Creativity Commons

How to set up your own private, offline AI on a Mac

unsplash.com/@planetvolumes

Abstract

A step-by-step guide to setting up a private, offline AI on Mac using Ollama, covering RAM requirements, installation, uncensored models, and connecting to LibreOffice.

Running an AI model on your own Mac gives you three things online tools can't: complete privacy (nothing you type ever leaves your computer), it works with no internet, and it's free once the download is done. Here's the whole setup from scratch.

When people decide between an online AI (like the ones you use in a web browser) and an offline AI running on your own Mac, it usually comes down to a tradeoff between power and privacy. Online models live on huge company servers, so they tend to be the smartest and most capable, but everything you type travels over the internet and needs a connection to work. Offline models run entirely on your own machine, which means they're private, free to use, and work anywhere — but your computer's memory limits them and generally isn't as sharp as the biggest cloud models. Here's how the two stack up side by side.

Feature	Strength Online AI (cloud)	Offline AI (your Mac)
Raw capability	✅ Access to the largest, most advanced models	⚠️ Limited by your RAM; smaller and less powerful
Privacy	⚠️ Your input is sent to a company's servers	✅ Nothing leaves your computer
Cost	⚠️ Often a monthly fee or usage limits	✅ Free to run once downloaded
Works offline	❌ Needs an internet connection	✅ Runs with no internet at all
Speed	✅ Fast, powered by industrial hardware	⚠️ Depends on your Mac ; can be slower
Setup	✅ Just open a website and start	⚠️ Requires installing and downloading a model
Fewer refusals	⚠️ Heavier built-in content filters	✅ Can run less-restricted models if you choose
Always up to date	✅ Company updates it automatically	⚠️ You manage updates and new versions yourself

The short takeaway: pick an online model when you want the most powerful AI with zero setup, and pick an offline model when privacy, cost, and working without internet matter most to you.

Step 1: Check your RAM and understand model sizes

Click the Apple icon in the top left, choose "About This Mac," and look at your Memory (RAM). Your RAM decides which models will run well.

Models come in different sizes, measured in billions of parameters — that's the "b" in a name like llama3.1:8b. Think of parameters as the number of tiny knobs a model uses to think. More parameters usually mean smarter, more detailed answers, but they also require more memory and run more slowly. A "3b" model is small and fast. A "14b" model is bigger and sharper, but hungrier.

A rough guide:

8 GB RAM: small models (around 3b)
16 GB RAM: mid-size models (around 8b) — the sweet spot for most people
24 GB or more: large models (14b and up)

Step 2: Download Ollama

Go to ollama.com and click the macOS download button. You'll get a .dmg file.

Step 3: Install it

Open the .dmg, then drag the Ollama icon into your Applications folder. Launch Ollama. The first time it runs, it sets up a command-line tool and starts running quietly in the background. A small icon appears in your menu bar at the top of the screen.

Step 4: Make sure it worked

Open the Terminal app (search "Terminal" in Spotlight, the magnifying glass in the top right). Type this and press Enter:

% ollama --version

You should see a version number. If it says "command not found," restart Terminal and try again.

Step 5: Download a model

Still in Terminal, pull a model that fits your RAM. For example:

% ollama pull llama3.1:8b

Swap the name for whatever size fits your Mac. The download is a few gigabytes and takes a few minutes, with a progress bar to show how it's going.

Step 6: Try it out

Once it finishes, run:

% ollama run llama3.1:8b

You'll see a >>> prompt. Type a question, press Enter, and start chatting. Type /bye to exit.

Uncensored models (optional)

If you want a model with fewer built-in refusals, look for "abliterated" or "dolphin" versions. Abliterated models are standard models with their refusal behavior removed. Dolphin models are fine-tuned to be more open and direct. They install the same way:

% ollama pull dolphin-llama3:8b (good for 8–16 GB)

% ollama pull dolphin-mixtral:8x7b (needs 24 GB+)

New versions come out often, so you can also browse ollama.com/library and search "uncensored" to see what's current. One honest tradeoff: these have had their safety filters removed, so nothing is catching harmful requests — how you use it is fully on you. They're also fine-tunes, so quality varies and they sometimes make more mistakes than the standard versions.

Connecting it to LibreOffice

Ollama automatically runs a small local server at http://localhost:11434. Tools like the localwriter extension use that address to reach your model. To point localwriter to a specific model, open its settings and set the model name to the one you installed, such as llama3.1:8b or dolphin-llama3:8b.

The short version

Check your Mac's RAM first, since it determines how large a model you can run — the "b" in a model name means billions of parameters, and larger models are smarter but need more memory. Download and install Ollama from ollama.com, confirm it works with ollama --version, then use ollama pull to download a model and ollama run to chat with it. Uncensored "abliterated" or "dolphin" models install the same way, just with the tradeoff that no safety layer is left. And because Ollama runs its own local server, tools like localwriter can connect to your model automatically.