How Much RAM Do You Need for Local AI? I Tested 8 GB.

Why I wanted it local

Last episode I promised you a free coding model. I’ll get there. But first I needed one for something a lot more personal. My family is going through something hard, and I needed to find specific text messages and get them to a lawyer. I searched my phone for a contract, a name, a place, and got back a pile of results. Not the messages I needed, and no easy way to export them.

So I did what I always do: I built an app. iMessage Search copies your messages from your iPhone backup and lets you search by words, person, conversation and date. Then it summarizes the results, citing every message, and exports a report a lawyer can actually read.

But that summary came from Claude, in the cloud, and these are my texts. So the question for this episode: can a free model running on this laptop do that job instead?

Every message on screen in the video, and on this page, is fictional demo data. My real messages stay off camera.

The setup, and the watchdog

I’m working with an M1 MacBook Pro with 8 GB of memory. On a Mac, the AI model and everything else share those same 8 GB: your browser, your editor, macOS itself. The graphics chip only gets about two-thirds of it. That’s about 5.5 GB, best case.

I ran the models with Ollama, with the safest settings I could find:

• One model at a time, one request at a time.
• A compressed scratch pad for its memory (flash attention and a q8_0 KV cache).
• A fixed amount of text it can read at once: 8,192 tokens, set explicitly. On small Macs Ollama’s default is 4,096, and it silently cuts anything longer.

And this time I didn’t just hope. I had Claude write me a watchdog. Every 2 seconds it checks memory pressure. Green means fine, yellow means the Mac is squeezing, and red means it’s out of room. That’s when things freeze. If pressure hits red, free memory drops under 8%, or swap grows by more than 2 GB, the watchdog kills the model before it can take the Mac down with it.

Round one: Qwen 3.5 4B

Qwen 3.5, 4 billion parameters, is a popular small model with a download of about 3 GB. On paper, it fits. The first time I ran it: red in 6 seconds. My watchdog killed it.

I ran it again for the video, and this time it survived, barely. Free memory dropped to 9%, one point above where the watchdog pulls the plug, and to stay alive the Mac pushed 3.2 GB out to disk. That’s swap: your Mac quietly struggling. It was also slower than the smaller model, 38 seconds for the same summary.

Here’s what surprised me. Qwen 3.5 can read pictures too, and Ollama loaded that picture part, about 600 MB, even though I was only sending text. On an 8 GB Mac, that’s 600 MB you don’t have.

Round two: Granite 4.1 3B

So I went smaller: IBM’s Granite 4.1, 3 billion parameters, text only, about a 2 GB download. Memory pressure went to yellow and never went red. The lowest free memory was 17%, it ran 100% on the GPU in about 2.5 GB, and it wrote about 22 tokens a second, faster than you can read it.

That’s the first answer. It’s not about squeezing the biggest model in. It’s about picking the right size.

Wait, it was written in Python?

A confession. When I built this app, I never told the AI which language to use. It picked Python, and I didn’t even notice. I don’t always read the code it writes. I like building things, so I move on to the next step. But I write Go, and if something breaks, Go is what I know how to fix.

So I had Claude Code rewrite the whole thing in Go: one binary, one dependency. Then I checked it against the old version, message by message, all 224,999 of them. Every single one matched.

The part I love: the app talks to AI through one tiny Go interface, and Claude is just one plug. So a free local model should be one new file. This time I actually read it, and checked three things:

1. Where does it send my messages? Localhost. This Mac, nowhere else.
2. How much can it read at once? 8,000 tokens. That’s what keeps me in the yellow. Crank it up and you’re back in the red.
3. Thinking off. Some models write out their reasoning before they answer. On 8 GB, every token counts.

The real test: speed, privacy, mistakes

The demo is a made-up story: a tenant, a roommate and a landlord fighting over a security deposit. I searched “deposit” and hit Summarize. 27 seconds. Claude does the same summary in 7.5. So Granite is two to three and a half times slower.

Then I ran it on my real messages, the ones for the lawyer. 85 messages, 55 seconds. And honestly, it was really good: close to what Claude gave me, and exactly what I needed. Would I wait 55 seconds when it’s free and private? Absolutely. To prove nothing was leaving, I turned the Wi-Fi off and ran it again. Still works.

But it’s not perfect, and this matters. I checked Granite’s demo summary line by line. It:

• said Sam pays Priya half, when it’s the other way around;
• moved a showing from April to March;
• wrote December 14th, not the 19th;
• and invented new tenants who don’t exist.

Claude got every one of those right. So here’s my rule: free, private, good, but if it’s going to a lawyer, you check the citations. The app makes that easy, because every point links to the exact message it came from.

Why not just use Apple’s search?

Fair question. I have Apple Intelligence on an iPhone 16e, and in iOS 26 Messages search understands plain English. So I asked it for messages about signing a contract. It found the thread, but not the messages inside it. Fewer results, it only seemed to reach back so far, and I have years of messages on that phone. And there’s still no export.

So how much RAM do you need?

For this job, 8 GB is enough, if you’re smart about it. My five rules:

1. Pick the right size. 3B fit every time. 4B was a coin flip.
2. Text only. Don’t run a model that loads a picture part you’ll never use.
3. Keep what it reads small. I used 8K tokens of context. Narrow your search.
4. Watch memory pressure, not the RAM number. Green, yellow, red, and let something kill it before red.
5. Check its work. Free and private doesn’t mean right.

Try it yourself

Start small. A file renamer that reads your PDFs and gives them names that actually make sense. Or a tone checker: does this sound rude, before you hit send? Privately. The app, the watchdog and every number in this episode are linked below.

Next up: the PR review agent that froze this Mac, rebuilt on Granite. Let’s see if it survives. And if you missed it, episode 2 is 5 AIs built the same Go app.