Setting Up a Mini PC for Local AI

Why a mini PC, not a cloud subscription or a bigger rig

The obvious alternative is just paying for a cloud AI subscription instead of setting any of this up. That's a fair question, and the honest answer is it depends what you're optimizing for.

A cloud subscription is faster to start and skips picking hardware entirely. But the monthly cost keeps going whether you use it or not, and every prompt you send leaves your machine and lives on someone else's server. A mini PC costs more upfront and effectively nothing after that, and nothing you type ever leaves the room.

The other alternative is a full desktop with a dedicated GPU. That's genuinely faster for larger models, but it costs three to four times as much, draws a lot more power, and most people running 7-8B models for everyday tasks won't notice the difference in practice. A mini PC is where cost, noise, and capability actually line up for a first setup.

Picking hardware

You don't need a gaming rig. A mini PC with a modern CPU and enough RAM will run 7-8B parameter models at a comfortable speed for everyday use - chatting, coding help, summarizing documents.

The two numbers that matter most:

• RAM: 16GB minimum, 32GB if you want headroom
• CPU: a recent Ryzen or Intel Core chip with at least 6 cores

Installing Ollama

Ollama is the easiest way to run local models. It handles downloading, quantization, and serving a simple API - no manual model wrangling required.

curl -fsSL https://ollama.com/install.sh | sh

Once installed, confirm it's running:

ollama --version

Running your first model

Pull and run a small model to confirm everything works end to end:

ollama run llama3.2

If you get a response, your mini PC is now a private AI server. No cloud, no API key, no monthly bill.

What actually goes wrong

Every guide shows the happy path. Here's what actually trips people up, based on running this exact setup.

Ollama isn't responding

Check whether the background service is actually running:

ollama ps

If that comes back empty when you know a model should be loaded, the service likely didn't start. On Linux:

sudo systemctl status ollama
sudo systemctl restart ollama

On Mac, quit and reopen the Ollama app from the menu bar - it runs the background service for you.

A model download stalls partway through

Models run several gigabytes, and a slow or unstable connection can stall the pull. Just re-run the same command:

ollama run llama3.2

Ollama resumes the download where it left off instead of starting over.

Responses are painfully slow

This is almost never a CPU problem. It's almost always RAM. If the system is swapping to disk because the model doesn't fit in memory, every response crawls no matter how fast the processor is. Check memory usage (Activity Monitor on Mac, htop on Linux) while a model is running - if it's maxed out with swap climbing, the fix is more RAM or a smaller, more quantized model, not a faster chip.

Common questions