Building a Full PC for Local AI

When you actually need this

The mini PC guide covers 7-8B parameter models comfortably, without a GPU, for a few hundred dollars. That's genuinely enough for most everyday use - chatting, coding help, summarizing documents. This guide is for three specific situations where that stops being true:

• You want to run larger models (13B, 30B, or bigger) that need more memory and compute than a CPU-only setup can move through quickly
• You want image generation - Stable Diffusion-style workloads lean on a GPU far more heavily than text models do
• Response speed matters to you specifically, and CPU inference feels too slow even for smaller models

If none of those apply, the mini PC setup is genuinely the better choice - not a compromise you're settling for.

The GPU

This is the one component that actually matters here. Everything else in this build is close to a normal desktop PC - the GPU is what's specific to running AI models well.

Within NVIDIA's lineup, the number that actually determines what you can run is VRAM - the memory built into the graphics card itself, separate from your system's regular RAM (the RAM figures in Choosing Your First Model are a different number for a CPU-only setup, not this one). A faster GPU with less VRAM will often run a large model worse than a slower GPU with more, because if the model doesn't fit in VRAM, performance falls off sharply.

Understanding VRAM

Here's the rough math that actually matters when picking a card: a model at a common quantization level needs roughly half to one gigabyte of VRAM per billion parameters.

"How many billions of parameters" and "how much VRAM" are really the same question, asked two different ways.

Buy VRAM for the largest model you actually plan to run regularly, not the one you might try once - see the GPU pick for what to actually look for on Amazon. Not interested in sourcing and assembling the parts yourself? The prebuilt RTX 4090 workstation pick covers the same 24GB tier as a finished machine.

The rest of the build

Once the GPU is picked, everything else exists to support it well:

• PSU (power supply): check the GPU's rated power draw and add real headroom - a common mistake is sizing a power supply for the GPU alone and forgetting the rest of the system needs power too. See the PSU pick for exact wattage and certification to look for
• CPU: matters much less here than the GPU does. A mid-range modern CPU is fine - the GPU is doing the heavy lifting for inference
• System RAM: 32GB is a reasonable baseline, same as the mini PC guide - this is separate from the GPU's VRAM and still matters for everything else the system is doing. See the RAM pick for what to check before buying a kit
• Case and cooling: a GPU under sustained load generates real heat for long stretches - make sure the case has enough airflow, not just enough physical space

What actually goes wrong

The power supply is undersized

People size a PSU for the GPU's listed power draw and forget the CPU, drives, and fans also pull from the same supply. Add at least 150-200 watts of headroom above the GPU's rated draw for the rest of the system, and don't buy a PSU at the exact edge of what a build needs.

Drivers are out of date or missing entirely

A fresh GPU install needs the latest NVIDIA driver before most inference tools will actually use the GPU instead of quietly falling back to the CPU. Packaged Ollama bundles the CUDA runtime it needs, so a separate CUDA Toolkit install generally isn't required just for this - the driver is the actual requirement. If a tool runs but feels no faster than your old mini PC, this is the first thing to check - confirm the GPU is actually being used:

nvidia-smi

If that command isn't found or shows no active processes while a model is running, the driver is the problem, not the model or the tool.

Drivers current and the GPU still not being used? Why Isn't Ollama Using My GPU? covers the checks past this point - what ollama ps should show, and the VRAM-fit issue that causes a silent CPU fallback.

The case looks like it fits the GPU, but airflow is an afterthought

A GPU can physically fit in a case and still overheat and throttle under sustained load if there isn't real airflow across it. Check a case's airflow reviews specifically, not just its size specs, before buying.

Everything works, but it's mysteriously slow

If a model loads and responds but feels far slower than the VRAM math above would suggest, it likely doesn't fully fit in VRAM. Ollama and most inference tools handle this automatically by offloading the overflow to system RAM rather than failing outright - which keeps things technically working, but at a real cost: those offloaded layers now route through the much slower connection between GPU and CPU on every single token, not just once. A 5-15x slowdown from this kind of partial offloading isn't unusual, and nothing about it throws an error to warn you.

Check VRAM usage with nvidia-smi while a response is generating - if it's pinned at 100% and inference still crawls, that's the tell. The fix isn't a setting to change, it's math: drop to a smaller model, a more aggressive quantization, or accept that this particular model genuinely needs a bigger card than the one you have.

Get your first model running

The build is done and the drivers are verified. The rest is the same Ollama workflow as the mini PC guide - the difference is what you can actually run:

curl -fsSL https://ollama.com/install.sh | sh

Then pull and run a model sized for the VRAM you actually built for. A 13B-class model is a reasonable first pick on most cards covered here - it's genuinely out of reach for CPU-only hardware, which is the whole reason to build this instead of the mini PC:

ollama run qwen2.5:14b

While it responds, confirm the GPU is actually doing the work rather than quietly falling back to the CPU:

nvidia-smi

If VRAM usage climbs and a GPU process shows up during generation, that's it - the build is doing its job.

Common questions