Building a Full PC for Local AI

When you actually need this

The mini PC guide covers 7-8B parameter models comfortably, without a GPU, for a few hundred dollars. That's genuinely enough for most everyday use - chatting, coding help, summarizing documents. This guide is for three specific situations where that stops being true:

• You want to run larger models (13B, 30B, or bigger) that need more memory and compute than a CPU-only setup can move through quickly
• You want image generation - Stable Diffusion-style workloads lean on a GPU far more heavily than text models do
• Response speed matters to you specifically, and CPU inference feels too slow even for smaller models

If none of those apply, the mini PC setup is genuinely the better choice - not a compromise you're settling for.

The GPU

This is the one component that actually matters here. Everything else in this build is close to a normal desktop PC - the GPU is what's specific to running AI models well.

Within NVIDIA's lineup, the number that actually determines what you can run is VRAM - the memory built into the graphics card itself, separate from your system's regular RAM. A faster GPU with less VRAM will often run a large model worse than a slower GPU with more, because if the model doesn't fit in VRAM, performance falls off sharply.

Understanding VRAM

Here's the rough math that actually matters when picking a card: a model at a common quantization level needs roughly half to one gigabyte of VRAM per billion parameters. A 13B model needs somewhere around 8-10GB. A 30B model needs closer to 20GB or more, depending on quantization.

This is why "how many billions of parameters" and "how much VRAM" are really the same question asked two different ways. Buy VRAM for the largest model you actually plan to run regularly, not the one you might try once.

The rest of the build

Once the GPU is picked, everything else exists to support it well:

• PSU (power supply): check the GPU's rated power draw and add real headroom - a common mistake is sizing a power supply for the GPU alone and forgetting the rest of the system needs power too
• CPU: matters much less here than the GPU does. A mid-range modern CPU is fine - the GPU is doing the heavy lifting for inference
• System RAM: 32GB is a reasonable baseline, same as the mini PC guide - this is separate from the GPU's VRAM and still matters for everything else the system is doing
• Case and cooling: a GPU under sustained load generates real heat for long stretches - make sure the case has enough airflow, not just enough physical space

What actually goes wrong

The power supply is undersized

People size a PSU for the GPU's listed power draw and forget the CPU, drives, and fans also pull from the same supply. Add at least 150-200 watts of headroom above the GPU's rated draw for the rest of the system, and don't buy a PSU at the exact edge of what a build needs.

Drivers are out of date or missing entirely

A fresh GPU install needs the latest NVIDIA drivers plus the CUDA toolkit before most inference tools will actually use the GPU instead of quietly falling back to the CPU. If a tool runs but feels no faster than your old mini PC, this is the first thing to check - confirm the GPU is actually being used:

nvidia-smi

If that command isn't found or shows no active processes while a model is running, the drivers or CUDA toolkit are the problem, not the model or the tool.

The case looks like it fits the GPU, but airflow is an afterthought

A GPU can physically fit in a case and still overheat and throttle under sustained load if there isn't real airflow across it. Check a case's airflow reviews specifically, not just its size specs, before buying.

Common questions