Ollama Says "Out of Memory" - Now What?

First, figure out which memory it means

"Out of memory" can mean two genuinely different things depending on your setup, and the fix is different for each:

• GPU / VRAM out of memory - on a GPU build, this shows up as an error containing CUDA error: out of memory. The model (or its context) doesn't fit in the graphics card's VRAM.
• System RAM out of memory - on a CPU-only setup, this looks more like the whole system slowing to a crawl, swapping heavily, or Ollama getting killed by the OS rather than a clean CUDA error.

GPU out of memory (CUDA error)

This means the model doesn't fit in VRAM at the quantization and context length you're running it at. The math is covered in Understanding VRAM, but the practical fixes, roughly in order of how much they help:

• Drop to a smaller quantization of the same model - a 4-bit tag needs meaningfully less VRAM than an 8-bit or full-precision one
• Reduce the context window (num_ctx) if it's set higher than you actually need - a longer context reserves more VRAM whether or not a conversation is using all of it. Persistent Instructions With Ollama Modelfiles covers adjusting this via a Modelfile
• Set OLLAMA_KEEP_ALIVE=0 so Ollama releases VRAM immediately after each response instead of holding the model loaded for a few minutes by default - useful if you're switching between models often on a card with limited headroom
• Accept that this specific model needs a bigger card - not every model fits every GPU, and that's a hardware ceiling, not a settings problem

System RAM out of memory (CPU-only setup)

Same underlying cause as the GPU case - the model doesn't fit in the memory available to it - but on a CPU-only mini PC, this is exactly the "responses are painfully slow" or outright crash scenario covered in the mini PC guide's troubleshooting section. The fix is the same short list: a smaller or more quantized model, or more RAM if the machine supports it - see the RAM pick for what to check before buying a kit.