Using a Local Model as a Coding Assistant

Why local for coding specifically

Coding assistants see everything - your actual source code, sometimes API keys or connection strings sitting in a config file, business logic you'd rather not hand to a third party. Cloud coding assistants are genuinely useful, but they also mean your code is leaving your machine constantly, all day, every file you touch.

Running this locally means the assistant is just as available, but nothing about your codebase goes anywhere. For anyone working under an NDA, on proprietary code, or just uncomfortable with the idea, that's the whole appeal.

Pulling a coding model

General-purpose models like llama3.2 can write code, but a model trained specifically for programming does noticeably better. qwen2.5-coder has been the reliable local standard, and Qwen3-Coder is the newer, stronger pick if your hardware has the room for it - it's not a typo or an odd naming convention, these are genuinely the model names.

ollama pull qwen2.5-coder

qwen2.5-coder answering a real coding question - working code plus an explanation, no editor required yet.

Installing Continue (or Cline)

Continue is a free, open-source VS Code extension that connects your editor to a model - local or cloud, your choice. Install it from the VS Code Extensions panel by searching "Continue."

Once installed, open Continue's configuration and point it at your local Ollama instance instead of a cloud provider. It auto-detects models you've already pulled, so your coding model should show up in the list once Ollama is running.

From there, Continue works the way any coding assistant does - inline suggestions as you type, a chat panel for asking questions about your code, and the ability to select code and ask for changes.

The two ways you'll actually use it day to day: as you type, gray "ghost text" suggestions appear inline - press Tab to accept one, or just keep typing to ignore it. For anything bigger than a one-line completion, select the relevant code and use Continue's edit shortcut to describe the change you want in plain language, rather than trying to get it right through autocomplete alone.

Cline is worth knowing as the alternative here - also free and open-source, also works with local Ollama models, but built around more autonomous, agentic editing with separate "Plan" and "Act" modes rather than Continue's lighter, more hands-on-the-wheel approach. Continue suits people who want suggestions they review line by line; Cline suits people comfortable handing over a bigger task and reviewing the result. Neither is strictly better - they're different levels of how much control you want to keep.

What actually goes wrong

Continue doesn't see your local model

Confirm Ollama is actually running with ollama ps, and that Continue's configuration points at the correct local address rather than a default cloud endpoint left over from setup.

Suggestions feel slow compared to a cloud assistant

This is expected on CPU-only hardware, more so than with chat - code completion wants to feel instant. If the delay is bothering you day to day, this is one of the stronger cases for the full PC build guide's dedicated GPU path, since coding assistance benefits more from raw speed than casual chat does.

The model gives confidently wrong answers about your code

Treat it the way you'd treat a very fast junior developer: useful for a first pass, but verify anything that touches something critical.

This isn't unique to local models - it's true of every current AI coding assistant, local or cloud.

It seems to "forget" earlier parts of a large file or lose track in a big repo

This is a context window limit, not the model getting confused - once a file or conversation exceeds what the model can hold in memory at once, older content genuinely falls out of what it's considering. Scope your requests to a single function or file rather than an entire large repo, and start a fresh conversation for a new unrelated task instead of letting one thread run indefinitely.

Common questions