Using a Local Model as a Coding Assistant

Why local for coding specifically

Coding assistants see everything - your actual source code, sometimes API keys or connection strings sitting in a config file, business logic you'd rather not hand to a third party. Cloud coding assistants are genuinely useful, but they also mean your code is leaving your machine constantly, all day, every file you touch.

Running this locally means the assistant is just as available, but nothing about your codebase goes anywhere. For anyone working under an NDA, on proprietary code, or just uncomfortable with the idea, that's the whole appeal.

Pulling a coding model

General-purpose models like llama3.2 can write code, but a model trained specifically for programming does noticeably better. qwen2.5-coder is the current standard choice for local coding assistance - it's not a typo or an odd naming convention, that's genuinely the model name.

ollama pull qwen2.5-coder

Installing Continue

Continue is a free, open-source VS Code extension that connects your editor to a model - local or cloud, your choice. Install it from the VS Code Extensions panel by searching "Continue."

Once installed, open Continue's configuration and point it at your local Ollama instance instead of a cloud provider. It auto-detects models you've already pulled, so qwen2.5-coder should show up in the list once Ollama is running.

From there, Continue works the way any coding assistant does - inline suggestions as you type, a chat panel for asking questions about your code, and the ability to select code and ask for changes.

What actually goes wrong

Continue doesn't see your local model

Confirm Ollama is actually running with ollama ps, and that Continue's configuration points at the correct local address rather than a default cloud endpoint left over from setup.

Suggestions feel slow compared to a cloud assistant

This is expected on CPU-only hardware, more so than with chat - code completion wants to feel instant. If the delay is bothering you day to day, this is one of the stronger cases for the full PC build guide's dedicated GPU path, since coding assistance benefits more from raw speed than casual chat does.

The model gives confidently wrong answers about your code

Treat it the way you'd treat a very fast junior developer: useful for a first pass, but verify anything that touches something critical. This isn't unique to local models - it's true of every current AI coding assistant, local or cloud.

Common questions