Own It

Giving Your AI Memory: Custom System Prompts

Use Ollama Modelfiles to create a custom version of a model that remembers how you want it to behave, every time, without repeating yourself.

12 minute read

The problem this solves

By default, every new chat starts from zero. If you always want responses kept short, or always want the model to answer as if explaining to a beginner, or always want it to avoid a certain tone, you'd normally have to type those instructions again at the start of every conversation.

A Modelfile fixes this by baking a system prompt directly into a custom model. Once it's built, that behavior is just how the model works, every time, without you repeating anything.

Writing a Modelfile

Create a plain text file named Modelfile (no extension) with contents like this:

FROM llama3.2

SYSTEM """
You are a direct, concise assistant. Keep answers short unless
asked to go deeper. Avoid corporate speak and hedging language.
"""

The FROM line points at a base model you already have installed. The SYSTEM block is the instruction that now applies automatically, every time this custom model is used.

Building your custom model

From the same folder as your Modelfile, run:

ollama create my-assistant -f ./Modelfile

That's it - my-assistant now shows up alongside your other models, in Open WebUI's dropdown and in the terminal, and it carries your system prompt automatically.

ollama run my-assistant

This is genuinely useful, not just a novelty. A few well-considered Modelfiles - one for quick answers, one for detailed explanations, one for a specific recurring task - save real repeated typing over time.

What actually goes wrong

The build command fails immediately

Check that the base model named in the FROM line is actually pulled already:

ollama list

If it's not there, pull it first, then run ollama create again.

The custom model doesn't seem to follow the system prompt

Smaller models are less reliable at consistently following system instructions than larger ones - this is a real model capability limit, not a setup mistake. If a 3B model is ignoring the prompt, the same Modelfile against a 7-8B base model usually behaves noticeably better.

You want to update the prompt later

Edit the Modelfile and run the same ollama create command again with the same name - it overwrites the existing custom model cleanly.

Common questions

Is this the same as RAG?

No. A system prompt shapes how the model behaves and responds in general. RAG pulls in specific document content. They solve different problems and work well together.

Does a custom Modelfile use more storage?

Barely any. It is a small text file plus a reference to the base model you already have - not a full copy of the model itself.

Go deeper

This guide covers one solid path. Here's where to go if you want something different.

r/LocalLLaMA Real examples of Modelfiles people have built for specific recurring tasks. Talking to Your Own Documents with RAG Pairs well with a custom Modelfile - combine a tuned system prompt with your own reference documents.

Written from hands-on security operations experience. More about this site →