Glossary
Plain-language definitions for the terms that come up across this site - not full explainers, just enough to keep reading without stopping to look something up elsewhere.
Agent
An AI system that decides what actions to take as part of a multi-step task - not just answering with text. Tools, MCP connections, and code execution are things an agent can use, but connecting a model to MCP doesn't automatically make it agentic on its own.
API
A defined way for one program to talk to another. Ollama's API is what Open WebUI actually talks to under the hood - see Ollama API Automation for scripting against it directly.
Learn more: Ollama API Automation →Benchmark
A standardized test used to compare models or hardware. Useful as a starting point, not a verdict - a model that wins a benchmark can still be the wrong pick for a specific real task.
Context window
How much of a conversation a model can actually "see" at once, measured in tokens. Once a conversation grows past it, the oldest parts stop being considered.
Learn more: Why It Seems to "Forget" Things →CUDA
NVIDIA's software platform for running compute workloads on its GPUs. It's why this site defaults to NVIDIA, not AMD, for GPU builds - most local AI tools are built around it first.
Embedding
A way of turning text into a list of numbers that captures its meaning, so a computer can compare how similar two pieces of text are. This is the mechanism RAG uses to find relevant document chunks.
Learn more: What RAG Actually Does →Endpoint
A specific URL an API exposes for one purpose - Ollama's /api/generate endpoint handles a single chat response, for example.
Learn more: Ollama API Automation →GGUF
The file format most local models are distributed in today. If you see a model file ending in .gguf, that's what it is - a single file containing the model's weights, ready for a tool like Ollama or llama.cpp to load.
Hallucination
When a model states something false with the same confidence as something true. It's not a bug being fixed over time so much as an inherent property of how these models generate text - local AI isn't immune to it.
Inference
The actual process of a model generating a response to a prompt. "Inference speed" is how fast it does that - distinct from training, the much more expensive process of creating the model in the first place.
LLM
Large language model - the general term for the kind of AI model this whole site is about. "Large" refers to the number of parameters, not the file size directly, though the two are related.
MCP
Model Context Protocol - a standard way for a model to call external tools (reading files, fetching a web page, querying a database) instead of only answering from what it already knows.
Learn more: Giving Your Local AI Real Tools with MCP →Metal
Apple's equivalent to CUDA - the framework that lets Ollama and other tools use the GPU inside Apple Silicon chips.
Multimodal
A model that can work with more than just text - reading images, in most cases on this site. See Vision model below for what that actually looks like day to day.
Parameters
The internal values a model tunes during training - roughly, the number of "knobs" it has. A model's size in billions of parameters (7B, 27B, 70B) is the biggest single factor in both its capability and how much RAM or VRAM it needs.
Learn more: What Parameter Count Means →Quantization
A way to shrink a model so it uses less RAM or VRAM, usually at the cost of a small amount of quality. Most models you'll actually pull are already quantized by default, even when the tag doesn't say so directly - the level (Q4, Q5, Q8, etc.) sometimes appears in the tag name and sometimes only shows up in the model's details.
Learn more: Quantization, Briefly →RAG
Retrieval-Augmented Generation - having a model answer questions using your own documents instead of just what it learned during training.
Learn more: Chat With Your Documents →Reasoning model
A model optimized for more deliberate, multi-step problem solving before answering - useful for math and multi-step logic, less useful for tasks that just need a quick, direct answer. Some tools show the model's reasoning as it happens; others keep it hidden and only show the final answer.
Learn more: Reasoning Models, Briefly →ROCm
AMD's equivalent to CUDA. Support for it in local AI tools is real but generally arrives later and with more rough edges - part of why this site defaults to NVIDIA for a first GPU build.
System prompt
An instruction given to a model before the actual conversation starts, shaping how it behaves for the rest of the chat.
Learn more: Persistent Instructions With Ollama Modelfiles →Temperature
A setting that controls how random a model's output is. Lower values give more focused, repeatable answers; higher values give more varied, creative ones.
Learn more: Persistent Instructions With Ollama Modelfiles →Tokens
Roughly, chunks of a word - the unit a model actually processes text in, not quite the same as a word or a character. Context windows and generation speed are both measured in tokens, not words.
Tokens per second
How fast a model generates a response, measured in tokens (see above). The number that actually matters for "does this feel fast," more than any benchmark score.
Top-p
A setting related to temperature that controls how many different next-word options a model considers before picking one. Most people never need to touch this directly - it's worth knowing the name exists more than tuning it yourself.
Unified memory
Apple Silicon's approach where CPU and GPU share one pool of memory instead of the GPU having its own separate VRAM. It's why a Mac can address far more memory for a model than a comparable graphics card.
Learn more: Apple's New Mac Studio and Mac Mini Are Serious Local AI Machines Now →Vector database
Storage built specifically for embeddings, organized so "find things similar to this" is fast. Open WebUI's built-in RAG handles this for you automatically - most people reading this site never interact with one directly.
Vision model
A model that can also read images, not just text - "what's in this photo" or "describe this diagram."
Learn more: Vision Models →VRAM
The memory built into a graphics card itself, separate from a computer's regular system RAM. It's the number that actually determines what size model a GPU can run.
Learn more: Understanding VRAM →