Locking Down Your Local AI Setup

What "local" already solves

The moment you started running models on your own mini PC instead of a cloud API, you solved the biggest privacy problem there is: nothing you type gets sent anywhere.

No company logs your prompts, no conversation trains a future model, no data breach at a provider you've never heard of exposes what you asked.

That's the real win, and it happened automatically. Everything below is about the smaller things that are easy to get wrong once this setup becomes part of your daily routine instead of a one-time experiment.

Network exposure

By default, both Ollama's own API and Open WebUI only listen on localhost - neither is reachable from any other device, not even others on your home network. That's the safe default this whole site installs, and for most people it should stay that way.

The most common way people change this for Ollama: wanting to reach it from a phone or another computer directly, and setting an environment variable like OLLAMA_HOST=0.0.0.0 to make that possible. This does work, but it also means any device on your network can now talk to your Ollama instance, and Ollama has no built-in login or password. On a trusted home network that's a small risk. On anything else - a shared apartment network, a coffee shop, a hotel - it's not one worth taking casually.

Open WebUI works the same way, just through Docker's port binding instead of an environment variable - Accessing Your Setup From Other Devices covers widening it on purpose. Unlike Ollama, Open WebUI does have its own login, so the stakes of widening it are lower - but "lower" isn't "none," especially on a network you don't fully control.

For anything beyond your own trusted home network, a reverse proxy with authentication in front of Ollama specifically is the fix - a small piece of software that sits between the network and Ollama and only lets a request through once it's supplied a username and password, since Ollama itself has no login of its own. Caddy is a genuinely beginner-friendly option for this. That setup is worth its own dedicated walkthrough rather than a rushed one here - treat this as the shape of the fix, and look up a current Caddy basic-auth guide before actually leaving anything open this way.

Backups

Model files themselves usually aren't worth backing up - they're multiple gigabytes each and can simply be re-downloaded with ollama pull if something goes wrong. What's actually worth protecting is everything you've built on top of them: saved chat history, custom system prompts, any documents you've fed into a RAG setup down the road.

Ollama stores its models and configuration in ~/.ollama on Mac, or under /usr/share/ollama/.ollama on Linux, since the standard install runs it as a system service under its own account rather than yours - see the storage guide if you need the exact path. Open WebUI's data - your accounts, chat history, settings - lives inside the Docker volume created during setup. A simple external drive with a scheduled copy of these two locations covers the realistic failure case: a drive dying, not a sophisticated attack.

For the actual backup and restore commands, see Backing Up Your Local AI Setup.

Disk encryption

Here's a way to think about it that makes this concrete: your mini PC is now, in a real sense, the "cloud" your data lives in - except it's a physical object sitting in your house instead of a data center. If that machine were ever lost or stolen, everything on it is readable by whoever has it, unless the disk itself is encrypted.

Turning this on takes a few minutes and runs invisibly afterward:

• Mac: System Settings → Privacy & Security → FileVault → Turn On
• Windows: Settings → Privacy & Security → Device Encryption, or BitLocker on Pro editions
• Linux: depends on distribution - LUKS is the standard, usually offered as an option during OS install

What actually goes wrong

You set OLLAMA_HOST=0.0.0.0 for one thing and forget about it

This is the single most common way people end up with more network exposure than they intended - usually to test something from a phone once, and then the setting just stays. Check what Ollama is actually bound to:

echo $OLLAMA_HOST

If that returns 0.0.0.0 and you don't have a specific, ongoing reason for it, unset it and restart Ollama to go back to the localhost-only default.

Check what Open WebUI is actually bound to

This is the one command worth running periodically rather than trusting memory of what you set up months ago:

docker ps

Look at the PORTS column. Local-only, matching the default install, reads like this:

127.0.0.1:3000->8080/tcp

Widened to your network - deliberately, per Accessing Your Setup From Other Devices - it reads like this instead:

0.0.0.0:3000->8080/tcp

If you see the second form and don't have a specific reason for it, that's worth a second look. This same check applies to any other container you add to this setup later - don't assume "it's in Docker" means "it's private."

Backups exist but were never tested

A backup you've never restored from isn't a backup yet, it's a hope. Once you have a copy of ~/.ollama and your Open WebUI volume somewhere external, actually try restoring it on a spare machine or a fresh user account at least once.

Common questions