How to Deploy a Zero-Cloud Local AI Agent for Home Assistant Using Ollama

This guide walks you through wiring a local large language model into Home Assistant using Ollama, so every command, every automation decision, and every scrap of conversation stays on hardware you own. No subscription. No API key. No round-trip to a data center in Virginia. Let’s build it.

Why Local AI Actually Makes Sense for Your Smart Home

Before you touch a terminal, it’s worth understanding what you’re trading and what you’re gaining.

The privacy case is obvious, but the practical case is just as strong. A local setup keeps working when your internet goes down, since Home Assistant talks to Ollama over your LAN, not the open internet. Latency also becomes something you control instead of something a third party’s server load dictates. And because you’re not paying per-token API fees to a cloud provider, your only real cost is the electricity your server draws.

The trade-offs are real too, and a good systems administrator names them instead of glossing over them:

• Response speed: a cloud assistant on a fast connection can respond in under a second. A local model on modest hardware might take two to four seconds. Noticeable, but tolerable once you’re used to it.
• Ecosystem breadth: Alexa has thousands of pre-built “skills.” Your local assistant only knows what you expose to it and teach it.
• Hardware dependency: you need a machine that can actually run a model fast enough to feel responsive, which brings us to the next section.

What Hardware You Actually Need

You don’t need a rack of enterprise GPUs. A 7B-to-8B parameter model running at 4-bit quantization needs roughly 8 GB of VRAM to respond quickly enough for real-time voice or text commands. If you’re running Ollama on a machine without a discrete GPU, a modern CPU with 16 GB of system RAM will still work — just expect longer response times. A Raspberry Pi 5 can technically run this stack, but temper your expectations on speed. If you’re serious about snappy performance, a small dedicated box with an NVIDIA or AMD GPU is the sweet spot.

Setting Up Ollama as Your Local AI Engine

Ollama is the piece that actually loads and runs the model. Home Assistant never runs the model itself — it just talks to Ollama over the network.

1. Install Ollama on a machine that Home Assistant can reach on your LAN (this can be the same box running Home Assistant, or a separate always-on machine — a mini PC or home server works well).
2. Verify the Ollama service is running. By default, it listens on http://localhost:11434. Confirm it’s live with:

   curl http://localhost:11434

   A response confirms the API is reachable.

3. Pull a tool-calling model. This matters more than most people expect — not every model can reliably translate “turn off the kitchen lights” into an actual service call. Models trained for tool/function calling do this far more reliably than general chat models. Pull one with:

   ollama pull qwen3:8b

   Llama 3.3 8B Instruct is a solid general-purpose fallback if you want a second option to compare.

4. Avoid “thinking” or reasoning-mode model variants for this use case. They add deliberation time before responding, and that extra latency doesn’t buy you better accuracy on short, direct commands like “dim the bedroom lights to 30%.”

Connecting Ollama to Home Assistant

With Ollama running and a model pulled, it’s time to bring Home Assistant into the picture. Home Assistant has shipped a native Ollama integration since its 2024.4 release, and it has matured considerably since.

Adding the Integration

1. In Home Assistant, go to Settings → Devices & Services.
2. Click Add Integration and search for Ollama.
3. Enter the host URL and port of your Ollama server (for example, http://192.168.1.50:11434 if it’s on a separate machine).
4. Select your pulled model from the dropdown once the connection succeeds.

Understanding the Two-Agent Pattern

Here’s a detail that trips people up: Home Assistant’s documentation recommends adding the Ollama integration twice, using the same model each time, for two distinct purposes.

• The first instance is a plain conversation agent — good for chatting, asking questions, no device control.
• The second instance enables “control Home Assistant”, granting the model access to the Assist API so it can actually flip switches and adjust thermostats.

This separation isn’t bureaucratic overhead — it’s a safety boundary. It keeps a casual chat session from accidentally having the authority to unlock your front door.

Controlling What the AI Can Touch

Once the control-enabled agent exists, go to the exposed entities page and explicitly choose which devices the model can see and act on. This is opt-in by design. If a device isn’t exposed, the model can’t touch it, full stop — which keeps the blast radius of a misunderstood command small.

Tuning the Context Window

Home Assistant defaults the context window to 8,000 tokens, larger than Ollama’s own 2,000-token default. This matters for larger homes: more exposed entities means more tokens spent just describing your house’s current state before the model even reads your command. If you have a smaller home, you can lower this to reduce RAM usage on the Ollama server; larger homes with lots of entities benefit from keeping it high or increasing it further.

Wiring It Into a Full Voice Pipeline

Text commands are great, but voice is where this setup really shines. Home Assistant’s Assist pipeline lets you chain together:

• Wake word detection (openWakeWord or similar)
• Speech-to-text via Whisper, running locally
• Your Ollama conversation agent for intent and response
• Text-to-speech via Piper, also local

Every stage of that pipeline can run without touching the internet. One useful trick: custom sentences and intents get matched before a request ever reaches Ollama. This gives you deterministic, instant handling for critical commands (like “goodnight” triggering a specific automation) while letting the LLM handle everything else conversationally. It’s the best of both worlds — speed and reliability where you need it, flexibility where you don’t.

Wrapping Up: Your Next Steps

You now have everything you need to run a genuinely private, genuinely capable AI layer for Home Assistant:

1. Provision hardware matched to your model size (8 GB VRAM is your realistic starting line for real-time response).
2. Install Ollama and pull a tool-calling-capable model like Qwen3 8B.
3. Add two Ollama integrations in Home Assistant — one for chat, one for control.
4. Expose only the entities you want the AI touching.
5. Layer in Whisper and Piper if you want a full local voice pipeline.

None of this requires a subscription, a cloud account, or trust in a third party’s data-handling policy. It requires a spare afternoon and a machine with a bit of horsepower. Once it’s running, you’ll have a smart home assistant that answers to you — and only you.