Give your voice assistant a local brain: Ollama in Home Assistant
Whisper hears and Piper speaks, but plain Assist still thinks in exact phrases. The 2026 step is a local large language model as the conversation agent: 'make it cozy in here' becomes lights, temperature and music understood in context — with nothing leaving your house. The stack has converged (Ollama serving a Qwen3 model that supports tool calling), and so has the hard-won wisdom about what makes it feel smart instead of sluggish.
Published Nov 18, 2026
The architecture, honestly
Three parts you already may have, plus one new one. Whisper (speech-to-text) and Piper (text-to-speech) are covered in our local voice guide. The new piece is the conversation agent: instead of Assist's pattern matching, requests route to a local LLM served by Ollama, which interprets intent and calls Home Assistant's tools — turning devices on, setting scenes, answering questions about the house. Home Assistant 2026 even added a LiteLLM conversation integration for the cloud-flexible; for the fully-local path, the Ollama integration is the standard. The satellite hardware does not change: Voice Preview Edition or ESP32 satellites keep being ears and mouths.
Pick the model by your VRAM, not by hype
- The non-negotiable: tool calling. The model must support function/tool calling, or it literally cannot operate your devices — it will chat charmingly and do nothing. This single requirement eliminates half the models people try first.
- ~24 GB VRAM: Qwen3 30B-A3B (MoE — fast for its size) is the community's top pick at this tier; genuinely impressive household reasoning.
- ~16 GB: Qwen3 mid-size dense models, or the 30B-A3B at tighter quantization — still strong.
- ~8 GB: Qwen3 4B Instruct-class models: smaller, quicker, perfectly serviceable for device control (the actual job) if your expectations are set to 'excellent butler, not philosopher.'
- No GPU: CPU-only LLM voice is a demo, not a product — multi-second to multi-minute replies kill the magic. A NAS without a GPU can still run everything else; put Ollama on the one machine with a GPU, or stay with plain Assist (which is better than a slow LLM).
Deploy Ollama + connect Home Assistant
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- "11434:11434"
volumes:
- ./ollama:/root/.ollama
# GPU hosts: add the runtime/deploy block for your vendor
# (NVIDIA: runtime nvidia + driver capabilities; see the voice guide notes)
restart: unless-stopped
- Pull your chosen model (ollama pull qwen3:… the tag matching your VRAM tier) and test it in Ollama's own chat first — confirm it answers and, in the API sense, emits tool calls.
- In Home Assistant: Settings → Devices & Services → Add Integration → Ollama, pointed at the host and port. Then Settings → Voice assistants → your assistant → set the conversation agent to the Ollama model.
- Turn on the 'prefer handling commands locally' style option where offered, so simple commands stay on the fast path and the LLM handles the fuzzy ones.
Exposed entities: the discipline that decides everything
Every entity you expose to the voice assistant rides along in the model's context on every request. The community's rule of thumb: around 30 well-chosen exposed entities ≈ 1,300 tokens of context and snappy, accurate control; expose 400 entities and even a big model slows down and starts grabbing the wrong 'light.' Expose rooms' main actors — lights, climate, covers, media, a few scenes — with clean names and good aliases (the 2026.11 naming cleanup in our naming guide pays off double here). Unexpose the sensors, the diagnostic switches, and the seventeen smart plugs named after their MAC addresses. This one discipline is the difference between the demos that wow people and the installs that get unplugged.
Latency: making it feel alive
- Speech-to-Phrase (Home Assistant's newer on-device recognizer) resolves common commands in well under a second on modest hardware — pair it as the front door with the LLM behind it.
- Keep the model warm: Ollama unloads idle models; a first request after tea takes the load hit. A tiny periodic keep-alive automation (or accepting the first slow answer of the evening) is the standard shrug.
- Stream responses where the pipeline supports it — hearing the first words at one second beats a perfect paragraph at six.
Gear for this build
Prices are editorial and change often; buttons show live prices. Some links may earn a commission at no cost to you — see the disclosure page.
Official documentation and references
- Home Assistant documentation — the Ollama conversation agent integration
- Home Assistant documentation — the conversation integration and exposed entities
- Home Assistant documentation — voice control and Assist
- Ollama documentation — running and managing local models
- Ollama model library — the Qwen3 model family and sizes
Frequently asked questions
Do I need a GPU for local LLM voice?
For the LLM part, realistically yes — CPU inference is too slow for conversation. The rest of the pipeline (Whisper/Piper/Speech-to-Phrase) runs happily on modest hardware; if you have no GPU anywhere, plain Assist with good aliases is the better experience.
Which model should I start with?
The largest Qwen3 tool-calling model your VRAM fits comfortably alongside its context: 30B-A3B around 24 GB, mid-size dense models at 16 GB, the 4B Instruct class at 8 GB. Model choice expires fast in this space — the tool-calling requirement and the VRAM tiers are the durable parts.
Is my voice data still private with an LLM in the loop?
Yes, on this architecture: audio is transcribed locally, the LLM runs on your hardware, and actions execute locally. Nothing in this guide's stack sends voice or text to a cloud — that is the point of building it this way.

