Give your voice assistant a local brain: Ollama in Home Assistant

Whisper hears and Piper speaks, but plain Assist still thinks in exact phrases. The 2026 step is a local large language model as the conversation agent: 'make it cozy in here' becomes lights, temperature and music understood in context — with nothing leaving your house. The stack has converged (Ollama serving a Qwen3 model that supports tool calling), and so has the hard-won wisdom about what makes it feel smart instead of sluggish.

Published Nov 18, 2026

The architecture, honestly

Three parts you already may have, plus one new one. Whisper (speech-to-text) and Piper (text-to-speech) are covered in our local voice guide. The new piece is the conversation agent: instead of Assist's pattern matching, requests route to a local LLM served by Ollama, which interprets intent and calls Home Assistant's tools — turning devices on, setting scenes, answering questions about the house. Home Assistant 2026 even added a LiteLLM conversation integration for the cloud-flexible; for the fully-local path, the Ollama integration is the standard. The satellite hardware does not change: Voice Preview Edition or ESP32 satellites keep being ears and mouths.

Whisper + Piper (ears/mouth)→Ollama serves the model→HA conversation agent→Expose entities carefully→Tune latency

Pick the model by your VRAM, not by hype

Deploy Ollama + connect Home Assistant

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ./ollama:/root/.ollama
    # GPU hosts: add the runtime/deploy block for your vendor
    # (NVIDIA: runtime nvidia + driver capabilities; see the voice guide notes)
    restart: unless-stopped
  1. Pull your chosen model (ollama pull qwen3:… the tag matching your VRAM tier) and test it in Ollama's own chat first — confirm it answers and, in the API sense, emits tool calls.
  2. In Home Assistant: Settings → Devices & Services → Add Integration → Ollama, pointed at the host and port. Then Settings → Voice assistants → your assistant → set the conversation agent to the Ollama model.
  3. Turn on the 'prefer handling commands locally' style option where offered, so simple commands stay on the fast path and the LLM handles the fuzzy ones.

Exposed entities: the discipline that decides everything

Every entity you expose to the voice assistant rides along in the model's context on every request. The community's rule of thumb: around 30 well-chosen exposed entities ≈ 1,300 tokens of context and snappy, accurate control; expose 400 entities and even a big model slows down and starts grabbing the wrong 'light.' Expose rooms' main actors — lights, climate, covers, media, a few scenes — with clean names and good aliases (the 2026.11 naming cleanup in our naming guide pays off double here). Unexpose the sensors, the diagnostic switches, and the seventeen smart plugs named after their MAC addresses. This one discipline is the difference between the demos that wow people and the installs that get unplugged.

Latency: making it feel alive

Gear for this build

Prices are editorial and change often; buttons show live prices. Some links may earn a commission at no cost to you — see the disclosure page.

Official documentation and references

Frequently asked questions

Do I need a GPU for local LLM voice?

For the LLM part, realistically yes — CPU inference is too slow for conversation. The rest of the pipeline (Whisper/Piper/Speech-to-Phrase) runs happily on modest hardware; if you have no GPU anywhere, plain Assist with good aliases is the better experience.

Which model should I start with?

The largest Qwen3 tool-calling model your VRAM fits comfortably alongside its context: 30B-A3B around 24 GB, mid-size dense models at 16 GB, the 4B Instruct class at 8 GB. Model choice expires fast in this space — the tool-calling requirement and the VRAM tiers are the durable parts.

Is my voice data still private with an LLM in the loop?

Yes, on this architecture: audio is transcribed locally, the LLM runs on your hardware, and actions execute locally. Nothing in this guide's stack sends voice or text to a cloud — that is the point of building it this way.

Related guides