The gap
Every voice agent example in examples/ assumes cloud APIs somewhere in the loop. The pieces for a fully local pipeline already ship, openai.LLM.with_ollama(), silero VAD, the turn-detector model, and the openai plugin's base_url override reaches any OpenAI-compatible local server for STT and TTS. But nothing in the repo composes them, so anyone building private, on-prem, or edge voice agents is reverse-engineering the combination from plugin docs and Discord threads.
#5351 proposed a local HF LLM plugin. This is the cheaper move: no new plugin surface, just an example wiring together what already exists.
What I'd contribute
examples/voice_agents/local_only/ — a voice agent with zero cloud calls:
- STT: openai plugin → speaches (faster-whisper) via local OpenAI-compatible endpoint
- LLM:
openai.LLM.with_ollama() → Ollama
- TTS: openai plugin → kokoro-fastapi, same pattern
- VAD + turn detection: silero + turn-detector (already local)
Plus a README that treats latency honestly: per-stage numbers (STT, TTFT, TTS first-byte) measured on named consumer hardware, model choices with tradeoffs, and the failure modes I hit. Not "it works", but what it actually costs to run this loop offline.
Why me
I ship an offline edge-AI device for a living (safety-critical foraging ID, RPi5 + Hailo NPU, the whole pipeline runs without a network), and I run local LLM infra daily. This example would be maintained by someone who actually lives on this stack.
One thing I hit while building this
livekit.plugins.turn_detector is marked deprecated in favor of livekit.agents.inference.TurnDetector, but the replacement takes api_key/api_secret and routes through the inference gateway. If the plugin is removed on schedule, there's no local turn-detection path left. Is local turn detection intended to survive the migration? The example currently uses the plugin with a comment explaining why; happy to update it when there's a local-capable replacement.
The ask
Any objection to the shape or location before I open the PR? Happy to adjust scope, split STT/TTS server choices differently, or fold it into an existing example dir if you'd rather keep voice_agents/ flat.
The gap
Every voice agent example in
examples/assumes cloud APIs somewhere in the loop. The pieces for a fully local pipeline already ship,openai.LLM.with_ollama(), silero VAD, the turn-detector model, and the openai plugin'sbase_urloverride reaches any OpenAI-compatible local server for STT and TTS. But nothing in the repo composes them, so anyone building private, on-prem, or edge voice agents is reverse-engineering the combination from plugin docs and Discord threads.#5351 proposed a local HF LLM plugin. This is the cheaper move: no new plugin surface, just an example wiring together what already exists.
What I'd contribute
examples/voice_agents/local_only/— a voice agent with zero cloud calls:openai.LLM.with_ollama()→ OllamaPlus a README that treats latency honestly: per-stage numbers (STT, TTFT, TTS first-byte) measured on named consumer hardware, model choices with tradeoffs, and the failure modes I hit. Not "it works", but what it actually costs to run this loop offline.
Why me
I ship an offline edge-AI device for a living (safety-critical foraging ID, RPi5 + Hailo NPU, the whole pipeline runs without a network), and I run local LLM infra daily. This example would be maintained by someone who actually lives on this stack.
One thing I hit while building this
livekit.plugins.turn_detectoris marked deprecated in favor oflivekit.agents.inference.TurnDetector, but the replacement takesapi_key/api_secretand routes through the inference gateway. If the plugin is removed on schedule, there's no local turn-detection path left. Is local turn detection intended to survive the migration? The example currently uses the plugin with a comment explaining why; happy to update it when there's a local-capable replacement.The ask
Any objection to the shape or location before I open the PR? Happy to adjust scope, split STT/TTS server choices differently, or fold it into an existing example dir if you'd rather keep
voice_agents/flat.