A fully local voice assistant for the home — your own Jarvis, built from open-source pieces, running on hardware you control.
No cloud APIs. No subscriptions. Nothing leaves the house except web searches you ask it to make.
Talk to it out loud, or through Telegram from anywhere. It remembers what you tell it, sets reminders and recurring routines on its own, and asks before doing anything that touches the outside world.
- Why local?
- What it can do today
- What it can't do yet
- Architecture
- Stack
- Getting started
- Roadmap status
- A note on scope
Because a home assistant that phones a cloud API for every sentence is a home assistant that stops working when your internet does, and one whose "brain" belongs to someone else. Every piece of IDUS — speech recognition, the language model, text-to-speech, web search — runs on hardware you control.
Voice loop — say "Hey Jarvis," wait for the pause, then talk. IDUS listens, transcribes, thinks, and answers out loud, in Russian or English (it follows whichever language you use, mid-sentence code-switching included).
A real conversational brain — backed by a local LLM (Ollama), with a personality that doesn't lecture you and doesn't pretend to know things it doesn't. If it needs a fact, it looks it up; if it mishears you, it says so instead of guessing.
Tool use — the model doesn't just talk, it acts:
| Time, date, calculator, timers | Web search — self-hosted, no API keys, no tracking |
| Shopping lists & voice notes | Reminders — survive a restart, reach you by voice or Telegram |
| Automation engine — recurring rules, not one-off answers | Cooking assistant — step-by-step, with per-step timers |
| On-the-fly translation | Jokes and trivia on request |
The bottom row needed zero new code — just a model doing what models do.
Memory that grows on its own — long-term facts persist across restarts, and once a day IDUS quietly reviews the last day's conversations, writes itself a short summary, and picks out anything worth remembering that you never explicitly asked it to save.
Knows who's talking — voice-based speaker identification, so it can recognize you (and anyone else you enroll) by the sound of your voice.
A second front door — a Telegram bot wired into the exact same brain, text or voice messages, so you can talk to your home from anywhere. Only responds to you.
An agentic core with real guardrails — before doing anything with consequences (sending a message, building a shopping cart on a real account), it stops and asks "yes or no?" — and your answer is understood by meaning, not by matching a fixed list of keywords. Some actions (placing an actual order, for instance) aren't just gated behind confirmation — the code to complete them simply doesn't exist. That's a design choice, not a missing feature.
| Not yet | Why |
|---|---|
| Control smart home devices | No hardware purchased yet — next once it is |
| Media on the TV, presence awareness, reliable multi-step chains | Same hardware decision, or a bigger local model than a modest GPU can run |
| Place a real food order | Cart-building works end-to-end; the delivery platform's bot detection blocks automated login, and defeating that wasn't a line worth crossing — it stops at "cart's ready, go finish it" |
| Message your contacts as you | Sending as you to someone who's never messaged the bot needs your own Telegram account automated — a deliberate, not-yet-made call |
voice/ wake word · voice activity detection · speech-to-text · text-to-speech · speaker ID
brain/ the LLM loop, tool-calling, confirmation gate, nightly reflection
tools/ everything the model can actually do — one file per capability
memory/ SQLite-backed facts, reminders, automations, episodes, shopping list, notes
scripts/ entry points — the voice loop, the Telegram bot, and test/setup utilities
config/ downloaded models and voices, local service configuration
The split between
voice/(how you talk to it) andbrain/(what it thinks) is deliberate — the Telegram bot is a second front door onto the same brain, not a separate assistant, and a future phone-companion channel can be added the same way without touching the core.
| Piece | What it does | Why this one |
|---|---|---|
| Ollama + Qwen2.5 7B | The language model | Runs fully local, strong at Russian |
| faster-whisper | Speech-to-text | Fast, accurate enough, runs on CPU |
| Silero TTS | Text-to-speech | Natural-sounding, fully local |
| Silero VAD | Voice activity detection | Small, fast, reliable |
| openWakeWord | Wake word detection | Trainable, no cloud account needed |
| SpeechBrain | Speaker identification | Pretrained ECAPA-TDNN embeddings |
| SearXNG | Web search | Self-hosted, no API keys |
| python-telegram-bot | Remote access channel | Mature async Telegram API wrapper |
| SQLite | Everything persistent | One file, zero setup, plenty for one household |
Requires Python 3.11+, uv, Ollama, and Docker (for the search backend).
uv sync
ollama pull qwen2.5:7b
docker compose up -d # starts the SearXNG search backend
cp .env.example .env # fill in your Telegram bot token, if you want that channelRun the voice assistant:
uv run scripts/jarvis_loop.pyRun the Telegram bot (separately, same brain):
uv run scripts/telegram_loop.pyIndividual scripts/*_test.py files exercise each component (VAD, STT, TTS, wake word,
memory, reminders, automations) in isolation — useful when something's not behaving and you
want to know which layer to blame.
Full stage-by-stage breakdown
| # | Stage | Status |
|---|---|---|
| 0 | Environment & tooling |  |
| 1 | Voice loop (wake word → STT → TTS) | |
| 2 | Local LLM connection | |
| 3 | Tool-calling | |
| 4 | Internet access (search) | |
| 5 | Long-term memory | |
| 6 | Smart home foundation |  |
| 7 | Reminders, media, scenes |  reminders done; media/scenes need devices |
| 8 | UX & reliability | latency + degradation done; barge-in needs a headset |
| 9 | Security & privacy | |
| 10 | Packaging & autostart | Ubuntu-specific |
| 11 | Remote access (Telegram) | |
| 12 | Agentic core | confirmation gate done; full planner needs a bigger model |
| 13 | Media stack | |
| 14 | External actions (ordering, messaging) | owner-messaging done; third-party & ordering blocked |
| 15 | Memory 2.0 & proactivity | episodes + reflection done; procedural memory needs devices |
Everything marked done has been verified against real hardware and a real voice, not just tested in isolation.
A few things were deliberately not built, on purpose, not by oversight:
- No feature here will ever complete a payment or place a real order — that boundary is structural in the code, not a setting you can flip
- No automation here defeats another service's bot protection — when that's the only path forward, the feature stops at "here's what I found, you finish it"
Local-first and user-controlled cuts both ways: it means no one else's cloud can pull the plug on your assistant, and it means your assistant doesn't get to cut corners on your behalf either.