Skip to content

Voice Control

Jump to bottom Edit New page
KNOOP edited this page Aug 27, 2026 · 8 revisions

Voice Control

Voice control is the core feature of Ava, allowing you to control smart home devices by speaking.

Compatible with Android 5-16.

0.6.0+ brought native status_panel controls, new manifest fields and on-device TTS/STT references. 0.6.3 rebuilt TTS playback, added voice input health detection, and added a second microWakeWord verification pass. 0.6.5 expanded Voice Stats across the complete Voice Config experience and added smart echo cancellation with room-matched strength. 0.6.6 added a speech gate before wake scoring and on-device noise/echo preview recording. 0.6.7 moved reply text to a restrained caption below the voice orb. 0.6.9 added expressions that follow the reply and clearer speech during audio playback. 0.7.0+ replaced the legacy vsWakeWord engine with OpenWakeWord — a native C++ streaming engine using a shared 96-dimensional acoustic embedding and lightweight ONNX keyword classifiers. 0.7.3 added Chorus Wake multi-device arbitration, ESPHome Noise encryption, ADB Control Gate, graded WebView performance scheduling, settings search, and complete French localization.

How It Works

┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│  You speak  │ -> │ Ava records │ -> │Home Assistant│ -> │ Ava plays   │
│ wake word + │    │ sends audio │    │   speech    │    │   voice     │
│   command   │    │             │    │ recognition │    │   reply     │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘

Detailed Flow:

  1. Standby: Ava continuously listens for wake word (local processing, no internet)
  2. Wake Detection: When wake word detected, plays prompt sound, starts recording
  3. Audio Transmission: Recording sent to Home Assistant via ESPHome protocol
  4. Speech Recognition: Home Assistant's voice assistant performs speech-to-text
  5. Intent Processing: Home Assistant understands intent and executes action
  6. Speech Synthesis: Home Assistant generates voice response
  7. Playback: Ava receives and plays voice response

Topics

Topic Description
LLM Integration Claw Assistant — multi-turn tool calls, Ava device detection, voice intent map, ADB runbook, workspace persona, 50+ tools
Wake Word Engines Dual engines: microWakeWord (uint8 TFLite) and OpenWakeWord (native C++ streaming, shared 96-dim embedding) — architecture, switching, visual feedback, stop words
Custom Wake Words Import custom wake word models via in-app library or APK injection — microWakeWord and OpenWakeWord formats
Voiceprint Recognition On-device speaker identification — Manual mode (enrollment-based gating) and Automatic mode (passive learning)
Audio Event Detection Local detection of household sounds — alarm, baby crying, doorbell, glass breaking, siren, cough, speech
Audio Configuration Microphone settings, noise suppression, echo cancellation, gain, continuous conversation, subtitles, mute
Text to Speech Standard vs Streaming TTS playback, whisper response, announcement, pipeline error i18n, recommended TTS engine
Settings Summary & Services Complete settings reference, Home Assistant services, and FAQ

On-Device TTS & STT Engines

Ava ships with two self-developed voice engine mods that run entirely on the Android device — no separate server, no cloud dependency for STT:

Engine Protocol Port Description
HA Edge TTS Wyoming TTS 10301 Cloud-based TTS using Microsoft Edge TTS voices. 400+ voices in 40+ languages. Natural neural voices without API costs.
HA STT Engine Wyoming STT 10300 Offline speech recognition using SenseVoice model. Supports zh / en / ja / ko / Cantonese. ~230MB model download.

Both mods expose Wyoming protocol servers for Home Assistant auto-discovery via mDNS. Install them from the Mod Store — see the Mod Catalog for full details, configuration, and setup instructions.

Setup:

  1. Install HA Edge TTS and/or HA STT Engine from Settings -> Advanced -> Mod Store
  2. For STT, download the model (~230MB) in mod settings
  3. In Home Assistant, add the Wyoming Protocol integration
  4. Connect to the Ava device IP on port 10301 (TTS) or 10300 (STT)
  5. Select the engine in your voice assistant pipeline

Voice Feedback & Session Control (0.6.2)

Voice Feedback Effects

The wake ripple and screen-edge glow can now be enabled or disabled independently. Users with a custom assistant feedback implementation can turn off Ava's built-in effects so multiple animations do not overlap.

Path: Ava Settings → Voice Config → Voice feedback colors

End Voice Session Manually

Enabling End voice session manually automatically restarts the voice service and registers a Just stop button under Home Assistant Controls. Pressing it while Ava is listening, processing, or speaking immediately terminates the current session and prevents the remaining response from playing. It can also be used in automations as the opposite of Manual Wake.

Path: Ava Settings → Voice Config → Voice replies → End voice session manually

Wake-Word Change Protection

Changing the wake-word model during an active conversation now ends the current Assist run before rebuilding the microphone pipeline. A cancelled session cannot play delayed TTS after the model change or restore processing state that has already ended. This protection applies when changing a wake word in Settings, selecting a model from the library, or changing wake-word configuration through Home Assistant.

Path: Ava Settings → Voice Config → Wake Word

Voice Stats (0.6.5)

When voice behavior looks wrong, users no longer have to begin with guesswork or logs. Voice Stats now covers the complete Voice Config experience, not only the microphone page. Open any of the following locations and pull up the bottom handle to see live status focused on that page:

  • Voice Config overview — quickly confirm whether the complete voice channel is healthy
  • Wake Word — see the active wake method, the latest detection, and recognition behavior
  • Wake Word Library — confirm local wake models, availability, and current detection state
  • Microphone — see whether sound is arriving, which input is active, and the current level and recording state
  • Smart Noise Suppression — confirm which suppression method is active and whether it is working
  • Echo Cancellation — compare microphone, playback echo, and the resulting voice path at a glance
  • Voice Print — see enrollment, verification, quality, and the most recent recognition state
  • Audio Event Detection — see monitored sound types, recognition behavior, and the latest event
  • Streaming TTS — see whether a voice reply is playing, queued, or complete
  • Feedback Accent — confirm the active feedback state used during wake and reply

Each location shows only information relevant to the page that is open. Long values expand on demand and time is presented in a readable form. Voice Stats starts only when opened and stops immediately when closed. Sensitive audio, voiceprint data, and Home Assistant credentials are never displayed.

Path: Ava Settings → Voice Config → any page above → bottom-page handle

Speech Gate for Wake Word (0.6.6)

Room noise could still cause false wakes. A lightweight speech gate now waits for something closer to human speech before wake scoring, reducing empty triggers from TV hum, bumps, and similar sounds. Threshold display in Voice Stats is also more consistent.

Path: Ava Settings → Voice → Wake Word / Voice Stats

Smart Echo Cancellation (0.6.5)

With software echo cancellation enabled, Ava can keep hearing wake and stop words even while playing music, TTS, or a timer alarm. It identifies and reduces Ava's own speaker output inside the microphone signal, leaving the voice that actually came from the room.

Cancellation strength offers Light, Standard, and Strong:

  • Light favors natural speech
  • Standard fits most homes
  • Strong for difficult rooms or louder speakers

An Adaptive mode is also available. Most homes can keep the default Adaptive + Standard setting.

Path: Ava Settings → Voice → Microphone -> Echo cancellation

Hear It Yourself: Noise Suppression and Echo Preview (0.6.6)

At the bottom of Smart noise suppression and Echo cancellation, record a short clip and listen on the spot to compare before and after processing; the echo page also offers a quick test, with data kept only on the device.

Path: Ava Settings → Voice → Microphone -> Smart noise suppression / Echo cancellation

TTS Playback Rebuild (0.6.3)

TTS playback has been comprehensively rebuilt across both classic and streaming modes. Long responses are no longer terminated by a fixed 20-second timeout. Ava now follows actual playback progress, uses duration information when available, and distinguishes active playback, rebuffering, network failure, and a genuinely stalled stream.

If streaming PCM provides no usable audio, stops unexpectedly, or misses an upstream completion event, Ava can recover through URL playback without playing both sources. A stale playback callback could previously terminate the next voice session immediately after a progressive TTS response. Playback state is now fully detached between sessions.

Home Assistant installations serving TTS directly over HTTPS can now enable Use original TTS URL. Ava plays the address returned by Home Assistant instead of rewriting it to LAN HTTP.

Path: Ava Settings → Voice Replies → Maintenance → Use original TTS URL

Voice Input Health Detection (0.6.3)

On affected devices, Ava tests available microphone profiles, sources, and channel layouts during startup. If the selected source remains silent or produces invalid downmixed audio, Ava can rebuild the recording pipeline with a healthier configuration.

microWakeWord now performs a second offline verification before accepting a wake-word detection, reducing false activations in noisy environments.

Reply Captions Below the Orb (0.6.7)

After Ava wakes, recognition and response text now appears like a restrained caption below the voice orb. It finds a suitable position in portrait and landscape, stays within three lines, and changes more gently. There is no large panel covering the dashboard, and the text no longer competes with the sound animation along the bottom.

Expressions That Follow the Reply (0.6.9)

The voice orb can show happy eyes for good news and different expressions for an apology or an important warning. During a long answer, the expression can change as the captions move to the next page. The eyes change during a blink while the orb color stays steady. The text is evaluated directly on the device.

Clearer Speech During Audio Playback (0.6.9)

While Ava is playing music, a prompt, or a long spoken reply, software echo cancellation works to preserve the nearby person's voice. Several sounds playing together are handled as one playback mix, reducing speaker audio without making the person's words unnecessarily thin.

The processing stays on the device. Most homes can keep the default Adaptive + Standard setting. Increase cancellation strength gradually only when recordings received by Home Assistant still contain clear playback from Ava.

Timer Ring and Announcements (0.7.3)

Timer Ring

When a Home Assistant timer finishes, Ava receives a VoiceAssistantTimerEventResponse and enters timer-ringing mode:

  1. Ava ducks media audio and plays a timer-finished chime
  2. Stop-word detection activates (it is normally off during idle)
  3. Saying "stop" stops the timer ring
  4. The chime plays in a 2.5-second cycle — the loud first second masks the stop template, so the DSP can only complete in the quiet part. The 2.5 s gap widens the usable window to ~2.25 s so a reactive "stop" almost always lands in the quiet part across all echo levels
  5. When the timer is cancelled or stopped, media audio un-ducks and stop detection returns to its normal gating

Timer STARTED/UPDATED/CANCELLED events from HA do not affect the ring — HA drops a timer from its registry the moment it fires, so a cancel can only refer to a still-pending timer, never the one ringing.

Announcements and Ask-Question Mode

Home Assistant can send a VoiceAssistantAnnounceRequest to play an announcement on Ava:

Parameter Description
media_id URL of the announcement audio
preannounce_media_id Optional pre-announcement audio (played first)
start_conversation If true, start listening for user's answer after announcement

When start_conversation is true (ask-question / start_conversation pipeline):

  1. Ava plays the announcement
  2. Sends VoiceAssistantAnnounceFinished to HA
  3. Wakes the satellite to listen for the user's answer
  4. Starts a 10-second timeout — if the user does not speak, the pipeline stops automatically

ESPHome Voice Assistant Feature Flags

Ava advertises the following feature flags to Home Assistant:

Flag When Active Purpose
VOICE_ASSISTANT Voice channel enabled Basic voice satellite
API_AUDIO Always (when voice enabled) Audio over ESPHome API
TIMERS Always (when voice enabled) Timer events from HA
ANNOUNCE Always (when voice enabled) HA can send announcements
START_CONVERSATION Always (when voice enabled) Ask-question / start_conversation support
SPEAKER Streaming TTS subtitles enabled Speaker mode for TTS playback
MULTI_CHANNEL_AUDIO Reserved Multi-channel audio (not yet active)

Chorus Wake (0.7.3)

When multiple Ava devices hear the same wake word, Chorus Wake arbitrates which device responds. Only one device opens the assistant — the others yield. See Chorus Wake for full details.

ESPHome Noise Encryption (0.7.3)

Voice satellite connections can now be encrypted using the Noise NNpsk0 protocol. See ESPHome Encryption for full details.

Settings Search (0.7.3)

Ava's settings screen now includes a search bar. Type any setting name or keyword to jump directly to it, without navigating through categories.

Back to Home

Add a custom footer

Core

Optional Modules

Advanced

Clone this wiki locally