Repository navigation
Renderer UI
A React 18 + TypeScript + Tailwind app rendered inside Electron's
sandboxed renderer process. It owns the visible surface of Voice
Gateway and the audio I/O (mic + speakers), and talks to the main
process exclusively through the
[[IPC-Layer|window.vg contextBridge surface]].
Entry points:
| File | Purpose |
|---|---|
main.tsx |
React root + global error boundary |
App.tsx |
View router (main vs settings vs wizard) |
components/MainScreen.tsx |
Default chat surface |
components/PairingWizard.tsx |
First-run wizard |
components/SettingsPanel.tsx |
Tabs: Voz, Microfone, Reconhecimento, Ativação, Conexão, Avançado |
flowchart TB
Main[Main window<br>BrowserWindow #1] -. ipcRenderer.send<br>open-window .-> SettingsWin[Settings window<br>BrowserWindow #2]
Main --> Router[App.tsx<br>detectView]
SettingsWin --> Router
Router -- view=settings --> Panel[SettingsPanel]
Router -- view=main && pairing --> Screen[MainScreen]
Router -- view=main && !pairing --> Wizard[PairingWizard]
The view query parameter is set by loadRendererInto(win, 'settings')
in
src/main/index.ts.
A single index.html bundle serves both windows; the React tree
branches on detectView().
We avoid a heavy global store. Two layers:
-
Zustand for cross-component shared state (settings snapshot,
connection info). See
store/app-store.ts. -
React hooks for view-local state —
useConversation(),useSettings(). Each owns its IPC subscriptions and cleans them up on unmount.
Source:
src/renderer/hooks/useConversation.ts.
The hook that powers the whole call experience. Mounted once by
MainScreen. It:
- Subscribes to FSM state, transcripts, response_text, tts_chunk, error, warning, connection status, hotkey, STT status.
- Holds an
AudioPlaybackinstance (created viauseMemo). - Spins up a fresh
AudioCaptureon everyCAPTURINGentry (replacing the previous one is more bulletproof than reusing). - Calls
playback.beginUtterance()when state →SPEAKING,playback.endUtterance()when state →IDLE/LISTENING_WAKE. - Returns the conversation API (
state,transcript,error,pressTalk,releaseTalk,cancel,bargeIn).
useEffect(() => {
if (state !== 'CAPTURING') {
setLevel(0);
return;
}
let cancelled = false;
const cap = new AudioCapture();
void cap.start({ deviceId: inputDeviceId ?? null }).then(() => {
if (cancelled) { void cap.stop(); return; }
cap.onFrame((frame) => {
const copy = new ArrayBuffer(frame.byteLength);
new Uint8Array(copy).set(new Uint8Array(frame.buffer, frame.byteOffset, frame.byteLength));
window.vg.conversation.sendAudioFrame(copy);
});
cap.onLevel((rms) => { if (!cancelled) setLevel(rms); });
}).catch((err) => {
if (!cancelled) setError(`Não consegui aceder ao microfone: ${err.message}`);
});
return () => { cancelled = true; void cap.stop(); };
}, [state, inputDeviceId]);The cancelled flag handles the press-then-release-fast race: if the
user releases PTT while cap.start() is still awaiting
audioWorklet.addModule(...), the cleanup runs first and the
subsequent .then() callback bails out before attaching listeners to
a stopped capture. The audio-side mirror of this race is documented
in Audio-Pipeline#the-ptt-press-release-race.
Source:
src/renderer/hooks/useSettings.ts.
Two exported hooks:
-
useSettingsBootstrap()— runs once at app start, fetches the currentSettingsand seeds the Zustand store, then subscribes tovg:settings:changedand keeps the store in sync. -
useSettings()— reads the snapshot from Zustand and exposes a typedupdate(patch)that callswindow.vg.settings.set(patch).
The Settings panel reads from this hook and writes through it; the main process broadcasts back any deep-merged result so both windows stay coherent.
SVG-based wordmark + standalone glyph. Rendered in the header and on
the system tray icon. Source:
components/Logo.tsx.
The big circle in the middle that pulses based on FSM state:
| State | Colour | Animation |
|---|---|---|
IDLE |
grey | static |
LISTENING_WAKE |
grey + ring | slow pulse |
CAPTURING |
green | fast pulse, scaled by level
|
STREAMING |
yellow | spinning |
THINKING |
yellow | spinning |
SPEAKING |
purple | wave animation |
ERROR |
red | static |
level (RMS 0..1) comes from useConversation → AudioCapture.onLevel.
The round purple PTT button. Disabled when connection.status !== 'connected' OR (STT isn't ready AND we're not in ERROR — the carve-out
is the State-Machine#error-recovery-via-ptt
path).
disabled={
conv.connection.status !== 'connected' ||
(conv.sttStatus.state !== 'ready' && conv.state !== 'ERROR')
}Press/release map straight to conv.pressTalk() / conv.releaseTalk(),
which fire the IPC events that drive the Conversation-Orchestrator.
Auto-scrolling transcript pane. Each turn shows the user's transcript
followed by the assistant's streaming reply. Empty turns (cancelled or
silent) are filtered out — useConversation's response_text handler
skips empty deltas.
3-step flow:
-
URL →
window.vg.pair.test({ url, token: '' })to validate the URL shape locally. -
Token + Test →
window.vg.pair.test({ url, token })opens a short-lived WS, expectswelcome. On success, the button lights green. Errors map to friendly Portuguese (see IPC-Layer#pairing). - Activation mode → two cards (PTT / wake word). Selection just stages the choice; nothing is persisted until Done!.
On Done! it calls window.vg.pair.save({...}) then sets pairing in
settings, which triggers the main process to bootstrapConversation().
A 6-tab single-window panel. Each tab is a controlled form that calls
update({ … }) on change with a debounced flush (most settings persist
on blur, except booleans/selects which persist immediately).
| Tab | Source |
|---|---|
| Voz | TTS provider switch, Piper voice picker, ElevenLabs key + catalogue, voice-test textarea (type a phrase, hit Reproduzir — uses prepareTestText from shared/tts-test-text.ts to sanitise + cap at 500 chars) |
| Microfone | Input device dropdown, mic status pill (see macOS-Permissions#diagnostic-ui), live VU meter from AudioCapture.onLevel
|
| Reconhecimento | STT provider switch, Whisper model size, language, OpenAI API key |
| Ativação | PTT vs wake word, hotkey input, Minimum audio length slider (filters accidental taps) |
| Conexão | Bridge URL + token (re-pair UI), shows current pairing status |
| Avançado | "Factory reset" button (calls window.vg.settings.reset()) |
Tailwind 3 with a small custom theme in
tailwind.config.cjs.
We use Tailwind utility classes throughout — no CSS Modules, no
styled-components.
The drag region for the frameless macOS window is opt-in via custom
classes in
styles/index.css:
.vg-drag { -webkit-app-region: drag; }
.vg-no-drag { -webkit-app-region: no-drag; }Combine them: any region marked vg-drag lets you drag the window;
any nested vg-no-drag (typically buttons) stays clickable.
The renderer's HTML hardcodes a strict Content-Security-Policy:
<meta http-equiv="Content-Security-Policy" content="
default-src 'self';
script-src 'self';
style-src 'self' 'unsafe-inline';
img-src 'self' data: blob:;
media-src 'self' blob:;
connect-src 'self';
" />The blob: source on media-src is required by the audio-capture
worklet, which is loaded via URL.createObjectURL(new Blob([source]))
(see Audio-Pipeline#worklet-source-embedded-as-a-string).
'unsafe-inline' on style-src is needed because Tailwind's runtime
adds inline <style> tags.
The E2E suite leans on a small set of data-testid attributes that
the renderer treats as public contract. Don't rename them without
updating the matching Playwright assertions — the suite (65 specs
across 19 files) will fail loud.
| Test ID | Component | What it surfaces |
|---|---|---|
call-button |
CallButton |
The PTT button; pointer events drive press/release |
call-button-wrapper |
MainScreen |
The row that wraps the call button + cancel X; carries title with the disabled-reason tooltip when the button is greyed out |
cancel-capture |
MainScreen |
The small "×" button that appears next to call-button while in CAPTURING — clicking it ends the turn without sending audio |
hotkey-hint |
MainScreen |
Subtle hint under the call button — adapts copy to push-to-talk / wake-word mode and renders the configured shortcut |
state-orb |
StateOrb |
Wrapper div. Also carries data-state="…" (the live FSM state) for visual assertions |
transcript |
TranscriptView |
The scroller container |
transcript-empty |
TranscriptView |
The empty-state hint (mode-aware copy) — visible only when the transcript has no turns yet |
transcript-count / transcript-copy / transcript-clear
|
TranscriptView |
Action bar above the transcript — N-mensagens chip, "copiar" (formatted clipboard dump), "limpar" (also bound to Cmd+L). Only render when the transcript has at least one turn. |
transcript-user / transcript-assistant
|
(rows in TranscriptView) | One per turn; lets specs assert ordering + text |
connection-indicator |
MainScreen |
<button> carrying "Ligado (N ms)" / "A ligar…" (with attempt counter) / "Sem ligação". Carries data-clickable="true" while disconnected; clicking then asks main to reconnectNow(). |
mute-toggle |
MainScreen |
Speaker icon in the header. Toggles settings.audio.outputMuted; the renderer drops every incoming TTS chunk while data-muted="true". FSM is untouched. |
main-vu-meter |
MainScreen |
Slim level bar that only renders while state === 'CAPTURING'. Carries data-level (0..1) so specs can sample without clipping. |
auto-launch-toggle |
Avançado tab | Checkbox bound to settings.ui.autoLaunch; main reconciles via app.setLoginItemSettings on every change. |
reveal-log-file / reveal-log-path
|
Avançado tab | Button + path-display chip for "Abrir registo de eventos". Clicking shells out to shell.showItemInFolder(electron-log.path). |
url-input / recent-bridge-chip / bridge-url-history
|
PairingWizard step 1 |
URL input + <datalist> + chip list of the last-N paired URLs (MAX_RECENT_BRIDGE_URLS = 3). |
warning-toast / error-toast
|
MainScreen |
The CommandHint banners. warning-toast auto-dismisses after 4 s |
tab-microfone / tab-voz / tab-ativacao / tab-avancado etc. |
SettingsPanel |
One per tab in the Settings window |
settings-saved-indicator |
SettingsPanel |
Transient "Guardado" pill that flashes for ~1.2 s after a setting change reaches the renderer via settings.onChange
|
mic-permission |
SettingsPanel |
Carries `data-status="granted |
mic-start-test / mic-stop-test / vu-meter
|
Microfone tab | Live VU meter test ride |
output-device-select / output-test-button
|
Microfone tab | Speaker picker + 440 Hz tone test |
tts-test-text / tts-test-button / tts-test-reset / tts-test-char-count
|
Voz tab | Custom-text voice tester |
wake-phrase-input / wake-phrase-hint
|
Ativação tab | Phrase mode input + validation hint |
wake-test-button / wake-test-stop / wake-test-status / wake-test-transcript / wake-tester
|
Ativação tab | "Testar agora" panel for wake detection |
factory-reset / factory-reset-confirm
|
Avançado tab | Two-step danger button |
url-next / token-next / probe-test / probe-result / finish-pairing / pairing-done / open-app
|
PairingWizard |
Step navigation + connection probe in the wizard |
wizard-step-label / wizard-cancel
|
PairingWizard |
"passo X de 3" label + the Cancelar link that wipes typed input and returns to step 1 |
The renderer also exposes a couple of state attributes on
window.document.querySelector('main') so specs can poll
visual-state changes without screenshot diffs:
-
data-just-woke="true"— flashes for ~600 ms when the FSM transitions into CAPTURING via a wake event (wake-word mode only). Used bytests/e2e/ux-round8.spec.tsto confirm the brief visual cue fires. -
document.title— suffixed with the friendly FSM state, e.g.Voice Gateway — A captar áudioor… — A pensar. Updated every transition; seeTITLE_SUFFIXinMainScreen.tsx.
MainScreen.tsx installs a single keydown listener with these
contracts (all bound on Cmd-or-Ctrl):
| Shortcut | Behaviour |
|---|---|
| Escape | Cancel CAPTURING if active, otherwise dismiss the error toast |
| Cmd+, | Open the Settings BrowserWindow (macOS standard) |
| Cmd+L | Wipe the local transcript (new conversation; FSM untouched) |
| Cmd+R | While in ERROR only: dismissError + auto press/release PTT to retry |
In addition to the test ID, StateOrb exposes the live FSM state
as data-state="IDLE" | "LISTENING_WAKE" | "CAPTURING" | "STREAMING" | "THINKING" | "SPEAKING" | "ERROR".
This is the visual contract for any styling test that wants to assert
"the orb turned green during capture" without screenshot comparison.
Note: the attribute reflects whatever the React tree re-rendered to,
which can race past STREAMING/THINKING faster than a 50 ms poll. If
you want to assert every transition was visited, read
window.__vg_state_log (populated by the rig's
instrumentTtsCounter) — that's event-driven and never misses.
UI is exercised by Playwright at
tests/e2e/
(launches the packaged app, drives the pairing wizard, presses the
call button, asserts the FSM state transitions). See
Testing-Guide#the-e2e-rig for the full setup.
Component unit tests live next to their components when sane (small
pure components like Logo, Button) and otherwise are folded into
the integration suite.
Voice Gateway — VivaldiCode/voice-gateway. MIT licensed. Wiki sources live under docs/; run ./docs/sync-wiki.sh to publish.
Getting started
Architecture
- Architecture — system overview
- State-Machine — conversation FSM
- Conversation-Orchestrator — main glue
- WebSocket-Client — Hermes connection
- IPC-Layer — main ↔ renderer
- Settings-And-Storage — config persistence
Audio
- Audio-Pipeline — capture & playback
- Speech-To-Text — STT adapters
- Text-To-Speech — TTS adapters
- Wake-Word-Detection — openWakeWord
Server
- Hermes-Voice-Bridge — Python bridge
- Protocol — WebSocket spec
Platform
- Build-And-Packaging — electron-builder
- macOS-Permissions — TCC, signing
- Renderer-UI — React components
Project
- Testing-Guide — unit, integration, E2E
- Development-Guide — contributing
- Glossary — terms