Repository navigation
Architecture
┌──────────────────────────────────────────────────────────────────────┐
│ Voice Gateway desktop app │
│ ┌──────────────┐ ┌────────────────┐ ┌───────────────────────────┐ │
│ │ Renderer │ │ Main process │ │ Spawned children │ │
│ │ (React 18) │ │ (Electron) │ │ • python wake-word │ │
│ │ │ │ │ │ • whisper.cpp (STT) │ │
│ │ - StateOrb │ │ - Settings │ │ • piper (TTS) │ │
│ │ - Wizard │ │ - IPC bridge │ └───────────────────────────┘ │
│ │ - useConv.. │◄►│ - WS client │ │
│ │ - audio I/O │ │ - FSM driver │ │
│ └──────────────┘ └────────┬───────┘ │
└─────────────────────────────┼────────────────────────────────────────┘
│ WebSocket (Bearer-auth)
▼
┌──────────────────────────────────────────────────────────────────────┐
│ hermes-voice-bridge │
│ aiohttp WS server ◄───► HermesAdapter ◄───► your Hermes HTTP API │
└──────────────────────────────────────────────────────────────────────┘
-
Audio I/O in the renderer. Microphone capture and Web Audio playback live in the renderer process because that's where
navigator.mediaDevicesandAudioContextlive. The renderer streams frames to the main process via IPC asArrayBuffer. -
Pipeline state in the main process. The
ConversationOrchestratorowns the FSM, theHermesClient, the STT adapter, and the TTS adapter. This is the single source of truth for conversation state. The renderer is a view layer that subscribes to state events. - Heavy native bits as child processes. whisper.cpp, piper, and openwakeword run as subprocesses behind small adapter interfaces. The main process pipes audio through them. This isolates native crashes from the Electron event loop, and makes each adapter trivially swappable / mockable.
- Server bridge for the agent. The desktop app never talks to Hermes directly. The bridge enforces auth and adapts whatever Hermes' real HTTP shape happens to be, so the protocol stays stable as Hermes evolves.
src/
├── shared/ # ← imported by main, preload, renderer; no Node deps
│ ├── constants.ts # IPC channels, audio rates, error codes
│ ├── protocol.ts # WS message types + parsers + capability negotiation
│ ├── types.ts # Settings, PairingInfo, ConnectionInfo, …
│ └── state-machine.ts # pure FSM reducer (29 tests)
│
├── main/
│ ├── index.ts # boot, window, hotkey, tray, pipeline wiring
│ ├── ipc-handlers.ts # settings + pair-test/save handlers
│ ├── tray.ts # menu bar / system tray
│ ├── global-shortcut.ts # hotkey registration
│ └── services/
│ ├── settings-store.ts # electron-store + deep-merge
│ ├── hermes-client.ts # WS client (reconnect, heartbeat)
│ ├── stt-service.ts # SttAdapter + Whisper + OpenAI
│ ├── tts-service.ts # TtsAdapter + Piper + ElevenLabs
│ ├── wake-word-service.ts # spawn supervisor for python runner
│ └── conversation-orchestrator.ts # FSM driver, glue
│
├── preload/
│ └── index.ts (compiled to index.cjs) # contextBridge surface
│
└── renderer/
├── App.tsx
├── components/ # Wizard, MainScreen, StateOrb, CallButton, TranscriptView
├── hooks/ # useSettings, useConversation
├── lib/ # audio-capture.ts (AudioWorklet), audio-playback.ts
└── store/ # Zustand
| Action | Process |
|---|---|
| Microphone capture / playback | Renderer (Web Audio) |
| Conversation FSM | Main |
| WebSocket to bridge | Main |
| STT (whisper.cpp or OpenAI API) | Main |
| TTS (Piper or ElevenLabs) | Main |
| Wake word (openwakeword) | Spawned Python child |
| Global hotkey | Main |
| Tray | Main |
| settings.json storage | Main (electron-store) |
| UI / state subscriptions | Renderer (Zustand) |
The preload script is compiled to CommonJS (index.cjs) — Electron with sandbox: true cannot load ESM preloads, and we use contextIsolation: true so the renderer only sees the explicit window.vg API surface.
The renderer's HTML CSP allows ws:, wss:, and https: for connect-src (the bridge URL is user-chosen) but blocks inline scripts entirely.
- The FSM is pure — same
(ctx, event, env)always produces the same next context. 29 vitest cases pin its transitions. - External I/O lives behind adapter interfaces (
SttAdapter,TtsAdapter,wsFactory,spawnImpl). Tests inject fakes. No mocking framework required. - The mock bridge in
tests/integration/__mocks__/mock-bridge-server.tsis the same shape as the real bridge and is used by both vitest integration tests and Playwright E2E. - The renderer is decoupled from electron via
window.vg. Components can be exercised by Playwright against the real Electron build (tests/e2e/).
electron-vite natively understands the main/preload/renderer split, can run all three in one dev process with hot reload, and outputs the right module formats for each (ESM for main, CJS for preload, ESM bundle for renderer). It also wires the ELECTRON_RENDERER_URL env var so the main process knows whether to load a dev URL or a built index.html.
Voice Gateway — VivaldiCode/voice-gateway. MIT licensed. Wiki sources live under docs/; run ./docs/sync-wiki.sh to publish.
Getting started
Architecture
- Architecture — system overview
- State-Machine — conversation FSM
- Conversation-Orchestrator — main glue
- WebSocket-Client — Hermes connection
- IPC-Layer — main ↔ renderer
- Settings-And-Storage — config persistence
Audio
- Audio-Pipeline — capture & playback
- Speech-To-Text — STT adapters
- Text-To-Speech — TTS adapters
- Wake-Word-Detection — openWakeWord
Server
- Hermes-Voice-Bridge — Python bridge
- Protocol — WebSocket spec
Platform
- Build-And-Packaging — electron-builder
- macOS-Permissions — TCC, signing
- Renderer-UI — React components
Project
- Testing-Guide — unit, integration, E2E
- Development-Guide — contributing
- Glossary — terms