Architecture
COMSTAR splits into a thin I/O terminal (Raspberry Pi) and a fat brain (AI server). The Mac is a third machine used only for development.
flowchart TB
subgraph pi ["Raspberry Pi 4 - terminal"]
direction TB
cam["USB camera"]
mic["Microphone"]
display["HDMI display"]
speakers["Speakers"]
audio["comstar-audio<br/>wake word + VAD"]
bridge["comstar-bridge Dart<br/>attention / vision / speech clients<br/>ao_reach / WS / HTTP"]
kiosk["Chromium kiosk<br/>SVG avatar<br/>HTTP :8776 / WS :8777"]
cam -->|"ffmpeg JPEG in-process"| bridge
mic --> audio
audio -->|"WS :8778 PCM"| bridge
bridge -->|"speak + audioUrl"| kiosk
kiosk --> display
kiosk -->|"HTMLAudio"| speakers
end
subgraph server ["AI server - RTX 4000 Ada"]
direction TB
ao["agentic-orchestration<br/>:8765"]
stt["AO speech STT sidecar<br/>:8090"]
tts["AO speech TTS sidecar<br/>:8091"]
cpai["CodeProject.AI<br/>:32168"]
end
bridge -->|"HTTP detect / recognize"| cpai
bridge -->|"HTTP/WS session overlay"| ao
ao -.->|"hello.speech URLs"| bridge
bridge -->|"POST /v1/audio/transcriptions"| stt
bridge -->|"POST /v1/audio/speech"| tts
| Runs on the Pi | Runs on the AI server |
|---|---|
| Camera grab, mic, wake, VAD, kiosk, playback | CodeProject.AI (YOLO + face) |
| Bridge (attention + clients); optional local STT/TTS fallback | agentic-orchestration :8765 + speech sidecars :8090/:8091
|
| Hosted MCPs (e.g. Home Assistant) |
The split: the Pi captures and plays. The server thinks (AO), sees (CPAI), and — when speech is enabled — transcribes/synthesizes via AO-advertised sidecars (SessionBridge.speechClient). Env COMSTAR_STT_URL / COMSTAR_TTS_URL remain for Mac/dev and when Ada speech is off.
The kiosk has no camera preview — the bridge owns the camera for vision only.
| Process | Language | Ports | Job |
|---|---|---|---|
comstar-bridge |
Dart | WS :8777 kiosk, :8778 audio, HTTP :8776
|
Attention, vision poll, AO session, STT/TTS clients |
comstar-audio |
Python | client → :8778
|
Wake word, VAD, PCM capture |
comstar-kiosk |
Chromium + JS | client → :8777
|
SVG avatar + HTMLAudio playback |
comstar-stt (optional) |
Python | :8090 |
Local faster-whisper fallback |
comstar-tts (optional) |
Python | :8091 |
Local Piper/sherpa fallback |
Phase 1: LAN only.
- Camera frames → CPAI over LAN (not persisted by COMSTAR).
- Utterance PCM → STT (Ada sidecar preferred, or local
127.0.0.1). - Transcripts → AO Reach session.
- Production WebSockets bind
127.0.0.1only. LAN bind requires the triple gate.
See Privacy Model and ADR 0003.
The bridge uses ao_reach (Dart) for:
- Ephemeral per-session agent overlays
- WebSocket reverse tunnel for Pi-local MCP tools
- Optional
hello.speechdiscovery for STT/TTS sidecars
Do not ferry PCM on the Reach WebSocket or route turns through the planner just for STT.
| ADR | Decision |
|---|---|
| 0001 | Kiosk is the TTS audio sink |
| 0002 | Local HTMLAudio / SVG now; TalkingHead GLB next |
| 0003 | Prefer Reach speech on Ada |
| 0004 | Sleep + volume via tunnelled terminal MCP |