Architecture

Architecture

COMSTAR splits into a thin I/O terminal (Raspberry Pi) and a fat brain (AI server). The Mac is a third machine used only for development.

---

System diagram

flowchart TB
  subgraph pi ["Raspberry Pi 4 - terminal"]
    direction TB
    cam["USB camera"]
    mic["Microphone"]
    display["HDMI display"]
    speakers["Speakers"]

    audio["comstar-audio<br/>wake word + VAD"]
    bridge["comstar-bridge Dart<br/>attention / vision / speech clients<br/>ao_reach / WS / HTTP"]
    kiosk["Chromium kiosk<br/>SVG avatar<br/>HTTP :8776 / WS :8777"]

    cam -->|"ffmpeg JPEG in-process"| bridge
    mic --> audio
    audio -->|"WS :8778 PCM"| bridge
    bridge -->|"speak + audioUrl"| kiosk
    kiosk --> display
    kiosk -->|"HTMLAudio"| speakers
  end

  subgraph server ["AI server - RTX 4000 Ada"]
    direction TB
    ao["agentic-orchestration<br/>:8765"]
    stt["AO speech STT sidecar<br/>:8090"]
    tts["AO speech TTS sidecar<br/>:8091"]
    cpai["CodeProject.AI<br/>:32168"]
  end

  bridge -->|"HTTP detect / recognize"| cpai
  bridge -->|"HTTP/WS session overlay"| ao
  ao -.->|"hello.speech URLs"| bridge
  bridge -->|"POST /v1/audio/transcriptions"| stt
  bridge -->|"POST /v1/audio/speech"| tts

Loading

---

What runs where

Runs on the Pi	Runs on the AI server
Camera grab, mic, wake, VAD, kiosk, playback	CodeProject.AI (YOLO + face)
Bridge (attention + clients); optional local STT/TTS fallback	agentic-orchestration :8765 + speech sidecars :8090/:8091
	Hosted MCPs (e.g. Home Assistant)

The split: the Pi captures and plays. The server thinks (AO), sees (CPAI), and — when speech is enabled — transcribes/synthesizes via AO-advertised sidecars (SessionBridge.speechClient). Env COMSTAR_STT_URL / COMSTAR_TTS_URL remain for Mac/dev and when Ada speech is off.

The kiosk has no camera preview — the bridge owns the camera for vision only.

---

Process map (Pi)

Process	Language	Ports	Job
comstar-bridge	Dart	WS :8777 kiosk, :8778 audio, HTTP :8776	Attention, vision poll, AO session, STT/TTS clients
comstar-audio	Python	client → :8778	Wake word, VAD, PCM capture
comstar-kiosk	Chromium + JS	client → :8777	SVG avatar + HTMLAudio playback
comstar-stt (optional)	Python	:8090	Local faster-whisper fallback
comstar-tts (optional)	Python	:8091	Local Piper/sherpa fallback

---

Trust & network boundaries

Phase 1: LAN only.

• Camera frames → CPAI over LAN (not persisted by COMSTAR).
• Utterance PCM → STT (Ada sidecar preferred, or local 127.0.0.1).
• Transcripts → AO Reach session.
• Production WebSockets bind 127.0.0.1 only. LAN bind requires the triple gate.

See Privacy Model and ADR 0003.

---

Client SDK: AO Reach

The bridge uses ao_reach (Dart) for:

• Ephemeral per-session agent overlays
• WebSocket reverse tunnel for Pi-local MCP tools
• Optional hello.speech discovery for STT/TTS sidecars

Do not ferry PCM on the Reach WebSocket or route turns through the planner just for STT.

---

Related ADRs

ADR	Decision
0001	Kiosk is the TTS audio sink
0002	Local HTMLAudio / SVG now; TalkingHead GLB next
0003	Prefer Reach speech on Ada
0004	Sleep + volume via tunnelled terminal MCP

---

Related pages

• Attention Model
• Software Stack
• Contracts
• MCP Topology