Skip to content

Overseer voice: wire estate Chatterbox TTS behind VoiceTransportShim (kill ElevenLabs readback dependency) #97

Description

@heavygee

Goal

Kill the ElevenLabs dependency for overseer voice readback by wiring the estate-local Chatterbox TTS (already operated on proxmox, OpenAI-compatible HTTP) behind the existing VoiceTransportShim. Free + local, minimal engineering. This is a bridge win that is independent of the larger agentic-overseer plan — worth doing regardless of where the full-duplex / brain decisions land.

Context

  • Overseer audio summaries in headset work today via ElevenLabs TTS (AGENT_NOTIFY_SUMMARY → corruption-tolerant parse → event/inbox → TTS readback). Operator does not intend to keep paying ElevenLabs.
  • Evaluation + framing: docs/plans/2026-07-29-unmute-voice-evaluation.md (committed). Destination is a free/local agentic voice overseer; mouth/ears are nearly solved on the estate (Speaches STT + Chatterbox TTS). This ticket is Phase 0/1 (free local readback).
  • Existing seam: feat/overseer-stt-tts-endpointsshared/src/voiceTransport.ts (capabilities {stt,tts}, models, default voice) + VoiceTransportShim { transcribe(), synthesize() } + registry in hub/src/voice/transport/. Chatterbox is OpenAI /v1/audio/speech-shaped → drops in with almost no protocol invention.

Scope

  • Add a local-openai (or chatterbox) entry to the voice transport registry (hub/src/voice/transport/).
  • Implement synthesize(text → audio) against Chatterbox /v1/audio/speech (base URL + voice id config).
  • Wire config so overseer readback can select this backend (env / settings).
  • Voice-id mapping + a sane default voice.
  • Tests with fixture audio (mirror existing transport tests).
  • Dogfood: point one overseer summary readback at Chatterbox on :3006, confirm parity with ElevenLabs on the same summary strings.
  • (Optional follow-up, separate) STT via Speaches for PTT dictation — Phase 2, not this ticket.

Out of scope

  • Full-duplex conversation / VAD / barge-in (that's the agentic Phase 3, X-vs-Y decision).
  • The local tool-using LLM "brain" (separate spike).
  • Kyutai / unmute (upgrade path only if Chatterbox fails quality/latency dogfood).

Done when

ElevenLabs can be switched off for overseer summary readback with Chatterbox covering it, free + local, dogfooded on :3006.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions