Skip to content

v0.5.0 · 实时语音问诊 Realtime Voice Consultation

Latest
Latest

Choose a tag to compare

View all tags
@wananing wananing released this 02 Oct 12:20
· 1 commit to main since this release
v0.5.0
5a99d18

🎙️ 实时语音问诊 · Realtime Voice Consultation

像打电话一样问诊：开放麦连续对话，助手说话时随时开口打断，屏幕同步显示实时字幕、症状要点和分诊卡片。

A phone-call style triage conversation: open mic, interrupt the assistant at any time, live captions, symptom checklist and triage card on screen.

全双工交互体验，级联实现：流式 ASR → LangGraph（与文字通道共用的预问诊子图）→ 流式 TTS，加打断仲裁。每一句话出声前都是完整文本，先过急症规则与出声前检查。
Full-duplex interaction, cascade implementation: streaming ASR → LangGraph (the same clinic subgraph as the text chat) → streaming TTS, with barge-in arbitration. Every sentence is complete text before it is spoken, so it passes emergency rules and a pre-speech check.

Highlights

  • /api/voice WebSocket gateway: server-side endpointing, barge-in, output arbitration, playback receipts, reconnect recovery
  • Emergency rule gate on every utterance, uninterruptible safety line, full-screen call-120 takeover
  • Template read-back confirmation, first sentence spoken first, consultation summary card after hang-up
  • Volcengine streaming ASR + bidirectional TTS (verified live), fake providers for offline development
  • contracts/voice-frames.json: one wire contract checked by backend and frontend tests
  • Design system (tokens, shared result cards, elder-mode text scale) and a redesigned UI
  • Medicine and nearby-pharmacy cards now render; lab reports show every item

Measured with live services and synthetic speech: end of speech → turn committed 0.2–0.3 s, TTS first audio 0.3–0.5 s, follow-up spoken ~2.6–3.0 s.

Known limitations: echo cancellation on real phone speakers is not yet verified (try headphones first); voice covers the AI clinic only.

Full notes: CHANGELOG.md · Design: docs/voice-clinic.md

Assets 2
Loading