Skip to content

Ember v0.6.6 — multimodal patch

Choose a tag to compare

View all tags
@voidwest voidwest released this 24 Aug 22:39
· 56 commits to main since this release
v0.6.6
5729a77

Multimodal patch on top of the v0.6.5 foundation: live voice conversation and Arabic speech output at reference parity.

Highlights

  • ember voice --converse — full model-in-the-loop conversation in one command: capture ring → energy VAD → streaming audio → VoiceSession → LLM → speech out → playback ring, with barge-in (generation-phase cancel + KV rollback, playback-phase interrupt) and a hermetically pinned transition graph.
  • MMS-TTS (VITS) engine in Rust (src/tts/vits.rs) — first genuinely Arabic-capable open-weight TTS path (character vocab over raw Arabic script, no G2P). Full parity ladder vs HuggingFace reference: embeddings exact, encoder 2.5e-7 rms_rel, flow z 3.4e-7, decoder substages ≤5e-6, waveform rms_rel 8.8e-6 / cosine 0.99999999996 with exact sample counts; residual proven below torch's own f32-vs-f64 cross-precision floor.
  • SpeechOut seam — OuteTTS and MMS-VITS interchangeable behind one streaming API (--vits-model); Arabic S2S integration test drives bank audio → transcript → reply → PCM against real weights.
  • Duplex rework: onset+endpoint survive one chunk, utterance head never lost, single validated resample path.

Fixed

  • CI: benches/multimodal_batch.rs exited 2 when run argless by cargo test --all-targets, turning the v0.6.5 release run red on both tiers; now skips silently.
Assets 2
Loading