Skip to content

v0.2.24

Latest

Choose a tag to compare

@QuentinFuxa QuentinFuxa released this 11 Jul 21:49

Streaming LLM translation, refreshed benchmarks and loud failures.

Added

  • AlignAtt translation backend (--translation-backend alignatt): streaming LLM translation through an Alignatt4LLM sidecar. The model drafts ahead over the unstable ASR tail and commits only target words whose attention lands on committed source words, so translations are append-only and released the instant the ASR commits. See docs/translation-alignatt.md.
  • Benchmark figures re-measured on an H100 with the current backends (the README scatter plots dated from March and misrepresented the qwen3 backends). Raw results in benchmarks/h100_scatter/.

Fixed

  • mlx-whisper produced empty captions with torch >= 2.13 (#383, thanks @RobertBartelds-FleetEnergies).
  • The server now fails loudly when the ASR backend produces nothing: warmup errors abort startup with the cause, and a watchdog logs an explicit error if audio flows but no text is ever produced.