Skip to content

TranscrIA 0.3.6 — served STT runtimes

Choose a tag to compare

View all tags
@Martossien Martossien released this 12 Jul 21:45
· 430 commits to main since this release
v0.3.6
762cdb7

Served STT runtimes

audio.cpp and parakeet.cpp become first-class engines, managed with the same lifecycle discipline as the local arbitration LLM: TranscrIA installs them pinned, starts them on demand before jobs, health-checks them, admits them against available VRAM, and stops them. Both were qualified on our published benchmark of real French meetings (docs/STT_BENCHMARK_REAL_MEETINGS.md).

New

  • qwen3asr (audio.cpp + Qwen3-ASR-1.7B, Apache-2.0) — second-best text WER of our entire benchmark (0.421), ~12 s per 5-minute window on one card, ~4 GB of weights. Install: venv/bin/python -m transcria.installer.cli audiocpp --with-model.
  • nemotron (parakeet.cpp + Nemotron 3.5 ASR 0.6B GGUF) — 0.492 WER at ~2 s per 5-minute window, 1.4 GB of weights. Install: … installer.cli parakeetcpp, GGUF from the Models page.
  • On-demand start in both topologies. All-in-one: a loopback backend URL plus a matching engine declared in resource_node.engines is enough — the job preflight launches the engine in-process (same A/B/C supervisor cycle as the GPU node's /engines/ensure; verified end-to-end with the engine stopped). Split: the resource node path is unchanged.
  • Per-engine health checks (health_path, health_mode) — parakeet-server has no /v1/models but a clean /health; the http_any mode covers single-model servers that load weights before binding (never for vLLM).
  • Native fallback, never implicit: inference.stt.backends.<name>.fallback_backend picks the native engine to fall back to; a served backend without one fails explicitly instead of silently loading a 6 GB gated model.
  • Pinned everywhere: installer builds check out exact commits (CMAKE_CUDA_ARCHITECTURES=native — a wrong default arch SIGABRTs at first inference); the GPU Docker images (:latest, :bundled, resource-node) now bake both binaries pinned under /opt/runtimes (models stay in volumes). Upgrading a runtime = new SHA + rebuild + re-qualification on the benchmark.
  • Models page rows for served engines (Qwen3-ASR download delegated to audio.cpp's own manager; Nemotron GGUF via the standard single-file path).
  • Doctor check for provisioned runtimes: a declared qwen3asr/nemotron engine whose binary is missing or built from a stale commit gets a WARN with the exact provisioning command.
  • install.sh --with-stt-runtimes provisions both runtimes at install time (opt-in).

Fixed

  • Config schema now accepts a served backend name in models.stt_backend when it is routed (inference.stt.backends.<name>.url) — validation used to reject what the factory routed correctly.
  • No more phantom 6 GB VRAM reservation for remotely-routed backends (get_backend_vram_mb returns 0 for them).

No database migration, no new Python dependency. Everything is opt-in; an existing config.yaml stays valid. Full details: CHANGELOG · setup guide: docs/EXTERNAL_STT_RUNTIMES.md · upgrade notes: docs/UPGRADE.md.

🤖 Generated with Claude Code

Assets 2
Loading