Skip to content

Qwen3-TTS Native v0.4.0

Latest
Latest

Choose a tag to compare

View all tags
@luka-loehr luka-loehr released this 18 Jul 16:03
· 6 commits to main since this release
v0.4.0
7acd6d5

Qwen3-TTS Native v0.4.0

The second performance round: a fast research vocoder mode, flash-style decode
attention, INT8 epilogue fusion, and an exact-parity parallel sampling kernel.
At six concurrent streams the engine now synthesizes at more than nine times
real time in aggregate while every individual stream stays faster than real
time.

Immutable deployment

ghcr.io/luka-loehr/qwen3-tts-native@sha256:1761d5f55d6d1d64359e0b240ee75b59a1cd26fbe8815de54aaa2e07b3a45b13
  • Source commit: 7acd6d51eb63a78c120226370a2a4755c05abee7
  • Annotated source tag: v0.4.0
  • OCI platform: linux/arm64
  • Model: Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
  • Model revision: 5ecdb67327fd37bb2e042aab12ff7391903235d3

Deploy only the complete digest reference above.

What changed

  • Fast vocoder mode (QWEN3_TTS_CODEC_FAST=1): a numerically exact
    coalesced weight repack for the transposed-conv upsampler (71 percent of
    codec GPU time was uncoalesced weight reads), block-parallel norms, and
    per-(session, frame-count) CUDA-graph capture of the packet decode.
    Waveform deviation versus the default codec is 3.2e-6 on the parity
    fixtures — about a tenth of one int16 bit. TF32 GEMMs are a further opt-in
    (QWEN3_TTS_CODEC_TF32=1). The default codec path is byte-identical to
    v0.3.0 and its parity gates are unchanged.
  • Flash-style decode attention and INT8 epilogue fusion (residual
    additions and a fused gate/up/SiLU kernel) in the INT8 research mode; each
    transformer layer drops from 17 to 13 kernel nodes.
  • Exact-parity parallel sampling: the sampling kernel was rewritten as a
    parallel sort under the identical total order; verified bit-identical
    selected tokens and RNG state across 1.2 million randomized cases, with
    20-30x speedups on sampled configurations.

Evidence-backed negative results

  • RTN INT4 talker weights: ~12 percent output error on real model matrices
    (INT8: 0.9 percent), at most 1.6x at M=1, slower than INT8 at M=6. Rejected.
  • Codec INT8: exceeds a 1e-3 waveform budget by 10-3600x in every subset;
    continuous vocoder output has no token-quantization cushion. Rejected.

Measured performance (idle DGX Spark, all 24 workload languages natural-EOS)

Metric v0.3.0 INT8 v0.4.0 INT8 + fast codec
Single-stream RTF 0.47 0.287
Single-stream TTFA p50 60 ms 55 ms
B3 aggregate RTF / per-request 0.33 / 0.90 0.139 / 0.37
B6 aggregate RTF / per-request 0.31 / 1.69 0.107 / 0.54

Engineering measurements from short runs with the repository's external HTTP
client; the published controlled benchmark and research paper remain those of
the v0.1.0 release. The recommended deployment enables
QWEN3_TTS_INT8_DECODE=1 and QWEN3_TTS_CODEC_FAST=1; without them the
engine behaves as the BF16 contract path.

Qualification performed

  • fmt, clippy -D warnings, and tests for all six crates at the release commit;
    in-build model identity and license/SBOM gates; codec parity gates in
    default mode.
  • Container qualification on DGX Spark under the hardened profile: readiness
    under 20 s, all 24 workload languages natural-EOS in the recommended mode,
    and the benchmark table above.
  • Not performed: the full schema-v1.2 controlled comparison, SoX validation,
    and formal listening review (informal listening approved by the maintainer).
Assets 2
Loading