Qwen3-TTS Native v0.4.0

Qwen3-TTS Native v0.4.0

The second performance round: a fast research vocoder mode, flash-style decode
attention, INT8 epilogue fusion, and an exact-parity parallel sampling kernel.
At six concurrent streams the engine now synthesizes at more than nine times
real time in aggregate while every individual stream stays faster than real
time.

Immutable deployment

ghcr.io/luka-loehr/qwen3-tts-native@sha256:1761d5f55d6d1d64359e0b240ee75b59a1cd26fbe8815de54aaa2e07b3a45b13

• Source commit: 7acd6d51eb63a78c120226370a2a4755c05abee7
• Annotated source tag: v0.4.0
• OCI platform: linux/arm64
• Model: Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
• Model revision: 5ecdb67327fd37bb2e042aab12ff7391903235d3

Deploy only the complete digest reference above.

What changed

• Fast vocoder mode (QWEN3_TTS_CODEC_FAST=1): a numerically exact
  coalesced weight repack for the transposed-conv upsampler (71 percent of
  codec GPU time was uncoalesced weight reads), block-parallel norms, and
  per-(session, frame-count) CUDA-graph capture of the packet decode.
  Waveform deviation versus the default codec is 3.2e-6 on the parity
  fixtures — about a tenth of one int16 bit. TF32 GEMMs are a further opt-in
  (QWEN3_TTS_CODEC_TF32=1). The default codec path is byte-identical to
  v0.3.0 and its parity gates are unchanged.
• Flash-style decode attention and INT8 epilogue fusion (residual
  additions and a fused gate/up/SiLU kernel) in the INT8 research mode; each
  transformer layer drops from 17 to 13 kernel nodes.
• Exact-parity parallel sampling: the sampling kernel was rewritten as a
  parallel sort under the identical total order; verified bit-identical
  selected tokens and RNG state across 1.2 million randomized cases, with
  20-30x speedups on sampled configurations.

Evidence-backed negative results

• RTN INT4 talker weights: ~12 percent output error on real model matrices
  (INT8: 0.9 percent), at most 1.6x at M=1, slower than INT8 at M=6. Rejected.
• Codec INT8: exceeds a 1e-3 waveform budget by 10-3600x in every subset;
  continuous vocoder output has no token-quantization cushion. Rejected.

Measured performance (idle DGX Spark, all 24 workload languages natural-EOS)

Metric	v0.3.0 INT8	v0.4.0 INT8 + fast codec
Single-stream RTF	0.47	0.287
Single-stream TTFA p50	60 ms	55 ms
B3 aggregate RTF / per-request	0.33 / 0.90	0.139 / 0.37
B6 aggregate RTF / per-request	0.31 / 1.69	0.107 / 0.54

Engineering measurements from short runs with the repository's external HTTP
client; the published controlled benchmark and research paper remain those of
the v0.1.0 release. The recommended deployment enables
QWEN3_TTS_INT8_DECODE=1 and QWEN3_TTS_CODEC_FAST=1; without them the
engine behaves as the BF16 contract path.

Qualification performed

• fmt, clippy -D warnings, and tests for all six crates at the release commit;
  in-build model identity and license/SBOM gates; codec parity gates in
  default mode.
• Container qualification on DGX Spark under the hardened profile: readiness
  under 20 s, all 24 workload languages natural-EOS in the recommended mode,
  and the benchmark table above.
• Not performed: the full schema-v1.2 controlled comparison, SoX validation,
  and formal listening review (informal listening approved by the maintainer).

Qwen3-TTS Native v0.3.0

Qwen3-TTS Native v0.3.0

A performance-focused release: the native engine now performs cross-request
lockstep batched decoding, captures the complete decode frame as a CUDA graph,
and offers an opt-in INT8 weight-only decode mode.

Immutable deployment

ghcr.io/luka-loehr/qwen3-tts-native@sha256:276990b3b693136d6bd93b098682f3860751afb3f5baa99eb5aec2b85c086575

• Source commit: c193e259cac16219d18d312755e4bbeef82f04f0
• Annotated source tag: v0.3.0
• OCI platform: linux/arm64
• Model: Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
• Model revision: 5ecdb67327fd37bb2e042aab12ff7391903235d3

Deploy only the complete digest reference above.

What changed

• Lockstep batched decode (talker ABI v3). When several requests generate
  concurrently, one batched native call performs the talker and 15-step
  code-predictor decode for every active session, so each shared weight matrix
  is read from memory once per codec frame instead of once per session. KV
  caches, sampling state, histories, random states, and delivery leases stay
  request-local.
• CUDA-graph frames. The complete batched predictor+talker frame is
  captured once per session tuple and replayed thereafter, eliminating
  per-frame launch overhead for the ~1,600-kernel decode sequence.
• Opt-in INT8 decode (QWEN3_TTS_INT8_DECODE=1): decode GEMM weights are
  quantized to per-output-channel symmetric INT8 at engine load, directly from
  the pinned BF16 artifact; the checkpoint identity and artifact contract are
  unchanged, and prefill stays BF16. The default engine remains BF16.
• First-frame priority: a session's first frame bypasses the lockstep
  rendezvous, so first-audio latency does not pay a batching penalty.

Measured performance (idle DGX Spark, all 24 workload languages natural-EOS)

Metric	v0.2.0 engine	v0.3.0 BF16	v0.3.0 INT8
Single-stream RTF	0.76	0.72	0.47
Single-stream TTFA p50	81 ms	75 ms	60 ms
B3 aggregate RTF / per-request	0.59 / 1.65	0.41 / 1.10	0.33 / 0.90
B6 aggregate RTF / per-request	0.59 / 3.22	0.36 / 1.91	0.31 / 1.69

These figures come from short engineering runs with the repository's external
HTTP client on an otherwise idle Spark. They are engine-comparison numbers,
not a replacement for the full schema-v1.2 controlled evidence bundle; the
published controlled benchmark and research paper remain those of the
v0.1.0 release.

INT8 mode changes decode numerics (per-channel quantization error is on the
order of 4e-3 relative); all transport, EOS, and language gates pass in both
modes. Listening review of INT8 output is the deployer's responsibility.

Qualification performed for this release

• cargo fmt, clippy -D warnings, and tests for all six crates at the
  release commit.
• In-build model identity gates (manifest and weight SHA-256, full native
  artifact validation) and release-metadata license/SBOM validation.
• Container qualification on DGX Spark: hardened run profile (read-only root,
  cap-drop ALL, no-new-privileges), readiness within 20 s, all 24 workload
  languages with natural EOS in BF16 and INT8 modes, and the B1/B3/B6
  benchmark table above.
• Not performed for this release: the full 12-run schema-v1.2 controlled
  comparison, SoX audio validation, and human listening review.

Qwen3-TTS Native v0.1.0

Qwen3-TTS Native v0.1.0

Qwen3-TTS Native is a Rust and CUDA runtime for progressive Qwen3-TTS 1.7B
VoiceDesign inference on NVIDIA DGX Spark. The runtime and image do not include
Python, Node.js, PyTorch, SGLang, vLLM, voice cloning, reference-audio
conditioning, or the retired 0.6B model.

Immutable deployment

ghcr.io/luka-loehr/qwen3-tts-native@sha256:5efbe3143e5d305ae3b82603263a503574da15160691bb6de363becfff5612d6

• Source commit: d745ebcbaf8014b6053f103470d83595d87878e3
• Annotated source tag: v0.1.0
• OCI platform: linux/arm64
• Model: Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
• Model revision: 5ecdb67327fd37bb2e042aab12ff7391903235d3
• OCI transfer size: 3,871,303,232 bytes
• Docker-reported local image size in an initially empty content store:
  3,871,309,521 bytes

Deploy only the complete digest reference above. v0.1.0 and latest are
convenience aliases verified to resolve to the same digest; they are not the
deployment identity.

API

• Progressive multipart PCM16 at 24 kHz mono.
• Buffered PCM16 WAV.
• Text plus natural-language voice_description.
• language=auto or Chinese, English, Japanese, Korean, German, French,
  Russian, Portuguese, Spanish, or Italian.
• Request UUIDs, bounded cancellation, backpressure, natural-EOS reporting,
  liveness/readiness, capabilities, privacy-safe metrics, and graceful
  shutdown.

Controlled benchmark

The release includes a validated two-round Native-versus-stock-SGLang matrix
on one NVIDIA DGX Spark: B1/B3/B6, 24 warm-ups per cell, 200/210/240 measured
requests per cell, 2,600 successful measured requests overall, 100 ms
telemetry, and no competing CUDA process.

• Native aggregate RTF across the two accepted rounds was 0.800–0.803 at B1,
  0.641–0.642 at B3, and 0.617–0.618 at B6.
• Stock SGLang aggregate RTF was 0.497–0.499 at B1, 0.186–0.197 at B3, and
  0.102–0.112 at B6; stock therefore had higher aggregate throughput.
• Native TTFA p95 was 93.89–95.58 ms at B1, 215.55–216.81 ms at B3, and
  405.38–406.04 ms at B6. Stock completion-buffered TTFA p95 was
  2,691.51–3,145.88 ms across the three profiles.
• Peak observed GPU unified memory was 5.68 GB for Native and 108.90 GB for
  stock SGLang, approximately a 19.2-fold reduction.

Stock SGLang is completion-buffered and exposes no authoritative EOS metadata;
its TTFA and completion semantics are therefore reported explicitly rather
than presented as equivalent to Native progressive delivery. Performance does
not constitute a human speech-quality claim.

Supply-chain and GPU acceptance

• BuildKit SPDX SBOM: verified against the immutable OCI index.
• Maximum SLSA provenance: verified; no credential-like field or private host
  path was present.
• Embedded CycloneDX Rust SBOM and third-party license report: verified.
• Gitleaks full-history result: 163 commits scanned, no leaks.
• Grype high/critical findings: 0 with Grype 0.116.0 and the current database.
• Keyless Cosign identity:
  https://github.com/luka-loehr/qwen3-tts-native/.github/workflows/sign-ghcr-image.yml@refs/heads/main;
  issuer https://token.actions.githubusercontent.com; workflow run
  29634701244 passed exact-claim verification.
• Pull into a distinct, initially empty Docker content store: passed. After
  publication, a second pull with a completely empty Docker client profile
  also resolved the public package anonymously to the exact release digest.
• Digest-specific hardened startup: passed as UID 10001:10001, read-only,
  all Linux capabilities dropped, no-new-privileges, PID limit 256.
• Digest-specific functional smoke: one progressive German request completed
  with exactly one start and one terminal natural-EOS event; one buffered WAV
  completed and passed SoX validation (24 kHz, 16-bit, mono, 5.92 seconds,
  maximum amplitude 0.792145, no clipping). Observed GPU process memory was
  4,821 MiB.
• Performance acceptance is the already completed 12-cell, 2,600-request
  controlled matrix described above. It was intentionally not repeated during
  tag promotion.

Assets

• qwen3-tts-native-vs-sglang-stock-dgx-spark-2026-07-17-428307c-report.pdf
• qwen3-tts-native-paper.pdf
• qwen3-tts-native-paper-arxiv-source.tar.gz
• qwen3-tts-native-benchmark-evidence-428307c.tar.zst
• qwen3-tts-native-rejected-benchmark-audits.tar.zst
• qwen3-tts-native-release-evidence-d745ebc.tar.zst
• qwen3-tts-native-v0.1.0-sample.wav
• SHA256SUMS

Scope and limitations

• Qualified only on the recorded NVIDIA DGX Spark/GB10 software stack.
• VoiceDesign only; no voice cloning or reference audio.
• Turkish has no explicit language ID in the pinned checkpoint and is not
  advertised as supported.
• Transport and systems qualification do not replace subjective listening,
  pronunciation, or instruction-adherence evaluation.
• The paper is published with the release; submission to arXiv itself remains
  an author-controlled action outside this automated release.