Skip to content

Support Qwen3-TTS prompt modes without ICL reference codes - #202

Open
fedeizzo wants to merge 1 commit into
0xShug0:mainfrom
fedeizzo:fedeizzo/qwen3-tts-validation
Open

Support Qwen3-TTS prompt modes without ICL reference codes#202
fedeizzo wants to merge 1 commit into
0xShug0:mainfrom
fedeizzo:fedeizzo/qwen3-tts-validation

Conversation

@fedeizzo

@fedeizzo fedeizzo commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Originated from #170

What does this PR do?

  • include/engine/models/qwen3_tts/talker.h, src/models/qwen3_tts/talker.cpp: add validate_qwen3_talker_voice_clone_prefill() to validate prompt mode parameters and add support for x_vector_only_mode prompt embedding construction when voice prompts do not include ICL reference codes.
  • src/models/qwen3_tts/session.cpp: support voice prompt execution without ICL reference codes by conditionally routing to speech_decoder_->decode() or speech_decoder_->decode_and_trim_reference() when reference codes are present/absent, and ensure release_talker_cached_step_graph cleanup is exception-safe.

Option parsing in assets.cpp and session.cpp is left as-is in this PR to keep the review surface focused prior to the spec-v1 migration.

Motivation

When running Qwen3-TTS in base or voice-prompt mode without ICL reference codes, the session required reference codes to be present even if an x-vector speaker embedding was supplied. Supporting x-vector-only prefill allows Qwen3-TTS to execute without requiring reference codes while keeping reference audio trimming clean and safe.

Gains

Correctness-matched Ryzen AI Max / Strix Halo results with HIP graphs disabled:

Metric Before After Gain
Mean synthesis latency 10.400 s 9.454 s 9.1%
Median synthesis latency 10.499 s 9.354 s 10.9%
Decoder latency 2.369 s 1.304 s 45.0%
Decoder graph build 140 ms 0 ms 100%

All 29 cold, warmup, and measured calls produced the same audio digest and sample metadata.

Mixed-Shape & Extended Longform Validation (Addressing Issue #105)

To ensure this PR does not reintroduce the decoder graph reuse issues described in #105 across varying request lengths, we validated both consecutive mixed-shape requests as well as extended multi-paragraph longform synthesis:

1. Consecutive Mixed-Shape Requests (Short -> Long -> Short)

Request Sequence Text Length / Shape Generated Audio Duration Speech Decoder Compute Time Status
Request 1 (Short) Short prompt (3 words) 2.08 s 765.89 ms PASSED
Request 2 (Long) Long prompt (23 words) 9.76 s 1103.90 ms PASSED
Request 3 (Short) Short prompt (3 words) 1.84 s 755.23 ms PASSED

2. Extended Multi-Paragraph Longform Synthesis Test

  • Prompt: Multi-sentence paragraph (65 words)
  • Generated Audio Duration: 43.2 seconds (24 kHz audio)
  • Total Wall Time: 25.4 seconds (1.70x Real-Time Speed, RTF 0.59)
  • Speech Decoder Compute Time: 2,453.65 ms
  • Status: PASSED (clean audio output, zero memory leaks, exact reference audio trimming, and zero state corruption across 1,080 audio frames).

All consecutive and extended longform requests completed cleanly without state leakage, audio distortion, or decoder graph shape mismatch errors.

How to use it

Use audiocpp_cli normally with Qwen3-TTS:

audiocpp_cli --task tts --model models/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
  --text "Hello from Qwen3 TTS." --out out.wav

AI usage disclosure

AI was used for the following purposes:

  • understand project structure and validate hypothesis about code organization
  • clean-up code after manual definition

AI was NOT used for the following purposes:

  • basic implementation

Add support for x-vector prompt modes and optional reference audio decoding
when running Qwen3-TTS without ICL reference codes.
@mirek190

Copy link
Copy Markdown
Contributor

fedeizzo

AI was NOT used for the following purposes:

basic implementation

There is literally nothing wrong with that.
Even llamacpp is fully allowing use AI to evething now.

That is not early 2026 ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants