[Model Request]: Qwen3-ASR-1.7B and Qwen3-TTS-12Hz-1.7B-Base support

[kylin7226]

This is an excellent project! I hope it can support Qwen's ASR and TTS models, specifically Qwen3-ASR-1.7B and Qwen3-TTS-12Hz-1.7B-Base.

[sxwxs]

I investigated this request and started an implementation on an XDNA2 machine (Ryzen AI 7 PRO 350, Linux/XRT 2.21.75).

The good news is that the Qwen3-ASR text backbones exactly match models FastFlowLM already supports:

• Qwen3-ASR-0.6B text config == Qwen3-0.6B (hidden 1024, 28 layers, 16 Q heads / 8 KV heads)
• Qwen3-ASR-1.7B text config == Qwen3-1.7B (hidden 2048, 28 layers, 16 Q heads / 8 KV heads)

So the proposed MVP is:

1. Reuse the existing Qwen3 NPU decoder.
2. Add the AuT audio encoder (starting with 0.6B).
3. Reuse FastFlowLM's audio reader / spectrogram infrastructure.
4. Initially support offline /v1/audio/transcriptions, without streaming or timestamps.

Work-in-progress branch: https://github.com/sxwxs/FastFlowLM/tree/feat/qwen3-asr

I have completed the first upstreamable pieces:

• a C++ port of Qwen3ASRFeatureExtractor (128-bin Slaney log-mel, minimum-length padding, 2*n_window frame padding, exact CNN output-length calculation);
• a standalone Transformers parity test (max abs error 3.44e-05, mean abs error 2.97e-07);
• exact single-audio prompt construction and ASR output parsing helpers.

The current blocker is the binary qwen3_npu ABI. Qwen3-ASR must replace <|audio_pad|> token embeddings with continuous embeddings from the audio tower, while the public interface currently only exposes:

buffer<bf16> prefill(std::vector<int>& ids, void* payload = nullptr);

Could the maintainers please clarify or expose one of the following?

• the expected payload type for qwen3_npu::prefill, if it already supports embedding replacement;
• a generic prefill_embeddings(...) / mixed token-and-embedding API;
• or a Qwen3-ASR-specific engine/kernel interface.

I can continue implementing the wrapper, model registration, REST integration, tests, and documentation. I am also happy to adapt the current preprocessing work to the interface preferred by the maintainers.

Additional weight-conversion finding

I also downloaded and inspected the official Qwen3-ASR-0.6B-hf checkpoint. Its language-model tensors can be renamed directly to the standard Qwen3 layout, confirming decoder architecture compatibility. However, invoking the public Q4NX::convert_model() on the BF16 checkpoint fails looking for lm_head.weight_scale, so the exposed converter appears to expect an already quantized source checkpoint rather than the official BF16 safetensors.

Could you also share the supported BF16/AWQ-to-Q4NX conversion procedure, or provide an optimized Qwen3-ASR text/audio checkpoint artifact?