Repository navigation
Ultrafast Qwen3-TTS 1.7B at sub-50 ms TTFA at 10 requests per second (beats vLLM-Omni, SGLang-Omni) #358
shamuiscoding
started this conversation in
Show and tell
Ultrafast Qwen3-TTS 1.7B at sub-50 ms TTFA at 10 requests per second (beats vLLM-Omni, SGLang-Omni)
#358
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We built a Qwen3-TTS CustomVoice 1.7B implementation that achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) while maintaining real-time playback on 1x NVIDIA H100. Even at 20 RPS, it sustains sub-80 ms p95 TTFA.
The code and container is open sourced here: https://github.com/nari-labs/nari-qwen3-tts.
Read the blog for a detailed breakdown: https://nari-labs.com/blog/qwen3-tts-speed-cost-frontier/
All reactions