Model Information
Why support this model?
CosyVoice is the most popular open-source TTS project on GitHub with ~19k stars. CosyVoice 2/3 uses an LLM backbone (Qwen2.5-0.5B) for speech token generation combined with chunk-aware causal flow matching for waveform synthesis. This LLM-based architecture means vllm-omni's existing optimizations for transformer inference could directly benefit serving performance. CosyVoice 3 supports 9 languages and 18+ Chinese dialects.
Technical Notes
- Two-stage: LLM (Qwen2.5-0.5B backbone) generates speech tokens via FSQ, then causal flow matching generates audio
- Finite Scalar Quantization (FSQ) replaces VQ for full codebook utilization
- Unified streaming/non-streaming synthesis (150ms latency in streaming mode)
- CosyVoice 3 released mid-2025 with improved content consistency and prosody
- Zero-shot cross-lingual voice cloning, emotion/dialect control
- NVIDIA TensorRT-LLM runtime support contributed upstream
Closest supported model in vllm-omni
Qwen3-TTS / MiMo-Audio — CosyVoice shares the same LLM-based TTS architecture pattern: an LLM generates discrete speech tokens, then a vocoder/flow-matching module converts them to audio. Qwen3-TTS uses a similar Qwen backbone + Code2Wav stage, so much of the existing pipeline infrastructure (LLM inference, token-to-waveform conversion) could be reused.
Model Information
Why support this model?
CosyVoice is the most popular open-source TTS project on GitHub with ~19k stars. CosyVoice 2/3 uses an LLM backbone (Qwen2.5-0.5B) for speech token generation combined with chunk-aware causal flow matching for waveform synthesis. This LLM-based architecture means vllm-omni's existing optimizations for transformer inference could directly benefit serving performance. CosyVoice 3 supports 9 languages and 18+ Chinese dialects.
Technical Notes
Closest supported model in vllm-omni
Qwen3-TTS / MiMo-Audio — CosyVoice shares the same LLM-based TTS architecture pattern: an LLM generates discrete speech tokens, then a vocoder/flow-matching module converts them to audio. Qwen3-TTS uses a similar Qwen backbone + Code2Wav stage, so much of the existing pipeline infrastructure (LLM inference, token-to-waveform conversion) could be reused.