English | Español
This is a community fork of fishaudio/fish-speech that adds bitsandbytes NF4 4-bit quantization support, enabling inference on GPUs with as little as 12 GB of VRAM.
Huge thanks to the amazing team at Fish Audio for building and open-sourcing the original Fish Speech model — all credit for the core research and architecture belongs to them.
Important
License Notice
This codebase and its associated model weights are released under FISH AUDIO RESEARCH LICENSE. Please refer to LICENSE for more details. We will take action against any violation of the license.
Warning
Legal Disclaimer
We do not hold any responsibility for any illegal usage of the codebase. Please refer to your local laws about DMCA and other related laws.
This fork is tuned to make Fish Speech S2-Pro feel practical on everyday hardware. The default path is now a polished RTX 3060 / 12 GB deployment with:
- bitsandbytes NF4 4-bit quantization via
--bnb4 - lazy model loading so the API starts fast and loads weights on first inference
- an OpenAI-compatible API on
http://0.0.0.0:8880/v1 - a refreshed Gradio frontend tuned for reference-audio workflows
- a bundled default voice sample so generations stay on a consistent voice even when no reference is supplied
- an automatic 5 minute idle shutdown to give VRAM back when the server is not being used
- a one-command installer and one-command launcher
The bundled default sample uses the repo's included Spanish reference clip and transcript. If you want a different default voice, replace sample.mp3 and sample.lab with your preferred reference pair.
If you want the fastest path from clone to audio, use this:
git clone https://github.com/groxaxo/fish-speech-int4-patch
cd fish-speech-int4-patch
./install_bnb4_3060.sh
./start_bnb4_3060.shThe launcher defaults to:
GPU_INDEX=0PORT=8880--bnb4 --half--lazy-load--idle-timeout-seconds 300--max-seq-len 4096
The direct entrypoints now follow the same defaults too:
python tools/api_server.pystarts on0.0.0.0:8880with--bnb4 --halfpython tools/run_webui.pyloads the WebUI with--bnb4 --half- pass
--no-bnb4or--no-halfif you need to opt out
Note
--bnb4 is designed for the Groxaxo-hosted s2-pro NF4 checkpoint. Do not point it at legacy int4 or int8 checkpoint directories.
- Hugging Face model:
groxaxo/s2-pro - Loader path: keep
--bnb4 --halfenabled when targeting this checkpoint - Export helper:
python tools/llama/export_nf4.py --checkpoint-path checkpoints/s2-pro --output-path /tmp/s2-pro-nf4 - Smoke-tested flow: exported NF4
model.pthreloads correctly throughinit_model(...)
- It turns upstream S2-Pro into a cleaner 12 GB-friendly deployment with sensible
bnb4defaults - It ships the live NF4 publishing path used for
groxaxo/s2-pro - It keeps the original Fish Audio model front and center while making self-hosting dramatically easier
The upstream S2-Pro model is outstanding, but the default out-of-box setup assumes more GPU headroom than many single-card workstations have. This fork closes that gap and turns S2-Pro into a professional, API-first voice stack for 12 GB cards without sacrificing the flagship model experience.
- 12 GB install guide
- Server guide
- Apple Silicon (MLX) local serving — run the 8-bit build natively on a Mac, OpenAI
/v1API, no CUDA - Command line inference
- WebUI inference
- Docker setup
Important
For SGLang server deployment, read the SGLang-Omni README.
Clone the repo, run ./install_bnb4_3060.sh, then run ./start_bnb4_3060.sh. This launches the OpenAI-compatible API on port 8880 with BnB NF4, lazy loading, and a 5 minute idle timeout. The canonical model name is `s2-pro`; compatible OpenAI-style model IDs include `tts-1` and `tts-1-hd`.
Best text-to-speech system among both open source and closed source
Fish Audio S2 is the latest model developed by Fish Audio. Trained on over 10 million hours of audio across approximately 50 languages, S2 combines reinforcement learning alignment with a Dual-Autoregressive architecture to generate speech that sounds natural, realistic, and emotionally rich.
S2 supports fine-grained inline control of prosody and emotion using natural-language tags like [laugh], [whispers], and [super happy], as well as native multi-speaker and multi-turn generation.
Visit the Fish Audio website for live playground. Read the blog post and technical report for more details.
| Model | Size | Availability | Description |
|---|---|---|---|
| S2-Pro | 4B parameters | HuggingFace | Groxaxo-hosted NF4 build of the flagship model |
More details of the model can be found in the technical report.
| Benchmark | Fish Audio S2 |
|---|---|
| Seed-TTS Eval — WER (Chinese) | 0.54% (best overall) |
| Seed-TTS Eval — WER (English) | 0.99% (best overall) |
| Audio Turing Test (with instruction) | 0.515 posterior mean |
| EmergentTTS-Eval — Win Rate | 81.88% (highest overall) |
| Fish Instruction Benchmark — TAR | 93.3% |
| Fish Instruction Benchmark — Quality | 4.51 / 5.0 |
| Multilingual (MiniMax Testset) — Best WER | 11 of 24 languages |
| Multilingual (MiniMax Testset) — Best SIM | 17 of 24 languages |
On Seed-TTS Eval, S2 achieves the lowest WER among all evaluated models including closed-source systems: Qwen3-TTS (0.77/1.24), MiniMax Speech-02 (0.99/1.90), Seed-TTS (1.12/2.25). On the Audio Turing Test, 0.515 surpasses Seed-TTS (0.417) by 24% and MiniMax-Speech (0.387) by 33%. On EmergentTTS-Eval, S2 achieves particularly strong results in paralinguistics (91.61% win rate), questions (84.41%), and syntactic complexity (83.39%).
Righto — the H200 cloud numbers further down are mean as, but most of us aren't running an H200 under the desk. So here's how the 8-bit MLX build actually goes on a bog-standard Apple Silicon Mac: measured on the warmed local_mlx server (:8881), GPU all to itself, no fudging the figures.
| Clip | Audio out | Gen time | RTF (gen ÷ audio) | Semantic tok/s |
|---|---|---|---|---|
| Short | 4.55 s | 17.0 s | 3.74 | 5.8 |
| Medium | 7.66 s | 28.4 s | 3.70 | 5.8 |
| Long | 11.89 s | 45.4 s | 3.82 | 5.6 |
| All up | 24.1 s | 90.7 s | 3.77 | ~5.7 |
Straight up, no spin: she sits at about RTF 3.8 — roughly 0.27× real-time — so it's an offline workhorse, not a live-yarn machine. The handbrake is semantic-token generation chugging along at ~5.7 tok/s; the codec's barely breaking a sweat. Cold start's ~23 s while the weights load, then she's good as gold — no per-request load tax after that.
Every clip got run back through ASR on :5093 to make sure it's actual speech and not a fistful of static — transcribed clean, peaks sitting 0.53–0.93. And don't bother with the MLX 4-bit conversions, eh (the Apple-Silicon affine-quant builds — a different beast entirely to this fork's CUDA bnb4 NF4 build, which is grand): both of them decode to noise on MLX, ASR comes back empty, so on Mac 8-bit's the one. Sweet as.
Full setup — the OpenAI /v1 API, the two servers (:8881 eager, :8882 lazy), env vars and the ASR check — lives in local_mlx/README.md.
Honest single-box measurements, not a leaderboard flex. Your mileage'll shift with the chip and how warm the caches are.
S2 enables localized control over speech generation by embedding natural-language instructions directly at specific word or phrase positions within the text. Rather than relying on a fixed set of predefined tags, S2 accepts free-form textual descriptions — such as [whisper in small voice], [professional broadcast tone], or [pitch up] — allowing open-ended expression control at the word level.
S2 builds on a decoder-only transformer combined with an RVQ-based audio codec (10 codebooks, ~21 Hz frame rate). The Dual-AR architecture splits generation into two stages:
- Slow AR operates along the time axis and predicts the primary semantic codebook.
- Fast AR generates the remaining 9 residual codebooks at each time step, reconstructing fine-grained acoustic detail.
This asymmetric design — 4B parameters along the time axis, 400M parameters along the depth axis — keeps inference efficient while preserving audio fidelity.
S2 uses Group Relative Policy Optimization (GRPO) for post-training alignment. The same models used to filter and annotate training data are directly reused as reward models during RL — eliminating distribution mismatch between pre-training data and post-training objectives. The reward signal combines semantic accuracy, instruction adherence, acoustic preference scoring, and timbre similarity.
Because the Dual-AR architecture is structurally isomorphic to standard autoregressive LLMs, S2 directly inherits all LLM-native serving optimizations from SGLang — including continuous batching, paged KV cache, CUDA graph replay, and RadixAttention-based prefix caching.
On a single NVIDIA H200 GPU:
- Real-Time Factor (RTF): 0.195
- Time-to-first-audio: ~100 ms
- Throughput: 3,000+ acoustic tokens/s while maintaining RTF below 0.5
S2 supports high-quality multilingual text-to-speech without requiring phonemes or language-specific preprocessing. Including:
English, Chinese, Japanese, Korean, Arabics, German, French...
AND MORE!
The list is constantly expanding, check Fish Audio for the latest releases.
Fish Audio S2 allows users to upload reference audio with multi-speaker, the model will deal with every speaker's feature via <|speaker:i|> token. Then you can control the model's performance with the speaker id token, allowing a single generation to include multiple speakers. You no longer need to upload reference audio separately for each speaker.
Thanks to the expansion of the model context, our model can now use previous information to improve the expressiveness of subsequent generated content, thereby increasing the naturalness of the content.
Fish Audio S2 supports accurate voice cloning using a short reference sample (typically 10–30 seconds). The model captures timbre, speaking style, and emotional tendencies, producing realistic and consistent cloned voices without additional fine-tuning. Please refer to SGLang-Omni README to use the SGLang server.
@misc{fish-speech-v1.4,
title={Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesis},
author={Shijia Liao and Yuxuan Wang and Tianyu Li and Yifan Cheng and Ruoyi Zhang and Rongzhi Zhou and Yijin Xing},
year={2024},
eprint={2411.01156},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2411.01156},
}
@misc{liao2026fishaudios2technical,
title={Fish Audio S2 Technical Report},
author={Shijia Liao and Yuxuan Wang and Songting Liu and Yifan Cheng and Ruoyi Zhang and Tianyu Li and Shidong Li and Yisheng Zheng and Xingwei Liu and Qingzheng Wang and Zhizhuo Zhou and Jiahua Liu and Xin Chen and Dawei Han},
year={2026},
eprint={2603.08823},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2603.08823},
}