Skip to content

V3.0.1 — BlueV3: seven languages, calibrated speed

Latest

Choose a tag to compare

@maxmelichov maxmelichov released this 16 Nov 20:58
· 20 commits to main since this release

Multilingual zero-shot TTS at 44.1 kHz in Hebrew, English, Spanish, Italian, German, Russian and Yiddish — try it at bluetts.com.

git clone https://github.com/maxmelichov/BlueTTS.git && cd BlueTTS && uv sync
uv run hf download notmax123/BlueV3-onnx --include "onnx/*" "voices/*" "frontend/*" --local-dir ./bluev3_onnx
uv pip install ./bluev3_onnx/frontend/polyg2p-*.whl     # G2P for en / de / es / it / ru (Linux x86-64, glibc >= 2.34)
import soundfile as sf
from blue_onnx import BlueTTS

tts = BlueTTS(onnx_dir="bluev3_onnx/onnx", style_json="voices/noa.json")
samples, sr = tts.synthesize("שלום, זהו מודל דיבור בעברית.", lang="he")
sf.write("out.wav", samples, sr)

Highlights

BlueV3 model (v3.0.1, checkpoint 702000). A new audio autoencoder, a text-to-latent model trained from scratch on its latent space with the RobustSpeechFlow contrastive loss (arXiv 2605.22083), and reinforcement learning on top with FlowTTS-GRPO (arXiv 2606.23190): 1,500 iterations with the paper's rewards, then 500 more with a reward that moves the share of pauses toward read speech. Weights: BlueV3-onnx, BlueV3 (PyTorch), blue-codec.

Seven languages. Russian and Yiddish join Hebrew, English, Spanish, Italian and German. All take plain text; mix languages inline with <en>…</en>.

Calibrated speed. speed=1.0 is natural read-speech pace for every voice and language. Each voice JSON carries a per-language speed_scale, and the duration is divided by speed * speed_scale[lang]. Voice files without the field behave as before.

New duration model. Hebrew is paced slower and closer to read speech.

15 voices (LibriTTS-R speakers, CC BY 4.0): female maya, noa, lily, tamar, shira, yael, emma, sara; male adam, daniel, david, jonathan, ethan, michael, ariel.

OpenAI-compatible server. examples/ has a server that answers the OpenAI speech API with BlueTTS voices (#60).

G2P. English, German, Spanish, Italian and Russian use polyg2p (Apache-2.0); Hebrew uses RenikudPlus with the weights shipped in the bundle.

Text handling. Prices, ordinals, dates and times are read inline as words (#54); a sentence longer than a chunk is cut, so no chunk exceeds max_chunk_tokens (#56); text and JSON files are read as UTF-8 (#57).

Training code. The model, the three training stages and the FlowTTS-GRPO trainer and rewards are in training/ (#42, #43, #51, #53).

Scores

Unseen test set and unseen voices, 8 steps, CFG 4, both with the v3.0.1 duration model.

v3.0.0 v3.0.1
Hebrew phone error rate 1.81 % 1.80 % (no change)
Hebrew speaker similarity 0.664 0.670
Hebrew DNSMOS OVRL 3.41 3.43
English word error rate 1.81 % 1.95 % (no change)
English speaker similarity 0.676 0.679
Pause share, he / en / de (read speech 11.4 / 11.0 / 13.4 %) 8.9 / 8.7 / 8.9 % 9.6 / 9.6 / 10.0 %

Behavior changes

  • BlueV3 voices are not interchangeable with earlier ones. A voice JSON is valid only for the checkpoint it was exported from. The previous voices moved to voices/v2/ and still work with the earlier bundles.
  • Earlier bundles still load. BlueTTS2.5-onnx and blue-onnx-v2 run unchanged, e.g. BlueTTS(onnx_dir="onnx_models", style_json="voices/v2/noa.json").
  • The BlueV3 bundle runs on blue_onnx. blue_trt does not read it yet.
  • v3.0.0 (checkpoint 701500) stays on Hugging Face under the git tag v3.0.0.