Qwen3-TTS Pure C Implementation

A lightweight, cross-platform C inference engine for Qwen3-TTS text-to-speech models (0.6B and 1.7B). No Python, no PyTorch, no ONNX runtime — just C, a BLAS library, and raw model weights.

The engine runs the complete TTS pipeline: BPE tokenization, a 28-layer causal transformer (Talker), a multi-pass code predictor, and a convolutional speech decoder. Weights are memory-mapped directly from safetensors files in BF16, so loading is near-instant and memory usage stays low.

📍 Where does a voice live in the model? See docs/speaker-map.md for a readable map of which layers/stages carry timbre vs language/prosody vs emotion (and how the preset voices like ryan work). Essential background for voice cloning and expressivity.

Audio Samples

All samples generated with the 0.6B model — sub-realtime on a 2020 Apple M1 CPU (RTF 0.52 --int4 / 0.69 --int8, no GPU; see Performance):

Language	Speaker	Sample	Text
English	ryan	listen	Hello, this is a test of the text to speech system.
Italian	ryan	listen	Buongiorno a tutti, questa e una dimostrazione del sistema di sintesi vocale.
Italian	vivian	listen	Buongiorno a tutti, questa e una dimostrazione del sistema di sintesi vocale.
Spanish	ryan	listen	Hola, esta es una demostracion del sistema de sintesis de voz.
Portuguese	ryan	listen	Ola, esta e uma demonstracao do sistema de sintese de voz.
French	ryan	listen	Bonjour a tous, ceci est une demonstration du systeme de synthese vocale.
German	ryan	listen	Guten Tag, dies ist eine Demonstration des Sprachsynthesesystems.
Japanese	Ono_Anna	listen	こんにちは、私の名前はアンナです。今日はとても良い天気ですね。東京の桜がとても綺麗です。
Japanese	Ono_Anna	listen	頑張れ、アンドレア！あなたならできるよ。毎日少しずつ前に進もう。夢を諦めないで。応援してるよ！

Clone and play locally: afplay samples/english_ryan.wav (macOS) or aplay samples/english_ryan.wav (Linux)

Quick Start

# Clone and build
git clone https://github.com/gabriele-mastrapasqua/qwen3-tts.git
cd qwen3-tts
make blas

# Download a model (interactive: small, large, voice-design, base-small, base-large)
./download_model.sh

# Synthesize speech
./qwen_tts -d qwen3-tts-0.6b --text "Hello, how are you today?" -o hello.wav

Dependencies: Only a C compiler and BLAS (Accelerate on macOS, OpenBLAS on Linux). See docs/building.md for Linux, Windows/WSL2, and other build targets.

Features

• Pure C, minimal dependencies — Only requires a C compiler and BLAS. No Python runtime needed.
• Runs on macOS, Linux and Windows/WSL2 (ARM/x86) — the hot matvec/attention kernels have NEON+SDOT (ARM), AVX2 and AVX-512/VNNI/BF16 (x86) twins with a scalar fallback + runtime ISA guard, and decode threading runs on a cross-OS pool (GCD on macOS, pthread elsewhere). Validated on Apple M1, Ryzen 7 6800H, and EPYC 9555P (Zen5). Single-stream RTF is memory/cache-bound, so the chip's cache matters most (see Performance); measure yours with bash tests/x86_bench.sh.
• Optional GPU backends (opt-in) — Apple Metal (make metal) and NVIDIA CUDA (make cuda) run the whole fused pipeline resident on the GPU (0.28 RTF for 0.6B on an M4; ~0.44 for 1.7B on a mainstream NVIDIA GPU), plus server request-batching for throughput. CPU stays the default. → Performance § GPU backends · docs/hardware-testing.md (Metal) · docs/cuda-performance.md (CUDA).
• Both model sizes — Automatically detects 0.6B or 1.7B from weight files.
• 9 preset voices — ryan, vivian, serena, aiden, eric, dylan, uncle_fu, ono_anna, sohee.
• 10 languages — English, Chinese, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian.
• Memory-mapped weights — BF16 safetensors mmap'd directly. 0.6B ~3 GB, 1.7B ~8 GB.
• Voice cloning — Clone any voice from a short WAV clip. Ship it as a compact ~25 MB graft .qvoice (tests/qvoice_to_graft.py → --icl-only): keeps the CustomVoice weights so emotion levers (--instruct, --expr, --ml-steer) all work, with full prosody (sighs/pauses). An 8 KB --xvector-only .bin is the ultra-lean alternative (identity only). See docs/icl-graft-portability.md.
• Voice management — List, inspect, delete .qvoice profiles (--list-voices, --delete-voice). No model required.
• Style control — --instruct for emotion/style on 1.7B: angry, whisper, cheerful, and more.
• Emotion in one flag (🧪 beta; paralinguistics [laugh]/[sigh] 🧪 alpha) — --emotion <sad\|joy\|anger\|fear\|disgust\|surprise> (1.7B) auto-applies the ear-validated recipe (per-language fine-tune .expr + steering vector + a default English instruct + temperature), on presets and cloned voices, in every Qwen language. Plus 7 blended "dyads" (contempt, awe, nostalgia, disapproval, remorse, outrage, despair) and inline [emotion] switching — many emotions from one prompt in a single generation. A vivid English --instruct and -T override. Pitch-preserving --rate/--volume and a --roughness grit knob are still available. See docs/emotion-THE-recipe.md.
• The small 0.6B is expressive too — and stays sub-realtime 🆕 — for a long time --emotion did nothing on the 0.6B: unlike the 1.7B it has no steerable emotion subspace. It does, however, clone voices very well — so on the small model the emotion rides on the voice. All 9 presets ship ready (240 KB of 4 KB voice assets) and any cloned voice emotes with zero setup via six shipped emotion directions — so the small model has the whole expressive stack out of the box: 6 emotions + inline [tag] paralinguistics + voice cloning, together, at RTF ≈ 0.8 under --int8 on an M1. Try it with make emo-06b-demo. See docs/emotion-06b-recipe.md.
• Inline markup for audiobooks — write one text with ElevenLabs/Bark-style tags and get a multi-emotion take in one pass: --text "I won! [joy] ...amazing! [pause:500ms] [sad] But it's over. [sigh]". Mid-text emotion switches, [pause:400ms]/[break:1s] pauses, and [sigh]/[huff] paralinguistic fillers — auto-detected in --text (no flag) or explicit via --compose. Spans are model-generated and concatenated seamlessly. See docs/markup.md.
• VoiceDesign — Create new voices from text descriptions.
• HTTP server — /v1/tts, /v1/tts/stream, OpenAI-compatible /v1/audio/speech; JSON body takes emotion/instruct/volume/rate (same recipe as the CLI). Inline [mood] markup works over the API too — one request can switch emotion sentence-by-sentence ("text":"[joy] Great news! [sad] But I must go."), auto-detected and streamed span-by-span. See docs/server.md.
• Streaming — Real-time audio via --stream (WAV) or --stdout (raw PCM).
• INT8 / INT4 quantization — --int8 / --int4 quantize Talker + Code Predictor (native SDOT on ARM, AVX-512/VNNI on x86), near-bf16 quality, and work with presets and custom .qvoice voices. On cache-rich Apple Silicon both go sub-realtime (0.6B best 0.52 int4 / 0.69 int8; 1.7B best ~1.53 quant-mixed); on memory-starved x86, int8+VNNI wins the wall clock. See Performance.
• Configurable sampling — Temperature, top-k, top-p, and repetition penalty.
• 24 kHz WAV output — 16-bit PCM, mono.

Usage

./qwen_tts [options]

Required:
  -d, --model-dir <path>     Model directory
  --text <string>            Text to synthesize

Optional:
  -o, --output <path>        Output WAV file (default: output.wav)
  -s, --speaker <name>       Speaker voice (default: ryan)
  -l, --language <lang>      Target language (default: English)
  -I, --instruct <text>      Style/emotion instruction (1.7B model only)
  --temperature <f>          Sampling temperature (default: 0.5)
  --top-k <n>                Top-k sampling (default: 50)
  --top-p <f>                Top-p nucleus sampling (default: 1.0)
  --rep-penalty <f>          Repetition penalty (default: 1.05)
  --max-tokens <n>           Max audio tokens (default: 8192)
  --max-duration <secs>      Max audio duration in seconds
  --seed <n>                 Random seed for reproducible output
  --ref-audio <path>         Reference audio for voice cloning (Base model)
  --save-voice <path>        Save voice profile (.qvoice = full, .bin = x-vector only)
  --load-voice <path>        Load voice profile (.qvoice or .bin)
  --xvector-only             Clone via speaker x-vector only — clean, 8KB .bin (recommended for expr/emotion)
  --icl-only                 Graft mode: keep CV weights, use the .qvoice ICL prefix (max timbre mimicry)
  --target-cv <dir>          CV model dir for delta encoding (bit-identical cross-model)
  --list-voices <dir>        List .qvoice files in directory (no model needed)
  --delete-voice <path>      Delete a .qvoice file
  --voice-name <name>        Name for the voice (stored in .qvoice metadata)
  --voice-design             VoiceDesign mode (create voice from --instruct)
  --stream                   Stream audio (decode chunks during generation)
  --stdout                   Output raw s16le PCM to stdout (implies --stream)
  --int8                     INT8 quantized (0.6B & 1.7B; faster, ~same quality) — recommended; uses VNNI on AVX-512 x86, SDOT on ARM
  --int4                     Q4_0 quantized (experimental; slower than --int8 on CPU)
  -j, --threads <n>          Worker threads (default: 4)
  --silent                   Suppress status output
  --debug                    Verbose diagnostics
  --serve <port>             Start HTTP server

Examples

# Basic English
./qwen_tts -d qwen3-tts-0.6b --text "The quick brown fox jumps over the lazy dog." -o fox.wav

# Italian with a specific voice
./qwen_tts -d qwen3-tts-0.6b -s ryan -l Italian \
    --text "Ciao, questa e una prova del sistema di sintesi vocale." -o test_it.wav

# Style/emotion control (1.7B only)
./qwen_tts -d qwen3-tts-1.7b -s ryan -l English \
    --text "I cannot believe you did that to me." \
    --instruct "Speak in a very angry and aggressive tone" -o angry.wav

# Reproducible output with seed
./qwen_tts -d qwen3-tts-0.6b --text "Hello world" --seed 42 -o hello.wav

Voice Cloning

Clone any voice from a reference audio clip. Requires a Base model.

# Clone a voice
./qwen_tts -d qwen3-tts-0.6b-base --ref-audio reference.wav \
    --text "Hello, this is my cloned voice." -o cloned.wav

Full guide: reference audio tips, model comparison, samples → docs/voice-cloning.md

Ready-to-use reference voices (CC0 / Public Domain). Four lite ~25 MB graft .qvoice clones of LibriVox public-domain readers (Italian, Spanish, English, French) so the demos/tests run out of the box and you have voices to listen to and reuse:

bash download_voices.sh    # fetch galatea(IT)/quijote(ES)/ohenry(EN)/hugo(FR) into voices/ (sha256-verified)
./qwen_tts -d qwen3-tts-1.7b --load-voice voices/galatea_graft.qvoice --icl-only -l Italian \
    --text "Buongiorno, questa è la mia voce clonata." -o out.wav

🎭 Want these clones to emote (--emotion)? See the Emotion & expressivity section below — it needs bash download_assets.sh first. Hosted on Hugging Face → gabrione/qwen3-tts-voices (CC0, LibriVox attribution).

Custom Voices — small, portable, and emotable .qvoice

Clone a voice once, save it as a portable .qvoice, reuse it forever on the CustomVoice model — with --instruct, --emotion, streaming, and the HTTP server.

The default .qvoice is now a ~25 MB "graft" — it keeps the CustomVoice weights, so it stays small, carries full prosody, and the emotion / instruct levers still work on your clone (no more multi-GB weight-delta files):

# Create — default = ~25 MB graft (one-time; needs the Base model)
./qwen_tts -d qwen3-tts-0.6b-base --ref-audio mario.wav -l Italian \
    --voice-name "Mario" --save-voice mario.qvoice

# Use it on CustomVoice — --icl-only keeps the CV weights (→ instruct/emotion work)
./qwen_tts -d qwen3-tts-0.6b --load-voice mario.qvoice --icl-only \
    --text "Ciao, come stai?" -o output.wav

# ...with an emotion, on your OWN cloned voice:
./qwen_tts -d qwen3-tts-0.6b --load-voice mario.qvoice --icl-only \
    --emotion joy -l Italian --text "Ce l'abbiamo fatta!" -o joy.wav

# Server / manage
./qwen_tts -d qwen3-tts-0.6b --load-voice mario.qvoice --icl-only --serve 8080
./qwen_tts --list-voices ./my_voices/

Other formats → docs/custom-voices.md: 8 KB .bin x-vector (--xvector-only, tiniest & cleanest) · heavy WDELTA (--target-cv, ~0.8–3 GB, bit-identical — only if you need exact fidelity).

Voice clone samples — cloned voices on 0.6B CustomVoice (25 MB grafts):

Language	Voice	Source	Output	Text
Italian	Pirandello Reader	LibriVox Public Domain	input → clone	Buongiorno a tutti, questa e una dimostrazione della clonazione vocale.
English	Sarac (F)	LibriTTS-R CC-BY	listen	Good morning everyone, this is a demonstration of voice cloning using a custom voice profile.
English	Peter (M)	LibriTTS-R CC-BY	listen	I love reading books aloud, there is something magical about bringing stories to life with your voice.
French	Baudelaire Reader	LibriVox Public Domain	listen	Bonjour a tous, ceci est une demonstration du clonage vocal avec un profil de voix personnalise.
Spanish	Lu	LibriVox Public Domain	listen	Buenos dias a todos, esta es una demostracion de la clonacion de voz con un perfil de voz personalizado.

Full guide: delta vs standard, format internals, troubleshooting → docs/custom-voices.md

Emotion & expressivity (1.7B) · 🧪 Beta

🧪 Beta quality. Ear-validated after ~a month of tuning, but results vary by language/voice and can still be imprecise — expect rough edges (please gauge that before filing issues 🙏). Improvements will come, just not today.

⚙️ Setup (once): run bash download_assets.sh to fetch the emotion fine-tunes (.expr, ~200 MB for Italian) from Hugging Face → gabrione/qwen3-tts-italian-expr. The steering vectors already ship in this repo. --emotion then works on the 9 presets AND on your own cloned voices. No clone yet? Grab ready-made CC0 graft voices with bash download_voices.sh → gabrione/qwen3-tts-voices and emote them straight away.

🎧 Hear it in one command (after the two downloads): make emotion-demo renders a batch of emotion clips — every language × emotion, on presets and the galatea clone — so you can judge the current quality by ear. make emotion-para-demo adds the alpha [laugh]/[sigh].

Emotion is one flag. Pick an emotion with --emotion and the engine auto-composes the validated COMBINE stack for you — the per-language fine-tune (.expr) plus the steering vector for that voice and emotion, at the ear-validated weights. No file paths, no layer ranges. A vivid English or Chinese --instruct on top is optional but recommended — it drives the strongest, most natural result.

# emotion on a CLONED voice (galatea = a ready-made CC0 graft) — same one flag
./qwen_tts -d qwen3-tts-1.7b --load-voice voices/galatea_graft.qvoice --icl-only \
    -l Italian --emotion sad --text "Ho perso tutto, e adesso non so più cosa fare." -o sad.wav

# emotion in ONE flag — works on presets AND cloned voices
./qwen_tts -d qwen3-tts-1.7b -s ryan -l Italian -T 1.1 --emotion sad \
    --instruct "Speak softly, with quiet sadness." \
    --text "Allora, lascia che ti spieghi come stanno le cose." -o sad.wav

🔊 Hear it — committed examples in samples/emotion_examples/ (play after clone: afplay samples/emotion_examples/<file>.wav, or click to download):

Language	Voice	Emotion	Text	Listen
Italian	ryan (preset)	😢 sad	Ho perso tutto quello che avevo, e adesso non so più cosa fare.	▶ play
Italian	ryan (preset)	😄 joy	Non ci posso credere, è la notizia più bella della mia vita!	▶ play
Italian	ryan (preset)	🤢 disgust	Ma che roba è questa? Fa davvero schifo, non riesco neanche a guardarla.	▶ play
Italian	ryan (preset)	😲 surprise	Cosa?! Non me lo aspettavo per niente, è incredibile!	▶ play
Italian	galatea (cloned voice)	😢 sad	Ho perso tutto quello che avevo, e adesso non so più cosa fare.	▶ play
Italian	galatea (cloned voice)	😠 anger	Come ti permetti di parlarmi così? Questo non lo accetto!	▶ play
English	ryan (preset)	😢 sad	I've lost everything I had, and now I don't know what to do anymore.	▶ play
English	ryan (preset)	😠 anger	How dare you talk to me like that? I will not accept this!	▶ play
English	ryan (preset)	😨 fear	There's someone in the house, I heard footsteps... I'm so scared, I don't know what to do.	▶ play
German	vivian	😠 anger	Also, lass mich dir in Ruhe erklären, wie die Dinge wirklich stehen.	▶ play
French	vivian	😢 sad	Bon, laisse-moi t'expliquer calmement comment les choses se passent.	▶ play
French	vivian	😲 surprise	Quoi ? Je ne m'y attendais pas du tout, c'est incroyable.	▶ play
Spanish	vivian	😄 joy	Bueno, déjame explicarte con calma cómo están realmente las cosas.	▶ play
Spanish	vivian	🤢 disgust	¿Pero qué es esto? Es asqueroso, ni siquiera puedo mirarlo.	▶ play
Portuguese	ryan	😄 joy	Não acredito, é a melhor notícia da minha vida!	▶ play
Chinese	vivian	😄 joy	我简直不敢相信，这是我一生中最好的消息，我太高兴了！	▶ play
Russian	ryan	😠 anger	Как ты смеешь так со мной разговаривать? Это неприемлемо!	▶ play
Japanese	ono_anna	😢 sad	私が持っていたものを全て失って、もうどうすればいいのか分からない。	▶ play
Japanese	ono_anna	😨 fear	家に誰かいる、足音が聞こえた……怖くてどうすればいいのか分からない。	▶ play
Korean	sohee	😠 anger	네가 어떻게 나한테 그렇게 말할 수 있어? 이건 절대 받아들일 수 없어!	▶ play

• Emotions: 6 primaries — sad · joy · anger · fear · disgust · surprise (synonyms like happy/angry work too) — plus 7 blended "dyads" (below).
• The recipe: a preset voice → pure STEER (the steering vector @ w12, clean in every language); a cloned voice → COMBINE (the language .expr + steer). Use the native preset per language (JA ono_anna, KO sohee, ZH vivian, EN/Romance ryan); the engine prints a hint. Full recipe → docs/emotion-THE-recipe.md.
• Works in every Qwen3-TTS language (EN, IT, DE, ZH, RU, KO, JA, ES, FR, PT) — just set -l <Language>.

Blended emotions (dyads) · new

Emotion steering directions add: summing two primary vectors yields a coherent new emotion. Seven ear-validated Plutchik dyads ship as first-class --emotion values — no new capture, no fine-tune:

Dyad	= blend of	Reads as	Listen (English, ryan)
contempt	anger + disgust	sneering disdain	▶ play
awe	fear + surprise	hushed wonder	▶ play
nostalgia	joy + sad	bittersweet fondness	▶ play
disapproval	surprise + sad	let-down reproach	▶ play
remorse	sad + disgust	guilty regret	▶ play
outrage	anger + surprise	indignant shock	▶ play
despair	fear + sad	hopeless dread	▶ play

./qwen_tts -d qwen3-tts-1.7b -s ryan -l English --emotion contempt \
    --text "Oh, sure, that's a truly brilliant idea." -o contempt.wav

Inline emotion switching — many emotions from ONE prompt · new

Write [emotion] tags inside --text — any primary or dyad — and the engine switches emotion sentence by sentence in a single generation, clean at the seams, one output file. [neutral] resets; composes with a global --emotion and with [laugh]/[sigh] tags.

./qwen_tts -d qwen3-tts-1.7b -s ryan -l English -T 1.1 --text \
  "[contempt] Oh, sure, that's a brilliant idea. [nostalgia] We used to spend every summer by the sea. [despair] And now there's nothing left." \
  -o switch.wav

🔊 Switch-inside-one-prompt audio examples (EN + IT) → docs/emotion-THE-recipe.md.

• Paralinguistics → inline [tags], also automatic · 🧪 Alpha. Write [laugh], [sigh], [yawn], [wow], [giggle] or [scoff] in --text and the engine performs the event (it picks the onomatopoeia anchor + the right seed per voice for you) — no flags. [wow]/[yawn]/[scoff] compose well with the matching --emotion; [giggle] is best standalone (stacking it with --emotion joy over-drives the laugh). Alpha quality: hit-or-miss across voices/languages (laughs land best); expect misses for now:

  ./qwen_tts -d qwen3-tts-1.7b -s ryan -l Italian -T 1.1 \
      --text "Che giornata... [sigh] non ce la faccio più. [laugh]" -o para.wav

Emotion + paralinguistics, and tuning the --instruct (experimental 🧪)

You can put a paralinguistic [tag] inside an emotional sentence (e.g. --emotion joy + [laugh]) and get both at once — the engine switches to the COMBINE stack so the .expr language-correction keeps the event on-accent. Still a bit unstable across some languages/voices; clearest on [laugh]/[sigh] with ryan/vivian. Reproduce with make emotion-para-demo.

--instruct is an optional vivid English (or Chinese) line on top of the recipe (matters for cloned voices; preset pure-emotion needs none): a stronger, more vivid instruct pushes emotion harder, and plain English "speak faster/slower" shifts pacing (~±15 %). Don't use a slot template (Tempo:+40% comes out slower — Qwen doesn't parse it); plain prose wins.

🔊 Emotion+[tag] audio examples, the per-emotion strong/very-strong instruct library, and the manual override flags (--expr / --ml-steer — you normally never touch them) → docs/emotion-THE-recipe.md · docs/emotion-instruct-control.md.

Assets: bash download_assets.sh fetches the .expr packs; --verify re-checks integrity. Full set ≈ 1.4 GB; Italian-only emotion needs just italian_csp_topk6.expr (203 MB).

Deeper docs: docs/expressivity-assets.md (asset catalog + recipes) · docs/csp-ft-emotion.md (how the .expr packs were trained, cross-language transfer) · docs/expressivity-lora.md (which layers, the .expr format, train your own) · docs/paralinguistics-tags.md (laugh/sigh tags + vectors).

Emotion & expressivity on the small 0.6B · 🆕

The 0.6B used to be the fast neutral voice: --emotion was a no-op there, and steering or fine-tuning it never worked. The reason is structural — at half the width the small model has no emotion subspace disjoint from language and timbre, so there is nothing to steer.

But it clones voices very well. So on the small model the emotion is not an inference-time lever, it is a property of the voice: you clone from emotional audio and get an emotional voice.

A cloned voice gets all six emotions for free. Clone a voice the usual way and --emotion just works on it — no asset to build, no 1.7B, no Base model:

./qwen_tts -d qwen3-tts-0.6b --load-voice myvoice.qvoice --icl-only --int8 \
    -l Italian --emotion anger --text "[sigh] ..." -o out.wav
# → Emotion 'ang' on 0.6B: generic direction @ 0.25 (no per-voice asset needed)

This works because the emotional offset in ECAPA speaker space turns out to be largely speaker-independent — between two different speakers' deltas the cosine sits at 0.48-0.68 (random 1024-dim vectors would be ~0). Six averaged unit directions ship in this repo (4 KB each, 24 KB total) and the engine adds one to whatever x-vector you loaded, preserving its norm. It costs nothing at runtime. Dose it with --emotion-strength (default 0.25; 0.35 pushes harder).

All 9 presets work out of the box too — every one ships with its six dedicated assets (60 files, 240 KB total), each built from a donor in the language that voice speaks natively. So --emotion on a preset needs no setup either:

# emotion on the SMALL model — nothing to install, the 1.7B is not in the path
./qwen_tts -d qwen3-tts-0.6b -s ryan -l Italian --int8 --emotion anger \
    --text "Non è possibile che succeda sempre la stessa cosa." -o anger.wav

# everything composes — emotion + inline paralinguistics, one generation
./qwen_tts -d qwen3-tts-0.6b -s ryan -l Italian --int8 --emotion sad \
    --text "[sigh] Non è possibile che succeda sempre la stessa cosa." -o sad_sigh.wav

Optional upgrade — a dedicated asset per (voice × emotion). The generic direction is instant and needs nothing; a dedicated asset, rendered from ~25 s of that voice actually performing the emotion, is stronger and more faithful. One command builds all six and caches them — you never map anything by hand:

make emovoice VOICE=vivian                                      # another preset
make emovoice VOICE=galatea LOAD=voices/galatea_graft.qvoice    # your own cloned voice
make emovoice VOICE=ryan TTS_LANG=English                       # another language

This route needs the 1.7B and the 0.6B Base model present (the first renders the emotional donor audio, the second extracts the 4 KB voice), one-time and offline. It is an upgrade, not a prerequisite — without it a cloned voice still emotes via the generic direction. When a dedicated asset exists the engine prefers it automatically.

Resolution order for --emotion on the 0.6B: dedicated asset → generic direction on the loaded x-vector → an explicit error naming the missing piece. It never silently falls back to neutral.

🎧 make emo-06b-demo renders the whole stack (6 emotions + 5 [tag]s + both together + a clone) and prints the RTF of each.

Speed — M1, -j4, quiet machine. The full expressive stack costs ~0.09 RTF over the bare model:

on the 0.6B	bf16	int8	int4
emotional voice (4 KB)	1.16	0.73	0.56
emotional voice (16.8 MB graft)	1.14	0.72	—
emotional voice + [tag]	1.18	0.78	—
bare 0.6B (reference)	1.17	0.69	—

Notes. Works on presets and cloned voices. Prefer the 16.8 MB graft for anger — with a bare 4 KB x-vector the high-arousal delivery compresses and can swallow a short word; the graft's prosody scaffolding holds it together, at the same speed. The [tag] seeds differ from the 1.7B's (the engine picks the right table per model automatically).

→ Full recipe, limits and what was tried and rejected: docs/emotion-06b-recipe.md

HTTP Server

# Start server
./qwen_tts -d qwen3-tts-0.6b --serve 8080

# Serve many users at once — step their requests together (vLLM-style request batching)
./qwen_tts -d qwen3-tts-0.6b --serve 8080 --batch-size 4

# Generate speech
curl -s http://localhost:8080/v1/tts \
  -d '{"text":"Hello, how are you?"}' -o output.wav

# With emotion (same recipe as the CLI --emotion; joy/sad/angry/calm/…)
curl -s http://localhost:8080/v1/tts \
  -d '{"text":"What a wonderful day!","speaker":"ryan","language":"English","emotion":"joy"}' -o joy.wav

# Stream with real-time playback (emotion works on the streaming path too)
curl -sN http://localhost:8080/v1/tts/stream \
  -d '{"text":"Hello, how are you?","emotion":"sad"}' | \
  play -t raw -r 24000 -e signed -b 16 -c 1 -

# OpenAI-compatible endpoint
curl -s http://localhost:8080/v1/audio/speech \
  -d '{"input":"Hello world","voice":"ryan"}' -o output.wav

Full guide: all endpoints, request body, performance → docs/server.md

Streaming

# Stream to WAV file
./qwen_tts -d qwen3-tts-0.6b --text "Hello world" --stream -o hello.wav

# Pipe raw PCM to audio player
./qwen_tts -d qwen3-tts-0.6b --text "Hello world" --stdout | \
    play -t raw -r 24000 -e signed -b 16 -c 1 -

How It Works

Text --> BPE Tokenizer --> Talker (LLM) --> Code Predictor --> Speech Decoder --> 24 kHz WAV

Component	What it does
Talker	28-layer Qwen3 transformer with GQA, RoPE, SwiGLU. Generates one audio frame token per step.
Code Predictor	5-layer transformer running 15 sequential passes per frame. Predicts the remaining 15 codebook entries.
Speech Decoder	Causal ConvNet with 16-codebook RVQ dequantization and 480x upsampling. Converts codes to waveform.

	0.6B	1.7B
Talker hidden dim	1024	2048
Heads (Q/KV)	16/8	16/8
Layers	28	28
Code Predictor	1024 hidden, 5 layers	1024 hidden, 5 layers (+2048→1024 projection)
Memory	~3 GB	~8 GB

Performance

⚡ Faster than real-time on Apple Silicon — both --int8 and --int4 go sub-1.0 RTF

On a 2020 M1 (CPU, no GPU) the 0.6B model runs ~2× faster than real time — CLI, streaming and server — with no perceptible quality loss by ear (cloned .qvoice voices included).

Apple M1 (8-core, 16 GB, 4 threads), 0.6B — best RTF per precision. int4 is the fastest lever on Apple's cache-rich SLC; int8 is the safest quality/speed pick — both beat real time:

Precision	Best 0.6B RTF	vs real-time
bf16	1.3–1.8	slower
--int8	0.69 ⚡	~1.4× faster
--int4	0.52 ⚡	~1.9× faster

Every delivery mode stays sub-realtime — bf16 vs --int8 across CLI / streaming / server / cloned voice:

Mode	bf16 RTF	--int8 RTF	First audio (TTFA)
CLI (short, ~4 s)	1.5–1.8	0.90 ⚡	0.96 s
CLI (long, ~14 s)	~1.3	0.80 ⚡	—
Streaming (--stream, short)	1.5–1.8	0.89 ⚡	0.46 s
Streaming (long)	~1.3	0.81 ⚡	0.50 s
HTTP server (--serve, warm)	~1.3	0.88 ⚡	—
Custom voice .qvoice (streamed)	1.34	0.93 ⚡	0.47 s

Expressive and sub-realtime — the 0.6B's full expressive stack (emotional voice + inline [tag] paralinguistics + cloning, see Emotion on the small 0.6B) costs only ~0.09 RTF over the bare model:

0.6B, everything on	bf16	--int8	--int4
emotional voice (4 KB asset)	1.16	0.73 ⚡	0.56 ⚡
emotional voice (16.8 MB graft)	1.14	0.72 ⚡	—
emotional voice + [tag]	1.18	0.78 ⚡	—
bare 0.6B (reference)	1.17	0.69	—

RTF = processing_time / audio_duration; < 1.0 = faster than real-time. Quantization reads fewer weight bytes per frame (native SDOT on ARM, AVX-512/VNNI on x86): 0.6B ~1.5 (bf16) → 0.69 (int8) → 0.52 (int4); 1.7B ~2.0 (bf16) → 1.79 (int8) → ~1.53 (quant-mixed: int4 Talker + int8 CP, the fastest 1.7B config on M1) — no perceptible quality loss, works with .qvoice voices (details).

📊 Benchmark your CPU

Want to know how this runs on your machine (Apple Silicon, AMD/Intel x86, ARM server)? The repo ships a one-command per-box report — no setup beyond the model:

make bench              # quick RTF: short+long, normal+stream (both models)
make bench-full         # + server, instruct, INT8, .qvoice

# Per-CPU report (copy onto any rented ARM/x86 box):
./qwen_tts --caps       # what SIMD your CPU actually has (NEON/SDOT/bf16/i8mm/SVE • AVX2/AVX-512/VNNI/AMX)
./qwen_tts --self-test  # are the kernels numerically correct on this ISA?
make bench-matrix       # caps + self-test + RTF matrix (single vs batch × bf16/int8/int4)
make bench-matrix-full  # + streaming + server + request-batching throughput
make bench-server       # concurrent-request throughput alone (N users vs single-stream, per precision)

The full cross-hardware workflow (which boxes have which SIMD, where to rent, what to measure) lives in docs/hardware-testing.md.

Cross-device CPU (single-stream 0.6B, this repo's best config — reproduce with bash tests/x86_bench.sh):

Device	SIMD + threads	RAM	Best 0.6B RTF	Config
Apple M1 8-core	NEON + SDOT int8/int4, GCD 4-thread	16 GB	0.52 int4 / 0.69 int8	--int4 -j4
Neoverse-N1 (Ampere Altra Max, Scaleway)	NEON + SDOT, pthread 4-thread	16 GB / 4 vCPU	1.28 stream int4 + conv-int8 (1.49 default)	--int4 --stream
Graviton3 (Neoverse-V1, AWS c7g.2xlarge)	NEON + SDOT + i8mm SMMLA + BFMMLA, pthread 4-thread	16 GB / 8 vCPU	0.66 (1.7B int8: 0.95, sub-RT!)	--int8 -j4
Apple M4 (Mac mini, Scaleway)	NEON + SDOT + i8mm + bf16 + SME	16 GB / 10-core	0.32 (1.7B qm: 0.57!)	--int4 -j4
Ryzen 7 6800H (Zen3+, 16 MB L3, bare metal)	AVX2 + FMA, pthread 4-thread	32 GB	2.02	--int4 -j4
EPYC 9555P (Zen5, AVX-512+VNNI+BF16, Scaleway VM)	AVX-512 attention + VNNI + VDPBF16PS, pthread 4-thread	16 GB / 4 vCPU	0.95 (int4 = int8; -j1: int4 1.05 beats int8 1.21)	--int4 -j4, SIMD=avx512bf16

Numbers refreshed 2026-08-04 after the AVX-512 parity round (16-wide attention/rms/conversions, native-bf16 VDPBF16PS matvec, q4-VNNI v3-default + fused-QKV VNNI twin — bf16 mode −21% single-thread on Zen5, and int4 now beats int8 single-thread on x86 0.6B for the first time). Single-stream RTF is memory/cache-bound (the Code Predictor re-reads its weights 16×/frame): SIMD width and thread count matter less than fewer weight bytes (--int8/--int4) and a cache that fits the working set (Apple's SLC, an X3D chip's V-cache). On cache-rich Apple Silicon int4 is the fastest lever; on x86 it now depends on the model: 0.6B → --int4, 1.7B → --int8 (still the 1.7B wall-clock king; pure --int8 beats --quant-mixed, which is the Apple-silicon config). Many-core servers are best for throughput (concurrent requests), not single-stream latency. Check yours: ./qwen_tts --caps (on x86, build with make blas SIMD=avx512bf16 on Zen4/5 / Cooper Lake+, or SIMD=avx512vnni if the CPU lacks avx512bf16 — the default build is portable AVX2).

Concurrent serving — request batching (--serve --batch-size N). For N users at once, the server can step their requests together through the model (vLLM-style): weights are read from memory once and reused across all in-flight requests, instead of re-read per user. A continuous scheduler keeps the batch full (a finished request's slot is refilled immediately) and streaming composes — each user still gets their own progressive audio stream. This trades a little per-request latency for much higher total throughput on bandwidth-bound boxes. Measure it on your CPU with make bench-server; details in docs/server-batching.md.

vs other implementations:

Hardware	0.6B RTF	Notes
This project (C, Apple M1 CPU, --int4)	0.52	Pure C, no GPU — 2× faster than real-time (post-PR#17 decoder work)
This project (C, Apple M1 CPU, bf16)	1.26–1.39	Pure C, no GPU
Python + PyTorch (Ryzen 9 7950X CPU)	4.5–5.8	Official Python, CPU-only
NVIDIA RTX 3090	0.52–0.68	Python + PyTorch + FlashAttention 2

5–7x faster than Python on CPU, and faster than real-time with --int8 — on a 2020 laptop with no GPU.

Per-component breakdown, full GPU table, optimization history → docs/performance.md x86 AVX2/AVX-512/VNNI findings + how to benchmark your CPU → docs/x86-optimization.md

🖥️ GPU backends — Apple Metal & NVIDIA CUDA (opt-in)

Optional --backend metal|cuda runs the whole fused pipeline resident on the GPU (weights + KV + activations on device, one command buffer / step). The CPU path stays the default — GPU is purely additive. Full numbers: Metal / Apple Silicon · CUDA / NVIDIA.

Apple Metal — make metal CC=clang, then QWEN_METAL_FUSED_TALKER=1 ./qwen_tts --backend metal. Single-stream latency (one request — CLI, or a warm --serve server; the two match):

Device	0.6B RTF	1.7B RTF	Streaming TTFA (single client)
Apple M1 8-core (dev box)	~0.60 (int4)	—	469 ms (0.6B)
Apple M2 Pro 16-core GPU	0.36–0.39	0.48–0.53	314 ms / 517 ms
Apple M4 10-core GPU	0.28 (int4)	0.41 (int4)	—

RTF = processing_time ÷ audio_duration (< 1.0 = faster than real time); TTFA = time to first audio for a single --stream client, warm server (the first request after startup pays a one-time weight→GPU-buffer upload — e.g. ~3.6 s cold vs 469 ms warm on M1-0.6B). Metal beats the native M2 CPU path ~1.5–2×; int8 is the sweet spot on Apple Silicon (bandwidth-rich → int4's nibble-unpack doesn't pay). Resident decode is bit-identical to the CPU path. (Multi-user concurrency → the batching table below.)

NVIDIA CUDA — make cuda (resident fused + cuBLAS pointwise convs + CUDA graphs). One multi-arch binary (Ampere/Ada/Blackwell). The CUDA toolkit is auto-detected (nvcc on PATH, else /usr/local/cuda, else /opt/cuda — Arch Linux); point at a different prefix with make cuda CUDA_HOME=/path/to/cuda. Single-stream latency (one request), measured:

GPU	Config	RTF (single stream)
RTX 4060-class (~270 GB/s)	1.7B --quant-mixed	0.44
A100-SXM4-40GB (cloud)	0.6B (bf16 or int4)	0.39
A100-SXM4-40GB (cloud)	1.7B --quant-mixed + QWEN_CUDA_DP4A=1	0.50

QWEN_CUDA_DP4A=1 (new, opt-in) runs int4 weights against int8-quantized activations with integer __dp4a dots: 1.7B Talker −33% ms/f on the A100, ear-validated. Note the honest scaling limit: decode is bandwidth-bound only up to a point — past it, single-stream becomes launch-latency-bound (the A100's 5–6× bandwidth did not translate to 5× RTF), so big cards pay off in batching, not single-stream. Details → docs/cuda-performance.md.

Throughput — server request-batching (--serve --batch-size N, continuous batching + per-request streaming). Batching is a throughput / parallelism lever, not a per-request speedup — it serves N concurrent users in roughly the time of one by reading each weight once for all B sequences (matvec → matmat). CPU, CUDA and Metal all batch:

Backend	Batch speedup	Notes
CUDA (RTX 4060-class)	~3.35× at B=8	per-step (Talker 4.1× · CP 2.7×), ~3× end-to-end; output bit-identical solo-vs-batch
CUDA (A100, cloud)	aggregate RTF 0.47 at B=8	8 concurrent users, all served faster than real-time (1.7B; ~2.1× throughput)
Apple Metal (M2 Pro)	~2.8× at B=4	0.6B 2.81× · 1.7B 2.82× (consistent); batch output bit-identical to single-stream
CPU x86	~N on bandwidth-bound servers	~1× on cache-rich M1 (single-stream is already fast)
CPU ARM (Graviton3+, i8mm)	int8 batch matmat 2.1×, int4 1.6× (native SMMLA GEMM)	e2e batched server −19% wall @ B=4 vs the pre-SMMLA twin; bf16 1.5× (BFMMLA)

Documentation

Guide	Contents
Voice Cloning	Reference audio tips, ECAPA-TDNN internals, model comparison, samples
Custom Voices	.qvoice format, delta vs standard, managing profiles, troubleshooting
HTTP Server	All endpoints, request body, streaming, server performance
Server request-batching	vLLM-style --batch-size N: serve N concurrent users together, continuous batching, per-request streaming
VoiceDesign	Creating voices from text descriptions
Emotion — THE recipe	The one-and-only --emotion recipe: preset → STEER @ w12, clone → COMBINE; native preset per language. Single source of truth
Emotion on the small 0.6B	🆕 The small model has no steerable emotion subspace — so the emotion rides on the voice (4 KB asset per emotion, make emovoice). Emotion + paralinguistics + cloning at RTF ≈ 0.8
Expressivity packs .expr	Per-language emotion LoRA: which layers, why it's ~16–63 MB, file format, --expr/--expr-weight, per-voice rank. Train your own: training/expressivity-lora/
Inline markup	Audiobook/podcast tags in --text: [sad]/[joy] mid-text emotion switches, [sigh]/[huff] fillers, [pause:400ms]
Quantization	INT8/INT4, comparison table, recommendations
Performance	RTF benchmarks, component breakdown, CPU vs GPU, optimization history
x86 optimization	AVX2 / AVX-512 / VNNI findings, why it's memory-bound, how to benchmark your CPU
Hardware testing / benchmark your CPU	One-command per-box report (make bench-matrix), which CPU has which SIMD, where to rent ARM/x86, the RTF + throughput matrix to fill in
Building	All platforms, build targets, testing (golden-reference test needs Python + librosa)

Blog Posts

Post	Topic
Voice Cloning Internals	ECAPA-TDNN architecture deep-dive
Cross-Model Voice Analysis	Why delta format works (weight analysis)
Optimization Notes	RTF 3.5 → 1.3: the full M1 bf16 optimization story
Emotion on the Small Model	Why steering and fine-tuning both failed on the 0.6B, the cosine≈0 measurement that killed transfer, and the reframe that solved it: emotion as a property of the voice
Fast on Every CPU	SDOT (sub-1.0 on M1) + AVX2/AVX-512/VNNI on x86; why it's memory-bound

Credits & Acknowledgments

• Salvatore Sanfilippo (antirez) — This project wouldn't exist without his qwen-asr, a pure C Qwen2-Audio ASR engine that proved you can do real neural inference in plain C with mmap'd safetensors, BF16 NEON kernels, and zero dependencies. The entire architecture of this TTS engine — the approach, the style, the philosophy of minimal C inference — is directly inspired by his work. If you like this project, go star qwen-asr first.
• Michael Abrash — His Graphics Programming Black Book (1997) shaped how we think about performance. The chapters on data alignment, struct layout, and cache-friendly access patterns for the 386/486 are still relevant today — we got a 24% speedup from cache-line alignment (posix_memalign(64)), applying the same principles Abrash taught 30 years ago to modern SIMD and BLAS.
• John Carmack — His .plan files and QuakeCon talks on micro-optimization and cache friendliness were a constant reference. Where Abrash gave you the systematic rules and benchmarks, Carmack showed you the mindset: always think about how data flows through the CPU.
• Qwen3-TTS by the Qwen team at Alibaba — the model architecture, weights, and research. Models on Hugging Face. Paper.
• Qwen2.5 by the Qwen team — the base LLM architecture (GQA, RoPE, SwiGLU) used in the Talker and Code Predictor.

License

MIT