Skip to content

feat(voice): add VoxCPM2 (OpenBMB) as a fifth TTS backend candidate [spec 02] - #10

Merged
wine-fall merged 1 commit into
mainfrom
zachg-0704--add-voxcpm2
Jul 4, 2026
Merged

feat(voice): add VoxCPM2 (OpenBMB) as a fifth TTS backend candidate [spec 02]#10
wine-fall merged 1 commit into
mainfrom
zachg-0704--add-voxcpm2

Conversation

@wine-fall

@wine-fall wine-fall commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Implements specs/spec02/02-voice-provider.md (also touches specs/DESIGN.md).

Summary

Adds VoxCPM2 (mlx-community/VoxCPM2-8bit) to the MLX voice-backend registry as a post-L0 blind-A/B candidate (specs/spec02/02-voice-provider.md §3.3). The thin MlxAudioBackend + MlxProfile seam means the whole model is one PROFILES row — the standardized SynthesisRequest → wav layer does the rest.

Smoke-verified end-to-end through the production path (build_backend → load → warm → synthesize):

  • ✅ loads — mlx-audio #649 (model_type: voxcpm2) does not block on the installed version
  • ✅ 48 kHz native output, real non-empty wav via the standard seam (I/O conforms — no core/backend class change)
  • ⏱️ measured RTF ≈ 1.6 on M3 Pro / 8bit (whole-clip, warm) → slower than real-time, so it's a quality-reference / pre-generated candidate, not the real-time default (Qwen3-TTS stays the real-time pick; Spark the fallback)

Bug fixed (caught by closing code-review)

The CLI --voice argparse choices were hardcoded and rejected voxcpm2 even though build_voice() accepted it (unit tests bypassed argparse, so they missed it). Fixed at altitude — choices now derived from PROFILES so future rows are CLI-selectable automatically — and guarded by a new regression test.

Changes

  • src: PROFILES row; --voice choices derived from the registry
  • spec: specs/spec02/02-voice-provider.md §3.3/§4/§5/§6 + specs/DESIGN.md shortlist + README/pyproject aligned
  • tests (test-first): registry + integration params, and test_cli_voice_flag_accepts_every_registered_voice

Test plan

  • pytest -m "not integration"101 passed; ruff + format clean
  • On-demand: pytest -m integration now includes voxcpm2 (needs [tts-mlx] + weights)
  • Owed (human): blind A/B by ear among the candidates — not self-verifiable

AI coding brief

Original request — While evaluating OpenBMB's new TTS for the murmur companion-radio project, confirmed VoxCPM2 supports MLX and compared it against the already-installed candidates (Spark/Qwen3/Chatterbox/Dia). Asked to pull the model via HuggingFace and add it to the candidate pool on a new branch. Why: widen the blind-A/B voice pool with a higher-fidelity (48 kHz, tokenizer-free) contender.

Manual interventions — The human pushed back hard on "you just added comments — does the new model's I/O actually match the standardized layer?", which drove a real end-to-end smoke (proving the seam conforms, measuring RTF 1.61, and clearing the #649 load risk) instead of taking the design on faith. 8bit-over-4bit was settled in discussion (quality reference for the A/B; 18 GB fits). The human then said to ship, then to open the PR.

Retro — Two things would have shipped this faster: (1) the CLI registry-drift bug was foreseeable — prompting "add the model and make sure every entry point (CLI choices, config, docs) derives from the registry, not a hardcoded list" up front would have pre-empted it; (2) the first huggingface-cli download silently no-op'd (deprecated in hub 1.21) — a one-line "verify the weights landed in the cache before proceeding" check would have caught it a step earlier.

🤖 Generated with Claude Code

…spec 02]

Adds VoxCPM2 (mlx-community/VoxCPM2-8bit) to the MLX backend registry as a
post-L0 blind-A/B candidate. The thin MlxAudioBackend + profile seam means the
whole model is one PROFILES row — smoke-verified end-to-end through the standard
SynthesisRequest -> wav path: loads (mlx-audio #649 does not block), 48 kHz
native, measured RTF ~=1.6 on M3 Pro/8bit (slower than real-time -> a
quality-reference / pre-generated candidate, not the real-time default).

Also fixes a registry-drift bug the review caught: the CLI --voice choices were
hardcoded and rejected 'voxcpm2' even though build_voice() accepted it. Now
derived from PROFILES so future rows are CLI-selectable automatically, guarded
by a new regression test.

- src: PROFILES row; --voice choices derived from the registry
- spec 02 §3.3/§4/§5/§6 + DESIGN shortlist + README/pyproject aligned
- tests (test-first): registry/integration params + CLI-accepts-every-voice guard

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wine-fall
wine-fall force-pushed the zachg-0704--add-voxcpm2 branch from a2b0b89 to 6385617 Compare July 4, 2026 15:01
@wine-fall
wine-fall merged commit 2d262dd into main Jul 4, 2026
3 checks passed
@wine-fall
wine-fall deleted the zachg-0704--add-voxcpm2 branch July 4, 2026 15:02
wine-fall added a commit that referenced this pull request Jul 4, 2026
Codifies the murmur PR workflow so the check_pr.py title/description gate is
run LOCALLY and required green before pushing — the failure mode that just cost
a CI round-trip on #10 (body linked "spec 02" but no on-disk specs/*.md path).

Captures the session's hard-won specifics: org PR rules (branch naming, base
main, no direct push to master, squash merge, required AI coding brief), what
check_pr.py enforces per commit-type, and the gh-pr-edit-silently-no-ops
workaround (PATCH the body via gh api REST instead).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wine-fall added a commit that referenced this pull request Jul 4, 2026
Codifies the murmur PR workflow so the check_pr.py title/description gate is
run LOCALLY and required green before pushing — the failure mode that just cost
a CI round-trip on #10 (body linked "spec 02" but no on-disk specs/*.md path).

Captures the session's hard-won specifics: org PR rules (branch naming, base
main, no direct push to master, squash merge, required AI coding brief), what
check_pr.py enforces per commit-type, and the gh-pr-edit-silently-no-ops
workaround (PATCH the body via gh api REST instead).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant