Skip to content

fix(cli): map zh to espeak-ng's cmn for Kokoro TTS synthesis#2094

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/tts-zh-cmn-espeak-lang
Jul 9, 2026
Merged

fix(cli): map zh to espeak-ng's cmn for Kokoro TTS synthesis#2094
miguel-heygen merged 2 commits into
mainfrom
fix/tts-zh-cmn-espeak-lang

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

hyperframes tts --lang zh (or any zh-prefixed voice like zf_xiaobei) now translates the language code to cmn only at the point it's handed to the Python/kokoro-onnx/espeak-ng boundary. The public --lang zh value is unchanged everywhere else (progress messages, SynthesizeOptions, return values).

Why

espeak-ng 1.52.0 maps Mandarin Chinese to the ISO 639-3 code cmn, not zh. zh is Kokoro's own voice-ID-prefix convention (and matches Kokoro's docs), but synthesize() forwarded it verbatim into kokoro_onnx.Kokoro.create(lang=lang), which surfaces as espeak-ng's language table and fails with:

language zh is not supported by espeak backend

Root cause confirmed by a user: calling kokoro_onnx.Kokoro.create(lang='cmn') directly (bypassing our CLI) works fine.

How

Added a small ESPEAK_LANG_OVERRIDES map in packages/cli/src/tts/synthesize.ts and applied it only to the value sent as argv[7] to the Python subprocess, right before the execFileSync call. SUPPORTED_LANGS/the public lang value in manager.ts are untouched — zh stays correct CLI-facing UX.

Test plan

  • Unit tests added/updated
  • Manual testing performed
  • Documentation updated (if applicable)

Added packages/cli/src/tts/synthesize.test.ts, mocking execFileSync, manager.js's ensureModel/ensureVoices, and fs.existsSync/mkdirSync so synthesize() can run without a real Python/espeak install:

  • Repro: ran the new test against the pre-fix code — the zh case failed with expected 'zh' to be 'cmn' (captured argv sent "zh" verbatim), confirming the reported failure mode.
  • Post-fix: all 3 new tests pass (zhcmn translation, es passes through unchanged, public langApplied/return value unaffected).
  • packages/cli/src/tts/* full suite: 19/19 passing.
  • oxlint + oxfmt --check on changed files: clean.
  • tsc --noEmit (via the repo's lefthook pre-commit typecheck): clean.

espeak-ng 1.52.0 recognizes Mandarin Chinese as the ISO 639-3 code "cmn",
not Kokoro's own voice-prefix convention "zh". `hyperframes tts --lang zh`
was forwarding "zh" straight through to kokoro_onnx.Kokoro.create(), which
failed with "language zh is not supported by espeak backend". Translate
only at the Python/espeak boundary; the public --lang value stays "zh"
since that matches Kokoro's own docs and voice-ID prefixes.

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R1 — hyperframes #2094 at 0283fe99

🟢 LGTM. Minimum-viable fix for a real user-reported failure, placed at the correct boundary. Verified per-line at head SHA.

Verified

  • Root cause called correctly. hyperframes tts --lang zh (or -voice zf_*) sent "zh" as argv[7] to the Python subprocess, which forwarded to kokoro_onnx.Kokoro.create(lang="zh") → espeak-ng's language table → "language zh is not supported by espeak backend". Miguel's user-confirmation trace (calling Kokoro.create(lang="cmn") directly works fine) pins the layer.
  • Boundary placement is the correct choice. synthesize.ts:164 computes espeakLang = ESPEAK_LANG_OVERRIDES[lang] ?? lang and only that value is substituted in the execFileSync argv at position 7. Everything else keeps lang:
    • SynthesizeOptions.lang remains typed SupportedLang = "zh" in the public API (:105).
    • SynthesizeResult.langApplied is a boolean echoed from Python — unaffected by the value (:114).
    • Progress message at :162 uses lang, not espeakLang — users still see "Generating speech with voice zf_xiaobei (zh)...", not (cmn).
    • Voice-ID inference at :128 (inferLangFromVoiceId) still returns zh for z* prefixes.
  • Only one Python-exec site passes lang. Grepped all four TTS files at head — packages/cli/src/tts/{manager,index,python,synthesize}.ts. python.ts's execFileSync calls are all Python probes (--version, -c "import <pkg>"). Only synthesize.ts:165 runs the synth script with lang. So there's no second execFileSync bypassing the override. Complete coverage.
  • Type shape is honest. Partial<Record<SupportedLang, string>> at :58 constrains keys to SupportedLang (compiler catches typos on the key side). Values are open string, which lets future entries land without a type change. Trade-off is fair for a map with one entry.
  • Test file structure. synthesize.test.ts uses vi.hoisted for the shared capture + vi.mock("node:child_process") / vi.mock("node:fs") / vi.mock("./manager.js") before the dynamic await import("./synthesize.js") — the standard vitest ESM mock ordering. execFileSyncMock disambiguates the mocked calls by argv shape (args[0] === "--version" / args[0] === "-c" / cmd === "which"|"where" / else → capture) — no accidental catches. LANG_ARGV_INDEX = 7 matches the actual argv position at synthesize.ts:167.
  • Regression coverage is real. PR body notes: "ran the new test against the pre-fix code — the zh case failed with expected 'zh' to be 'cmn'." That's the correct proof-of-repro — the test fails against pre-fix code and passes against post-fix, so a future refactor that reverts the override can't sneak through green.
  • CI fully green at 0283fe99 — Test, Typecheck, Fallow, Lint, Format, CLI smoke (required), CLI: npx shim (macos/ubuntu/windows), Build, SDK unit+contract+smoke, Studio load smoke, Test: runtime contract, Render on windows-latest, Tests on windows-latest, Preview parity, Semantic PR title, Smoke: global install. 41 checks pass.

Observations (non-blocking)

O1 — Sibling-mismatch surface: ESPEAK_LANG_OVERRIDES has one entry, but the same class of Kokoro-vs-espeak drift could affect other supported langs. SUPPORTED_LANGS at manager.ts:23-33 includes "pt-br", "fr-fr", "en-us", "en-gb" (all BCP-47 with region), plus "es", "hi", "it", "ja", "zh". espeak-ng generally accepts BCP-47 forms with -region suffixes, so en-us/en-gb/fr-fr/pt-br likely work as-is. Two watchpoints if reports come in:

  1. es — espeak-ng ships both es (Iberian Spanish) and es-419 (Latin American Spanish). Kokoro passes "es" regardless of the voice's region. If Kokoro's Spanish voices are trained on LatAm accents but espeak phonemizes as Iberian, it's a quality mismatch, not a "not supported" hard-fail — but a symptom would look like garbled prosody, easy to miss.
  2. ja — espeak-ng's Japanese support exists (ja) but is documented as experimental; some downstream tools have historically preferred routing Japanese through a different phonemizer (misaki/cutlet-mecab). If a jf_*/jm_* voice ever produces language ja is not supported on some user's espeak build, the fix pattern is a one-line map entry.

Not this PR's problem — you correctly scoped to the empirically-confirmed failure. Naming the pattern here so a future report doesn't get re-diagnosed from scratch.

O2 — Test 3 (returns the public zh value in the result...) doesn't actually assert what its comment claims. SynthesizeResult (:109-115) doesn't carry a lang string — only langApplied: boolean. So there's no cmn string in the return path to leak. The assertion expect(result.langApplied).toBe(true) is a bool echo from the Python-mock, verifying the mock returned what the mock returned; it can't distinguish leaked-cmn from correct-zh because that distinction doesn't exist in the API surface. Test 1's expect(argv[LANG_ARGV_INDEX]).not.toBe("zh") already covers the actual swap. Test 3 is safe but redundant; if you want it to earn its keep, either drop it or extend to also assert the argv wasn't tampered with (e.g., that voice: "zf_xiaobei" and the input text still made it through untouched — a sanity check on the whole argv shape).

O3 — Explicit --lang zh path isn't test-covered separately. Both new tests pass options.voice but not options.lang, so lang: SupportedLang = options?.lang ?? inferLangFromVoiceId(voice) at :128 always resolves via inference. Same code branch as the explicit --lang path, so it doesn't miss a bug shape today — but a defensive symmetry test (await synthesize(text, path, { lang: "zh" }) → argv[7] === "cmn") would prevent a future refactor of the option-resolution from breaking one path silently. Trivial add, not asking for it now.

Awareness only

  • espeak-ng version dependency. cmn was added to espeak-ng around v1.51; users on older builds would still fail even with this fix, this time with a different error. If hyperframes documents a minimum espeak-ng version somewhere (I didn't grep), worth ensuring it's ≥ 1.51.

R1 by Via

@miguel-heygen miguel-heygen merged commit 995db68 into main Jul 9, 2026
41 checks passed
@miguel-heygen miguel-heygen deleted the fix/tts-zh-cmn-espeak-lang branch July 9, 2026 04:55

@james-russo-rames-d-jusso james-russo-rames-d-jusso left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R2 — hyperframes #2094 at bdf0e89

Reviewed at bdf0e89 (post-merge). Layering onto Via's R1 at 0283fe9 — I verified R1's findings and the follow-up commit that landed on top of them, so this stays tight rather than re-treading.

🟢 R1 observations — all resolved by the follow-up commit

Miguel pushed test(cli): cover Kokoro zh language override (bdf0e89) between R1 and merge. Reading it as a direct response to R1:

  • O1 (sibling-mismatch surface) — resolved by structural enforcement. New EXPECTED_ESPEAK_LANGS satisfies Record<SupportedLang, string> table + the keeps every non-Mandarin public lang unchanged at the Python boundary iteration test makes the drift check compile-time-coupled: adding a lang to SUPPORTED_LANGS forces an explicit entry in the expected table or tsc fails. Not a correctness guarantee (a future "zh-TW": "zh-TW" still typechecks and would fail at runtime), but it forces the espeak-alignment discussion into the diff of whoever adds the lang — which is the practical outcome R1 was after.
  • O2 (tautological langApplied test) — resolved by removal. The returns the public zh value… test was dropped rather than extended. Right call — it was asserting a bool the mock itself returned.
  • O3 (explicit --lang zh path uncovered) — resolved by new test. translates an explicit zh override too passes voice: "af_heart" + lang: "zh" so options?.lang wins over inferLangFromVoiceId(voice). Bonus: also asserts the progress-message string contains "(zh)" not "(cmn)", guarding against the override leaking into user-facing UX. Cleaner than the symmetry check R1 asked for.

Cross-checks independent of R1

  • Regression is real, not tautological. Tests drive synthesize() (the actual production entry point) and capture argv[7] from a mocked execFileSync. Pre-fix code passed lang verbatim into argv[7], so expect(argv[7]).toBe("cmn") genuinely fails against pre-fix source. Matches Miguel's stated repro in the PR body.
  • No behavior-enshrinement risk. Tests assert the correct mapping, so a future revert of the override in synthesize.ts (ESPEAK_LANG_OVERRIDES block) trips the assertion at the argv boundary. Safe against silent regressions.
  • CI green at merge. 41 required checks pass at bdf0e89 (Test, Typecheck, Lint, Format, CLI smoke, CLI: npx shim on macos/ubuntu/windows, Build, SDK unit+contract+smoke, Studio load smoke, runtime contract, Render / Tests on windows-latest, Preview parity, Smoke: global install, etc.).

🟡 Awareness-only, for future espeak drift (not for this PR)

Miguel — one thought worth carrying forward: satisfies Record<SupportedLang, string> catches missing keys but not incorrect espeak values. If zh-Hant, yue, or a Hakka variant ever land in SUPPORTED_LANGS, a well-meaning "mirror the string" default ("zh-Hant": "zh-Hant") would compile, pass the invariance test, and fail at runtime with the same espeak error class this PR fixed. Not this PR's scope — but if the Kokoro-vs-espeak drift keeps biting, a build-time cross-check against espeak-ng --voices output would turn "did I remember an override?" into a caught error rather than a customer bug report. Filing under "if this bug shape recurs, escalate to catalog-source-of-truth."

Clean execution — minimum-viable fix at the correct boundary, tests that survive refactors, and a follow-up commit that took R1 seriously without over-reacting. LGTM from my side; leaving as a comment since the merge already happened.

Review by Rames D Jusso

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants