Skip to content

Bug: ChatterboxMultilingualTTS forces early EOS on every generation — all languages, all voice profiles #587

Description

@Blaecklight

Bug: ChatterboxMultilingualTTS forces early EOS on every generation — all languages, all voice profiles

Environment

  • Voicebox v0.5.0, Windows 11, RTX 5090
  • Engine: Chatterbox Multilingual
  • Occurs on first generation after fresh Voicebox start
  • Reproducible with all voice profiles, German and English

Observed behavior
Every generation is cut off early. The log consistently shows:

alignment_stream_analyzer - WARNING - 🚨 Detected 2x repetition of token 6405
alignment_stream_analyzer - WARNING - forcing EOS token, long_tail=tensor(True), token_repetition=True
t3 - INFO - ✅ EOS token detected! Stopping generation at step 82

Generation stops between step 42 and step 231 regardless of text, text length, language, or voice profile.

Root cause (suspected)
Two related issues in chatterbox/models/t3/inference/alignment_stream_analyzer.py:

  1. Aggressive token_repetition check — the analyzer triggers forced EOS when only 2 consecutive identical tokens are detected (len(set(self.generated_tokens[-2:])) == 1). Natural speech tokens for long vowels, pauses, or specific phonemes can legitimately repeat, causing false positives even on the very first generation call.

  2. Forward hook leak (related: issue Low-End Laptop + Voicebox: Performance Limits, Model Advice, and Remote Usage Guide #504, PR miniaudio is missing #505) — each ChatterboxMultilingualTTS.generate() call registers new forward hooks in the AlignmentStreamAnalyzer without removing old ones, corrupting the EOS heuristic over repeated calls.

Suggested fix
For the token_repetition threshold, changing:

token_repetition = (
    len(self.generated_tokens) >= 3 and
    len(set(self.generated_tokens[-2:])) == 1
)

to:

token_repetition = (
    self.complete and
    len(self.generated_tokens) >= 4 and
    len(set(self.generated_tokens[-3:])) == 1
)

would make the heuristic less aggressive and require the generation to be in a complete state before forcing EOS.

The forward hook leak fix from PR #505 should also be included to prevent progressive degradation over multiple calls.

Impact
Chatterbox Multilingual is unusable for real-time TTS applications (e.g. voice , agents) where multiple short generations are triggered in sequence. All languages and voice profiles are affected.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions