captioned_speech endpoint is not returning the timestamps for non-english languages

[hsavit1]

Describe the bug
The /dev/captioned_speech endpoint returns timestamps: null instead of word-level timestamps, even when using the latest Docker image that includes the merged streaming word timestamps feature from PR #173. Additionally, enabling streaming mode causes connection errors with chunked encoding issues.

Screenshots or console output

# Response (successful but no timestamps)
Response status: 200
Response headers: {'date': 'Wed, 11 Jun 2025 21:16:09 GMT', 'server': 'uvicorn', 'content-disposition': 'attachment; filename=speech.wav', 'cache-control': 'no-cache', 'content-length': '3794545', 'content-type': 'application/json'}
Saved audio file: chapter_30_tv1_es_ef_dora.wav (2845864 bytes)
Word-level timestamps: None  # ← Should contain timestamp array
Duration: unknowns

# Same payload but with "stream": true
Error processing streaming response: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

Branch / Deployment used
Branch: Latest master branch (post PR #173 merge)
Docker image: ghcr.io/remsky/kokoro-fastapi-cpu:latest
Deployment: Local Docker container
Docker command:
docker run -d --name kokoro-audio-temp -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest
Image digest: sha256:fb639bcaa170ef653a78c385c8701e5f0c7f502f52fe944f2c7aa7febf8a8906

Operating System
Platform: macOS Darwin 24.4.0
Docker version: Latest (pulled image successfully on June 11, 2025)
Hardware: CPU-only (using cpu variant of the image)
Architecture: Apple Silicon (M-series)

Additional context
Audio generation works perfectly - files are created successfully with correct content and size
Only timestamps are affected - the timestamps field consistently returns null
PR #173 context: This issue occurs despite the "Streaming word timestamps" feature being merged on Feb 23, 2025
Endpoint behavior:
/dev/captioned_speech with stream: false → Audio works, timestamps null
/dev/captioned_speech with stream: true → Connection errors
/v1/audio/speech → Crashes server entirely
Expected behavior: Based on the GitHub documentation example, timestamps should contain an array of word-level timing data
Workaround: Currently using --no-timestamps flag to generate audio without timestamp data

[hsavit1]

I think the error might be due to the fact that im trying to generate timestamps for ef_dora - it's a spanish text. Can you generate timestamps for non-english texts?

[hsavit1]

im able to generate timestamps for english voices but not spanish voices

[hsavit1]

to get timestamped audio for spanish text, i had to do this

// Spanish voices don't support timestamps → Use English voices
es: { 
  voices: ['ef_dora', 'em_alex'],           // Normal generation
  timestampVoices: ['af_heart', 'am_adam']  // Timestamp generation
}

[fireblade2534]

@hsavit1 This appears to be an issue with the kokoro library itself rather then kokoro-FastAPI.

[RBEmerson970]

Is there any sign of development of Kokoro itself?

[hsavit1]

Is there any sign of development of Kokoro itself?

yeah, also an issue to link to over there

[lionel-rowe]

to get timestamped audio for spanish text, i had to do this

// Spanish voices don't support timestamps → Use English voices
es: { 
  voices: ['ef_dora', 'em_alex'],           // Normal generation
  timestampVoices: ['af_heart', 'am_adam']  // Timestamp generation
}

@hsavit1 How does this work? Surely the timestamps wouldn't line up if you used different voices, with the English ones pronouncing with English phonology?