fix(nemo): extract Hypothesis.text for TDT/RNNT ASR models

[fqscfqj]

Problem

NeMo Parakeet TDT models (e.g. parakeet-tdt-0.6b-v3) deployed via the nemo backend always produce empty transcription output.

Root Cause

NeMo's transcribe() returns different types depending on the model architecture:

• CTC models (e.g. Whisper): List[str] — works fine
• TDT/RNNT models (e.g. parakeet-tdt-0.6b-v3): List[Hypothesis] — the decoded text lives in the Hypothesis.text attribute

The backend code at backend/python/nemo/backend.py line 105 did:

text = results[0]  # Hypothesis object, not a str!

This assigned the entire Hypothesis dataclass to the protobuf string field. When protobuf tried to serialize it, it either raised a TypeError (caught by the except block → returns empty) or silently converted to an empty string. Either way, the transcript was always blank.

Fix

Check the return type and extract .text from Hypothesis objects when present:

result = results[0]
if isinstance(result, str):
    text = result
elif hasattr(result, 'text'):
    text = result.text if result.text else 
else:
    text = str(result) if result else 

This is backward-compatible — CTC models still return strings and take the first branch.

Testing

Verified by reading the NeMo source code:

• nemo/collections/asr/parts/submodules/rnnt_decoding.py → rnnt_decoder_predictions_tensor() always returns List[Hypothesis] even when return_hypotheses=False
• nemo/collections/asr/models/rnnt_models.py → _transcribe_output_processing() passes this through unchanged

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates transcript extraction logic in the NeMo backend to correctly handle different model output types (CTC strings vs. RNNT/TDT Hypothesis objects).

Changes:

• Adds type-aware handling for results[0] to extract text from either str or an object with a .text attribute.
• Improves inline documentation clarifying expected return types from different model families.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mudler]

thanks!