nvidia/nemotron-3.5-asr-streaming-0.6b: transcribe() fails with "ValueError: Unknown prompt key: 'None'"

[deneme777342-ai]

Describe the bug

When attempting to run inference using the pretrained nvidia/nemotron-3.5-asr-streaming-0.6b model via ASRModel.from_pretrained, a ValueError is raised stating "Unknown prompt key: 'None'". The model expects a specific language prompt/key to be passed or configured, but standard usage of .transcribe() defaults to None, causing the transcription to fail out of the box. Moreover, https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b page doesn't provide any instruction on how to pass the target_lang (including how to use the auto option).

Steps/Code to reproduce bug

Steps:

Install NeMo and its ASR dependencies.
Load the model nvidia/nemotron-3.5-asr-streaming-0.6b using ASRModel.from_pretrained().
Call transcribe() on a valid WAV file.
Observe the exception:
ValueError: Unknown prompt key: 'None'. Available: ['en-US', 'en', 'en-GB', 'enGB', 'es-ES', 'esES', 'es-US', 'es', 'zh-CN', 'zh-ZH']...

Expected behavior

The model should either default to a reasonable fallback language prompt (e.g., 'en') when no prompt is provided, or the .transcribe() method/documentation should clearly indicate how to supply the mandatory prompt key argument for this specific streaming model architecture so it doesn't fail with None.

Environment overview

• Environment location: Bare-metal
• Method of NeMo install: Used these 3 commands "
  apt-get update && apt-get install -y libsndfile1 ffmpeg
  pip install Cython packaging
  pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]
  " as described in https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b

Environment details

• OS version: Ubuntu 22.04
• PyTorch version: 2.12.1
• Python version: 3.12.0

Additional context

The nvidia/nemotron-3.5-asr-streaming-0.6b model is a prompt-conditioned multilingual streaming ASR model. The underlying error seems to stem from the model's expectation of a language/prompt token during the forward pass, which is not being implicitly handled or exposed cleanly through the standard asr_model.transcribe() pipeline. It is possible for you to share code examples for this model where target_lang is specified or auto option is used.

[rtxlovelace-collab]

# cfg = model.get_transcribe_config()
# cfg.target_lang = "zh-CN"

result = model.transcribe(
    ["voice.mp3"],
    # override_config=cfg
)

Configure the language identification method. The default value is set to auto.

[deneme777342-ai]

It looks like a strange method to do that. Why isn't the library already providing this feature by default. In addition, when I use multiple samples within a batch like 16 or 128, I still get this error even if I use the technique described.

[enas-albasiri]

Pass target_lang explicitly via override_config:

model = ASRModel.from_pretrained("nvidia/nemotron-3.5-asr-streaming-0.6b")
cfg = model.get_transcribe_config()
cfg.target_lang = "en-US"        # or "auto" for auto-detection
result = model.transcribe(["voice.wav"], override_config=cfg)

We will add a fix to configure .transcribe() to use langid when passed and default to 'auto' when no langID provided (similar behavior to examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py )

[deneme777342-ai]

Thanks for your answer, yet it is identical to what @rtxlovelace-collab described. Even if it partially solves the problem, I still get this error when I use multiple samples within a batch like 16 or 128. Moreover, the performance is really poor when I use the method described. Is it normal?

[rehan243]

Oh yeah, I ran into something super similar with this model a couple of weeks ago. The issue is that transcribe() doesn't auto-handle the prompt_key for some of these streaming models, which is... not great. IIRC, the trick is to explicitly pass a prompt dict with a prompt_key set to one of the valid options (like 'en-US') when you call transcribe(). So, something like:

model.transcribe(audio_file, prompt={'prompt_key': 'en-US'})

It's annoying because you'd expect the default to do something sane instead of throwing a ValueError. Also, totally agree that the Hugging Face model card is missing key info, it's not obvious at all how target_lang interacts with prompt_key, and I'm pretty sure "auto" doesn't even work properly for this model.

Quick question: are you using the pip version of NeMo or the latest from GitHub? I've seen differences in behavior depending on the version.

[deneme777342-ai]

Thanks for the response @rehan243 ; nevertheless, it didn't solve the problem. It is still throwing ValueError. By the way, I installed NeMo using the command pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr] as described in hf page.