Deepgram Flux dropping short and soft utterances in non-English languages (German, Italian)

[crocodile85]

Across multiple voice deployments using Deepgram Flux, we are seeing a consistent pattern in non-English languages where caller speech is not transcribed at all — i.e. Flux emits no transcript rather than emitting a wrong one. From the application's point of view this looks like silence: the AI agent stays mute for 15–20 seconds waiting for a turn that never gets signaled, then either prompts again or the caller gives up.

We've ruled out microphone quality as the primary cause — issues reproduce on multiple mic setups, including ones with audibly clean input.

Observed behaviors:

1. Short utterances dropped.

• Single-word language selections at the start of a call (e.g. caller saying "Deutsch" in response to a German/English autodetect greeting) are frequently not transcribed at all. Easier to reproduce when said softly.

• Short backchannel/confirmation answers in Italian like "sì" or "l'ho trovato" ("I've found it") are commonly missed.

2. Quiet speech dropped entirely. The same phrase said at normal volume is transcribed; said quietly, no transcript is emitted (we'd expect a low-confidence guess rather than total silence).

3. Long silences indicating missed turn endpointing. In Italian sessions, testers report frequent 15–20s gaps where they spoke but the agent didn't respond. Reviewing the recordings vs transcripts confirms the audio was not transcribed by Flux at all. We suspect utterance detection and/or turn-end detection is materially weaker for these languages than for English.

4. Interruption misbehaviour. In at least one Italian session, the caller attempted to interrupt the agent, and the interrupting speech was not detected.

5. Confusable single-word triggers. When the single-word selection ("Deutsch") is transcribed, it's often misrecognized as similar-sounding English/Dutch words ("Dutch", "Torage", "Voight").

Request IDs for affected sessions:
019e455a-acf0-77e2-8072-e3952f2b863c
019e3b6b-c294-77b1-9d27-d7f106993721

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The programming language you are working in (e.g. JavaScript, Python).
• The deepgram product you are using (e.g Speech to Text, Agent API)

[crocodile85]

• TypeScript
• Speech to Text

[andreisergiu98]

We also noticed (for flux-multi in Italian) that some utterances never receive the EndOfTurn event (even after 10 seconds). My understanding is that regardless of confidence EndOfTurn should eventually fire after eot_timeout_ms, instead it's stuck in an Update loop when the user is silent, the user having to start speak to get out of the loop.

Also the docs previously mentioned that the default value for eot_timeout_ms was 5000 and for eot_threshold was 0.7, but now are unspecified. Did defaults change with the release of flux-multi?

[nkaimakis]

Also the docs previously mentioned that the default value for eot_timeout_ms was 5000 and for eot_threshold was 0.7

these are still the defaults for Flux multilingual as well.

We also noticed (for flux-multi in Italian) that some utterances never receive the EndOfTurn event (even after 10 seconds). My understanding is that regardless of confidence EndOfTurn should eventually fire after eot_timeout_ms, instead it's stuck in an Update loop when the user is silent, the user having to start speak to get out of the loop.

do you have audio recordings where these behaviors are reproducible? we definitely want to dig into this unexpected behavior further.

[deepgram-community[bot]]

I don't see anything off with your setup, and unfortunately we're not able to take a look at the audio since you're using mip_opt_out=true. do you have audio recordings where these behaviors are reproducible?

language hinting should help with the short utterances being mistranscribed in different languages. beyond this, we are actively working on some broader model architecture improvements that should specifically deliver improvements on short utterances in particular 🙂 we are also looking at making StartOfTurn more sensitive per your feedback on missed interruptions. any audio reproducers you can share of these issues are immensely helpful.

This message was sent by nick kaimakis from Deepgram, via our community automation.

[crocodile85]

Thanks for the update Nick! Happy to share the recordings - how do we do that securely?

[nkaimakis]

@crocodile85 send me an email at nickDOTkaimakisATdeepgramDOTcom