Voice Agent API (V1) — intermittent STT stall: UserStartedSpeaking fires but no ConversationText/UtteranceEnd returned for delivered audio (no error, no close)

[mustafamahmood]

Summary

On the Voice Agent API, a subset of our sessions intermittently stall on speech-to-text. The agent greets normally; the caller speaks; we receive UserStartedSpeaking — but then no user ConversationText and no UtteranceEnd are ever emitted, even though we keep streaming caller audio. There is no error event and no Close from the server. The session produces nothing further until the caller hangs up. Most sessions transcribe fine; this hits a minority.

Configuration

• Voice Agent API (V1), real-time, AgentV1* API.
• Listen / STT: Deepgram nova-3, smart_format=true ← the stage that stalls
• Think / LLM: BYO OpenAI gpt-4o-mini (AgentV1OpenAiThinkProvider)
• Speak / TTS: Deepgram (aura voice)
• Audio in: Twilio Media Streams → μ-law 8 kHz, 20 ms frames (~50 fps), streamed continuously into the agent socket.

Specific failing session

request_id: 019eadbd-4218-7b40-aa5d-ccd219ed60b7

Timeline (UTC, 2026-06-09):

18:55:22.192  Welcome received; greeting played normally
              (SettingsApplied → assistant ConversationText → AgentAudioDone)
18:55:33.570  UserStartedSpeaking received  (caller began speaking)
18:55:33 → 18:56:01  we KEPT streaming caller audio the whole time
              (~216 additional 20ms frames ≈ 4.3s of speech forwarded
               after UserStartedSpeaking)
   throughout: ZERO user ConversationText, ZERO UtteranceEnd,
               ZERO errors, NO Close
18:56:01.673  caller hung up; we tore the connection down

Expected vs actual

• Expected: after UserStartedSpeaking, with audio still flowing, a user ConversationText (and/or UtteranceEnd) so the agent can respond.
• Actual: nothing further from STT for the rest of the session — silent, with no error and no close.

Frequency

Intermittent; not load-correlated (reproduced on a single isolated call).

Questions

1. What could cause nova-3 STT in Voice Agent to emit UserStartedSpeaking but never emit ConversationText/UtteranceEnd for audio that is still being delivered — with no error and no Close?
2. Known issue? Any recommended keep-alive / endpointing / timeout config to prevent or surface it?
3. Is there a server-side signal we can detect to recover (reconnect) instead of waiting for the caller to hang up?
4. Can you confirm, from request_id 019eadbd-4218-7b40-aa5d-ccd219ed60b7, what happened to that session's transcription?

Any help will be appreciated, thanks.

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[mustafamahmood]

Its connected

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The programming language you are working in (e.g. JavaScript, Python).

[mustafamahmood]

Python