[Gemini Live API] gemini-3.1-flash-live-preview — VAD turn thrashing + incorrect language detection across multiple tenants

[Zerohunter36]

Description of the bug:

We're operating a multi-tenant voice agent platform built on the Gemini
Live API. Starting on the weekend of 2026-06-07, we observed a systemic
regression in gemini-3.1-flash-live-preview affecting ALL agents in our
platform regardless of language, voice, prompt, or tenant.

The model:

1. Generates excessive turn events (~1-2 turns per second of conversation)
   even when the automaticActivityDetection config explicitly sets
   conservative thresholds.
2. Incorrectly identifies user audio as a different language than the one
   declared in inputAudioTranscription.languageCode and produces wrong
   transcripts (e.g. Spanish speech transcribed as Portuguese tokens
   "É. Não" appearing in es-MX sessions).

No changes were made to our integration code in the time frame the issue
appeared. The bridge code has been stable since 2026-06-03 (last commit
touching Live API integration).

== Reproduction ==

Endpoint: generativelanguage.googleapis.com
SDK: @google/genai v1.5.0
Model: gemini-3.1-flash-live-preview
Config sent:
{
"responseModalities": ["AUDIO"],
"speechConfig": {
"voiceConfig": { "prebuiltVoiceConfig": { "voiceName": "Orus" } }
},
"inputAudioTranscription": { "languageCode": "es-MX" },
"outputAudioTranscription": { "languageCode": "es-MX" },
"realtimeInputConfig": {
"automaticActivityDetection": {
"startOfSpeechSensitivity": "START_SENSITIVITY_LOW",
"endOfSpeechSensitivity": "END_SENSITIVITY_LOW",
"prefixPaddingMs": 500,
"silenceDurationMs": 3000
}
},
"systemInstruction": ""
}

User audio: PCM 16-bit 16kHz mono Spanish (es-MX), real human speech with
natural pauses (typical conversational rhythm).

Expected behavior:

• ~5-15 turn events per minute of conversation.
• Transcripts in es-MX.

Observed behavior:

• 60-160 turn events per minute (10-30× expected).
• Random Portuguese tokens in transcripts of Spanish speech.
• Model interrupts the user mid-sentence (turn_complete fires <500ms after
  user speech starts, while the user is still talking).
• silenceDurationMs: 3000 appears to be ignored.

== Evidence ==

We collected 65 conversations from our production platform across 5
unrelated tenants (different prompts, voices, languages, agents). All
exhibited the same pattern when using gemini-3.1-flash-live-preview:

Agent Tenant Voice Convs Avg turns/sec
abf5ab03 tenant_A Orus 20 1.99
e3913a4b tenant_B Kore 5 1.46
abf5ab03 tenant_C Orus 3 1.31
915306d8 tenant_D Kore 28 1.01
8869861b tenant_E Kore 6 0.88

A control group using gemini-2.5-flash-native-audio-latest with the same
prompts, voices, and audio characteristics did NOT exhibit thrashing.

== What we ruled out ==

• Network/jitter from client to our origin: clean tests from <100ms RTT
  clients also exhibit the issue.
• Our VAD config not being applied: verified at the SDK level that
  realtimeInputConfig.automaticActivityDetection is passed correctly
  (logged the exact payload sent to genai.live.connect()).
• Tenant-specific config: same agent under two different tenants both
  exhibit the issue.
• Voice-specific: multiple voices (Orus, Kore) all affected.
• Code regression: no changes to bridge/Live API integration in the
  affected time window.

== What we need ==

1. Confirmation of whether gemini-3.1-flash-live-preview was updated
   server-side around 2026-06-07.
2. Confirmation that silenceDurationMs and endOfSpeechSensitivity are
   supposed to be honored in the current preview release.
3. Whether the language detection bug (Portuguese tokens in es-MX session)
   is known.
4. Workaround / config recommendation while a fix is in progress.

== Our project ID / API key ==

(provide your GCP project ID + the API key used in Live sessions when
support asks — DO NOT paste it in the public ticket if it's an open
form; wait for a private support engineer to ask)

== Sample timestamps (UTC) for log lookup ==

We can provide specific conversation timestamps where the bug manifests
upon request. Recent example: 2026-06-09 15:05:11 UTC, 79-second session
that generated 80 turn events.

Thanks.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[kkorpal]

Hey @Zerohunter36,

Thanks so much for this incredibly detailed debugging!

I will bring your findings directly to the Live API engineering team for investigation.

Thanks!

[IngLP]

Hi, Any news?

[kkorpal]

Hi @Zerohunter36, following up on this!

To help us investigate the VAD thrashing and language detection issues directly, could you share a minimal code snippet showing your setup and audio streaming logic? Having your exact setup will make it much easier to reproduce and track this down on our end. Thanks!