fix(google): support gemini-3.1-flash-live-preview realtime API for g…#5262
Closed
acharan-tech-200037 wants to merge 4 commits intolivekit:mainfrom
Closed
fix(google): support gemini-3.1-flash-live-preview realtime API for g…#5262acharan-tech-200037 wants to merge 4 commits intolivekit:mainfrom
acharan-tech-200037 wants to merge 4 commits intolivekit:mainfrom
Conversation
…enerate_reply and update_chat_ctx
Author
|
@livekit/maintainers This PR is ready for review! Summary: Adds support for Gemini 3.1 Flash Live Preview by switching from Changes:
Testing:
Ready for review when you have time. Thanks! |
Author
|
@theomonnom @tinalenguyen can you please review this pr. |
Author
|
Closing this in favor of #5251 by @Hormold which takes a cleaner Key differences:
Tested and Hormold's approach is architecturally correct. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr resolves #5260
Adds support for
gemini-3.1-flash-live-previewin the Google Realtime plugin.Problem
The
gemini-3.1-flash-live-previewmodel (released March 26, 2026) rejects allsend_client_contentWebSocket calls with a 1007 error (Request contains an invalid argument). This causes:generate_reply()to timeout waiting forgeneration_createdeventupdate_instructions()/update_chat_ctx()to failThe model only accepts
send_realtime_inputfor all content delivery.How it was confirmed
A minimal test script was used to isolate the issue:
send_client_content→ ❌ 1007 error (even with minimal config)send_realtime_inputwith text → ✅ Works perfectlyChanges
File:
livekit-plugins-google/livekit/plugins/google/realtime/realtime_api.pygemini-3.1-flash-live-previewtoKNOWN_GEMINI_API_MODELSREALTIME_INPUT_ONLY_MODELSconstant to detect models that rejectsend_client_contentgenerate_reply()— Routes instructions as text viaLiveClientRealtimeInputinstead ofLiveClientContent_main_task()— Converts initial chat context tosend_realtime_inputwith[role]: textprefixes_send_task()— Catch-all: converts anyLiveClientContenttosend_realtime_inputwith role prefixes (coversupdate_instructionsandupdate_chat_ctxflows)All changes are backward-compatible — older models continue using
send_client_contentas before.