LLMs that support bidirectional mode

[TanejaAnkisetty]

My team is building a voice based chatbot with bidirectional streaming via WebSocket. The backend has Agentic workflow using ADK and Gemini models. Currently we are using the gemini-2.0-flash-exp model. It is clear that it is an experimental model and hence it has low quota and Provisioned Throughput is not supported. Hence, the recommendation was to migrate to a stable model. But I'm unable to find a LLM that supports our requirements.

Following are the different errors I have received when I tested various LLMs -

1. gemini-2.0-flash : ERROR echo_ai.consumer consumer agent_to_client: Error during agent streaming: received 1007 (invalid frame payload data) gemini-2.0-flash is not supported in the live api.
2. gemini-2.5-flash : websockets.exceptions.ConnectionClosedError: received 1007 (invalid frame payload data) gemini-2.5-flash is not supported in the live api.; then sent 1007 (invalid frame payload data) gemini-2.5-flash is not supported in the live api
3. gemini-2.0-flash-live-preview-04-09 : google.genai.errors.ClientError: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'gemini-2.0-flash-live-preview-04-09 is not supported in the generateContent API.', 'status': 'INVALID_ARGUMENT'}}
4. gemini-live-2.5-flash-preview-native-audio : google.genai.errors.ClientError: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'gemini-live-2.5-flash-preview-native-audio is not supported in the generateContent API.', 'status': 'INVALID_ARGUMENT'}}

Following is the high level workflow of the solution -

• User speaks to frontend (browser)
• Audio is translated into PCM and then sent to WebSocket to backend
• Backend is powered by ADK. The ASCII text (of the user input) is provided to LLMs
• LLMs provide textual response, then sent to WebSocket and sent to frontend
• Frontend translates into audio and then plays the audio on the browser

An important feature that is absolutely required in out workflow is the bidirectional mode - where we can provide end users with the experience of natural, human-like voice conversations, including the ability for the user to interrupt the agent's responses with voice commands. https://google.github.io/adk-docs/streaming/

Currently, I found only gemini-2.0-flash-exp LLM that works for our solution without throwing any errors except for the 429 error. Every other stable LLM, I tried gave me errors as shown above. So could you please suggest the alternative for gemini-2.0-flash-exp LLM , so that my workflow works without any errors and without 429 resource exhaustion errors.

[matt-buckley]

Also facing this issue where bidi-streaming is only supported on the quota-limited experimental 2.0 model. Whilst I understand bidi-streaming itself is still an experimental feature in ADK, it would be good to know when and how support for bidi-streaming will be extended to other models as it's currently very difficult to use and test bidi-streaming (even as an experimental feature) when quota limits are frequently being hit on the 2.0-flash-exp model.

[GrahamGGreig]

@TanejaAnkisetty for option 2, this sounds like the LiveAPI bug documented issue here. Likely because you are using the text modality. Streaming back audio instead of text to translate would probably help. However, I agree, the documentation is not great as option 1 does not work even though it is listed with Vertex AI. Also of note, the native audio models do not handle function calling well which is a real breaking issue given the announcement here that the half-cascade models will be depreciated on December 9th.

I personally have only had success with the half-cascade models available via the Google AI Studio:

Half-cascade audio: This option uses a cascaded model architecture (native audio input and text-to-speech output). It offers better performance and reliability in production environments, especially with tool use. Half-cascaded audio is supported by the following models:
gemini-live-2.5-flash-preview
gemini-2.0-flash-live-001

Which interestingly are recommended by the LiveAPI for production environments while ADK recommends the Vertex AI API. Unfortunately, as mentioned before, they are slated for depreciation in a little over a month. It would be great if ADK could provide a roadmap for the Bidi support as right now it is a challenge to develop on.

[hangfei]

@kazunori279 Could you share your tests and how your tests are setup ? This might help users to address those issues.

[kazunori279]

Hi, this is my test. Hope this helps:
Test setup: https://github.com/kazunori279/adk-streaming-test
The latest report: https://github.com/kazunori279/adk-streaming-test/blob/main/test_report.md

[GrahamGGreig]

Hey @hangfei and @kazunori279 this looks great, thanks for sending it over! However, I don't think it will help people hitting quota limits as this is single input single output tests. My experience is that most models work well for a few turns but then crash or have issues due to:

• Quota limits
• Calling a function and crashing (mentioned issue in LiveAPI documentation)
• When the context window becomes too large due to all audio being cached (i think?) on the gemini model because the parameter ContextWindowCompression is only available in LiveConnectConfig which requires an extension of the LlmAgent and BaseLlmFlow to set
• On refresh or from large input prompts if not using context window compression (see above) due to audio from some models still being saved to the DatabaseSessionService due to the TranscriptionManager not filtering for parametrized mimetypes (see my open issue here
• On refresh not having context due to issue I have mentioned in the link above (Output transcriptions are not saved to the DatabaseSessionService because they are no longer saved as context and re not a field in the database)
• Due to randomly being sent:

Truncated response body. Usually implies that the request timed out or the application exited before the response was finished.
by Gemini which seems to be an issue with a lot of models that Google has not addressed - months old thread here

I think we all understand that this is a bleeding edge, experimental function but some guidance from Google on where things are planning to go and how this can be made stable in the meantime would be extremely helpful.

[kazunori279]

@GrahamGGreig Hi, I recommend to find/open issues for those issues you shared and continue discussing on them. For the quota limits, I recommend to use Vertex AI Live API rather than Gemini Live API since the former can accept your quota increase request. I believe the latter can only take a few concurrent connections. See also: https://github.com/kazunori279/adk-streaming-guide/blob/main/docs/part4_run_config.md#session-limits-and-constraints

[hangfei]

Closing this as it's not related to ADK.

[matt-buckley]

@hangfei I believe this is related to ADK as ADK claims to (experimentally) support bidi-streaming, yet in practice only supports it via one soon-to-be-deprecated model. If a replacement will not be forthcoming then the claim to support bidi-streaming as a feature should be dropped as there will be no way to implement bidi-streaming via ADK if no models support it.

[kazunori279]

@matt-buckley actually ADK Bidi-streaming is designed as transparent as possible in terms of supporting the Live API models (please check the source code). It supports any models that are officially supported by either Gemini Live API or Vertex AI Live API as my latest test result shows. So, the current status only supports it via one soon-to-be-deprecated model is the status of Gemini Live API and Vertex AI Live API themselves. One those APIs release any GA models, they will be automatically available via ADK Bidi-streaming too.