generate_reply times out on first dialogue with gemini-3.1-flash-live-preview — works fine with gemini-2.5-flash-

[acharan-tech-200037]

Bug Description

When using gemini-3.1-flash-live-preview as the realtime model, the agent fails to generate the first reply immediately after joining the room. It times out waiting for the generation_created event. The exact same code with gemini-2.5-flash works without any issues.

Expected Behavior

Agent should generate and speak the first reply immediately after joining the room, same as it does with gemini-2.5-flash

Reproduction Steps

1.Create an AgentSession with google.realtime.RealtimeModel using model gemini-3.1-flash-live-preview
2.Start the session with session.start()
3.Call session.generate_reply() for the opening dialogue

Minimal Reproducible Example

from livekit import agents
from livekit.plugins import google, silero
from livekit.agents import AgentSession

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: agents.JobContext):
    session = AgentSession(
        llm=google.realtime.RealtimeModel(
            model="gemini-3.1-flash-live-preview",  # ❌ fails
            voice="Puck",
        ),
        vad=silero.VAD.load(),
    )

    await session.start(ctx.room, participant=ctx.room.remote_participants)
    await session.generate_reply(
        instructions="Greet the user."
    )

Operating System

Windows11

Models Used

gemini-3.1-flash-live-preview

Package Versions

livekit-agents==1.5.1

Logs

ERROR… livekit.…ns.google  error in receive task: 1007 None.
                             Request contains an invalid argument.

websockets.exceptions.ConnectionClosedError:
  received 1007 (invalid frame payload data) Request contains an invalid argument.;
  then sent 1007 (invalid frame payload data) Request contains an invalid argument.

google.genai.errors.APIError: 1007 None. Request contains an invalid argument.

ERROR… livekit.agents  failed to generate a reply:
                        generate_reply timed out waiting for
                        generation_created event.

WARNI… livekit.…ns.google  received server content but no active generation.

[oEdyb]

Root cause analysis

Confirmed on livekit-plugins-google==1.5.1 + google-genai==1.68.0.

What's happening

RealtimeSession.generate_reply() (line 715-716 in realtime_api.py) uses LiveClientContent (i.e. send_client_content) to inject a placeholder "." user turn:

turns.append(types.Content(parts=[types.Part(text=".")], role="user"))
self._send_client_event(types.LiveClientContent(turns=turns, turn_complete=True))

Gemini 3.1 Flash Live rejects all send_client_content calls during a session with 1007 invalid argument. Per Google's migration guide, send_client_content is now only for seeding initial context history (with initial_history_in_client_content set). Text during conversation must use send_realtime_input.

Reproduction (minimal, no LiveKit needed)

import asyncio
from google.genai import Client, types

async def main():
    client = Client(api_key="YOUR_KEY")
    config = types.LiveConnectConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Kore")
            )
        ),
    )
    async with client.aio.live.connect(model="gemini-3.1-flash-live-preview", config=config) as session:
        # ❌ This fails with 1007:
        await session.send_client_content(
            turns=[types.Content(parts=[types.Part(text="Hello")], role="user")],
            turn_complete=True,
        )
        # ✅ This works:
        # await session.send_realtime_input(text="Hello")
        response = await asyncio.wait_for(session._receive(), timeout=5.0)
        print(response)

asyncio.run(main())

What needs to change in the plugin

generate_reply() should use LiveClientRealtimeInput(text=...) instead of LiveClientContent when connected to a 3.1+ model. The initial chat context seeding in _main_task (lines 820-825) may also need updating per the initial_history_in_client_content flag.

The monkey-patch in #5234 (comment by @smittie2000) works as a temporary workaround.

[davidzhao]

generate_reply makes use of send_client_content to create a generation with specific model instructions.

it seems that 3.1 isn't supporting this mode very well.

[davidzhao]

generate_reply() should use LiveClientRealtimeInput(text=...) instead of LiveClientContent

LiveClientRealtimeInput isn't the same thing. It represents input coming from the user, where as we need to add a piece of context that's coming from the assistant/model