-
Notifications
You must be signed in to change notification settings - Fork 1
Google GenAI: Live API (real-time multimodal streaming) not instrumented #125
Description
Gap
The Google GenAI Live API (google.genai.live) is not instrumented. This is Google's GA API for real-time, bidirectional multimodal interactions with Gemini models — supporting streaming text, audio, and video input/output over WebSocket connections.
The current Google GenAI wrapper (py/src/braintrust/wrappers/google_genai/__init__.py) instruments three surfaces:
| Method | Instrumented? |
|---|---|
models.generate_content() |
Yes |
models.generate_content_stream() |
Yes |
models.embed_content() |
Yes |
client.aio.live.connect() |
No |
live.send() / live.receive() |
No |
The wrap_client() function (line 50) is defined but is explicitly a no-op ("noop for now"), so Live API sessions created from a wrapped client are completely untraced.
What is missing
At minimum:
- Session-level span: A parent span created when
client.aio.live.connect()opens a session, capturing model config, session duration, and aggregate token usage. - Turn-level spans: Child spans for each send/receive turn within a session, capturing input content, output content, and per-turn token usage.
- Tool use within sessions: The Live API supports function calling mid-session; tool calls and results should be captured as nested spans.
Braintrust docs status
not_found — the Gemini integration page documents generate_content, streaming, function calling, structured outputs, thinking tokens, and context caching, but does not mention the Live API or real-time streaming.
Upstream sources
- Google GenAI Live API guide: https://ai.google.dev/gemini-api/docs/live
- Google GenAI Python SDK Live API: https://googleapis.github.io/python-genai/genai.html#genai.Client.aio (the
liveattribute) - Live API supports: streaming text, audio, video input; function calling; code execution; session-level context caching
Local files inspected
py/src/braintrust/wrappers/google_genai/__init__.py:setup_genai()(lines 60–145) — patchesModelsandAsyncModelsforgenerate_content,generate_content_stream, andembed_contentonlywrap_client()(line 50) — defined as a no-op- No references to
live,connect,LiveSession, orLiveConnectConfig
py/src/braintrust/wrappers/test_google_genai.py— no tests reference the Live APIpy/noxfile.py—test_google_genai(latest)session exists but Live API is not tested