Skip to content

Google GenAI: Live API (real-time multimodal streaming) not instrumented #125

@braintrust-bot

Description

@braintrust-bot

Gap

The Google GenAI Live API (google.genai.live) is not instrumented. This is Google's GA API for real-time, bidirectional multimodal interactions with Gemini models — supporting streaming text, audio, and video input/output over WebSocket connections.

The current Google GenAI wrapper (py/src/braintrust/wrappers/google_genai/__init__.py) instruments three surfaces:

Method Instrumented?
models.generate_content() Yes
models.generate_content_stream() Yes
models.embed_content() Yes
client.aio.live.connect() No
live.send() / live.receive() No

The wrap_client() function (line 50) is defined but is explicitly a no-op ("noop for now"), so Live API sessions created from a wrapped client are completely untraced.

What is missing

At minimum:

  • Session-level span: A parent span created when client.aio.live.connect() opens a session, capturing model config, session duration, and aggregate token usage.
  • Turn-level spans: Child spans for each send/receive turn within a session, capturing input content, output content, and per-turn token usage.
  • Tool use within sessions: The Live API supports function calling mid-session; tool calls and results should be captured as nested spans.

Braintrust docs status

not_found — the Gemini integration page documents generate_content, streaming, function calling, structured outputs, thinking tokens, and context caching, but does not mention the Live API or real-time streaming.

Upstream sources

Local files inspected

  • py/src/braintrust/wrappers/google_genai/__init__.py:
    • setup_genai() (lines 60–145) — patches Models and AsyncModels for generate_content, generate_content_stream, and embed_content only
    • wrap_client() (line 50) — defined as a no-op
    • No references to live, connect, LiveSession, or LiveConnectConfig
  • py/src/braintrust/wrappers/test_google_genai.py — no tests reference the Live API
  • py/noxfile.pytest_google_genai(latest) session exists but Live API is not tested

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions