Skip to content

fix(gemini live): use parameters instead of parameters_json_schema for raw schema function tools#5560

Merged
longcw merged 1 commit intomainfrom
longc/fix-raw-schema-gemini-live
Apr 28, 2026
Merged

fix(gemini live): use parameters instead of parameters_json_schema for raw schema function tools#5560
longcw merged 1 commit intomainfrom
longc/fix-raw-schema-gemini-live

Conversation

@longcw
Copy link
Copy Markdown
Contributor

@longcw longcw commented Apr 27, 2026

Summary

  • Gemini Live doesn't accept parameters_json_schema on FunctionDeclaration, so this PR switches the realtime session to the parameters field (simplified via _GeminiJsonSchema).
  • The regular Gemini LLM keeps using parameters_json_schema to preserve the richer JSON Schema features it supports.

Closes #5450

Ref: googleapis/python-genai#1147

…r raw schema function tools

Gemini Live doesn't support parameters_json_schema, so raw schema
function tools were sent without parameters and the model could not
call them properly. Switch the realtime session to the parameters
field, while keeping parameters_json_schema for the regular Gemini LLM.

Closes: #5450
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

@longcw longcw merged commit 8a3293c into main Apr 28, 2026
26 checks passed
@longcw longcw deleted the longc/fix-raw-schema-gemini-live branch April 28, 2026 01:36
Copy link
Copy Markdown
Contributor

This is an automated Claude Code Routine created by @toubatbrian. Right now it is in experimentation stage. The automation will start porting this PR into agents-js automatically.

Heuristic classification: this PR touches the Google plugin's Gemini Live realtime integration (plugin work, not Python-only deps), so it is in scope for porting. A draft PR will be opened against livekit/agents-js shortly.


Generated by Claude Code

Copy link
Copy Markdown
Contributor

Update from the automated routine: after inspecting livekit/agents-js, no port is being opened for this PR.

Reasoning:

  • This fix targets raw-schema function tools (info.raw_schema -> FunctionDeclaration) being routed through parameters_json_schema, which Gemini Live rejects. agents-js has no rawSchema / raw-schema tool concept yet — function tools always go through llm.toJsonSchema(tool.parameters, false).
  • The JS Google plugin's toFunctionDeclarations (plugins/google/src/utils.ts) already emits the simplified parameters field via convertJSONSchemaToOpenAPISchema(...), which is the same destination Python's Gemini Live is now switching to. Both regular LLM and RealtimeModel in agents-js consume that helper, so Gemini Live in JS was never affected by the parameters_json_schema regression this PR fixes.

If a future PR introduces raw-schema tools to agents-js (or splits the regular Gemini LLM onto parameters_json_schema for richer JSON Schema features), this same special-case will need to be ported then. For now there's nothing actionable on the JS side.

cc @toubatbrian


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants