feat(google): Native grounding, builtin tools, Gemini tools & MCP support

[olivier-lacroix]

This PR refactors how Gemini-specific tools (like Vertex AI Search) are integrated into the pipeline, introducing a generic mechanism for native Gemini features.

• Unified Tool Detection: Added logic to pipelines/google/google_gemini.py that automatically detects and executes Open WebUI tools with a types.Tool return type hint. These tools are executed locally, and their output is passed directly to the Gemini API as a native tool. This architecture replaces specialized, hardcoded logic blocks with a uniform loop, making the pipeline cleaner and more extensible.
• Decoupled Vertex AI Search: Moved the Vertex AI Search grounding logic from a hardcoded block in the pipeline to a standalone tool in tools/vertex_ai_search.py.
• Improved grounding: Gemini UrlContext & GoogleSearch tools replace the OWUI builtin tools fetch_url and search_web. No need for a filter anymore.
• Builtin tool support: Builtin tools are now properly added, fixing [Bug]: Gemini pipeline does not recognize the built-in tools of OWUI #155.
• Native MCP Tool Support: Directly integrate Model Context Protocol (MCP) tool sessions with Gemini's native tool calling capabilities.
• Cleaner Valves: Relocated grounding configuration (like VERTEX_AI_RAG_STORE) to the respective tool valves, reducing clutter in the main Gemini pipeline.
• Updated Documentation: Updated docs/google-gemini-integration.md to reflect the new architecture and tool-based grounding configuration.

These rely on getting tools from __metadata__["tools"] and MCP clients from __metadata__["mcp_clients"] rather than __tools__, as __tools__ only contains user defined tools for now. This requires function calling to be set to native to work properly.

[olivier-lacroix]

@owndev what do you think of these changes? they are breaking changes, but hopefully for the better :).

Something I have not added to the doc: it does allow users to define tools that return a type.Tool, and they will automatically be added & handled by Gemini

[owndev]

When I enable native function calling, I don't get a response from the model. How did you get it working?

I don't see any errors or other anomalies in the logs.

I tried testing Microsoft Learn MCP, but it doesn't seem to work. Which tool did you use to test it?

[olivier-lacroix]

I'll take a look. I have tested it with Egnyte MCP.

[olivier-lacroix]

It seems to be working for me with Gemini 3 flash with the same MCP server. Maybe flash-lite is 'too small a model'?

[owndev]

It doesn't work for me with the Pro model either.

Do you have native function calling enabled, and is the tool you're using displayed? See this image:

For it to work correctly for me, I have to set function calling to default and explicitly activate the MCP tool in the chat.

[olivier-lacroix]

Ah so you definitely need to activate the tool explicitly in the chat, with native mode enabled. otherwise, the mcp session is not passed to the genai SDK

[owndev]

Okay, thanks for the info, I must have misunderstood something.

Are you also experiencing the issue where, when native function calling is enabled and an MCP tool is active, you only receive a response if streaming is disabled? Because when streaming is enabled, I don't get a response at all.

[olivier-lacroix]

Yeah, I do have this sometimes. I had not realised it was linked to streaming. Is this linked to MCP tools you think?

An alternative, rather than passing the mcp session, would be to pass the individual mcp tools. on the pros side, this would honor the OWUI whitelist. on the cons side, it requires to add the right signature to the tools, as they come with no type hinting whatsoever. I had a working proto that did this. I thought it may be more brittle, but maybe genai handling of MCP is no that robust...

[olivier-lacroix]

@owndev I have switched the approach for MCP tools to adding them as regular callable tools, rather than passing the MCP session object. let me know if that works better for you?

[owndev]

It's still not working for me in native mode. But as long as it works in default mode, it doesn't matter to me.

Is it working for you? Could it be an MCP server issue or a Gemini problem, and not the pipeline itself?

[flueckiger]

Should we enforce google-genai>=1.68.0 in the requirements and apply the following configuration when combining built-in tools with function calling?

gen_config_params["tool_config"] = types.ToolConfig(
    include_server_side_tool_invocations=True
)

[olivier-lacroix]

Should we enforce google-genai>=1.68.0 in the requirements and apply the following configuration when combining built-in tools with function calling?

gen_config_params["tool_config"] = types.ToolConfig(

    include_server_side_tool_invocations=True

)

I don't see why @flueckiger ? All callable tools are executed client side

[flueckiger]

Should we enforce google-genai>=1.68.0 in the requirements and apply the following configuration when combining built-in tools with function calling?

gen_config_params["tool_config"] = types.ToolConfig(

    include_server_side_tool_invocations=True

)

I don't see why @flueckiger ? All callable tools are executed client side

I encountered the following error when running gemini-3.1-pro-preview with 'Web Search' (built-in) and 'Code Interpreter' (function calling) enabled:

Error during streaming: 400 Bad Request. {'message': '{\n "error": {\n "code": 400,\n "message": "Please enable tool_config.include_server_side_tool_invocations to use Built-in tools with Function calling.",\n "status": "INVALID_ARGUMENT"\n }\n}\n', 'status': 'Bad Request'}