Problem or Use Case
Currently, users wanting to run Gemini models (like gemini-3.1-pro-preview) natively must either use a middleman service or attempt to use Google's OpenAI-compatible endpoint. However, routing through the OpenAI compatibility layer often causes fragile tool calling, dropped streaming tokens, or agent crashes. This happens because of schema translation mismatches between Hermes's strict OpenAI-style loops and Google's actual backend expectations. The use case is to allow developers to drop in a GOOGLE_API_KEY and run Gemini 3.1 Pro natively with 100% stable multi-turn tool calling, uninterrupted streaming, and full context support without relying on fragile wrappers.
Proposed Solution
Implement a native provider: "google" class in Hermes Agent that communicates directly via the Google GenAI API, bypassing the OpenAI translation layer entirely.
Key technical requirements include:
• Native Schema Translation: Programmatically map Hermes's standard JSON schema for tools into Google's specific FunctionDeclaration format, and correctly intercept functionCall payloads to return them cleanly to the sandbox as functionResponse arrays.
• Message Normalization: Google's API enforces strict role alternation (user, model) in the contents array. The new provider must include a normalizer to merge adjacent messages of the same role before transmitting the payload to prevent 400 Bad Request errors.
• System Prompt Extraction: Hermes's system prompt must be cleanly separated from the main message array and mapped specifically to Google's systemInstruction parameter.
• Streaming Parity: Utilize Google's native streaming endpoint (stream=True) to properly yield text chunks to the Hermes TUI/WebUI while correctly pausing to buffer and parse tool argument streams.
Alternatives Considered
OpenRouter: Works stably for Gemini tool calling right now, but adds a third-party dependency, potential latency, and requires users to manage a separate OpenRouter account instead of using their direct Google API keys.
• LiteLLM Local Proxy: Requires users to run and configure a separate local server to handle schema translation, adding unnecessary overhead and complexity to the basic setup process.
• Google's OpenAI Compatibility Endpoint: Currently too unstable for heavy multi-turn tool calling, often resulting in silent failures or dropped context during autonomous agent loops.
Feature Type
CLI improvement
Scope
Medium (few files, < 300 lines)
Contribution
Problem or Use Case
Currently, users wanting to run Gemini models (like gemini-3.1-pro-preview) natively must either use a middleman service or attempt to use Google's OpenAI-compatible endpoint. However, routing through the OpenAI compatibility layer often causes fragile tool calling, dropped streaming tokens, or agent crashes. This happens because of schema translation mismatches between Hermes's strict OpenAI-style loops and Google's actual backend expectations. The use case is to allow developers to drop in a GOOGLE_API_KEY and run Gemini 3.1 Pro natively with 100% stable multi-turn tool calling, uninterrupted streaming, and full context support without relying on fragile wrappers.
Proposed Solution
Implement a native provider: "google" class in Hermes Agent that communicates directly via the Google GenAI API, bypassing the OpenAI translation layer entirely.
Key technical requirements include:
• Native Schema Translation: Programmatically map Hermes's standard JSON schema for tools into Google's specific FunctionDeclaration format, and correctly intercept functionCall payloads to return them cleanly to the sandbox as functionResponse arrays.
• Message Normalization: Google's API enforces strict role alternation (user, model) in the contents array. The new provider must include a normalizer to merge adjacent messages of the same role before transmitting the payload to prevent 400 Bad Request errors.
• System Prompt Extraction: Hermes's system prompt must be cleanly separated from the main message array and mapped specifically to Google's systemInstruction parameter.
• Streaming Parity: Utilize Google's native streaming endpoint (stream=True) to properly yield text chunks to the Hermes TUI/WebUI while correctly pausing to buffer and parse tool argument streams.
Alternatives Considered
OpenRouter: Works stably for Gemini tool calling right now, but adds a third-party dependency, potential latency, and requires users to manage a separate OpenRouter account instead of using their direct Google API keys.
• LiteLLM Local Proxy: Requires users to run and configure a separate local server to handle schema translation, adding unnecessary overhead and complexity to the basic setup process.
• Google's OpenAI Compatibility Endpoint: Currently too unstable for heavy multi-turn tool calling, often resulting in silent failures or dropped context during autonomous agent loops.
Feature Type
CLI improvement
Scope
Medium (few files, < 300 lines)
Contribution