Conversation
Part A: Bug Fix for Core OpenAI Converter - Fix tool choice logic that was forcing 'required' mode - Ensure models can respond with text before tool calls - Prevent Claude Code client from hanging on tool requests Part B: Native Gemini Provider Integration - Add GeminiClient for direct native API communication - Implement robust schema sanitization in GeminiConverter - Add automatic routing logic for Gemini endpoints - Support both OpenAI-compatible and native Gemini backends Key Changes: - internal/converter/gemini_converter.go: Robust schema reconstruction - internal/client/gemini.go: Native Gemini API client - internal/handlers/enhanced_messages.go: Gemini routing logic - docs/NATIVE_GEMINI_SETUP.md: User documentation This resolves critical hanging issues and enables reliable tool execution with Gemini backends through proper schema transformation.
🚀 Native Gemini Integration: - Add complete native Gemini API client with direct Google API communication - Implement robust Claude-to-Gemini request/response conversion - Add automatic provider detection and routing for Gemini endpoints - Support all Claude Code tools including web_search with proper schema sanitization - Enable streaming responses and comprehensive error handling 🔧 Code Quality Improvements: - Fix ineffective break statement in streaming context cancellation - Remove unused functions and parameters across multiple handlers - Modernize Go code with 'any' type and improved string operations - Convert if-else chains to switch statements for better readability - Add comprehensive debug logging for troubleshooting 📚 Documentation: - Add complete setup guide for native Gemini API configuration - Include usage examples for Claude Code integration and direct API calls - Document troubleshooting steps and performance benefits ✅ Backward Compatibility: - All changes are non-breaking and preserve existing functionality - OpenAI-compatible Gemini endpoint continues to work unchanged - Existing configurations and integrations remain fully functional Files changed: - internal/client/gemini.go: Enhanced native API client - internal/handlers/enhanced_messages.go: Fixed streaming and added Gemini routing - internal/converter/detector.go: Improved parameter handling - internal/handlers/config.go: Code quality improvements - docs/NATIVE_GEMINI_SETUP.md: Complete setup documentation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @Mike-37, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
I've implemented native Gemini API support in ccany and resolved a critical bug that caused the Claude Code client to hang during tool call operations. This significantly enhances ccany's capabilities as a multi-provider proxy, ensuring reliable tool execution and better performance with Gemini.
Highlights
- Native Gemini API Integration: ccany can now directly communicate with Google's native Gemini API, bypassing the unreliable OpenAI-compatible endpoint. This includes automatic provider detection based on endpoint URLs, full support for Claude Code tools (web_search, file ops), robust schema sanitization for complex Claude schemas, and real-time streaming support.
- Critical Bug Fix for Claude Client Hanging: Addressed an issue where the Claude Code client would hang during tool calls. The root cause was ccany incorrectly forcing tool_choice: "required", which prevented the model from returning initial text responses. The fix involves removing this forced behavior and ensuring text content is prioritized.
- New Core Components: Introduced internal/client/gemini.go for the native API client, internal/converter/gemini_converter.go for Claude-to-Gemini schema transformation (including a sanitizeSchema function), and internal/models/gemini.go for Gemini API structs.
- Enhanced Routing and Documentation: Updated internal/handlers/enhanced_messages.go to intelligently route requests to either the native Gemini or existing OpenAI path. A new setup guide, docs/NATIVE_GEMINI_SETUP.md, has been added for configuring native Gemini support.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request adds native Gemini API support, which is a great enhancement. The implementation includes a new Gemini client, request/response converters, and routing logic in the message handler. The code is well-structured, but I've identified a few areas for improvement. My main concerns are around the robustness of the Gemini endpoint detection, the use of a non-unique ID generator, placeholder converter functions, and some inconsistencies in documentation and hardcoded model data. Addressing these points will improve the reliability and maintainability of the new feature.
| - "gemini-1.5-flash" | ||
| - "gemini-1.5-pro" | ||
| - "gemini-2.0-flash-exp" |
There was a problem hiding this comment.
The Gemini model names listed in the documentation are inconsistent with other parts of the codebase and the PR description. For example, this file mentions gemini-1.5-flash and gemini-2.0-flash-exp, while other parts of the code reference gemini-2.5-flash. To avoid confusion, please ensure all model names are consistent across the documentation and the application.
| func (c *GeminiClient) GetModelInfo(ctx context.Context) (map[string]any, error) { | ||
| // This would typically call the models endpoint, but for now return static info | ||
| return map[string]any{ | ||
| "models": []string{ | ||
| "gemini-1.5-flash", | ||
| "gemini-1.5-flash-latest", | ||
| "gemini-1.5-pro", | ||
| "gemini-1.5-pro-latest", | ||
| "gemini-2.5-flash", | ||
| }, | ||
| "supports_tools": true, | ||
| "supports_streaming": true, | ||
| "supports_vision": true, | ||
| "max_tokens": 8192, | ||
| "context_window": 1000000, | ||
| }, nil | ||
| } |
There was a problem hiding this comment.
The GetModelInfo function returns a hardcoded map of model information. This can easily become outdated and is inconsistent with model names used elsewhere in the PR. To improve maintainability, this should be implemented to fetch the list of available models directly from the Gemini API's v1beta/models endpoint.
| // ConvertFromOpenAI converts OpenAI format request to Gemini format | ||
| func (c *GeminiConverter) ConvertFromOpenAI(openaiReq *models.OpenAIChatCompletionRequest) (*models.GeminiRequest, error) { | ||
| // This is a placeholder implementation - convert OpenAI to Claude first, then to Gemini | ||
| // For now, return a basic implementation | ||
| geminiReq := &models.GeminiRequest{ | ||
| Contents: []models.GeminiContent{}, | ||
| } | ||
| } | ||
|
|
||
| func (c *GeminiConverter) mapClaudeRoleToGemini(role string) string { | ||
| switch role { | ||
| case "user": | ||
| return "user" | ||
| case "assistant": | ||
| return "model" | ||
| default: | ||
| return "user" | ||
| } | ||
| } | ||
| // Convert OpenAI messages to Gemini format | ||
| for _, msg := range openaiReq.Messages { | ||
| role := "user" | ||
| if msg.Role == "assistant" { | ||
| role = "model" | ||
| } | ||
|
|
||
| func (c *GeminiConverter) mapGeminiFinishReasonToClaudeStopReason(finishReason string) string { | ||
| switch finishReason { | ||
| case "STOP": | ||
| return "end_turn" | ||
| case "MAX_TOKENS": | ||
| return "max_tokens" | ||
| case "SAFETY": | ||
| return "stop_sequence" | ||
| case "RECITATION": | ||
| return "stop_sequence" | ||
| default: | ||
| return "end_turn" | ||
| geminiReq.Contents = append(geminiReq.Contents, models.GeminiContent{ | ||
| Role: role, | ||
| Parts: []models.GeminiPart{{Text: msg.Content}}, | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func (c *GeminiConverter) extractTextFromParts(parts []models.GeminiPart) string { | ||
| var texts []string | ||
| for _, part := range parts { | ||
| if part.Text != "" { | ||
| texts = append(texts, part.Text) | ||
| } | ||
| // Handle generation config | ||
| config := &models.GeminiGenerationConfig{} | ||
| if openaiReq.Temperature != nil { | ||
| config.Temperature = openaiReq.Temperature | ||
| } | ||
| return strings.Join(texts, " ") | ||
| if openaiReq.MaxTokens != nil { | ||
| config.MaxOutputTokens = openaiReq.MaxTokens | ||
| } | ||
| geminiReq.GenerationConfig = config | ||
|
|
||
| return geminiReq, nil | ||
| } |
There was a problem hiding this comment.
The ConvertFromOpenAI function is a placeholder and does not seem to handle all features, such as tool calls. This could lead to unexpected behavior if this conversion path is used. If this conversion is intended to be supported, it should be fully implemented to match the capabilities of the Claude-to-Gemini conversion.
| // generateRandomID generates a random ID for Claude responses | ||
| func generateRandomID() string { | ||
| return fmt.Sprintf("%d", time.Now().UnixNano()) | ||
| } |
There was a problem hiding this comment.
The generateRandomID function uses time.Now().UnixNano(), which is not guaranteed to be unique, especially in high-concurrency scenarios. This could lead to ID collisions for Claude responses. It's recommended to use a more robust method for generating unique IDs, such as a UUID. The github.com/google/uuid package is already used elsewhere in the project and would be a good choice here. You will need to add the import for it.
| // generateRandomID generates a random ID for Claude responses | |
| func generateRandomID() string { | |
| return fmt.Sprintf("%d", time.Now().UnixNano()) | |
| } | |
| // generateRandomID generates a random ID for Claude responses | |
| func generateRandomID() string { | |
| return uuid.New().String() | |
| } |
| // isGeminiEndpoint checks if the base URL is a Gemini endpoint | ||
| func (h *EnhancedMessagesHandler) isGeminiEndpoint(baseURL string) bool { | ||
| return strings.Contains(baseURL, "generativelanguage.googleapis.com") | ||
| } |
There was a problem hiding this comment.
The isGeminiEndpoint function uses strings.Contains to detect a Gemini URL. This can lead to false positives if the Gemini URL string appears in a query parameter or other part of the URL. For more robust detection, it's better to parse the URL and check the host directly. You will need to add an import for net/url.
// isGeminiEndpoint checks if the base URL is a Gemini endpoint
func (h *EnhancedMessagesHandler) isGeminiEndpoint(baseURL string) bool {
u, err := url.Parse(baseURL)
if err != nil {
h.logger.WithError(err).Warnf("Could not parse URL to check for Gemini endpoint: %s", baseURL)
return strings.Contains(baseURL, "generativelanguage.googleapis.com")
}
return u.Host == "generativelanguage.googleapis.com"
}
|
thanks for your pr @Mike-37 |
2 participants
Native Gemini API Support
Had to make this as gemini doesn't seem to work on OpenAI compatible API
Summary
Adds native Gemini API integration to ccany, enabling direct transformation to Google's Gemini API.
Key Features
Native Gemini Integration
Core Changes
Gemini-Specific Files
internal/client/gemini.go- Enhanced native API client for direct Google API communicationinternal/handlers/enhanced_messages.go- Added Gemini provider detection and routing + fixed critical streaming context cancellation bugdocs/NATIVE_GEMINI_SETUP.md- Complete setup and usage guide for native Gemini APIAdditional Improvements
Testing Results
Configuration
Usage Example