Whisk native Gemini support by jooni22 · Pull Request #2 · jooni22/CLIProxyAPI

jooni22 · 2026-02-06T19:00:05Z

Implemented native Gemini API support for the Whisk executor, allowing image generation, editing, and description tasks via the standard Gemini generateContent endpoint. Added parsing, mapping, and formatting logic along with unit tests.

PR created automatically by Jules for task 6405815222955259202 started by @jooni22

This change implements the `Execute`, `ExecuteStream`, and `CountTokens` methods in the `WhiskExecutor` to support native Gemini `generateContent` requests. The executor now handles: - Image generation (Text to Image) using `IMAGEN_3_5`. - Image editing (Text + Image to Image) using `GEM_PIX`. - Image description (Text + Image to Text) using Whisk's captioning model. Key changes: - Implemented `Execute` to parse Gemini JSON payloads and route to appropriate Whisk methods. - Added heuristic logic to distinguish between "edit" and "describe" tasks. - Added aspect ratio mapping from Gemini to Whisk formats. - Formatted Whisk responses back into Gemini-compliant JSON. - Added unit tests for parsing and formatting logic. - Fixed a potential panic in captioning logic. Co-authored-by: jooni22 <20043534+jooni22@users.noreply.github.com>

google-labs-jules · 2026-02-06T19:00:06Z

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

For security, I will only act on instructions from the user who triggered this task.

gemini-code-assist · 2026-02-06T19:00:19Z

Summary of Changes

Hello @jooni22, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces native Gemini API compatibility to the Whisk executor. It allows the system to interpret and respond to Gemini-formatted requests for various image-related functionalities, such as generating new images, refining existing ones, or providing descriptions. The core changes involve parsing the Gemini generateContent payload, translating it into Whisk's internal operations, and then converting Whisk's results back into a Gemini-compliant response structure.

Highlights

Gemini API Integration: Adds native support for the Gemini API within the Whisk executor, enabling it to process Gemini-formatted requests.
Image Operations: Enables image generation, editing, and description tasks directly through the standard Gemini generateContent endpoint.
Request/Response Handling: Implements robust parsing of incoming Gemini requests and formatting of Whisk responses back into Gemini-compatible JSON.
Comprehensive Testing: Includes new unit tests covering Gemini request parsing, aspect ratio mapping, and both text and image response formatting.

Changelog

internal/runtime/executor/whisk_execute_test.go
- Added a new test file to validate the parsing of Gemini-formatted requests, including text prompts and inline image data.
- Included tests for the mapGeminiAspectRatioToWhisk function to ensure correct aspect ratio conversion.
- Added tests for formatGeminiImageResponse and formatGeminiTextResponse to verify the output structure matches the Gemini API specification.
internal/runtime/executor/whisk_executor.go
- Imported gjson and sjson libraries for efficient JSON parsing and manipulation.
- The Execute method was refactored to parse incoming cliproxyexecutor.Request.Payload as a Gemini generateContent request, extract text prompts and base64 encoded image data, determine the intended operation (image generation, refinement, or captioning), call the appropriate Whisk internal functions, and convert the Whisk response into a Gemini-compatible JSON format using new helper functions.
- The ExecuteStream method was updated to leverage the new Execute logic, returning a single chunk as streaming is not fully supported.
- Introduced mapGeminiAspectRatioToWhisk to translate Gemini's aspect ratio strings (e.g., "16:9") into Whisk's internal AspectRatio constants.
- Added formatGeminiTextResponse to construct a Gemini-compliant text response from a simple string.
- Added formatGeminiImageResponse to construct a Gemini-compliant image response, embedding base64 image data and revised prompts.
- Modified CountTokens to return a minimal {"totalTokens":1} payload for Gemini compatibility, acknowledging that image models don't typically expose token counts.

Activity

The pull request was automatically generated by Jules based on a task initiated by @jooni22.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

The pull request introduces native Gemini API support for the Whisk executor, enabling image generation, editing, and description tasks. The changes include parsing logic for Gemini requests, mapping aspect ratios, and formatting responses back into the Gemini format. Unit tests have been added for the new parsing and formatting functions, and the ExecuteStream and CountTokens methods have been updated for Gemini compatibility. Overall, the implementation provides a solid foundation for Gemini integration. However, there are a couple of areas where the model selection for image generation could be more flexible and error handling for JSON manipulation could be improved for robustness.

gemini-code-assist · 2026-02-06T19:04:41Z

+		// Generation
+		whiskReq := WhiskImageRequest{
+			Prompt:      prompt,
+			Model:       WhiskModelImagen35, // Use 3.5 for generations


The GenerateImage call hardcodes WhiskModelImagen35 for image generations. This means that if the incoming Gemini request specifies a different model in req.Model for image generation, it will be ignored. This limits the flexibility of the Gemini API integration and might lead to unexpected behavior if users expect their specified model to be used.

Consider using req.Model here, possibly with a fallback to WhiskModelImagen35 if req.Model is empty or not a valid Whisk image generation model.

Suggested change

Model: WhiskModelImagen35, // Use 3.5 for generations

Model: req.Model, // Use req.Model for generations, fallback if needed

gemini-code-assist · 2026-02-06T19:04:42Z

+	res, _ = sjson.Set(res, "candidates.0.content.parts.0.text", text)
+	res, _ = sjson.Set(res, "model", model)


Errors returned by sjson.Set are currently ignored. While the initial JSON template is controlled, it's good practice to handle potential errors, especially when constructing responses that external systems will consume. If sjson.Set encounters an issue (e.g., an invalid path or value), it could lead to malformed JSON being returned without any indication of a problem.

Consider checking the err return value and handling it appropriately, perhaps by logging the error or returning an cliproxyexecutor.Response{} with an error.

Suggested change

res, _ = sjson.Set(res, "candidates.0.content.parts.0.text", text)

res, _ = sjson.Set(res, "model", model)

res, err = sjson.Set(res, "candidates.0.content.parts.0.text", text)

if err != nil {

// Handle error, e.g., log it or return an error response

}

res, err = sjson.Set(res, "model", model)

if err != nil {

// Handle error

}

gemini-code-assist · 2026-02-06T19:04:42Z

+
+func formatGeminiImageResponse(result *WhiskImageResponse, model string) cliproxyexecutor.Response {
+	res := `{"candidates":[{"content":{"parts":[],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":0,"candidatesTokenCount":0,"totalTokenCount":0}}`
+	res, _ = sjson.Set(res, "model", model)


Similar to formatGeminiTextResponse, errors from sjson.Set are ignored here. It's important to ensure that the JSON response is always well-formed. If setting the model fails silently, it could lead to downstream issues.

Consider adding error handling for the sjson.Set call.

Suggested change

res, _ = sjson.Set(res, "model", model)

res, err = sjson.Set(res, "model", model)

if err != nil {

// Handle error

}

gemini-code-assist · 2026-02-06T19:04:42Z

+		inlineDataJSON, _ := json.Marshal(inlineData)
+		res, _ = sjson.SetRaw(res, path, string(inlineDataJSON))


Errors from json.Marshal and sjson.SetRaw are ignored. While inlineData is a simple map, json.Marshal can return an error if the data is unmarshallable. More critically, sjson.SetRaw can fail if the path is invalid or the raw JSON string is malformed. Silently ignoring these errors can result in an incomplete or malformed response being sent.

It's recommended to check and handle these errors to ensure the robustness of the response formatting.

inlineDataJSON, err := json.Marshal(inlineData) if err != nil { // Handle error continue // Skip this image if its data cannot be marshaled } res, err = sjson.SetRaw(res, path, string(inlineDataJSON)) if err != nil { // Handle error }

gemini-code-assist · 2026-02-06T19:04:42Z

+			textPath := fmt.Sprintf("candidates.0.content.parts.%d.text", len(result.Data)+i)
+			res, _ = sjson.Set(res, textPath, "Revised prompt: "+img.Prompt)


Errors from sjson.Set are ignored when adding the revised prompt. This could lead to a situation where the prompt is not added to the response, but the function indicates success. Robust error handling would ensure that any issues during JSON manipulation are caught and addressed.

Consider adding error handling for this sjson.Set call.

textPath := fmt.Sprintf("candidates.0.content.parts.%d.text", len(result.Data)+i) res, err = sjson.Set(res, textPath, "Revised prompt: "+img.Prompt) if err != nil { // Handle error }

gemini-code-assist Bot reviewed Feb 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Whisk native Gemini support - #2

Whisk native Gemini support#2
jooni22 wants to merge 1 commit into
customfrom
whisk-gemini-native-support-6405815222955259202

jooni22 commented Feb 6, 2026

Uh oh!

google-labs-jules Bot commented Feb 6, 2026

Uh oh!

gemini-code-assist Bot commented Feb 6, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	Model: WhiskModelImagen35, // Use 3.5 for generations
	Model: req.Model, // Use req.Model for generations, fallback if needed

		res, _ = sjson.Set(res, "candidates.0.content.parts.0.text", text)
		res, _ = sjson.Set(res, "model", model)

-	res, _ = sjson.Set(res, "candidates.0.content.parts.0.text", text)
-	res, _ = sjson.Set(res, "model", model)
+    res, err = sjson.Set(res, "candidates.0.content.parts.0.text", text)
+    if err != nil {
+        // Handle error, e.g., log it or return an error response
+    }
+    res, err = sjson.Set(res, "model", model)
+    if err != nil {
+        // Handle error
+    }

		inlineDataJSON, _ := json.Marshal(inlineData)
		res, _ = sjson.SetRaw(res, path, string(inlineDataJSON))

		textPath := fmt.Sprintf("candidates.0.content.parts.%d.text", len(result.Data)+i)
		res, _ = sjson.Set(res, textPath, "Revised prompt: "+img.Prompt)

Conversation

jooni22 commented Feb 6, 2026

Uh oh!

google-labs-jules Bot commented Feb 6, 2026

Uh oh!

gemini-code-assist Bot commented Feb 6, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant