Add feature-gated standalone image generation extension by won-openai · Pull Request #24723 · openai/codex

won-openai · 2026-05-27T07:47:11Z

Why

Add a standalone image generation path that can be exercised independently of hosted Responses image generation, while retaining the hosted tool as fallback unless the extension is actually available to the model.

What changed

Added the codex-image-generation-extension crate with standalone generate/edit execution, prior-image selection for edits, model-visible image output, and local generated-image persistence.
Installed the extension in app-server behind the disabled-by-default imagegenext feature and backend eligibility checks.
Updated core tool planning so eligible image_gen.imagegen exposure replaces hosted image_generation, while unavailable configurations retain hosted fallback.
Added coverage for extension behavior, edit history reuse, feature gating, auth eligibility, and hosted-tool replacement.
The extension is installed through app-server only in this PR; other execution paths retain hosted image generation because hosted replacement occurs only when the standalone executor is actually registered and model-visible.
The initial extension contract intentionally fixes the image model to gpt-image-2 and uses automatic image parameters.
Native generated-image history/card parity and rollout persistence cleanup are intentionally deferred follow-up work.

Validation

just test -p codex-image-generation-extension
just test -p codex-features
just test -p codex-core hosted_tools_follow_provider_auth_model_and_config_gates
just test -p codex-app-server
just fix -p codex-image-generation-extension -p codex-features -p codex-core -p codex-app-server
just fmt
just bazel-lock-update
just bazel-lock-check

won-openai · 2026-05-27T07:49:33Z

@@ -0,0 +1,11 @@
+The `image_gen.imagegen` tool enables image generation from descriptions and editing of existing images based on specific instructions. Use it when:


placeholder, subject to change

Honestly not sure we need a doc file then... just drop it

placeholder as in we need to change the guidelines later depending on how we want the model to use this image gen tool, we should still give it guidelines on generate and edit etc bc the model isn't trained to do it

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce9ab1187e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-27T07:52:55Z

+fn standalone_image_generation_model_visible(turn_context: &TurnContext) -> bool {
+    image_generation_runtime_enabled(turn_context)
+        && turn_context.features.get().enabled(Feature::ImageGenExt)
+        && namespace_tools_enabled(turn_context)


Gate imagegenext behind image_generation

When a user disables [features].image_generation but enables [features].imagegenext, this predicate still returns true for ChatGPT auth on an image-capable OpenAI model, so append_extension_tool_executors exposes image_gen.imagegen even though the stable image generation kill switch is off. The existing hosted path used image_generation_tool_enabled, so this regresses configurations that intentionally disable all model image-generation tools; include Feature::ImageGeneration in the standalone visibility check as well.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-05-27T07:52:55Z

+        ResponseInputItem::FunctionCallOutput {
+            call_id: call_id.to_string(),
+            output: FunctionCallOutputPayload {
+                body: FunctionCallOutputBody::ContentItems(content),


Surface standalone generated images to clients

When the standalone image tool succeeds, the image is only recorded as a FunctionCallOutput for the follow-up model request; app-server thread reconstruction ignores ResponseItem::FunctionCallOutput and only creates visible image-generation thread items from ImageGenerationBegin/End events. Since the new tool instructions tell the model to say nothing after generation, users in the app/TUI can get a successful generation with no visible image or saved-path item, so this path should emit/reuse image-generation events or otherwise publish the generated artifact to the client.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-05-27T07:52:55Z

+impl ToolOutput for GeneratedImageOutput {
+    /// Avoids copying image bytes into tool-call telemetry.
+    fn log_preview(&self) -> String {
+        "[generated image]".to_string()
+    }


Avoid piping generated images into hooks

When PostToolUse hooks are configured, the default CoreToolRuntime::post_tool_use_payload fallback serializes this output's to_response_item body because GeneratedImageOutput does not override post_tool_use_response; that includes the full data:image/png;base64,... payload for every generated image. Multi-MB image results can make hook stdin very large or expose image bytes to hooks despite the telemetry preview being redacted, so return a small hook-facing response (for example the saved path/hint) instead of letting the default fallback serialize the image content.

Useful? React with 👍 / 👎.

jif-oai · 2026-05-27T10:04:39Z

 use tracing::warn;

 const MULTI_AGENT_V2_NAMESPACE_DESCRIPTION: &str = "Tools for spawning and managing sub-agents.";
+const IMAGE_GEN_NAMESPACE: &str = "image_gen";


What is the plan to drop this? (and drop the old tool)

once the standalone path is ready to replace the hosted tool, I plan to remove the hosted tool and this replacement detection together

jif-oai · 2026-05-27T10:07:42Z

+    }
+
+    /// Returns generated bytes and persisted-artifact context for the model's follow-up response.
+    fn to_response_item(&self, call_id: &str, _payload: &ToolPayload) -> ResponseInputItem {


OOC do you also plan to emit events after this? FunctionCallOutput might not be rich enough? Not sure what is the product expecting

ya i am keeping this PR scoped to the standalone extension and fallback wiring, and will handle event emission in a follow-up PR

jif-oai · 2026-05-27T10:08:10Z

@@ -0,0 +1,11 @@
+The `image_gen.imagegen` tool enables image generation from descriptions and editing of existing images based on specific instructions. Use it when:


Honestly not sure we need a doc file then... just drop it

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d121f9377a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

jif-oai

Is there a test or an eval to prove equivalence with the original tool?

jif-oai · 2026-05-28T13:27:49Z

    .await;
    image_generation.assert_visible_contains(&["image_generation"]);

+    let extension_flag_without_imagegen_tool = probe(|turn| {


This only covers the fallback when imagegenext is on but the executor is absent. The replacement path itself is untested now

note: doing as follow-up / as the need comes to invest time in making sure we are at parity

jif-oai

Ok after my comment

jif-oai · 2026-05-28T18:00:20Z

+        if role != "user" {
+            return None;
+        }
+        let images = content


This reverses the user-visible attachment order before calling the edit API. If the user/model prompt says “edit the first image” after uploading [A, B], we now send [B, A], so positional references can target the wrong image. I guess this is a big deal but I'm not sure

draft

ce9ab11

won-openai requested a review from a team as a code owner May 27, 2026 07:47

won-openai commented May 27, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed May 27, 2026

View reviewed changes

jif-oai reviewed May 27, 2026

View reviewed changes

github-actions Bot mentioned this pull request May 28, 2026

📊 AI CLI 工具社区动态日报 2026-05-28 ivanweng2077/big_model_radar#101

Open

addressed comments

d121f93

chatgpt-codex-connector Bot reviewed May 28, 2026

View reviewed changes

Comment thread codex-rs/Cargo.toml

Comment thread codex-rs/ext/image-generation/src/tool.rs Outdated

won-openai and others added 2 commits May 27, 2026 20:48

nit

89b205c

Merge branch 'main' into imagegen_extension

7d514b0

jif-oai reviewed May 28, 2026

View reviewed changes

fix

f5f5311

jif-oai approved these changes May 28, 2026

View reviewed changes

fix

480b1db

won-openai merged commit ecb41fc into main May 28, 2026
46 checks passed

won-openai deleted the imagegen_extension branch May 28, 2026 18:44

github-actions Bot locked and limited conversation to collaborators May 28, 2026

		@@ -0,0 +1,11 @@
		The `image_gen.imagegen` tool enables image generation from descriptions and editing of existing images based on specific instructions. Use it when:

Conversation

won-openai commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Validation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

jif-oai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jif-oai left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

won-openai commented May 27, 2026 •

edited

Loading