Skip to content

Expose local image paths to models#25944

Merged
won-openai merged 7 commits into
mainfrom
attach_image_paths
Jun 3, 2026
Merged

Expose local image paths to models#25944
won-openai merged 7 commits into
mainfrom
attach_image_paths

Conversation

@won-openai

@won-openai won-openai commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

Why

Local image attachments include image bytes, but the adjacent model-visible label omits the source path. Exposing the path lets model-selected workflows refer back to the intended local image explicitly.

What changed

  • Include an escaped path attribute in model-visible local image opening tags.
  • Reuse the path-aware marker generator in rollout coverage.
  • Update protocol, replay, and rollout coverage for the new request shape.

Validation

  • just fmt
  • just test -p codex-protocol
  • just test -p codex-core skips_local_image_label_text
  • just test -p codex-core copy_paste_local_image_persists_rollout_request_shape
  • git diff --check

@won-openai won-openai requested a review from a team as a code owner June 2, 2026 21:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bd82b8913

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/protocol/src/models.rs Outdated
content: vec![
ContentItem::InputText {
text: codex_protocol::models::local_image_open_tag_text(/*label_number*/ 1),
text: format!(r#"<image name=[Image #1] path="{}">"#, abs_path.display()),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't you use your new method?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the function is set to private, we can set to public and use it here

Comment thread codex-rs/protocol/src/models.rs Outdated
let path = path
.display()
.to_string()
.replace('&', "&amp;")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need this escaping

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd92f422aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

let label = local_image_label_text(label_number);
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label}{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")
let path = path.display();
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label} path=\"{path}\"{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 Badge Bound local image paths before injecting them

For local images under very long absolute paths, this writes the entire path into a new model-visible InputText item, and the string has no hard cap before it becomes part of the request context. The model-context review rule requires every injected item to be bounded and flags new individual items that can exceed 1k tokens as P0; see AGENTS.md lines 87-90. Please truncate or otherwise cap only the displayed path while keeping the full path in non-model state.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extremely unlikely for the path (codex home + thread_id + call_id) to get to any level that is concernin

let label = local_image_label_text(label_number);
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label}{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")
let path = path.display();
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label} path=\"{path}\"{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Escape path text before embedding it in the tag

When the local image filename contains characters such as " or > that are valid on Unix paths, this emits a malformed model-visible tag and lets part of the path be interpreted as ordinary prompt text rather than the path attribute. For example, the added test path /tmp/a&"<b>.png produces path="/tmp/a&"<b>.png", so please escape or encode attribute delimiters before building this wrapper.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the local image filename will not ocntain these characters..

@won-openai won-openai enabled auto-merge (squash) June 3, 2026 19:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d77965f07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

let label = local_image_label_text(label_number);
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label}{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")
let path = path.display();
format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label} path=\"{path}\"{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 Badge Cap local image path text before injecting it

This sends the full Path::display() string into a model-visible InputText without any hard cap. The context rule requires bounded injected context and calls out any new item that can exceed 1k tokens as P0 (AGENTS.md lines 87-90); fresh evidence since the earlier thread is that UserInput::LocalImage also comes from CLI/exec/app-server user-supplied paths, not only generated Codex home/thread/call paths, so a valid long path can still inflate every replayed request and churn cache. Please truncate or otherwise bound the displayed path while retaining the full path only in non-model state.

Useful? React with 👍 / 👎.

@won-openai won-openai merged commit 57ab4c8 into main Jun 3, 2026
46 of 47 checks passed
@won-openai won-openai deleted the attach_image_paths branch June 3, 2026 19:50
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 3, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants