Skip to content

Restore legacy image detail values#24644

Merged
rhan-oai merged 1 commit into
mainfrom
rhan/image-detail-backwards-compatbility
May 26, 2026
Merged

Restore legacy image detail values#24644
rhan-oai merged 1 commit into
mainfrom
rhan/image-detail-backwards-compatbility

Conversation

@rhan-oai
Copy link
Copy Markdown
Collaborator

@rhan-oai rhan-oai commented May 26, 2026

Why

Older persisted rollouts can contain input_image.detail values of auto or low from before ImageDetail was narrowed to high/original. Current deserialization rejects those values, which can make resume skip later compacted checkpoints and reconstruct an oversized raw suffix before the next compaction attempt.

Confirmed Sentry reports fixed by this compatibility path:

Background

openai/codex#20693 added image-detail plumbing for app-server UserInput so input images could explicitly request detail: original. The Slack discussion behind that PR was about ScreenSpot / bridge evals where user input images were resized, while tool output images already had MCP/code-mode ways to request image detail.

In review, the intended new API surface was narrowed to high and original: default to high, allow original when callers need unchanged image handling, and avoid encouraging new auto or low usage. That policy still makes sense for newly emitted values.

The missing compatibility piece is persisted history. Older rollouts can already contain auto and low, and resume reconstructs typed history by deserializing those rollout records. Rejecting old values at that boundary causes valid compacted checkpoints to be skipped. This PR restores auto and low as real variants so old records deserialize and round-trip without being rewritten as high, while product paths can continue to default to high and avoid emitting auto for new behavior.

What changed

  • Restored ImageDetail::Auto and ImageDetail::Low as first-class protocol values.
  • Preserved auto/low through rollout deserialization, MCP image metadata, code-mode image output, and schema/type generation.
  • Kept local image byte handling conservative: only original switches to original-resolution loading; auto/low/high continue through the resize-to-fit path while retaining their detail value.
  • Added regression coverage for enum round-tripping and code-mode low detail handling.

Testing

  • just write-app-server-schema
  • just test -p codex-protocol
  • just test -p codex-tools
  • just test -p codex-code-mode
  • just test -p codex-app-server-protocol
  • just test -p codex-core suite::rmcp_client::stdio_image_responses_preserve_original_detail_metadata
  • just test -p codex-core suite::code_mode::code_mode_can_use_mcp_image_result_with_image_helper
  • Loaded broken rollouts on local fixed builds, and started/completed new turns.

I also attempted just test -p codex-core; the local broad run did not finish green: 2559 tests run, 2467 passed, 55 flaky, 91 failed, 1 timed out. The failures were broad timeout/deadline failures across unrelated areas; targeted changed-path core tests above passed.

@rhan-oai rhan-oai changed the title Accept legacy image detail values Restore legacy image detail values May 26, 2026
@rhan-oai rhan-oai force-pushed the rhan/image-detail-backwards-compatbility branch from 6fd663d to 28bff79 Compare May 26, 2026 22:18
@rhan-oai rhan-oai requested a review from a team as a code owner May 26, 2026 22:18
@rhan-oai rhan-oai merged commit dc4e54d into main May 26, 2026
31 checks passed
@rhan-oai rhan-oai deleted the rhan/image-detail-backwards-compatbility branch May 26, 2026 23:24
@github-actions github-actions Bot locked and limited conversation to collaborators May 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants