feat(v1): support image-bearing Messages prompts#1974
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 57c6e4c74a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
xeophon
force-pushed
the
harness-image-prompts
branch
from
57c6e4c to
b463059
Compare
xeophon
changed the title
xeophon
force-pushed
the
harness-image-prompts
branch
from
b463059 to
72492f9
Compare
xeophon
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 72492f99b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
xeophon
force-pushed
the
harness-image-prompts
branch
from
72492f9 to
5d5ab11
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d5ab11990
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
xeophon
force-pushed
the
harness-image-prompts
branch
from
5d5ab11 to
c6343cb
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6343cbe54
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
xeophon
force-pushed
the
harness-image-prompts
branch
from
c6343cb to
8adb1eb
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8adb1eb. Configure here.
Default and null move serialized Messages out of the environment and into trace-specific transient files, deleting the handoff before any MCP setup begins. Codex maps user Messages onto its initial prompt surface, preserves system text, accepts parameterized image data URLs, attaches every materialized image with a repeated -i argument, and removes its rollout directory with best-effort cleanup.
xeophon
force-pushed
the
harness-image-prompts
branch
from
8adb1eb to
e5d5f21
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5d5f21cf3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Resolves the conflict in verifiers/v1/harnesses/null/program.py: this branch's asyncio.timeout(60) wrap around connect_mcp() lands adjacent to main's --initial-messages-file refactor (#1974). Kept both: the timeout wrap, and main's file-based `initial` (dropping this branch's now-stale `initial = json.loads(os.environ.get("INITIAL_MESSAGES", "[]"))`, which would have silently clobbered the value main's refactor already computed earlier in main()). Also re-ran ruff format: this branch's incidental single-line reformatting of chat()/connect_mcp() no longer matches the pinned ruff (0.15.12) target — main's original multi-line wrapping is what ruff format now produces, confirmed by running ruff format --diff against main's own unmodified file (no changes). Restored it so the diff is limited to the actual timeout fix + the requires-python bump. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 participants
Overview
Support image-bearing
Messagesprompts across the default, null, and Codex harnesses.Messagesto a trace-specific transient file, pass only its path to the harness program, and delete the file immediately after parsing arguments, before MCP setup.-iargument.Why
Default and null previously put the full serialized prompt in
INITIAL_MESSAGES. Base64 image data can exceed process-launch limits because argv and environment data share the same budget, causingE2BIGbefore the harness program starts.Codex accepts image files rather than OpenAI-style
image_urlcontent. Materializing those image parts bridges the existingMessagesrepresentation onto the Codex CLI while preserving system instructions and multiple images in their original order.Trace-specific paths avoid task-file collisions. Prompt handoff files are removed before setup work, and Codex image files use best-effort cleanup when the program exits with runtime teardown as a fallback.
Note
Add image-bearing Messages prompt support to harnesses
CodexHarnessnow accepts structured message prompts containing image parts. Text parts are concatenated into the prompt string; base64data:imageURLs are decoded to temporary files and passed via-iarguments. Onlysystemanduserroles are accepted; invalid roles or non-base64 image URLs raiseValueError. Temporary files are cleaned up on exit.INITIAL_MESSAGESenvironment variable to writing them to a per-trace JSON file and passing--initial-messages-file=<path>on the CLI. The file is deleted after reading.INITIAL_MESSAGESfrom the environment will no longer receive it there.Macroscope summarized cb1c6d2.
Note
Medium Risk
Behavior change removes INITIAL_MESSAGES for default/null; Codex adds new prompt parsing and temp file I/O on the rollout path, with validation errors for unsupported message shapes.
Overview
Adds multimodal
Messagesprompt handling so base64 images no longer blow process launch limits (E2BIG) when prompts are passed through argv or environment.Default and null harnesses stop using the
INITIAL_MESSAGESenvironment variable. For non-string prompts they write wire-format JSON to a trace-specific.vf-initial-messages-{trace.id}.jsonfile, pass only--initial-messages-file, and the harness program loads that file and deletes it immediately on startup.Codex harness advertises
SUPPORTS_MESSAGE_PROMPT = True, can take structuredMessageswhen bothsystem_promptand a non-stringpromptare set, folds system/user text into Codex’s single initial prompt string, decodesdata:image/...;base64,...parts into a per-trace image directory, passes them with repeated-iflags, and best-effort removes that directory aftercodex execfinishes. Non–system/user roles or non–base64 image URLs raiseValueError.Reviewed by Cursor Bugbot for commit cb1c6d2. Bugbot is set up for automated code reviews on this repo. Configure here.