feat: image gen, code interpreter, canvas + 10 new MCP tools (25 total) - #5
Merged
Conversation
Three new MCP tools, four P0 bug fixes, and a repositioning of the package around "codex login → full ChatGPT Pro inside any MCP client". New tools (19 total now, up from 16): - agent — ChatGPT Agent Mode (262K context, autonomous browse/code/tools) - gpt_chat — chat through one of your private g-p-* Custom GPTs (experimental gizmo_id payload routing, RE'd from chatgpt.com web bundle) - memory_create_via_chat — model-initiated memory write (workaround for POST /backend-api/memories returning 405) Fixes: - heavy DR returned the connector-dispatch JSON instead of the real report. _emit_done fired on the dispatch envelope's finished_successfully before the actual report started streaming. Gated with state[is_connector_dispatch]. - "Research is not currently supported in temporary chats" — both DR payload builders were forcing history_and_training_disabled=True, marking the conversation as Temporary Chat. Both DR paths now use False; chat keeps True. - Quota probe blocked the asyncio event loop — wrapped in asyncio.to_thread. - BackendClient now reloads the bearer when ~/.codex/auth.json mtime changes, so codex's background refresh propagates without restart (30-min heavy DR runs no longer 401 mid-poll). - server.py tool-register failures used logging.warning (no traceback); switched to logging.exception for diagnosability. Additions: - deep_research / deep_research_heavy gained auto_confirm: bool = True — prefixes an imperative directive so the model proceeds without asking. - [models].heavy_dr and [models].agent config keys for slug overrides. - gizmo_id parameter on ConversationClient.stream / .complete and _build_payload. Tests: - tests/test_heavy_dr_parser.py — synthetic SSE replay covering the two-message pattern (dispatch envelope → tool invocation → tool response → real report envelope → progress patches → final done). Asserts exactly one done event with the real report text, no dispatch JSON in progress stream, payload temp-chat fix, heavy DR model override, gizmo_id propagation. - tests/test_backend_token.py — codex auth.json mtime-triggered reload, no-op when unchanged, tolerant of file deletion. - tests/test_deep_research.py — updated to assert temp-chat fix. All tests: 12 passed, 9 skipped (live tests gated by SKIP_LIVE=1). build_server smoke: 19 tools registered. Refs upstream audit AUDIT_2026-05-15.md P0 #1-#4, P1 #5, #7, #10.
…on first question
ChatGPT's `research` model often opens with a clarifying question
("Could you confirm whether you'd like 6 or 12 months?") rather than starting
research. Before this commit, `deep_research` exited on that first `done`
event and the caller never saw real research.
New behaviour in `ConversationClient.deep_research`:
- Capture `conversation_id` (top-level SSE field) and the latest assistant
message id during streaming, so a follow-up turn can land in the same
thread with the correct `parent_message_id`.
- Detect clarification-shaped `done` events via `_looks_like_clarification`:
text < 800 chars ending in "?", or matching a curated phrase list
("could you confirm", "before I start", "just one key clarification",
"shall I proceed", …).
- When detected, auto-reply with "Proceed with your best interpretation of
any ambiguity. Do not ask further clarifying questions. Begin the research
now." Capped at `max_clarification_rounds=2`.
- Surface `{"type": "clarification_auto_reply", "round": N, "question": text}`
events so callers (and tests) can observe the auto-reply.
`_build_dr_payload` gained optional `conversation_id` + `parent_message_id`
kwargs to support the follow-up turn.
Tests: 4 new in `tests/test_dr_clarification.py` — scripted SSE replay
covering (1) clarification → auto-reply → real report, (2) long real report
NOT mis-detected as clarification, (3) `_looks_like_clarification` unit
heuristic, (4) `_build_dr_payload` continuation field propagation. All 16
tests pass.
Heavy DR (`deep_research_heavy`) is unchanged — its connector-orchestrated
path doesn't typically surface clarifications. If field reports show
otherwise, the same pattern can be applied to that state machine.
…release
Three deliverables for "install anywhere, ship anywhere":
1. `openai-mcp install` subcommand (openai_mcp/install.py)
- Targets: claude-code (~/.claude.json mcpServers.openai), codex
(~/.codex/config.toml [mcp_servers.openai]). Auto-detects with
`--client all` (default).
- Idempotent + atomic write + .bak-openai-mcp backup of prior file.
- --transport stdio (default) or http --http-port N.
- --dry-run prints actions without writing.
- Bundles the deep-research skill into ~/.claude/skills/ on
claude-code installs (--no-skill to opt out).
2. One-line installer (install.sh — rewrite; was macOS LaunchAgent only)
- Cross-platform. Detects Python 3.10+, installs pipx if missing,
pipx-installs openai-mcp, checks for codex auth, then runs the
install subcommand.
- Curl-pipe-bash entry. Flags for source path / git URL / per-client.
3. GitHub Actions (.github/workflows/)
- ci.yml — pytest on {ubuntu, macos} × Python 3.10-3.13 + shellcheck
install.sh, on push/PR to main.
- release.yml — on v* tag: verify tag matches pyproject version,
test matrix, build wheel + sdist, twine check, publish to PyPI via
OIDC trusted publishing (one-time setup at pypi.org; no token
secret needed), then create GitHub Release with this version's
CHANGELOG section as body. Auto-marks -rc/-alpha/-beta as prerelease.
Bundling: openai_mcp/skills/deep-research/{SKILL.md,bin/*} ships in the
wheel via pyproject [tool.setuptools.package-data]. The install module
shutil.copytree's it into the user's ~/.claude/skills/.
Tests: tests/test_install.py — 17 new tests covering Claude Code config
preservation (large existing files), Codex section replace-or-append,
backup creation, idempotence, dry-run, broken-JSON refusal, HTTP transport,
skill bundle copy + overwrite-with-backup, _replace_or_append_toml_section
unit cases, and detect_clients HOME-aware behaviour. All 33 tests pass
(was 16) — no regressions.
README: front-page reorganised — one-line curl|bash up top, step-by-step
underneath, per-client manual snippets at the bottom. New Development →
Release section documents the tag-driven release flow.
Doesn't touch any user runtime data unless --dry-run is False.
Independent code review by sonnet pr-review-toolkit/code-reviewer flagged
3 blockers + 3 highs against feat/full-features. All addressed:
BLOCKERS
* sse.py:deep_research — sentinel + token reread INSIDE the multi-turn
clarification loop. Previously the sentinel from round 1 was reused for
the auto-proceed POST in round 2; sentinels are short-lived and the
reuse silently 403s, defeating the entire multi-turn feature.
* sse.py:stream + deep_research_heavy — call _reload_token_if_stale()
before snapshotting headers. The SSE path uses a fresh AsyncSession and
doesn't go through backend.get/post, so it never benefited from the
mtime-based reload. Long heavy-DR runs (5–30 min) could 401 mid-stream
if codex refreshed ~/.codex/auth.json.
HIGH
* install.py:_atomic_write — preserve existing file mode on rename
(or 0o600 for new files). Previously inherited process umask (typically
0o644), exposing ~/.claude.json (which contains MCP commands the agent
will exec) to other users on shared systems.
* release.yml — tighten tag trigger from "v*" to "v[0-9]+.[0-9]+.[0-9]+"
(+ "-*" for pre-releases). 'v', 'vfoo', 'vlatest' no longer fire the
workflow and burn a publish attempt before failing version-check.
* ci.yml + release.yml — add libcurl4-openssl-dev install step on Linux
runners. curl_cffi typically ships manylinux wheels but falls back to
source build on edge cases (esp. Python 3.13 fresh after release);
missing libcurl headers would fail the matrix silently.
MEDIUM
* sse.py:_looks_like_clarification — drop the bare "len<800 + endswith('?')"
branch. False-positives on real reports ending with rhetorical "?"
(caught in updated test_clarification_detection_unit). Phrase-list-only
is conservative but avoids wasting a clarification round on real answers.
* sse.py:deep_research finally — only emit the synthetic terminated_abnormally
done event when the stream ended NORMALLY. On exception, propagate cleanly
without faking a done event so callers don't mistake partial output for a
complete answer.
Tests:
- _FakeBackend in 3 test files now exposes a no-op _reload_token_if_stale()
to mirror the production interface.
- test_clarification_detection_unit updated: "Which timeframe?" now correctly
asserted negative; added rhetorical-question false-positive case as
asserted negative.
All 33 tests still pass; 9 skipped (live).
Why: "openai" is a registered OpenAI trademark; PyPI may take the package
down under their trademark policy and there's no implied affiliation. The
new name continues this repo's `chatgpt2agent` naming pattern (drop "chat")
and reads as "GPT to agent" — i.e. the package makes any GPT account
addressable as an agent.
Renames:
- PyPI package + CLI: openai-mcp → gpt2agent
- Python module: openai_mcp/ → gpt2agent/
- Default MCP server-name in client configs:
- Claude Code: mcpServers.openai → mcpServers.gpt2agent
- Codex: [mcp_servers.openai] → [mcp_servers.gpt2agent]
- Bundled Claude Code skill: openai-mcp → gpt2agent
- Config dir: ~/.openai-mcp/ → ~/.gpt2agent/
- All imports `from openai_mcp.…` → `from gpt2agent.…`
Mechanical: sed across all .py/.toml/.md/.sh/.yml (except CHANGELOG, which
keeps the historical "openai-mcp 0.0.1" mention in the 0.0.1 section), then
git mv openai_mcp/ → gpt2agent/. install.py default `server_name` updated
to "gpt2agent". Tests updated to assert new section/key names.
Migration for users on 0.0.1:
pipx uninstall openai-mcp && pipx install gpt2agent
gpt2agent install # writes new keys in client configs
mv ~/.openai-mcp ~/.gpt2agent # if you had a config dir
# then manually remove the stale "openai" entry from ~/.claude.json.
Verification:
- 33 pytest pass / 0 fail / 9 skipped
- pipx list: gpt2agent 0.0.2 (editable from worktree)
- python -m build → gpt2agent-0.0.2.{tar.gz,whl}
- twine check: PASSED for both
- Fresh venv install: gpt2agent CLI works, 19 tools register including
agent / gpt_chat / memory_create_via_chat
- gpt2agent install --dry-run from /tmp: detects claude-code + codex,
writes mcpServers.gpt2agent / [mcp_servers.gpt2agent]
1. POW solver fingerprint mismatch (audit P2 #11) _vendored/pow.py:103 had `language−zh-CN` in the navigator-key pool while the request side sets `OAI-Language: en-US`. Cloudflare's bot manager cross-checks the proof-of-work fingerprint against the request headers and 403s on mismatch — explained intermittent sentinel failures. Fix: use `language−en-US` to match. 2. _redact_error duplicated verbatim (audit P2 #15) sse.py and sentinel.py each defined the same `_SENSITIVE_KEY_RE` + `_redact_error()`. Consolidated into new top-level `_log_redact.py` module (`redact_error`); both call sites now import the single source. Module docstring clarifies the scope split vs the existing `tools/_redact.py` (PII redaction in MCP tool outputs vs header/auth redaction in error logs — distinct concerns kept separate). Removed now-unused `import re` from both sse.py and sentinel.py. 3. Legacy `mcpServers.openai` migration in `gpt2agent install` Users on 0.0.1 (`openai-mcp`) who run `gpt2agent install` after the rename ended up with both the new `mcpServers.gpt2agent` AND a stale `mcpServers.openai` entry pointing at the gone openai-mcp binary — Claude Code would spawn-and-fail on every restart. Now both `install_claude_code` and `install_codex` detect the legacy entry/section (matched by `command == "openai-mcp"`) and drop it as part of the install, with a `_ok` log line so the user knows. Unrelated `mcpServers.openai` entries (e.g. someone's other bridge) are left alone — strict match on command, not on key name. Tests: tests/test_install.py adds 4 new tests: - test_claude_migrates_legacy_openai_entry - test_claude_keeps_unrelated_openai_entry - test_codex_migrates_legacy_openai_section - test_codex_keeps_unrelated_openai_section 37 pass / 0 fail / 9 skipped.
Real production root cause for the heavy-DR-12s-failure (re-discovered
2026-05-15 from /home/robot/workspace/11-autobio-isaac/research/heavy-review/events.jsonl):
The dispatch envelope arrives as `parts=[""]` (EMPTY assistant text)
with `status=finished_successfully`, NOT the `{"path":...}` connector JSON
the original audit hypothesized. The previous P0 #1 fix's
``is_connector_dispatch`` text heuristic doesn't match an empty string, so
``_emit_done`` fired with text=""; the wrapper exited in 12s before any
real report arrived. The server then sent `server_ste_metadata` with
`tool_invoked: true` and `turn_mode: "deep research"`, but `done_emitted`
was already True so Phase 2 polling never kicked in.
Fix: add `state["asst_text"]` non-empty guard alongside the existing
`is_connector_dispatch` check, in BOTH `_on_envelope` and the
`_apply_path("/message/status", ...)` branch. An empty assistant text
buffer reaching `finished_successfully` is *always* a placeholder (real
reports stream content first, then flip status; one-shot short replies
arrive with non-empty parts in the same envelope).
Test: tests/test_heavy_dr_parser.py adds
test_empty_dispatch_envelope_done_suppressed which replays the exact
production frames captured today: empty dispatch envelope → meta with
tool_invoked → real report envelope → progress patch → status flip.
Asserts exactly one done event, with the real report text.
All 38 tests pass; 9 skipped (live).
User impact: heavy DR runs that previously exited in ~12s with empty
output now correctly suppress the dispatch-placeholder done and (a) keep
streaming if the report arrives in Phase 1, or (b) trigger Phase 2 polling
if the SSE stream closes after the dispatch (because tool_invoked will be
True from the meta event by then).
Co-authored-by: Codex <noreply@openai.com>
…ng tools - Add generate_image MCP tool with async polling + download URLs - Add code_interpreter and canvas_execute MCP tools - Add get_conversation, get_file_info, get_file_download_url tools - Add temporary param to _build_payload/stream/complete (False enables tool-based features: image gen, code interpreter, canvas) - Fix SSE parser to handle multimodal_text content type - Enhance list_models to return 22 models with full metadata - Enhance list_tasks to return full task fields - New ConversationClient.image_gen() and tool_call() methods 25 MCP tools total (up from 15). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Fix null-safety: msg.get("content") or {} in stream() Format B
- Fix image_gen: guard against empty-asset early return
- Fix image poll: sort by create_time to avoid stale data
- Fix gpt_chat: use temporary=False for GPT memory_scope
- Fix get_conversation: add max_messages param (default 100)
- Fix images: surface download errors instead of silent pass
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 43 minutes and 19 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughThis PR completes a comprehensive rebranding and feature enhancement of the Changesgpt2agent refactor: identity, auth, SSE, tools, and installation
🎯 4 (Complex) | ⏱️ ~60 minutes
✨ Finishing Touches🧪 Generate unit tests (beta)
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eccafc42a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+79
to
+83
| "id": data.get("id"), | ||
| "title": redact(data.get("title") or ""), | ||
| "create_time": data.get("create_time"), | ||
| "update_time": data.get("update_time"), | ||
| "message_count": len(messages), |
There was a problem hiding this comment.
This block is a dangling dict fragment (there is no return { ... }), which makes the module fail to import with a SyntaxError (unmatched '}'). In practice, importing gpt2agent.tools then fails at conversations, so register_all cannot load backend tools and the new conversation/task tooling is unavailable at runtime.
Useful? React with 👍 / 👎.
- sse.py: tool_call content:null crash (msg.get("content") or {})
- sse.py: image_gen timeout 120s → 300s for complex prompts
- sse.py: poll loop exponential backoff (max 5 consecutive errors)
- sse.py: removeprefix instead of global replace for sediment://
- sse.py: text_parts fallback when last_text is empty in tool_call
- server.py: agent() and memory_create_via_chat use temporary=False
- images.py: asyncio.to_thread for sync HTTP calls (event loop safety)
- images.py/tools_features.py: reuse conv singleton, no redundant sentinel
- tools/__init__.py: pass conv to images and tools_features registration
- conversations.py: null safety for list_conversations and list_tasks
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
generate_imageMCP tool with async polling + download URLscode_interpreterandcanvas_executeMCP tools via ChatGPT tool callsget_conversation,get_file_info,get_file_download_urltoolshistory_and_training_disabled=Trueprevented all tool-based featureslist_modelsto return 22 models with full metadata (enabled_tools, thinking_efforts)list_tasksto return full task fieldsKey changes
sse.py: Addtemporaryparam to_build_payload/stream/complete. Handlemultimodal_textin SSE parser. Newimage_gen(),tool_call(),_poll_image_result()methods.server.py: Propagatetemporaryparam tochat(). Fixgpt_chatto usetemporary=Falsefor GPT memory.tools/images.py(new):generate_image,get_file_info,get_file_download_urltools/tools_features.py(new):code_interpreter,canvas_executetools/conversations.py: Addget_conversationwithmax_messageslimit, enhancedlist_taskstools/account.py: Full metadata inlist_modelsTool count: 15 → 25
Test plan
generate_imagetested live: prompt → SSE → async poll → sediment asset → download URLlist_modelsreturns 22 models with enabled_tools🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
Chores
openai-mcptogpt2agent