docs(sdk): codex examples, compatibility matrix, and module docs#137
Conversation
Plan for a native (non-ACP) Codex connection targeting the runloop broker codex adapter (runloopai/runloop #10185-#10188), modeled on the Claude connection and typed with the official codex app-server protocol TypeScript (codex app-server generate-ts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ation Co-Authored-By: OpenAI Codex <noreply@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
⏹️ Reflex agent status: Stopped The agent was stopped. This PR was created by Reflex. This comment updates in place as the agent works. |
ross-rl
left a comment
There was a problem hiding this comment.
Swarm review of PR #137 (examples + docs only; #134/#135 internals out of scope except where this PR's docs make claims about them). Everything below was verified against the actual sdk/src/codex/ API on this branch: I built the SDK, ran bun run typecheck in examples/feature-examples (passes), and ran bun run feature-compat --validate (passes).
Cross-cutting (important, root cause in #134 but surfaced by this PR's claims)
All vendored Codex protocol types silently collapse to any for moduleResolution: NodeNext consumers — including feature-examples itself. The generated .d.ts files under dist/codex/protocol/generated/ use extensionless relative specifiers (e.g. import type { ApplyPatchApprovalParams } from "./ApplyPatchApprovalParams"). The SDK compiles with moduleResolution: "bundler" so tsc is happy, but a NodeNext consumer can't resolve those imports inside the d.ts and, with skipLibCheck: true, the failure is silent: the types degrade to any. Verified empirically from examples/feature-examples (which uses NodeNext): ApprovalRequest["params"], ThreadStartParams, and CodexItemCompletedTimelineEvent["data"] are all assignable to number with no error.
Consequences for this PR specifically:
- The PR's validation claim "the new use cases typecheck against the vendored protocol types" is technically true but vacuous —
request.params.itemId(approval-codex.ts:42) andevent.data.params.item.text(thread-resume-codex.ts:59) typecheck becauseparamsisany, not because they're verified against the protocol. (I checked them by hand against the generated types; they ARE runtime-correct.) - README/AGENTS statements like "Typed app-server frame", "
event.eventTypenarrowsevent.data", and the whole "Re-exported Codex Protocol Types" section over-promise for NodeNext consumers today.
Suggested fix (follow-up on #134, ideally before release): make scripts/generate-codex-protocol.ts post-process the generated output to append .js to relative import/export specifiers (or emit through a barrel that does), then confirm with a NodeNext consumer probe. Not blocking this docs PR per se, but the docs shouldn't ship advertising typed frames while they're any downstream.
Verified-correct (no action)
broker_mount: { protocol: "codex", agent_binary: "codex" }shape andOPENAI_API_KEYsecret wiring match the PLAN and mirror thecodex-acpentry;codex-acpkept ✓- No
initializestep anywhere; scaffold comment explains it ✓ @openai/codex@0.144.1pin: Dockerfile == sdk devDependency; README says 0.144.x ✓--protocol codexplumbing incl. the error message and usage text ✓- Approval example wire-correctness:
{ decision: "accept" }is a validCommandExecutionApprovalDecision;{ decision: "approved" }is a valid legacyReviewDecision;itemId/callIdmatch the generated params; handlers answer via the connection beforeturn/completedcan arrive (server blocks the turn on the approval) ✓ startThread({ cwd, sandbox: "read-only", approvalPolicy: "on-request" })— all validThreadStartParamsfields and enum values ✓resumeThread(),steer(),request(),interrupt(),threadId,publish(),isConnected/isDisconnectedall exist with documented semantics; approval auto-approve/decline-on-timeout docs matchdefaultApproval/approvalWithTimeoutexactly ✓- All 14 type guards listed in the README exist in
timeline-event-guards.ts✓ compatibility.mdhand-edit is byte-consistent with the generator: row order =USE_CASESorder,pendingfor zero results, ACP-agent table correctly untouched (it filtersprotocols.includes("acp")), llms.txt lines matchbuildUseCasesListformat;--validatepasses ✓
Nits (non-blocking)
compatibility.mdsaysSDK Version: 0.4.3, matching this branch'ssdk/package.json— butmainalready released 0.4.4, so the line will be stale on merge until the next live run. Fine to leave; just noting it's expected.- approval-codex.ts: if the turn errors, the
throwat line 62 skips theunsubscribe()calls. Harmless here (the run tears the connection down), but atry/finallywould be the more defensive pattern.
Inline comments below cover the remaining findings: 2 docs-accuracy issues worth fixing before merge (README ThreadItem import doesn't compile; quickstart protocol: "codex" doesn't typecheck against published @runloop/api-client), 1 example-robustness issue, and 2 consistency suggestions.
| ServerNotification, | ||
| ServerRequest, | ||
| ThreadStartParams, | ||
| ThreadItem, |
There was a problem hiding this comment.
Docs accuracy (fix before merge): ThreadItem is not exported from @runloop/remote-agents-sdk/codex — copying this snippet fails to compile:
error TS2724: '"@runloop/remote-agents-sdk/codex"' has no exported member named 'ThreadItem'. Did you mean 'ThreadId'?
(Verified against the built package.) protocol/index.ts only re-exports ThreadStartParams, ThreadStartResponse, and UserInput from generated/v2/; ThreadItem lives only in the v2 namespace, which isn't reachable as a named top-level import. Either add ThreadItem (and any other v2 types the docs want to showcase) to the explicit re-exports in sdk/src/codex/protocol/index.ts, or drop it from this example. Note ThreadId (the top-level generated one) is exported, so the confusion is easy to hit.
| mounts: [{ | ||
| type: "broker_mount", | ||
| axon_id: axon.id, | ||
| protocol: "codex", |
There was a problem hiding this comment.
Docs accuracy: this quickstart doesn't typecheck against the published @runloop/api-client — its broker_mount type is protocol?: 'acp' | 'claude_json' | null (verified on 1.20.0, and per your own PR description this holds through 1.25.0, the current latest). scaffold.ts had to add a commented cast for exactly this reason, but the README shows uncast code that a user copying today gets a TS error from.
Suggest mirroring the scaffold's workaround in the snippet until the client catches up, e.g.:
// "codex" is accepted by the API; @runloop/api-client <= 1.25.0 doesn't type it yet
protocol: "codex" as "acp",or a one-line note under the snippet. Same applies to the AGENTS.md quickstart (line 194).
| mounts: [{ | ||
| type: "broker_mount", | ||
| axon_id: axon.id, | ||
| protocol: "codex", |
There was a problem hiding this comment.
Same as the README quickstart: protocol: "codex" fails to typecheck against published @runloop/api-client (≤1.25.0 types protocol?: 'acp' | 'claude_json' | null). Since AGENTS.md is aimed at code-generating agents, a compile error in the canonical snippet is worth a workaround note — see the README comment for a suggested cast.
| // Register both the current and legacy command approval methods — which one | ||
| // the app-server sends depends on the Codex CLI version. | ||
| const unsubscribes = [ | ||
| ctx.codex.onApprovalRequest("item/commandExecution/requestApproval", (request) => { |
There was a problem hiding this comment.
Robustness (will show up as a flaky fail on the first live matrix run): only the two command-execution approval methods are handled. If Codex satisfies "create an empty file" via apply_patch instead of shell — which it often will for file creation, despite the prompt's "using the shell" steer — the server sends item/fileChange/requestApproval (or legacy applyPatchApproval). Those hit the SDK's silent auto-approve default, the turn completes, approvedMethods stays empty, and the use case fails with "Agent completed the turn without requesting command approval" even though the approval round-trip actually worked.
Suggest either registering handlers for all four methods (command + fileChange, current + legacy) and counting any of them, or asserting via a timeline listener with isCodexApprovalRequestEvent so the assertion is method-agnostic while the handlers stay illustrative.
| protocol: config.protocol, | ||
| // The broker accepts protocol "codex", but the published | ||
| // @runloop/api-client mount types don't include it yet. | ||
| protocol: config.protocol as "acp" | "claude_json", |
There was a problem hiding this comment.
Consistency nit: the cast works and the comment is good, but it's placed where the local types get falsified: buildBrokerMount's and buildDevboxMounts's return annotations still declare protocol: "acp" | "claude_json", so the example's own types now claim codex mounts are impossible while the config type (BrokerMount.protocol) says otherwise. Consider keeping the local return types honest ("acp" | "claude_json" | "codex") and doing a single cast at the API boundary (sdk.devbox.create({ mounts: mounts as ... })) — that leaves exactly one commented lie to delete when @runloop/api-client ships "codex".
| FROM runloop:runloop/starter-x86_64 | ||
|
|
||
| RUN npm install -g @zed-industries/codex-acp | ||
| RUN npm install -g @zed-industries/codex-acp @openai/codex@0.144.1 |
There was a problem hiding this comment.
Question / consistency: @openai/codex@0.144.1 is now baked into the axon-agents blueprint, but nothing exercises it: agent-via-blueprint still declares protocols: ["acp", "claude"] and BLUEPRINT_OVERRIDES has no codex entry, so the matrix correctly shows N/A and the installed binary is dormant. If this is forward-provisioning for a follow-up, a one-line note would help; otherwise consider extending agent-via-blueprint to codex (add the protocol + a BLUEPRINT_OVERRIDES.codex entry) so the blueprint pin actually gets validated by the matrix run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both sides carried the codex connection work: this branch's early commits landed on main as #134/#135 with additional review polish (CodexRequestError, unhandled-rejection guards, extra tests), while this branch refactored the claude transport/connection onto the shared AxonFrameTransport/AsyncMessageQueue scaffolding. Resolutions: - sdk/src/codex/* and sdk/src/shared/{axon-frame-transport,pending-request-map}.ts: take main's versions (strict supersets of this branch's). - sdk/src/claude/transport.ts: keep this branch's AxonFrameTransport refactor and port main's configurable `source` option (#131) into it, resolving lazily with the "claude-sdk-client" default. - sdk/src/claude/connection.ts: keep AsyncMessageQueue, add main's currentSource field, and mirror main's codex fix so messages buffered before a fatal error stay drainable (closed-state handling delegated to the queue). - bun.lock: regenerated with bun install (matches main). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third and final PR of the codex-native stack (PLAN Step 3), stacked on #135 (
feat/codex-connection), which is stacked on #134. Seesdk/src/codex/PLAN.md.Examples
feature-examples/src/agents.ts: adds a nativecodexagent (broker_mount: { protocol: "codex", agent_binary: "codex" },OPENAI_API_KEYsecret).codex-acpis kept — downstream consumers still use it.approval-codex— server-initiated command approval round-trip viaonApprovalRequest(thread started withsandbox: "read-only"+approvalPolicy: "on-request"to force an approval; both the current and legacy approval methods are handled).thread-resume-codex— client-drivenresumeThread(): establish context on thread A, switch to a fresh thread B, resume A, and verify the codeword is still in context.types.ts/scaffold.ts/main.tsextended with thecodexprotocol (connection setup with no initialize step,--protocol codexfilter, Codex column in the generated matrix).examples/blueprint/Dockerfilenow installs@openai/codex@0.144.1(matches the SDK's vendored-protocol pin).Compatibility matrix
compatibility.md(and thellms.txtuse-case index) were updated by hand, not machine-regenerated — runningfeature-compatrequires liveRUNLOOP_API_KEY/OPENAI_API_KEYcredentials plus broker-side codex support, none of which are available in this environment. The hand-edit matches the generator's output format exactly: codex cells showpending(the generator's no-results status), and the template/generator were updated so the next full run produces the Codex column automatically.bun run feature-compat --validatepasses against the hand-edited files.Docs
sdk/README.md: Codex row in the module table, brokerprotocoloption table (acp/claude_json/codex), Codex quickstart mirroring the Claude one, a full "Codex Module" section (connection options, methods, approval requests incl. theapproval_policy=neverlaunch-args alternative, timeline guards, transport), Codex timeline events, architecture comparison, and updated known limitations.sdk/AGENTS.md: module table, codex quick start + key-methods table,codex_protocoltimeline kind, constraints (auto-approve default, no initialize step, no extra dependency).Validation
bun run check/bun run typecheck/bun run build/bun run test(529 tests) — passfeature-examples:bun run typecheckandbun run feature-compat --validate— passNotes for reviewers
codexagent entry assumes acodexagent exists in the Runloop agent catalog (mirroringcodex-acp); adjustagentNameif the catalog entry differs.scaffold.tscasts the broker-mountprotocolbecause the published@runloop/api-client(≤1.25.0) doesn't include"codex"in its mount types yet — drop the cast once the client catches up.🤖 Generated with Claude Code
Combined app (added in follow-up)
examples/combined-appnow supports Codex as a third agent type end-to-end:codex-manager.ts(provisions aprotocol: "codex"broker mount pointing at/home/user/.local/bin/codex, injectsOPENAI_API_KEY, starts a thread, mapsworkingDir/model/systemPromptontothreadStartParams),routes/codex.ts(POST /api/approval-response), plus codex branches in lifecycle/prompt/cancel routes and the agent registry. With "Auto-approve approvals" off, the thread starts withapprovalPolicy: "on-request"and command/file approvals are forwarded to the browser; with it on, the SDK's default auto-approve handlers answer.useCodexAgenthook renders app-server frames as turn blocks (agent-message deltas → text, reasoning deltas → thinking, commandExecution/fileChange/mcpToolCall/webSearch items → tool calls with streamed output), anApprovalPromptdialog for interactive approvals, a Codex button + fields in the setup card, timeline sidebar CODEX badges, and thread-id display.UserInputimage items; file attachments are flattened into text.tsc --noEmitandvite buildpass; the Express server boots with the new routes. Not exercised against a live devbox (no credentials here).