[Bug] subagent_codex fails with Product subagent failure (product: Codex; stage: turn; category: product-error) and no diagnostic

[tigerwho1888]

Summary

subagent_codex failed twice in a row, both times with the same notice and no underlying detail:

Background subagent <id> failed before it finished. It left no closing message.
Product subagent failure (product: Codex; stage: turn; category: product-error)

The task itself was trivial (generate one image and save it to a path). The same task run through the Codex CLI completed, so the Codex side is healthy — only the subagent path fails.

Environment

• dsh 0.2.1-alpha.2 (Homebrew install, /opt/homebrew/lib/node_modules/@deepseek-ai/dsh)
• @deepseek-ai/dsh-subagent-codex 0.2.1-alpha.2
• codex-cli 0.162.1, auth mode chatgpt
• macOS (Apple Silicon), session run from the dsh web host
• Working directory: a git repository

Steps to reproduce

1. From a dsh web session, call subagent_codex with a short, self-contained prompt, e.g.

   Generate one image with your image generation tool and save it as a PNG at
   /<dir>/.tmp/imagegen-test/lighthouse.png (create the folder if needed).
   Then report the path, byte size and pixel dimensions.
   

2. The dispatch is accepted and the tool returns a subagent id.

3. The subagent then fails with the notice quoted above and no closing message.

Observed twice, roughly 3 minutes apart, with two different prompts (one long, one short) and the same result:

• subagent f2824c71-e3e3-43e2-820a-7bd6ff330536
• subagent 58f5826e-004f-49b5-b395-2fd37a1974e9

What the Codex side looked like at that moment

$ codex doctor
20 ok · 1 notes · 1 warn · 0 fail   (degraded)
  app-server   running (ephemeral mode), version 0.162.1
  websocket    connected (HTTP 101), endpoint wss://chatgpt.com/backend-api/…
  auth mode    chatgpt

codex features list reports image_generation stable true. So neither auth, nor the app server, nor the image feature was down.

No diagnostic is emitted anywhere

• ~/.dsh/logs/dsh-web.err.log gained no line (newest entry predates the failure by a day).
• ~/.dsh/logs/model-failover.jsonl records no hop for these turns.
• The parent notice carries only the product, stage and category.
• The failed child sessions leave no readable record of the failure.

Source of the message (for whoever fixes it)

node_modules/@deepseek-ai/dsh-subagent-codex/lib/index.js:

• L261-278 — the turn ends with status !== "completed"; failureInfo() maps error.codexErrorInfo and, for badRequest / threadRollbackFailed / other (L90-92), returns category: "product-error". The error thrown to the parent is
  subagent-codex: Codex turn ended with status failed: product-error
  — the Codex-side error text is never included.
• The stage is turn, not turn-start, so turn/start succeeded: the thread was created and the turn started, then failed.

Workaround

Running the same prompt through the CLI works:

codex exec -s workspace-write -C <dir> --skip-git-repo-check "<prompt>"

It completed in about a minute and wrote a valid 1254x1254 PNG (1,482,458 bytes).

Requested improvement

Surface the Codex-side failure detail — at minimum codexErrorInfo, preferably the provider's error message — in the parent notice or in a log line, so that product-error is diagnosable instead of terminal.

[argszero]

Source-level confirmation of your analysis — and the layer where the detail is still present.

Traced on master d743267388 (the tree @deepseek-ai/dsh@0.2.1-alpha.2 matches). Your reading of lib/index.js is right, and the reduction happens one layer below the notice: the specific Codex-side error id is known at the wire and is discarded before it is projected.

1. The wire layer already knows the exact failure id.
packages/subagent/subagent-codex/src/wire.ts → failureInfo(turn) reads turn.error.codexErrorInfo. As a string it is matched against a fixed list — contextWindowExceeded → limit, serverOverloaded / internalServerError → service, cyberPolicy / misalignmentPolicyViolation / unauthorized → access-policy, sandboxError → access-policy — and badRequest, threadRollbackFailed and other all collapse into the single product-error. Any string outside that list becomes unknown. As an object, only five transport keys plus activeTurnNotSteerable are recognised; every other shape is unknown as well.

So category: product-error is the union of at least three distinct Codex ids, and the actual value is available exactly here and dropped exactly here.

2. The projection to the parent is deliberate and lossy.
run.ts → failureDiagnostic(facts) renders only product: Codex; stage: …; category: … (plus HTTP status, exit code, signal when known), and the doc comment on codexStartupFailure() states the intent verbatim: "Hide an unpublished Host failure behind fixed safe startup facts." The full cause is retained on the CodexRunFailure instance — nothing reads it, so it never leaves the process.

The logging side explains your silent logs: the provider's onError hook writes error.message only (which is the already-projected line), and a failure raised before publication logs nothing at all. Neither dsh-web.err.log nor model-failover.jsonl is fed by this path, so their silence is expected rather than evidence that the failure was elsewhere.

3. stage: turn is the fingerprint of a turn that started. turn/start resolved and the turn frame came back with status !== 'completed'; a thread-creation or initialize failure would have reported stage: thread-start / stage: initialize. Your deduction holds.

4. Where the detail can still be recovered. The provider spawns the app-server with every stream explicitly piped — stdio: { stdin: 'pipe', stdout: 'pipe', stderr: 'pipe' } in startCodexRun — and decodes the JSON-RPC frames off the child's stdout. That pipe is reachable: a plugin that wraps the subprocess service's spawn can tee the handle it returns for the Codex app-server argv, mirror the frames into a bounded local capture, and read error.codexErrorInfo (and any error.message) out of the failing turn frame. That keeps the parent notice as narrow as it is designed to be while making the failure diagnosable locally. For completeness, stderr is already forwarded to the host's own stderr descriptor by the provider.

Interim answer to "why is this terminal": with the projection closed, the raw app-server frame is the only surviving copy of the value, so a capture at that seam is the difference between seeing product-error and seeing the id behind it.

[argszero]

The wire frames are recoverable from a plugin.

The cause analysis above still holds: wire.ts failureInfo() reduces codexErrorInfo to a coarse category, run.ts failureDiagnostic() renders fixed fields by design, the full error stays on the thrown failure as cause and nothing reads it, and the app-server's stdout is therefore the only surviving copy of the id. What that left open was whether anything could read that stream. One thing can.

ctx.get('subprocess') answers Symbol.for('cordis.original') with the object behind the tracking view, and spawn is a prototype method on the local provider — so one own-property write on that instance shadows it for every caller, including the Codex provider's own call-time lookup in subagent-codex/src/index.ts. Matching the app-server by the argv its own codexAppServerArgv() produces, and teeing that handle's stdout into a PassThrough, is then enough to keep every frame — including one the child answers before the transport attaches.

Published as a plugin:

• npm: @argszero/dsh-codex-wire-capture@0.1.0
• source: https://github.com/argszero/dsh-codex-wire-capture

npm install --save @argszero/dsh-codex-wire-capture

It ships its own bundle patch, so a profile that installs it by name gets the mount row without writing one.

Once mounted, a failing turn frame logs codexErrorInfo=<id> together with the provider's own message, in the same turn it arrives, and says what it would otherwise have been reduced to; the captured tail lands under <DSH_HOME>/logs/codex-wire/ when the child settles, with a JSON header carrying the same id, the message, the exit facts and the frame count.

It is an observer, deliberately: it does not change the parent notice, does not classify or retry, and never touches the spawn spec, the argv, the environment, the exit facts or the child's stdin. If the notice itself should carry the cause, that is a change in subagent-codex — this plugin is an argument for making it, not a substitute for it.

Probed against the three published lines (dsh-subprocess@0.1.7-rc.2, 0.2.0-rc.2, 0.2.1-alpha.2) with the whole line pinned in a scratch tree; 123 tests and 28 injection arms, none silent.