Skip to content

SIGKILLed foreground companion orphans a live app-server turn; /cancel can't reach it #42

Description

@axisrow

Problem

A foreground companion run that is SIGKILLed by Claude Code's Bash tool (~120s ceiling) leaves the underlying Codex turn running inside the detached broker, still able to modify the workspace. The job is then reconciled to failed, and /codex:cancel refuses to touch it — so the orphaned turn cannot be stopped through the plugin at all.

This is not introduced by #41. It is reachable on main today. Verified against 15f22e6: the broker's socket-close handler, the /cancel status filter, and the missing turnTimeoutMs on the adversarial-review path are all identical there. #41's idle semantics widen the window for actively-progressing turns, but every link in the chain predates it.

Failure chain

  1. Foreground runs the turn in-process. handleTask (plugins/codex/scripts/codex-companion.mjs:905-922) and handleReviewCommand (:828-843) both go through runForegroundCommand (:744-755). Only task --background spawns a detached worker (spawnDetachedTaskWorker, :757-768). Note handleReviewCommand parses --background (:801) but never consults options.background — for reviews the flag only raises the idle budget via resolveTurnTimeoutMsFromOptions (:726), it does not background anything.

  2. An active turn never trips the internal deadline. armIdleDeadline (plugins/codex/scripts/lib/codex.mjs:713-726) resets on every lifecycle and in-item progress notification, so a healthy long-running foreground turn sails past the host's Bash ceiling.

  3. SIGKILL leaves no trace. There are no signal handlers anywhere in plugins/codex/scripts/, and runTrackedJob (plugins/codex/scripts/lib/tracked-jobs.mjs:194-226) has no finally, so the job record stays status: "running".

  4. The broker never interrupts. socket.on("close") / socket.on("error") (plugins/codex/scripts/lib/broker-controller.mjs:353-370) call only sockets.delete + clearSocketOwnership (:149-157) + scheduleIdleShutdown. No turn/interrupt is issued; the app-server child keeps executing the turn.

  5. Notifications are dropped and the broker reports itself idle. routeNotification (:159-175) returns early once ownership is cleared, and scheduleIdleShutdown arms the 15-minute timer — so probeBroker sees an idle broker while a turn is live, and a reuse path may even shut it down mid-turn.

  6. Reconciliation marks the job failed. reconcileRunningJobs (plugins/codex/scripts/lib/state.mjs:151-196) lazily flips runningfailed with UNREPORTED_PROCESS_EXIT_MESSAGE on the next listJobs.

  7. /cancel can no longer reach it. resolveCancelableJob (plugins/codex/scripts/lib/job-control.mjs:326-353) calls listJobs (which reconciles first) and then filters status === "queued" || "running". The job is already failed, so handleCancel throws before ever reaching interruptAppServerTurn (plugins/codex/scripts/lib/codex.mjs:1166).

Why the fix is cheap

Everything needed for recovery is already in place:

  • createJobProgressUpdater (plugins/codex/scripts/lib/tracked-jobs.mjs:89-97) persists threadId and turnId incrementally, as soon as each is observed — so a killed job's record already carries valid ids.
  • interruptAppServerTurn passes allowBusyStaleBroker: true (plugins/codex/scripts/lib/codex.mjs:1186-1188), so it can attach to a busy broker.
  • The broker already has a dedicated bypass for interrupting a stream owned by another socket (plugins/codex/scripts/lib/broker-controller.mjs:298-323), and it tracks activeStreamThreadIds (:99, :334).

The machinery exists; only the reachability gate is missing.

Proposed fixes (independent, both worth doing)

  • Broker-side (defense in depth). In socket.on("close") / on("error"), if the closing socket was activeStreamSocket, issue turn/interrupt for the active stream before clearing ownership. This needs the turn id as well as the thread id — buildStreamThreadIds currently retains only thread ids, so it would need to capture turnId too.
  • Cancel-side (user-facing). Let /cancel reach a job that reconcileRunningJobs just marked failed with UNREPORTED_PROCESS_EXIT_MESSAGE and a non-null threadId/turnId, so interruptAppServerTurn becomes reachable. Either widen the filter at job-control.mjs:328 or introduce a distinct terminal state (e.g. orphaned) that cancel accepts.
  • Companion-side (partial). Install SIGTERM/SIGINT handlers that interrupt the in-flight turn. Does not help against SIGKILL, but covers ordinary termination.

Related

Verification

  • /codex:status must not report an idle broker while a turn is live.
  • After a kill that orphans a turn, /codex:cancel <id> must successfully interrupt it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions