driver codex leaves its app-server process group alive on SIGTERM

[myobie]

Problem

st2 driver codex does not reap its isolated app-server process group when the wrapper receives SIGTERM.

The wrapper exits immediately. The app-server group survives and keeps the control socket live.

A replacement wrapper then refuses a second control owner. The supervisor retries on every reconcile pass without recovery.

Fleet evidence

The observed restart sample had six Codex seats and four Claude seats.

All six Codex seats left app-server groups that required TERM. All four Claude seats cleared their provider processes without a manual signal.

One Codex seat was the only agent on its host. Its orphan held the socket for 17 minutes and blocked 34 supervisor retries.

The service stayed active and retried every 30 to 31 seconds. Each PTY attempt exited with code 1 and reported:

Codex app-server socket ... is already live; refusing a second control owner

A manual TERM to the verified orphan group cleared the blocker. The existing supervisor then started a healthy replacement on its next pass.

Cause

run_controlled_owned starts the app server with spawn_process_group. That function calls setsid, so the app server owns a process group separate from the wrapper.

The function calls terminate_process_group only after run_connected returns.

The Codex wrapper installs no SIGTERM handler. It also has no guard whose Drop implementation owns the app-server group.

A direct SIGTERM therefore stops the wrapper before Rust reaches terminate_process_group. The separate app-server group survives and retains the socket.

The hook-trust preflight uses the same call-after-return cleanup shape.

The Claude session wrapper has different behavior. It catches SIGTERM, leaves its poll loop, and calls stop_provider_group before exit.

Required behavior

The Codex driver must reap each app-server group on every graceful wrapper exit path.

This includes SIGTERM during startup, hook preflight, control connection, thread binding, and steady operation.

The driver must remove the socket only after the owned group cannot retain it.

The driver must not signal an unverified or foreign process group.

SIGKILL cannot run in-process cleanup and is outside this guarantee.

Acceptance

• A model-free test starts the Codex wrapper with a fake app-server group.
• The test sends SIGTERM to the wrapper process.
• The wrapper exits within a bounded time.
• The app-server leader and its descendant both exit.
• The control socket is not live after wrapper exit.
• A replacement wrapper starts without the second-owner refusal.
• Tests cover SIGTERM during preflight and after the main app server starts.
• Existing normal-return and error cleanup stays green.
• Cleanup does not signal a foreign or reused process group.

Related work

Issue #28 covers leaked eval and E2E PTY daemons. This issue covers the production Codex driver process group.

Issue #162 owns the longer-term harness-driver boundary.