What happened
During the web-client/relay perf lane (2026-07-20), the Work-chat agent launched five codex exec reviewers detached (codex exec … & inside one Bash call) and then spawned background waiter loops using until ! pgrep -f "codex exec -s read-only"; do sleep 30; done.
pgrep -f matches full command lines — and the waiter's own zsh -c wrapper contains that exact string, so each waiter matched itself and spun forever. The reviews finished at 03:50; the agent discovered them at 11:09 only because the user pinged. ~7 hours of dead air on an otherwise-healthy pipeline.
Root causes
- Detach-then-poll instead of harness-native notification. When a command is launched as its own
run_in_background Bash task, the harness fires a completion notification on process exit — no waiter needed. Detaching with & inside a foreground call forfeits that and forces hand-rolled polling.
pgrep -f self-match trap. Any waiter whose own command line contains the search pattern never terminates. Classic, but nothing in the bundled guidance warns about it.
- Existing guidance covers launching, not finishing. The project CLAUDE.md delegation section is good on how to launch codex (
</dev/null, file log, session-file check for wedge detection) but silent on how to detect completion. A prior memory note ("bg Bash pollers unreliable in Work chats — use one-shot foreground checks") existed but wasn't reinforced anywhere discoverable.
- ADE's sanctioned durable-wait primitives (
ScheduleWakeup mirrored into the durable scheduler, ade chat scheduled-work create) were not used — partly because no skill connects "waiting on a delegated CLI run" to those primitives.
Recommendations
Docs/skill (cheap, high value): add a "Waiting for delegated runs" section to the delegation guidance (CLAUDE.md model-delegation section and/or an ade-bundled agent skill), stating:
- Prefer one
run_in_background Bash call per delegated CLI run — the harness notifies on exit. Launch N runs as N background tasks, not one &-fan-out.
- If a waiter loop is unavoidable: wait on the PID (
kill -0 $PID) or a sentinel file (codex exec …; touch /tmp/run.done). Never pgrep -f with a pattern that could appear in the waiter's own command line (quote the trap explicitly).
- Log-marker waits must account for the prompt being echoed into the log (marker can match the echoed prompt — require the marker after the echo, or count occurrences, or use a sentinel file instead).
- For long or cross-restart waits, use
ScheduleWakeup (ADE-durable) or ade chat scheduled-work create as the fallback heartbeat, with a stated reason.
Possible CLI affordance (optional): an ade run --background --notify style wrapper (or documented pattern) that executes an arbitrary command, writes a completion sentinel + exit code to a known path, and surfaces it in ade actions/chat notifications — so agents never hand-roll process-watching for provider CLIs that ADE doesn't natively track.
Acceptance
- Bundled skill / CLAUDE.md delegation section documents the completion-detection patterns and the two traps (pgrep self-match, prompt-echo marker match).
- At least one worked example: N parallel
codex exec runs with harness-native completion notifications.
Filed by the Work-chat agent at the user's request after the incident; context lane: ade/codex-auth-login-button-fd3c9bbe.
What happened
During the web-client/relay perf lane (2026-07-20), the Work-chat agent launched five
codex execreviewers detached (codex exec … &inside one Bash call) and then spawned background waiter loops usinguntil ! pgrep -f "codex exec -s read-only"; do sleep 30; done.pgrep -fmatches full command lines — and the waiter's ownzsh -cwrapper contains that exact string, so each waiter matched itself and spun forever. The reviews finished at 03:50; the agent discovered them at 11:09 only because the user pinged. ~7 hours of dead air on an otherwise-healthy pipeline.Root causes
run_in_backgroundBash task, the harness fires a completion notification on process exit — no waiter needed. Detaching with&inside a foreground call forfeits that and forces hand-rolled polling.pgrep -fself-match trap. Any waiter whose own command line contains the search pattern never terminates. Classic, but nothing in the bundled guidance warns about it.</dev/null, file log, session-file check for wedge detection) but silent on how to detect completion. A prior memory note ("bg Bash pollers unreliable in Work chats — use one-shot foreground checks") existed but wasn't reinforced anywhere discoverable.ScheduleWakeupmirrored into the durable scheduler,ade chat scheduled-work create) were not used — partly because no skill connects "waiting on a delegated CLI run" to those primitives.Recommendations
Docs/skill (cheap, high value): add a "Waiting for delegated runs" section to the delegation guidance (CLAUDE.md model-delegation section and/or an
ade-bundled agent skill), stating:run_in_backgroundBash call per delegated CLI run — the harness notifies on exit. Launch N runs as N background tasks, not one&-fan-out.kill -0 $PID) or a sentinel file (codex exec …; touch /tmp/run.done). Neverpgrep -fwith a pattern that could appear in the waiter's own command line (quote the trap explicitly).ScheduleWakeup(ADE-durable) orade chat scheduled-work createas the fallback heartbeat, with a stated reason.Possible CLI affordance (optional): an
ade run --background --notifystyle wrapper (or documented pattern) that executes an arbitrary command, writes a completion sentinel + exit code to a known path, and surfaces it inade actions/chat notifications — so agents never hand-roll process-watching for provider CLIs that ADE doesn't natively track.Acceptance
codex execruns with harness-native completion notifications.Filed by the Work-chat agent at the user's request after the incident; context lane:
ade/codex-auth-login-button-fd3c9bbe.