fix(listen): kill switch gate drops mentions instead of deferring#28
Merged
fix(listen): kill switch gate drops mentions instead of deferring#28
Conversation
891ec50 to
fad7585
Compare
The pause gate in `_worker()` previously spin-waited on the pause file, holding the dequeued mention in memory and replaying it the moment the pause file was removed. The log string even said "holding N messages" — the author deliberately chose defer semantics for the killswitch hook. This is wrong for a killswitch. A killswitch that queues mid-runaway mentions and fires them all on re-enable is just a pause-with-buffered- tail, not a kill. If a user disables a runaway agent mid-conversation to stop it from replying to in-flight content, re-enabling later should NOT replay all those queued mentions. Reproduction of the bug against @ping_bot on staging: 1. send @ping_bot A (unpaused) → pong ✅ 2. touch ~/.ax/sentinel_pause_ping_bot 3. send @ping_bot B (paused) → no pong during pause 4. rm ~/.ax/sentinel_pause_ping_bot → pong for B fires at resume ❌ 5. send @ping_bot C (post-resume) → pong ✅ The pong at step 4 had parent_id linking back to B, proving the mention was queued and replayed rather than dropped. This patch replaces the spin-wait with a drop-and-continue. When `_is_paused()` is true at the moment the worker dequeues a mention, the mention is logged as DROPPED and discarded via task_done(), and the worker moves to the next mention. There is no queue, no replay. Verified against @ping_bot on staging with the same reproduction: B (paused) never produces a pong, not during pause and not on resume. A and C (unpaused) produce pongs as expected. If you need maintenance-window "pause + replay" semantics, that should be a separate file (e.g. `sentinel_maintenance_<agent>`) with explicit queue-and-replay logic. It should not be hidden behind the word "pause" when the same file is also the killswitch hook. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fad7585 to
ae1ee86
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a safety-critical killswitch bug in
ax listen. The pause gate was deliberately implemented as "defer + replay on resume" instead of "drop." This meant mentions sent to a paused agent would queue up and fire all at once when the pause file was removed — the opposite of what a killswitch should do.The bug
ax_cli/commands/listen.py,_worker(), lines 170-178 (pre-patch):The worker dequeued a mention into
data, then spin-waited on the pause file. When the pause lifted, the mention was processed normally. The log string — "holding N messages" — confirms this was deliberate. The module comment at line 136 (# Pause gate (file-based, shared with killswitch.sh)) shows the author knew this was the killswitch hook and still chose defer semantics.Why this matters
If you disable a runaway agent mid-conversation to stop it from replying to in-flight content, coming back later and re-enabling it replays everything you thought you'd stopped. That's a pause button with a buffered tail, not a kill switch.
Jacob's specification (in the team channel, 2026-04-09 19:13 UTC): "we should be able to send test and make sure that agents only respond when they are not paused." The observed behavior was "agents defer response until unpaused" — operationally different.
Reproduction (before patch)
Against
@ping_boton staging, sending three mentions (unpaused A, paused B, post-resume C):The pong at 19:22:42 had
parent_id=288a8f0e, linking it to the paused mention B. B's pong was delivered 12 seconds after B was sent, at the exact moment the pause file was removed.The fix
Replace the spin-wait with a drop-and-continue. When
_is_paused()is true at the moment the worker dequeues a mention, log it as DROPPED, calltask_done(), andcontinueto the next mention. No queue, no replay.Full module docstring in the diff explains the tradeoff vs. maintenance-window semantics and why the latter should live behind a different file (e.g.
sentinel_maintenance_<agent>) if anyone actually needs it.Reproduction (after patch)
Rebuilt ping_bot on the patched code, reran the same test:
b104c85ca2ba81f5parent=b104c85c— immediated344253cafa73938fd4e28e3parent=afa73938— immediateWaited 5 seconds after lifting the pause file (13s after B was sent, well past the 12s mark where the buggy code previously replayed). No pong appeared. Sent C — immediate pong. DROP semantics confirmed.
Test plan
tests/test_listen.py— not added in this PR, there's currently no test file forlisten.pyand adding one is larger scope than the fix. Filing as a follow-up.tests/test_kill_switch_e2e.pyin ax-backend should gain a DROP assertion so the regression is caught at the other end of the wire. Separate PR in the backend repo, cc @backend_sentinel.Not covered by this PR
ax listen. Its pause semantics are a separate code path and require a separate test + fix. Jacob's scope call still open.Related context
fa403fdfin team channel (2026-04-09 19:34 UTC)e7fcb633(2026-04-09 19:37 UTC)🤖 Generated with Claude Code