Skip to content

feat(buzz-acp): durable pending-turn ledger for agent auto-resume after app restart#2177

Draft
wpfleger96 wants to merge 47 commits into
mainfrom
duncan/acp-resume-ledger
Draft

feat(buzz-acp): durable pending-turn ledger for agent auto-resume after app restart#2177
wpfleger96 wants to merge 47 commits into
mainfrom
duncan/acp-resume-ledger

Conversation

@wpfleger96

Copy link
Copy Markdown
Member

Summary

  • Adds a durable pending-turn ledger (ledger.rs) that mirrors the in-memory EventQueue to disk after every queue mutation, enabling managed agents to resume interrupted work after app restarts.
  • Wires a full boot-recovery pipeline in lib.rs: staged ledger load → membership gate → TTL expiry → chunked REST fetch with per-event reconciliation (signature/id/channel verification) → ordered queue import → unresolved ordering barrier registration → single atomic commit.
  • Implements barrier-deadline timer arm in the main select! loop (rev 6.1): a dedicated tokio::time::Sleep arm fires at the earliest unresolved barrier deadline, ensuring the timeout is a real bound even on a completely quiet harness.
  • Adds live-admission resolution seam: recovered events resolved via WebSocket bypass the steer/interrupt path and dispatch with recovery framing; suppression set prevents duplicate processing across boot fetch and live delivery.
  • Shutdown-drain completion classifies results through complete_batch + ledger sync under both grace-period drain sites; join-error arm applies queue-only panic disposition.
  • queue.rs gains complete_batch (atomic payload-owning completion), shared enforce_cap, import_recovered/admit_recovered, set_unresolved_barrier/next_unresolved_barrier_deadline, and dirty_channels tracking.
  • config.rs gains --resume-enabled/--resume-ttl-secs CLI flags.
  • format_prompt emits recovery header and per-event [recovered-from-restart] markers for recovered batches.
  • 21 new integration tests covering boot recovery, suppression (both orderings), REST-fail resolution, membership gating, shutdown-grace (success/fail/panic in both dedup modes), large-ledger budget, chunk reconciliation (valid/wrong-channel/duplicate/unrequested), steer-path short-circuit, quiet-harness barrier expiry, and ledger persistence. 587/587 tests green.

npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 11 commits July 20, 2026 12:27
…ess)

RecoverableTrigger, complete_batch/BatchDisposition, shared enforce_cap,
recoverable_triggers/take_dirty_channels, slash-command guard for
restart_recovery batches. All requeue*/mark_complete call sites in
lib.rs (pool-exhausted, handle_prompt_result, recover_panicked_agent)
now route through complete_batch. import_recovered/admit_recovered and
the unresolved ordering barrier are not yet implemented.

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 1.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… ordering barrier (Task 1, in progress)

import_recovered() applies the boot promotion rule (whole snapshot
exempt iff no persisted exempt record survived); admit_recovered()
bypasses Drop-mode rejection and cap accounting to insert a resolved
unresolved record at its original admission_seq position.

set_unresolved_barrier/next_unresolved_barrier_deadline/
expire_due_unresolved_barriers implement the R6-F1 per-channel
ordering barrier: flush_next and has_flushable_work now hold any
queued/cancelled event above a channel's lowest unresolved seq until
one of the three release paths (resolution via admit_recovered,
invalidation via drain_channel, or deadline expiry) fires. The
select! timer arm that makes expiry an actual bound on a quiet
harness (rev 6.1) is not yet wired — that lands with the rest of the
lib.rs boot-recovery sequence in Task 3.

104 existing queue.rs tests still pass; no unit tests for the new
surface yet (todo, still Task 1).

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 1.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Unit tests for import_recovered (promotion rule both branches, empty
no-op, cap bypass), admit_recovered (seq-position insertion, Drop-mode
bypass, barrier release), and the R6-F1 unresolved ordering barrier
(all three release paths — resolution, invalidation, deadline expiry
— plus cross-channel earliest-deadline selection, empty-seqs no-op,
and cancelled-batch-fallback enforcement).

Completes Task 1 of the auto-resume ledger plan: cargo test -p
buzz-acp passes 537/537, clippy clean (only expected dead-code
warnings for the barrier/recovery surface not yet wired into lib.rs,
which is Task 3).

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 1.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
New module: JSON ledger types (incl. cap_exempt), atomic load/save
(temp file + rename), per-channel sync() with skip-identical-write and
unresolved-entry preservation across ordinary rewrites, unresolved
resolve/invalidate ops, staged-load + single-commit transaction API
for boot, corruption/version-mismatch recovery, per-pubkey filename
isolation, and TTL filtering at load time.

cargo test -p buzz-acp passes 549/549 (12 new), clippy clean. Not yet
wired into lib.rs (Task 3) — module is otherwise self-contained and
unreferenced outside its own tests.

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 2.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… resume CLI flags (Task 3/5, in progress)

Adds --resume-on-restart/--resume-ttl-secs/--state-dir CLI flags and
Config fields (Task 5), and fixes the 7 dispatch_pending() call sites
in run() to pass &mut ledger now that the function signature requires
it. cargo check -p buzz-acp is green; boot recovery sequence, barrier
timer arm, and the remaining direct-mutator sync_dirty call sites are
still outstanding (Task 3 continues).

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Tasks 3 and 5.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…n completion (Task 3, in progress)

Fixes two silent-drift gaps in has_flushable_work()'s two call sites:
its opportunistic expiry loop can dirty a channel even when it returns
false, so both call sites now sync_dirty() unconditionally instead of
only inside the true branch. Also syncs after drain_channel() on the
membership-removal path, which previously fell through to a bare
continue with no dispatch_pending() call after it to drain the dirty
set.

Implements shutdown-drain completion (R5-F2/R6-F4): both grace-period
result-drain sites (the select! loop and the late-arrival while-let)
now classify every received PromptResult through the same disposition
logic as the normal completion path (extracted into
classify_and_complete_batch, shared with handle_prompt_result) before
reaping the agent, with no dispatch. The join-error arm in the
select! loop applies the same queue-only panic disposition
recover_panicked_agent uses (Queue -> Retry, Drop -> Dropped) via a
new three-way pool split-borrow (rx_join_set_and_task_map), also with
no respawn/dispatch. Both close the gap where a turn that completed or
panicked during the 30s shutdown grace period left stale in-flight
triggers in the ledger mirror for the next boot to needlessly
re-resume.

cargo test -p buzz-acp passes 549/549, clippy clean (only expected
dead-code warnings for the boot-recovery/barrier surface not yet
wired in). Remaining Task 3 work: boot recovery sequence, barrier
timer arm, suppression/admission seam, slash-command guard.

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 3.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…(Task 4)

Adds FlushBatch::has_recovered_events() and a recovery_marker() helper
that annotates each event's rendered header line — in both the fresh
events section and the merged cancelled_events section — with
" [restart recovery]" when the underlying BatchEvent.restart_recovery
is set. A single scoped [Restart recovery] header is prepended to the
prompt when has_recovered_events() is true, explicitly limiting the
"may already be done, re-read first" caveat to marked events so a
mixed batch's fresh events are never mistaken for already-answered
recovered ones (R5-F6's per-event provenance rule).

Also adds a regression test confirming slash_command_for_batch()
rejects a recovered single-event slash command (must route through
the wrapped-context path rather than bare pass-through re-execution).

cargo test -p buzz-acp: 553/553 passing, clippy clean (same expected
dead-code warnings for the not-yet-wired boot-recovery surface).

Ref: PLANS/AGENT_AUTO_RESUME_LEDGER.md rev 6.1, Task 4.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…progress)

Purge durable live + unresolved ledger records together when a channel
is removed, so re-adding it later doesn't resurrect discarded work
(P3-F3 invalidation exit).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ogress)

Adds boot_recover() call site after Ledger::load and implements the
function: membership gate against this boot's discovered channel_ids,
chunked REST id-set fetch under a global 60s deadline, per-event
reconciliation (signature verify + requested-id check + h-tag match)
mirroring the canvas-fetch pattern in pool.rs, import_recovered in
global admission_seq order with cap promotion, unresolved barrier
registration, and a single ledger commit.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… (Task 3 complete)

Live-admission seam: incoming relay events that match a
boot-recovered suppression id are suppressed (already queued via
import_recovered). Events matching an unresolved ledger record are
resolved in-place via admit_recovered → resolve_unresolved
(consume-after-ownership ordering). Both paths skip try_native_steer
(R6-F2 steer short-circuit).

Barrier-deadline timer arm: a new select! branch sleeps until the
earliest unresolved barrier deadline. On fire it expires due barriers,
syncs dirty channels, and calls dispatch_pending — ensuring held
suffix events dispatch even when all external inputs are silent
(the rev 6.1 quiet-harness binding requirement).
@wpfleger96
wpfleger96 requested a review from a team as a code owner July 20, 2026 16:29
…ce, chunk reconciliation, barrier expiry

Covers all plan line 246 integration scenarios: boot recovery
resume/empty, suppression both orderings, REST-fail resolve/resurrection,
unresolved retention, shutdown-grace success/fail/panic (both dedup modes),
large-ledger budget, chunk reconciliation (valid/wrong-channel/duplicate/
unrequested), steer-path short-circuit, quiet-harness barrier expiry
(both timer arm and select! wake), and ledger-commit persistence.
@wpfleger96
wpfleger96 force-pushed the duncan/acp-resume-ledger branch from 8d72643 to a17863c Compare July 20, 2026 16:54
…es (#1990)

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
@wpfleger96
wpfleger96 marked this pull request as draft July 20, 2026 16:57
baxen and others added 14 commits July 20, 2026 09:59
)

Signed-off-by: Bradley Axen <baxen@squareup.com>
Signed-off-by: npub19x6jnl6rhepymwyl2xlltz3ce7rfg2ktllle3g2vu59n3s490k8s9n40l3 <29b529ff43be424db89f51bff58a38cf86942acbffff98a14ce50b38c2a57d8f@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: npub19x6jnl6rhepymwyl2xlltz3ce7rfg2ktllle3g2vu59n3s490k8s9n40l3 <29b529ff43be424db89f51bff58a38cf86942acbffff98a14ce50b38c2a57d8f@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub19x6jnl6rhepymwyl2xlltz3ce7rfg2ktllle3g2vu59n3s490k8s9n40l3 <29b529ff43be424db89f51bff58a38cf86942acbffff98a14ce50b38c2a57d8f@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub19x6jnl6rhepymwyl2xlltz3ce7rfg2ktllle3g2vu59n3s490k8s9n40l3 <29b529ff43be424db89f51bff58a38cf86942acbffff98a14ce50b38c2a57d8f@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Bradley Axen <baxen@squareup.com>
Signed-off-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Pinky <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
…al, activity-aware requeue, LLM stall surfacing (#2175)
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Signed-off-by: npub1gjuws2a2dc8z2nszprtg7v6u9q7ffeah3hgl5yx45jwn7y7aqs6s5e9xj6 <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Pinky <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co>
tlongwell-block and others added 20 commits July 20, 2026 19:22
Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1ux8n2yfs8qfvgd75s7kyhar2mztac355v6vmrz4juc9l3msw4pgstums9e <e18f3511303812c437d487ac4bf46ad897dc46946699b18ab2e60bf8ee0ea851@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1ux8n2yfs8qfvgd75s7kyhar2mztac355v6vmrz4juc9l3msw4pgstums9e <e18f3511303812c437d487ac4bf46ad897dc46946699b18ab2e60bf8ee0ea851@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1ft62tztwwm2x9xamk25smmuaj4sfckdkldksruf2x2jwqalffkrq0g7arr <4af4a5896e76d4629bbbb2a90def9d95609c59b6fb6d01f12a32a4e077e94d86@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1rf6fvdj6ut0c4kcmjv4p5mmgh89nj58n69uu3fz3cvk3jn500hqs7emz79 <1a7496365ae2df8adb1b932a1a6f68b9cb3950f3d179c8a451c32d194e8f7dc1@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1rf6fvdj6ut0c4kcmjv4p5mmgh89nj58n69uu3fz3cvk3jn500hqs7emz79 <1a7496365ae2df8adb1b932a1a6f68b9cb3950f3d179c8a451c32d194e8f7dc1@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1rf6fvdj6ut0c4kcmjv4p5mmgh89nj58n69uu3fz3cvk3jn500hqs7emz79 <1a7496365ae2df8adb1b932a1a6f68b9cb3950f3d179c8a451c32d194e8f7dc1@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1rf6fvdj6ut0c4kcmjv4p5mmgh89nj58n69uu3fz3cvk3jn500hqs7emz79 <1a7496365ae2df8adb1b932a1a6f68b9cb3950f3d179c8a451c32d194e8f7dc1@sprout-oss.stage.blox.sqprod.co>
#2207)

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
…n/main

Integrates the full boot-recovery/ledger subsystem (a17863c) onto
origin/main (daf1f40). Semantic conflict resolution:

lib.rs: kept origin/main's handle_prompt_result with the new
TimeoutKind::Hard { recently_active: bool } split (#2175).  Updated
classify_and_complete_batch (the shutdown-drain seam) to handle both
recently_active variants with matching retry/dead-letter semantics.
Updated three BatchEvent initializers in #2175 tests to include the
four admission/recovery fields added by our boot-recovery work.

queue.rs: took both sides — origin/main's extend_in_flight_deadline
and its tests (#2175 steer renewal) + our set_next_admission_seq,
recoverable_triggers, take_dirty_channels, and their tests.  Made
requeue/requeue_as_cancelled pub to match origin/main (handle_prompt_
result calls them directly in the main loop; classify_and_complete_
batch calls them via complete_batch — both usages are valid).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ership gate

When every staged channel was absent from the current boot's membership,
boot_recover returned early without writing the ledger — stale records
survived on disk and could resurrect on a later boot after membership
re-add. Remove the early return so the empty set falls through to the
single transactional commit, durably clearing the stale snapshot.
…LI opt-out

F1: dispatch_pending() after boot recovery before main loop so recovered
work dispatches immediately on a quiet harness instead of waiting 30s.

F2: compute epoch promotion per channel from ALL gated records before the
fetch split, not just the fetched subset. Apply the settled class to both
fetched imports and unresolved records so the cap invariant holds through
resolution and reload.

F3: separate fetch_deadline (REST budget) from barrier_deadline (created
after reconciliation). A slow bridge no longer consumes the unresolved
barrier's resolution window.

F4: make --resume-on-restart a value-setting flag (ArgAction::Set +
num_args 0..=1 + default_missing_value) so --resume-on-restart=false
works. Bare flag and env var behavior preserved.
Removes import_recovered's internal promotion re-derivation (P2-F2
residual): boot_recover is now the sole computer of the whole-snapshot
epoch class, applying it to both fetched and unresolved records before
the fetch split; import_recovered takes cap_exempt verbatim from the
caller. Adds the exempt-unfetched/counted-fetched discriminating test
(red on the reverted internal rule).

Reworks three P2-F5 manufactured tests to drive real production seams
instead of mirroring them: the quiet-boot dispatch test now calls
dispatch_pending() and asserts a claimed batch instead of only
has_flushable_work(); the barrier-deadline test uses a real 50s mock-
server delay instead of paused virtual time (std::time::Instant cannot
observe a virtual-clock advance); the timer-arm test drops start_paused
and drives expire_due_unresolved_barriers on the real clock through to
dispatch instead of handing it a synthetic past_deadline. Also makes
resume_on_restart_env_false_disables actually set/clear the env var
(previously exercised only the CLI flag as a proxy), serialized via a
new RESUME_ENV_LOCK against the only other env-observing resume test.

Adds the remaining plan:244/246 binding coverage: chunk-reconciliation
invalid-signature and omitted-id cases, and a suppression mode/skew-
window matrix.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Extract admit_live_event and drain_shutdown_grace as standalone fns
so tests can drive the exact production code paths rather than
hand-rolling queue/ledger calls with hardcoded outcomes.

Steer tests (3):
- test_unresolved_resolution_skips_steer_path: drives admit_live_event
  with a pre-loaded unresolved ledger record; asserts accepted=true,
  skip_steer=true, record consumed.
- test_live_event_with_no_unresolved_record_takes_steer_path: companion
  that asserts skip_steer=false for an ordinary live event, closing the
  mutation gap where always-true skip_steer would go undetected.
- test_recovered_suppressed_event_skips_steer_and_is_not_readmitted:
  verifies suppressed duplicate is not re-queued and still skips steer.

Shutdown-drain tests (4): all four drive the real drain_shutdown_grace
select! loop instead of hand-calling queue.complete_batch(). Two
deliver a PromptResult through pool.result_tx and assert trigger
retention/clearing; two abort via join_set.abort_all and assert
Queue/Drop-mode panic disposition. Red-on-old confirmed for result
arm: null-batch mutation failed test_failed_shutdown_grace_result_
persists_triggers. Panic-arm mutation proofs in prior commit.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…esult

The e45d408 merge-commit resolution kept origin/main's inline requeue
block in handle_prompt_result instead of restoring the branch's single
classify_and_complete_batch call. The inline block calls requeue() and
requeue_as_cancelled() directly but never calls mark_complete — so
every channel stayed in in_flight_channels permanently after its first
turn, and flush_next skipped it forever.

Fix:
- Delete the inline requeue block; restore the single
  classify_and_complete_batch(queue, config, &mut result, ...) call.
  It already handles all dispositions (Dropped/Cancelled/Retry/Success)
  through complete_batch and performs the mark_complete release via the
  complete_batch(*ch, None, Success) arm for batchless channel results.
- Have classify_and_complete_batch return Option<&'static str> (the
  hard-timeout fate suffix) so handle_prompt_result's death_message
  construction keeps the #2175 recently_active wording. All three
  hard_timeout_recently_active fate-suffix tests stay green.
- Re-privatize requeue/requeue_as_cancelled (doc comments now true
  again; P3-F1 restored: one atomic completion writer).
- Add paul_probe_channel_released_after_ok_completion regression test:
  push -> flush_next -> Ok completion via handle_prompt_result ->
  assert channel released + second event dispatches.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
P3-F1 (barrier-aware skip_steer): add has_active_unresolved_barrier()
to EventQueue; admit_live_event's ordinary branch now returns
skip_steer = true while the channel's barrier is armed. Fresh live
events queue normally behind the hole rather than bypassing it through
the native steer side-door.

P3-F2 (sync-before-exit): three main-loop exit paths broke before
dispatch_pending's ledger sync call. All three now call sync_dirty
before break: handle_prompt_result -> Exit, drain_ready_join_results
-> Exit, and the all-agents-dead panic path. Prevents terminated work
from being resurrected on the next boot.

P3-F3 (persist success observable): sync()/commit()/invalidate_channel()
now stage changes in a candidate map and advance last_written only after
persist_candidate() returns true. A failed temp-write or rename leaves
last_written unchanged so the skip-identical check does not suppress the
healing retry.

P3-F4 (persisted prompt_tag): admit_live_event's unresolved-resolution
branch now passes unresolved.prompt_tag to from_recovered instead of
the live relay delivery's tag. A config or rule-priority change between
original admission and restart cannot alter recovery framing.

Discriminating tests (all red-on-old verified):
- test_live_event_skips_steer_while_barrier_armed (P3-F1 barrier gate)
- test_live_event_steers_after_barrier_expires (companion: no barrier)
- test_exit_via_handle_prompt_result_syncs_ledger_before_break (P3-F2)
- test_failed_persist_does_not_advance_last_written_allowing_retry (P3-F3)
- test_unresolved_resolution_uses_persisted_prompt_tag_not_live_tag (P3-F4)

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Switches P3-F3 from advance-on-success to a dirty flag, fixing a
regression in invalidate_channel: with advance-on-success, a failed
invalidation persist left the removed channel in last_written, and every
subsequent successful sync wrote it back — removal intent was permanently
lost until reboot.

The dirty-flag approach restores unconditional last_written mutation
(memory always reflects intent), keeps persist_candidate returning bool,
and sets self.dirty = !persist on failure. sync()'s skip-identical guard
becomes !changed && !self.dirty — guaranteeing the healing retry fires
on the next sync of any channel.

Changes:
- Add dirty: bool field to Ledger (false in all three constructors)
- invalidate_channel: remove from last_written unconditionally, then
  set dirty = !persist
- sync: mutate last_written before persist, use !changed && !self.dirty
  as the skip guard
- commit: update last_written before persist, set dirty on failure
- Add test_invalidate_channel_failed_persist_heals_on_subsequent_sync:
  blocked invalidation persist -> unblock -> sync different channel ->
  reload -> removed channel absent (red-on-old confirmed)
- Reword test_exit_via_handle_prompt_result_syncs_ledger_before_break
  comment: plainly states it is a non-discriminating model of the
  break-site contract, not a red-on-old mutation of production code

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants