Skip to content

sl/i178 cleanup merged worktrees false leaves a - #273

Merged
willprout merged 6 commits into
mainfrom
sl/i178-cleanup-merged-worktrees-false-leaves-a
Jul 21, 2026
Merged

sl/i178 cleanup merged worktrees false leaves a#273
willprout merged 6 commits into
mainfrom
sl/i178-cleanup-merged-worktrees-false-leaves-a

Conversation

@willprout

Copy link
Copy Markdown
Owner

Closes #178

cleanup_merged_worktrees is named for WORKTREES — an operator keeping the checkout to inspect it — but it was ANDed with auto_close_merged_windows (#168) to gate the whole merged-lane teardown. Setting it false skipped teardown entirely, so nothing closed the session and no pending_teardown marker was written either, so nothing ever retried.

Before #155 that was untidy but inert: only a FINISHED lane idling at the prompt could reach absorb_merged. #155 lets it fire on an IN-FLIGHT lane — a PR merged out of band while the worker is still building — and there the builder is left running against a branch that has already landed, free to commit and push to it and open a fresh PR nobody asked for, while local status already reads merged, the lane is freed, and nothing is watching.

What changed

_auto_close_merged() becomes _settle_merged_lane(iid) at all three call sites (_exec_merge, _exec_absorb_merged, _exec_absorb_close), asking one question each:

knob question
auto_close_merged_windows (default true) may the SESSION be ended?
cleanup_merged_worktrees (default true) may the CHECKOUT then be pruned?

The #168 property that motivated the AND survives: an operator who sets EITHER knob to keep a finished checkout still gets it kept — window knob off means no teardown at all, so the worktree persists too (a prune can never run under the live CLI it would leave open, #149).

Two review findings shaped the second commit:

  • The keep-the-checkout path must still OBSERVE the builder die. _close_pane is best-effort by contract, so without a wait a failed close would be recorded as a session ended while the pane markers tidy needs and the lock the liveness tiers read were deleted behind its back. _teardown_session grew an await_exit seam (defaulting to remove_worktree, so the D4 relaunch close is unchanged); a decline there clears nothing and writes no marker.
  • A stale pending_teardown marker from an earlier decline on the same lane would have let the drain — which retries with remove_worktree=True unconditionally — prune the kept checkout one tick after the settle cleared the lock, with no live pid left to refuse it. The settle now retires that marker when the prune knob is off.

Evidence

Full engine suite green (2544 passed). Six new unit tests plus a simulation test that drives real runner ticks against a real git repo and a real worker process: with cleanup_merged_worktrees: false, an out-of-band merge of a building lane now kills the builder (verified against the OS, not a recorded call) and keeps the checkout. Details, including a standalone before/after drive of the real Runner over a real git worktree, are in the report.

willprout and others added 4 commits July 21, 2026 03:38
…ilder

The merged-lane teardown gated BOTH the window close and the worktree prune on
`auto_close_merged_windows AND cleanup_merged_worktrees` (#168). So a knob whose
name is about WORKTREES — an operator keeping the checkout to inspect it — also
meant "leave the coding CLI running": with no teardown at all nothing closed the
session, and no pending_teardown marker was written either, so nothing retried.

Before #155 that was untidy but inert (only a FINISHED lane idling at the prompt
could reach absorb_merged). #155 lets absorb_merged fire on an IN-FLIGHT lane — a
PR merged out of band while the worker is still building — and there the worker is
left ACTIVELY BUILDING against a branch that already merged: free to commit and
push to it and open a fresh PR nobody asked for, while local status already reads
`merged`, the lane is freed, and nothing is watching.

Split the pair into one question each, in one place (_settle_merged_lane, replacing
the _auto_close_merged predicate at all three call sites — merge, absorb_merged,
absorb_close):

  auto_close_merged_windows  may the SESSION be ended?
  cleanup_merged_worktrees   may the CHECKOUT then be pruned?

The #168 property that motivated the AND survives: an operator who sets EITHER knob
to keep a finished checkout still gets it kept — window knob off means no teardown,
so the worktree persists too (a prune can never run under the live CLI it would
leave open, #149).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
P0 — a stale declined-prune marker outlived the keep-the-checkout settle. The drain
retries every pending_teardown marker with remove_worktree=True unconditionally, so a
marker written by an EARLIER decline on the lane (a re-approval that could not clear
it, the opt-in parked reaper) would prune the kept checkout one tick after the settle
cleared the lock — and prune it with no live pid left to refuse it, which is the #149
guarantee itself. The settle now retires the retry vehicle when the prune knob is off.

P1 — the keep-the-checkout path never observed the builder die, yet deleted every
handle to it. _close_pane is best-effort by contract (rc ignored, a missing pane record
a silent no-op), so a close that did not kill the worker was recorded as a session
ended while the pane markers `tidy` needs and the lock the liveness tiers read were
removed behind its back: the survivor became invisible AND unreachable. _teardown_session
grows an `await_exit` seam (defaulting to remove_worktree, so the D4 relaunch close is
unchanged) and the settle passes await_exit=True. A decline there writes NO deferral
marker — the drain's retry is a prune, the one thing that config forbids — and clears
nothing, so the window, its markers and the lock survive and tidy remains the owner's
working escape hatch.

Both are covered by regression tests that fail against the mutated fix. The existing
#178 unit tests now model the close actually killing the worker (_pid_alive flips on
close-surface) instead of stubbing a pid that never dies.

P2s: ADOPTING.md typo; the sim test reads closed.jsonl defensively so a regression
asserts instead of dying with FileNotFoundError.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
P1-1 — the split was half-fixed. `auto_close_merged_windows: false` took an early
return that dropped nothing, so a pending_teardown marker left by an earlier decline
survived the settle and the drain — which retried with a bare remove_worktree=True —
closed the window AND pruned the checkout that knob exists to preserve, one tick later
and unguarded (merged is not in REAPPROVAL_STATUSES, so #190's dirty/unpushed refusal
never ran either). Pre-existing under the old ANDed predicate, but this branch newly
asserts the opposite in runner.py, config.py and ADOPTING.md, and it is the exact hazard
the diff already fixed for the sibling knob.

P1-2 — a declined close on the keep-the-checkout path left the #178 harm running with
zero evidence anywhere: nothing retried (the drain is marker-driven and no marker was
written), nothing journaled, no ladder charged. The builder kept building against a
merged branch until a human noticed. That made the keep-checkout config strictly weaker
than the default one, which re-issues the close every tick until the pid goes.

Both close the same way: _drain_pending_teardowns now retries a MERGED lane through
_settle_merged_lane rather than a bare teardown, so the retry is knob-gated like the
settle that produced it, and the marker becomes the settle's to keep or drop — dropped
when the window knob is off or the settle succeeded, WRITTEN when the close was owed and
declined. Under the shipped defaults the drain makes the identical call it made before.

P2s: `wait = remove_worktree or bool(await_exit)` so await_exit can only ever ADD a wait
(remove_worktree=True + await_exit=False — a prune with no liveness check — is now
inexpressible); _reclaim_held popped wherever a marker is dropped without a prune,
matching the drain's own rule; the D4 "does not wait" docstring notes the await_exit
exception; the N x WORKER_EXIT_TIMEOUT tick-latency bound is stated.

Three regression tests, each verified to fail against the mutated fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
Third fresh-agent review, P0 + P1 — and the finding lands on the tick, not on the
settle. tick() snapshots issues.json BEFORE decide and hands that same snapshot to
_reclaim_terminal_worktrees / _drain_pending_teardowns AFTER the executors run. The
executors have just moved statuses and written the very markers the drain reads, so
the drain was judging every lane by a status one phase out of date.

Two concrete harms, both reachable on the tick the settle deferred:

- The owed CLOSE this branch newly records was deleted by the same tick that wrote it.
  The lane still read `running` in the snapshot, so the drain took its "back in flight"
  branch and dropped the marker — the retry the #149 safety argument rests on, gone on
  creation, and nothing else re-emits for a lane already settled to `merged`. (The
  default-path variant of this — _exec_merge's own declined-prune marker — is
  pre-existing on main.)
- Worse, and NEW to this branch: absorb_close settles a park-family lane to `merged`,
  but the snapshot still said `bounced`, so the drain took the park-family branch and
  pruned with remove_worktree=True the checkout `cleanup_merged_worktrees: false` had
  just asked to keep. #190's guard saves only a dirty tree, never a clean pushed one.

One re-read, once per tick, off the hot path: both sweeps now take self._load_state().

Both regression tests drive the REAL tick (only decide is stubbed, to emit one action)
rather than imitating its ordering — an earlier draft hand-rolled the sequence, passed
fresh state itself, and so failed to catch the mutation at all. Both fail against the
stale snapshot.

P2: the drain's "a MERGED lane never guards" comment was false for the absorb_close
route, whose work lives nowhere but that checkout; reworded, with the underlying
question filed separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
Fourth fresh-agent review, P1 — no P0. The retry this branch added for a declined
keep-checkout close was correct but SILENT: _defer_teardown writes a marker file and
nothing else, and the executors return "ok" regardless (correctly — what they report
is that the merge landed). So a builder that outlived its close, still standing in a
checkout whose branch is already on the mainline and free to keep pushing to it,
appeared nowhere: not in the journal, not in the report, not on the dashboard. The
docstring claimed the opposite.

_hold_close journals it once per (iid, pid) — the sibling of _hold_reclaim, bounded the
same way because the drain re-enters the settle every tick — and the entry is dropped
when the lane finally settles, so a later episode journals afresh. No ladder is charged:
charging one would park healthy lanes for finishing normally, the same reason the other
disk-hygiene deferrals do not charge.

P2s: the reclaim half of the post-execute re-read is now pinned by its own test (it was
possible to revert that argument alone and keep the suite green); the comment states the
two effects it has beyond the bug it fixes — the opt-in parked reaper now reaps a lane
parked by this tick's own executor rather than next tick's, and the two rebuild
executors lose one same-tick retry of a git removal that merely failed; and the
"settle succeeded -> nothing is owed" bullet no longer overstates the prune path, whose
marker _teardown_session re-writes on a failed removal and which is not this method's
to touch.

Two adjacent pre-existing defects the reviews surfaced are filed rather than fixed here:
#274 (absorb_close prunes a park-family checkout without #190's guard) and #275
(close_investigate settles to merged but never tears down).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
…led'

Final review P2. `_lock_pid` maps an absent, empty or garbage lock to None — and so does
`.get()` on an id never held. The dedup's `.get(iid) == pid` therefore read "already
journaled" for a lane journaled nowhere, and recorded nothing either, in the window
between the settle's decline and its re-read of the lock: exactly when the worker's own
EXIT trap frees it. Inert in outcome (the next drain finds no lock and retires the lane
cleanly), but it is a sentinel collision the sibling _hold_reclaim cannot hit, so its
shape offered no protection. Test drives that window and fails against the bare .get().

Also filed #276 for the pre-existing neighbour the reviews surfaced: _publish_view still
renders the pre-execute snapshot, so a lane that merged this tick shows OPEN until the
next one. Not fixed here — ist_map also feeds carry_titles/carry_prs, so moving it is a
wider change than it looks, and it should not ride a teardown-semantics PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FLi6MF3Z3rVjrY3PCgtKpP
@willprout

Copy link
Copy Markdown
Owner Author

Fresh-agent review verdict — CLEAN, no open findings at any severity

Reviewed: the cumulative diff origin/main (bc1ad95)...ed4d885 — all six commits, by agents that wrote none of it. Four review rounds, each re-deriving the delta from scratch rather than taking prior findings on trust. Two independent fresh reviewers were used across the rounds; neither authored any of the code.

What was reviewed: skills/superlooper/skill/bin/runner.py (_settle_merged_lane replacing the _auto_close_merged predicate at all three executor call sites plus the drain; the await_exit seam on _teardown_session; the pending_teardown marker lifecycle; _hold_close; the post-execute re-read feeding both hygiene sweeps in tick()), skill/lib/config.py, skill/docs/ADOPTING.md, and the tests in tests/test_runner.py + tests/test_simulation.py.

P0/P1 outcome: four P0/P1 findings were raised across the rounds and all are fixed, each pinned by a regression test verified to fail against the mutated fix:

# Severity Finding Fixed in
1 P0 A stale declined-prune marker let the drain prune the kept checkout one tick after the settle cleared the lock — unguarded, with no live pid left to refuse it 7e83b6a
2 P1 The keep-the-checkout path never observed the builder die, yet deleted every handle to it (tidy's pane markers, the liveness lock), making a failed close silent and unrecoverable 7e83b6a
3 P1 The same stale-marker hazard on the auto_close_merged_windows: false early return — the split was half-fixed, and a declined close had no retry or evidence at all 85a4040
4 P0+P1 tick() handed both hygiene sweeps a pre-execute snapshot, so the owed-close marker was eaten by the very tick that wrote it — and an absorb_close lane still reading bounced routed the kept checkout into an unguarded prune 878d63b

Remaining P2s were addressed in 57084d2 (the residual harm — a builder that outlived its close — is now journaled once per episode with the pid that makes it actionable) and ed4d885 (a sentinel collision where an unreadable or trap-freed lock read as "already journaled").

Final verdict, quoted:

I reviewed head ed4d885 as a fresh reviewer across four rounds, re-deriving each delta from scratch rather than taking prior findings on trust. The branch does what issue #178 asks: cleanup_merged_worktrees is now strictly about the checkout, the window question is asked once by auto_close_merged_windows alone, and the keep-the-checkout path observes the builder's pid go rather than assuming it — which matters precisely because _close_pane is best-effort by contract. The riskiest change, moving both hygiene sweeps onto a post-execute read of issues.json, is sound in both directions: the fresher read is strictly more current, so the drain's "an id could have been re-approved between the decline and the retry" safety argument is strengthened rather than weakened, and I confirmed by reachability analysis that no lane can carry a pending_teardown marker across a non-terminal-to-park-family transition inside a single tick, which makes that half inert for non-merged lanes in practice. Its two real side effects — the opt-in parked reaper acting a tick earlier, still behind the live-pid gate and #190's dirty/unpushed guard, and the two rebuild executors losing one same-tick retry of a merely-failed git removal — I measured independently and both are now stated at the call site. Every finding I raised was fixed and each fix is mutation-tested: reverting fresh to the stale snapshot fails three tests, reverting the reclaim sweep's argument alone fails one, and the _hold_close journal line fails four different ways when its guard, its call, its retirement pop, or its iid in membership test is tampered with. The final commit closes the last one — a sentinel collision where an unreadable or trap-freed lock read as "already journaled" — and its test drives the genuine decline sequence rather than the degenerate one an earlier draft would have. The residual harm this whole path exists to bound, a builder that outlived its close and is still standing in a checkout whose branch is already on the mainline, is now visible in the journal with the pid that makes it actionable, bounded to one line per episode. Engine suites and the real-process simulation are green in my copy, the adjacent staleness in _publish_view is filed as #276 with its reasoning rather than left to be read as an oversight, and I have no open findings at any severity. Ready to merge.

Out-of-scope defects the reviews surfaced, filed rather than fixed here: #274 (absorb_close prunes a park-family checkout without #190's unsaved-work guard), #275 (close_investigate settles to merged but never tears down), #276 (_publish_view renders the pre-execute snapshot). All pre-existing on main, all labelled needs-owner with parent: #178.

Evidence: full engine suite green at 2551 passed; CI tests green on this head (8m27s); and the changed behaviour was driven end-to-end outside pytest against the real Runner over a real git worktree with a real detached worker process — before the fix the builder was left STILL RUNNING with local status already merged, after it the builder is DEAD and the checkout kept. Details in reports/i178.md.

@willprout
willprout merged commit a9cf614 into main Jul 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cleanup_merged_worktrees:false leaves a LIVE builder running after an absorbed merge

1 participant