docs(rules): five multi-agent lessons, with evidence evicted to the archive - #313
Merged
Conversation
…NCH on it is Rebased onto current main. The original branch (2026-08-03) carried nine core rules and five archive anchors; eight of the nine have since landed on main via other PRs, and all five anchors would now be ORPHANS — main's core routes to 30 anchors, none of them these, and it already archives the same evidence under count-not-exit-code, worktree-envrc, isolation-seam and stale-observation. What survives the rebase is the one rule that did not land elsewhere: a type declaration is not a code path. Measured — a payload's notOwned was declared in the consumer's own type and never read, so the not-owned case rendered a zeroed dashboard as measured data, and 'set the field correctly' would have changed nothing on screen. +431 B; headroom 1,060 -> 629 against the 33,600 usable budget. Gate: 5 passed.
ZacxDev
force-pushed
the
zach/rules-multiagent-lessons
branch
from
August 5, 2026 18:19
ea82146 to
32a7b52
Compare
Contributor
Author
|
Rebased onto current This branch was opened 2026-08-03. Eight of the nine core rules have already landed via other PRs — All five archive anchors would now be orphans. What survives is the one genuinely new rule: a field that exists in a DTO is not a guard — only a BRANCH on it is. That one is not on Numbers:
Force-pushed with |
ZacxDev
added a commit
that referenced
this pull request
Aug 13, 2026
…m a merged branch (#447) * docs(rules): two multi-agent lessons that never landed, recovered from a merged branch `zach/rules-multiagent-lessons` merged as #313, so it read as fully absorbed. It scores 1% when measured by added-lines-present-in-main -- which means nothing for RULES.md, a file that is continuously reworded. Measured by ARCHIVE ANCHOR instead, 20 of its 22 lessons are in main and two are in neither RULES.md nor RULES-ARCHIVE.md: - cross-repo-worktree: `isolation: "worktree"` builds the worktree from the dispatching session's CURRENT repo, not the repo the task names. Cross-repo dispatch silently gives every agent a worktree of the wrong repo, with no error. - sibling-agent-kill: under a parallel fan-out, confirming `/proc/<pid>/cmdline` is not sufficient before a kill -- a sibling agent running the same suite matches it legitimately. Filter by `/proc/<pid>/cwd` to your own worktree too. Found while diffing the 23 preserved worktrees for keep-or-drop (verdict table in claudedocs/handoff-subsystem-store.md, landed in #446). Every other worktree was absorbed or is deliberately open (#355); this was the single real loss, and the "superseded intermediate" guess would have dropped it. RULES.md 31,726 -> 32,727 bytes, against a 35,200 ceiling with 900 required headroom -- fits with 1,573 to spare, so no eviction was needed. Numbers read from scripts/tests/test_rules_size.py, which owns them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): restore the measurements the first round paraphrased away Audit of #447 found the two archive sections were REWRITES, not recoveries. They dropped exactly what an archive exists to carry: `Measured 2026-08-02`, the repo names (`civit/civitai` dispatched from `civit/cli`), the kill pattern (`chrome-headless-shell|vitest|steam-run`, system-wide, ~15 PIDs), and n=2. Both sections are now ea82146's text verbatim, plus a remedy paragraph and the cross-links. Two invented sentences that read like evidence are gone -- "the loop can run several times before anyone asks" is contradicted by the n=2 measurement (one agent refused, one self-recovered; there was no loop). Also restores the SILENT failure mode the paraphrase lost. cross-repo-worktree has two, and the first round taught only the loud one: an agent may instead self-recover by quietly building its own worktree in the right repo, leaving the dispatcher's model of where the work happened wrong. A rule recovered and then narrowed is worse than the rule it replaced. Corrections to the claims themselves, both measured on the workbench today and neither present in the 2026-08-02 write-up: - `/proc/<pid>/cwd` is not sufficient as stated. This harness nests agent worktrees at `<dispatching-repo>/.claude/worktrees/agent-<id>`, so a prefix check against the repo root matches every sibling and discriminates nothing -- compare the EXACT path. And agents that share the base clone (the read-only ones, which the adjacent rule tells you not to give a worktree) have identical cwds, so the discriminator does not exist for them at all -- which is what the perpetrator of the original incident was. - The inference "confirming `cmdline` is not sufficient" was never measured: the auditor pattern-matched system-wide rather than running resolve-then-confirm. The widening is sound but it is reasoning, and is now labelled as such. - cross-repo-worktree is an 11-day-old harness measurement re-landed without re-verification. Labelled, with the structural corroboration that does hold today: the Agent tool schema exposes no target-repo parameter. Reconciles the new bullet with the "Mandatory" isolation rule directly above it -- having the agent make its own worktree still satisfies that mandate, so it is not an exception. Adds the missing `worktree-not-session` TOC entry (pre-existing gap) and moves `sibling-agent-kill` to the git/worktree TOC cluster where its section actually lives. RULES.md 32,727 -> 33,289 against the 34,300 budget; 1,011 spare. All 5 tests in scripts/tests/test_rules_size.py pass, including test_every_archive_pointer_resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): the cwd caveat I added was wrong about its own incident Delta re-audit of ff9589b caught an analytical error the previous fix round introduced -- not a restoration defect; the recovered text is faithful. Caveat (b) said the discriminator "does not exist" for base-clone-sharing agents "-- and the perpetrator above was an auditor", which reads as: cwd filtering would not have saved the recorded incident. That is false, and caveat (a) two sentences earlier already implied so. The retained ea82146 text says the victim died "in another worktree of the same repo". Perpetrator in the base clone, victim in a worktree -> the two cwds DIFFER -> exact-path comparison spares the victim. The prefix check is what kills it, which is exactly what (a) diagnoses. So (b)'s real scope is narrower than I stated: cwd fails only between two agents that BOTH sit in the base clone. Corrected, along with the premise it leaned on -- RULES.md:94 says read-only agents "don't need one", which is permissive, and I had paraphrased it as a prohibition. The consequence was worse than the overstatement. RULES.md:104 read "there, don't kill by pattern at all", aimed at read-only agents in the base clone -- i.e. at precisely the actor in the incident, whose stated need was to clear ITS OWN hung run. A prohibition with no substitute, for the one case the archive entry exists to describe. Replaced with a filter that is deterministic and strictly stronger than cwd in every case including that one: kill only your own descendants, walking PPid in /proc/<pid>/status back to your own pid, or track the pid you launched. Verified PPid is populated on this host. The same over-reach was published in the PR comment; a follow-up comment corrects it. RULES.md 33,289 -> 33,356 against the 34,300 budget; 944 spare. All 5 tests in scripts/tests/test_rules_size.py pass. Those tests were negative-controlled during the re-audit -- forcing the ceiling under the current size, the headroom above it, a missing file and a dangling anchor each turned the matching test red with its own message -- so the 5/5 is not vacuous. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): the descendant filter I offered as a fix returns EMPTY for the case it was for Round-3 delta audit found a 🔴 in fe315e4: I replaced a fail-safe prohibition with a procedure that does not reach its targets. Verified both decisive claims myself rather than taking the audit's word. - MEASURED: nix-daemon workers are children of nix-daemon (PID 18717, PPid 1). A `nix build` an agent launches does its real work in a process that is NOT the agent's descendant, so a PPid walk returns the empty set for exactly the hung-run case sibling-agent-kill is about. Same for anything that daemonizes or re-parents -- including Chrome's zygote, and the original incident's pattern was `chrome-headless-shell`. A walk is also per-tool-call: each Bash call is a fresh `zsh -c`, so an earlier call's strays are not descendants of the current one. - MEASURED: RULES.md:14 mandates killing an orphan "by resolved PID". An orphan is PPid 1 by construction, so it is never your descendant -- line 104's "kill only your own descendants" contradicted it outright. So "strictly stronger in every case" was wrong twice over. cwd and descendant are INCOMPARABLE filters, not ordered. cwd is now primary (it demonstrably separates the recorded pair); the PPid walk only NARROWS a cwd-filtered set and can never build one; orphan hunts are explicitly scoped out to the deploy rule. Restores a fail-safe, which is what the previous round actually removed: if no filter leaves a confident set, kill nothing and hand it to the operator. An empty descendant set cannot distinguish "no strays" from "the walk cannot see them" -- the archive's own empty-result trap, wearing a procedure. Also strikes "the perpetrator sat in the base clone" (RULES-ARCHIVE.md:659). The original never places the perpetrator; that was my inference stated as fact, three sentences above the same paragraph's "it is reasoning, not an observation". The victim-side fact -- the victim was in another worktree -- carries the argument alone. My control last round ("verified PPid is populated") validated that the FIELD exists, not that the procedure reaches the targets. PPid is always populated on Linux. That is the instrument-validation rule's negative control missing: the check needed was whether a hung run from an earlier tool call is reachable by the walk. It is not. ea82146's recovered bodies remain byte-identical (cmp-verified). RULES.md 33,356 -> 33,596 against the 34,300 budget; 704 spare. All 5 tests in scripts/tests/test_rules_size.py pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): evict my own error trail from the core to the archive Review with Zach: the sibling-agent-kill bullet had accreted a REFUTATION of an idea no reader has had yet -- descendant/PPid filtering -- which exists in the core only because I proposed that fix in fe315e4 and had to walk it back in 0679fe0. A reader arriving fresh needs the imperative, not my error trail. The core now carries the three actionable facts: filter by the EXACT worktree path (a prefix matches every sibling, because agent worktrees nest under the base repo); a box-wide pattern reaches a sibling agent's processes, with the incident; and if no filter leaves a confident set, kill nothing and hand it over. Everything evicted is preserved in the archive entry, where it costs nothing until recalled -- that the two filters are incomparable rather than ordered, the measured nix-daemon/PPid-1 re-parenting that makes a descendant walk return empty, the per-tool-call shell, and the scoping of orphan hunts to the deploy rule. Verified `nix-daemon`, `PPid` and `incomparable` now appear in RULES-ARCHIVE.md and NOT in RULES.md. RULES.md 33,596 -> 33,145 against the 34,300 budget; spare goes 704 -> 1,155. All 5 tests in scripts/tests/test_rules_size.py pass. Not addressed here, deliberately: the size module's own comment says the answer to the next real rule is consolidation, not another bump, and this PR takes the worktree-related bullets from four to five. Zach's call was to land the recovery and consolidate separately rather than entangle a refactor with a salvage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): the trim cut one clause too many, and it was the one carrying scope Round-4 delta audit found a 🔴 in 37dbbe8 -- a regression from the eviction I proposed. Verified both findings against the files and live /proc before acting. 1. SCOPE LOSS (blocking). Trimming the bullet dropped every mention of the case cwd CANNOT answer. `grep "base clone"` on bullet 104 returned 0. The core then read as an unqualified "confirm /proc/<pid>/cwd is your OWN worktree -- the EXACT path", framing exact-match as THE fix for the sibling problem. Two read-only agents in the base clone -- a configuration RULES.md:94 explicitly permits -- both match, and the auditor kills the sibling. That is the 2026-08-02 incident verbatim, re-opened. The retained fail-safe does not catch it: "if no filter leaves a set you are confident in" never fires, because the reader IS confident -- the exact-path check matched and nothing left in the core says a match can still be a sibling. This is precisely what scripts/tests/test_rules_size.py:128-133 forbids in bold: trimming a "not just X" qualifier to buy bytes reintroduces the failure mode. The qualifier lived in the archive, which is not auto-loaded. Restored, ~150 B against 1,155 spare. Also restores the 🔴 marker the trim dropped from the fail-safe (:142 lists the priority marker under what stays). 2. UNMEASURED CLAIM INSIDE A "MEASURED" PARAGRAPH (again). The archive asserted a descendant walk misses "anything that daemonizes or re-parents -- including Chrome's zygote". Measured on this host: all Brave --type=zygote processes have real parents (2869800/2869802 <- 2869781; 2869804 <- 2869802); only chrome_crashpad sits at PPid 1. The zygote does NOT re-parent. Struck, and the entry now says so explicitly, because the wrong version was in a paragraph headed "Measured 2026-08-13" and in 0679fe0's commit message under "MEASURED:". The conclusion is unaffected -- the nix-daemon measurement and the per-tool-call mechanism each independently establish the empty walk, and both reproduce. It is the per-tool-call shell, not zygote re-parenting, that actually orphans a chrome-headless-shell from an earlier call; the entry now says that instead. Fourth recurrence in this PR of one error class: stating an inference as an observation. Logged in the entry itself rather than only in a commit message. 3. The archive's `Supports:` line still quoted core text 37dbbe8 had deleted. Requoted. No test covers archive->core quote fidelity (test_every_archive_pointer _resolves only checks core->anchor), so this one is eyes-only. RULES.md 33,145 -> 33,300 against the 34,300 budget; 1,000 spare. All 5 tests in scripts/tests/test_rules_size.py pass, negative-controlled by the auditor on all five plus a positive control on the pointer test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(rules): the nix-daemon premise was measured; the "so" that followed it was not Round-5 delta audit was CLEAN on 37dbbe8..fa1c54f -- first clean round in five -- but found this pre-existing 🟡, introduced back in 0679fe0 and untouched since. Verified live before acting. The paragraph said a `nix build`'s real work runs in a nix-daemon child of PID 1, "so a PPid walk from the agent returns the EMPTY SET". The premise is true; the conclusion does not follow, and measurement refutes it. Sampled while this very gate ran: nix 368350 <- 366150(zsh) <- 328364(.claude-wrapped) <- the CLIENT is a descendant nix-daemon 368389 <- 18717 <- 1 <- the BUILDER is not So the walk returns something -- the client stub -- it just never reaches the process doing the work. Killing what it returns does not stop the build. Emptiness DOES hold, but via the other ground the paragraph already gave: the per-tool-call shell. Measured seven sequential Bash calls, seven distinct shell pids, and an actual orphaned stray from an earlier call (`nix 2085612 <- 1`). The entry now separates the two grounds and states which one yields which conclusion, and records the retraction inline -- because the wrong version sat inside a paragraph headed "Measured 2026-08-13", which is the whole failure mode being catalogued. Fifth instance in this PR of one class: stating a conclusion as measured when only its premise was. The imperative is unchanged and is now better supported than by the reason originally given: use a descendant walk to NARROW a cwd-filtered set, never to build one. Archive-only; RULES.md untouched at 33,300 B, 1,000 spare. All 5 tests in scripts/tests/test_rules_size.py pass. ea82146's recovered body still cmp-identical at 885 bytes. 🔴 Committed from a dedicated worktree at fa1c54f, NOT the base clone: another session checked ~/workspace/devrc out onto main and fast-forwarded it to b2fb680 mid-session. The gate I had just launched "on fa1c54f" resolved its derivation from that working tree and therefore measured MAIN -- collected=9429 against 9267 on every prior run, which was the tell I read past. That result is retracted; this commit's gate runs against the real branch tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Durable lessons measured in one long multi-agent session. Four new rules, one widening.
New rules
isolation: "worktree"builds the worktree from your current repo, not the repo the task names. Two agents dispatched at a TS monorepo while cwd was a Go repo both got Go worktrees — one refused and reported (the good outcome), one silently rebuilt its own. Cross-repo work must not pass the flag.mainadvanced 3 commits, one rewriting 779 lines of the router file both PRs touch, while both PRs reportedMERGEABLEthroughout.notOwnedflag declared in the consumer's own type and never read rendered a fully zeroed dashboard as if it were measured data.prettier --check --stdin-filepath <path>prints(stdin), not the path, so a path-grep reported a false CLEAN. Caught only by a negative control.Widened
The
pkill -frule now covers the parallel-agent case explicitly. A box-wide pattern reaches a sibling agent's processes: an auditor clearing its own hung run killed ~15 PIDs and destroyed another agent's in-flight test run, whose next attempt then collapsed with0 files collectedandexit 144— both readable as code defects rather than as the kill. Filter resolved PIDs by/proc/<pid>/cwdbefore killing.Budget
scripts/tests/test_rules_size.pyleaves 495 B of free budget; these additions are ~1.9 KB. Rather than ratchetMAX_BYTES, ~1.4 KB of evidence moved toclaude/RULES-ARCHIVE.md:cross-repo-worktree,stale-gate-base,count-tests,sibling-agent-kill,cheap-controlparsing-tool-output,base-clone-driftNo rule lost its imperative, priority marker, triggers, widest scope, or procedure. What moved out is worked-incident detail, enumerated ground cases where one representative carries the point, and two ✅/❌ blocks that restated their own bullet verbatim.
RULES.md33,105 → 33,507 B (993 B headroom against a 900 B floor).Verification
nix build .#checks.x86_64-linux.pytests—TOTAL collected=5868 passed=5867 skipped=1 failed=0(floor 5600);scripts/tests collected=1068 passed=1068.test_rules_md_under_hard_ceiling, and a bogus→ archive: no-such-anchortag reddenstest_every_archive_pointer_resolves— each failing on its own assertion, not a neighbour's. The file was restored byte-identically afterwards (cmp).~/.claude/RULES.mdresolves to/nix/store/…-hm_RULES.md(a read-only copy), so this change is inert until ahome-manager switch/ship.sh. Not run here.🤖 Generated with Claude Code
https://claude.ai/code/session_01858ymA3tEJQi83435u7npi