docs: audit the in-flight ledger against reality, then split it so PRs stop colliding - #112
Conversation
The ledger had grown to 677 lines and drifted into being a record of finished work: entries told to "delete when #98 merges" outlived three merges, the rebalance-commit fix was still filed as "awaiting PR" after landing as #100, and the largeOffsetMap static-state flake was still listed as deliberately-unfixed after #101 fixed it at source. Meanwhile the largest thing actually in flight - #80, carrying the drain-zombie fix - appeared only as a footnote about quarantine ownership, and #105/#106/#111 were absent entirely. Every remaining claim is now checked against GitHub, git and the code, and dated. Obsolete entries are deleted rather than annotated: the jscpd cap is above baseline, the docs-only path-filter inconsistency cannot happen (no workflow has path filters), the ManagedPCInstance header carries its Modifications line, and the CommitFailedException follow-up shipped in #108. What remains open is stated with what would close it - the reviewer's missing actionlint/todo-index grants, the ungated stacked-PR path, the undiagnosed MultiInstanceMetricsTest flake. Upstream references now follow the convention the changelog and refactoring backlog already use. Fork numbering reaches #111, so every bare reference at or above #162 was silently upstream and read as if it were ours; those are now written "upstream #NN". The convention, including the trap that branch names encode the upstream number, is stated at the top of the file. Four items moved to docs/refactoring.md, which is where deferred work belongs: the eleven SpotBugs thread-visibility findings (verified still present), the produce-lock double release surfaced by #110, the commitAndWait timeout message that misreports its own wait, and jacoco's single exec file under forked surefire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ync a rule Five entries disagreed with reality. bug-857 recorded no fork PR and a todo to open one, while #29 was open, #100 had merged and #80 was awaiting review - it is a family of three distinct defects behind one upstream symptom, now written that way. fix-909 recorded no PR though #31 is open; bug-912 was "in-progress" when the schema's own word for pushed-but-unPR'd is "ready"; the release entry listed none of #55/#56/#72 and did not say the release has not happened. The manifest is the declared source of truth for this mapping, so drift here is worse than drift in a scratch file - it is what a future session trusts instead of re-deriving. Nothing catches it: upstream-map.py validate only checks the schema, and upstream-sweep.sh only watches upstream, so "prs: []" beside an open fork PR passes every check we have. Hence the AGENTS.md rule: update the entry at every lifecycle transition of your own work, in the commit that causes it, not only when starting. The two wrong-base PRs are now todo lines rather than buried prose, so upstream-map.py todo surfaces them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These paths were only excluded in .git/info/exclude, which is local to one checkout - so every fresh clone, and every agent session, rediscovered 7.6G of session transcripts and 6.9M of tool artifacts sitting untracked. Sharing the rules means nobody has to. .claude is ignored by contents rather than as a directory, so shared config can later be un-ignored with a negation; git will not descend into a fully-excluded directory, which would make that impossible. .compound-engineering holds a local config byte-identical to its own example, and .vscode a single personal Java setting - neither is a project convention. Also removes an empty duplicate-code-cross-check clone from the tree: a git init from April that never fetched anything (no objects, no refs, no commits). The action itself is real and pinned in maven.yml; this was scaffolding. Deleted rather than ignored, so a genuine nested clone would be visible rather than silently swallowed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
✅ Duplicate Code ReportTwo engines run in parallel for cross-validation. Each has its own thresholds tuned to its baseline - the real safety net is the per-engine "max increase vs base" check. ✅ PMD CPD
No new clones introduced by this PR. ✅ jscpd (language-agnostic)
No new clones introduced by this PR. Powered by astubbs/duplicate-code-cross-check |
|
Claude finished @astubbs's task in 5m 35s —— View job Review completeTasks
SummaryThis is an unusually well-verified docs PR. I spot-checked essentially every falsifiable claim and they all held up:
One thing I could not verify: the PR/branch ahead/behind counts in the Minor finding
Both are absent from Everything else — the |
✅ SpotBugs ReportNo bugs found (new bugs only — baseline from base branch excluded). |
🧪🔒 Quarantine Lane Report
🔴 expected while the owner PR is open · 🟡🎲 flapper, pass proves nothing · 🚨 a deterministic quarantined test passing means its fix landed: delete its |
…-and-upstream-map # Conflicts: # docs/inflight.md
|
Claude finished @astubbs's task in 3m 56s —— View job Review completeTasks
SummaryThis push merged Specifically re-checked since the prior automated review:
Not independently verifiable in this sandbox (unchanged from the prior pass): the ahead/behind counts in the Open fork PRs table (e.g. No new issues found. The audit's central claim — that every surviving line was checked against GitHub, git and the code — held up under a second, independent pass focused on what changed in this sync. |
…deferred thread-parallel check The header now says that work your own PR resolves is tracked by that PR, so its entry goes in that PR - never leave a marker on master saying to delete it later. A merge is exactly when nobody is reading this file, so such markers outlive the work: the one #80 left behind survived the merge and read as live afterwards. Also restores an open item this rewrite dropped. #68 made integration reliable by forking per broker, which sidesteps the upstream confluentinc#857 deadlock rather than proving it gone - so re-running with -Dparallel-tests=true on a shared broker, once #29 lands, is still outstanding. Recorded against #29, which is what gates it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 2m 46s —— View job Review completeTasks
SummaryThis third pass covers what changed since the previous two automated reviews: commit The new header rule is well-grounded, not just asserted. I checked whether the "#80 left a marker that survived the merge" claim is actually true:
That's a self-referential marker — impossible to act on, since the commit containing it is the merge it names. It sat on The restored "Step 2" thread-parallel item checks out in full:
No inaccuracies, no dangling references, no duplication with the surrounding Not independently verifiable in this sandbox (same gap as both prior passes): the ahead/behind counts in the "Open fork PRs" table (e.g. Nothing to fix. This audit continues to hold up under independent verification, including the "delete when #NN merges" anti-pattern it just self-corrected. |
The rebuilt ledger opened with a table of every open PR, its branch, its worktree and its ahead/behind counts. That is a second PR tracker: `gh pr list` is always right, this table is wrong within a day, and a reader cannot tell which one they are looking at. Same for branch divergence and worktree paths, which `git` and bin/worktree-status.sh already report. Replaced with the part no command can answer - #29/#31 pointing at the pinned mirror and why the retarget is not mechanical, #38 blocked on a missing ArchUnit engine rather than the version bump, #51 colliding with #57 over PCMetrics, and which PR owns which files. Divergence counts are dropped throughout; what stays is the consequence ("rebase before review"), not the number behind it. Measured flake rates stay - an experiment's results are not re-derivable from a command. The rule is in the header, where the file's other writing rules live. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 1m 58s —— View job Review completeTasks
SummaryThis is a fourth pass, covering only what changed since the last automated review: commit The change itself resolves a gap flagged in all three prior reviews. Each earlier pass noted it could not verify the ahead/behind counts in the old "Open fork PRs" table (e.g. "#57: 28 ahead, 11 behind") because this sandbox has no Verified every new factual claim against GitHub (this session did have
Not independently verifiable in this sandbox: existence of the no-PR branches listed ( No inaccuracies found in the new content. The commit is a clean, well-motivated tightening of the file's own scope rule, and the rationale in its message matches what's actually in the diff. |
The stacked-PR gap is closed: an "All branches: PR dependency gate" ruleset (~ALL, requiring only "Check PR Dependencies") now covers PRs whose base is a feature branch, which the master-only ruleset never matched. Verified live on #112. The concede optimizer is abandoned, not parked - it was removed from #75 by review, re-introduced and dropped again on #80, and highcpu stays purely advisory. A five-point revival checklist for something nobody intends to revive is the "keep it in case" habit this file is meant to resist; the findings survive in #75's and #80's review history if it ever comes back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 1m 46s —— View job Review completeTasks
SummaryFifth pass, scoped to what changed since the last automated review: commit Two entries were deleted; both deletion rationales check out:
No orphaned references: grepped both Not independently verifiable in this sandbox (recurring, unchanged gap): GitHub-side ruleset configuration — No inaccuracies found. This pass's deletions are well-supported by verifiable history rather than assertion. |
… site Two scope points that were only in one person's head. Known defects on the current branch belong in this file even when a GitHub issue exists. An agent picking up work scans this ledger; it does not read every issue on the tracker, so an unrecorded defect is one the next session rediscovers or builds on top of. Link the issue, keep it short. The docs are also heading for a generated site, which removes the include::CHANGELOG.adoc[] coupling that currently publishes the whole changelog as a README chapter. Recorded now so nothing new is built on that coupling, and so changelog decisions stop being driven by it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 1m 11s —— View job Review completeTasks
SummarySixth pass, scoped to what changed since the last automated review: commit This commit adds two scope points to the file, both checked:
Whole-diff sanity check: cumulative diff vs No inaccuracies found in this delta. Across all six passes on this PR, every falsifiable claim I've been able to check locally (file contents, line numbers, |
docs/inflight.md appeared in 26 of the last 30 master commits. Unrelated PRs conflicted on it constantly - not because they disagreed, but because their notes were adjacent, and the merge that resolves such a conflict silently drops one side's update. It happened between #108 and #110. So each item is now its own file under docs/inflight/, named <category>-<slug>.md. Two PRs recording unrelated work touch disjoint files and cannot conflict, and finishing work is `git rm` - a deletion, which never conflicts with an edit elsewhere. The category prefix is the structure: `ls docs/inflight/` shows the shape of what is open without opening anything. No committed index. It would be edited by every PR, which is the problem this directory exists to solve; docs/TODO_INDEX.md is the cautionary case, generated and committed and stale until a reviewer caught it. `ls` and `grep -r` are the index. The rules that lived in the old file's header now live in docs/inflight/AGENTS.md, where an agent will find them: delete a file when its work lands rather than rewriting it into a FIXED narrative, delete it in the PR that resolves it rather than leaving a "delete when #NN merges" marker, never record what gh or git can answer, and record known defects here even when an issue exists. This was parked on docs/inflight-as-directory with migrating ~600 lines of stale entries named as the reason not to do it. This branch's audit already did that work, which is why it is cheap now. That parked note is one of the entries the split drops. docs/refactoring.md deliberately stays a single file - 2 of the last 30 commits touched it, so it has none of this problem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @astubbs's task in 5m 39s —— View job Review completeTasks
SummarySeventh pass, scoped to what changed since the last automated review: commit The rename left two dangling references this pass caught:
A pre-existing factual drift, carried forward unverified into new content — Everything else checked out:
Not independently verifiable in this sandbox (recurring, unchanged gap across all seven passes): |
astubbs
changed the title
Conflict: #112 split docs/inflight.md into docs/inflight/, one file per item, because every PR edited the single file. This branch had rewritten that file, so it conflicted structurally rather than textually. Resolved master's way. The mutation note is now docs/inflight/ci-mutation-testing.md, following the new <category>-<slug> convention and the directory's rules - open items only, no FIXED/DONE narrative, no committed index. Also swept master's own notes for claims this PR invalidates, rather than only resolving my side of the conflict. ci-disabled-jobs-and-runner-load.md said "the highcpu lane runs six suites per branch on one box, including mutation sweeps ... consider moving mutation off-box. Mutation strategy is being reconsidered wholesale in #111" - #111 is this branch, and it did exactly that: the lane is down to two suites, both mutation entries are off-box, and Unit/Integration went with them. Left the runner-lost-communication symptom noted but marked for re-checking, since its stated cause is largely gone. Two references in the plan doc pointed at ledger content that the #112 audit dropped (the arcmutate shelved plan). Rather than leave a dangling pointer, the plan doc now says it is itself the record of that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…from the changelog It was true until this PR removed the include::. README.adoc is still generated - from README_TEMPLATE.adoc - but no longer from CHANGELOG.adoc, so the row contradicted this PR's own Changelog section a few lines below it. The table is the thing a reader skims first, by design, so a stale row there is worse than a stale paragraph. The row now also states the two facts a reader of that table needs: the file is frozen up to 0.6.0.0 and generated from the commit log after it, and it is not a per-PR chore. Found by the review bot on the merge of #112, which is what put the two statements in the same file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… from the commit log (#113) The unreleased section had reached 15 entries and 1,828 words - longer than every 0.5.x release note combined - and every one was Build & CI. Median entry 104 words against 8-15 in the upstream-era sections; the longest, at 316 words, explained that a tool-grant string did prefix matching. That is a postmortem, and postmortems belong in the commit message, the PR body and docs/solutions/, which is where this project already puts them. The problem was audience, not length. A changelog answers one question - should I upgrade, and will anything change for me - so an entry now has to pass a test: can a user or operator observe it without reading our repo? CI lanes, reviewer tool grants, PR checklists and TODO indexes cannot, and are gone. What survives of that work is a single Build & CI entry listing the five things that tell a reader how carefully the library is tested. Surviving entries are one sentence of about 25 words: what a reader would have SEEN, plus who it hits when that is not everyone. The rule names both failure modes, because the upstream-era entries show the other one - "fix: Paused consumption across multiple consumers" is six words from which you cannot tell whether you are affected. Section total: 2,570 words to 569. Two headings that both meant "not released" are collapsed into one. Entries were split between == Unreleased and == 0.6.0.0 while neither had shipped, which is how #80's drain fix - the most user-visible change in the release - ended up filed apart from everything else. Beyond that, a PR now does nothing about the changelog at all. The file appeared in 30 of the last 30 master commits and dragged the generated README.adoc with it, and it forced an entry to cite a PR number that does not exist when the entry is written. Everything up to 0.6.0.0 is frozen; from the next release an agent reads git log <last-tag>..HEAD and drafts the section, and a human re-applies the same judgement before it freezes. What is written down is that judgement - the entry test, the one-sentence rule, and assembling the release as a SET, merging related commits and dropping what did not matter, which no per-PR entry could ever do. The cost is that a commit message now does double duty, so a lazy subject line becomes a worse changelog. The README stops embedding the changelog - 505 lines - and keeps a Change Log section pointing at it. The checklist line goes too: with nothing to do per PR, a box that always reads N/A is the ritual this removes. Also corrects the "where things live" table, which still said README.adoc is generated FROM CHANGELOG.adoc. It is still generated, from README_TEMPLATE.adoc, but no longer from the changelog. That contradiction existed in neither change alone - #112 rewrote the row, this PR removed the include - and appeared only when the merge put both in one file. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Description
An audit of the in-flight ledger against reality, and then a restructure of it - plus the same audit applied to the manifest it defers to.
1. The audit
The ledger had reached 677 lines and turned into a record of finished work. Entries told to "delete when #98 merges" survived three merges; the rebalance-commit fix was still filed as awaiting PR after landing as #100; the
largeOffsetMapstatic-state flake was still listed as deliberately-unfixed after #101 fixed it at source. Meanwhile the largest thing actually in flight - #80, carrying the drain-zombie fix - appeared only as a footnote about quarantine ownership, and #105/#106/#111 were absent entirely. The file's own scope rule says entries shrink or vanish when work lands; it had stopped being applied.Every surviving claim is now checked against GitHub, git and the code, and dated. 677 lines of prose → 281, before the split below.
Obsolete entries deleted rather than annotated, each verified first:
ManagedPCInstanceneeds a fork header lineCommitFailedExceptionhandler marks uncommitted offsets cleanlargeOffsetMapstatic leak "worth doing soon"@ResourceLock)Check PR DependenciesUpstream references now follow the convention
CHANGELOG.adocanddocs/refactoring.mdalready use. Fork numbering reaches #111, so every bare reference at or above #162 was silently upstream and read as if it were ours -#857,#859,#893/#905,#909,#912,#629/#631/#640and the rest are now writtenupstream #NN, including the trap that fork branch names encode the upstream number (bugs/857-...), so a number in a branch name is never a fork issue.Four items moved to
docs/refactoring.md, which owns deferred work: the eleven SpotBugs thread-visibility findings (verified still present on master), the produce-lock double release surfaced by #110, thecommitAndWaittimeout message that misstates its own wait, and jacoco's single exec file under forked surefire.2. The split: one file per item
With the content correct, the file became a directory.
docs/inflight.mdappeared in 26 of the last 30 master commits, so unrelated PRs conflicted on it constantly - not because they disagreed, but because their notes were adjacent, and the merge that resolves such a conflict silently drops one side (it happened between #108 and #110).Each item is now its own file,
docs/inflight/<category>-<slug>.md. Two PRs recording unrelated work touch disjoint files and cannot conflict, and finishing work isgit rm- a deletion never conflicts with an edit elsewhere. The prefix is the structure:ls docs/inflight/shows the shape of what is open without reading anything (bug-,test-,ci-,deps-,pr-,branch-,release-,parked-,next-).There is deliberately no committed index. It would be edited by every PR, which is the problem the directory solves;
docs/TODO_INDEX.mdis the cautionary case, generated and committed and stale until a reviewer caught it on #110.lsandgrep -rare the index.This was parked on
docs/inflight-as-directory, with "migrating ~600 lines of existing entries" named as the reason not to do it. The audit is what made it cheap.docs/refactoring.mddeliberately stays one file - 2 of the last 30 commits touched it, so it has none of this problem.Four rules now live in
docs/inflight/AGENTS.md, where an agent will find them, rather than in one person's head:ghorgitcan answer - which is why the open-PR table an earlier revision of this PR added is gone again.3.
upstream-map.yaml: five entries disagreed with realityThe manifest is the declared source of truth for the fork↔upstream mapping, so drift here is worse than drift in a scratch file - it is what a future session trusts instead of re-deriving.
bug-857recorded no fork PR and a todo to open one, while fix: paused consumption after rebalance with multiple consumers (#119, upstream #857) #29 was open, fix(core): a rebalance-time commit no longer kills the broker-poll thread (#857 family) #100 had merged and fix(core): draining PC stops polling - 10kHz busy-spin + zombie partition hold (#857-family silent stall) #80 was in review. It is a family of three distinct defects behind one upstream symptom - thesynchronized(commitCommand)deadlock, theRebalanceInProgressExceptionthat killed the broker-poll thread, the drain-path zombie - and is now written that way, with fix(core): draining PC stops polling - 10kHz busy-spin + zombie partition hold (#857-family silent stall) #80 recorded as merged atbd717241.fix-909recorded no PR though fix: replace stale container at same offset after rebalance (carries upstream #909) #31 is open;bug-912wasin-progresswhen the schema's own word for pushed-but-unPR'd isready; the release entry listed none of build: change Maven groupId to bz.stub.parallelconsumer #55/docs: prepare for 0.6.0.0 release (changelog + README) #56/docs(release): fix 0.6.0.0 changelog + use it as the GitHub release notes #72 and did not say the release has not happened (pom is still-SNAPSHOT, no tag).master-confluent, the pinned pre-rebrand mirror - merging either would land the fix where no user can reach it. Nowtodo:lines, soupstream-map.py todosurfaces them.Nothing catches this class of drift:
upstream-map.py validateonly checks the schema,upstream-sweep.shonly watches upstream.prs: []beside an open fork PR passes every check we have. Hence the newAGENTS.mdrule - update the entry at every lifecycle transition of your own work, in the commit that causes it, not only when starting.4.
.gitignoreAgent and tool state was only excluded in
.git/info/exclude, which is local to one checkout, so every fresh clone and every agent session rediscovered 7.6G of session transcripts and 6.9M of tool artifacts sitting untracked..claudeis ignored by contents (/*) rather than as a directory, so shared config can later be un-ignored with a negation - git will not descend into a fully-excluded directory, which would make that impossible.Also removes an empty
duplicate-code-cross-check/clone: agit initfrom April that never fetched anything (0 objects, no refs, no commits). The action itself is real and pinned inmaven.yml; this was scaffolding. Deleted rather than ignored, so a genuine nested clone would be visible rather than silently swallowed.Not included
Raised for the maintainer to decide, deliberately not acted on here: retargeting #29/#31 off
master-confluent(their fixes need reconciling with what #100 and #80 landed, not a mechanical rebase), pruning the supersededcherry-pick/*andupstream-pr-*branches, and refreshinglast_swept(over a week stale - needs an actual sweep run).docs/inflight-as-directorycan also be deleted once this merges: its idea has shipped.The changelog rework is #113, deliberately separate - it touches the generated
README.adoc, which would have made this diff noisier to read.changelog-ref: N/A - internal documentation and gitignore only, no user- or operator-visible change
Checklist
N/A - internal docs + gitignore, nothing user- or operator-visibledocs/inflight/,docs/refactoring.md,AGENTS.md,upstream-map.yaml); every in-repo reference todocs/inflight.mdupdated, includingbin/todo-index.sh,dependabot.ymlandmaven.yml. Historical mentions indocs/plans/anddocs/solutions/are left alone: they record what was true when writtenN/A - no code changed; upstream-map.py validate passes (24 entries, no schema errors) and the changelog-ref gate self-test passesN/A - no workflow or runner changes(themaven.ymledit is a comment pointing at the moved doc)🤖 Generated with Claude Code