Skip to content

Why the Backlog Grows: Prose Review Has No Fixed Point, and a Plan to Keep the Rigor Without It #1311

Description

@ptr727

A fresh assessment of why the open-issue count keeps growing, what is producing the issues, and how to keep the rigor while stopping the growth. Filed as the umbrella for the sub-issues listed at the end, each of which can be executed on its own.

Summary

The backlog is not growing because the fleet is getting worse. It is growing because the review machinery added in the last two weeks is doing exactly what it was designed to do, reading long prose units whole and adversarially, while the disposition rules turn every finding on pre-existing text into a filed issue, and the corpus it reads is large, cross-referential, and restated in several places. The result is a pump: one small edit to one section produces ten to thirteen findings, most of them pre-existing, each fix push re-reads the unit and finds defects in the fix, and every bundle issue filed from that pass fans out over several units when someone works it. Nothing in the loop converges, and the loop is triggered on nearly every pull request because nearly every pull request touches a canonical unit.

The fix is not to weaken the rules. It is to separate what is law from what is commentary, give every rule one home, put the mechanical rules into gates, treat prose review as bounded advice rather than as an unbounded contract audit, and stop carrying prose byte-for-byte to every repository so a wording change is not a fleet-wide change.

Evidence

All numbers are from the live repository on 2026-09-04. The per-issue classification (origin, kind, surfaces, size) was done by reading every open issue body.

Volume and velocity

Week issues were filed Filed (open and closed)
Jul 29 to Aug 4 22
Aug 5 to 11 21
Aug 12 to 18 54
Aug 19 to 25 39
Aug 26 to Sep 1 99
Sep 2 to 4 (three days) 101

Where the open issues came from

Discovered by Count
Whole-unit canonical pass or local strict review on a hub PR 54
Reviewer bot or reviewer on a hub PR, deferred as follow-up 34
Audit, resync, or standup run against a fleet repo 23
Something failing in use, or the maintainer noticing 23
A downstream repository's review of carried hub content 9
Feature request 5

97 of 148 were found by a review process. 23 were found by something actually going wrong.

What kind of defect they are

Kind Count
Procedure gap (missing actor, timing, unbounded loop, unsatisfiable condition) 36
Script or shell bug 24
Needs a maintainer decision, not a reword 24
Prose claim the shipped code or workflow falsifies 15
Rule exists in prose with no mechanical check, or the check misses a shape 14
A rule restated in a second place, and the copies drifted 12
Two documents or skills disagree 9
Fleet chore (registry, labels, rollout) 9
Feature 5

72 of 148 (49%) are defects in prose or procedure text. 38 (26%) are code, and those are the ones that carry the fleet's actual rigor. 81 issues are bundles, together holding 426 enumerated sub-findings, and 90 need edits in more than one canonical unit.

Where they land

Surface Open issues touching it
GOVERNANCE.md 35
WORKFLOW.md 32
backlog-burndown skill 23
drive-pr skill 16
AGENTS.md 14
spec/audit.py 12
validate-task.yml 11
local-strict-review skill 10
pr-review-conduct skill 9

The corpus being reviewed

  • The governing prose (AGENTS, GOVERNANCE, WORKFLOW, AUDIT, RESYNC, STANDUP, CODESTYLE, OPERATIONS, and 25 skills) is about 90,800 words. The skills alone are 42,500 words. GOVERNANCE.md is 14,700 words in 22 sections, and its "Verification Discipline" section alone is 18 KB. Eleven single bullets in GOVERNANCE.md exceed 1,500 characters.
  • The canonical review engine counts 304 units. 87 have had a whole read, 217 (71%) never have. Every one of those 217 is a future pass that will return findings.
  • Every skill exists in three copies (.agents/skills, .github/skills, .claude-plugin/fleet-skills). Reviewer bots comment on each copy. PR Add the backlog-burndown Skill and Seat Its Orchestration in the Skills It Drives #1291's Copilot review lists every suppressed finding three times.
  • GOVERNANCE.md carries 30 cross-references to other documents or skills, AGENTS.md 23. The same rule appears in up to four places: the GOVERNANCE section, the Skill, the agent-conduct summary list, and a condensed catalog or a sibling skill's copy. Review-loop skills disagree with each other and stop short of decisions the loop needs #1164 records agent-conduct listing 11 of the source section's roughly 20 checks, and the class-sweep rule stated three different ways.

The mechanism

flowchart TD
  edit["small edit to one canonical unit"] --> hook["pre-push: whole-unit adversarial pass on every unit the diff touched"]
  hook --> findings["10 to 13 findings, mostly pre-existing"]
  findings --> fixdiff["fix what the diff introduced"]
  findings --> file["file the rest (outcome 4), usually as one bundle issue per pass"]
  fixdiff --> repush["fix push owes a new pass"] --> hook
  file --> bundle["bundle issue names 5 to 19 defects across several units"]
  bundle --> work["working it touches every named unit"] --> hook
  edit --> bots["three reviewer bots, one on an assertive profile, on three mirrored copies"]
  bots --> gate["Merge Gate: every finding on head closed, including out-of-diff and low-confidence"]
  gate --> file
  edit --> sweep["'fix the class, sweep for siblings' widens the diff"] --> hook
Loading

Each arrow is a rule that is individually reasonable. Together they have no fixed point.

Root causes

  1. Prose has no oracle, so adversarial prose review does not converge. A test or a linter decides code review. Nothing decides whether a sentence is "correct", so a reviewer told to be adversarial can always find a stricter reading, and a fix to one sentence changes the reading of its neighbors. The whole-unit review loop has no committed stop condition, so it does not converge #1267 states the consequence: a finding count is not a stop condition, and the loop currently has none. The evidence is that the rounds find defects in the previous round's fix, not in the original text.

  2. Every sentence is written as a checkable contract, including the sentences that are only rationale. The house style justifies each rule with its history, the tool behavior it depends on, and its exceptions, in one long bullet. That is good for a reader and terrible for a reviewer, because the rationale makes claims about scripts and workflows that go stale the moment those change. The "claim falsified" and "two documents disagree" classes (24 issues) are overwhelmingly rationale sentences, not rule sentences. The rule and its commentary are reviewed to the same standard because nothing marks which is which.

  3. Rules have several homes, and each home is a unit that can drift. A rule lives in a GOVERNANCE section, is packaged in a Skill, summarized in agent-conduct, condensed in a catalog, copied into a sibling skill's step list, and mirrored three times by the build. The 12 restatement-drift issues, the 9 inconsistency issues, and a large share of the 36 procedure gaps are the copies disagreeing. AGENTS.md already says not to restate rules. The rule is not enforced, and skills need enough self-contained text to work in isolation, so they restate.

  4. Skills are coupled through step numbers and shared tracked files, not through interfaces. drive-pr step 2 requires local-strict-review, whose carried-content pass writes a tracked report with global counts, which makes backlog-burndown's central no-shared-file rule unsatisfiable (backlog-burndown's no-shared-file rule is unsatisfiable: every canonical-content worker writes the shared ledger #1290, The canonical-review ledger conflicts across concurrent branches, and its generated report cannot be hand-merged #1268). The single .source-digest hash makes any two concurrent skill edits conflict (workflow-ci-contract Skill Restates D1.6's uv.lock Guard and D7.3's Unscoped Boolean Rule #1240). Skills reference each other by step number ("drive-pr's Drive Loop step 2"), so a renumbering in one skill is a defect in three others. Review the skills for isolation: tightly coupled, loosely integrated #1295 names the principle: tightly coupled internally, loosely integrated with siblings. The current tree is the reverse.

  5. The disposition rules convert every observation into a tracked item. pr-review-conduct outcome 4 files an issue for anything real and deferred. The Merge Gate requires every finding on the head closed, including CodeRabbit's out-of-diff comments and Copilot's low-confidence collapsed findings, so a bot's remark on a sentence the diff never touched becomes an issue or a written decline. Three bots, one on an assertive profile, run over prose. GOVERNANCE's "a review flags an instance, so fix the class" widens the diff into more units, which owe more passes. None of this is wrong for code. For prose it is the pump.

  6. Prose is carried byte-for-byte to every repository. The skills tree is a verbatim tree and many GOVERNANCE and AGENTS sections are verbatim. So every wording change is a fleet-wide change, every carrier's first review re-discovers the same defects (Hub-Owned Carried Content Gets Its First Real Review Only When a Downstream Repo Carries It #1138, A Resync Pull Request Is the First Review a Downstream Repo's Own Long-Standing Files Have Had #1139), the fidelity model cannot express legitimate per-repo variance (Verbatim Fidelity Admits No Per-Carrier Variance, in Either Direction #1141, .markdownlint-cli2.jsonc is verbatim, but a repo with a committed data root needs its own globs/ignores #1118), and a carrier holds sentences that are false in the carrier (Carried Text Says the Skills Are Not Carried, While the Manifest Carries Them to Every Repo #1184). The bootstrap already tells an agent to fetch the hub and read it there. Carrying the prose as well is what couples every repository to the hub's wording.

  7. The gates in code are the minority of the rigor, and they are where the real defects are. 38 open issues are script bugs or gate gaps. Those are the fleet's actual controls, they have tests, they converge, and they are starved for attention while sessions spend 150 tool calls on a sentence (The whole-unit review loop has no committed stop condition, so it does not converge #1267). The three-condition test in GOVERNANCE for promoting a rule to a hook is sound, and 14 open issues describe rules that pass it and have no gate.

Principles for the fix

  • Law and commentary are different things and are reviewed differently. A rule is one short normative statement. Everything under it is rationale, non-normative, and a stale rationale is a cleanup rather than a defect. Only the normative statement is a contract a reviewer can hold a change to.
  • One rule, one home. Every other appearance is a pointer or is generated from the home. A pointer cannot drift. A generated copy is checked by the build, not by a reader.
  • Gates in code, guidance in prose. A rule that is decidable from bytes gets a script, a test that watches it fail, and a CI or hook seat. A rule that needs judgment stays prose and is not reviewed as if it were a contract.
  • A review pass has a stop condition stated as a question, not a count. "Does the diff introduce a false or contradictory normative claim?" closes. "Did you find anything?" never does.
  • A skill is a unit with an interface. It names its inputs, outputs, tracked-file side effects, and the siblings it depends on, by name and purpose, never by step number. Nothing a skill writes to the tree may collide with a sibling running concurrently.
  • Carry the bootstrap, reach the rules. A repository carries the pointer to the hub and the files the tooling needs. It does not carry the prose.

Disposition of the current 148

  • 38 script bugs and gate gaps. Keep, label, and work them first. They are the rigor and they converge.
  • 24 decisions. Put them to the maintainer as one list in one session rather than one at a time as each is re-encountered. Half of them are the same three questions (where a rule earns a hook, whether verbatim fidelity admits variance, what the whole-unit loop's stop rule is).
  • 72 prose and procedure defects. Freeze them. Do not fix them one bundle at a time, because each fix re-enters the pump. Roll them into one tracker issue per canonical unit, then close them as superseded when that unit is rewritten in the normative format under this plan. The 23 against backlog-burndown are a special case, covered by P2 (decision): Decide backlog-burndown's Shape Before Its 23 Open Defects Are Worked One at a Time #1323.
  • 14 features and chores. Normal backlog.

What to watch

Tracked weekly, by hand or by the health report #1095 asks for: issues filed against issues closed, issues filed per merged pull request, review rounds per pull request, whole-unit passes per pull request, and the share of open issues that are review-discovered prose defects. The plan is working when filed drops below closed and issues per merged pull request drops below one.

Sub-issues, in priority order

Each carries its own scope, the rules that bind it, and an acceptance test.

Now, to stop the pump (each is one pull request or one session):

  1. P0: Commit a Stop Condition and Round Budget for the Whole-Unit and Local Review Passes #1312 Commit a stop condition and round budget for the whole-unit and local review passes.
  2. P0: Decouple the Review Ledger and the Skills Digest From the Working Tree #1313 Decouple the review ledger and the skills digest from the working tree so concurrent branches stop conflicting.
  3. P0: Scope Reviewer Bots Away From Generated Mirrors and Bound What the Merge Gate Owes an Out-of-Diff Prose Finding #1314 Scope the reviewer bots away from generated mirrors, and bound what the Merge Gate owes an out-of-diff prose finding.
  4. P0: Triage the Open Backlog Into a Label Set, One Tracker Per Canonical Unit, and One Decision List #1315 Triage the open backlog: a label set, one tracker issue per canonical unit, and the decision list for the maintainer.

Next, structural:

  1. P1: Define the Normative-Statement Format, Lint It, and Pilot It on "Verification Discipline" #1316 Define the normative-statement format, lint it, and pilot it on GOVERNANCE.md "Verification Discipline".
  2. P1: Give Every Rule One Home, Replacing Restatements With Pointers or Generated Includes #1317 Give every rule one home: inventory the restatements and replace each with a pointer or a generated include.
  3. P1: Bind Every WORKFLOW.md Section 4 Guarantee to an Audit Check, or Mark It Guidance #1318 Bind every WORKFLOW.md section 4 guarantee to an audit check id, or mark it guidance.
  4. P1: Implement the Open Gate Gaps That Pass the Three-Condition Hook Test #1319 Implement the open gate gaps that pass the three-condition hook test.
  5. P1: Give Every Skill a Declared Interface and Run the Isolation Review #1320 Give every skill a declared interface and run the isolation review Review the skills for isolation: tightly coupled, loosely integrated #1295 asks for.

Then, the larger changes that need a maintainer decision:

  1. P2 (decision): Carry Only the Fleet Bootstrap Verbatim, and Reach Everything Else From the Hub #1321 Carry only the Fleet Bootstrap verbatim, and reach everything else from the hub.
  2. P2: Rewrite GOVERNANCE.md and the Review-Loop Skills Into the Normative Format, One Unit Per Pull Request #1322 Rewrite GOVERNANCE.md and the review-loop skills into the normative format, one unit per pull request, closing the frozen prose issues as each unit lands.
  3. P2 (decision): Decide backlog-burndown's Shape Before Its 23 Open Defects Are Worked One at a Time #1323 Decide backlog-burndown's shape: hold it experimental until its 23 open defects settle, or shrink it to the round loop and file claims and drop what it restates.
  4. P2: Add Weekly Backlog and Review-Cost Metrics #1324 Add the weekly backlog and review-cost metrics.

Dependencies: #1316 and #1317 before #1322. #1315 before #1314's tracker batching has somewhere to file. #1313 before any dispatched #1323 run. Everything else is independent.

Related

#1267 (no stop condition), #1295 (skill isolation), #1083 (prose versus hooks), #1164 (review-loop skills disagree), #1290 and #1268 (ledger collisions), #1138 and #1139 (first review happens downstream), #1104 (local review before every fix), #1300 (brief as single point of failure), #1271 (labels).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentsAgents instructionsdocumentationImprovements or additions to documentationproseA defect in rule or procedure textskillsAgent skill

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions