|
| 1 | +# SDD Fix-Loop Redesign — Design Spec |
| 2 | + |
| 3 | +**Status:** Approved design (brainstormed with Jesse 2026-07-15); implementation |
| 4 | +plan to follow. |
| 5 | +**Objective:** make the subagent-driven-development skill's review-fix loop |
| 6 | +convergent and autonomous, and make the document readable, without rewriting |
| 7 | +its eval-tuned language. |
| 8 | +**Hard invariant:** existing eval-tuned sentences move; they do not get |
| 9 | +reworded. New machinery ships with drill evidence. |
| 10 | + |
| 11 | +## Problems |
| 12 | + |
| 13 | +Four, all observed in real sessions: |
| 14 | + |
| 15 | +1. **Pathological review loops.** The loop is literally "Repeat until |
| 16 | + approved" — no round cap. Each re-review is a fresh full review of the |
| 17 | + whole diff, so a nondeterministic frontier reviewer surfaces new findings |
| 18 | + every round instead of verifying fixes. Result: implement, review, fix, |
| 19 | + review, review, fix, review, fix — with no circuit breaker. The |
| 20 | + strict-cost spec (2026-06-10) independently measured review-loop count as |
| 21 | + the biggest run-to-run cost variance. |
| 22 | +2. **Contradictory fix policy.** The process diagram and "Constructing |
| 23 | + Reviewer Prompts" dispatch dedicated fix subagents; Red Flags says |
| 24 | + "Implementer (same subagent) fixes them"; implementer-prompt.md's "After |
| 25 | + Review Findings" section assumes the implementer will be re-engaged. Three |
| 26 | + answers to "who fixes?" in one skill. |
| 27 | +3. **Accreted structure.** Thirteen top-level sections; guidance for one |
| 28 | + activity is scattered across four of them. "Constructing Reviewer Prompts" |
| 29 | + is a grab-bag holding reviewer guidance, fix policy, final-review policy, |
| 30 | + and plan-conflict adjudication. |
| 31 | +4. **Red Flags format.** Seven sibling skills use the `| Excuse | Reality |` |
| 32 | + rationalization table; SDD carries a 17-bullet "Never" list plus three |
| 33 | + "If X" mini-blocks. |
| 34 | + |
| 35 | +## Design Decisions |
| 36 | + |
| 37 | +| # | Decision | Rationale | |
| 38 | +|---|----------|-----------| |
| 39 | +| 1 | The original implementer fixes its own review findings — resume it in place. | It already holds the task context; ownership beats a drive-by patcher. Fresh "fix subagents" rebuild context per finding and lack the task frame. | |
| 40 | +| 2 | Re-reviews are scoped to the findings. | Fresh full reviews each round are the churn engine. Scoped re-reviews make the loop structurally convergent; the final whole-branch review remains the broad safety net. | |
| 41 | +| 3 | Circuit breaker at five fix rounds: three resumes, then two fresh dispatches on a more capable model. | Jesse's call. A loop that survives three resumes usually means the implementer cannot see its own problem — the fresh capable dispatch de-anchors and capability-bumps in one move. | |
| 42 | +| 4 | At trip, the controller adjudicates and routes. No new human checkpoint — structural failures reach the existing BLOCKED stop. | SDD's point is autonomous execution. The controller holds the plan and cross-task context the reviewer lacks; the existing text already sanctions it ("adjudicate it in the review loop") without ever specifying the mechanism. | |
| 43 | +| 5 | Reorganize SKILL.md by lifecycle, preserving tuned sentences. | Fixes "hard to follow" at the root. Content moves to its point of use, matching the house direction (recent commits fold recap sections into points of use). | |
| 44 | +| 6 | Convert Red Flags to a `| Excuse | Reality |` rationalization table; relocate hard rules to their points of use. | Matches the other seven skills. Excuses get rebuttals; rules get enforced where the reader acts. | |
| 45 | + |
| 46 | +## The Fix Loop |
| 47 | + |
| 48 | +Trigger: a task review returns spec ❌ or any Critical/Important finding. |
| 49 | + |
| 50 | +**Rounds 1–3 — resume the original implementer.** Send the findings verbatim |
| 51 | +(Critical/Important plus spec gaps). The implementer fixes, re-runs the |
| 52 | +covering tests, appends the fix report to its existing report file, and |
| 53 | +returns the short contract. On a harness without agent resume, a "resume" is |
| 54 | +a fresh dispatch carrying the brief, the report file, and the findings — the |
| 55 | +report file is the persistent memory either way. |
| 56 | + |
| 57 | +**Rounds 4–5 — fresh implementer, more capable model.** Full task context: |
| 58 | +brief, report file, open findings, and the framing "a prior implementer |
| 59 | +attempted this N times; you own the task now." |
| 60 | + |
| 61 | +**Every round's re-review is scoped.** The re-reviewer receives the brief, |
| 62 | +the updated report, the original findings list, and a fix-scoped diff package |
| 63 | +(`review-package FIX_BASE HEAD`, where FIX_BASE is the head the reviewer |
| 64 | +last reviewed; the script already takes arbitrary ranges). |
| 65 | +It verdicts each finding addressed / not addressed and flags new breakage in |
| 66 | +the fix diff only. Novel findings on code the fix did not touch are reported |
| 67 | +as non-blocking; the controller ledgers them for the final review. |
| 68 | + |
| 69 | +**Fix-report completeness gate (existing rule, kept):** before dispatching a |
| 70 | +re-review, confirm the fix report names the covering tests, the command run, |
| 71 | +and the output. |
| 72 | + |
| 73 | +**No early exit.** The controller never adjudicates before the cap — an early |
| 74 | +exit reopens the "pre-judge findings to spare yourself a review loop" hole |
| 75 | +the current content deliberately closed. One exception, unchanged from |
| 76 | +today: a finding that conflicts with what the plan's text mandates goes to |
| 77 | +the human immediately (plan authority, not loop churn). |
| 78 | + |
| 79 | +**Minor findings** never enter the loop: ledger them as they arrive (existing |
| 80 | +rule, kept). |
| 81 | + |
| 82 | +### Adjudication at Trip |
| 83 | + |
| 84 | +After round five fails, the controller stops dispatching and judges each open |
| 85 | +finding against the brief, the plan, and cross-task context: |
| 86 | + |
| 87 | +- **Contested or wrong** → ledger with a one-line adjudication ("controller: |
| 88 | + reviewer wrong because X"), continue. The final review sees both sides. |
| 89 | +- **Real, not load-bearing** → ledger as known-open, continue. Later |
| 90 | + dispatches touching that area carry a pointer to the entry. |
| 91 | +- **Real and load-bearing** (later tasks build on it, or it reveals a plan |
| 92 | + defect) → the existing BLOCKED stop. Park-and-continue defers a structural |
| 93 | + failure to the most expensive point and lets dependents build on it, so |
| 94 | + structural failures stop the run — through the stop condition that already |
| 95 | + exists, not a new checkpoint. |
| 96 | + |
| 97 | +Every adjudication is a ledger entry. Silent discards stay forbidden. |
| 98 | + |
| 99 | +## Document Restructure |
| 100 | + |
| 101 | +New skeleton, in execution order: |
| 102 | + |
| 103 | +1. Intro — why subagents, core principle, narration, continuous execution |
| 104 | +2. When to Use — unchanged, including the decision graph |
| 105 | +3. The Process — diagram updated for the new loop |
| 106 | +4. Setup — worktree, ledger check/resume, pre-flight plan review, todos |
| 107 | +5. Model Selection — stays one cross-cutting section; every dispatch |
| 108 | + consults it, so folding it into points of use would repeat it five times |
| 109 | +6. The Task Loop — five numbered steps: |
| 110 | + 1. Dispatch the implementer (task-brief script, five-part dispatch |
| 111 | + composition, model line required) |
| 112 | + 2. Handle the report (DONE / DONE_WITH_CONCERNS / NEEDS_CONTEXT / BLOCKED) |
| 113 | + 3. Review the task (review-package script, reviewer dispatch composition, |
| 114 | + constraints lens, no pre-judging, ⚠️ handling) |
| 115 | + 4. Fix loop (the machinery above) |
| 116 | + 5. Complete the task (ledger append, todo update) |
| 117 | +7. Final Review — package, model pin, one fix wave, one scoped re-review, |
| 118 | + adjudication |
| 119 | +8. Finish — finishing-a-development-branch |
| 120 | +9. Common Rationalizations — the table |
| 121 | +10. Example Workflow — updated to show a resume-based fix round and the |
| 122 | + breaker not tripping |
| 123 | + |
| 124 | +"Constructing Reviewer Prompts," "File Handoffs," and "Durable Progress" |
| 125 | +dissolve into the steps where each rule applies. Every eval-tuned sentence |
| 126 | +lands in exactly one new location; a move map in the implementation plan |
| 127 | +tracks source → destination so review can verify nothing was dropped or |
| 128 | +reworded. |
| 129 | + |
| 130 | +## Rationalization Table |
| 131 | + |
| 132 | +Excuse-shaped Never items convert to rows; new rows cover the loop |
| 133 | +pathology. Draft rows (final wording at implementation): |
| 134 | + |
| 135 | +| Excuse | Reality | |
| 136 | +|--------|---------| |
| 137 | +| "Close enough on spec compliance" | Reviewer found gaps = not done. | |
| 138 | +| "I'll fix it myself, dispatching is overhead" | Controller fixes pollute your context and skip review. Resume the implementer. | |
| 139 | +| "One more round will converge" | Past the cap, rounds don't converge. Adjudicate. | |
| 140 | +| "The reviewer will just find something new anyway" | Scoped re-reviews check fixes, not taste. New findings on untouched code go to the ledger, not the loop. | |
| 141 | +| "This finding is obviously wrong, I'll drop it" | You adjudicate only at the cap, and every adjudication is a ledger entry. Silent discards are forbidden. | |
| 142 | +| "The fix was small, skip the re-review" | Unreviewed fixes are how regressions land. | |
| 143 | + |
| 144 | +Hard rules that are not excuses (never parallel implementers, never dispatch |
| 145 | +a reviewer without a diff file, model line required, never re-dispatch |
| 146 | +ledger-complete tasks) move to their points of use. |
| 147 | + |
| 148 | +## Prompt Templates |
| 149 | + |
| 150 | +- **implementer-prompt.md** — "After Review Findings" rewritten for resume |
| 151 | + semantics: you will be resumed with findings; fix, re-run covering tests, |
| 152 | + append to your report file, return the short contract. |
| 153 | +- **task-reviewer-prompt.md** — initial review only; the trailing re-review |
| 154 | + sentence moves out. |
| 155 | +- **re-review-prompt.md (new)** — the scoped re-review contract: inputs are |
| 156 | + brief, updated report, original findings, fix-scoped diff package; output |
| 157 | + is a per-finding verdict (addressed / not addressed), new breakage in the |
| 158 | + fix diff, and non-blocking observations outside it. A separate template |
| 159 | + because it is a different contract — overloading the full-review template |
| 160 | + produced the current ambiguity. |
| 161 | +- **Takeover dispatch (rounds 4–5)** — composed from implementer-prompt.md |
| 162 | + plus SKILL.md guidance (brief, report path, open findings, takeover |
| 163 | + framing); no new template file. |
| 164 | + |
| 165 | +## Final Review Loop |
| 166 | + |
| 167 | +Unchanged: merge-base package, most capable model, ONE fixer with the |
| 168 | +complete findings list. New: exactly one scoped re-review of the fix wave, |
| 169 | +then controller adjudication. Residual load-bearing findings surface at |
| 170 | +finishing-a-development-branch, where the human already is. The end of the |
| 171 | +branch gets a bounded loop too. |
| 172 | + |
| 173 | +## Evals |
| 174 | + |
| 175 | +Three new drill scenarios in `evals/`: |
| 176 | + |
| 177 | +1. **Resume, don't re-dispatch:** a task review returns findings; the |
| 178 | + controller must resume the same implementer rather than dispatch a fix |
| 179 | + subagent. |
| 180 | +2. **Breaker trips:** a seeded never-satisfied reviewer; the controller must |
| 181 | + stop dispatching after the fifth round fails, adjudicate, ledger, and |
| 182 | + continue — not loop. |
| 183 | +3. **Structural finding stops:** a load-bearing finding (later tasks depend |
| 184 | + on it); the controller must stop via BLOCKED rather than park. |
| 185 | + |
| 186 | +Plus before/after runs of the existing SDD scenarios to catch regressions |
| 187 | +from the reorganization. |
| 188 | + |
| 189 | +## Non-Goals |
| 190 | + |
| 191 | +- Ledger session-scoping — PR #1943 owns it. This work touches the same |
| 192 | + sections, so the implementation plan notes the collision risk. |
| 193 | +- Script changes — task-brief and review-package already do what the new |
| 194 | + loop needs. |
| 195 | +- Changes to executing-plans or requesting-code-review beyond the final- |
| 196 | + review pointer continuing to resolve. |
0 commit comments