Skip to content

Commit 55bbb52

Browse files
obraarittr
authored andcommitted
docs(specs): SDD fix-loop redesign design spec
Review-fix loop gets resume-the-implementer semantics, scoped re-reviews, a five-round circuit breaker, and controller adjudication at trip. SKILL.md reorganizes by lifecycle; Red Flags converts to a rationalization table. Brainstormed with Jesse 2026-07-15.
1 parent 0e13ad8 commit 55bbb52

1 file changed

Lines changed: 196 additions & 0 deletions

File tree

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# SDD Fix-Loop Redesign — Design Spec
2+
3+
**Status:** Approved design (brainstormed with Jesse 2026-07-15); implementation
4+
plan to follow.
5+
**Objective:** make the subagent-driven-development skill's review-fix loop
6+
convergent and autonomous, and make the document readable, without rewriting
7+
its eval-tuned language.
8+
**Hard invariant:** existing eval-tuned sentences move; they do not get
9+
reworded. New machinery ships with drill evidence.
10+
11+
## Problems
12+
13+
Four, all observed in real sessions:
14+
15+
1. **Pathological review loops.** The loop is literally "Repeat until
16+
approved" — no round cap. Each re-review is a fresh full review of the
17+
whole diff, so a nondeterministic frontier reviewer surfaces new findings
18+
every round instead of verifying fixes. Result: implement, review, fix,
19+
review, review, fix, review, fix — with no circuit breaker. The
20+
strict-cost spec (2026-06-10) independently measured review-loop count as
21+
the biggest run-to-run cost variance.
22+
2. **Contradictory fix policy.** The process diagram and "Constructing
23+
Reviewer Prompts" dispatch dedicated fix subagents; Red Flags says
24+
"Implementer (same subagent) fixes them"; implementer-prompt.md's "After
25+
Review Findings" section assumes the implementer will be re-engaged. Three
26+
answers to "who fixes?" in one skill.
27+
3. **Accreted structure.** Thirteen top-level sections; guidance for one
28+
activity is scattered across four of them. "Constructing Reviewer Prompts"
29+
is a grab-bag holding reviewer guidance, fix policy, final-review policy,
30+
and plan-conflict adjudication.
31+
4. **Red Flags format.** Seven sibling skills use the `| Excuse | Reality |`
32+
rationalization table; SDD carries a 17-bullet "Never" list plus three
33+
"If X" mini-blocks.
34+
35+
## Design Decisions
36+
37+
| # | Decision | Rationale |
38+
|---|----------|-----------|
39+
| 1 | The original implementer fixes its own review findings — resume it in place. | It already holds the task context; ownership beats a drive-by patcher. Fresh "fix subagents" rebuild context per finding and lack the task frame. |
40+
| 2 | Re-reviews are scoped to the findings. | Fresh full reviews each round are the churn engine. Scoped re-reviews make the loop structurally convergent; the final whole-branch review remains the broad safety net. |
41+
| 3 | Circuit breaker at five fix rounds: three resumes, then two fresh dispatches on a more capable model. | Jesse's call. A loop that survives three resumes usually means the implementer cannot see its own problem — the fresh capable dispatch de-anchors and capability-bumps in one move. |
42+
| 4 | At trip, the controller adjudicates and routes. No new human checkpoint — structural failures reach the existing BLOCKED stop. | SDD's point is autonomous execution. The controller holds the plan and cross-task context the reviewer lacks; the existing text already sanctions it ("adjudicate it in the review loop") without ever specifying the mechanism. |
43+
| 5 | Reorganize SKILL.md by lifecycle, preserving tuned sentences. | Fixes "hard to follow" at the root. Content moves to its point of use, matching the house direction (recent commits fold recap sections into points of use). |
44+
| 6 | Convert Red Flags to a `| Excuse | Reality |` rationalization table; relocate hard rules to their points of use. | Matches the other seven skills. Excuses get rebuttals; rules get enforced where the reader acts. |
45+
46+
## The Fix Loop
47+
48+
Trigger: a task review returns spec ❌ or any Critical/Important finding.
49+
50+
**Rounds 1–3 — resume the original implementer.** Send the findings verbatim
51+
(Critical/Important plus spec gaps). The implementer fixes, re-runs the
52+
covering tests, appends the fix report to its existing report file, and
53+
returns the short contract. On a harness without agent resume, a "resume" is
54+
a fresh dispatch carrying the brief, the report file, and the findings — the
55+
report file is the persistent memory either way.
56+
57+
**Rounds 4–5 — fresh implementer, more capable model.** Full task context:
58+
brief, report file, open findings, and the framing "a prior implementer
59+
attempted this N times; you own the task now."
60+
61+
**Every round's re-review is scoped.** The re-reviewer receives the brief,
62+
the updated report, the original findings list, and a fix-scoped diff package
63+
(`review-package FIX_BASE HEAD`, where FIX_BASE is the head the reviewer
64+
last reviewed; the script already takes arbitrary ranges).
65+
It verdicts each finding addressed / not addressed and flags new breakage in
66+
the fix diff only. Novel findings on code the fix did not touch are reported
67+
as non-blocking; the controller ledgers them for the final review.
68+
69+
**Fix-report completeness gate (existing rule, kept):** before dispatching a
70+
re-review, confirm the fix report names the covering tests, the command run,
71+
and the output.
72+
73+
**No early exit.** The controller never adjudicates before the cap — an early
74+
exit reopens the "pre-judge findings to spare yourself a review loop" hole
75+
the current content deliberately closed. One exception, unchanged from
76+
today: a finding that conflicts with what the plan's text mandates goes to
77+
the human immediately (plan authority, not loop churn).
78+
79+
**Minor findings** never enter the loop: ledger them as they arrive (existing
80+
rule, kept).
81+
82+
### Adjudication at Trip
83+
84+
After round five fails, the controller stops dispatching and judges each open
85+
finding against the brief, the plan, and cross-task context:
86+
87+
- **Contested or wrong** → ledger with a one-line adjudication ("controller:
88+
reviewer wrong because X"), continue. The final review sees both sides.
89+
- **Real, not load-bearing** → ledger as known-open, continue. Later
90+
dispatches touching that area carry a pointer to the entry.
91+
- **Real and load-bearing** (later tasks build on it, or it reveals a plan
92+
defect) → the existing BLOCKED stop. Park-and-continue defers a structural
93+
failure to the most expensive point and lets dependents build on it, so
94+
structural failures stop the run — through the stop condition that already
95+
exists, not a new checkpoint.
96+
97+
Every adjudication is a ledger entry. Silent discards stay forbidden.
98+
99+
## Document Restructure
100+
101+
New skeleton, in execution order:
102+
103+
1. Intro — why subagents, core principle, narration, continuous execution
104+
2. When to Use — unchanged, including the decision graph
105+
3. The Process — diagram updated for the new loop
106+
4. Setup — worktree, ledger check/resume, pre-flight plan review, todos
107+
5. Model Selection — stays one cross-cutting section; every dispatch
108+
consults it, so folding it into points of use would repeat it five times
109+
6. The Task Loop — five numbered steps:
110+
1. Dispatch the implementer (task-brief script, five-part dispatch
111+
composition, model line required)
112+
2. Handle the report (DONE / DONE_WITH_CONCERNS / NEEDS_CONTEXT / BLOCKED)
113+
3. Review the task (review-package script, reviewer dispatch composition,
114+
constraints lens, no pre-judging, ⚠️ handling)
115+
4. Fix loop (the machinery above)
116+
5. Complete the task (ledger append, todo update)
117+
7. Final Review — package, model pin, one fix wave, one scoped re-review,
118+
adjudication
119+
8. Finish — finishing-a-development-branch
120+
9. Common Rationalizations — the table
121+
10. Example Workflow — updated to show a resume-based fix round and the
122+
breaker not tripping
123+
124+
"Constructing Reviewer Prompts," "File Handoffs," and "Durable Progress"
125+
dissolve into the steps where each rule applies. Every eval-tuned sentence
126+
lands in exactly one new location; a move map in the implementation plan
127+
tracks source → destination so review can verify nothing was dropped or
128+
reworded.
129+
130+
## Rationalization Table
131+
132+
Excuse-shaped Never items convert to rows; new rows cover the loop
133+
pathology. Draft rows (final wording at implementation):
134+
135+
| Excuse | Reality |
136+
|--------|---------|
137+
| "Close enough on spec compliance" | Reviewer found gaps = not done. |
138+
| "I'll fix it myself, dispatching is overhead" | Controller fixes pollute your context and skip review. Resume the implementer. |
139+
| "One more round will converge" | Past the cap, rounds don't converge. Adjudicate. |
140+
| "The reviewer will just find something new anyway" | Scoped re-reviews check fixes, not taste. New findings on untouched code go to the ledger, not the loop. |
141+
| "This finding is obviously wrong, I'll drop it" | You adjudicate only at the cap, and every adjudication is a ledger entry. Silent discards are forbidden. |
142+
| "The fix was small, skip the re-review" | Unreviewed fixes are how regressions land. |
143+
144+
Hard rules that are not excuses (never parallel implementers, never dispatch
145+
a reviewer without a diff file, model line required, never re-dispatch
146+
ledger-complete tasks) move to their points of use.
147+
148+
## Prompt Templates
149+
150+
- **implementer-prompt.md** — "After Review Findings" rewritten for resume
151+
semantics: you will be resumed with findings; fix, re-run covering tests,
152+
append to your report file, return the short contract.
153+
- **task-reviewer-prompt.md** — initial review only; the trailing re-review
154+
sentence moves out.
155+
- **re-review-prompt.md (new)** — the scoped re-review contract: inputs are
156+
brief, updated report, original findings, fix-scoped diff package; output
157+
is a per-finding verdict (addressed / not addressed), new breakage in the
158+
fix diff, and non-blocking observations outside it. A separate template
159+
because it is a different contract — overloading the full-review template
160+
produced the current ambiguity.
161+
- **Takeover dispatch (rounds 4–5)** — composed from implementer-prompt.md
162+
plus SKILL.md guidance (brief, report path, open findings, takeover
163+
framing); no new template file.
164+
165+
## Final Review Loop
166+
167+
Unchanged: merge-base package, most capable model, ONE fixer with the
168+
complete findings list. New: exactly one scoped re-review of the fix wave,
169+
then controller adjudication. Residual load-bearing findings surface at
170+
finishing-a-development-branch, where the human already is. The end of the
171+
branch gets a bounded loop too.
172+
173+
## Evals
174+
175+
Three new drill scenarios in `evals/`:
176+
177+
1. **Resume, don't re-dispatch:** a task review returns findings; the
178+
controller must resume the same implementer rather than dispatch a fix
179+
subagent.
180+
2. **Breaker trips:** a seeded never-satisfied reviewer; the controller must
181+
stop dispatching after the fifth round fails, adjudicate, ledger, and
182+
continue — not loop.
183+
3. **Structural finding stops:** a load-bearing finding (later tasks depend
184+
on it); the controller must stop via BLOCKED rather than park.
185+
186+
Plus before/after runs of the existing SDD scenarios to catch regressions
187+
from the reorganization.
188+
189+
## Non-Goals
190+
191+
- Ledger session-scoping — PR #1943 owns it. This work touches the same
192+
sections, so the implementation plan notes the collision risk.
193+
- Script changes — task-brief and review-package already do what the new
194+
loop needs.
195+
- Changes to executing-plans or requesting-code-review beyond the final-
196+
review pointer continuing to resolve.

0 commit comments

Comments
 (0)