Self Improving Retro

The method improves itself

Section 16 of the method. A scheduled retro that turns the numbers into changes to the method.

Sections 7, 8 and 15 all say to measure and to attack the method. None of it ran by itself. On the day this section was written, more than twenty adversarial reviews ran and log-review.sh was called for none of them. A loop that depends on somebody remembering is not a loop.

So the loop is a script and a routine, run after every batch, and it changes the method only through a pull request with evidence.

flowchart TD
    B[A batch merges] --> S["retro-signals.sh collects facts<br/>fix rounds, failed checks, days to merge,<br/>escaped fixes, unlogged reviews"]
    S --> R["A cheap agent runs the lean-retro routine<br/>against fixed triggers"]
    R --> P["Proposed changes, each with<br/>the signal that caused it and<br/>the signal that would show it worked"]
    P --> H{"You approve?"}
    H -- yes --> M["Pull request to the method,<br/>plus memory and CLAUDE.md"]
    H -- no --> X[Recorded as rejected, with the reason]
    M --> N[Next batch]
    N --> B
    R --> C{"Did the last change help?"}
    C -- no --> V[Propose reverting it]
    V --> P

Loading

Facts first, from a script. retro-signals.sh --since <date> <repos> prints one row per merged pull request: commits pushed after it opened (fix rounds), failed checks, days to merge, later issues or pull requests that point back at it as the cause of a fix (a proxy for escaped defects), and whether its review was logged. It reasons about nothing, so the retro starts from the same numbers every time.

The first run, on one repository for two days, already said something: a stack of four pull requests with no review of the tickets beforehand took 5, 7, 9 and 10 commits after opening; four smaller ones with a clear scope took 0 to 2.

Fixed triggers, not taste. The retro checks the numbers against rules decided in advance:

Signal	Change it proposes
A finding category seen in two or more changes	A scripted check, a Risks line in section 0, or a critic question
A check or critic question that found nothing for three batches	Drop it for that tier
Something an agent reasoned through twice	A script, per section 4a
A step that needed a person in two or more changes	Automate it, or script it with confirmations
Fix rounds or cost per change rising	Name the step that grew
An escaped defect	A check that would have caught it, tested against that defect
Reviews ran but were not logged	Make logging a step of the review script, not a rule

Every change carries its own test. A proposed change names the signal that triggered it and the number that should move if it works. The next retro reads that number first. If it did not move, or moved the wrong way, the retro proposes reverting the change. That is section 8 on a timer.

One experiment running now: review the ticket before the code. Section 0 predicts that answering the review's questions in the ticket, plus one cheap agent reading the ticket and the code before anyone builds, cuts fix rounds and findings per change. The baseline is the table above. The claim holds if fix rounds and findings fall while the review after the code still finds something and escaped defects stay at zero. If the review after the code drops to nothing and escaped defects rise, the review before has made everyone overconfident, and it comes out.

After each milestone, a lean pass over what shipped. No number triggers this one. Agent-written code and comments over-explain, and none of the gates asks whether the result is lean, so the pass deletes comments that restate the code and shortens what a newer language feature says more plainly, without changing behaviour, as its own pull request.

What it does not do. It never changes the method, the scripts or anyone's instructions by itself. It proposes; a person decides. A method that rewrites itself unattended drifts toward whatever the numbers reward, which is section 15's warning about goals that poison themselves.

The routine is skills/lean-retro/SKILL.md in this repository.

---

Previous: What a reader should observe | Home