docs/building.md: the build loop a parity study proved out - #73
Merged
Conversation
SKILL.md's Step 5 ends at the first VERIFY PASS, and both gates measure geometry only — neither has read the brief. A parity study (30 decks, five briefs, seven setups, judged blind against a hand-built reference per brief) measured the gap: SKILL.md alone 0.878 of the reference; a review checklist read against the verify PNGs 0.952 mean but 0.691 on its worst deck; the same checklist behind a coverage manifest written before the model 0.994 mean, 0.944 worst, beating the reference on three briefs of five at the same 14-minute mean wall time. docs/building.md is that loop in the engine's own voice: manifest, build, validate --strict, create, verify --strict, then the mandatory second pass — tick the manifest against the PNGs, then shape / facts / chrome — fix, rebuild, verify again. It carries the concrete rules the study found (arrows on a process, a chart row on a series, a delta on a before/after, one row per option, every number traced, spell flags into spell.ignore, no empty lower 40%, no orphaned last word, no phrase said three times in the chrome). SKILL.md links it in one sentence at Step 5 and stays inside its 7,000-word budget (6,995 -> 6,999, one word trimmed from that same paragraph). README and llms.txt name the eighth doc; test/skill.test.mjs checks it like the other seven plus five rules the page must keep. Verified: node --test test/**/*.test.mjs -> 307 pass, 0 fail, 0 skipped (Playwright present, live tests ran). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new reference page,
docs/building.md, stating the engine's recommended build loop, and a one-sentence link to it from SKILL.md's Step 5.The loop, in order: write
MANIFEST.mdbefore the model (every must-include and every brief number gets a slide and a shape; the brief's slide count is the deck's) → build →validate --strict→create→verify --strict→ the mandatory second pass: tick the manifest against the PNGs (a line with no PNG is a missing item), then read every PNG for shape / facts / chrome → fix, rebuild, verify again.The concrete rules the page keeps: a process gets arrows, a series gets a chart row, a before/after gets a delta rather than two sentences, options get one row per option; every number traced to the brief and every derivation computed;
create's spell flags go intospell.ignore; no reading slide with an empty lower 40%; no orphaned last word; the cover, footer and supertitle do not say one thing three times.Why
validateandverifymeasure geometry. Neither has read the brief, so a deck passes both and still drops a must-include, puts a five-year series in a bullet list, or prints a number the brief never gave.A parity study measured that gap: 30 decks from five briefs by seven setups, each judged blind against a hand-built reference deck for the same brief.
The manifest variant beat the reference on three briefs of five, at the same 14-minute mean wall time as the checklist alone. Two runs show where it pays: one caught two pricing captions asserting billing terms the brief never gave, another caught a missing arithmetic tie between a 62% → 35% share shift and the 17% blended cost before a line of JSON was written.
How verified
Playwright was installed, so the live browser tests ran rather than no-opping.
test/skill.test.mjsis extended two ways:building.mdjoins the seven docs checked for existence, a(docs/building.md)link in SKILL.md and >80 words of real text; and five regexes hold the rules the page must keep (MANIFEST.md, must-include, second pass, lower 40%,spell.ignore).Word budget
SKILL.md is gated at 7,000 words and sat at 6,995. It is now 6,999: the added sentence is
The hand-off needs a second pass: [docs/building.md](docs/building.md).and the four words it needed came out of the same paragraph (Fix the model, not the output. Re-run until VERIFY PASS;→Fix the model; re-run to VERIFY PASS,). No other author's rule was touched.README and llms.txt both name the eighth doc in the lists the gate reads.
🤖 Generated with Claude Code