-
Notifications
You must be signed in to change notification settings - Fork 0
Draft Rubric Integrated Loop
Status: IN FORCE as of 2026-08-23. This arm won the vibe test and its content now lives in Reviewer Brief STEP 1b. This page is kept as the record of how it was arrived at and what it cost to get right. The brief is the source of truth; where they differ, the brief wins.
What actually moved: the depth table, both trigger tables and the radius table are in STEP 1b. The three gaps in §3 did not become rubric checks — the brief carries them as three things the reviewer must apply that the rubric does not cover, and R1g-surface, R18, R18b and R18c are all still in Critic Rules. Read §3 and §4 as the plan, not the outcome. Where the rubric sits in the pipeline today: Running It.
Six PRs, both arms blind, same PR each time.
| PR | outcome |
|---|---|
| #5318 | tie |
| #5352 | tie |
| #5317 | tie |
| #5324 | tie |
| #5343 | A — all three of B's misses were outside the component |
| #5310 | tie — both reached R3e independently |
| #5350 | tie — B's radius table produced both findings |
| #5344 | B — found the competing PR; A did not |
The loss is why it won. On #5343
B missed three things and every one was outside the component: the bug
surviving past the fix's own breakpoint, 100dvw ignoring its container, and a
sibling PR already fixing it properly. The rubric is a component-audit
instrument — every check points inward. That diagnosis produced the radius
table, and with it B stopped losing and started finding things A could not:
#5344's competitor came from
"is anyone else already fixing this", a question arm A has no reason to ask.
Cindy's bar for promotion: "Since arm B is starting to look good and not missing things from arm A let's promote." Not that B wins more — that it stopped losing.
B's honest self-report on #5344: §1 fired loudly, it enumerated the accessibility tree option by option, found the divider correctly ignored, and nearly produced an approve-with-nits. Check coverage said pass; the axe run said critical. All three of its findings came from the radius table.
The critic has grown its own opinions about component quality — ossification, effects, DOM ownership, perf tests. The Component Audit Rubric already owns most of that, better and in more detail: C1 justifies every effect, C13 wants stable hook returns, C6/C7 cover observers, P38 covers escape hatches, P10 says converge rather than diverge.
Two sources of truth for the same judgement is the problem. The split this arm proposes:
| owns | |
|---|---|
| The rubric | what good CODE is. True with no reviewer in the room. |
| The critic | what good REVIEW is. Meaningless outside a review. |
"A theme target ossifies the system" is rubric. "Approve summaries are ≤30 words" is critic. The test: could a contributor read it before writing the PR? Then it belongs in the rubric, where they and the nightly auditor can find it.
frame → RUBRIC selects the checks → run them → draft → CRITIC grades the draft
The rubric is private to the reviewer. Nothing from it reaches the comment: no check ids, no grades, no tiers, no severities. R12c stands — every finding is translated into what it costs a real person.
Straight from the rubric's own "Reviewing a change". Never the whole checklist.
| Change | The bar |
|---|---|
| Bug fix | Evidence it was broken and is fixed — red→green test, or before/after frames. Plus a code-smell pass. Evidence real, fix clean ⇒ approve and move on. |
| New feature | Every automatable check, then judgment on what the change touches. |
| New component → core | Full audit, visual evidence. |
| New component → lab | Deliberately lax. Flag for the promotion gate rather than blocking. |
(a) Syntactic triggers — the rubric's existing table. What the diff literally
touches: themeProps → §2 · aria-* → §1 · useEffect → §7 · a physical CSS
property → §9. Use it as written.
(b) Semantic triggers — NEW, and the reason this arm exists. What the change means, which is not always visible in what it touches:
| The change's nature | Also run |
|---|---|
| Behavior changes | §1 accessibility — AT dispatch, focus order, keyboard path · §7 code health · §4 state coverage |
| API surface changes | §3 · §8 docs and changeset · §2 if the prop is style-driving |
| A theme target is added | §2 · §5b screenshots · §3 — a target is public surface |
| Anything rendered moves | §5b · §9 RTL · §10 responsive · §1 contrast |
| O(N) work on a hot flow | §6 evidence · §7 · a perf test |
Why this table is the point. #5018
changed dismissal behavior. Its real defect was an accessibility one — a screen
reader dispatches a bare click with no pointerdown, and the guard swallowed
it. Nothing in the diff said "aria"; the syntactic table fired §1 only because
six unrelated lines happened to match. A behavior change should reach §1 by
design, not by luck.
The rubric is a component-audit instrument: every check points at the component. Route a review through it and the reviewer gets more thorough at exactly the wrong radius. On #5343 all three findings this arm missed were outside the component, and the arm without the rubric found all three.
So after both tables above, run a third — four questions, none of which the rubric asks:
| ask | what it caught on #5343 |
|---|---|
| Where does the fix NOT reach — is the bug still there? | Keyed to a 640px breakpoint; at 1440px the same label still clips. The PR's own story renders that state on any normal monitor. |
| Does an EXISTING target change value, not just count? | No target added, so §2 came back clean — but the footer stack's data-direction flips to vertical, and a theme selecting the horizontal stack stops matching. |
| Is anyone else already fixing this? | #5352, same author, open, already clamping in Dialog with no breakpoint. The ask became a deletion. |
| Render it in a real container, not the story |
100dvw ignores its container: the docs preview goes 468px inside a 458px column and runs off the page. |
Three of the four are about radius: the change's blast radius past its own branch, past its own file, past its own PR. The fourth is R15d — a component alone in a story is the one place reflow never shows.
Why the rubric cannot own these: "is someone else fixing this" is not a property of the code, and "does the fix reach the whole bug" needs the bug, which only a review has. These are review questions, so they stay critic-side.
Findings the current critic carries that have no rubric home. If this arm wins, they move — as rubric checks with ids, not critic prose.
| Gap | Nearest existing | Why it does not reach |
|---|---|---|
| New surface must solve a CLASS of problems, not one case | P38 | P38 asks for "a demonstrated use case" — one callsite. The bar is the class, because a single-case addition becomes the precedent for the next one and the surface ratchets shut. |
| A new theme target is permanent surface | §2's 5/5 anchor | The anchor says "justified by a named consumer need", but it is prose in a grading anchor, so nothing can cite it. Targets look free — no runtime behavior — so nobody counts them. |
| O(N) work on a hot flow needs a perf test | §6 | §6 has no perf row at all. The repo already has the convention: *-perf.test.tsx, colocated, build-excluded, asserting render counts, never milliseconds — a wall-clock budget is flaky on CI and gets deleted within a month. |
The cost of adding them: each is a new FIX/BLOCK-capable check, so under the rubric's versioning rule that is a minor bump and existing §2, §3 and §6 scores stop being comparable. The semantic trigger table is routing only — a patch, no re-audits.
Everything about the review as an artifact: length caps, voice, the presentation slots, staleness (R14d), anchor verification (R14e), attribution, recording (R17), contributor policy (R1g), order of consequence (R1b).
What it loses: R1g-surface, R18, R18b, R18c — the component-quality rules — which become rubric checks the reviewer cites privately.
Same PR, both arms, blind. A wins if the split costs nothing; B wins if routing through the rubric finds what A missed.
| A — current | B — this draft | |
|---|---|---|
| reads | Brief + Critic | Brief + Critic + Rubric with both trigger tables |
| judged on | did it find the real defect · did it invent one · did the presentation survive |