Evidence Templates
Complementary data a reviewer produces so a human can judge fast. Not the review — this is a separate bot-attributed comment, tables only, no prose.
Gap-filler rule: skip anything CI already reports (the PR Analysis Report already posts bundle size, a11y audit, Storybook + sandbox preview links) and anything the PR description already says. Post only what is missing.
Before choosing frames, record:
VISUAL CHECK: manual frames required | visual gate sufficient | not applicable
WHY: <evidence>
A behavior-only change may use the visual gate instead of manual frames only
when that job is green on the exact head and named stories cover both visible
endpoints. A green check without endpoint-story coverage is not enough. Pixels
changing intentionally, a claim of no visual change despite render/style work,
an uncovered visible endpoint, or a missing/red/skipped gate requires manual
frames. not applicable is reserved for genuinely non-rendering changes and
names the changed paths that prove it.
The decision is part of the review evidence; screenshots are never silently omitted.
When manual frames are required, start with:
- One finding pair — before/after of the exact visual claim.
- One control or far-side pair only when needed — the nearest case that should remain unchanged, or the first value just outside the fix's condition.
Examples:
- hidden/modal ancestor bug → same component outside the Dialog
-
max-width: 640pxfix → 641px - error/persistent-toast fix → ordinary auto-hiding info toast
- RTL fix → LTR
- selected-state fix → unselected state
The control proves attribution: the change acts where expected without becoming a general regression. Do not build a gallery. Add another frame only when it settles a separate visual claim, boundary, theme, or composition seam that the first two pairs cannot.
All frames still require sensor receipts. A single receipt may cover a batch whose non-state sensors are identical, but every semantic state has its own state assertion.
A screenshot is not evidence until the capture proves it is the intended render. Emit this table with every screenshot matrix; one row may cover a batch only when every value except state is identical.
| Sensor | Expected | Observed | Pass? |
|---|---|---|---|
| Build | before/head SHA + server/worktree | ||
| Story | exact Storybook story id | ||
| Theme |
astryxTheme global + rendered data-astryx-theme
|
||
| Color mode |
colorMode global + computed root color-scheme + surface luminance class |
||
| Direction |
direction global + computed direction on the subject |
||
| Viewport/media | CSS width×height, DPR, forced-colors, reduced-motion, pointer/hover mode | ||
| Rendered state | role/name/value/ARIA/text/count that uniquely identifies the intended state | ||
| Subject geometry | selector count, visible, non-zero box, intended clipping/viewport relationship | ||
| Settled render | fonts loaded; expected animation state; no Storybook error screen/page error | ||
| Image | output path, pixel dimensions, non-blank hash |
The state sensor is the most important row. #storybook-root * or “a button exists” is not a state sensor. Assert something unique to the build and case: the error toast is persistent and has exactly one dismiss control; the selected row has aria-selected=true; the narrow card actually wraps; the theme override moved the target property.
Concrete Storybook failure: core-button--primary contains three injected,
zero-size button controls before the real component. target: 'button'
selected a 0×0 decoy while the actual .astryx-button was the fourth match at
123×32. A viewport screenshot still looked correct. The geometry sensor failed
it; the old “a button exists” readiness check would have accepted it.
targetCount is an authored expectation, not a discovery query. State the count
from the component contract (for example, exactly one primary Button), then let
the sensor compare it. Reading the count first and feeding the same value back
is a tautology and earns no evidence.
Before/after pairs must have matching receipts for story, theme, mode, direction, viewport/media and state. Only Build may differ. If another sensor differs, the pair is not a comparison.
A valid receipt proves which build rendered; it does not prove that two builds differ only in the change being tested. For every hand-built/mutated arm, emit:
ARM: <label>
BASE: <commit/worktree>
DELTA: <exact files + hunks/declarations changed from base>
UNRELATED DELTA: none | <list>
Bank the arm diff beside the receipts (<arm>.diff) and inspect it before
interpreting pixels. A bundled declaration, removed margin, changed alignment,
or extra state invalidates attribution even when every screenshot sensor passes.
Checked-out parent/head pairs may use the reviewed PR diff; mutation/spike arms
always need their own delta.
Sensor expectations come from the intended contract before observing the
page. A failure is evidence: inspect the DOM/state or fix the capture. Do not
copy the observed value into expected, derive targetCount from the page, or
weaken the selector until it passes.
State sensors use semantic/user-observable values: role/name/ARIA/value,
innerText, or an accessibility query. Avoid textContent: components may
render aria-hidden sizer/measurement copies (DetailsDetails) that a user and
AT perceive once. When the DOM contains intentional duplicates, the state
receipt records the accessible value and separately records the expected DOM
shape if that shape matters.
Fail closed:
- unknown Storybook globals are ignored silently, so assert their rendered effect
- a URL parameter is not evidence that the decorator applied it
-
networkidleis not readiness under Storybook dev - a generic DOM selector can match the previous HMR render
- byte-identical light/dark frames require proving that identical output is expected; otherwise treat them as a sensor failure
- a screenshot captured during an unintended transition is invalid; either wait for stable pixels or record the exact animation time/state intentionally captured
Use captureWithSensors() from ~/astryx/probe-kit/lib.cjs; it writes a .sensors.json receipt beside the PNG and throws before capture when a required expectation does not match.
A layout change is never proved by one before/after at one width. Parents and grandparents change how an element renders, so the evidence has to name the containment assumptions being relied on.
State what the fix depends on. Each row is a claim a human can challenge.
| Assumption | This change relies on | Verified |
|---|---|---|
| Parent display | e.g. flex row |
yes/no |
| Which element shrinks | e.g. the <button> flex child, not the inner span |
yes/no |
| Width source | container-defined / children auto-size | both tested? |
| Min-size defaults | e.g. flex children default min-width: auto
|
yes/no |
| Ancestor constraints | any overflow, contain, grid track, or fixed height above it |
yes/no |
The #5035 lesson: minWidth: 0 on the inner span does nothing while its parent
flex child is unshrinkable. Always say which box is being constrained.
One row per case, before and after. Include at least one case where the change is visible, and say plainly when a case is unchanged.
| Case | Before | After | What changed |
|---|---|---|---|
| Baseline (short content, wide) | identical / … | ||
| Long content, container-defined width | |||
| Long content, auto-sized container | |||
Component's own layout variants (e.g. hug vs fill) |
|||
| Narrow viewport (320px) | |||
| Mixed: one long child + short siblings | |||
| RTL | |||
| Text zoom 200% | |||
| Forced colors | |||
| Icon-only / label-hidden path |
Drop rows that genuinely do not apply — say why, do not silently omit.
If the change does not achieve its stated goal, add a third column showing the minimal delta that does, and name the one-line diff. That is what turns "this looks wrong" into an actionable review.
| Added | Nearest sibling precedent | Matches? |
|---|---|---|
propName?: Type |
Sibling.tsx:123 uses otherName for the same axis |
yes/no |
Plus: prop-overlap count against the base component if it wraps one; whether the new surface is reachable from the components that motivate it (a Calendar prop the date inputs cannot forward is a gap); exports added to the public barrel.
| Path | Covered by the change | Covered by a test |
|---|---|---|
| pointer click | ||
| keyboard (Enter/Space/Escape/arrows) | ||
| IME composition | ||
| paste | ||
| AT / synthetic activation | ||
| programmatic call |
Interaction fixes cannot be proved in jsdom (no popover light dismiss, no invokers) — say so, and drive real Chromium when it matters.
- What was not verified, and why.
- Which claims are read off source versus observed in a browser.