Visual-regression snapshots that are the docs screenshots - #166
Merged
Conversation
One pipeline, two jobs. tests/visual.test.ts drives the real dashboard (built SPA + real platform on ephemeral Postgres, seeded through the real ingest wire) in a real Chromium across the 9 documented surfaces and compares each capture against its committed baseline — where the baseline IS the image the docs site embeds. A UI change either fails here as a visual regression, or is accepted with VX_UPDATE_SNAPSHOTS=1, which rewrites the baselines and therefore the docs screenshots in the same commit. Docs can no longer rot behind the product. Determinism by construction: the seed is anchored to a fixed epoch, the browser clock is frozen to that instant, animations are disabled, and the shutter fires only when two consecutive captures are byte-identical (a measurement, not a magic timeout). Comparison is dependency-free — helpers/png.ts hand-rolls the PNG reader and a tolerance differ, the tar.ts/sigv4.ts precedent. Building it surfaced two real defects: - box-sizing was never reset, so every padded full-width element overflowed by exactly its padding — scrollWidth 1648 vs clientWidth 1600 on every dashboard page, a permanent horizontal scrollbar (and why the old screenshots clipped their right column). Fixed in the UnoCSS preflight; measured 1600/1600 after. - The committed perf guard was silently skipping: bun test doesn't consult NODE_PATH and playwright here is a global install, so it had never resolved a browser. A shared helpers/playwright.ts loader now serves both suites and the perf guard runs for the first time. Differentially verified: perturbing shell padding reds 5 shots at 8-11% of pixels with the capture parked for inspection; reverted, green. Skips stay honest — no browser or no built SPA means skip, never fail. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RW7aso5j5CrBo7cjyET23D
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One pipeline, two jobs.
packages/cloud/tests/visual.test.tsdrives the real dashboard (built SPA + real platform on ephemeral Postgres + fake S3, seeded through the real/v1/ingestwire) in a real Chromium across the 9 documented surfaces, and compares each capture against its committed baseline — where the baseline IS the image the docs site embeds (apps/docs/src/assets/screenshots/*.png, the exact files the cloud docs already reference).So a UI change either fails here as a visual regression, or is accepted with
VX_UPDATE_SNAPSHOTS=1, which rewrites the baselines and therefore the docs screenshots in the same commit. Docs screenshots can no longer silently rot behind the product.Determinism by construction
addInitScript, so "2h ago" renders identically forever.stableShot— a measurement, not a magic timeout).helpers/png.tshand-rolls the PNG reader (IHDR/IDAT +node:zlibinflate + all five unfilters incl. Paeth → RGBA) and a per-channel-tolerance differ, following thetar.ts/sigv4.tsprecedent. ~100 ms per 3200×2000 image.Two real defects surfaced by building it
box-sizingwas never reset. The UnoCSS preflight lacked the universal border-box rule, so every padded full-width element overflowed by exactly its padding —scrollWidth1648 vsclientWidth1600 on every dashboard page (a permanent horizontal scrollbar, and why the old screenshots clipped their right-hand column). Fixed; measured 1600/1600 after.bun testdoesn't consultNODE_PATHand this container's Playwright is a global install, soui-perf.test.tshad never resolved a browser. A sharedhelpers/playwright.tsloader (env override → NODE_PATH → conventional global prefixes, importing the package directory to bypass node resolution) now serves both suites — the perf guard runs for the first time (5 pass).Verification
p-6→p-10reds 5 shots at 8–11% of pixels, with each capture parked in tmp for eyeballing and an actionable message; reverted → green.oxfmt --check .clean ·oxlint --type-aware --type-check0 · cloud 549/0 · core from root 1286/0 · the docs site builds clean on the regenerated images.No schema/wire/CACHE bump (test infra + one CSS reset rule).
🤖 Generated with Claude Code
https://claude.ai/code/session_01RW7aso5j5CrBo7cjyET23D
Generated by Claude Code