We scored 30 real websites against a 40-check design contract. Here is what we found. #14
LE-VAI
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We scored 30 real websites against a 40-check design-system contract — tokens, motion, accessibility, cadence, takt, poise, identity, interaction, performance, responsive, semantic, security, spec, and copywriting. No LLM, no heuristics. Every check is deterministic and reproducible.
The results were worse than we expected.
The headline
60% of sites scored D or F. Zero sites scored a B. The gap between "looks good" and "passes a deterministic design contract" is enormous.
What "pass" looks like
Across 30 sites × 40 checks = 1,110 total checks:
More checks warn than pass. A "warn" means the check detected something partial — a token file exists but uses bare hex instead of structured color, or reduced-motion is present but not tiered. The industry is in a half-adopted state.
Where sites fail the most
Tokens (avg 40.8 — the worst dimension)
11 of 30 sites score below 50 on token compliance. The common failures:
$type: colorwith colorSpace + components)$schemapointer — token files don't declare which spec they conform toThis is the W3C DTCG 2025.10 gap. The spec went stable in October 2025, and 84% of teams say they use design tokens — but most are using them informally, not in a conformant format.
Motion (avg 52.5 — second worst)
11 of 30 sites score below 50 on motion. The common failures:
prefers-reduced-motionis either absent or a kill switchwidth,height,top,leftinstead oftransform/opacityApple's HIG is the reference here — they tier reduced-motion (completely remove, soften, or keep) instead of treating accessibility as a binary kill switch. Only 1 site in our sample does this correctly.
The top 5
No site scored a B. The jump from C (top non-self score: 77.7) to A (100) is 22 points — that's the gap between "has tokens but uses bare hex" and "has structured DTCG-conformant tokens with a $schema pointer and colorSpace."
The bottom 5
The bottom 5 tells a story: the sites that award "good design" fail a deterministic design contract.
The Geist vs. Vercel split
The most interesting finding: Vercel's marketing site (vercel.com) scores 73.6 (C) while Vercel's design system (geist.dev) scores 50.3 (F). The marketing site outperforms the design system by 23 points. Marketing teams optimize for visual polish. Design system teams optimize for API stability. The contract rewards the former.
What this means
Try it
The scoring engine is deterministic, open, and free. No API key, no auth. The methodology is published, the contract is public, and the GitHub Action gates your CI on the same 40 checks.
Designesy is a design-system contract verification engine. 40 deterministic checks, A–F grade, DTCG token validation, WCAG 2.2 AA accessibility, Lottie motion validation. designesy.org.
All reactions