Skip to content

test(goldens): scoped 0.5% diff threshold for chart-line drift#624

Merged
TaprootFreak merged 1 commit into
stagingfrom
chore/golden-diff-threshold-home
Jun 1, 2026
Merged

test(goldens): scoped 0.5% diff threshold for chart-line drift#624
TaprootFreak merged 1 commit into
stagingfrom
chore/golden-diff-threshold-home

Conversation

@TaprootFreak
Copy link
Copy Markdown
Contributor

Summary

Scopes a diffThreshold: 0.005 only on the home_page_loaded golden via AlchemistConfig.runWithConfig. Every other golden stays at the default 0.0 (exact-match).

Why

The home_page_loaded golden renders a stroked price-chart path. Sub-pixel anti-aliasing along that path produces coverage values that jitter across runs even on the locked dfx01 renderer — hardware-lock is not enough for stroked-path AA. Evidence:

  • 044852b (28.05.26, bot regen) refreshed home_page_loaded and buy_registration_required
  • d25c61d (01.06.26, fix(kyc): keep financial-data answers on submit failure (retryable) #620) refreshed home_page_loaded again — only 4 days later, no UI change in scope
  • Pixel diff between the two home_page_loaded versions: 0.24% of pixels changed, clustered in the chart-curve region only (rows 129–244, cols 252–385).

Both regens were the correct fix in the moment (a real pixel drift had to be reconciled), but they're paying recurring cost for a structural property of the chart widget.

Why scoped, not global

A global diffThreshold would weaken the gate for every screen — including 93 static-content goldens where 0.0 is exactly right. A small icon removal (~256 px on 329 160-pixel surface) is well under 0.5% and would be silently swallowed.

Local scope is the surgical fix: the chart-drift screen tolerates ~0.5%, everything else stays strict.

Why 0.005 (not lower or higher)

  • Current chart drift: 0.238% of pixels → 0.005 absorbs it with ~2× headroom for next time Skia jitters slightly more.
  • Sanity ceiling: any meaningful UI regression in a 390×844 viewport (icon, button, label) covers >2% — well above the threshold. The gate still catches real changes.

Alternative considered — not chosen

Hardening the chart with StrokeJoin.bevel + StrokeCap.butt in the production code would remove drift at the source, but at the cost of making the chart visually less smooth for end users to satisfy a test concern. Wrong direction.

Verification

  • flutter analyze test/goldens/screens/home/home_golden_test.dart → No issues found.
  • Threshold is read at goldenTest registration time via Zone.current (see alchemist/src/golden_test.dart:163), so the zone-based override correctly reaches the comparator's variantConfig.diffThreshold (golden_test.dart:195).
  • The wrapper goldenTest from test/helper/golden_test_with_assets.dart is preserved (alchemist import uses show AlchemistConfig, PlatformGoldensConfig to avoid ambiguous import on goldenTest).

The home_page_loaded golden renders a stroked price-chart path whose
sub-pixel anti-aliasing coverage values jitter across runs even on the
locked dfx01 renderer — bot commit 044852b and the d25c61d follow-up
regenerated the same baseline twice within five days, both touching
only the curve region.

Wraps just this one goldenTest in AlchemistConfig.runWithConfig with
diffThreshold: 0.005. All other goldens stay at the 0.0 default — they
have no comparable drift surface (static text/icons, no stroked paths).

The threshold is well below the size of any meaningful UI change: a
typical icon, glyph, or button removal covers >2% of pixels, so the
gate still catches real regressions while absorbing the AA noise that
drove the manual regen cycle.
@TaprootFreak TaprootFreak marked this pull request as ready for review June 1, 2026 14:43
@TaprootFreak TaprootFreak changed the base branch from develop to staging June 1, 2026 14:44
@TaprootFreak TaprootFreak merged commit bd2830d into staging Jun 1, 2026
14 of 15 checks passed
@TaprootFreak TaprootFreak deleted the chore/golden-diff-threshold-home branch June 1, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant