Skip to content

doc-tests: ui/gallery.ts retina screenshot baseline mismatch (recurring) #244

@proggeramlug

Description

@proggeramlug

Context

./target/release/doc-tests --skip-xcompile reports 79/80 passing — the lone fail is ui/gallery.ts flagging a screenshot baseline mismatch. This shows up in CLAUDE.md entries v0.5.323, v0.5.324, v0.5.343 etc. as a pre-existing noise item that the author of each surrounding fix has explicitly noted "unrelated to this commit". It's been the lone fail for 30+ commits, so it warrants its own ticket so it can be properly tracked + fixed rather than continuously hand-waved past.

Symptom

The captured screenshot dimensions (or pixel content) don't match the blessed baseline at docs/examples/_baselines/<host-os>/gallery.png. The macOS baseline is at 900×970 per the v0.5.344 entry; v0.5.344 also notes the Linux baseline was re-blessed at 900×1024 from the v0.5.343 CI artifact. The actual variance the host run is hitting is undocumented.

Likely causes (need investigation)

  1. Retina scaling: macOS captures at 2× backing scale; the screenshot infrastructure may emit a 1800×1940 PNG against a 900×970 baseline (or vice versa). The doc-tests harness should either pin the baseline at the retina dimensions or down-scale the capture.
  2. Font kerning / sub-pixel hinting: Apple fonts (SF Pro / Segoe UI / Linux fontconfig) render slightly differently across host versions; even a one-version macOS bump can shift glyph positions enough to break a strict per-pixel diff.
  3. Color profile: macOS Display P3 vs sRGB roundtrip in the PNG encoder.

Repro

cargo build --release -p perry-runtime -p perry-stdlib -p perry
./target/release/doc-tests --skip-xcompile --filter ui/gallery.ts --verbose

The diff/SSIM score (or whatever the harness reports) will pin the actual delta.

Acceptance criteria

  • doc-tests --skip-xcompile reports 80/80 passing on a fresh macOS-14 CI runner without any --filter-exclude.
  • The chosen fix is documented (re-bless + tolerance, retina-aware capture, font-stable rendering, etc.) so the next baseline drift has a path forward.

Out of scope

Other doc-tests skips (e.g., the platform-banner-skipped ones documented in v0.5.345) are intentional, not regressions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions