Skip to content

fix(SnapshotKitTesting): stabilize async snapshots on slow CI - #232

Merged
kyleve merged 3 commits into
mainfrom
codex/stabilize-snapshot-readiness
Aug 10, 2026
Merged

fix(SnapshotKitTesting): stabilize async snapshots on slow CI#232
kyleve merged 3 commits into
mainfrom
codex/stabilize-snapshot-readiness

Conversation

@kyleve

@kyleve kyleve commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Posted by an AI agent on kve's behalf.

Summary

  • add a hosted onReadyToMeasure seam for deterministic intrinsic/full-content readiness, with cancellation-aware timeout handling and distinct timing attribution
  • give snapshot-only GitHub runners an explicit 2× maximum settle/hook ceiling while preserving local defaults, minimum floors, quiet-window proof, cadence, and image tolerances
  • make Flyover publish the latest nonempty visible-preview expectation and await every active variant/generation load before intrinsic measurement
  • preserve already-loaded Flyover preview content across the measurement probe's transient rehost and isolate each appearance configuration with a fresh model

Why

The shard-2 failure on #227 showed FlyoverCanvas_iPad still changing after its four-second settle budget. Pixel settling is only a proxy for async readiness: on slower hosted runners, Flyover's serial preview builders could still be resolving when intrinsic measurement or capture began. Broadly increasing all waits would make failures slower without proving the intended content had loaded.

This change gives height-changing async content a deterministic pre-measure seam, uses Flyover's production loading lifecycle as that signal, and reserves the CI multiplier for genuine runner slowness at maximum ceilings only.

Review focus

  • SnapshotMeasurementHook races the hosted hook against the effective capture ceiling and relies on hook implementations cooperating with cancellation.
  • FlyoverPreviewReadiness tracks active load identity separately from the latest viewport expectation so early completions count, stale completions do not, and cancellation resumes every waiter exactly once.
  • The existing 1.5-second Flyover capture floor remains for time-based glass material adaptation; the new signal replaces only preview-loading guesswork.
  • No snapshot reference images changed.

Testing

  • ./swiftformat --lint
  • bash -n test
  • ./test SnapshotKitTests SnapshotKitTestingTests FlyoverTests — 117 tests passed
  • ./test FlyoverTests — 36 tests passed after the final active-load hardening
  • ./test --snapshots --only 'FlyoverSnapshotTests/FlyoverSnapshotTests/canvasAndList()' — five consecutive runs passed against the pushed commit
  • SNAPSHOT_SETTLE_TIMEOUT_MULTIPLIER=2 ./test --snapshots --snapshot-shard 2/2 — 22 snapshot tests passed

@kyleve
kyleve merged commit 599b4af into main Aug 10, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant