Skip to content

compare a screenshot against a baseline, and what that costs the dependency-free promise #17

Description

@Flatts3000

The problem

A screenshot is currently an illustration a human looks at. To be an assertion, something has to
compare it against a known-good baseline and fail when it differs.

The proposal

gamebridge shot hall --baseline docs/baselines/hall.png --tolerance 0.02

Exit non-zero when the difference exceeds the tolerance, and write a diff image beside the shot so a
failure is inspectable rather than just red.

The dependency tension, stated up front

gamebridge is deliberately stdlib-only and zero-dependency, and that is a documented property in
its pyproject.toml and README. Image comparison wants Pillow.

Three options:

  1. An optional extra (pip install gamebridge[image]), keeping the core install clean.
  2. Hand-rolled PNG decoding for the narrow case we emit. Minecraft writes 8-bit RGBA PNGs, and
    zlib is stdlib, so this is smaller than it sounds but is still a decoder to maintain.
  3. Refuse, and leave image comparison to whatever the calling project already uses.

Option 1 is probably right, but it is a decision about the package's character, not just a feature.

Notes

  • Tolerance has to account for real nondeterminism. Even with frozen weather and time, dithering and
    driver differences move pixels, so an exact-match assertion will be flaky on a different machine.
  • Depends on the determinism issues to be worth much: comparing shots that legitimately differ every
    run just produces noise.

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions