Take screenshots from pub_integration tests (store locally). #8575
Conversation
isoos
commented
Feb 19, 2025
isoos
commented
- Automatically takes separate screenshots for typical mobile, tablet and desktop widths and for dark and light themes (6 combinations total).
- Starting with 11 locations (66 screenshots), some full-page, some per-component screenshots.
- Comparison tool and github CI integrations in subsequent PRs.
| final _themes = ['dark', 'light']; | ||
| final _viewports = { | ||
| 'mobile': DeviceViewport(width: 400, height: 800), | ||
| 'tablet': DeviceViewport(width: 768, height: 1024), |
There was a problem hiding this comment.
Might I propose we don't do tablet (just because of the number of screenshots)? Also don't those normally have much higher resolution (same with phones).
I'm assuming this is more about aspect ratio?
There was a problem hiding this comment.
Why don't do tablet? I think it is a good way to check how the site behaves in narrower screen, and tbh. I don't think we will manually check tablet screens, but may catch if something degrades.
There was a problem hiding this comment.
I'm assuming this is more about aspect ratio?
Mostly the width.
There was a problem hiding this comment.
I'm mostly thinking that we already have a LOT of screenshots, so I don't mind it, I'm only concerned that it's hard to get an overview.
There was a problem hiding this comment.
Let's keep them for now, in the hope they are useful. If they are not, create too much noise, we may remove them or ignore them in most comparison reports.
| // Default screen with 16:10 ratio. | ||
| final desktopDeviceViewport = DeviceViewport(width: 1280, height: 800); | ||
|
|
||
| final _screenshotDir = Platform.environment['PUB_SCREENSHOT_DIR']; |
There was a problem hiding this comment.
How about we just always store the screenshots in .dart_tool/screenshots/.
- The
.dart_tool/folder is already there. - The
.dart_tool/folder is always in.gitignore.
There was a problem hiding this comment.
Borderline crazy idea:
.dart_tool/
screenshots/
<revision>/
image1.png
current/
image1.png
comparison.html
Whenever, we create screenshots we:
- Always write to
.dart_tool/screenshots/current/ - If
git status --porcelain(meaning we have no changes), then:- We also write to
.dart_tool/screenshots/<revision>/
- We also write to
- We always create
.dart_tool/screenshots/comparison.html- A simple dump HTML file showing screenshots side by side
- Contents of
.dart_tool/screenshots/current/on the right side - Contents of
<revision>on the left side, where you pick<revision>from a dropdown- We create a dropdown at the top of the HTML file.
- We run
git logto find relevant revisions- We put relevant revisions in dropdown, if
.dart_tool/screenshots/<revision>/exists locally. - We order the dropdown by
git log, and default to the most recent revision we have.
- We put relevant revisions in dropdown, if
We could also use gh api to download artifacts from github:
https://docs.github.com/en/rest/actions/artifacts?apiVersion=2022-11-28#get-an-artifact
Note
- An artifact on github is a zip folder.
- It's not possible to download an artifact without being authenticated!
- Artifacts can't be embedded in comments as images :(
If wanted to have before/after images on github, we'll probably have to do something borderline crazy like uploading the images to a separate branch or a GCS bucket.
Which probably involve more infrastructure than we care to stand up and maintain.
There was a problem hiding this comment.
Not a crazy idea, but let's not rush to it yet, and reach it maybe step-by-step. There is an immediate benefit with this + a comparison tool, and then we can add further automatization with github artifacts and/or per-revision storage...
3 participants
