Screenshot Tests

[justinfagnani]

We need tests!

Testing this is complicated because of the service worker. Ideally we'd test the service worker under a number of the conditions that cause it to reload: initial visit, second visit, repeat visit with a stale worker, interacting with a page after the worker has been shut down, etc. The Workbox team might have some resources to help drive these scenarios.

We also need basic interaction tests: edit a file, see the change. And we need some visual regression tests.

We should probably look at using Web Test Runner, and use whatever it recommends for visual snapshot testing, if it does.

[justinfagnani]

The Workbox team pointed me to this for an example: https://github.com/GoogleChromeLabs/sw-appcache-behavior/blob/master/tests/test-suite.js

[e111077]

This came into my twitter feed yesterday

https://www.npmjs.com/package/@web/test-runner-visual-regression

[43081j]

Keep in mind they use pixelmatch internally, which can have trouble with font smoothing differences across different OSes. Most libraries will have the same problem, looks-same tries to handle this by some clever algorithms but still trips up on it sometimes.

Just something to take into account I guess. The one true solution is to always generate them in CI so they are consistent i think, knowing that people who clone the repo are likely to get slightly different screenshots.