New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🏗✅ Make visual diff tests hermetic #21935
🏗✅ Make visual diff tests hermetic #21935
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! How does this look on Travis?
Approving to unblock.
Literally just sent this PR to review, it didn't even have time to run on Travis 😅 |
Hi, I wrote the GPT.js test which is the only test remaining that requires external access. The test never loaded AMP locally since there is no support for proxying (I could change GPT itself so it loads AMP locally but didn't find the time to do so), and with your submitted changes it would likely break entirely. Even with that fixed the test relies on loading |
@zombifier done - I removed the optional flag and disabled (but not completely removed) your test. I'll be happy to work with you to modify this test to be hermetic! |
@rsimha approve the Percy build for me? Can always use an extra pair of eyes instead of blindly approving mine. Things to note:
|
Approved diffs. Some images appear to have changed, which seems reasonable given what you're doing in this PR. A good check might be to re-run this and see if the same diffs occur. |
I reran this several times locally (I don't want to jinx this successful execution by rerunning it on Travis… there's still #21882 that should help fix the timeouts) and the results are consistent. Merging this now… |
This PR:
allow_external_network_access
set totrue
cdn.ampproject.org
orfonts.googleapis.com
. Those are now MOCKED by the test runner (i.e., the test runner returns a local, frozen copy of those external assets)Progress on #21056
*all that are required to make the tests pass