Upload screenshots to S3 and post comment on Github #107
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The comment is posted on the commit itself, which should show up in PRs. The one open question I have is how to deal with “auto merge” in Github. It could make actual regressions go unnoticed. Any ideas?
janpaul123
force-pushed
the
screenshots-s3
branch
from
7f5e9be
to
9999e05
Compare
So we don’t suddenly get different screenshots unintentionally.
To fight very minor rendering artifacts.
|
Is your thought to disable auto-merge or just recommend that we don't use it? |
stevekrouse
reviewed
Mar 2, 2022
zaplib/scripts/ci/browser_tests.sh
Outdated
| then | ||
| echo "SCREENSHOT_GITHUB_MESSAGE=[✅ No screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA)" >> $GITHUB_ENV | ||
| else | ||
| echo "SCREENSHOT_GITHUB_MESSAGE=[🤔 Screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA) Please look at the screenshots and tag this comment with 👍 or 👎. Only merge when both the PR author and all the reviewers are happy with the changes." >> $GITHUB_ENV |
There was a problem hiding this comment.
Suggested change
| echo "SCREENSHOT_GITHUB_MESSAGE=[🤔 Screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA) Please look at the screenshots and tag this comment with 👍 or 👎. Only merge when both the PR author and all the reviewers are happy with the changes." >> $GITHUB_ENV | |
| echo "SCREENSHOT_GITHUB_MESSAGE=[🤔 Screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA) Please look at the screenshots and tag this comment with 👍 or 👎. Only merge when both the PR author and a reviewer are 👍." >> $GITHUB_ENV |
There was a problem hiding this comment.
^ to make it clearer that 👍 means that there is no diff
There was a problem hiding this comment.
And also maybe just require only two 👍 s
There was a problem hiding this comment.
#nit and up to you if you like it
There was a problem hiding this comment.
^ to make it clearer that 👍 means that there is no diff
Oh but there can be a diff! 👍🏻 in that case means that the diff is intentional / desired.
stevekrouse
approved these changes
Mar 2, 2022
|
Exciting PR! This is now officially the codebase with the most CI I've ever worked on |
|
I see, so 👍 just means that things are good
…On Tue, Mar 1, 2022 at 5:31 PM JP Posma ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In zaplib/scripts/ci/browser_tests.sh
<#107 (comment)>:
> mv screenshots/ screenshots_report/
mv previous_screenshots/ screenshots_report/
mv diff_screenshots/ screenshots_report/
+aws s3 cp --recursive screenshots_report/ s3://zaplib-screenshots/$GITHUB_SHA
+
+if grep --fixed-strings '"newItems":[]' screenshots_report/reg.json && grep --fixed-strings '"deletedItems":[]' screenshots_report/reg.json && grep --fixed-strings '"failedItems":[]' screenshots_report/reg.json
+then
+ echo "SCREENSHOT_GITHUB_MESSAGE=[✅ No screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA)" >> $GITHUB_ENV
+else
+ echo "SCREENSHOT_GITHUB_MESSAGE=[🤔 Screenshot diffs found.](http://zaplib-screenshots.s3-website-us-east-1.amazonaws.com/$GITHUB_SHA) Please look at the screenshots and tag this comment with 👍 or 👎. Only merge when both the PR author and all the reviewers are happy with the changes." >> $GITHUB_ENV
^ to make it clearer that 👍 means that there is no diff
Oh but there can be a diff! 👍🏻 in that case means that the diff is
intentional / desired.
—
Reply to this email directly, view it on GitHub
<#107 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AARO2K7EWLZTHXVRDFZDT6TU53AFTANCNFSM5PSTVYZA>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
Per feedback
Missed this earlier. It's a good question. I'll disable it for now, and if we feel like that is too annoying then we find a better solution. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The comment is posted on the commit itself, which should show up in PRs.
The one open question I have is how to deal with “auto merge” in Github.
It could make actual regressions go unnoticed. Any ideas?