-
Notifications
You must be signed in to change notification settings - Fork 177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix flaky Playwright tests #3759
Conversation
Signed-off-by: Olga Bulat <obulat@gmail.com>
Signed-off-by: Olga Bulat <obulat@gmail.com>
Signed-off-by: Olga Bulat <obulat@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! I wonder if it's worth excluding fonts from the playwright test, or at least, excluding them from maybe all but one specific one? It feels like whack-a-mole if we're adding sleep to tests as they appear flaky, when I suppose any test could technically have suddenly-new flakiness due to font loading?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
I think I understood why the font loading issue happens for the internal header in storybook. The first rendered element does not contain any text, only the mobile header with logo and a button. That's why we would only load the font after opening the modal, which does have some text. I'll review the tests for similar situations. |
Fixes
Fixes #3713 by @sarayourfriend
Fixes #3714 by @sarayourfriend
Description
This PR attempts to fix the flaky Storybook Playwright tests by adding waits or testing that a font request was sent after the menu button is clicked, to make sure the fonts are loaded before taking the snapshots.
I merged the sensitive results test fix into this PR since both PRs were failing the CI due to the other test failure (sensitive result fix was failing due to Storybook failure, and the Storybook fix was failing due to sensitive results failure).
Instead of going to the homepage, and making a new search to check that sensitive results were loaded, I added a check for sent API request with the sensitive parameter.
Testing Instructions
The CI should pass.
Checklist
Update index.md
).main
) or a parent feature branch.Developer Certificate of Origin
Developer Certificate of Origin