Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix two flaky e2e tests #47562

Merged
merged 6 commits into from
May 20, 2024
Merged

Fix two flaky e2e tests #47562

merged 6 commits into from
May 20, 2024

Conversation

lanej0
Copy link
Contributor

@lanej0 lanej0 commented May 16, 2024

Submission Review Guidelines:

Changes proposed in this Pull Request:

Fixes two flaky tests: basic.spec.js (which was identified by Buildkite as being flaky) and Closes: #46666

How to test the changes in this Pull Request:

Using the WooCommerce Testing Instructions Guide, include your detailed testing instructions:

  1. Ensure CI passes
  2. pnpm env:restart && pnpm test:e2e-pw basic.spec.js --repeat-each 5
  3. pnpm test:e2e-pw checkout-block.spec.js --repeat-each 5

Changelog entry

  • Automatically create a changelog entry from the details below.

Significance

  • Patch
  • Minor
  • Major

Type

  • Fix - Fixes an existing bug
  • Add - Adds functionality
  • Update - Update existing functionality
  • Dev - Development related task
  • Tweak - A minor adjustment to the codebase
  • Performance - Address performance issues
  • Enhancement - Improvement to existing functionality

Message

Comment

@lanej0 lanej0 changed the title E2e/fix/flaky test fixes Fix two flaky tests May 16, 2024
@woocommercebot woocommercebot requested a review from a team May 16, 2024 22:35
@github-actions github-actions bot added focus: e2e tests Issues related to e2e tests plugin: woocommerce Issues related to the WooCommerce Core plugin. labels May 16, 2024
@lanej0 lanej0 changed the title Fix two flaky tests Fix two flaky e2e tests May 16, 2024
Copy link
Contributor

github-actions bot commented May 16, 2024

Hi @adimoldovan,

Apart from reviewing the code changes, please make sure to review the testing instructions as well.

You can follow this guide to find out what good testing instructions should look like:
https://github.com/woocommerce/woocommerce/wiki/Writing-high-quality-testing-instructions

@@ -1,7 +1,8 @@
const { expect } = require( '../fixtures/fixtures' );
const logIn = async ( page, username, password, assertSuccess = true ) => {
await page.waitForSelector( 'form' );
await page.getByLabel( 'Username or Email Address' ).click();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meant to add a comment on this:

I know this is a goofy thing, and a .fill() shouldn't need a .click() preceding it. It looks like in about 10% of executions, Playwright starts to fill in the login form before the page has finished loading and that there is a focus() on the username field that fires right before we start filling out the password -- so the password ends up overwriting the username (and that causes the flakiness/failure). Explicitly clicking on each field before filling it out seems to have fixed that.

@adimoldovan adimoldovan self-requested a review May 20, 2024 07:25
Copy link
Contributor

@adimoldovan adimoldovan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lanej0 for working on this. It looks much better.

@adimoldovan adimoldovan merged commit 4d7a691 into trunk May 20, 2024
21 checks passed
@adimoldovan adimoldovan deleted the e2e/fix/flaky-test-fixes branch May 20, 2024 07:50
@github-actions github-actions bot added this to the 9.0.0 milestone May 20, 2024
@github-actions github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label May 20, 2024
@rodelgc rodelgc added status: analysis complete Indicates if a PR has been analysed by Solaris and removed needs: analysis Indicates if the PR requires a PR testing scrub session. labels May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
focus: e2e tests Issues related to e2e tests plugin: woocommerce Issues related to the WooCommerce Core plugin. status: analysis complete Indicates if a PR has been analysed by Solaris
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Flaky] shopper/checkout-block.spec.js
3 participants