Update Playwright to 1.44.1 from 1.41.1 #48291

lanej0 · 2024-06-07T18:17:19Z

Submission Review Guidelines:

I have followed the WooCommerce Contributing Guidelines and the WordPress Coding Standards.
I have checked to ensure there aren't other open Pull Requests for the same update/change.
I have reviewed my code for security best practices.
Following the above guidelines will result in quick merges and clear and detailed feedback when appropriate.

Changes proposed in this Pull Request:

We had fallen a bit behind in Playwright in the monorepo, mostly because Playwright introduced this breaking change a few versions back. This PR updates Playwright to the latest release and refactors some shopper/merchant tests in core so that they're compatible with the latest version.

Also updates the Blocks e2e tests to the latest version (ran tests, and no apparent issues with new version of Playwright)

How to test the changes in this Pull Request:

Using the WooCommerce Testing Instructions Guide, include your detailed testing instructions:

Ensure CI passes for both Core and Blocks e2e tests
pnpm env:restart && pnpm test:e2e-pw (for core tests). Will probably be prompted to install updated browsers first.
pnpm env:restart && pnpm test:e2e (for blocks tests)

Changelog entry

Automatically create a changelog entry from the details below.

This Pull Request does not require a changelog entry. (Comment required below)

Changelog Entry Details

Significance

Patch
Minor
Major

Type

Fix - Fixes an existing bug
Add - Adds functionality
Update - Update existing functionality
Dev - Development related task
Tweak - A minor adjustment to the codebase
Performance - Address performance issues
Enhancement - Improvement to existing functionality

Message

Changelog Entry Comment

Comment

github-actions · 2024-06-07T18:17:51Z

Hi @WunderBart, @gigitux, @woocommerce/vortex

Apart from reviewing the code changes, please make sure to review the testing instructions as well.

You can follow this guide to find out what good testing instructions should look like:
https://github.com/woocommerce/woocommerce/wiki/Writing-high-quality-testing-instructions

github-actions · 2024-06-07T18:27:22Z

Test using WordPress Playground

The changes in this pull request can be previewed and tested using a WordPress Playground instance.
WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Test this pull request with WordPress Playground.

Note that this URL is valid for 30 days from when this comment was last updated. You can update it by closing/reopening the PR or pushing a new commit.

github-actions · 2024-06-07T18:32:02Z

Size Change: +10 B (0%)

Total Size: 2.47 MB

_{compressed-size-action}

lanej0 · 2024-06-07T21:35:33Z

@WunderBart @gigitux -- I'd like to update Playwright to the latest version in the Monorepo (we're about 3 versions old currently). Looks like everything is okay, except there's one Blocks Playwright test that seems to have turned flaky with this PR.

Releasing the `page.goto` before the `load` event (default) doesn't really speed up the tests because we still need to wait for the locator that we're going to interact with. Furthermore, it can destabilize the test because the said locator timer starts earlier than it normally would (after page "load"), so it might timeout in a situation when a page takes longer to load than usually.

…ailable

WunderBart · 2024-06-10T12:18:31Z

plugins/woocommerce-blocks/tests/e2e/utils/mini-cart/mini-cart-utils.page.ts

+		// The mini cart button scripts are loaded when the button is either
+		// hovered or focused. The click event alone does not trigger neither of
+		// those actions so we need to perform one explicitly.
+		await miniCartButton.hover();
+		await miniCartButton.click();


This should fix the failing mini cart tests. Those tests failed because the cart scripts aren't loaded until the cart button is explicitly either hovered or focused. This is an implementation detail and, ideally, it should not be necessary to account for it in the test. However, it is weird that the Playwright's physical click event does not seem to lead with hover, as it's only possible to click with hovering first. I'll try to investigate whether it's a bug or an intentional change with the newer release.

cc: @Aljullu, since you last worked on mini cart frontend scripts.

Thanks for the explanation, @WunderBart! It makes sense to me. 👍

I've investigated it a bit further, and the issue seems to be that the mini cart button scripts (loaded lazily on button hover) aren't hydrating the button element — they remove it and replace it with a new one. This means there's no continuity in the click handler, which should be enough to load the scripts lazy and open the mini cart drawer. A workaround could be re-querying the button, for which I put together a PoC here: #48330.

In general, the current implementation should not be an issue with fast machines/connection, but it can cause further flakiness due to the disconnection between the button being ready to be clicked and the actual click happening. Let me know if it makes sense.

I've investigated it a bit further, and the issue seems to be that the mini cart button scripts (loaded lazily on button hover) aren't hydrating the button element — they remove it and replace it with a new one. This means there's no continuity in the click handler, which should be enough to load the scripts lazy and open the mini cart drawer.

Right, good catch! I didn't think that was the reason of the issue, but it makes total sense. 💯

I know the issue is clear, but let me share these awesome diagrams I created by explaining the issue to GPT and asking for a mermaid output. 😅 TIL that it works that well! 💪

Successful scenario

sequenceDiagram participant User participant CartButton participant Scripts User->>CartButton: Hover CartButton->>Scripts: Trigger lazy load Scripts-->>CartButton: Scripts loaded User->>CartButton: Click CartButton->>Scripts: Open cart drawer Scripts-->>User: Cart drawer opened

Loading

Failure scenario

sequenceDiagram participant User participant CartButton participant Scripts User->>CartButton: Hover CartButton->>Scripts: Trigger lazy load User->>CartButton: Click CartButton->>User: No action Scripts-->>CartButton: Scripts loaded

Loading

lanej0 · 2024-06-10T18:49:46Z

Not sure why the JS Unit tests are having issues. Seems to be different ones, so maybe just flaky?

WunderBart · 2024-06-11T08:23:58Z

Not sure why the JS Unit tests are having issues. Seems to be different ones, so maybe just flaky?

Looks like pnpm 9.1.4 is breaking @wordpress/data dependency. The solution seems to be downgrading to 9.1.3 (see p1717495487961929/1717436616.890969-slack-C03CPM3UXDJ)

adimoldovan · 2024-06-11T17:39:16Z

🤔 I wonder if the lint errors in @woocommerce/block-library are caused by #48234

adimoldovan · 2024-06-11T17:50:04Z

I wonder if the lint errors in @woocommerce/block-library are caused by #48234

It looks like it. I reverted that change with 8186094

adimoldovan

Looks good. Thank you @lanej0 for this much needed update.

Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com> Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>

Jon Lane added 4 commits June 7, 2024 10:59

Update Playwright

34cab2e

Fix test syntax

fceea42

Add changelog

4d094f9

Update syncpack and blocks

72d57ff

github-actions bot added focus: e2e tests Issues related to e2e tests focus: monorepo infrastructure Issues and PRs related to monorepo tooling. plugin: woocommerce Issues related to the WooCommerce Core plugin. labels Jun 7, 2024

woocommercebot requested a review from a team June 7, 2024 18:17

lanej0 requested review from WunderBart and gigitux June 7, 2024 18:18

Update lockfile for Blocks

6a59f82

Jon Lane added 4 commits June 7, 2024 12:58

Merge branch 'trunk' into dev/update-playwright-1_44_1

c8c3cae

Fix for new flaky test

873778c

Fix flaky variations test

6d3785b

Fix flaky test

9cb9bed

WunderBart added 2 commits June 10, 2024 12:25

Address the issue of mini cart button script not being immediately av…

1d627f3

…ailable

WunderBart force-pushed the dev/update-playwright-1_44_1 branch from 8877a70 to 1d627f3 Compare June 10, 2024 10:29

fix the mini cart button by triggering scripts load

eed21fc

WunderBart reviewed Jun 10, 2024

View reviewed changes

WunderBart and others added 2 commits June 10, 2024 14:32

fix linter error

bbc6995

Merge branch 'trunk' into dev/update-playwright-1_44_1

03c4338

Remove upload plugin test (no longer used)

fa20592

Downgrade pnpm to 9.1.3

f2a0f3b

Merge branch 'trunk' into dev/update-playwright-1_44_1

feaf3a5

Build the project in lint jobs

8186094

adimoldovan approved these changes Jun 11, 2024

View reviewed changes

adimoldovan merged commit e5e51a4 into trunk Jun 11, 2024
96 checks passed

adimoldovan deleted the dev/update-playwright-1_44_1 branch June 11, 2024 18:52

github-actions bot added this to the 9.1.0 milestone Jun 11, 2024

github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label Jun 11, 2024

Stojdza added status: analysis complete Indicates if a PR has been analysed by Solaris and removed needs: analysis Indicates if the PR requires a PR testing scrub session. labels Jun 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Playwright to 1.44.1 from 1.41.1 #48291

Update Playwright to 1.44.1 from 1.41.1 #48291

lanej0 commented Jun 7, 2024

github-actions bot commented Jun 7, 2024 •

edited

Loading

github-actions bot commented Jun 7, 2024 •

edited

Loading

github-actions bot commented Jun 7, 2024 •

edited

Loading

lanej0 commented Jun 7, 2024

WunderBart Jun 10, 2024 •

edited

Loading

Aljullu Jun 10, 2024

WunderBart Jun 10, 2024 •

edited

Loading

Aljullu Jun 11, 2024

WunderBart Jun 11, 2024

lanej0 commented Jun 10, 2024

WunderBart commented Jun 11, 2024

adimoldovan commented Jun 11, 2024

adimoldovan commented Jun 11, 2024

adimoldovan left a comment

Update Playwright to 1.44.1 from 1.41.1 #48291

Update Playwright to 1.44.1 from 1.41.1 #48291

Conversation

lanej0 commented Jun 7, 2024

Submission Review Guidelines:

Changes proposed in this Pull Request:

How to test the changes in this Pull Request:

Changelog entry

Significance

Type

Message

Comment

github-actions bot commented Jun 7, 2024 • edited Loading

github-actions bot commented Jun 7, 2024 • edited Loading

Test using WordPress Playground

github-actions bot commented Jun 7, 2024 • edited Loading

lanej0 commented Jun 7, 2024

WunderBart Jun 10, 2024 • edited Loading

Choose a reason for hiding this comment

Aljullu Jun 10, 2024

Choose a reason for hiding this comment

WunderBart Jun 10, 2024 • edited Loading

Choose a reason for hiding this comment

Aljullu Jun 11, 2024

Choose a reason for hiding this comment

WunderBart Jun 11, 2024

Choose a reason for hiding this comment

Successful scenario

Failure scenario

lanej0 commented Jun 10, 2024

WunderBart commented Jun 11, 2024

adimoldovan commented Jun 11, 2024

adimoldovan commented Jun 11, 2024

adimoldovan left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 7, 2024 •

edited

Loading

github-actions bot commented Jun 7, 2024 •

edited

Loading

github-actions bot commented Jun 7, 2024 •

edited

Loading

WunderBart Jun 10, 2024 •

edited

Loading

WunderBart Jun 10, 2024 •

edited

Loading