Puppeteer to Playwright Migration Guide for e2e Testing

Background

In 2022, the end-to-end tests for WooCommerce Core were migrated from a Puppeteer/Jest-based setup to running in Playwright using Playwright's native test runner. There were a number of reasons for this migration:

• Increase the reliability/decrease the flakiness of the end-to-end test suite
• Simplify the code base, eliminate a good deal of boilerplate code in favour of built-in functions/conventions
• Speed up the test suite execution
• Position the test suite to use some of the advanced features available in Playwright (cross browser/device, visual testing)

A fair amount of research was done on how tests could be migrated from Puppeteer directly to Playwright, but at the end of the day there were so many inter-dependencies between the test environment and various helper packages that had been written, we had to take a step back and re-engineer a lot of the test suite.

Approach

The first step in the migration was installing the Playwright dependency in the WooCommerce monorepo and creating a basic test to ensure that all components were functioning as intended. One of the goals was to be as non-destructive as possible with the migration so that anybody depending on the existing end-to-end suite would still be able to make use of the Puppeteer framework that had been created. With that in mind, we opted to install Playwright alongside Puppeteer.

$ /plugins/woocommerce/tests/e2e/

Puppeteer tests and dependencies were kept in their original location.

$ /plugins/woocommerce/tests/e2e-pw

Playwright was installed in a new folder within the tests directory.

A standard installation of Playwright was performed:

$ pnpm dlx create-playwright

This creates a few files and folders automatically for you. package.json and package-lock.json are self-explanatory, and depending on your setup you may want to keep these or just make sure that you add @playwright/test as a dependency to your existing package.json.

playwright.config.js (or playwright.config.ts if you picked TypeScript) is the key configuration file for your test suite. You can actually create any number of configurations and select which configuration to use at runtime:

$ pnpx playwright test --config=tests/e2e-pw/playwright.config.js

Here are a few of the key options specified in the configuration:

• globalSetup is (as the name suggests) a file that runs before the test suite executes. There's more details about the contents of this file below.
• globalTeardown similar to above, but this executes after all tests run.
• reporter lets you specify any number of report formatters (list, html, json). Enabling the HTML reporter is very useful for debugging and troubleshooting your tests.
• baseURL set the root URL for the web site you'll be testing.
• stateDir used to save the authentication "states" for use during your tests. One of the unique features of Playwright is that you can sign in as a registered user (in global-setup) and save that signed in state for re-use in all your tests.
• projects is where you can configure which browser(s)/configuration(s) you want to test against. Playwright supports Chrome, Safari and Firefox right out of the gates.

global-setup.js (or .ts) is where you can run a number of things before executing your tests. If you need to populate the database, or "reset" a test site, this is a pretty good place to do it. global-setup is also a great place to sign in with any/all accounts you'll use during your test runs and save the authenticated state (so that you aren't wasting time signing in before individual tests). In the core tests, we also add a REST API token so that we can make use of the API for any setup/cleanup tasks throughout execution.

global-teardown.js (or .ts) is where you can run a number of things after all of your tests have executed. It's always a good idea to try to leave the test site in the same shape as when you started, so we remove the API key that we created during setup.

Writing tests

Playwright has a similar structure/organization to Puppeteer, with a few differences in syntax. The README gives an overview of test organization/structure. Additionally, the Playwright documentation has an exhaustive reference of all the actions and assertions available in Playwright.

Re-usability

The current implementation of Playwright is evolving as further tests are implemented and existing tests are refactored. Part of the refactoring effort is to increase the re-usability of code.

test-data/data.js is the start of centralizing test data. Structures exist within for customer and store information that can be used across tests.

utils/ is a collection of helpers for re-use throughout the tests