Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add infrastructure for writing Playwright E2E tests #38570

Merged
merged 6 commits into from
Mar 22, 2022

Conversation

kevin940726
Copy link
Member

@kevin940726 kevin940726 commented Feb 7, 2022

Not set in stone, feedback welcome.

Follows the previous proposal and the meeting notes.

Prior art: #34089, #38401.

Notable changes

  • Use @playwright/test rather than jest + playwright. The playwright test runner offers more features out of the box specifically for e2e testing. It's also one of the primary focuses of the playwright team (seems like they're now branding playwright as a test framework).
  • I created a separated package e2e-test-utils-playwright for all the utils used in Playwright. The new tests written in Playwright should be in test/e2e/specs.
  • This PR adds a sanity.spec.js just to show that it works as expected. It should be removed once we have other tests migrated.
  • I haven't written any doc yet, but I have some ideas of what we could do during the migration (written down below). We can discuss and finalize it before publishing them to the docs folder.
  • It is still not clear how to run the playwright tests in Core, but it should be doable.

Migration steps

  1. Choose a test suite to migrate in packages/e2e-tests/specs, rename .test.js into .spec.js and put it in the same folder structure inside test/e2e/specs.
  2. Require the test helpers from @wordpress/e2e-test-utils-playwright:
    const { test, expect } = require( '@wordpress/e2e-test-utils-playwright' );
  3. Change all occurrences of describe, beforeAll, beforeEach, afterEach and afterAll with the test. prefix. For instance, describe is turning into test.describe.
  4. Use the fixtures API to require previously global variables like page and browser.
  5. Delete all the imports of e2e-test-utils, instead use the fixtures API to get the pageUtils and requestUtils directly. (However, pageUtils is not allowed in beforeAll and afterAll, rewrite them using requestUtils instead.)
  6. If there's a missing util, go ahead and migrate it. Copy the existing file in e2e-test-utils and paste it in the page or request folder in e2e-test-utils-playwright depending on whether it's a page util or a request util.
  7. Global page and browser are available in PageUtils's this.page and this.browser. You can get autocomplete and type inference by adding @this {import('./').PageUtils} to the JSDoc.
  8. All the other utils in the same class are available in this and bound to the same instance. You can remove the internal imports and just use this to access them.
  9. Import the newly migrated util in index.ts, and put it inside the PageUtils/RequestUtils class as an instance field.
  10. You can now use the util inside the pageUtils or the requestUtils fixture in test/e2e.
  11. Manually migrate other details in the tests following the proposed best practices below. Note that even though the differences in the API of playwright and puppeteer are similar, some manual changes are still required.

Proposed best practices

Forbid $, use locator instead

In fact, any API that returns ElementHandle is discouraged. This includes $, $$, $eval, $$eval, etc. Locator is a much better API and can be used with playwright's assertions. This also works great with Page Object Model since that locator is lazy and doesn't return a promise.

Use accessible selectors

This PR introduces a new selector engine role-selector. It enables us to write accessible queries without having to rely on internal implementations. It's an experimental library and could be swapped out if playwright supports it natively instead. The syntax should be straightforward and looks like this:

// Select a button with the accessible name "Hello World".
page.locator( 'role=button[name="Hello World"]' );

It can also be chained with built-in selector engines to perform complex queries:

// Select a button with a name ends with `Back` and is visible on the screen.
page.locator( 'role=button[name=/Back$/] >> visible=true' );
// Select a button with name "View options" under `#some-section`.
page.locator( 'css=#some-section >> role=button[name="View options"]' );

role-selector under the hood uses @testing-library/dom to compute the query the elements and compute the accessible attributes.

Selectors are strict by default

To encourage better practices for querying elements, selectors are strict by default, meaning that it will throw an error if the query returns more than one element.

Don't overload test-utils, inline simple utils

e2e-test-utils are too bloated with too many utils. Most of them are simple enough to be inlined directly in tests. With the help of accessible selectors, simple utils are easier to write now. For utils that only take place on a certain page, use Page Object Model instead (with an exception of clearing states with rest which are better placed in e2e-test-utils). Otherwise, only create an util if the action is complex and repetitive enough.

Favor Page Object Model over utils

As mentioned above, Page Object Model is the preferred way to create reusable utility functions on a certain page. (You can see an inline POM example in #38570 (comment))

The rationale behind using a POM is to group utils under namespaces to be easier to discover and use. In fact, PageUtils in e2e-test-utils` package is also a POM, which avoids the need for global variables, and utils can reference each other easily.

Restify actions to clear or set states.

It's slow to set states manually before or after tests, especially when they're repeated multiple times between tests. It's recommended to set them via API calls. Use requestUtils.rest and requestUtils.batchRest instead to call the REST API. We should still add a test for manually setting them, but that should only be tested once.

Avoid global variables

page and browser are not exposed as global variables, this makes it easier to work with when we have multiple pages/tabs in the same test, or if we want to run multiple tests in parallel. @playwright/test has the concept of fixtures which allows us to inject page, browser, and other parameters into the tests.

By default, @playwright/test will use different page instances for each test to isolate them. However, the nature of WordPress means most of our tests depend on the server state and we can't easily isolate them. We instead run the tests in serial to prevent race-condition.

Make explicit assertions

We can insert as many assertions in one test as needed. It's better to make explicit assertions whenever possible. For instance, if we want to assert that a button exists before clicking on it, we can do expect( locator ).toBeVisible() before performing locator.click(). This makes the tests flow better and easier to read.

Testing Instructions

# Be sure to install the dependencies first
npm install

# Run all tests
npm run test-e2e:playwright

# Run individual test
npm run test-e2e:playwright -- <path/to/test>

# Run in headed mode
npm run test-e2e:playwright -- --headed

# Run in debug mode
npm run test-e2e:playwright -- --debug

Types of changes

New feature

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • I've tested my changes with keyboard and screen readers.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR (please manually search all *.native.js files for terms that need renaming or removal).
  • I've updated related schemas if appropriate.

@kevin940726 kevin940726 added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Feb 7, 2022
@github-actions
Copy link

github-actions bot commented Feb 7, 2022

Size Change: +1.57 kB (0%)

Total Size: 1.21 MB

Filename Size Change
build/block-editor/index.min.js 146 kB +257 B (0%)
build/block-library/blocks/pullquote/style-rtl.css 370 B -19 B (-5%)
build/block-library/blocks/pullquote/style.css 370 B -18 B (-5%)
build/block-library/blocks/separator/editor-rtl.css 140 B +41 B (+41%) 🚨
build/block-library/blocks/separator/editor.css 140 B +41 B (+41%) 🚨
build/block-library/blocks/separator/theme-rtl.css 194 B +22 B (+13%) ⚠️
build/block-library/blocks/separator/theme.css 194 B +22 B (+13%) ⚠️
build/block-library/blocks/site-logo/editor-rtl.css 759 B +15 B (+2%)
build/block-library/blocks/site-logo/editor.css 759 B +15 B (+2%)
build/block-library/editor-rtl.css 9.98 kB +25 B (0%)
build/block-library/editor.css 9.98 kB +24 B (0%)
build/block-library/index.min.js 169 kB +504 B (0%)
build/block-library/style-rtl.css 11.2 kB -9 B (0%)
build/block-library/style.css 11.2 kB -7 B (0%)
build/block-library/theme-rtl.css 689 B +24 B (+4%)
build/block-library/theme.css 694 B +24 B (+4%)
build/components/index.min.js 218 kB +124 B (0%)
build/components/style-rtl.css 15.6 kB +44 B (0%)
build/components/style.css 15.6 kB +52 B (0%)
build/core-data/index.min.js 14.3 kB -8 B (0%)
build/edit-navigation/index.min.js 16.1 kB -50 B (0%)
build/edit-post/index.min.js 29.8 kB +25 B (0%)
build/edit-site/index.min.js 45 kB +137 B (0%)
build/edit-site/style-rtl.css 7.58 kB +144 B (+2%)
build/edit-site/style.css 7.56 kB +141 B (+2%)
ℹ️ View Unchanged
Filename Size
build/a11y/index.min.js 993 B
build/admin-manifest/index.min.js 1.24 kB
build/annotations/index.min.js 2.77 kB
build/api-fetch/index.min.js 2.27 kB
build/autop/index.min.js 2.15 kB
build/blob/index.min.js 487 B
build/block-directory/index.min.js 6.49 kB
build/block-directory/style-rtl.css 1.01 kB
build/block-directory/style.css 1.01 kB
build/block-editor/default-editor-styles-rtl.css 378 B
build/block-editor/default-editor-styles.css 378 B
build/block-editor/style-rtl.css 15.1 kB
build/block-editor/style.css 15.1 kB
build/block-library/blocks/archives/editor-rtl.css 61 B
build/block-library/blocks/archives/editor.css 60 B
build/block-library/blocks/archives/style-rtl.css 65 B
build/block-library/blocks/archives/style.css 65 B
build/block-library/blocks/audio/editor-rtl.css 150 B
build/block-library/blocks/audio/editor.css 150 B
build/block-library/blocks/audio/style-rtl.css 111 B
build/block-library/blocks/audio/style.css 111 B
build/block-library/blocks/audio/theme-rtl.css 125 B
build/block-library/blocks/audio/theme.css 125 B
build/block-library/blocks/block/editor-rtl.css 161 B
build/block-library/blocks/block/editor.css 161 B
build/block-library/blocks/button/editor-rtl.css 445 B
build/block-library/blocks/button/editor.css 445 B
build/block-library/blocks/button/style-rtl.css 560 B
build/block-library/blocks/button/style.css 560 B
build/block-library/blocks/buttons/editor-rtl.css 292 B
build/block-library/blocks/buttons/editor.css 292 B
build/block-library/blocks/buttons/style-rtl.css 275 B
build/block-library/blocks/buttons/style.css 275 B
build/block-library/blocks/calendar/style-rtl.css 207 B
build/block-library/blocks/calendar/style.css 207 B
build/block-library/blocks/categories/editor-rtl.css 84 B
build/block-library/blocks/categories/editor.css 83 B
build/block-library/blocks/categories/style-rtl.css 79 B
build/block-library/blocks/categories/style.css 79 B
build/block-library/blocks/code/style-rtl.css 103 B
build/block-library/blocks/code/style.css 103 B
build/block-library/blocks/code/theme-rtl.css 124 B
build/block-library/blocks/code/theme.css 124 B
build/block-library/blocks/columns/editor-rtl.css 108 B
build/block-library/blocks/columns/editor.css 108 B
build/block-library/blocks/columns/style-rtl.css 406 B
build/block-library/blocks/columns/style.css 406 B
build/block-library/blocks/comment-author-avatar/editor-rtl.css 125 B
build/block-library/blocks/comment-author-avatar/editor.css 125 B
build/block-library/blocks/comment-template/style-rtl.css 127 B
build/block-library/blocks/comment-template/style.css 127 B
build/block-library/blocks/comments-pagination-numbers/editor-rtl.css 123 B
build/block-library/blocks/comments-pagination-numbers/editor.css 121 B
build/block-library/blocks/comments-pagination/editor-rtl.css 222 B
build/block-library/blocks/comments-pagination/editor.css 209 B
build/block-library/blocks/comments-pagination/style-rtl.css 235 B
build/block-library/blocks/comments-pagination/style.css 231 B
build/block-library/blocks/comments-query-loop/editor-rtl.css 95 B
build/block-library/blocks/comments-query-loop/editor.css 95 B
build/block-library/blocks/cover/editor-rtl.css 546 B
build/block-library/blocks/cover/editor.css 547 B
build/block-library/blocks/cover/style-rtl.css 1.56 kB
build/block-library/blocks/cover/style.css 1.56 kB
build/block-library/blocks/embed/editor-rtl.css 293 B
build/block-library/blocks/embed/editor.css 293 B
build/block-library/blocks/embed/style-rtl.css 417 B
build/block-library/blocks/embed/style.css 417 B
build/block-library/blocks/embed/theme-rtl.css 124 B
build/block-library/blocks/embed/theme.css 124 B
build/block-library/blocks/file/editor-rtl.css 300 B
build/block-library/blocks/file/editor.css 300 B
build/block-library/blocks/file/style-rtl.css 255 B
build/block-library/blocks/file/style.css 255 B
build/block-library/blocks/file/view.min.js 353 B
build/block-library/blocks/freeform/editor-rtl.css 2.44 kB
build/block-library/blocks/freeform/editor.css 2.44 kB
build/block-library/blocks/gallery/editor-rtl.css 961 B
build/block-library/blocks/gallery/editor.css 964 B
build/block-library/blocks/gallery/style-rtl.css 1.51 kB
build/block-library/blocks/gallery/style.css 1.51 kB
build/block-library/blocks/gallery/theme-rtl.css 122 B
build/block-library/blocks/gallery/theme.css 122 B
build/block-library/blocks/group/editor-rtl.css 159 B
build/block-library/blocks/group/editor.css 159 B
build/block-library/blocks/group/style-rtl.css 57 B
build/block-library/blocks/group/style.css 57 B
build/block-library/blocks/group/theme-rtl.css 78 B
build/block-library/blocks/group/theme.css 78 B
build/block-library/blocks/heading/style-rtl.css 114 B
build/block-library/blocks/heading/style.css 114 B
build/block-library/blocks/html/editor-rtl.css 332 B
build/block-library/blocks/html/editor.css 333 B
build/block-library/blocks/image/editor-rtl.css 731 B
build/block-library/blocks/image/editor.css 730 B
build/block-library/blocks/image/style-rtl.css 529 B
build/block-library/blocks/image/style.css 535 B
build/block-library/blocks/image/theme-rtl.css 124 B
build/block-library/blocks/image/theme.css 124 B
build/block-library/blocks/latest-comments/style-rtl.css 284 B
build/block-library/blocks/latest-comments/style.css 284 B
build/block-library/blocks/latest-posts/editor-rtl.css 199 B
build/block-library/blocks/latest-posts/editor.css 198 B
build/block-library/blocks/latest-posts/style-rtl.css 447 B
build/block-library/blocks/latest-posts/style.css 446 B
build/block-library/blocks/list/style-rtl.css 94 B
build/block-library/blocks/list/style.css 94 B
build/block-library/blocks/media-text/editor-rtl.css 266 B
build/block-library/blocks/media-text/editor.css 263 B
build/block-library/blocks/media-text/style-rtl.css 493 B
build/block-library/blocks/media-text/style.css 490 B
build/block-library/blocks/more/editor-rtl.css 431 B
build/block-library/blocks/more/editor.css 431 B
build/block-library/blocks/navigation-link/editor-rtl.css 708 B
build/block-library/blocks/navigation-link/editor.css 706 B
build/block-library/blocks/navigation-link/style-rtl.css 94 B
build/block-library/blocks/navigation-link/style.css 94 B
build/block-library/blocks/navigation-submenu/editor-rtl.css 299 B
build/block-library/blocks/navigation-submenu/editor.css 299 B
build/block-library/blocks/navigation-submenu/view.min.js 375 B
build/block-library/blocks/navigation/editor-rtl.css 2.03 kB
build/block-library/blocks/navigation/editor.css 2.04 kB
build/block-library/blocks/navigation/style-rtl.css 1.89 kB
build/block-library/blocks/navigation/style.css 1.88 kB
build/block-library/blocks/navigation/view.min.js 2.85 kB
build/block-library/blocks/nextpage/editor-rtl.css 395 B
build/block-library/blocks/nextpage/editor.css 395 B
build/block-library/blocks/page-list/editor-rtl.css 363 B
build/block-library/blocks/page-list/editor.css 363 B
build/block-library/blocks/page-list/style-rtl.css 175 B
build/block-library/blocks/page-list/style.css 175 B
build/block-library/blocks/paragraph/editor-rtl.css 157 B
build/block-library/blocks/paragraph/editor.css 157 B
build/block-library/blocks/paragraph/style-rtl.css 273 B
build/block-library/blocks/paragraph/style.css 273 B
build/block-library/blocks/post-author/style-rtl.css 175 B
build/block-library/blocks/post-author/style.css 176 B
build/block-library/blocks/post-comments-form/style-rtl.css 446 B
build/block-library/blocks/post-comments-form/style.css 446 B
build/block-library/blocks/post-comments/style-rtl.css 521 B
build/block-library/blocks/post-comments/style.css 521 B
build/block-library/blocks/post-excerpt/editor-rtl.css 73 B
build/block-library/blocks/post-excerpt/editor.css 73 B
build/block-library/blocks/post-excerpt/style-rtl.css 69 B
build/block-library/blocks/post-excerpt/style.css 69 B
build/block-library/blocks/post-featured-image/editor-rtl.css 721 B
build/block-library/blocks/post-featured-image/editor.css 721 B
build/block-library/blocks/post-featured-image/style-rtl.css 153 B
build/block-library/blocks/post-featured-image/style.css 153 B
build/block-library/blocks/post-template/editor-rtl.css 99 B
build/block-library/blocks/post-template/editor.css 98 B
build/block-library/blocks/post-template/style-rtl.css 323 B
build/block-library/blocks/post-template/style.css 323 B
build/block-library/blocks/post-terms/style-rtl.css 73 B
build/block-library/blocks/post-terms/style.css 73 B
build/block-library/blocks/post-title/style-rtl.css 80 B
build/block-library/blocks/post-title/style.css 80 B
build/block-library/blocks/preformatted/style-rtl.css 103 B
build/block-library/blocks/preformatted/style.css 103 B
build/block-library/blocks/pullquote/editor-rtl.css 198 B
build/block-library/blocks/pullquote/editor.css 198 B
build/block-library/blocks/pullquote/theme-rtl.css 167 B
build/block-library/blocks/pullquote/theme.css 167 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 122 B
build/block-library/blocks/query-pagination-numbers/editor.css 121 B
build/block-library/blocks/query-pagination/editor-rtl.css 221 B
build/block-library/blocks/query-pagination/editor.css 211 B
build/block-library/blocks/query-pagination/style-rtl.css 234 B
build/block-library/blocks/query-pagination/style.css 231 B
build/block-library/blocks/query/editor-rtl.css 131 B
build/block-library/blocks/query/editor.css 132 B
build/block-library/blocks/quote/style-rtl.css 201 B
build/block-library/blocks/quote/style.css 201 B
build/block-library/blocks/quote/theme-rtl.css 223 B
build/block-library/blocks/quote/theme.css 226 B
build/block-library/blocks/read-more/style-rtl.css 132 B
build/block-library/blocks/read-more/style.css 132 B
build/block-library/blocks/rss/editor-rtl.css 202 B
build/block-library/blocks/rss/editor.css 204 B
build/block-library/blocks/rss/style-rtl.css 289 B
build/block-library/blocks/rss/style.css 288 B
build/block-library/blocks/search/editor-rtl.css 165 B
build/block-library/blocks/search/editor.css 165 B
build/block-library/blocks/search/style-rtl.css 397 B
build/block-library/blocks/search/style.css 398 B
build/block-library/blocks/search/theme-rtl.css 64 B
build/block-library/blocks/search/theme.css 64 B
build/block-library/blocks/separator/style-rtl.css 233 B
build/block-library/blocks/separator/style.css 233 B
build/block-library/blocks/shortcode/editor-rtl.css 474 B
build/block-library/blocks/shortcode/editor.css 474 B
build/block-library/blocks/site-logo/style-rtl.css 181 B
build/block-library/blocks/site-logo/style.css 181 B
build/block-library/blocks/site-tagline/editor-rtl.css 86 B
build/block-library/blocks/site-tagline/editor.css 86 B
build/block-library/blocks/site-title/editor-rtl.css 84 B
build/block-library/blocks/site-title/editor.css 84 B
build/block-library/blocks/social-link/editor-rtl.css 177 B
build/block-library/blocks/social-link/editor.css 177 B
build/block-library/blocks/social-links/editor-rtl.css 674 B
build/block-library/blocks/social-links/editor.css 673 B
build/block-library/blocks/social-links/style-rtl.css 1.37 kB
build/block-library/blocks/social-links/style.css 1.36 kB
build/block-library/blocks/spacer/editor-rtl.css 332 B
build/block-library/blocks/spacer/editor.css 332 B
build/block-library/blocks/spacer/style-rtl.css 48 B
build/block-library/blocks/spacer/style.css 48 B
build/block-library/blocks/table/editor-rtl.css 471 B
build/block-library/blocks/table/editor.css 472 B
build/block-library/blocks/table/style-rtl.css 481 B
build/block-library/blocks/table/style.css 481 B
build/block-library/blocks/table/theme-rtl.css 188 B
build/block-library/blocks/table/theme.css 188 B
build/block-library/blocks/tag-cloud/style-rtl.css 226 B
build/block-library/blocks/tag-cloud/style.css 227 B
build/block-library/blocks/template-part/editor-rtl.css 235 B
build/block-library/blocks/template-part/editor.css 235 B
build/block-library/blocks/template-part/theme-rtl.css 101 B
build/block-library/blocks/template-part/theme.css 101 B
build/block-library/blocks/text-columns/editor-rtl.css 95 B
build/block-library/blocks/text-columns/editor.css 95 B
build/block-library/blocks/text-columns/style-rtl.css 166 B
build/block-library/blocks/text-columns/style.css 166 B
build/block-library/blocks/verse/style-rtl.css 87 B
build/block-library/blocks/verse/style.css 87 B
build/block-library/blocks/video/editor-rtl.css 571 B
build/block-library/blocks/video/editor.css 572 B
build/block-library/blocks/video/style-rtl.css 173 B
build/block-library/blocks/video/style.css 173 B
build/block-library/blocks/video/theme-rtl.css 124 B
build/block-library/blocks/video/theme.css 124 B
build/block-library/common-rtl.css 934 B
build/block-library/common.css 932 B
build/block-library/reset-rtl.css 474 B
build/block-library/reset.css 474 B
build/block-serialization-default-parser/index.min.js 1.12 kB
build/block-serialization-spec-parser/index.min.js 2.83 kB
build/blocks/index.min.js 46.8 kB
build/compose/index.min.js 11.2 kB
build/customize-widgets/index.min.js 11.2 kB
build/customize-widgets/style-rtl.css 1.39 kB
build/customize-widgets/style.css 1.39 kB
build/data-controls/index.min.js 663 B
build/data/index.min.js 8.19 kB
build/date/index.min.js 31.9 kB
build/deprecated/index.min.js 518 B
build/dom-ready/index.min.js 336 B
build/dom/index.min.js 4.53 kB
build/edit-navigation/style-rtl.css 4.04 kB
build/edit-navigation/style.css 4.05 kB
build/edit-post/classic-rtl.css 546 B
build/edit-post/classic.css 547 B
build/edit-post/style-rtl.css 7.07 kB
build/edit-post/style.css 7.07 kB
build/edit-widgets/index.min.js 16.5 kB
build/edit-widgets/style-rtl.css 4.39 kB
build/edit-widgets/style.css 4.39 kB
build/editor/index.min.js 38.4 kB
build/editor/style-rtl.css 3.71 kB
build/editor/style.css 3.71 kB
build/element/index.min.js 4.29 kB
build/escape-html/index.min.js 548 B
build/format-library/index.min.js 6.62 kB
build/format-library/style-rtl.css 571 B
build/format-library/style.css 571 B
build/hooks/index.min.js 1.66 kB
build/html-entities/index.min.js 454 B
build/i18n/index.min.js 3.79 kB
build/is-shallow-equal/index.min.js 535 B
build/keyboard-shortcuts/index.min.js 1.83 kB
build/keycodes/index.min.js 1.41 kB
build/list-reusable-blocks/index.min.js 1.75 kB
build/list-reusable-blocks/style-rtl.css 838 B
build/list-reusable-blocks/style.css 838 B
build/media-utils/index.min.js 2.94 kB
build/notices/index.min.js 957 B
build/nux/index.min.js 2.12 kB
build/nux/style-rtl.css 751 B
build/nux/style.css 749 B
build/plugins/index.min.js 1.98 kB
build/preferences/index.min.js 1.2 kB
build/primitives/index.min.js 949 B
build/priority-queue/index.min.js 611 B
build/react-i18n/index.min.js 704 B
build/react-refresh-entry/index.min.js 8.44 kB
build/react-refresh-runtime/index.min.js 7.31 kB
build/redux-routine/index.min.js 2.69 kB
build/reusable-blocks/index.min.js 2.24 kB
build/reusable-blocks/style-rtl.css 256 B
build/reusable-blocks/style.css 256 B
build/rich-text/index.min.js 11.1 kB
build/server-side-render/index.min.js 1.61 kB
build/shortcode/index.min.js 1.52 kB
build/token-list/index.min.js 668 B
build/url/index.min.js 1.99 kB
build/vendors/react-dom.min.js 38.5 kB
build/vendors/react.min.js 4.34 kB
build/viewport/index.min.js 1.08 kB
build/warning/index.min.js 280 B
build/widgets/index.min.js 7.21 kB
build/widgets/style-rtl.css 1.16 kB
build/widgets/style.css 1.16 kB
build/wordcount/index.min.js 1.07 kB

compressed-size-action

@kevin940726 kevin940726 force-pushed the try/playwright-test branch 8 times, most recently from f39fab1 to 565c7b6 Compare February 7, 2022 17:51
strategy:
fail-fast: false
matrix:
node: ['14']
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add matrix here in the future when test suites grow and take advantage of the sharding feature.

@@ -112,6 +113,7 @@
"@types/eslint": "7.28.0",
"@types/estree": "0.0.50",
"@types/highlight-words-core": "1.2.1",
"@types/istanbul-lib-report": "3.0.0",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea why I have to add this to make tsc happy.

};

if ( isReducedMotion ) {
handleTransitionEnd();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to fix a bug in one of the existing tests which surfaces itself during the migration. For some reason, transitionend event below never fires in CI, but I couldn't reproduce it locally. Anyway, it makes sense to skip the event when the user agent prefers reduced motion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be extracted into a separate fix; if it addresses a bug with the current tests we should get it in asap. Also it's best not to mix actual code changes with the test migration 😛

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! In fact, both these two tests (customizing-widgets and new-post) are included here only for demonstration purposes, and they could be separated into new PRs along with the fix here. This should help minimize the changes introduced in this PR and make it easier to review.

packages/e2e-test-utils-playwright/src/blocks.js Outdated Show resolved Hide resolved
packages/e2e-test-utils-playwright/src/login-user.js Outdated Show resolved Hide resolved
packages/e2e-test-utils-playwright/src/posts.js Outdated Show resolved Hide resolved
packages/e2e-tests-playwright/playwright.config.ts Outdated Show resolved Hide resolved
timeout: parseInt( process.env.TIMEOUT || '', 10 ) || 100_000, // Defaults to 100 seconds.
// Don't report slow test "files", as we will be running our tests in serial.
reportSlowTests: null,
testDir: new URL( './specs', 'file:' + __filename ).pathname,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code might seem weird, but it makes it easier to swap into ESM (import.meta.url) in the future if needed.

} );
} );

class WidgetsCustomizerPage {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a Page Object Model we use in this test. We can also move this to e2e-test-utils if there are needs.

@juhi123
Copy link
Contributor

juhi123 commented Feb 11, 2022

This looks great.

Looking forward to working on playwright tests.

I was hoping we would have the Playwright package in WP core itself but I think because of internal dependencies of packages it would have not been feasible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants