Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks #42903

Aljullu · 2023-12-18T12:31:00Z

Changes proposed in this Pull Request:

This PR is a port of woocommerce/woocommerce-blocks#11975.

This PR adds Playwright tests to the All Reviews, Reviews by Product and Reviews by Category blocks and removes the Puppeteer tests for the same blocks.

Closes #42065.

How to test the changes in this Pull Request:

Using the WooCommerce Testing Instructions Guide, include your detailed testing instructions:

Verify Playwright e2e tests pass.

Changelog entry

Automatically create a changelog entry from the details below.

Significance

Patch
Minor
Major

Type

Fix - Fixes an existing bug
Add - Adds functionality
Update - Update existing functionality
Dev - Development related task
Tweak - A minor adjustment to the codebase
Performance - Address performance issues
Enhancement - Improvement to existing functionality

Message

Comment

Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks

github-actions · 2023-12-18T12:55:48Z

Test Results Summary

Commit SHA: ee863d8

Test 🧪	Passed ✅	Failed 🚨	Broken 🚧	Skipped ⏭️	Unknown ❔	Total 📊	Duration ⏱️
API Tests	259	0	0	2	0	261	0m 36s
E2E Tests	258	0	0	3	0	261	14m 43s

To view the full API test report, click here.
To view the full E2E test report, click here.
To view all test reports, visit the WooCommerce Test Reports Dashboard.

github-actions · 2023-12-18T13:40:00Z

Hi @sunyatasattva, @imanish003,

Apart from reviewing the code changes, please make sure to review the testing instructions as well.

You can follow this guide to find out what good testing instructions should look like:
https://github.com/woocommerce/woocommerce/wiki/Writing-high-quality-testing-instructions

…mmerce-blocks

sunyatasattva · 2023-12-19T23:28:31Z

Popping up unsolicited in this PR again 🙃

You might not be aware of it since we had these conversations while you were on sabbatical, but during the meetup we thought of trying to minimize (if not entirely remove) the ubiquitous clickity steps in our E2E tests to reduce run time by using fixtures (more info: per0F9-16b-p2 — or feel free to ping me).

I'd love to see this starting to get implemented, rather than adding more tests that we'll then need to refactor. I understand this increases the scope (if not entirely is out of scope) of this PR. So perhaps we shouldn't put this on hold, but I wanted to make you aware of it.

plugins/woocommerce-blocks/tests/e2e/tests/all-reviews/all-reviews.block_theme.spec.ts

…one single place

…the already-created post

Aljullu · 2023-12-20T11:56:45Z

Thanks for the feedback and the extra context, @sunyatasattva, it was super-valuable! 💯

I'd love to see this starting to get implemented, rather than adding more tests that we'll then need to refactor. I understand this increases the scope (if not entirely is out of scope) of this PR. So perhaps we shouldn't put this on hold, but I wanted to make you aware of it.

In 7159454 I updated some tests so they go to the post created on setup instead of creating a new post. I still kept one test per block that verifies the block can be inserted into the page. My understanding is that we want to avoid creating a post + inserting the block in each test, but there is still value in having at least one test per block that verifies the block can be inserted, is that right?

These changes have the drawback that we can no longer create the products in parallel, as we need IDs to be deterministic.

sunyatasattva · 2023-12-20T14:55:48Z

Thanks a bunch for doing that!

My understanding is that we want to avoid creating a post + inserting the block in each test, but there is still value in having at least one test per block that verifies the block can be inserted, is that right?

This would be my suggestion to: I think testing that the block is insertable is still valuable, but limiting to only once per suite.

These changes have the drawback that we can no longer create the products in parallel, as we need IDs to be deterministic.

Could you elaborate a bit on this? What do you mean create products in parallel?

Aljullu · 2023-12-20T16:08:17Z

Could you elaborate a bit on this? What do you mean create products in parallel?

Sure thing. As I understand it, right now posts, pages and products are created in parallel, as the scripts that create them are inside the parallel folder:

https://github.com/woocommerce/woocommerce/tree/0ba8f7848df7c6930924d33ee63d5a349d978069/plugins/woocommerce-blocks/tests/e2e/bin/scripts/parallel

That means that we can't know in advance which ids will be assigned to each product, as it will depend on when they are created relative to the posts and pages.

However, we do need to know the id of the product (and category) we add the reviews to so we can create the post on setup with the correct productId:

https://github.com/woocommerce/woocommerce/blob/trunk/plugins/woocommerce-blocks/tests/e2e/bin/posts/reviews-by-product.html

That's why I needed to move products.sh outside of the scripts that run in parallel:

https://github.com/woocommerce/woocommerce/pull/42903/files#diff-fc116cb558a2c0728b71c6047fe4684b8fe036014f9797702ffc0766443d3792R11-R12

Does it make sense? Any ideas on how to solve that?

One alternative approach would be not to create the post during the setup, but inside the test file (in a beforeAll() function, for example). So for the Reviews by Product block we would remove its fixture and instead we would create the post in the test file as we are currently doing, but reusing the same post across tests.

Thoughts? That approach would mean we don't make use of fixtures, but given that we still need one test to make sure the block can be inserted, it shouldn't add any overhead.

sunyatasattva · 2023-12-21T16:58:27Z

Oh thank you for the clarification, I understand now. I still like the idea of keeping the fixture, I find a couple of advantages there:

It's very explicit, and easy to read, parse and navigate.
If the insertion in the beforeAll were to fail, for some reason, the entire test suite would fail (also, failure in test hooks are often a bit more frustrating than tests themselves). By testing insertion in isolation, we would have correct data on what exactly fails (specific functionality vs. insertion workflow).

The drawback of removing them from parallel creation is not such a big deal for me, because usually generating entities via the CLI is way faster than E2E insertion workflow, so I don't see incredibly big problems.

In theory, there are solutions to this, but they are all overkill for this use-case, I think. We should progressively add complexity as we start seeing the necessity. A bigger issue for me is fixing the DOM structure inside the fixture, but we discussed this during the meetup, and it seems that the consensus was that this is also not a big deal; at least, not outweighing the pros.

All in all, thanks for the great work! I'm going to approve this PR, personally. Feel free to do more modifications or ping me more.

Aljullu · 2023-12-22T08:33:11Z

Makes a lot of sense! Thanks for the reviews and all the feedback, @sunyatasattva!

…by Category blocks (#42903) * Remove Reviews blocks Puppeteer tests * Minor code cleanup * Typos * Create publishAndVisitPost() editor util * Fix subcategories when importing products in Playwright and add reviews * Add Reviews blocks tests in Playwright * More typos * Add changefile(s) from automation for the following project(s): woocommerce-blocks * Create a 'reviews' object in data.ts so we can store reviews data in one single place * Update test so instead of creating a new post in each test, we go to the already-created post * Add source comments to reviews data to match it with the script --------- Co-authored-by: github-actions <github-actions@github.com>

…s documentation (#43055) * Update woocommerce.com URLs in documentation and code files * Add changelog * Fix github repository link in extend-rest-api-add-custom-fields.md Co-authored-by: Albert Juhé Lluveras <contact@albertjuhe.com> * Add github reporter (#42974) * Add github reporter * Add changelog --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> * Fix product task redirect to support grouped and external products (#43051) * Rearrange product redirection logic to better accept grouped and external produc types * Add changelog * Modify feedback modal actions (#43005) * Adapt feedback modal actions * Add changelogs * Modify comments * Fix tests * Fix test * Update class-wc-gateway-bacs.php (#43054) * Update class-wc-gateway-bacs.php Fix typo in textdomain * Add changefile(s) from automation for the following project(s): woocommerce --------- Co-authored-by: github-actions <github-actions@github.com> * [Product Block Editor]: Add `Linked product` tab (#43009) * add linked-products to group IDs * add Linked Products tab * tweak hideConditions condition * changelog * fix typo in doc comment * Introduce a product type selection within the new experience (#41823) * Create a relation between the product type and the product block template * Add 'patterns' to name the kind of products that can be created for a specific template * Resolve template using its id as a template query param * Rename ProductEditPattern to ProductTemplate * Rename get_patterns hook to woocommerce_product_editor_get_product_templates * Return the list of templates to the client * Set layout template events as array * Register the layout template based on the product template or the post type in case of product variations * Registering non supported product types * Create and register the woocommerce/product-details-section-description block * Add the product type to the section description * Create product type selector * Fix menu item style * Highlight selected menu item * Set the selected product template * Set product template title to lowercase in the content description * Rename blocks by blockTemplates under the AbstractBlockTemplate class * Rename to woocommerce_product_editor_product_templates filter * Remove product_template_ prefix from the supported_product_types map * Rename get_formatted to to_JSON and convert the props to client side like * Refactor get_product_templates * Fix icon resolution * Add a confirmation modal for unsupported product templates * Add changelog files * Remove product types using for testing * Fix redirection when changing to a non supported product template * Set the change button state to busy when it is saving the product * Fix php linter errors * Fix rebase conflict * Move ProductTemplate to Automattic\WooCommerce\Admin\Features\ProductBlockEditor namespace * Add the to_json definition to the BlockTemplateInterface * Create default product template by custom product type if it does not have a template associated yet * Fix some comments and product template creation validation * Add support to load the product template icon from an external resource * Fix php linter * Fix the changelog description * [Experimental] Interactivity Dropdown multi-select mode, ratings filter and introduce each directive (#42981) --------- Co-authored-by: David Arenas <david.arenas@automattic.com> * Introduce the transient files engine (#42877) Co-authored-by: Corey McKrill <916023+coreymckrill@users.noreply.github.com> * Change marketplace install API request to POST instead of GET (#43033) * Change marketplace install API to using POST instead of GET * Fix linting error * Add changefile(s) from automation for the following project(s): woocommerce --------- Co-authored-by: github-actions <github-actions@github.com> * Prep trunk for 8.6 cycle (#43021) Prep trunk for 8.6 cycle with version bump to 8.6.0-dev Co-authored-by: WooCommerce Bot <no-reply@woo.com> * Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks (#42903) * Remove Reviews blocks Puppeteer tests * Minor code cleanup * Typos * Create publishAndVisitPost() editor util * Fix subcategories when importing products in Playwright and add reviews * Add Reviews blocks tests in Playwright * More typos * Add changefile(s) from automation for the following project(s): woocommerce-blocks * Create a 'reviews' object in data.ts so we can store reviews data in one single place * Update test so instead of creating a new post in each test, we go to the already-created post * Add source comments to reviews data to match it with the script --------- Co-authored-by: github-actions <github-actions@github.com> * Release: Remove 8.5 change files (#43022) Delete changelog files from 8.5 release Co-authored-by: WooCommerce Bot <no-reply@woo.com> Co-authored-by: Alex López <alex.lopez@automattic.com> * Delete changelog files based on PR 43033 (#43079) Delete changelog files for 43033 Co-authored-by: WooCommerce Bot <no-reply@woo.com> * Delete changelog files based on PR 43051 (#43081) Delete changelog files for 43051 Co-authored-by: WooCommerce Bot <no-reply@woo.com> * Interactive Price Filter: use `context` instead of `state` (#42980) * feat: use context instead of state * fix: temporary move the context to inner element for diffing to work * fix: update context before navigation for optimistic UI * Load google analytics gtag script asynchronously in WooCommerce Blocks (#43040) Co-authored-by: github-actions <github-actions@github.com> * set WOOCOMMERCE_BLOCKS_PHASE to 1 for the production build (#43074) * set WOOCOMMERCE_BLOCKS_PHASE to 1 for the production build * Add changefile(s) from automation for the following project(s): woocommerce --------- Co-authored-by: github-actions <github-actions@github.com> * Revert "Fix schedule sales error" (#43094) Revert "Fix schedule sales error (#42700)" This reverts commit 9b800aa. * [Product Block Editor]: add Linked product sections. First approach. (#43013) * add Linked products, Upsell section * changelog * add Cross-lens section * add links to the Upsell sections * changelog * fix lint issues * fix lint issus * fix linting issue :-| * check whether the linked product group is defined * [Product Block Editor]: introduce ShoppingBags component (#43042) * add ShoppingBags component * Add ShoppingBag story * changelog * Fix: Collection data being leaked between Collection Filters blocks (#43044) * fix: CYS - change heading color (#43076) * fix: CYS - change heading color * Add changefile(s) from automation for the following project(s): woocommerce --------- Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Patricia Hillebrandt <patriciahillebrandt@gmail.com> * Delete changelog files based on PR 43074 (#43118) Delete changelog files for 43074 Co-authored-by: WooCommerce Bot <no-reply@woo.com> * [Product Block Editor]: fix feature flag to hide the Linked products (#43119) * fix flag to hide/show product editor * changelog * Add changelog --------- Co-authored-by: Albert Juhé Lluveras <contact@albertjuhe.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com> Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: louwie17 <lourensschep@gmail.com> Co-authored-by: Fernando Marichal <fernando.marichal@automattic.com> Co-authored-by: Marc Guay <marc.guay@gmail.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Damián Suárez <rdsuarez@gmail.com> Co-authored-by: Maikel David Pérez Gómez <maikel.perez@automattic.com> Co-authored-by: Sam Seay <samueljseay@gmail.com> Co-authored-by: Néstor Soriano <konamiman@konamiman.com> Co-authored-by: Corey McKrill <916023+coreymckrill@users.noreply.github.com> Co-authored-by: Kyle Nel <22053773+kdevnel@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: WooCommerce Bot <no-reply@woo.com> Co-authored-by: Alex López <alex.lopez@automattic.com> Co-authored-by: Tung Du <dinhtungdu@gmail.com> Co-authored-by: Thomas Roberts <5656702+opr@users.noreply.github.com> Co-authored-by: Luigi Teschio <gigitux@gmail.com> Co-authored-by: Patricia Hillebrandt <patriciahillebrandt@gmail.com>

Aljullu added focus: e2e tests Issues related to e2e tests block: reviews by product Issues related to the Reviews by Product block. block: reviews by category Issues related to the Reviews by Category block. block: all reviews Issues related to the All Reviews block. labels Dec 18, 2023

Aljullu self-assigned this Dec 18, 2023

github-actions bot removed the focus: e2e tests Issues related to e2e tests label Dec 18, 2023

Aljullu marked this pull request as ready for review December 18, 2023 13:37

Aljullu requested a review from imanish003 December 18, 2023 13:39

Aljullu added the team: Kirigami & Origami label Dec 18, 2023

Aljullu force-pushed the add/review-blocks-tests branch 2 times, most recently from 5df0f08 to 771666e Compare December 18, 2023 14:25

Aljullu and others added 8 commits December 18, 2023 15:44

Remove Reviews blocks Puppeteer tests

70911ab

Minor code cleanup

328f8cc

Typos

9752c02

Create publishAndVisitPost() editor util

5331b51

Fix subcategories when importing products in Playwright and add reviews

6af9b9c

Add Reviews blocks tests in Playwright

d869b13

More typos

2659021

Add changefile(s) from automation for the following project(s): wooco…

07fd26b

…mmerce-blocks

Aljullu force-pushed the add/review-blocks-tests branch from 7337e32 to 07fd26b Compare December 18, 2023 14:44

github-actions bot added the plugin: woocommerce Issues related to the WooCommerce Core plugin. label Dec 18, 2023

Aljullu mentioned this pull request Dec 19, 2023

Reviews by Product block: fix encoded in input aria-label #42944

Merged

11 tasks

sunyatasattva reviewed Dec 19, 2023

View reviewed changes

plugins/woocommerce-blocks/tests/e2e/tests/all-reviews/all-reviews.block_theme.spec.ts Outdated Show resolved Hide resolved

Aljullu added 2 commits December 20, 2023 12:47

Create a 'reviews' object in data.ts so we can store reviews data in …

d334554

…one single place

Update test so instead of creating a new post in each test, we go to …

7159454

…the already-created post

Add source comments to reviews data to match it with the script

ee863d8

sunyatasattva self-requested a review December 21, 2023 16:58

sunyatasattva approved these changes Dec 21, 2023

View reviewed changes

Aljullu merged commit 2da7b73 into trunk Dec 22, 2023
33 checks passed

Aljullu deleted the add/review-blocks-tests branch December 22, 2023 08:33

github-actions bot added this to the 8.6.0 milestone Dec 22, 2023

github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label Dec 22, 2023

veljkho added status: analysis complete Indicates if a PR has been analysed by Solaris and removed needs: analysis Indicates if the PR requires a PR testing scrub session. labels Dec 25, 2023

Aljullu mentioned this pull request Jan 5, 2024

fix - Product Button: don't trigger jQuery add to cart function when the button is clicked on classic themes #43325

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks #42903

Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks #42903

Aljullu commented Dec 18, 2023 •

edited

github-actions bot commented Dec 18, 2023 •

edited

github-actions bot commented Dec 18, 2023 •

edited

sunyatasattva commented Dec 19, 2023

Aljullu commented Dec 20, 2023 •

edited

sunyatasattva commented Dec 20, 2023

Aljullu commented Dec 20, 2023

sunyatasattva commented Dec 21, 2023

Aljullu commented Dec 22, 2023

Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks #42903

Add Playwright tests for All Reviews, Reviews by Product and Reviews by Category blocks #42903

Conversation

Aljullu commented Dec 18, 2023 • edited

Changes proposed in this Pull Request:

How to test the changes in this Pull Request:

Changelog entry

Significance

Type

Message

Comment

github-actions bot commented Dec 18, 2023 • edited

Test Results Summary

github-actions bot commented Dec 18, 2023 • edited

sunyatasattva commented Dec 19, 2023

Aljullu commented Dec 20, 2023 • edited

sunyatasattva commented Dec 20, 2023

Aljullu commented Dec 20, 2023

sunyatasattva commented Dec 21, 2023

Aljullu commented Dec 22, 2023

Aljullu commented Dec 18, 2023 •

edited

github-actions bot commented Dec 18, 2023 •

edited

github-actions bot commented Dec 18, 2023 •

edited

Aljullu commented Dec 20, 2023 •

edited