Extremely slow WebAPI GraphQL functional tests

[ilnytskyi]

Preconditions and environment

• Magento version: 2.4.7+
• Run graphql tests from dev/tests/api-functional https://developer.adobe.com/commerce/webapi/graphql/develop/functional-testing/

Steps to reproduce

1. Run graphql tests from dev/tests/api-functional
   1.1. Example command with debug flag to see test times: ../../../vendor/bin/phpunit -c $(pwd)/phpunit_graphql.xml --debug -v
2. tearDown function cleans full_page cache after every tests that adds about 5 seconds to every test: https://github.com/magento/magento2/blob/2.4-develop/dev/tests/api-functional/framework/Magento/TestFramework/TestCase/GraphQlAbstract.php#L215

3. Before every test the config cache is cleaned that causes graphql schema rebuilding on every new test. Related to this issue: Compiled Schema Stored In Config Cache community-features#226
   3.1. Each request takes at least 8 seconds -
   
   3.2. tearDown calls add about 5 seconds
   
   3.3. Config cache cleans before every tests causing issues as above
   

Expected result

1. WebAPI graphql tests work faster
2. WebAPI graphql tests times when run locally are similar to times in magento's repo pipeline

Actual result

1. WebAPI graphql tests work slow
2. WebAPI graphql tests times when run locally are about 10 times slower than in magento's repo pipeline

Additional information

1. In default setup the test takes at least 15 seconds to execute.
2. It's unclear how official CI modified in actual pipelines for this repo. As for reference GRAPHQL WebAPI tests for CE work much faster https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/39723/892caa62a7455d9aedcbe9153360bf32/WebApi/allure-report-ce-WebAPI-GRAPHQL/index.html

Also many other tests run a lot faster than locally:

I have been able to achieve similar to magento's CI tests times of 1.5 - 3 seconds only after these steps:

1. Disabling bin/magento cache:flush full_page in https://github.com/magento/magento2/blob/2.4-develop/dev/tests/api-functional/framework/Magento/TestFramework/TestCase/GraphQlAbstract.php#L215
2. Moving graphql schema cache into different cache type that is not cleaned that often - Compiled Schema Stored In Config Cache community-features#226 (comment)
3. Without those changes actual tests are executed about x10 times slower.
   It's unclear why official CI work faster in comparance to running same testsuite locally
   It also takes forever to do for developers when custom tests need to be debugged with xdebug, because whole testsuite becomes extremely slow, even on latest machines.

Documentation page: https://developer.adobe.com/commerce/webapi/graphql/develop/functional-testing/ does not mention anything about improving performance or debugging.

Release note

• GRAPHQL WebAPI tests now execute significantly faster when running locally. Previously excessive cache cleaning and schema stitching caused about x10 slowdown.

Triage and priority

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @ilnytskyi. Thank you for your report.
To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce.

• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this
• To learn more about issue processing workflow, refer to the Code Contributions.

---

Join Magento Community Engineering Slack and ask your questions in #github channel.
⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
🕙 You can find the schedule on the Magento Community Calendar page.
📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.