Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run all tests on daily run #39351

Merged
merged 7 commits into from Jul 23, 2023
Merged

Conversation

lanej0
Copy link
Contributor

@lanej0 lanej0 commented Jul 20, 2023

Submission Review Guidelines:

Changes proposed in this Pull Request:

Currently, our daily test run only executes 5 of the 261 tests in the API test suite. This PR changes that to execute the entire suite.

There are some test that fail when they aren't run agains wp-env. For the time being, we'll skip those tests and work to fix their execution on CI.

How to test the changes in this Pull Request:

Using the WooCommerce Testing Instructions Guide, include your detailed testing instructions:

  1. Go to https://github.com/woocommerce/woocommerce/actions/workflows/smoke-test-daily.yml
  2. Under "Run workflow", select this branch (instead of trunk)
  3. Run the workflow
  4. Verify in the results that all API tests were run (261) and that all tests pass/are skipped (results should look like this run: https://github.com/woocommerce/woocommerce/actions/runs/5616661454/job/15219866462).

Changelog entry

  • Automatically create a changelog entry from the details below.

Significance

  • Patch
  • Minor
  • Major

Type

  • Fix - Fixes an existing bug
  • Add - Adds functionality
  • Update - Update existing functionality
  • Dev - Development related task
  • Tweak - A minor adjustment to the codebase
  • Performance - Address performance issues
  • Enhancement - Improvement to existing functionality

Message

Comment

@lanej0 lanej0 requested review from a team and lsinger and removed request for a team July 20, 2023 22:56
@github-actions github-actions bot added focus: api tests Issues related to API tests. plugin: woocommerce Issues related to the WooCommerce Core plugin. labels Jul 20, 2023
@lanej0 lanej0 removed the plugin: woocommerce Issues related to the WooCommerce Core plugin. label Jul 20, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 20, 2023

Hi @lsinger, @nigeljamesstevenson,

Apart from reviewing the code changes, please make sure to review the testing instructions as well.

You can follow this guide to find out what good testing instructions should look like:
https://github.com/woocommerce/woocommerce/wiki/Writing-high-quality-testing-instructions

@lanej0 lanej0 marked this pull request as draft July 20, 2023 23:22
@github-actions
Copy link
Contributor

github-actions bot commented Jul 20, 2023

Test Results Summary

Commit SHA: 296f2aa

Test 🧪Passed ✅Failed 🚨Broken 🚧Skipped ⏭️Unknown ❔Total 📊Duration ⏱️
API Tests25900202610m 54s
E2E Tests1890019020814m 39s

To view the full API test report, click here.
To view the full E2E test report, click here.
To view all test reports, visit the WooCommerce Test Reports Dashboard.

@github-actions github-actions bot added the plugin: woocommerce Issues related to the WooCommerce Core plugin. label Jul 21, 2023
@lanej0 lanej0 marked this pull request as ready for review July 21, 2023 21:46
@lanej0
Copy link
Contributor Author

lanej0 commented Jul 21, 2023

This is just a start to running more of the API tests daily. There are still a large number of tests being skipped because they don't run reliably against an external host, but we can iterate on that.

Copy link
Contributor

@nigeljamesstevenson nigeljamesstevenson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested well - all (non-skipped) tests executed as expected. Thanks for this @lanej0 !

@nigeljamesstevenson nigeljamesstevenson merged commit d41d61e into trunk Jul 23, 2023
50 of 52 checks passed
@nigeljamesstevenson nigeljamesstevenson deleted the api/run-all-tests-on-daily branch July 23, 2023 20:00
@github-actions github-actions bot added this to the 8.1.0 milestone Jul 23, 2023
@github-actions github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label Jul 23, 2023
@alopezari alopezari removed the needs: analysis Indicates if the PR requires a PR testing scrub session. label Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
focus: api tests Issues related to API tests. plugin: woocommerce Issues related to the WooCommerce Core plugin.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants