New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run unit tests in parallel #1625
Conversation
Codecov Report
@@ Coverage Diff @@
## trunk #1625 +/- ##
==========================================
- Coverage 83.16% 83.15% -0.01%
==========================================
Files 397 397
Lines 86545 86542 -3
Branches 9197 9197
==========================================
- Hits 71979 71968 -11
- Misses 11436 11444 +8
Partials 3130 3130
Continue to review full report at Codecov.
|
scenarios when running tests in standalone fashion (so compute test didn't set token value on which test relies on).
This change uncovered some issues with some test files relying on shared state (aka when running all the test cases together in the same process some other import unintentionally setting some state which test relies on) which I'm trying to fix. |
On CI, tests now finish 30-40 seconds faster (2 min 20 vs 1 min 40). Not a massive speed up, but GHA runner likely doesn't have as many cores and it's likely more heavily over provisioned with other jobs. |
This PR attempts to run unit tests in parallel using
pytest-xdist
plugin.To avoid issues with some of the test functions not being "cross process safe" we run all the tests in a single test file in the same worker process.
Locally (with 16 cores) unit tests now finish ~5-10x faster, but it depends on number of available cores.