Remove global counters in tests

[justinchuby]

Remove global counters in tests from reference_evaluator_backend_test and automatic_upgrade_test to allow parallel test execution.

Run tests in parallel in CI.

[jcwchen]

I guess the original test stats will be covered by the new codecov: #5660 instead?

[justinchuby]

I guess the original test stats will be covered by the new codecov: #5660 instead?

Yep! Even without #5660, we should still remove global states in tests because they make parallelization hard

[justinchuby]

Need to test the notebooks separately because they are not parallelizable.

[justinchuby]

OK we need --dist loadscope

[codecov-commenter]

Codecov Report

All modified lines are covered by tests ✅

Files	Coverage Δ
...x/test/version_converter/automatic_upgrade_test.py	99.63% <100.00%> (+<0.01%)	⬆️

📢 Thoughts on this report? Let us know!.

[justinchuby]

cc @xadupre

[jcwchen]

Thanks for pytest -n auto --dist loadscope, but I am surprise that the CI running time seems similar.

[justinchuby]

Thanks for pytest -n auto --dist loadscope, but I am surprise that the CI running time seems similar.

The Python tests don’t take much time so we may not see a big difference. Nevertheless enabling parallel testing allows us to catch undesirable order dependencies or global state dependencies in our tests