Reenable pytest-xdist for running tests

[echoix]

At the end of the v5 cycle, pytest-xdist was added to the pytest plugins installed for running tests, but was removed soon after, in the same PR.

Added:
f093fb9

Removed:

e95d3b2

To reenable pytest-xdist and have parallel execution of tests, if possible, I added the -n flag, and to solve some issues with golangci lint and typescript standard, I added the --dist loadscope so that jobs from a same module and class run with the same worker.

If ever there were some new failures due to not being sequential, there was a way to have annotations that we could mark some tests with something like "serial" and run two times, one without enabling xdist for the serial tests, and split the remaining with xdist. Something like pytest-dev/pytest-xdist#84

For golangci lint, there was the option to use --allow-parallel-runners or --allow-serial-runners

It shaves off about 9-10 minutes of test time when all tests are done. For individual linters, with loadscope it doesn’t really change much.

Proposed Changes

1. …
2. …
3. …

Readiness Checklist

Author/Contributor

• Add entry to the CHANGELOG listing the change and linking to the corresponding issue (if appropriate)
• If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

• Label as breaking if this is a large fundamental change
• Label as either automation, bug, documentation, enhancement, infrastructure, or performance

[nvuillam]

Thhat's amazing @echoix , with this single PR you just save 10mn of CI validation for every PR ❤️

[echoix]

Just because of ordering with loadscope, it added back like 30-45 secs, but it's still better now!

I'm wondering about all the skipped tests. Do they ever get tested?

[nvuillam]

@echoix all test classes inherit from the same root test class, so for example to test the fixes, if the linter can not fix, we skip the test ;)

[nvuillam]

Example:

def test_linter_format_fix(linter, test_self):
    if (
        linter.disabled is True
        or "all" in getattr(linter, "descriptor_flavors_exclude", [])
        or (linter.is_formatter is False and linter.cli_lint_fix_arg_name is None)
    ):
        raise unittest.SkipTest("Linter doesn't format and can't apply fixes")