cli: add parallel test running and order randomizer #575
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test runner now uses worker_threads to dispatch test files. Every worker runs the global setup before executing tests and runs the global teardown when no test files are available.
The unit of distribution used are files, so tests in a single file will always run in serial.
When
--serial
is passed, the standard test runner is used, and the same happens when you execute a test file directly and callmainTestFn
.To find flaky tests you can use
--randomize-rounds=n
, where n is any integer. This will randomize the files passed to workers. Note that the workers are restarted after every full test run, and thus the global setup and teardown are execute multiple times.To enable these arguments, the cli parser also got upgraded. We try to match Node.js arguments against their long flags as specified in
node --help
.Closes #509