Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing the "shallow" torture mode #4699

wants to merge 2 commits into from


Copy link

@bagder bagder commented Dec 10, 2019

With a new option to, we can limit the number of failures to cause for each test case and should then possible be able to run over more tests in the CI and elsewhere.

When running over a large amount of tests, torture mode tests the same function failures (in the common code paths) a very large number of times. That then makes them so slow that we can never really run them much and thus they decrease in value.

When set, shallow mode limits runtests -t to make no more than NUM fails
per test case. If more are found, it will randomly discard entries until
the number is right. The random seed can also be set.

This is particularly useful when running MANY tests as then most torture
failures will already fail the same functions over and over and make the
total operation painfully tedious.
As a first step to enable it to run over a more diverse set of tests in
a reasonable time.
@bagder bagder added the tests label Dec 10, 2019
@bagder bagder mentioned this pull request Dec 10, 2019
7 tasks
@bagder bagder closed this in 7c1bd03 Dec 11, 2019
@bagder bagder deleted the bagder/tests-shallow-torture branch December 11, 2019 08:55
@lock lock bot locked as resolved and limited conversation to collaborators Mar 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Successfully merging this pull request may close these issues.

None yet

1 participant