Introducing the "shallow" torture mode #4699
With a new option to
When running over a large amount of tests, torture mode tests the same function failures (in the common code paths) a very large number of times. That then makes them so slow that we can never really run them much and thus they decrease in value.
When set, shallow mode limits runtests -t to make no more than NUM fails per test case. If more are found, it will randomly discard entries until the number is right. The random seed can also be set. This is particularly useful when running MANY tests as then most torture failures will already fail the same functions over and over and make the total operation painfully tedious.