Ideas : Better random testing #16434
Labels
Enhancement
Testing
Related to the test runner. Do not use for test failures unless it relates to the test runner itself
Well, I see some random tests are failing, but they are too difficult to debug.
e.g #16288, #14481
I only understand that RNG changes the state every time it computes, and in that is the case, if our only clue is the random seed, the test may have to follow lengthy steps run in the way exactly how the travis do, only to inspect it.
And also, there is a suspicion that random test may create spurious coverage #16196
I think it may be better to
Make more verbose log for random tests, so it may print the exact expression which had failed the testing.
We may have to categorize random tests, like using
@randtest
, and all random tests can be collected and run in a separate travis stage. It will be good if there is a pytest equivalent wrapper for that.The text was updated successfully, but these errors were encountered: