Add an option to retry package testing #240
Comments
While we are dreaming, it would also be nice if one could limit the retry to the specific tests or contexts that failed. |
@craigcitro Can't you just retry from the shell? |
@gaborcsardi sure -- but it'd be even better to ask |
Could you give a bit more context? |
I know this is @craigcitro's issue ... but what I meant when I chimed in was this: 1 out of every x times, a test fails for |
Hmm I think @craigcitro probably wants something that retries all the tests, but an expectation level thing also sounds useful |
the original context was indeed "just retry everything" -- which is an ugly way to do things, but i thought might be the most likely to get implemented as v0. the goal was to use retries to paper over any flakiness due to external factors (other APIs, network woes, hardware wonkiness, etc). that said, i think this would be far superior as something you could enable for a group of tests, in particular to avoid ignoring tests that are flaky but shouldn't be. (in my head, at least, "just rerun all the tests" is the same sort of band-aid solution as using sleep() when trying to work around race conditions.) then users could make their CI systems happy without creating new problems. and, as mentioned above, i like the idea of this being part of maybe those were all my issues? i have so many, it's hard to keep track. |
This would be especially useful in contexts like travis.
The text was updated successfully, but these errors were encountered: