Add an option to retry package testing #240
I know this is @craigcitro's issue ... but what I meant when I chimed in was this:
1 out of every x times, a test fails for
the original context was indeed "just retry everything" -- which is an ugly way to do things, but i thought might be the most likely to get implemented as v0. the goal was to use retries to paper over any flakiness due to external factors (other APIs, network woes, hardware wonkiness, etc).
that said, i think this would be far superior as something you could enable for a group of tests, in particular to avoid ignoring tests that are flaky but shouldn't be. (in my head, at least, "just rerun all the tests" is the same sort of band-aid solution as using sleep() when trying to work around race conditions.) then users could make their CI systems happy without creating new problems.
and, as mentioned above, i like the idea of this being part of
maybe those were all my issues? i have so many, it's hard to keep track.