When introducing a delicate change to a large code base (2000+ tests), I often run all tests (using xdist) to see what has been affected, then ooops, 400 broken tests.
I would like to go fixing module by module to get everything green again.
What happens currently is this:
Because of this, I often find myself copying/pasting the list of failed modules somewhere so I can run them selectively, instead of relying on the caching mechanism.
The workflow I want:
This PR attempts to fix this by making the cache plugin always remember which tests failed at a certain point, discarding a test failure only when it sees that test passing again.