Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Fixture finalizer failure causes other finalizers to not be run #287
When tearing down fixtures and one finalizer (registered via
Of course - the fixture finalizer could be fixed to not throw exceptions and the problem would go away.
However, I find it useful to be able to make assertions in the fixture teardown. I have a stubbing fixture (similar to pytest's monkeypatch). It checks that the stubbed methods actually was called during the tests, and triggers assertions otherwise. When this happens in one test case, it causes database fixture and all other kinds of failures for all other tests.
I am not sure this is a valid use case for fixtures, if not, feel free to close it. :)
Independently of whether it's a good idea to do post-condition checking in teardowns (not really IMO), i wonder if we should guard each call of a finalizer. Not running finalizers due to an early failing one can bork the whole test run more than neccessary.
I was also surprised to see the behavior that Andreas reported. The interaction with the tmpdir fixture is particularly painful. If tmpdir isn't "finalized" because an earlier finalizer failed, then the next test will use the previous test's tmpdir!
I came across this issue when a finalizer threw an exception due to a bug in the test (the finalizer tried to wait() on a pid that the test had erroneously waited on). The bug in the test was non-deterministic and the probability of it appearing was only non-infinitesimal when 10+ tests were running in parallel using xdist. Hence, occasionally during heavy test runs I'd see a bunch of tests fail because they got very confused by files in their tmpdirs that had been left around by other tests. If a fresh tmpdir had been provided every time, then I would have just seen the test with the buggy wait() fail and presumably fixed it much sooner ;-)
Here's a small test to demonstrate the problem with tmpdir:
And the output: