Fixture finalizer failure causes other finalizers to not be run

[pytestbot]

Originally reported by: Andreas Pelme (BitBucket: pelme, GitHub: pelme)

---

When tearing down fixtures and one finalizer (registered via request.addfinalizer) fails (with an uncaught exception or with pytest.fail) - no other finalizers will be run.

Of course - the fixture finalizer could be fixed to not throw exceptions and the problem would go away.

However, I find it useful to be able to make assertions in the fixture teardown. I have a stubbing fixture (similar to pytest's monkeypatch). It checks that the stubbed methods actually was called during the tests, and triggers assertions otherwise. When this happens in one test case, it causes database fixture and all other kinds of failures for all other tests.

I am not sure this is a valid use case for fixtures, if not, feel free to close it. :)

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/287

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Independently of whether it's a good idea to do post-condition checking in teardowns (not really IMO), i wonder if we should guard each call of a finalizer. Not running finalizers due to an early failing one can bork the whole test run more than neccessary.

[pytestbot]

Original comment by Andreas Pelme (BitBucket: pelme, GitHub: pelme):

---

(I am not sure it is a good idea either, just playing around with different approaches at the moment.)

It can be a bit surprising that the outcome of the test run is different because of the implicit fixture finalizer order.

[pytestbot]

Original comment by Peter Feiner (BitBucket: peterfeiner, GitHub: peterfeiner):

---

I was also surprised to see the behavior that Andreas reported. The interaction with the tmpdir fixture is particularly painful. If tmpdir isn't "finalized" because an earlier finalizer failed, then the next test will use the previous test's tmpdir!

I came across this issue when a finalizer threw an exception due to a bug in the test (the finalizer tried to wait() on a pid that the test had erroneously waited on). The bug in the test was non-deterministic and the probability of it appearing was only non-infinitesimal when 10+ tests were running in parallel using xdist. Hence, occasionally during heavy test runs I'd see a bunch of tests fail because they got very confused by files in their tmpdirs that had been left around by other tests. If a fresh tmpdir had been provided every time, then I would have just seen the test with the buggy wait() fail and presumably fixed it much sooner ;-)

Here's a small test to demonstrate the problem with tmpdir:

#!python

import pytest

@pytest.fixture(scope='function')
def fixture(request, tmpdir):
        print 'setup'
        def death():
                print 'teardown'
                raise Exception('death cometh')
        request.addfinalizer(death)
        return tmpdir

def test_a(fixture):
        print 'test_a running with', fixture

def test_b(fixture):
        print 'test_b running with', fixture

And the output:

#!txt

$ py.test simple_test.py  -v --capture=no
=== test session starts ===
platform linux2 -- Python 2.6.6 -- pytest-2.3.5 -- /usr/bin/python
plugins: xdist
collected 2 items

simple_test.py:12: test_a setup
test_a running with /tmp/pytest-221/test_a0
PASSEDteardown

simple_test.py:12: test_a ERROR
simple_test.py:16: test_b test_b running with /tmp/pytest-221/test_a0
PASSED

=== ERRORS ===
___ ERROR at teardown of test_a ___

    def death():
            print 'teardown'
>           raise Exception('death cometh')
E           Exception: death cometh

simple_test.py:8: Exception
=== 2 passed, 1 error in 0.12 seconds ===

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

Ensure all finalizations are run when one fails

Fixes issue287.