Make reported captured output include function scope fixture finalizer output.

[pytestbot]

Originally reported by: Jurko Gospodnetić (BitBucket: jurko, GitHub: jurko)

---

My suggestion is to make regular test output capture include output from function scoped fixture finalizers.

The same might not be desirable for fixtures with larger scopes as their finalizers are not run for every test using them, and they are not included in this proposal.

Here's a scenario that should help you understand why I think this function level fixture behaviour is desirable:

Imagine you are a new ptytest user not deeply familiar with pytest specific APIs. And you want your tests to output additional calculated information in case of failure (e.g. some test related output collected from an external process). Outputting that information to stdout will work fine, since pytest will actually display that output only for failing tests. The only remaining question is when to output that information.

If it is only a few tests, this output can simply be added in a finally clause wrapping the whole test code, but in case this is needed in a lot of tests, one tries to think of a better solution. The first thing that pops to mind is to implement a fixture that would output this information in its teardown.

You check the pytest documentation for 'teardown' and easily find that you implement this with pytest using a function scoped fixture with teardown code contained as its finalizer routine.

Now you add the finalizer, add your code and everything should work fine... but alas... it is not. You run your failing test and no output appears.

Example:

import pytest

@pytest.fixture
def passwd(request):
    print("setup before yield")
    def finalizer():
        print("teardown after yield")
    request.addfinalizer(finalizer)
    return "gaga"

def test_has_lines(passwd):
    print("test called (%s)" % (passwd,))
    pytest.fail()

And you're pretty much out of ideas other than the ugly solution to manually wrap all your tests in a try:/finally: as suggested before.

You take another stab at the docs, and run into an experimental yield based fixture feature so you decide to try that. You rework your code, but in the end it suffers from the same problem - the teartdown output simply does not get captured.

Example:

import pytest

@pytest.yield_fixture
def passwd():
    print("setup before yield")
    yield "gaga"
    print("teardown after yield")

def test_has_lines(passwd):
    print("test called (%s)" % (passwd,))
    pytest.fail()

Now you are completely out of options unless you dive into pytest source code or ask someone much more experienced with pytest who can point you in the direction of undocumented pytest report hooks or some other such expose-pytest-bowels solution.

Even if you find out how to do this you will have wasted a lot more time on this then you'd like.

Adding and/or another pytest specific solution could work, but a new user would still expect the approach above to work, and nothing in his way would warn him that this was a dead end.

My suggestion is to just let pytest consistently capture function scope fixture finalizer output (yield based or not). That should allow intuitive pytest usage using standard test framework patterns without having to search for or learn additional pytest specific functionality.

If this is added, additional pytest specific functionality could be used (if documented and made public) to improve the solution or make its results even nicer, but pytest should still allow the user to quickly solve his problem in a way consistent with typical testing framework usage.

Hope this helps.

Best regards,
Jurko Gospodnetić

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/442

[aequitas]

Would love to see this fixed.

[vincentbernat]

I would love this too. There is some workaround documented in #225.

[campbellr]

I ran into the same issue, and the following (somewhat hacky) change seems to make it behave as expected:

diff --git a/_pytest/terminal.py b/_pytest/terminal.py
index 825f553..85ae1ad 100644
--- a/_pytest/terminal.py
+++ b/_pytest/terminal.py
@@ -498,6 +498,11 @@ class TerminalReporter:

     def _outrep_summary(self, rep):
         rep.toterminal(self._tw)
+        # Grab the report that actually includes teardown info
+        for report in self.getreports(''):
+            if report.nodeid == rep.nodeid and report.when == 'teardown':
+                rep = report
         for secname, content in rep.sections:
             self._tw.sep("-", secname)
             if content[-1:] == "\n":

The problem is that there is a separate report for setup, call and teardown, with each successive report apparently a superset of the previous one (in terms of the captured output it contains anyway).

My change just makes sure that we always use the teardown report for a particular test which includes all the output captured in setup, call and teardown.

One issue is that the combination of --exitfirst and -n X (from xdist) still causes the teardown output to be lost because pytest-xdist doesn't wait to receive the teardown report before bailing out. I haven't figured out a good solution for that, but I'll probably end up filing an issue.

[matclab]

I've monkeypatched summary_failures as follow in my own plugin and it looks like its working:

def print_teardown_sections(self, rep):
    """ Monkey patched into TerminalReporter """
    for secname, content in rep.sections:
        if 'teardown' in secname:
            self._tw.sep('-', secname)
            if content[-1:] == "\n":
                content = content[:-1]
            self._tw.line(content)


def summary_failures_with_teardonw(self):
    """ To monkey patch TerminalReporter.summary_failure, in order to display
    teardown stdout and stderr sections """
    if self.config.option.tbstyle != "no":
        reports = self.getreports('failed')
        if not reports:
            return
        self.write_sep("=", "FAILURES")
        for rep in reports:
            if self.config.option.tbstyle == "line":
                line = self._getcrashline(rep)
                self.write_line(line)
            else:
                msg = self._getfailureheadline(rep)
                markup = {'red': True, 'bold': True}
                self.write_sep("_", msg, **markup)
                self._outrep_summary(rep)
                for report in self.getreports(''):
                    if report.nodeid == rep.nodeid and report.when == 'teardown':
                        #self._outrep_summary(report)
                        self.print_teardown_sections(report)


@pytest.hookimpl(tryfirst=True)
def pytest_terminal_summary(terminalreporter, exitstatus):
    terminalreporter.__class__.print_teardown_sections = print_teardown_sections
    terminalreporter.__class__.summary_failures = summary_failures_with_teardonw
    return

[RonnyPfannschmidt]

@matclab how about creating a pull request and a test?

[matclab]

@RonnyPfannschmidt not a bad idea !
I'll give a try.