self._outcome.errors no longer contains error data since pytest 5.4.0

[mdmintz]

self._outcome.errors no longer contains error data since pytest 5.4.0

My test framework uses pytest to run tests that use a unittest.TestCase structure, and in the tearDown() method I was able to get error output from failing tests by using:
self._outcome.errors
This was working fine with pytest 5.3.5, but stopped working with pytest 5.4.0

This technique was made possible by this very popular solution on Stack Overflow: https://stackoverflow.com/a/39606065

Here's a code snippet that uses this technique to get the error from the tearDown() step when a test fails an assertion:

import unittest

class MyTest(unittest.TestCase):

    def tearDown(self):
        if hasattr(self, '_outcome'):
            errors = self._outcome.errors
            ...

I would like pytest to restore the use of self._outcome.errors so that tearDown() methods can easily access the error output of tests.
The test framework that I'm using (which I've built) is called SeleniumBase => https://github.com/seleniumbase/SeleniumBase
My workaround for now has been to hard-code pytest 5.3.5 into the requirements file for SeleniumBase. Given the large number of companies using SeleniumBase for test automation, I have to keep things working while waiting for a fix to arrive.

[Zac-HD]

(I've tagged this as a proposal, because accessing a private attribute is almost the definition of unsupported. If it doesn't make anything else worse we'd probably accept a PR to restore the old behaviour, but no promises that it wouldn't get changed again later...)

[RonnyPfannschmidt]

this is a unittest internal, it should be available, i believe this is part of why teadown currently fails

[blueyed]

@RonnyPfannschmidt teardown is skipped, which includes unittest feeding the errors.
I think this should be clear, isn't it?

[mdmintz]

@RonnyPfannschmidt , @blueyed , I created a test that'll pass when this issue is fixed. You'll be able to use it to determine which commit broke the functionality. It passes on 5.3.5 and fails on 5.4.0. There's a commit somewhere in between that broke things.

def test_outcome_errors(testdir):
    testpath = testdir.makepyfile(
        """
        import unittest
        class MyTestCase(unittest.TestCase):
            def test_fail(self):
                raise Exception("FAIL!")
            def tearDown(self):
                print(self._outcome.errors)
    """
    )
    reprec = testdir.inline_run(testpath)
    passed, skipped, failed = reprec.countoutcomes()
    assert failed == 1, failed

[mdmintz]

That test will fit right here: https://github.com/pytest-dev/pytest/blob/master/testing/test_unittest.py

[blueyed]

@mdmintz
Thanks, the issue is known already: #6947
(as mentioned in seleniumbase/SeleniumBase#534 (comment) already). Closing as duplicate.