capture is not resumed after using disabled() context manager - impact capsys and capfd

[jcfr]

First, thanks for maintaining such a great testing framework. It rocks !

It looks like that capture is not resumed after using the disabled() context manager. The following example should help reproduce the problem.

The problem is reproducible on Ubuntu 15.10 using either python 2.7 / python 3.5, with pytest 3.0.2 or pytest master.

import os

def _check(cap, expect_fd=False):
    print("sys output")
    if expect_fd:
        os.write(1, "fd output\n".encode("ascii"))
    out, err = cap.readouterr()
    assert out == "sys output\n%s" % ("fd output\n" if expect_fd else "")


def test_capsys(capsys):
    _check(capsys)


def test_capfd(capfd):
    _check(capfd, expect_fd=True)


def test_capsys_resume(capsys):
    _check(capsys)

    with capsys.disabled():
        pass

    _check(capsys)


def test_capfd_resume(capfd):
    _check(capfd, expect_fd=True)

    with capfd.disabled():
        pass

    _check(capfd, expect_fd=True)

$ pytest -v
[...]
test_capfd_resume.py::test_capsys PASSED
test_capfd_resume.py::test_capfd PASSED
test_capfd_resume.py::test_capsys_resume FAILED
test_capfd_resume.py::test_capfd_resume FAILED

[jcfr]

I suspect that the call to readouterr() is messing things up. It is not used in test_capture.py::test_disabled_capture_fixture.

See

pytest/testing/test_capture.py

Lines 502 to 516 in 3301a1c

	@pytest.mark.parametrize('fixture', ['capsys', 'capfd'])
	def test_disabled_capture_fixture(self, testdir, fixture):
	testdir.makepyfile("""
	def test_disabled({fixture}):
	print('captured before')
	with {fixture}.disabled():
	print('while capture is disabled')
	print('captured after')
	""".format(fixture=fixture))
	result = testdir.runpytest_subprocess()
	result.stdout.fnmatch_lines("""
	*while capture is disabled*
	""")
	assert 'captured before' not in result.stdout.str()
	assert 'captured after' not in result.stdout.str()

    @pytest.mark.parametrize('fixture', ['capsys', 'capfd'])
    def test_disabled_capture_fixture(self, testdir, fixture):
        testdir.makepyfile("""
            def test_disabled({fixture}):
                print('captured before')
                with {fixture}.disabled():
                    print('while capture is disabled')
                print('captured after')
        """.format(fixture=fixture))
        result = testdir.runpytest_subprocess()
        result.stdout.fnmatch_lines("""
            *while capture is disabled*
        """)
        assert 'captured before' not in result.stdout.str()
assert 'captured after' not in result.stdout.str()

[jcfr]

To have a better understanding, I elaborated the test case and it look like the use of disabled() context manager prevent the successful use of @readouterr()

Here are the results I am getting (still on Ubuntu 15.10 with pytest 3.0.3/master and python2.7/3.5):

test_capfd_resume.py::test_capsys PASSED
test_capfd_resume.py::test_capfd PASSED
test_capfd_resume.py::test_capsys_resume___without_disabled___without_intermediate_readouterr PASSED
test_capfd_resume.py::test_capfd__resume___without_disabled___without_intermediate_readouterr PASSED
test_capfd_resume.py::test_capsys_resume___without_disabled___with____intermediate_readouterr PASSED
test_capfd_resume.py::test_capfd__resume___without_disabled___with____intermediate_readouterr PASSED
test_capfd_resume.py::test_capsys_resume___with____disabled___without_intermediate_readouterr FAILED
test_capfd_resume.py::test_capfd__resume___with____disabled___without_intermediate_readouterr FAILED
test_capfd_resume.py::test_capsys_resume___with____disabled___with____intermediate_readouterr FAILED
test_capfd_resume.py::test_capfd__resume___with____disabled___with____intermediate_readouterr FAILED

Using the following script

import os
import pytest

def _check(cap, expect_fd=False, with_readouterr=True, expected_text_twice=False):
    print("sys output")
    if expect_fd:
        os.write(1, "fd output\n".encode("ascii"))
    if with_readouterr:
        out, err = cap.readouterr()
        expected_text = "sys output\n%s" % ("fd output\n" if expect_fd else "")
        if expected_text_twice:
            expected_text += expected_text
        assert out == expected_text


def test_capsys(capsys):
    _check(capsys)


def test_capfd(capfd):
    _check(capfd, expect_fd=True)


def test_capsys_resume___without_disabled___without_intermediate_readouterr(capsys):
    _check(capsys, with_readouterr=False)
    _check(capsys, expected_text_twice=True)


def test_capfd__resume___without_disabled___without_intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True, with_readouterr=False)
    _check(capfd, expect_fd=True, expected_text_twice=True)


def test_capsys_resume___without_disabled___with____intermediate_readouterr(capsys):
    _check(capsys)
    _check(capsys)


def test_capfd__resume___without_disabled___with____intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True)
    _check(capfd, expect_fd=True)


def test_capsys_resume___with____disabled___without_intermediate_readouterr(capsys):
    _check(capsys, with_readouterr=False)
    with capsys.disabled():
        pass
    _check(capsys, expected_text_twice=True)


def test_capfd__resume___with____disabled___without_intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True, with_readouterr=False)
    with capfd.disabled():
        pass
    _check(capfd, expect_fd=True, expected_text_twice=True)


def test_capsys_resume___with____disabled___with____intermediate_readouterr(capsys):
    _check(capsys)
    with capsys.disabled():
        pass
    _check(capsys)


def test_capfd__resume___with____disabled___with____intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True)
    with capfd.disabled():
        pass
    _check(capfd, expect_fd=True)

[khpeek]

I'm facing the same issue in Pytest 3.1.3 (see https://stackoverflow.com/questions/46153983/pytests-capsys-disabled-not-working-as-expected).

[nicoddemus]

@jcfr thanks for providing the test cases, sorry nobody replied it seems this has gone under the radar. 😞

@khpeek thanks for the ping.

[ghost]

I'm looking at this issue as well. In particular, I want to suppress "Requesting Pages: 0 through 19" in the output.

============================= test session starts =============================
platform win32 -- Python 3.6.2, pytest-3.2.2, py-1.4.34, pluggy-0.4.0
rootdir: C:\Users\creimer\Documents\Programming\Python\Slashdot, inifile:
collected 11 items

..\tests\test_requestor.py ..........Requesting Pages: 0 through 19
.

========================== 11 passed in 1.13 seconds ==========================

This doesn't work:

    with capsys.disabled():
        requestor.request_pages(out_queue)

This does work but is ugly as heck.

    old_out = sys.stdout
    sys.stdout = open(os.devnull, 'w')
    requestor.request_pages(out_queue)
    sys.stdout = old_out

[The-Compiler]

@cdreimer Huh, it seems to me that capsys.disabled() does the opposite of what you want - that output should already be captured by pytest by default.

[ghost]

@The-Compiler Perhaps I misunderstood the capsys.disabled() documentation. I'm trying to suppress a print statement from appearing in the output. I got an ugly workaround for this. Is there a better approach to this problem?

[The-Compiler]

@cdreimer pytest does this by default unless you start it with -s. If it doesn't, that might be some kind of bug.

[ghost]

@The-Compiler Looks like the PyCharm IDE is adding something extra in the background. I don't have any problems running pytest from the CLI. Second time this week that PyCharm bit me in the ass -- and it's only Tuesday. :/

[chirinosky]

Hi, I seems this issue has regressed (at least for me). The output of statements run outside of with capfd.disabled(): continues to be suppressed. I'm currently having to call capfd._start() after the with statement to continue the capture.

I am running pytest 3.2.4 under Python 3.6.2.