Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

capture is not resumed after using `disabled()` context manager - impact capsys and capfd #1993

Closed
jcfr opened this Issue Oct 9, 2016 · 10 comments

Comments

Projects
None yet
5 participants
@jcfr
Copy link

jcfr commented Oct 9, 2016

First, thanks for maintaining such a great testing framework. It rocks !

It looks like that capture is not resumed after using the disabled() context manager. The following example should help reproduce the problem.

The problem is reproducible on Ubuntu 15.10 using either python 2.7 / python 3.5, with pytest 3.0.2 or pytest master.

import os

def _check(cap, expect_fd=False):
    print("sys output")
    if expect_fd:
        os.write(1, "fd output\n".encode("ascii"))
    out, err = cap.readouterr()
    assert out == "sys output\n%s" % ("fd output\n" if expect_fd else "")


def test_capsys(capsys):
    _check(capsys)


def test_capfd(capfd):
    _check(capfd, expect_fd=True)


def test_capsys_resume(capsys):
    _check(capsys)

    with capsys.disabled():
        pass

    _check(capsys)


def test_capfd_resume(capfd):
    _check(capfd, expect_fd=True)

    with capfd.disabled():
        pass

    _check(capfd, expect_fd=True)
$ pytest -v
[...]
test_capfd_resume.py::test_capsys PASSED
test_capfd_resume.py::test_capfd PASSED
test_capfd_resume.py::test_capsys_resume FAILED
test_capfd_resume.py::test_capfd_resume FAILED
@jcfr

This comment has been minimized.

Copy link
Author

jcfr commented Oct 9, 2016

I suspect that the call to readouterr() is messing things up. It is not used in test_capture.py::test_disabled_capture_fixture.

See

@pytest.mark.parametrize('fixture', ['capsys', 'capfd'])
def test_disabled_capture_fixture(self, testdir, fixture):
testdir.makepyfile("""
def test_disabled({fixture}):
print('captured before')
with {fixture}.disabled():
print('while capture is disabled')
print('captured after')
""".format(fixture=fixture))
result = testdir.runpytest_subprocess()
result.stdout.fnmatch_lines("""
*while capture is disabled*
""")
assert 'captured before' not in result.stdout.str()
assert 'captured after' not in result.stdout.str()

    @pytest.mark.parametrize('fixture', ['capsys', 'capfd'])
    def test_disabled_capture_fixture(self, testdir, fixture):
        testdir.makepyfile("""
            def test_disabled({fixture}):
                print('captured before')
                with {fixture}.disabled():
                    print('while capture is disabled')
                print('captured after')
        """.format(fixture=fixture))
        result = testdir.runpytest_subprocess()
        result.stdout.fnmatch_lines("""
            *while capture is disabled*
        """)
        assert 'captured before' not in result.stdout.str()
assert 'captured after' not in result.stdout.str()
@jcfr

This comment has been minimized.

Copy link
Author

jcfr commented Oct 9, 2016

To have a better understanding, I elaborated the test case and it look like the use of disabled() context manager prevent the successful use of @readouterr()

Here are the results I am getting (still on Ubuntu 15.10 with pytest 3.0.3/master and python2.7/3.5):

test_capfd_resume.py::test_capsys PASSED
test_capfd_resume.py::test_capfd PASSED
test_capfd_resume.py::test_capsys_resume___without_disabled___without_intermediate_readouterr PASSED
test_capfd_resume.py::test_capfd__resume___without_disabled___without_intermediate_readouterr PASSED
test_capfd_resume.py::test_capsys_resume___without_disabled___with____intermediate_readouterr PASSED
test_capfd_resume.py::test_capfd__resume___without_disabled___with____intermediate_readouterr PASSED
test_capfd_resume.py::test_capsys_resume___with____disabled___without_intermediate_readouterr FAILED
test_capfd_resume.py::test_capfd__resume___with____disabled___without_intermediate_readouterr FAILED
test_capfd_resume.py::test_capsys_resume___with____disabled___with____intermediate_readouterr FAILED
test_capfd_resume.py::test_capfd__resume___with____disabled___with____intermediate_readouterr FAILED

Using the following script

import os
import pytest

def _check(cap, expect_fd=False, with_readouterr=True, expected_text_twice=False):
    print("sys output")
    if expect_fd:
        os.write(1, "fd output\n".encode("ascii"))
    if with_readouterr:
        out, err = cap.readouterr()
        expected_text = "sys output\n%s" % ("fd output\n" if expect_fd else "")
        if expected_text_twice:
            expected_text += expected_text
        assert out == expected_text


def test_capsys(capsys):
    _check(capsys)


def test_capfd(capfd):
    _check(capfd, expect_fd=True)


def test_capsys_resume___without_disabled___without_intermediate_readouterr(capsys):
    _check(capsys, with_readouterr=False)
    _check(capsys, expected_text_twice=True)


def test_capfd__resume___without_disabled___without_intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True, with_readouterr=False)
    _check(capfd, expect_fd=True, expected_text_twice=True)


def test_capsys_resume___without_disabled___with____intermediate_readouterr(capsys):
    _check(capsys)
    _check(capsys)


def test_capfd__resume___without_disabled___with____intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True)
    _check(capfd, expect_fd=True)


def test_capsys_resume___with____disabled___without_intermediate_readouterr(capsys):
    _check(capsys, with_readouterr=False)
    with capsys.disabled():
        pass
    _check(capsys, expected_text_twice=True)


def test_capfd__resume___with____disabled___without_intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True, with_readouterr=False)
    with capfd.disabled():
        pass
    _check(capfd, expect_fd=True, expected_text_twice=True)


def test_capsys_resume___with____disabled___with____intermediate_readouterr(capsys):
    _check(capsys)
    with capsys.disabled():
        pass
    _check(capsys)


def test_capfd__resume___with____disabled___with____intermediate_readouterr(capfd):
    _check(capfd, expect_fd=True)
    with capfd.disabled():
        pass
    _check(capfd, expect_fd=True)

jcfr added a commit to scikit-build/scikit-ci that referenced this issue Oct 9, 2016

test_scikit_ci: Display output and error only if test_driver failed
This will facilitate fixing remaining issue.

Since disabled() can not be used with readouterr [1], we refactor the code
so that disabled() is called only once at the end of the test.

[1] See pytest-dev/pytest#1993
@khpeek

This comment has been minimized.

Copy link

khpeek commented Sep 11, 2017

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Sep 12, 2017

@jcfr thanks for providing the test cases, sorry nobody replied it seems this has gone under the radar. 😞

@khpeek thanks for the ping.

@ghost

This comment has been minimized.

Copy link

ghost commented Sep 12, 2017

I'm looking at this issue as well. In particular, I want to suppress "Requesting Pages: 0 through 19" in the output.

============================= test session starts =============================
platform win32 -- Python 3.6.2, pytest-3.2.2, py-1.4.34, pluggy-0.4.0
rootdir: C:\Users\creimer\Documents\Programming\Python\Slashdot, inifile:
collected 11 items

..\tests\test_requestor.py ..........Requesting Pages: 0 through 19
.

========================== 11 passed in 1.13 seconds ==========================

This doesn't work:

    with capsys.disabled():
        requestor.request_pages(out_queue)

This does work but is ugly as heck.

    old_out = sys.stdout
    sys.stdout = open(os.devnull, 'w')
    requestor.request_pages(out_queue)
    sys.stdout = old_out
@The-Compiler

This comment has been minimized.

Copy link
Member

The-Compiler commented Sep 12, 2017

@cdreimer Huh, it seems to me that capsys.disabled() does the opposite of what you want - that output should already be captured by pytest by default.

@ghost

This comment has been minimized.

Copy link

ghost commented Sep 12, 2017

@The-Compiler Perhaps I misunderstood the capsys.disabled() documentation. I'm trying to suppress a print statement from appearing in the output. I got an ugly workaround for this. Is there a better approach to this problem?

@The-Compiler

This comment has been minimized.

Copy link
Member

The-Compiler commented Sep 12, 2017

@cdreimer pytest does this by default unless you start it with -s. If it doesn't, that might be some kind of bug.

@ghost

This comment has been minimized.

Copy link

ghost commented Sep 13, 2017

@The-Compiler Looks like the PyCharm IDE is adding something extra in the background. I don't have any problems running pytest from the CLI. Second time this week that PyCharm bit me in the ass -- and it's only Tuesday. :/

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Sep 29, 2017

@nicoddemus nicoddemus closed this Oct 9, 2017

@chirinosky

This comment has been minimized.

Copy link

chirinosky commented Jan 4, 2018

Hi, I seems this issue has regressed (at least for me). The output of statements run outside of with capfd.disabled(): continues to be suppressed. I'm currently having to call capfd._start() after the with statement to continue the capture.

I am running pytest 3.2.4 under Python 3.6.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.