Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pdb prompt unusable if doctests ran first (macOS and Python 3) #985

Closed
jaraco opened this Issue Aug 31, 2015 · 29 comments

Comments

Projects
None yet
4 participants
@jaraco
Copy link
Contributor

jaraco commented Aug 31, 2015

Consider this simple test:

">>> 3\n3"

def test_stuff():
    import pdb; pdb.set_trace()

Create a virtualenv on Python 3.4.3, install pytest 2.7.2 and py 1.4.30 and run py.test:

(pytest) $ py.test
====================================== test session starts =======================================
platform darwin -- Python 3.4.3 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/jaraco/foo, inifile: 
collected 1 items 

test_stuff.py 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
--Return--
> /Users/jaraco/foo/test_stuff.py(4)test_stuff()->None
-> import pdb; pdb.set_trace()
(Pdb) q
F

============================================ FAILURES ============================================
___________________________________________ test_stuff ___________________________________________

    def test_stuff():
>       import pdb; pdb.set_trace()

test_stuff.py:4: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/bdb.py:52: in trace_dispatch
    return self.dispatch_return(frame, arg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pdb.Pdb object at 0x102a3f7f0>, frame = <frame object at 0x102a54848>, arg = None

    def dispatch_return(self, frame, arg):
        if self.stop_here(frame) or frame == self.returnframe:
            # Ignore return events in generator except when stepping.
            if self.stopframe and frame.f_code.co_flags & CO_GENERATOR:
                return self.trace_dispatch
            try:
                self.frame_returning = frame
                self.user_return(frame, arg)
            finally:
                self.frame_returning = None
>           if self.quitting: raise BdbQuit
E           bdb.BdbQuit

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/bdb.py:96: BdbQuit
==================================== 1 failed in 1.95 seconds ====================================

Everything's fine - the pdb prompt is displayed and works as expected. However, invoke pytest with --doctest-modules, and the story is different.

(pytest) $ py.test --doctest-modules
====================================== test session starts =======================================
platform darwin -- Python 3.4.3 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/jaraco/foo, inifile: 
collected 2 items 

test_stuff.py .
>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
--Return--
> /Users/jaraco/foo/test_stuff.py(4)test_stuff()->None
-> import pdb; pdb.set_trace()
^D
F

============================================ FAILURES ============================================
___________________________________________ test_stuff ___________________________________________

    def test_stuff():
>       import pdb; pdb.set_trace()

test_stuff.py:4: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/bdb.py:52: in trace_dispatch
    return self.dispatch_return(frame, arg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pdb.Pdb object at 0x102a51be0>, frame = <frame object at 0x102af8248>, arg = None

    def dispatch_return(self, frame, arg):
        if self.stop_here(frame) or frame == self.returnframe:
            # Ignore return events in generator except when stepping.
            if self.stopframe and frame.f_code.co_flags & CO_GENERATOR:
                return self.trace_dispatch
            try:
                self.frame_returning = frame
                self.user_return(frame, arg)
            finally:
                self.frame_returning = None
>           if self.quitting: raise BdbQuit
E           bdb.BdbQuit

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/bdb.py:96: BdbQuit
=============================== 1 failed, 1 passed in 8.34 seconds ===============================

Although the code stops in the right place, the pdb prompt does not appear. Keystrokes appear to be ignored. Only sending an EOF (^d) allows the test to fail and pytest to exit.

This issue seems to have emerged recently, though I can't imagine what could be causing it. I've tested in a virtualenv, tried upgrading and downgrading pytest, and tried older setuptools versions.

What's the next step to troubleshoot?

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Aug 31, 2015

for testing purposes i'd suggest to write s small script using python pexpect that then can be used to test against different versions an potentially for bisecting the responsible change

do you have a rough set of known good/bad releases

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Aug 31, 2015

I rolled back to pytest 2.6.x and 2.7.0, but the issue remained. I'll try some older versions. I bet there's something else in my environment causing the issue but I can't imagine what. I'll try even older versions. Good idea.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Aug 31, 2015

I can reproduce the issue back to pytest 2.4 or py 1.4.24 on Python 2.7 or Python 3, but only on OS X. I've been unable to reproduce the issue on Ubuntu.

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Sep 1, 2015

FWIW I can't reproduce the problem on Windows either. 😕

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

Given that I adopted Mac in May 2015 and that the issue appears to only affect Mac, even on Pytest 2.4 which was released in 2013, I believe the issue is not a recent regression, as I suspected, but simply a broken behavior on macOS (formerly OS X).

@jaraco jaraco changed the title pdb prompt unusable if doctests ran first pdb prompt unusable if doctests ran first (macOS) Oct 21, 2016

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Oct 21, 2016

i see, as such most of us are unable to test/experiment :/

@jaraco jaraco self-assigned this Oct 21, 2016

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

Would someone on the pytest team point me to the code where doctests are invoked and where pdb is invoked? From there, I'll try to devise a minimal test that replicates the failure.

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 21, 2016

@jaraco could you please try running pytest's own test suite first? The current tests might detect the problem already.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

The tests all passed on Python 2.6, but on 3.5, there are many failures.

I don't see anything in those results that appears to me to relate to doctests or pdb prompts. I do see a lot of skips in the pdb tests, but I can't see which skip message(s) are relevant. Aha, so running just the tests in test_pdb, I see the reason so many are skipped: "could not import 'pexpect'".

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 21, 2016

I see the reason so many are skipped: "could not import 'pexpect'".

Those are the ones we want to run, actually. 😁 Are you running using tox?

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

I installed pexpect, but now the tests xfail.

$ .tox/py35/bin/python -m pip install pexpect
...
Successfully installed pexpect-4.2.1 ptyprocess-0.5.1
$ python -m tox -e py35 -- -rsx -k test_pdb
GLOB sdist-make: /Users/jaraco/Dropbox/code/public/pytest/setup.py
py35 inst-nodeps: /Users/jaraco/Dropbox/code/public/pytest/.tox/dist/pytest-3.0.4.dev0.zip
py35 installed: hypothesis==3.5.3,mock==2.0.0,nose==1.3.7,pbr==1.10.0,pexpect==4.2.1,ptyprocess==0.5.1,py==1.4.31,pytest==3.0.4.dev0,requests==2.11.1,six==1.10.0,spark-parser==1.4.0,uncompyle6==2.9.2,xdis==3.1.0
py35 runtests: PYTHONHASHSEED='2412819224'
py35 runtests: commands[0] | pytest --lsof -rfsxX -rsx -k test_pdb
=========================================== test session starts ===========================================
platform darwin -- Python 3.5.2, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/public/pytest, inifile: tox.ini
plugins: hypothesis-3.5.3
collected 1724 items 

testing/test_pdb.py ....xxxxxxxxxxxxxx.x.
========================================= short test summary info =========================================
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_unittest_postmortem
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capture
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_exception
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_on_collection_issue181
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_on_internal_error
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capturing_simple
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_set_trace_interception
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_and_capsys
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_set_trace_capturing_afterwards
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_doctest
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capturing_twice
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_used_outside_test
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_used_in_generate_tests
  reason: pexpect does not work reliably on darwin?!
XFAIL testing/test_pdb.py::TestPDB::()::test_enter_pdb_hook_is_called
  reason: pexpect does not work reliably on darwin?!

========================================== 1703 tests deselected ==========================================
========================== 6 passed, 1703 deselected, 15 xfailed in 3.68 seconds ==========================
_________________________________________________ summary _________________________________________________
  py35: commands succeeded
  congratulations :)
@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

If I remove that xfail:

$ hg diff
diff -r 1d9397a85c2c _pytest/pytester.py
--- a/_pytest/pytester.py   Fri Oct 21 01:36:24 2016 +0200
+++ b/_pytest/pytester.py   Fri Oct 21 11:33:16 2016 -0400
@@ -1002,8 +1002,6 @@
         pexpect = pytest.importorskip("pexpect", "3.0")
         if hasattr(sys, 'pypy_version_info') and '64' in platform.machine():
             pytest.skip("pypy-64 bit not supported")
-        if sys.platform == "darwin":
-            pytest.xfail("pexpect does not work reliably on darwin?!")
         if sys.platform.startswith("freebsd"):
             pytest.xfail("pexpect does not work reliably on freebsd")
         logfile = self.tmpdir.join("spawn.out").open("wb")

The tests now hang partway in.

$ python -m tox -e py35 -- -k test_pdb -v 
GLOB sdist-make: /Users/jaraco/Dropbox/code/public/pytest/setup.py
py35 inst-nodeps: /Users/jaraco/Dropbox/code/public/pytest/.tox/dist/pytest-3.0.4.dev0.zip
py35 installed: hypothesis==3.5.3,mock==2.0.0,nose==1.3.7,pbr==1.10.0,pexpect==4.2.1,ptyprocess==0.5.1,py==1.4.31,pytest==3.0.4.dev0,requests==2.11.1,six==1.10.0,spark-parser==1.4.0,uncompyle6==2.9.2,xdis==3.1.0
py35 runtests: PYTHONHASHSEED='2630976406'
py35 runtests: commands[0] | pytest --lsof -rfsxX -k test_pdb -v
=========================================== test session starts ===========================================
platform darwin -- Python 3.5.2, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0 -- /Users/jaraco/Dropbox/code/public/pytest/.tox/py35/bin/python3.5
cachedir: .cache
rootdir: /Users/jaraco/Dropbox/code/public/pytest, inifile: tox.ini
plugins: hypothesis-3.5.3
collected 1724 items 

testing/test_pdb.py::TestPDB::test_pdb_on_fail PASSED
testing/test_pdb.py::TestPDB::test_pdb_on_xfail PASSED
testing/test_pdb.py::TestPDB::test_pdb_on_skip PASSED
testing/test_pdb.py::TestPDB::test_pdb_on_BdbQuit PASSED
testing/test_pdb.py::TestPDB::test_pdb_interaction PASSED
testing/test_pdb.py::TestPDB::test_pdb_unittest_postmortem PASSED
testing/test_pdb.py::TestPDB::test_pdb_interaction_capture PASSED
testing/test_pdb.py::TestPDB::test_pdb_interaction_exception PASSED
testing/test_pdb.py::TestPDB::test_pdb_interaction_on_collection_issue181 ^CERROR: KEYBOARDINTERRUPT


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
to show a full traceback on KeyboardInterrupt use --fulltrace
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35/lib/python3.5/site-packages/ptyprocess/ptyprocess.py:649: KeyboardInterrupt
========================================== 1703 tests deselected ==========================================
=============================== 8 passed, 1703 deselected in 32.65 seconds ================================
ERROR: keyboardinterrupt
@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 21, 2016

Hmm that's interesting, so it seems to reproduce the issue already then?

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

I don't think so. --pdb works for me in that scenario. It's only in the doctest scenario that it fails... so I'm thinking that pexpect is in fact not working here, as the xfail suggests.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

I added some debug output to determine where the test is hanging.

$ hg diff
diff -r 1d9397a85c2c _pytest/pytester.py
--- a/_pytest/pytester.py   Fri Oct 21 01:36:24 2016 +0200
+++ b/_pytest/pytester.py   Fri Oct 21 11:45:27 2016 -0400
@@ -1002,8 +1002,6 @@
         pexpect = pytest.importorskip("pexpect", "3.0")
         if hasattr(sys, 'pypy_version_info') and '64' in platform.machine():
             pytest.skip("pypy-64 bit not supported")
-        if sys.platform == "darwin":
-            pytest.xfail("pexpect does not work reliably on darwin?!")
         if sys.platform.startswith("freebsd"):
             pytest.xfail("pexpect does not work reliably on freebsd")
         logfile = self.tmpdir.join("spawn.out").open("wb")
diff -r 1d9397a85c2c testing/test_pdb.py
--- a/testing/test_pdb.py   Fri Oct 21 01:36:24 2016 +0200
+++ b/testing/test_pdb.py   Fri Oct 21 11:45:27 2016 -0400
@@ -141,10 +141,15 @@
         child = testdir.spawn_pytest("--pdb %s" % p1)
         #child.expect(".*import pytest.*")
         child.expect("(Pdb)")
+        print("got here 1")
         child.sendeof()
+        print("got here 2")
         child.expect("1 error")
+        print("got here 3")
         if child.isalive():
+            print("got here 4")
             child.wait()
+        print("got here 5")

     def test_pdb_interaction_on_internal_error(self, testdir):
         testdir.makeconftest("""

And it appears to be hanging on the .wait() call.

testing/test_pdb.py::TestPDB::test_pdb_interaction_on_collection_issue181 got here 1
got here 2
got here 3
got here 4
^CERROR: KEYBOARDINTERRUPT
@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

According to the docs, "this [call] will block forever if the child has unread output and has terminated." Maybe that's what's happening.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

After applying #2024, the failure is revealed in this test run:

$ python -m tox -e py35-pexpect                                         
GLOB sdist-make: /Users/jaraco/Dropbox/code/public/pytest/setup.py
py35-pexpect create: /Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect
py35-pexpect installdeps: pexpect
py35-pexpect inst: /Users/jaraco/Dropbox/code/public/pytest/.tox/dist/pytest-3.0.4.dev0.zip
py35-pexpect installed: pexpect==4.2.1,ptyprocess==0.5.1,py==1.4.31,pytest==3.0.4.dev0
py35-pexpect runtests: PYTHONHASHSEED='369771295'
py35-pexpect runtests: commands[0] | pytest -rfsxX test_pdb.py test_terminal.py test_unittest.py
=========================================== test session starts ===========================================
platform darwin -- Python 3.5.2, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/public/pytest, inifile: tox.ini
collected 162 items 

test_pdb.py ..............F......
test_terminal.py ....................................s.............................................................
test_unittest.py ...................ssssssss................
========================================= short test summary info =========================================
FAIL test_pdb.py::TestPDB::()::test_pdb_interaction_doctest
SKIP [4] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:367: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:353: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:314: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:340: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_terminal.py:483: xdist plugin not installed
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:327: could not import 'twisted.trial.unittest'

================================================ FAILURES =================================================
__________________________________ TestPDB.test_pdb_interaction_doctest ___________________________________

self = <pexpect.expect.Expecter object at 0x1037f55f8>, timeout = 9.601989030838013

    def expect_loop(self, timeout=-1):
        """Blocking expect"""
        spawn = self.spawn
        from . import EOF, TIMEOUT

        if timeout is not None:
            end_time = time.time() + timeout

        try:
            incoming = spawn.buffer
            spawn.buffer = spawn.string_type()  # Treat buffer as new data
            while True:
                idx = self.new_data(incoming)
                # Keep reading until exception or return.
                if idx is not None:
                    return idx
                # No match at this point
                if (timeout is not None) and (timeout < 0):
                    return self.timeout()
                # Still have time left, so read more data
>               incoming = spawn.read_nonblocking(spawn.maxread, timeout)

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:99: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pexpect.pty_spawn.spawn object at 0x1037f5ac8>, size = 2000, timeout = 9.601989030838013

    def read_nonblocking(self, size=1, timeout=-1):
        '''This reads at most size characters from the child application. It
            includes a timeout. If the read does not complete within the timeout
            period then a TIMEOUT exception is raised. If the end of file is read
            then an EOF exception will be raised.  If a logfile is specified, a
            copy is written to that log.

            If timeout is None then the read may block indefinitely.
            If timeout is -1 then the self.timeout value is used. If timeout is 0
            then the child is polled and if there is no data immediately ready
            then this will raise a TIMEOUT exception.

            The timeout refers only to the amount of time to read at least one
            character. This is not affected by the 'size' parameter, so if you call
            read_nonblocking(size=100, timeout=30) and only one character is
            available right away then one character will be returned immediately.
            It will not wait for 30 seconds for another 99 characters to come in.

            This is a wrapper around os.read(). It uses select.select() to
            implement the timeout. '''

        if self.closed:
            raise ValueError('I/O operation on closed file.')

        if timeout == -1:
            timeout = self.timeout

        # Note that some systems such as Solaris do not give an EOF when
        # the child dies. In fact, you can still try to read
        # from the child_fd -- it will block forever or until TIMEOUT.
        # For this case, I test isalive() before doing any reading.
        # If isalive() is false, then I pretend that this is the same as EOF.
        if not self.isalive():
            # timeout of 0 means "poll"
            r, w, e = select_ignore_interrupts([self.child_fd], [], [], 0)
            if not r:
                self.flag_eof = True
                raise EOF('End Of File (EOF). Braindead platform.')
        elif self.__irix_hack:
            # Irix takes a long time before it realizes a child was terminated.
            # FIXME So does this mean Irix systems are forced to always have
            # FIXME a 2 second delay when calling read_nonblocking? That sucks.
            r, w, e = select_ignore_interrupts([self.child_fd], [], [], 2)
            if not r and not self.isalive():
                self.flag_eof = True
                raise EOF('End Of File (EOF). Slow platform.')

        r, w, e = select_ignore_interrupts([self.child_fd], [], [], timeout)

        if not r:
            if not self.isalive():
                # Some platforms, such as Irix, will claim that their
                # processes are alive; timeout on the select; and
                # then finally admit that they are not alive.
                self.flag_eof = True
                raise EOF('End of File (EOF). Very slow platform.')
            else:
>               raise TIMEOUT('Timeout exceeded.')
E               pexpect.exceptions.TIMEOUT: Timeout exceeded.

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/pty_spawn.py:462: TIMEOUT

During handling of the above exception, another exception occurred:

self = <test_pdb.TestPDB object at 0x10390ea90>
testdir = <Testdir local('/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0')>

    def test_pdb_interaction_doctest(self, testdir):
        p1 = testdir.makepyfile("""
                import pytest
                def function_1():
                    '''
                    >>> i = 0
                    >>> assert i == 1
                    '''
            """)
        child = testdir.spawn_pytest("--doctest-modules --pdb %s" % p1)
>       child.expect("(Pdb)")

/Users/jaraco/Dropbox/code/public/pytest/testing/test_pdb.py:243: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/spawnbase.py:321: in expect
    timeout, searchwindowsize, async)
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/spawnbase.py:345: in expect_list
    return exp.expect_loop(timeout)
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:107: in expect_loop
    return self.timeout(e)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pexpect.expect.Expecter object at 0x1037f55f8>, err = TIMEOUT('Timeout exceeded.',)

    def timeout(self, err=None):
        spawn = self.spawn
        from . import TIMEOUT

        spawn.before = spawn.buffer
        spawn.after = TIMEOUT
        index = self.searcher.timeout_index
        if index >= 0:
            spawn.match = TIMEOUT
            spawn.match_index = index
            return index
        else:
            spawn.match = None
            spawn.match_index = None
            msg = str(spawn)
            msg += '\nsearcher: %s' % self.searcher
            if err is not None:
                msg = str(err) + '\n' + msg
>           raise TIMEOUT(msg)
E           pexpect.exceptions.TIMEOUT: Timeout exceeded.
E           <pexpect.pty_spawn.spawn object at 0x1037f5ac8>
E           command: /Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/python3.5
E           args: ['/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/python3.5', '/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pytest.py', '--basetemp=/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/pexpect', '--doctest-modules', '--pdb', '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/test_pdb_interaction_doctest.py']
E           buffer (last 100 chars): b'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> <doctest test_pdb_interaction_doctest.function_1[1]>(1)<module>()\r\n'
E           before (last 100 chars): b'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> <doctest test_pdb_interaction_doctest.function_1[1]>(1)<module>()\r\n'
E           after: <class 'pexpect.exceptions.TIMEOUT'>
E           match: None
E           match_index: None
E           exitstatus: None
E           flag_eof: False
E           pid: 34733
E           child_fd: 13
E           closed: False
E           timeout: 10.0
E           delimiter: <class 'pexpect.exceptions.EOF'>
E           logfile: <_io.BufferedWriter name='/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/spawn.out'>
E           logfile_read: None
E           logfile_send: None
E           maxread: 2000
E           ignorecase: False
E           searchwindowsize: None
E           delaybeforesend: 0.05
E           delayafterclose: 0.1
E           delayafterterminate: 0.1
E           searcher: searcher_re:
E               0: re.compile("b'(Pdb)'")

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:70: TIMEOUT
============================ 1 failed, 152 passed, 9 skipped in 33.68 seconds =============================
ERROR: InvocationError: '/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/pytest -rfsxX test_pdb.py test_terminal.py test_unittest.py'
_________________________________________________ summary _________________________________________________
ERROR:   py35-pexpect: commands failed
@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

Okay, now in the issue-985 branch, there's an additional test demonstrating the failure in another mode where doctests being present and run affect the PDB prompt, even when the doctests pass.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

All tests, including the additional one added above, pass on Python 2.7 with python -m tox -e py27-pexpect, so the issue seems to be isolated to Python 3.

@jaraco jaraco changed the title pdb prompt unusable if doctests ran first (macOS) pdb prompt unusable if doctests ran first (macOS and Python 3) Oct 21, 2016

@jaraco jaraco removed their assignment Oct 21, 2016

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Oct 21, 2016

I'm going to break on this for now, but I welcome further contribution.

@jaraco

This comment has been minimized.

Copy link
Contributor Author

jaraco commented Jan 2, 2018

This issue persists for me. I repeatedly find myself editing the pytest.ini and commenting out the --doctest-modules from the addopts directive to work around the issue (and trying to remember to restore it before committing code).

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 12, 2018

I'm experiencing this too, I'm looking into it.
This looks like a promising direction: https://stackoverflow.com/questions/2882885/pdb-is-not-working-in-django-doctests

Two libraries both playing around with sys.stdout and pdb.set_trace behind the scenes => recipe for disaster

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Jan 12, 2018

@brianmaissy can you try running with -s? This disables pytest's capture of sys.stdout.

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 12, 2018

Yeah I'll try that

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 14, 2018

Yeah that solves it!

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 14, 2018

I'm having trouble following the exact flow, but my best guess is that this is what's happening:

  1. When we run the doctest itself, it switches out the stdout, and restores it in the finally after __run. It also patches pdb.set_trace to restore it when set_trace is called using the _OutputRedirectingPdb object.
  2. Since we have output capturing on at the time we run the doctest, it copies aside the wrong stdout. It restores what it thinks is the real stdout, but actually it turns output capturing back on after we had turned it off ourselves.

I found a fix that works, but I'm far from confident that it's the correct fix. If we add the line:
self.config.pluginmanager.getplugin("capturemanager").suspend_global_capture(in_=True) in DoctestItem.runtest, before the call to self.runner.run (doctest.py:106), we get our prompt.

Sample implementation:
https://github.com/brianmaissy/pytest/blob/0e88c831057272196851e861c751166edde74af5/_pytest/doctest.py#L106-L111

And maybe we need to turn it back on afterwards?

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 25, 2018

@nicoddemus what do you think?

Alternatively, if this isn't worth the time (and only happens on mac for some reason) we can just call it a known issue and xfail the test on mac

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Jan 27, 2018

Alternatively, if this isn't worth the time (and only happens on mac for some reason) we can just call it a known issue and xfail the test on mac

@brianmaissy I think this is the way to go, at least until someone can dedicate some time to nail this down.

@brianmaissy

This comment has been minimized.

Copy link
Contributor

brianmaissy commented Jan 27, 2018

k, ill open a PR

jaraco added a commit that referenced this issue Feb 13, 2018

@jaraco jaraco referenced this issue Feb 13, 2018

Merged

Disable output capturing in doctest on macOS #3215

5 of 5 tasks complete

jaraco added a commit that referenced this issue Feb 13, 2018

jaraco added a commit that referenced this issue Feb 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.