Restore pexpect tests on macOS. Fixes #2022.

[jaraco]

This series of patches makes pexpect tests viable on macOS. After applying the patches, here's the output of a test run (apparently confirming the report in #985).

$ python -m tox -e py35-pexpect                                         
GLOB sdist-make: /Users/jaraco/Dropbox/code/public/pytest/setup.py
py35-pexpect create: /Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect
py35-pexpect installdeps: pexpect
py35-pexpect inst: /Users/jaraco/Dropbox/code/public/pytest/.tox/dist/pytest-3.0.4.dev0.zip
py35-pexpect installed: pexpect==4.2.1,ptyprocess==0.5.1,py==1.4.31,pytest==3.0.4.dev0
py35-pexpect runtests: PYTHONHASHSEED='369771295'
py35-pexpect runtests: commands[0] | pytest -rfsxX test_pdb.py test_terminal.py test_unittest.py
=========================================== test session starts ===========================================
platform darwin -- Python 3.5.2, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/public/pytest, inifile: tox.ini
collected 162 items 

test_pdb.py ..............F......
test_terminal.py ....................................s.............................................................
test_unittest.py ...................ssssssss................
========================================= short test summary info =========================================
FAIL test_pdb.py::TestPDB::()::test_pdb_interaction_doctest
SKIP [4] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:367: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:353: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:314: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:340: could not import 'twisted.trial.unittest'
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_terminal.py:483: xdist plugin not installed
SKIP [1] /Users/jaraco/Dropbox/code/public/pytest/testing/test_unittest.py:327: could not import 'twisted.trial.unittest'

================================================ FAILURES =================================================
__________________________________ TestPDB.test_pdb_interaction_doctest ___________________________________

self = <pexpect.expect.Expecter object at 0x1037f55f8>, timeout = 9.601989030838013

    def expect_loop(self, timeout=-1):
        """Blocking expect"""
        spawn = self.spawn
        from . import EOF, TIMEOUT

        if timeout is not None:
            end_time = time.time() + timeout

        try:
            incoming = spawn.buffer
            spawn.buffer = spawn.string_type()  # Treat buffer as new data
            while True:
                idx = self.new_data(incoming)
                # Keep reading until exception or return.
                if idx is not None:
                    return idx
                # No match at this point
                if (timeout is not None) and (timeout < 0):
                    return self.timeout()
                # Still have time left, so read more data
>               incoming = spawn.read_nonblocking(spawn.maxread, timeout)

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:99: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pexpect.pty_spawn.spawn object at 0x1037f5ac8>, size = 2000, timeout = 9.601989030838013

    def read_nonblocking(self, size=1, timeout=-1):
        '''This reads at most size characters from the child application. It
            includes a timeout. If the read does not complete within the timeout
            period then a TIMEOUT exception is raised. If the end of file is read
            then an EOF exception will be raised.  If a logfile is specified, a
            copy is written to that log.

            If timeout is None then the read may block indefinitely.
            If timeout is -1 then the self.timeout value is used. If timeout is 0
            then the child is polled and if there is no data immediately ready
            then this will raise a TIMEOUT exception.

            The timeout refers only to the amount of time to read at least one
            character. This is not affected by the 'size' parameter, so if you call
            read_nonblocking(size=100, timeout=30) and only one character is
            available right away then one character will be returned immediately.
            It will not wait for 30 seconds for another 99 characters to come in.

            This is a wrapper around os.read(). It uses select.select() to
            implement the timeout. '''

        if self.closed:
            raise ValueError('I/O operation on closed file.')

        if timeout == -1:
            timeout = self.timeout

        # Note that some systems such as Solaris do not give an EOF when
        # the child dies. In fact, you can still try to read
        # from the child_fd -- it will block forever or until TIMEOUT.
        # For this case, I test isalive() before doing any reading.
        # If isalive() is false, then I pretend that this is the same as EOF.
        if not self.isalive():
            # timeout of 0 means "poll"
            r, w, e = select_ignore_interrupts([self.child_fd], [], [], 0)
            if not r:
                self.flag_eof = True
                raise EOF('End Of File (EOF). Braindead platform.')
        elif self.__irix_hack:
            # Irix takes a long time before it realizes a child was terminated.
            # FIXME So does this mean Irix systems are forced to always have
            # FIXME a 2 second delay when calling read_nonblocking? That sucks.
            r, w, e = select_ignore_interrupts([self.child_fd], [], [], 2)
            if not r and not self.isalive():
                self.flag_eof = True
                raise EOF('End Of File (EOF). Slow platform.')

        r, w, e = select_ignore_interrupts([self.child_fd], [], [], timeout)

        if not r:
            if not self.isalive():
                # Some platforms, such as Irix, will claim that their
                # processes are alive; timeout on the select; and
                # then finally admit that they are not alive.
                self.flag_eof = True
                raise EOF('End of File (EOF). Very slow platform.')
            else:
>               raise TIMEOUT('Timeout exceeded.')
E               pexpect.exceptions.TIMEOUT: Timeout exceeded.

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/pty_spawn.py:462: TIMEOUT

During handling of the above exception, another exception occurred:

self = <test_pdb.TestPDB object at 0x10390ea90>
testdir = <Testdir local('/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0')>

    def test_pdb_interaction_doctest(self, testdir):
        p1 = testdir.makepyfile("""
                import pytest
                def function_1():
                    '''
                    >>> i = 0
                    >>> assert i == 1
                    '''
            """)
        child = testdir.spawn_pytest("--doctest-modules --pdb %s" % p1)
>       child.expect("(Pdb)")

/Users/jaraco/Dropbox/code/public/pytest/testing/test_pdb.py:243: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/spawnbase.py:321: in expect
    timeout, searchwindowsize, async)
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/spawnbase.py:345: in expect_list
    return exp.expect_loop(timeout)
/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:107: in expect_loop
    return self.timeout(e)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pexpect.expect.Expecter object at 0x1037f55f8>, err = TIMEOUT('Timeout exceeded.',)

    def timeout(self, err=None):
        spawn = self.spawn
        from . import TIMEOUT

        spawn.before = spawn.buffer
        spawn.after = TIMEOUT
        index = self.searcher.timeout_index
        if index >= 0:
            spawn.match = TIMEOUT
            spawn.match_index = index
            return index
        else:
            spawn.match = None
            spawn.match_index = None
            msg = str(spawn)
            msg += '\nsearcher: %s' % self.searcher
            if err is not None:
                msg = str(err) + '\n' + msg
>           raise TIMEOUT(msg)
E           pexpect.exceptions.TIMEOUT: Timeout exceeded.
E           <pexpect.pty_spawn.spawn object at 0x1037f5ac8>
E           command: /Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/python3.5
E           args: ['/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/python3.5', '/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pytest.py', '--basetemp=/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/pexpect', '--doctest-modules', '--pdb', '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/test_pdb_interaction_doctest.py']
E           buffer (last 100 chars): b'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> <doctest test_pdb_interaction_doctest.function_1[1]>(1)<module>()\r\n'
E           before (last 100 chars): b'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\r\n> <doctest test_pdb_interaction_doctest.function_1[1]>(1)<module>()\r\n'
E           after: <class 'pexpect.exceptions.TIMEOUT'>
E           match: None
E           match_index: None
E           exitstatus: None
E           flag_eof: False
E           pid: 34733
E           child_fd: 13
E           closed: False
E           timeout: 10.0
E           delimiter: <class 'pexpect.exceptions.EOF'>
E           logfile: <_io.BufferedWriter name='/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-53/testdir/test_pdb_interaction_doctest0/spawn.out'>
E           logfile_read: None
E           logfile_send: None
E           maxread: 2000
E           ignorecase: False
E           searchwindowsize: None
E           delaybeforesend: 0.05
E           delayafterclose: 0.1
E           delayafterterminate: 0.1
E           searcher: searcher_re:
E               0: re.compile("b'(Pdb)'")

/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/lib/python3.5/site-packages/pexpect/expect.py:70: TIMEOUT
============================ 1 failed, 152 passed, 9 skipped in 33.68 seconds =============================
ERROR: InvocationError: '/Users/jaraco/Dropbox/code/public/pytest/.tox/py35-pexpect/bin/pytest -rfsxX test_pdb.py test_terminal.py test_unittest.py'
_________________________________________________ summary _________________________________________________
ERROR:   py35-pexpect: commands failed

[RonnyPfannschmidt]

this looks like an overdue correction of quite seriously broken tests

i am most pleased, thanks for the hard work

[nicoddemus]

Thanks a lot @jaraco!