Merge 'origin/master' into 'setwinsize_on_spawn'

especially careful in pexpect/__init__.py, the definition
of the 'run' vs. '_run' function has changed; the phrase
'dimension' is removed entirely but functional: it should
be allowed through the **kwargs pass-through.

README.rst

@@ -32,7 +32,7 @@ You can install Pexpect using pip::
 
 `Docs on ReadTheDocs <http://pexpect.readthedocs.org/>`_
 
-PEXPECT LICENSE
+PEXPECT LICENSE::
 
     http://opensource.org/licenses/isc-license.txt
 

doc/api/index.rst

@@ -8,5 +8,10 @@ API documentation
    fdpexpect
    replwrap
    pxssh
-   screen
-   ANSI
+
+The modules ``pexpect.screen`` and ``pexpect.ANSI`` have been deprecated in
+Pexpect version 4. They were separate from the main use cases for Pexpect, and
+there are better maintained Python terminal emulator packages, such as
+`pyte <https://pypi.python.org/pypi/pyte>`__.
+These modules are still present for now, but we don't advise using them in new
+code.

doc/api/pexpect.rst

@@ -36,9 +36,9 @@ spawn class
 
       .. note::
 
-         With a :class:`spawn` instance, the log files should be open for
-         writing binary data. With a :class:`spawnu` instance, they should
-         be open for writing unicode text.
+         With :class:`spawn` in bytes mode, the log files should be open for
+         writing binary data. In unicode mode, they should
+         be open for writing unicode text. See :ref:`unicode`.
 
 Controlling the child process
 `````````````````````````````
@@ -69,31 +69,35 @@ Controlling the child process
 Handling unicode
 ````````````````
 
-For backwards compatibility, :class:`spawn` can handle some Unicode: its
-send methods will encode arbitrary unicode as UTF-8 before sending it to the
-child process, and its expect methods can accept ascii-only unicode strings.
-However, for a proper unicode API to a subprocess, use this subclass:
+By default, :class:`spawn` is a bytes interface: its read methods return bytes,
+and its write/send and expect methods expect bytes. If you pass the *encoding*
+parameter to the constructor, it will instead act as a unicode interface:
+strings you send will be encoded using that encoding, and bytes received will
+be decoded before returning them to you. In this mode, patterns for
+:meth:`~spawn.expect` and :meth:`~spawn.expect_exact` should also be unicode.
+
+.. versionchanged:: 4.0
 
-.. autoclass:: spawnu
-   :show-inheritance:
+   :class:`spawn` provides both the bytes and unicode interfaces. In Pexpect
+   3.x, the unicode interface was provided by a separate ``spawnu`` class.
 
-There is also a :func:`runu` function, the unicode counterpart to :func:`run`.
+For backwards compatibility, some Unicode is allowed in bytes mode: the
+send methods will encode arbitrary unicode as UTF-8 before sending it to the
+child process, and its expect methods can accept ascii-only unicode strings.
 
 .. note::
 
    Unicode handling with pexpect works the same way on Python 2 and 3, despite
    the difference in names. I.e.:
 
-   - :class:`spawn` works with ``str`` on Python 2, and :class:`bytes` on Python 3,
-   - :class:`spawnu` works with ``unicode`` on Python 2, and :class:`str` on Python 3.
+   - Bytes mode works with ``str`` on Python 2, and :class:`bytes` on Python 3,
+   - Unicode mode works with ``unicode`` on Python 2, and :class:`str` on Python 3.
 
 run function
 ------------
 
 .. autofunction:: run
 
-.. autofunction:: runu
-
 Exceptions
 ----------
 

doc/history.rst

@@ -12,6 +12,16 @@ Version 4.0
   coroutine. You can get the result using ``yield from``, or wrap it in an
   :class:`asyncio.Task`. This allows the event loop to do other things while
   waiting for output that matches a pattern.
+* Enhancement: allow method as callbacks of argument ``events`` for
+  :func:`pexpect.run` (:ghissue:`176`).
+* It is now possible to call :meth:`~.wait` multiple times, or after a process
+  is already determined to be terminated without raising an exception
+  (:ghpull:`211`).
+* Deprecated ``pexpect.screen`` and ``pexpect.ANSI``. Please use other packages
+  such as `pyte <https://pypi.python.org/pypi/pyte>`__ to emulate a terminal.
+* Removed the independent top-level modules (``pxssh fdpexpect FSM screen ANSI``)
+  which were installed alongside Pexpect. These were moved into the Pexpect
+  package in 3.0, but the old names were left as aliases.
 
 Version 3.4
 ```````````

doc/overview.rst

@@ -60,7 +60,7 @@ Special EOF and TIMEOUT patterns
 --------------------------------
 
 There are two special patterns to match the End Of File (:class:`~pexpect.EOF`)
-or a Timeout condition (:class:`~pexpect.TIMEOUT`). You you can pass these
+or a Timeout condition (:class:`~pexpect.TIMEOUT`). You can pass these
 patterns to :meth:`~pexpect.spawn.expect`. These patterns are not regular
 expressions. Use them like predefined constants.
 
@@ -84,13 +84,13 @@ The following code fragment gives an example of this::
     # We expect any of these three patterns...
     i = child.expect (['Permission denied', 'Terminal type', '[#\$] '])
     if i==0:
-        print('Permission denied on host. Can't login')
+        print('Permission denied on host. Can\'t login')
         child.kill(0)
-    elif i==2:
+    elif i==1:
         print('Login OK... need to send terminal type.')
         child.sendline('vt100')
         child.expect('[#\$] ')
-    elif i==3:
+    elif i==2:
         print('Login OK.')
         print('Shell command prompt', child.after)
 

pexpect/__init__.py

@@ -71,13 +71,13 @@
 from .pty_spawn import spawn, spawnu, PY3
 from .expect import Expecter, searcher_re, searcher_string
 
-__version__ = '3.3'
+__version__ = '4.0.dev'
 __revision__ = ''
 __all__ = ['ExceptionPexpect', 'EOF', 'TIMEOUT', 'spawn', 'spawnu', 'run', 'runu',
            'which', 'split_command_line', '__version__', '__revision__']
 
 def run(command, timeout=30, withexitstatus=False, events=None,
-        extra_args=None, logfile=None, cwd=None, env=None, dimensions=None):
+        extra_args=None, logfile=None, cwd=None, env=None, **kwargs):
 
     '''
     This function runs the given command; waits for it to finish; then
@@ -149,8 +149,8 @@ def print_ticks(d):
 
     Note that you should put newlines in your string if Enter is necessary.
 
-    Like the example above, the responses may also contain callback functions.
-    Any callback is a function that takes a dictionary as an argument.
+    Like the example above, the responses may also contain a callback, either
+    a function or method.  It should accept a dictionary value as an argument.
     The dictionary contains all the locals from the run() function, so you can
     access the child spawn object or any other variable defined in run()
     (event_count, child, and extra_args are the most useful). A callback may
@@ -159,29 +159,16 @@ def print_ticks(d):
     sent to the child. 'extra_args' is not used by directly run(). It provides
     a way to pass data to a callback function through run() through the locals
     dictionary passed to a callback.
-    '''
-    return _run(command, timeout=timeout, withexitstatus=withexitstatus,
-                events=events, extra_args=extra_args, logfile=logfile, cwd=cwd,
-                env=env,dimensions=dimensions, _spawn=spawn)
-
-def runu(command, timeout=30, withexitstatus=False, events=None,
-        extra_args=None, logfile=None, cwd=None, env=None, **kwargs):
-    """This offers the same interface as :func:`run`, but using unicode.
 
-    Like :class:`spawnu`, you can pass ``encoding`` and ``errors`` parameters,
-    which will be used for both input and output.
-    """
-    return _run(command, timeout=timeout, withexitstatus=withexitstatus,
-                events=events, extra_args=extra_args, logfile=logfile, cwd=cwd,
-                env=env, _spawn=spawnu, **kwargs)
-
-def _run(command, timeout, withexitstatus, events, extra_args, logfile, cwd,
-         env, _spawn, **kwargs):
+    Like :class:`spawn`, passing *encoding* will make it work with unicode
+    instead of bytes. You can pass *codec_errors* to control how errors in
+    encoding and decoding are handled.
+    '''
     if timeout == -1:
-        child = _spawn(command, maxread=2000, logfile=logfile, cwd=cwd, env=env,
+        child = spawn(command, maxread=2000, logfile=logfile, cwd=cwd, env=env,
                         **kwargs)
     else:
-        child = _spawn(command, timeout=timeout, maxread=2000, logfile=logfile,
+        child = spawn(command, timeout=timeout, maxread=2000, logfile=logfile,
                 cwd=cwd, env=env, **kwargs)
     if isinstance(events, list):
         patterns= [x for x,y in events]
@@ -206,15 +193,18 @@ def _run(command, timeout, withexitstatus, events, extra_args, logfile, cwd,
                 child_result_list.append(child.before)
             if isinstance(responses[index], child.allowed_string_types):
                 child.send(responses[index])
-            elif isinstance(responses[index], types.FunctionType):
+            elif (isinstance(responses[index], types.FunctionType) or
+                  isinstance(responses[index], types.MethodType)):
                 callback_result = responses[index](locals())
                 sys.stdout.flush()
                 if isinstance(callback_result, child.allowed_string_types):
                     child.send(callback_result)
                 elif callback_result:
                     break
             else:
-                raise TypeError('The callback must be a string or function.')
+                raise TypeError("parameter `event' at index {index} must be "
+                                "a string, method, or function: {value!r}"
+                                .format(index=index, value=responses[index]))
             event_count = event_count + 1
         except TIMEOUT:
             child_result_list.append(child.before)
@@ -229,4 +219,13 @@ def _run(command, timeout, withexitstatus, events, extra_args, logfile, cwd,
     else:
         return child_result
 
+def runu(command, timeout=30, withexitstatus=False, events=None,
+        extra_args=None, logfile=None, cwd=None, env=None, **kwargs):
+    """Deprecated: pass encoding to run() instead.
+    """
+    kwargs.setdefault('encoding', 'utf-8')
+    return run(command, timeout=timeout, withexitstatus=withexitstatus,
+                events=events, extra_args=extra_args, logfile=logfile, cwd=cwd,
+                env=env, **kwargs)
+
 # vim: set shiftround expandtab tabstop=4 shiftwidth=4 ft=python autoindent :

pexpect/async.py

@@ -6,10 +6,11 @@
 @asyncio.coroutine
 def expect_async(expecter, timeout=None):
     # First process data that was previously read - if it maches, we don't need
-    # async stuff.    
-    idx = expecter.new_data(expecter.spawn.buffer)
+    # async stuff.
+    previously_read = expecter.spawn.buffer
     expecter.spawn.buffer = expecter.spawn.string_type()
-    if idx:
+    idx = expecter.new_data(previously_read)
+    if idx is not None:
         return idx
 
     transport, pw = yield from asyncio.get_event_loop()\
@@ -36,7 +37,7 @@ def error(self, exc):
 
     def data_received(self, data):
         spawn = self.expecter.spawn
-        s = spawn._coerce_read_string(data)
+        s = spawn._decoder.decode(data)
         spawn._log(s, 'read')
 
         if self.fut.done():
@@ -56,6 +57,7 @@ def eof_received(self):
         # N.B. If this gets called, async will close the pipe (the spawn object)
         # for us
         try:
+            self.expecter.spawn.flag_eof = True
             index = self.expecter.eof()
         except EOF as e:
             self.error(e)
@@ -67,4 +69,4 @@ def connection_lost(self, exc):
             # We may get here without eof_received being called, e.g on Linux
             self.eof_received()
         elif exc is not None:
-            self.error(exc)
+            self.error(exc)

pexpect/fdpexpect.py

@@ -32,7 +32,8 @@ class fdspawn(SpawnBase):
     descriptor. For example, you could use it to read through a file looking
     for patterns, or to control a modem or serial device. '''
 
-    def __init__ (self, fd, args=None, timeout=30, maxread=2000, searchwindowsize=None, logfile=None):
+    def __init__ (self, fd, args=None, timeout=30, maxread=2000, searchwindowsize=None,
+                  logfile=None, encoding=None, codec_errors='strict'):
         '''This takes a file descriptor (an int) or an object that support the
         fileno() method (returning an int). All Python file-like objects
         support fileno(). '''
@@ -50,7 +51,8 @@ def __init__ (self, fd, args=None, timeout=30, maxread=2000, searchwindowsize=No
 
         self.args = None
         self.command = None
-        SpawnBase.__init__(self, timeout, maxread, searchwindowsize, logfile)
+        SpawnBase.__init__(self, timeout, maxread, searchwindowsize, logfile,
+                           encoding=encoding, codec_errors=codec_errors)
         self.child_fd = fd
         self.own_fd = False
         self.closed = False
@@ -84,3 +86,28 @@ def isalive (self):
 
     def terminate (self, force=False):  # pragma: no cover
         raise ExceptionPexpect('This method is not valid for file descriptors.')
+
+    # These four methods are left around for backwards compatibility, but not
+    # documented as part of fdpexpect. You're encouraged to use os.write#
+    # directly.
+    def send(self, s):
+        "Write to fd, return number of bytes written"
+        s = self._coerce_send_string(s)
+        self._log(s, 'send')
+
+        b = self._encoder.encode(s, final=False)
+        return os.write(self.child_fd, b)
+
+    def sendline(self, s):
+        "Write to fd with trailing newline, return number of bytes written"
+        s = self._coerce_send_string(s)
+        return self.send(s + self.linesep)
+
+    def write(self, s):
+        "Write to fd, return None"
+        self.send(s)
+
+    def writelines(self, sequence):
+        "Call self.write() for each item in sequence"
+        for s in sequence:
+            self.write(s)