Skip to content
This repository

Pexpect-u #969

Merged
merged 6 commits into from over 2 years ago

3 participants

Thomas Kluyver Min RK Fernando Perez
Thomas Kluyver
Collaborator

This probably needs some careful thought, so it might not get in before 0.12.

pexpect doesn't work with Python 3. I've previously made a few small fixes so it works for piping process output to run external commands over ZMQ. It still failed for irunner, though. This brings in a more complete port of pexpect that I've been working on recently, called (for now) pexpect-u. It's compatible with Python 2.6 and up, including 2to3 conversion to Python 3.

I've contacted the author of pexpect about working these changes back in. But this could take a while - pexpect has previously supported very old versions of Python (the docs claim it's been tested on Python 1.5), so I'll have to see how much compromise is possible.

Min RK
Owner

Since this is a significant change, does that mean we should not import vanilla pexpect if it is present, and just use pexpect-u (on python-3 at least)?

Thomas Kluyver
Collaborator

That's worth considering, although at this point approximately no-one is likely to have pexpect-u installed. I'm vaguely hoping to get some changes into pexpect itself, but I can't get in touch with the author. And I'm not sure what compromise he'd be willing to make on dropping support for older versions.

Min RK
Owner

But right now, we prioritize existing pexpect over bundled, so if users already have pexpect, your changes won't be used, even though they are necessary. So it would seem that the logic in external.pexpect.__init__ should change, at least for Python 3.

Thomas Kluyver
Collaborator

It's only necessary on Python 3, and vanilla pexpect isn't compatible with Python 3, so it's sort of the user's problem if they've managed to install it there. On Python 2, the IPython code should work with either pexpect or pexpect-u.

Min RK
Owner

Okay, fair enough.

Fernando Perez
Owner

@takluyver, just noticed this one is in need of a rebase...

Thomas Kluyver
Collaborator

Rebased, and re-run tests on Python 2.7 and 3.2.

To explain these changes a bit further, I've separated the pexpect spawn class into spawn, which is a text interface, and spawnb, which is a bytes interface. For piped subprocesses, I've used spawnb, because our existing architecture expects bytes (and decodes it according to platform default encodings). For irunner, I've used the text interface (spawn), which will by default decode text with UTF-8 (which is a reasonable default on any modern unix-y system).

Fernando Perez fperez merged commit 668e8a0 into from November 26, 2011
Fernando Perez fperez closed this November 26, 2011
Fernando Perez
Owner

Merged after final review on IRC with @takluyver. Great work, as usual :) Let's hope the pexpect author is responsive to these changes.

Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
259  IPython/external/pexpect/_pexpect.py
@@ -9,17 +9,19 @@
9 9
 the standard Python pty module. The Pexpect interface focuses on ease of use so
10 10
 that simple tasks are easy.
11 11
 
12  
-There are two main interfaces to Pexpect -- the function, run() and the class,
13  
-spawn. You can call the run() function to execute a command and return the
  12
+There are two main interfaces to the Pexpect system; these are the function,
  13
+run() and the class, spawn. The spawn class is more powerful. The run()
  14
+function is simpler than spawn, and is good for quickly calling program. When
  15
+you call the run() function it executes a given program and then returns the
14 16
 output. This is a handy replacement for os.system().
15 17
 
16 18
 For example::
17 19
 
18 20
     pexpect.run('ls -la')
19 21
 
20  
-The more powerful interface is the spawn class. You can use this to spawn an
21  
-external child command and then interact with the child by sending lines and
22  
-expecting responses.
  22
+The spawn class is the more powerful interface to the Pexpect system. You can
  23
+use this to spawn a child program then interact with it by sending input and
  24
+expecting responses (waiting for patterns in the child's output).
23 25
 
24 26
 For example::
25 27
 
@@ -28,16 +30,18 @@
28 30
     child.sendline (mypassword)
29 31
 
30 32
 This works even for commands that ask for passwords or other input outside of
31  
-the normal stdio streams.
  33
+the normal stdio streams. For example, ssh reads input directly from the TTY
  34
+device which bypasses stdin.
32 35
 
33 36
 Credits: Noah Spurrier, Richard Holden, Marco Molteni, Kimberley Burchett,
34 37
 Robert Stone, Hartmut Goebel, Chad Schroeder, Erick Tryzelaar, Dave Kirby, Ids
35 38
 vander Molen, George Todd, Noel Taylor, Nicolas D. Cesar, Alexander Gattin,
36  
-Geoffrey Marshall, Francisco Lourenco, Glen Mabey, Karthik Gurusamy, Fernando
37  
-Perez, Corey Minyard, Jon Cohen, Guillaume Chazarain, Andrew Ryan, Nick
38  
-Craig-Wood, Andrew Stone, Jorgen Grahn (Let me know if I forgot anyone.)
  39
+Jacques-Etienne Baudoux, Geoffrey Marshall, Francisco Lourenco, Glen Mabey,
  40
+Karthik Gurusamy, Fernando Perez, Corey Minyard, Jon Cohen, Guillaume
  41
+Chazarain, Andrew Ryan, Nick Craig-Wood, Andrew Stone, Jorgen Grahn, John
  42
+Spiegel, Jan Grant, Shane Kerr and Thomas Kluyver. Let me know if I forgot anyone.
39 43
 
40  
-Free, open source, and all that good stuff.
  44
+Pexpect is free, open source, and all that good stuff.
41 45
 
42 46
 Permission is hereby granted, free of charge, to any person obtaining a copy of
43 47
 this software and associated documentation files (the "Software"), to deal in
@@ -57,10 +61,8 @@
57 61
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
58 62
 SOFTWARE.
59 63
 
60  
-Pexpect Copyright (c) 2008 Noah Spurrier
  64
+Pexpect Copyright (c) 2008-2011 Noah Spurrier
61 65
 http://pexpect.sourceforge.net/
62  
-
63  
-$Id: pexpect.py 507 2007-12-27 02:40:52Z noah $
64 66
 """
65 67
 
66 68
 try:
@@ -83,10 +85,11 @@
83 85
 A critical module was not found. Probably this operating system does not
84 86
 support it. Pexpect is intended for UNIX-like operating systems.""")
85 87
 
86  
-__version__ = '2.3'
87  
-__revision__ = '$Revision: 399 $'
88  
-__all__ = ['ExceptionPexpect', 'EOF', 'TIMEOUT', 'spawn', 'run', 'which',
89  
-    'split_command_line', '__version__', '__revision__']
  88
+__version__ = '2.6.dev'
  89
+version = __version__
  90
+version_info = (2,6,'dev')
  91
+__all__ = ['ExceptionPexpect', 'EOF', 'TIMEOUT', 'spawn', 'spawnb', 'run', 'which',
  92
+    'split_command_line', '__version__']
90 93
 
91 94
 # Exception classes used by this module.
92 95
 class ExceptionPexpect(Exception):
@@ -140,7 +143,22 @@ class TIMEOUT(ExceptionPexpect):
140 143
 ##class MAXBUFFER(ExceptionPexpect):
141 144
 ##    """Raised when a scan buffer fills before matching an expected pattern."""
142 145
 
143  
-def run (command, timeout=-1, withexitstatus=False, events=None, extra_args=None, logfile=None, cwd=None, env=None):
  146
+PY3 = (sys.version_info[0] >= 3)
  147
+
  148
+def _cast_bytes(s, enc):
  149
+    if isinstance(s, unicode):
  150
+        return s.encode(enc)
  151
+    return s
  152
+
  153
+def _cast_unicode(s, enc):
  154
+    if isinstance(s, bytes):
  155
+        return s.decode(enc)
  156
+    return s
  157
+
  158
+re_type = type(re.compile(''))
  159
+
  160
+def run (command, timeout=-1, withexitstatus=False, events=None, extra_args=None,
  161
+         logfile=None, cwd=None, env=None, encoding='utf-8'):
144 162
 
145 163
     """
146 164
     This function runs the given command; waits for it to finish; then
@@ -212,12 +230,14 @@ def print_ticks(d):
212 230
     the next event. A callback may also return a string which will be sent to
213 231
     the child. 'extra_args' is not used by directly run(). It provides a way to
214 232
     pass data to a callback function through run() through the locals
215  
-    dictionary passed to a callback. """
  233
+    dictionary passed to a callback."""
216 234
 
217 235
     if timeout == -1:
218  
-        child = spawn(command, maxread=2000, logfile=logfile, cwd=cwd, env=env)
  236
+        child = spawn(command, maxread=2000, logfile=logfile, cwd=cwd, env=env,
  237
+                      encoding=encoding)
219 238
     else:
220  
-        child = spawn(command, timeout=timeout, maxread=2000, logfile=logfile, cwd=cwd, env=env)
  239
+        child = spawn(command, timeout=timeout, maxread=2000, logfile=logfile,
  240
+                      cwd=cwd, env=env, encoding=encoding)
221 241
     if events is not None:
222 242
         patterns = events.keys()
223 243
         responses = events.values()
@@ -235,7 +255,7 @@ def print_ticks(d):
235 255
                 child_result_list.append(child.before)
236 256
             if isinstance(responses[index], basestring):
237 257
                 child.send(responses[index])
238  
-            elif isinstance(responses[index], types.FunctionType):
  258
+            elif type(responses[index]) is types.FunctionType:
239 259
                 callback_result = responses[index](locals())
240 260
                 sys.stdout.flush()
241 261
                 if isinstance(callback_result, basestring):
@@ -251,19 +271,28 @@ def print_ticks(d):
251 271
         except EOF, e:
252 272
             child_result_list.append(child.before)
253 273
             break
254  
-    child_result = ''.join(child_result_list)
  274
+    child_result = child._empty_buffer.join(child_result_list)
255 275
     if withexitstatus:
256 276
         child.close()
257 277
         return (child_result, child.exitstatus)
258 278
     else:
259 279
         return child_result
260 280
 
261  
-class spawn (object):
262  
-
263  
-    """This is the main class interface for Pexpect. Use this class to start
264  
-    and control child applications. """
265  
-
266  
-    def __init__(self, command, args=[], timeout=30, maxread=2000, searchwindowsize=None, logfile=None, cwd=None, env=None):
  281
+class spawnb(object):
  282
+    """Use this class to start and control child applications with a pure-bytes
  283
+    interface."""
  284
+    
  285
+    _buffer_type = bytes
  286
+    def _cast_buffer_type(self, s):
  287
+        return _cast_bytes(s, self.encoding)
  288
+    _empty_buffer = b''
  289
+    _pty_newline = b'\r\n'
  290
+    
  291
+    # Some code needs this to exist, but it's mainly for the spawn subclass.
  292
+    encoding = 'utf-8'
  293
+
  294
+    def __init__(self, command, args=[], timeout=30, maxread=2000, searchwindowsize=None,
  295
+                 logfile=None, cwd=None, env=None):
267 296
 
268 297
         """This is the constructor. The command parameter may be a string that
269 298
         includes a command and any arguments to the command. For example::
@@ -399,7 +428,7 @@ def __init__(self, command, args=[], timeout=30, maxread=2000, searchwindowsize=
399 428
         self.logfile_read = None # input from child (read_nonblocking)
400 429
         self.logfile_send = None # output to send (send, sendline)
401 430
         self.maxread = maxread # max bytes to read at one time into buffer
402  
-        self.buffer = b'' # This is the read buffer. See maxread.
  431
+        self.buffer = self._empty_buffer # This is the read buffer. See maxread.
403 432
         self.searchwindowsize = searchwindowsize # Anything before searchwindowsize point is preserved, but not searched.
404 433
         # Most Linux machines don't like delaybeforesend to be below 0.03 (30 ms).
405 434
         self.delaybeforesend = 0.05 # Sets sleep time used just before sending data to child. Time in seconds.
@@ -407,13 +436,12 @@ def __init__(self, command, args=[], timeout=30, maxread=2000, searchwindowsize=
407 436
         self.delayafterterminate = 0.1 # Sets delay in terminate() method to allow kernel time to update process status. Time in seconds.
408 437
         self.softspace = False # File-like object.
409 438
         self.name = '<' + repr(self) + '>' # File-like object.
410  
-        self.encoding = None # File-like object.
411 439
         self.closed = True # File-like object.
412 440
         self.cwd = cwd
413 441
         self.env = env
414 442
         self.__irix_hack = (sys.platform.lower().find('irix')>=0) # This flags if we are running on irix
415 443
         # Solaris uses internal __fork_pty(). All others use pty.fork().
416  
-        if (sys.platform.lower().find('solaris')>=0) or (sys.platform.lower().find('sunos5')>=0):
  444
+        if 'solaris' in sys.platform.lower() or 'sunos5' in sys.platform.lower():
417 445
             self.use_native_pty_fork = False
418 446
         else:
419 447
             self.use_native_pty_fork = True
@@ -442,7 +470,7 @@ def __del__(self):
442 470
             # -- Fernando Perez
443 471
             try:
444 472
                 self.close()
445  
-            except AttributeError:
  473
+            except:
446 474
                 pass
447 475
 
448 476
     def __str__(self):
@@ -452,7 +480,7 @@ def __str__(self):
452 480
 
453 481
         s = []
454 482
         s.append(repr(self))
455  
-        s.append('version: ' + __version__ + ' (' + __revision__ + ')')
  483
+        s.append('version: ' + __version__)
456 484
         s.append('command: ' + str(self.command))
457 485
         s.append('args: ' + str(self.args))
458 486
         s.append('searcher: ' + str(self.searcher))
@@ -610,19 +638,24 @@ def __pty_make_controlling_tty(self, tty_fd):
610 638
 
611 639
         child_name = os.ttyname(tty_fd)
612 640
 
613  
-        # Disconnect from controlling tty if still connected.
614  
-        fd = os.open("/dev/tty", os.O_RDWR | os.O_NOCTTY);
615  
-        if fd >= 0:
616  
-            os.close(fd)
  641
+        # Disconnect from controlling tty. Harmless if not already connected.
  642
+        try:
  643
+            fd = os.open("/dev/tty", os.O_RDWR | os.O_NOCTTY);
  644
+            if fd >= 0:
  645
+                os.close(fd)
  646
+        except:
  647
+            # Already disconnected. This happens if running inside cron.
  648
+            pass
617 649
 
618 650
         os.setsid()
619 651
 
620 652
         # Verify we are disconnected from controlling tty
  653
+        # by attempting to open it again.
621 654
         try:
622 655
             fd = os.open("/dev/tty", os.O_RDWR | os.O_NOCTTY);
623 656
             if fd >= 0:
624 657
                 os.close(fd)
625  
-                raise ExceptionPexpect, "Error! We are not disconnected from a controlling tty."
  658
+                raise ExceptionPexpect, "Error! Failed to disconnect from controlling tty. It is still possible to open /dev/tty."
626 659
         except:
627 660
             # Good! We are disconnected from a controlling tty.
628 661
             pass
@@ -695,9 +728,8 @@ def waitnoecho (self, timeout=-1):
695 728
             p.waitnoecho()
696 729
             p.sendline(mypassword)
697 730
 
698  
-        If timeout is None then this method to block forever until ECHO flag is
699  
-        False.
700  
-
  731
+        If timeout==-1 then this method will use the value in self.timeout.
  732
+        If timeout==None then this method to block until ECHO flag is False.
701 733
         """
702 734
 
703 735
         if timeout == -1:
@@ -767,7 +799,7 @@ def setecho (self, state):
767 799
 
768 800
     def read_nonblocking (self, size = 1, timeout = -1):
769 801
 
770  
-        """This reads at most size characters from the child application. It
  802
+        """This reads at most size bytes from the child application. It
771 803
         includes a timeout. If the read does not complete within the timeout
772 804
         period then a TIMEOUT exception is raised. If the end of file is read
773 805
         then an EOF exception will be raised. If a log file was set using
@@ -832,19 +864,19 @@ def read_nonblocking (self, size = 1, timeout = -1):
832 864
                 self.flag_eof = True
833 865
                 raise EOF ('End Of File (EOF) in read_nonblocking(). Empty string style platform.')
834 866
 
  867
+            s2 = self._cast_buffer_type(s)
835 868
             if self.logfile is not None:
836  
-                self.logfile.write (s)
  869
+                self.logfile.write(s2)
837 870
                 self.logfile.flush()
838 871
             if self.logfile_read is not None:
839  
-                self.logfile_read.write (s)
  872
+                self.logfile_read.write(s2)
840 873
                 self.logfile_read.flush()
841 874
 
842 875
             return s
843 876
 
844 877
         raise ExceptionPexpect ('Reached an unexpected state in read_nonblocking().')
845 878
 
846  
-    def read (self, size = -1):   # File-like object.
847  
-
  879
+    def read (self, size = -1):         # File-like object.
848 880
         """This reads at most "size" bytes from the file (less if the read hits
849 881
         EOF before obtaining size bytes). If the size argument is negative or
850 882
         omitted, read all data until EOF is reached. The bytes are returned as
@@ -852,7 +884,7 @@ def read (self, size = -1):   # File-like object.
852 884
         immediately. """
853 885
 
854 886
         if size == 0:
855  
-            return ''
  887
+            return self._empty_buffer
856 888
         if size < 0:
857 889
             self.expect (self.delimiter) # delimiter default is EOF
858 890
             return self.before
@@ -864,14 +896,17 @@ def read (self, size = -1):   # File-like object.
864 896
         # worry about if I have to later modify read() or expect().
865 897
         # Note, it's OK if size==-1 in the regex. That just means it
866 898
         # will never match anything in which case we stop only on EOF.
867  
-        cre = re.compile('.{%d}' % size, re.DOTALL)
  899
+        if self._buffer_type is bytes:
  900
+            pat = (u'.{%d}' % size).encode('ascii')
  901
+        else:
  902
+            pat = u'.{%d}' % size
  903
+        cre = re.compile(pat, re.DOTALL)
868 904
         index = self.expect ([cre, self.delimiter]) # delimiter default is EOF
869 905
         if index == 0:
870 906
             return self.after ### self.before should be ''. Should I assert this?
871 907
         return self.before
872 908
 
873  
-    def readline (self, size = -1):    # File-like object.
874  
-
  909
+    def readline(self, size = -1):
875 910
         """This reads and returns one entire line. A trailing newline is kept
876 911
         in the string, but may be absent when a file ends with an incomplete
877 912
         line. Note: This readline() looks for a \\r\\n pair even on UNIX
@@ -882,12 +917,11 @@ def readline (self, size = -1):    # File-like object.
882 917
         object. If size is 0 then an empty string is returned. """
883 918
 
884 919
         if size == 0:
885  
-            return ''
886  
-        index = self.expect (['\r\n', self.delimiter]) # delimiter default is EOF
  920
+            return self._empty_buffer
  921
+        index = self.expect ([self._pty_newline, self.delimiter]) # delimiter default is EOF
887 922
         if index == 0:
888  
-            return self.before + '\r\n'
889  
-        else:
890  
-            return self.before
  923
+            return self.before + self._pty_newline
  924
+        return self.before
891 925
 
892 926
     def __iter__ (self):    # File-like object.
893 927
 
@@ -902,7 +936,7 @@ def next (self):    # File-like object.
902 936
         """
903 937
 
904 938
         result = self.readline()
905  
-        if result == "":
  939
+        if result == self._empty_buffer:
906 940
             raise StopIteration
907 941
         return result
908 942
 
@@ -936,22 +970,22 @@ def writelines (self, sequence):   # File-like object.
936 970
         for s in sequence:
937 971
             self.write (s)
938 972
 
939  
-    def send(self, s, encoding='utf-8'):
  973
+    def send(self, s):
940 974
 
941 975
         """This sends a string to the child process. This returns the number of
942 976
         bytes written. If a log file was set then the data is also written to
943 977
         the log. """
944 978
 
945  
-        if isinstance(s, unicode):
946  
-            s = s.encode(encoding)
947 979
         time.sleep(self.delaybeforesend)
  980
+        
  981
+        s2 = self._cast_buffer_type(s)
948 982
         if self.logfile is not None:
949  
-            self.logfile.write (s)
  983
+            self.logfile.write(s2)
950 984
             self.logfile.flush()
951 985
         if self.logfile_send is not None:
952  
-            self.logfile_send.write (s)
  986
+            self.logfile_send.write(s2)
953 987
             self.logfile_send.flush()
954  
-        c = os.write(self.child_fd, s)
  988
+        c = os.write (self.child_fd, _cast_bytes(s, self.encoding))
955 989
         return c
956 990
 
957 991
     def sendline(self, s=''):
@@ -959,7 +993,7 @@ def sendline(self, s=''):
959 993
         """This is like send(), but it adds a line feed (os.linesep). This
960 994
         returns the number of bytes written. """
961 995
 
962  
-        n = self.send(s)
  996
+        n = self.send (s)
963 997
         n = n + self.send (os.linesep)
964 998
         return n
965 999
 
@@ -1131,8 +1165,8 @@ def isalive(self):
1131 1165
 
1132 1166
         try:
1133 1167
             pid, status = os.waitpid(self.pid, waitpid_options)
1134  
-        except OSError, e: # No child processes
1135  
-            if e[0] == errno.ECHILD:
  1168
+        except OSError as e: # No child processes
  1169
+            if e.errno == errno.ECHILD:
1136 1170
                 raise ExceptionPexpect ('isalive() encountered condition where "terminated" is 0, but there was no child process. Did someone else call waitpid() on our process?')
1137 1171
             else:
1138 1172
                 raise e
@@ -1218,20 +1252,28 @@ def compile_pattern_list(self, patterns):
1218 1252
             compile_flags = compile_flags | re.IGNORECASE
1219 1253
         compiled_pattern_list = []
1220 1254
         for p in patterns:
1221  
-            if isinstance(p, basestring):
  1255
+            if isinstance(p, (bytes, unicode)):
  1256
+                p = self._cast_buffer_type(p)
1222 1257
                 compiled_pattern_list.append(re.compile(p, compile_flags))
1223 1258
             elif p is EOF:
1224 1259
                 compiled_pattern_list.append(EOF)
1225 1260
             elif p is TIMEOUT:
1226 1261
                 compiled_pattern_list.append(TIMEOUT)
1227  
-            elif type(p) is type(re.compile('')):
  1262
+            elif type(p) is re_type:
  1263
+                p = self._prepare_regex_pattern(p)
1228 1264
                 compiled_pattern_list.append(p)
1229 1265
             else:
1230 1266
                 raise TypeError ('Argument must be one of StringTypes, EOF, TIMEOUT, SRE_Pattern, or a list of those type. %s' % str(type(p)))
1231 1267
 
1232 1268
         return compiled_pattern_list
  1269
+    
  1270
+    def _prepare_regex_pattern(self, p):
  1271
+        "Recompile unicode regexes as bytes regexes. Overridden in subclass."
  1272
+        if isinstance(p.pattern, unicode):
  1273
+            p = re.compile(p.pattern.encode('utf-8'), p.flags &~ re.UNICODE)
  1274
+        return p
1233 1275
 
1234  
-    def expect(self, pattern, timeout = -1, searchwindowsize=None):
  1276
+    def expect(self, pattern, timeout = -1, searchwindowsize=-1):
1235 1277
 
1236 1278
         """This seeks through the stream until a pattern is matched. The
1237 1279
         pattern is overloaded and may take several types. The pattern can be a
@@ -1339,7 +1381,7 @@ def expect_exact(self, pattern_list, timeout = -1, searchwindowsize = -1):
1339 1381
         This method is also useful when you don't want to have to worry about
1340 1382
         escaping regular expression characters that you want to match."""
1341 1383
 
1342  
-        if isinstance(pattern_list, basestring) or pattern_list in (TIMEOUT, EOF):
  1384
+        if isinstance(pattern_list, (bytes, unicode)) or pattern_list in (TIMEOUT, EOF):
1343 1385
             pattern_list = [pattern_list]
1344 1386
         return self.expect_loop(searcher_string(pattern_list), timeout, searchwindowsize)
1345 1387
 
@@ -1383,7 +1425,7 @@ def expect_loop(self, searcher, timeout = -1, searchwindowsize = -1):
1383 1425
                 if timeout is not None:
1384 1426
                     timeout = end_time - time.time()
1385 1427
         except EOF, e:
1386  
-            self.buffer = b''
  1428
+            self.buffer = self._empty_buffer
1387 1429
             self.before = incoming
1388 1430
             self.after = EOF
1389 1431
             index = searcher.eof_index
@@ -1448,7 +1490,7 @@ def setwinsize(self, r, c):
1448 1490
         s = struct.pack('HHHH', r, c, 0, 0)
1449 1491
         fcntl.ioctl(self.fileno(), TIOCSWINSZ, s)
1450 1492
 
1451  
-    def interact(self, escape_character = chr(29), input_filter = None, output_filter = None):
  1493
+    def interact(self, escape_character = b'\x1d', input_filter = None, output_filter = None):
1452 1494
 
1453 1495
         """This gives control of the child process to the interactive user (the
1454 1496
         human at the keyboard). Keystrokes are sent to the child process, and
@@ -1484,9 +1526,10 @@ def sigwinch_passthrough (sig, data):
1484 1526
         """
1485 1527
 
1486 1528
         # Flush the buffer.
1487  
-        self.stdout.write (self.buffer)
  1529
+        if PY3: self.stdout.write(_cast_unicode(self.buffer, self.encoding))
  1530
+        else:   self.stdout.write(self.buffer)
1488 1531
         self.stdout.flush()
1489  
-        self.buffer = b''
  1532
+        self.buffer = self._empty_buffer
1490 1533
         mode = tty.tcgetattr(self.STDIN_FILENO)
1491 1534
         tty.setraw(self.STDIN_FILENO)
1492 1535
         try:
@@ -1499,7 +1542,7 @@ def __interact_writen(self, fd, data):
1499 1542
         """This is used by the interact() method.
1500 1543
         """
1501 1544
 
1502  
-        while data != '' and self.isalive():
  1545
+        while data != b'' and self.isalive():
1503 1546
             n = os.write(fd, data)
1504 1547
             data = data[n:]
1505 1548
 
@@ -1548,8 +1591,8 @@ def __select (self, iwtd, owtd, ewtd, timeout=None):
1548 1591
         while True:
1549 1592
             try:
1550 1593
                 return select.select (iwtd, owtd, ewtd, timeout)
1551  
-            except select.error, e:
1552  
-                if e[0] == errno.EINTR:
  1594
+            except select.error as e:
  1595
+                if e.args[0] == errno.EINTR:
1553 1596
                     # if we loop back we have to subtract the amount of time we already waited.
1554 1597
                     if timeout is not None:
1555 1598
                         timeout = end_time - time.time()
@@ -1558,22 +1601,34 @@ def __select (self, iwtd, owtd, ewtd, timeout=None):
1558 1601
                 else: # something else caused the select.error, so this really is an exception
1559 1602
                     raise
1560 1603
 
1561  
-##############################################################################
1562  
-# The following methods are no longer supported or allowed.
1563  
-
1564  
-    def setmaxread (self, maxread):
1565  
-
1566  
-        """This method is no longer supported or allowed. I don't like getters
1567  
-        and setters without a good reason. """
1568  
-
1569  
-        raise ExceptionPexpect ('This method is no longer supported or allowed. Just assign a value to the maxread member variable.')
1570  
-
1571  
-    def setlog (self, fileobject):
1572  
-
1573  
-        """This method is no longer supported or allowed.
1574  
-        """
1575  
-
1576  
-        raise ExceptionPexpect ('This method is no longer supported or allowed. Just assign a value to the logfile member variable.')
  1604
+class spawn(spawnb):
  1605
+    """This is the main class interface for Pexpect. Use this class to start
  1606
+    and control child applications."""
  1607
+    
  1608
+    _buffer_type = unicode
  1609
+    def _cast_buffer_type(self, s):
  1610
+        return _cast_unicode(s, self.encoding)
  1611
+    _empty_buffer = u''
  1612
+    _pty_newline = u'\r\n'
  1613
+    
  1614
+    def __init__(self, command, args=[], timeout=30, maxread=2000, searchwindowsize=None,
  1615
+                 logfile=None, cwd=None, env=None, encoding='utf-8'):
  1616
+        super(spawn, self).__init__(command, args, timeout=timeout, maxread=maxread,
  1617
+                    searchwindowsize=searchwindowsize, logfile=logfile, cwd=cwd, env=env)
  1618
+        self.encoding = encoding
  1619
+    
  1620
+    def _prepare_regex_pattern(self, p):
  1621
+        "Recompile bytes regexes as unicode regexes."
  1622
+        if isinstance(p.pattern, bytes):
  1623
+            p = re.compile(p.pattern.decode(self.encoding), p.flags)
  1624
+        return p
  1625
+    
  1626
+    def read_nonblocking(self, size=1, timeout=-1):
  1627
+        return super(spawn, self).read_nonblocking(size=size, timeout=timeout)\
  1628
+                                    .decode(self.encoding)
  1629
+    
  1630
+    read_nonblocking.__doc__ = spawnb.read_nonblocking.__doc__
  1631
+        
1577 1632
 
1578 1633
 ##############################################################################
1579 1634
 # End of spawn class
@@ -1582,6 +1637,8 @@ def setlog (self, fileobject):
1582 1637
 class searcher_string (object):
1583 1638
 
1584 1639
     """This is a plain string search helper for the spawn.expect_any() method.
  1640
+    This helper class is for speed. For more powerful regex patterns
  1641
+    see the helper class, searcher_re.
1585 1642
 
1586 1643
     Attributes:
1587 1644
 
@@ -1594,6 +1651,7 @@ class searcher_string (object):
1594 1651
         start - index into the buffer, first byte of match
1595 1652
         end   - index into the buffer, first byte after match
1596 1653
         match - the matching string itself
  1654
+
1597 1655
     """
1598 1656
 
1599 1657
     def __init__(self, strings):
@@ -1604,7 +1662,7 @@ def __init__(self, strings):
1604 1662
         self.eof_index = -1
1605 1663
         self.timeout_index = -1
1606 1664
         self._strings = []
1607  
-        for n, s in zip(range(len(strings)), strings):
  1665
+        for n, s in enumerate(strings):
1608 1666
             if s is EOF:
1609 1667
                 self.eof_index = n
1610 1668
                 continue
@@ -1625,8 +1683,7 @@ def __str__(self):
1625 1683
         if self.timeout_index >= 0:
1626 1684
             ss.append ((self.timeout_index,'    %d: TIMEOUT' % self.timeout_index))
1627 1685
         ss.sort()
1628  
-        ss = zip(*ss)[1]
1629  
-        return '\n'.join(ss)
  1686
+        return '\n'.join(a[1] for a in ss)
1630 1687
 
1631 1688
     def search(self, buffer, freshlen, searchwindowsize=None):
1632 1689
 
@@ -1677,7 +1734,8 @@ def search(self, buffer, freshlen, searchwindowsize=None):
1677 1734
 class searcher_re (object):
1678 1735
 
1679 1736
     """This is regular expression string search helper for the
1680  
-    spawn.expect_any() method.
  1737
+    spawn.expect_any() method. This helper class is for powerful
  1738
+    pattern matching. For speed, see the helper class, searcher_string.
1681 1739
 
1682 1740
     Attributes:
1683 1741
 
@@ -1723,8 +1781,7 @@ def __str__(self):
1723 1781
         if self.timeout_index >= 0:
1724 1782
             ss.append ((self.timeout_index,'    %d: TIMEOUT' % self.timeout_index))
1725 1783
         ss.sort()
1726  
-        ss = [a[1] for a in ss]
1727  
-        return '\n'.join(ss)
  1784
+        return '\n'.join(a[1] for a in ss)
1728 1785
 
1729 1786
     def search(self, buffer, freshlen, searchwindowsize=None):
1730 1787
 
@@ -1777,8 +1834,6 @@ def which (filename):
1777 1834
     else:
1778 1835
         p = os.environ['PATH']
1779 1836
 
1780  
-    # Oddly enough this was the one line that made Pexpect
1781  
-    # incompatible with Python 1.5.2.
1782 1837
     pathlist = p.split(os.pathsep)
1783 1838
 
1784 1839
     for path in pathlist:
@@ -1842,4 +1897,4 @@ def split_command_line(command_line):
1842 1897
         arg_list.append(arg)
1843 1898
     return arg_list
1844 1899
 
1845  
-# vi:ts=4:sw=4:expandtab:ft=python:
  1900
+# vi:set sr et ts=4 sw=4 ft=python :
6  IPython/lib/irunner.py
@@ -38,6 +38,7 @@
38 38
 # Third-party modules: we carry a copy of pexpect to reduce the need for
39 39
 # external dependencies, but our import checks for a system version first.
40 40
 from IPython.external import pexpect
  41
+from IPython.utils import py3compat
41 42
 
42 43
 # Global usage strings, to avoid indentation issues when typing it below.
43 44
 USAGE = """
@@ -287,6 +288,7 @@ def main(self,argv=None):
287 288
 
288 289
         self.run_file(args[0],opts.interact)
289 290
 
  291
+_ipython_cmd = "ipython3" if py3compat.PY3 else "ipython"
290 292
 
291 293
 # Specific runners for particular programs
292 294
 class IPythonRunner(InteractiveRunner):
@@ -302,7 +304,7 @@ class IPythonRunner(InteractiveRunner):
302 304
     prompts would break this.
303 305
     """
304 306
 
305  
-    def __init__(self,program = 'ipython',args=None,out=sys.stdout,echo=True):
  307
+    def __init__(self,program = _ipython_cmd, args=None, out=sys.stdout, echo=True):
306 308
         """New runner, optionally passing the ipython command to use."""
307 309
         args0 = ['--colors=NoColor',
308 310
                  '--no-term-title',
@@ -318,7 +320,7 @@ def __init__(self,program = 'ipython',args=None,out=sys.stdout,echo=True):
318 320
 class PythonRunner(InteractiveRunner):
319 321
     """Interactive Python runner."""
320 322
 
321  
-    def __init__(self,program='python',args=None,out=sys.stdout,echo=True):
  323
+    def __init__(self,program=sys.executable, args=None, out=sys.stdout, echo=True):
322 324
         """New runner, optionally passing the python command to use."""
323 325
 
324 326
         prompts = [r'>>> ',r'\.\.\. ']
40  IPython/lib/tests/test_irunner.py
@@ -14,6 +14,7 @@
14 14
 # IPython imports
15 15
 from IPython.lib import irunner
16 16
 from IPython.testing.decorators import known_failure_py3
  17
+from IPython.utils.py3compat import doctest_refactor_print
17 18
 
18 19
 # Testing code begins
19 20
 class RunnerTestCase(unittest.TestCase):
@@ -57,11 +58,11 @@ def _test_runner(self,runner,source,output):
57 58
         self.assert_(mismatch==0,'Number of mismatched lines: %s' %
58 59
                      mismatch)
59 60
 
60  
-    # irunner isn't working on Python 3 (due to pexpect)
  61
+    # The SyntaxError appears differently in Python 3, for some reason.
61 62
     @known_failure_py3
62 63
     def testIPython(self):
63 64
         """Test the IPython runner."""
64  
-        source = """
  65
+        source = doctest_refactor_print("""
65 66
 print 'hello, this is python'
66 67
 # some more code
67 68
 x=1;y=2
@@ -76,13 +77,13 @@ def testIPython(self):
76 77
 cos(pi)
77 78
 
78 79
 for i in range(5):
79  
-    print i,
  80
+    print i
80 81
 
81 82
 print "that's all folks!"
82 83
 
83 84
 exit
84  
-"""
85  
-        output = """\
  85
+""")
  86
+        output = doctest_refactor_print("""\
86 87
 In [1]: print 'hello, this is python'
87 88
 hello, this is python
88 89
 
@@ -119,24 +120,27 @@ def testIPython(self):
119 120
 
120 121
 
121 122
 In [10]: for i in range(5):
122  
-   ....:     print i,
  123
+   ....:     print i
123 124
    ....:
124  
-0 1 2 3 4
  125
+0
  126
+1
  127
+2
  128
+3
  129
+4
125 130
 
126 131
 In [11]: print "that's all folks!"
127 132
 that's all folks!
128 133
 
129 134
 
130 135
 In [12]: exit
131  
-"""
  136
+""")
132 137
         runner = irunner.IPythonRunner(out=self.out)
133 138
         self._test_runner(runner,source,output)
134 139
 
135  
-    @known_failure_py3
136 140
     def testPython(self):
137 141
         """Test the Python runner."""
138 142
         runner = irunner.PythonRunner(out=self.out)
139  
-        source = """
  143
+        source = doctest_refactor_print("""
140 144
 print 'hello, this is python'
141 145
 
142 146
 # some more code
@@ -147,11 +151,11 @@ def testPython(self):
147 151
 cos(pi)
148 152
 
149 153
 for i in range(5):
150  
-    print i,
  154
+    print i
151 155
 
152 156
 print "that's all folks!"
153  
-        """
154  
-        output = """\
  157
+        """)
  158
+        output = doctest_refactor_print("""\
155 159
 >>> print 'hello, this is python'
156 160
 hello, this is python
157 161
 
@@ -165,10 +169,14 @@ def testPython(self):
165 169
 -1.0
166 170
 
167 171
 >>> for i in range(5):
168  
-...     print i,
  172
+...     print i
169 173
 ...
170  
-0 1 2 3 4
  174
+0
  175
+1
  176
+2
  177
+3
  178
+4
171 179
 >>> print "that's all folks!"
172 180
 that's all folks!
173  
-"""
  181
+""")
174 182
         self._test_runner(runner,source,output)
2  IPython/lib/tests/test_irunner_pylab_magic.py
@@ -22,7 +22,6 @@ def setUp(self):
22 22
         self.out = StringIO.StringIO()
23 23
         #self.out = sys.stdout
24 24
 
25  
-    @decorators.known_failure_py3
26 25
     def _test_runner(self,runner,source,output):
27 26
         """Test that a given runner's input/output match."""
28 27
 
@@ -83,7 +82,6 @@ def test_pylab_import_all_enabled(self):
83 82
         runner = irunner.IPythonRunner(out=self.out)
84 83
         self._test_runner(runner,source,output)
85 84
 
86  
-    @decorators.known_failure_py3
87 85
     @decorators.skipif_not_matplotlib
88 86
     def test_pylab_import_all_disabled(self):
89 87
         "Verify that plot is not available when pylab_import_all = False"
6  IPython/utils/_process_posix.py
@@ -147,7 +147,11 @@ def system(self, cmd):
147 147
             # can set pexpect's search window to be tiny and it won't matter.
148 148
             # We only search for the 'patterns' timeout or EOF, which aren't in
149 149
             # the text itself.
150  
-            child = pexpect.spawn(self.sh, args=['-c', cmd])
  150
+            #child = pexpect.spawn(pcmd, searchwindowsize=1)
  151
+            if hasattr(pexpect, 'spawnb'):
  152
+                child = pexpect.spawnb(self.sh, args=['-c', cmd]) # Pexpect-U
  153
+            else:
  154
+                child = pexpect.spawn(self.sh, args=['-c', cmd])  # Vanilla Pexpect
151 155
             flush = sys.stdout.flush
152 156
             while True:
153 157
                 # res is the index of the pattern that caused the match, so we
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.