pexpect & Python 3 #798

Merged
merged 3 commits into from Sep 29, 2011

3 participants

@takluyver
IPython member

Changes to pexpect so running external processes from the Qt console works in Python 3.

@fperez
IPython member

These look fine to merge, but I'd suggest you split it into two separate commits. Since one entails changes to an external dependency, I think it's best to always have those available in nice, isolated commits. It makes it easier to send the patch upstream, reapply it if we update to a new version from upstream that doesn't have the changes, etc.

So my vote is: rebase by splitting into two commits, one for each file, then merge away.

Thanks!

takluyver added some commits Sep 17, 2011
@takluyver takluyver Changes to pexpect so it does what we need after conversion to Python 3. a69359f
@takluyver takluyver Decode output from unix subprocesses using platform default encoding. 2cda1d8
@takluyver takluyver Try locale encoding if stdin encoding is ascii.
Starting the Qt console on Python 3, the kernel's stdin ends up with a .encoding of 'ascii' (whereas on Python 2 it is None). Since most platforms can handle a superset of ASCII, we may as well try locale.getpreferredencoding() in this case.
c06689d
@takluyver
IPython member

OK, I've split them, but I found another minor bug. The kernel in Python 2 has a sys.stdin.encoding of None, so @minrk's getdefaultencoding() function fell back to using the locale. In Python 3, sys.stdin.encoding is 'ascii', although the system is using UTF-8.

On the principle that most systems now can handle a superset of ascii (either utf-8 or a Windows code page), I've made getdefaultencoding() try the locale encoding if sys.stdin.encoding is None or ascii. I think the only way this could make anything worse is if a subprocess is returning ascii output on a system where the locale encoding is not ascii compatible (e.g. UTF-16).

@takluyver
IPython member

I'll merge this soon unless anyone objects.

@minrk
IPython member

seems fine to me.

Do you know why Python3 incorrectly marks sys.stdin.encoding as ascii? Or is that the new replacement for None? The problem is that if the terminal really is ASCII, we might run into problems. But I wouldn't worry too much about that.

@takluyver
IPython member

I think that the object which is now used for sys.stdin (io.TextIOWrapper) has to have a non-None encoding attribute, so ascii is the default if it's not told anything else.

I don't think it should cause problems, because:

  1. If the terminal is ASCII, the default locale should presumably indicate ascii as well.
  2. If the terminal is ASCII and the default locale indicates another encoding, it will most likely be either UTF-8 or an ascii compatible code page. So any characters we get from the terminal should be correctly decoded anyway. The only situation in which it could is if the locale incorrectly indicates a non-ascii-compatible encoding, such as UTF-16.
@minrk
IPython member

Sounds good.

@takluyver takluyver merged commit 5590396 into ipython:master Sep 29, 2011
@takluyver
IPython member

Thanks, Min. Merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment