Fix encoding errors with stdout capture and echoing #211

Closed
wants to merge 2 commits into
from

Projects

None yet

3 participants

@pfmoore
Contributor
pfmoore commented Mar 3, 2015

No description provided.

@bitprophet
Member

Closing this as it's been merged into master now.

@bitprophet bitprophet closed this Mar 4, 2015
@pfmoore pfmoore deleted the pfmoore:xplatform-friendly-run-201 branch Mar 5, 2015
@frol

Is there a specific reason for use_setlocale=False in locale.getpreferredencoding()? It is the commit that first introduced this thing. The reason why I'm asking is the following:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

$ python2
Python 2.7.9 |Continuum Analytics, Inc.| (default, Apr 14 2015, 12:54:25)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import locale
>>> locale.getpreferredencoding(False)
'ANSI_X3.4-1968'
>>> locale.getpreferredencoding(True)
'UTF-8'
>>> locale.getpreferredencoding()
'UTF-8'

$ python3
Python 3.4.3 (default, Mar 25 2015, 17:13:50)
[GCC 4.9.2 20150304 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getpreferredencoding(False)
'UTF-8'
>>> locale.getpreferredencoding(True)
'UTF-8'
>>> locale.getpreferredencoding()
'UTF-8'

As you can see, Python 2 weirdly reports ANSI (ASCII) instead of UTF-8 when use_setlocale is False. This causes unicode characters to be replaced with ?? on, for example, run('echo "Я"')

@bitprophet bitprophet referenced this pull request May 27, 2016
Closed

Catchall 'encoding needs to get un-crap-ified' ticket #350

10 of 11 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment