New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix encoding errors with stdout capture and echoing #211

Closed
wants to merge 2 commits into
base: xplatform-friendly-run-201
from

Conversation

Projects
None yet
3 participants
@pfmoore
Contributor

pfmoore commented Mar 3, 2015

No description provided.

@pfmoore pfmoore force-pushed the pfmoore:xplatform-friendly-run-201 branch from 5a87d85 to f0dab4d Mar 3, 2015

@pfmoore pfmoore force-pushed the pfmoore:xplatform-friendly-run-201 branch from f0dab4d to 2de58e9 Mar 3, 2015

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 4, 2015

Closing this as it's been merged into master now.

@bitprophet bitprophet closed this Mar 4, 2015

@pfmoore pfmoore deleted the pfmoore:xplatform-friendly-run-201 branch Mar 5, 2015

@frol

This comment has been minimized.

frol commented on invoke/runner.py in 2de58e9 Sep 11, 2015

Is there a specific reason for use_setlocale=False in locale.getpreferredencoding()? It is the commit that first introduced this thing. The reason why I'm asking is the following:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

$ python2
Python 2.7.9 |Continuum Analytics, Inc.| (default, Apr 14 2015, 12:54:25)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import locale
>>> locale.getpreferredencoding(False)
'ANSI_X3.4-1968'
>>> locale.getpreferredencoding(True)
'UTF-8'
>>> locale.getpreferredencoding()
'UTF-8'

$ python3
Python 3.4.3 (default, Mar 25 2015, 17:13:50)
[GCC 4.9.2 20150304 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getpreferredencoding(False)
'UTF-8'
>>> locale.getpreferredencoding(True)
'UTF-8'
>>> locale.getpreferredencoding()
'UTF-8'

As you can see, Python 2 weirdly reports ANSI (ASCII) instead of UTF-8 when use_setlocale is False. This causes unicode characters to be replaced with ?? on, for example, run('echo "Я"')

@bitprophet bitprophet referenced this pull request May 27, 2016

Closed

Catchall 'encoding needs to get un-crap-ified' ticket #350

10 of 11 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment