print unicode in Windows console

[LeslieK]

BPO	21164
Nosy	@terryjreedy, @vstinner
Superseder	bpo-1602: windows console doesn't print or input Unicode

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-04-11.17:43:23.332>
created_at = <Date 2014-04-07.02:57:52.718>
labels = ['type-bug', 'OS-windows']
title = 'print unicode in Windows console'
updated_at = <Date 2014-04-11.18:51:09.783>
user = 'https://bugs.python.org/LeslieK'

bugs.python.org fields:

activity = <Date 2014-04-11.18:51:09.783>
actor = 'LeslieK'
assignee = 'none'
closed = True
closed_date = <Date 2014-04-11.17:43:23.332>
closer = 'terry.reedy'
components = ['Windows']
creation = <Date 2014-04-07.02:57:52.718>
creator = 'LeslieK'
dependencies = []
files = []
hgrepos = []
issue_num = 21164
keywords = []
message_count = 4.0
messages = ['215675', '215693', '215937', '215945']
nosy_count = 3.0
nosy_names = ['terry.reedy', 'vstinner', 'LeslieK']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '1602'
type = 'behavior'
url = 'https://bugs.python.org/issue21164'
versions = ['Python 3.3']

[LeslieK]

The console behaves by encoding a str (using the sys.getdefaultencoding()) and then decodes the bytes to a glyph for rendering. The decoder used is 'cp437'. Apparently, there is no way to override that!

See ipython notebook for summary and example of the issue.

nbviewer.ipython.org/gist/LeslieK/10013426

[vstinner]

Hi, when you say "The console" is the the old MS-DOS command ("Windows console") opened when you run the "cmd.exe" program? Or IDLE or another console?

For the Windows console, see the old issue bpo-1602 which is not fixed yet.

On Windows, sys.stdout.encoding is your OEM code page, which is usually different than the ANSI code page (locale.getpreferredencoding()).

[terryjreedy]

You can change the code page of a Command Prompt window, before calling Python, with ...> chcp <code-page>. There is 'something' called cp65001 that is supposed to be a utf-8 codepage. Once can change to it, but it does not work right. This has been discussed on StackOverflow and elsewhere.

The Idle Shell, running on tkinter, handles everything in the BMP because tcl/tk is unicode (ucs-2) based. The characters that display properly depend on the font you select. On my machine, all but three of the following arbitrary codepoints display proper characters.
>>> print('\u1111\u2222\u3333\u4444\u5555\u6666\u7777\u8888\u9999\uaaaa\ubbbb\ucccc\udddd\ueeee')
ᄑ∢㌳䑄啕晦睷袈香ꪪ뮻쳌�

[LeslieK]

I start ipython in Windows PowerShell. The "console" I am referring to is
ipython running in WindowsPowerShell.
I do not use the DOS cmd.exe

When running ipython notebook from WindowsPowerShell -- no problem printing
unicode.

When running in PTVS (Python Tools for Visual Studio) debugger -- no
problem printing unicode in Output window, or Interactive Python window.
Problem does appear in Console window (which is the DOS window I believe).

On Mon, Apr 7, 2014 at 7:08 AM, STINNER Victor <report@bugs.python.org>wrote:

STINNER Victor added the comment:

Hi, when you say "The console" is the the old MS-DOS command ("Windows
console") opened when you run the "cmd.exe" program? Or IDLE or another
console?

For the Windows console, see the old issue bpo-1602 which is not fixed yet.

On Windows, sys.stdout.encoding is your OEM code page, which is usually
different than the ANSI code page (locale.getpreferredencoding()).

----------
nosy: +haypo

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21164\>

---