Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

unicode problem in qtconsole for windows #529

Closed
jstenar opened this Issue · 17 comments

2 participants

Jörgen Stenarson Thomas Kluyver
Jörgen Stenarson
Collaborator

Hi

there is some unicode problem when running magic commands like ls from the qtconsole. I see the same problem both on master and newapp branch.

As you can see below there are problems with åäö (the directory with the strange name below).

In [1]: ls
Volymen i enhet C har etiketten Enhet C
Volymens serienummer är 3260-74A6

Innehåll i katalogen C:\python\slask

2011-06-20 18:17 .
2011-06-20 18:17 ..
2011-06-20 18:17 ├Ñ├ñ├Â
0 fil(er) 0 byte
3 katalog(er) 12 938 858 496 byte ledigt

Thomas Kluyver
Owner

Note: u"åäö".encode("utf-8").decode("cp850") == u"├Ñ├ñ├Â"

Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

The default depends on region, so I'm assuming that it produces output in cp850 for you. We must therefore be decoding it correctly, re-encoding as UTF-8, then trying to decode again using cp850.

Can you try the following commands, to check that the process is actually producing cp850 encoded text:

p = subprocess.Popen('dir /on', shell=True,
                         stdin=subprocess.PIPE,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE)
p.communicate()    # Let me know the results from this
Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

OK, and can you start the Qt console app, and try print u"åäö".encode(enc) with enc as 'cp850', 'cp1252' and 'utf-8' in turn.

Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

The newapp problem I found as well - it should be fixed if you pull again from git.

So it seems that Qt is rendering text in your system code page too. I guess somewhere we've assumed that Qt is expecting UTF-8 text. Don't know where off the top of my head.

Thomas Kluyver
Owner

@jstenar: Just to make sure, can you still replicate this with a fresh checkout?

Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

And if you do print u"åäö" at the qt console, I assume it works as expected?

Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

Oh, that's annoying. I've marked the bug as critical.

Can you check out my qtconsole-unicode-debug branch, and try the print command again. It should spit out reprs of the text it's trying to append at the terminal it's launched from.

https://github.com/takluyver/ipython/tree/qtconsole-unicode-debug

Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

OK, I think I've found it. Can you try with my iostream-unicode branch?

https://github.com/takluyver/ipython/tree/iostream-unicode

Thomas Kluyver takluyver was assigned
Jörgen Stenarson
Collaborator
Thomas Kluyver
Owner

Excellent, thanks. I've made PR #534 for it.

Thomas Kluyver
Owner

Closed by 19d5c41.

Thomas Kluyver takluyver closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.