PIP bad handling of diacritics in folder names.

[rin67630]

Just after a fresh install of the current downloaded Python with PIP on Windows.
on which the user name contains diacritics, i get upon starting PIP:

"Fatal error in launcher: Unable to create process using '"C:\Users
\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32
\python.exe"  "C:\Users\B³rgerGegenFluglõrm\AppData\Local\Programs\Python
\Python35-32\Scripts\pip.exe" '"

Where the correct path is supposed to be "C:\Users\BürgerGegenFluglärm\AppData..."

The funny thing is that the message mentions the path twice, with two
different wrong codings.

That appears to be an issue isn't it?
Isn't PIP supposed to be 100% UTF-8?

[Rosuav]

For what it's worth, the first one seems to have been encoded UTF-8 and then decoded CP437; and the second has been encoded Latin-1 and decoded CP850.

>>> "BürgerGegenFluglärm".encode("utf-8").decode("437")
'BürgerGegenFluglärm'
>>> "BürgerGegenFluglärm".encode("iso-8859-1").decode("850")
'B³rgerGegenFluglõrm'

No idea whether that helps at all. @rin67630, is your system code page set to any one of those four? If you type chcp without parameters, Windows should tell you what the current code page is.

[rin67630]

Ups! the reported shell's codepage is 850 indeed.

That, albeit the Registry has:
[HKEY_CURRENT_USER\Console]
"CodePage"=dword:fde9
???

[Rosuav]

Hex FDE9 (decimal 65001) is the Windows name for UTF-8. I'm not sure what that registry entry actually does, but it's theoretically saying that you should be using UTF-8.

[pfmoore]

Can you just clarify - what is the command you executed?

Also, can you confirm what the commands

py -c "import sys; print(sys.executable)"
py -m pip --version

return?

[pfmoore]

(The reason I ask is that I suspect that it's the exe wrapper that runs pip which is the problem here, rather than pip itself. That wrapper is supplied by distlib, so if that is where the problem lies, we may need to report the issue there).

[rin67630]

I am just running "pip" from the command line, nothing else and immediately get the error.
I have no .exe wrapper around.

Your py commands run as following:
C:\Users\BürgerGegenFluglärm>py -c "import sys; print(sys.executable)"
C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\python.exe
is that what is expected?
(it has the correct diacritics)

[Rosuav]

Just out of interest, what's this show?

py -c "import sys; print(ascii(sys.executable))"

Might tell us what the file system encoding is, if nothing else.

[rin67630]

sorry i forgot the second one:
C:\Users\BürgerGegenFluglärm>py -m pip --version
pip 7.1.0 from C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages (python 3.5)

Thank you for your help.

[rin67630]

C:\Users\BürgerGegenFluglärm>py -c "import sys; print(ascii(sys.executable))"
'C:\Users\B\xfcrgerGegenFlugl\xe4rm\AppData\Local\Programs\Python\Python35-32\python.exe'

[Rosuav]

Strange, I would expect ascii() to double all those backslashes. But I think I made an error in recommending this, as it's simply giving us back the same Unicode text we were expecting. The FS encoding isn't visible, but it also doesn't seem to matter, as the decode to Unicode is giving the right results. Also, the console encoding is being correctly handled, as normal Unicode output is giving correct display. My bad, sorry!

[wm75]

@pfmoore couldn't this possibly be identical to this distlib issue: https://bitbucket.org/pypa/distlib/issues/71/script-wrappers-dont-handle-non-ascii, which you reported back in March this year?

[pfmoore]

@rin67630 OK, the fact that the python -m pip invocation works means that the issue is with the wrapper, not with pip. (The bare pip runs an executable pip.exe that's installed with Python, and that in turn invokes what is essentially python -m pip but using the full pathname for python - but it appears to be doing so in a way that breaks when there are non-ASCII characters in that pathname.

@wm75 Yes, it does look like that one. And that was released in distlib 0.2.1, which doesn't appear to be in the version of pip that's included with Python 3.5rc2 (the latest version I have).

@rin67630 This should be fixed by upgrading pip. Can you please try

python -m pip install --upgrade pip```

That should upgrade pip from version 7.1.0 to 7.1.2, which includes a fixed distlib, and should fix the issue.

[pfmoore]

FWIW, the fixed version of pip is in Python 3.5rc4 (and hence will be in 3.5 final).

[rin67630]

I have tried the upgrade, but got a bech of exceptions listed below.

C:\Users\BürgerGegenFluglärm>python -m pip install --upgrade pip You are using pip version 7.1.0, however version 7.1.2 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. Exception: Traceback (most recent call last): File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\basecommand.py", line 223, in main status = self.run(options, args) File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\commands\install.py", line 270, in run wheel_cache File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\basecommand.py", line 275, in populate_requirement_set wheel_cache=wheel_cache File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\req\req_install.py", line 212, in from_line wheel_cache=wheel_cache, constraint=constraint) File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\req\req_install.py", line 66, in __init__ req = pkg_resources.Requirement.parse(req) File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2980, in parse reqs = list(parse_requirements(s)) File "C:\Users\BürgerGege nFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2924, in parse_requirements "version spec") File "C:\Users\BürgerGegenFluglärm\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2889, in scan_list raise RequirementParseError(msg, line, "at", line[p:]) pip._vendor.pkg_resources.RequirementParseError: Expected version spec in pip at ```

Since i am just starting using python, i am a bit confused by this firework of exceptions...
I have got the advice to reinstall python, with the Option to incude the Python Path Option.
I am not sure if i did the first time, but will try that tomorrow.
I will report my findings.

[Random832]

@rin67630 I think there was a typo in @pfmoore's reply - please try again without ``` at the end of the command.

[pfmoore]

Sigh. Agreed, that's a horrible error. To be honest, I have no idea what it means :-(

Ah - I see @Random832 just spotted my typo - thanks, yes that would cause you problems. Sorry about that. As stated, if you try again without the trailing ``` it should work better...

The advice to reinstall Python, with the newest Python 3.5 release candidate, should also fix your problem (alternatively, using the stable Python 3.4 would also fix the issue, as that would install in C:\Python34 and so avoid the non-ASCII path altogether).

I'm sorry your initial experience with Python has been less than ideal - thanks for taking the time to report the issue and help us to ensure we fix things like this, it's much appreciated.

[pfmoore]

@rin67630 did you manage to solve your issue?

[dstufft]

Closing this, if it continues to happen please reopen this issue or open a new one with reproduction steps.