CherryPy 7.1 install error on Windows

[marunguy]

• Pip version: 8.1.2
• Python version: 2.7.12 32bit
• Operating System: Windows 10 64bit Korean

I have report this issue alreay to CherryPy community.
https://groups.google.com/forum/#!topic/cherrypy-users/7G30-UfSqtU

The easy_install may have a similar issue.
cherrypy/cherrypy#1467
pypa/setuptools#704

I think that pip have a problem with a package included a file have a non-ascii filename on windows.

D:\pydevenv\win32>pip install -U CherryPy
Collecting CherryPy
  Using cached CherryPy-7.1.0.tar.gz
Exception:
Traceback (most recent call last):
  File "d:\devtool\py27x86\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\devtool\py27x86\lib\site-packages\pip\commands\install.py", line 299, in run
    requirement_set.prepare_files(finder)
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 810, in unpack_url
    hashes=hashes
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 653, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 577, in untar_file
    with open(path, 'wb') as destfp:
IOError: [Errno 22] invalid mode ('wb') or filename: 'd:\\users\\me\\appdata\\local\\temp\\pip-build-0vvcm3\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'

It seems that IOError raised because of non-ascii filename - "cherrypy/test/static/Слава Україні.html" .
The filename is passed as "utf-8" encoding.

>>> print 'd:\\users\\me\\appdata\\local\\temp\\pip-build-esnrji\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'.decode("utf-8")
d:\users\me\appdata\local\temp\pip-build-esnrji\CherryPy\cherrypy/test/static/Слава Україні.html

But, the default encoding on windows is not "utf-8".

>>> sys.getdefaultencoding()
'ascii'
>>> sys.getfilesystemencoding()
'mbcs'

the default encoding on korean windows is "cp949".
So, DecodeError raised.

>>> print 'd:\\users\\me\\appdata\\local\\temp\\pip-build-esnrji\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'.decode("cp949")
Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    print 'd:\\users\\me\\appdata\\local\\temp\\pip-build-esnrji\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'.decode("cp949")
UnicodeDecodeError: 'cp949' codec can't decode bytes in position 99-100: illegal multibyte sequence

>>> print 'd:\\users\\me\\appdata\\local\\temp\\pip-build-esnrji\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'.decode("cp949", "ignore")
d:\users\me\appdata\local\temp\pip-build-esnrji\CherryPy\cherrypy/test/static/鬼剋逵勻逵 叫克逵戟.html

NAME INVALID error found in process monitor log.
오후 1:19:58.3672120 python.exe 4616 CreateFile D:\Users\me\AppData\Local\Temp\pip-build-esnrji\CherryPy\cherrypy\test\static\鬼剋逵勻逵 叫克?逵?戟?.html NAME INVALID Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0

[jaraco]

I believe the issue is that pip has the same bug as setuptools when it comes to untarring a file with unicode filenames on python 2, namely where tarfile.open is invoked, filename could be bytes, in which case, tarfile returns filenames in bytes and nothing handles the encoding of those. The proper fix here is to make sure the filename is text (unicode in Python 2 parlance).

[hexagonrecursion]

This issue is marked as "python 2 only". pip 21.0 dropped support for Python 2. Should this be closed?

[uranusjr]

Thanks . I believe this is covered by #7667 anyway.