Install from requirements file (into a virtualenv) fails on Windows

[sffjunkie]

OS: Windows 7
Python 3.5.1

pip -V
pip 8.0.2 from c:\users\xxx\documents\development\venvs\glossary\lib\site-packages (python 3.5)

Installing from a requirements file generated from pip freeze > requirements.txt produces the following error

pip install -r .\requirements.txt

Traceback (most recent call last):
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\basecommand.py", line 209, in main
    status = self.run(options, args)
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\commands\install.py", line 287, in run
    wheel_cache
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\basecommand.py", line 289, in populate_requirement_set
    wheel_cache=wheel_cache):
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\req\req_file.py", line 92, in parse_requirements
    for req in req_iter:
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\req\req_file.py", line 154, in process_line
    isolated=isolated, options=req_options, wheel_cache=wheel_cache
  File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\req\req_install.py", line 181, in from_line
    path = os.path.normpath(os.path.abspath(name))
  File "c:\users\xxx\documents\development\venvs\glossary\lib\ntpath.py", line 535, in abspath
    path = _getfullpathname(path)
ValueError: _getfullpathname: embedded null character``

[izaim]

Looks like pip 8.0.2 doesn't play good with virtualenv in Windows. I'm in trouble too #3482. Trying to figure out both problems..

[izaim]

Follow #3482.

[xavfernandez]

@sffjunkie Does it work better with pip install -r requirements.txt ?

[sffjunkie]

No pip install -r requirements.txt still exhibits the same problem.

[xavfernandez]

@sffjunkie Could you add a print statement at:

File "c:\users\xxx\documents\development\venvs\glossary\lib\site-packages\pip\req\req_install.py", line 181, in from_line
    path = os.path.normpath(os.path.abspath(name))

and print the name triggering this issue ?

And/or remove lines from your requirements.txt file to figure out what it the error triggering line ?

[Ivoz]

@sffjunkie would it be possible for you to tell us the username you are using? Pip has been known to have some trouble with ones that are non-ascii.

[sffjunkie]

OK, it's failing on the first line. It looks like it's encoded in UTF-16 with a BOM and not being decoded properly.

First line is ÿþa s t r a l = = 0 . 9 . 1

[pfmoore]

Did you do the pip freeze > requirements.txt in Powershell? Output redirection in Powershell uses Unicode, you have to take care (maybe use pip freeze | Out-File -Encoding ASCII requirements.txt)

[sffjunkie]

@pfmoore Yes, it was in Powershell. Your Out-File suggestion works. Maybe that advice could be added to the pip freeze docs?

[xavfernandez]

Strongly related to #2865 and #733

[xavfernandez]

I think we should either implement BOM detection on requirements.txt files or require that they should be encoded in utf8.

[pfmoore]

+1 for BOM detection. Requiring UTF-8 wouldn't help Windows/powershell users, as Powershell is no more willing to give you UTF-8 than ASCII.

I'll take a look at this possibility at some point (although if anyone else gets there before me, I'm fine with that ;-))

[xavfernandez]

Closed by #3485