Failing to install activate script in Windows Server 2008

[pancho-villa]

I have installed Python 3.3.4, then installed pip, then ran pip3.3.exe install virtualenv. When I run C:\Python33\Scripts\virtualenv testfoo it doesn't create a testfoo\Scripts\activate.bat or deacativate.bat files

Here's the stacktrace after running the virtualenv command:

C:\Users\Administrator>c:\Python33\Scripts\virtualenv.exe testfoo
Using base prefix 'C:\\Python33'
New python executable in testfoo\Scripts\python.exe
Installing setuptools, pip...
  Complete output from command C:\Users\Administrat...o\Scripts\python.exe -c "i
mport sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'main'
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python33\lib\runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "c:\Python33\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
  File "C:\Python33\lib\site-packages\virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 992, in create_enviro
nment
    install_wheel(to_install, py_executable, search_dirs)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 960, in install_wheel

    'PIP_NO_INDEX': '1'
  File "C:\Python33\lib\site-packages\virtualenv.py", line 902, in call_subproce
ss
    % (cmd_desc, proc.returncode))
OSError: Command C:\Users\Administrat...o\Scripts\python.exe -c "import sys, pip
; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

And here's the directory listing of the newly created environment:

C:\Users\Administrator>dir testfoo\Scripts
 Volume in drive C has no label.
 Volume Serial Number is 346F-6A10

 Directory of C:\Users\Administrator\testfoo\Scripts

02/11/2014  11:21 AM    <DIR>          .
02/11/2014  11:21 AM    <DIR>          ..
02/11/2014  11:21 AM            40,448 python.exe
02/11/2014  11:21 AM            40,960 pythonw.exe
               2 File(s)         81,408 bytes
               2 Dir(s)  19,137,454,080 bytes free

[Ivoz]

Could you edit your post and put three backticks on new lines above and below the console output, so it formats nicely to read? e.g. it would look like this while editing it:

```
C:\Users\Administrator>c:\Python33\Scripts\virtualenv.exe testfoo
Using base prefix 'C:\Python33'
New python executable in testfoo\Scripts\python.exe
Installing setuptools, pip...
etc
```

https://help.github.com/articles/github-flavored-markdown#fenced-code-blocks

[pancho-villa]

It looks like someone got to it before me, but thanks for the heads up. I'll make sure I do that next time.

[Ivoz]

Hmm... how complicated is the python install history on this server?

That traceback is a little mangled so it's hard to see where it went wrong. Seems to be complaining that pip has no main(), which it most certainly does... :S

I assume you should have no permissions problems, running everything under Administrator.

If you're not too invested in the current install, I would suggest uninstalling Python3.3, making sure the C:\Python33\ folder is wiped, and then reinstalling.

[After,] If you run the script under C:\Python33\Tools\Scripts\win_add2path.py then you will have python in your %PATH% and should be able to just type python, pip, etc at the console rather than prefacing it with the install path. Just a handy hint. This should also be a tickbox in the Python 3 installer these days IIRC.

If you hadn't figured out already, reinstalling pip should be as simple as downloading get-pip.py and running it.

From my perspective, it would be nice to confirm that this was indeed a cleanroom failure of virtualenv on a Windows (Server '08) system. I've seen a great many bugs from multiple installs / unclean uninstalls breaking things in the past.

[pancho-villa]

The host is a brand new virtual machine, and I installed python on it that day. I also have access to a freshly installed Windows 7 machine, and tried your steps outlined above to the same error as before.

[Ivoz]

Running Windows 8.1 x64
Installed Python 3.3.4, For All Users, Choose to add Python to Path

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\Ivo>cd Downloads

C:\Users\Ivo\Downloads>python get-pip.py
Downloading/unpacking pip
Downloading/unpacking setuptools
Installing collected packages: pip, setuptools
Successfully installed pip setuptools
Cleaning up...

C:\Users\Ivo\Downloads>pip -V
pip 1.5.2 from C:\Python33\lib\site-packages (python 3.3)

C:\Users\Ivo\Downloads>pip install virtualenv
Downloading/unpacking virtualenv
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

C:\Users\Ivo\Downloads>virtualenv --version
1.11.2

C:\Users\Ivo\Downloads>virtualenv testfoo
Using base prefix 'C:\\Python33'
New python executable in testfoo\Scripts\python.exe
Installing setuptools, pip...
  Complete output from command C:\Users\Ivo\Downloa...o\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'pip'
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python33\lib\runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "C:\Python33\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
  File "C:\Python33\lib\site-packages\virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 992, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 960, in install_wheel
    'PIP_NO_INDEX': '1'
  File "C:\Python33\lib\site-packages\virtualenv.py", line 902, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command C:\Users\Ivo\Downloa...o\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

C:\Users\Ivo\Downloads>dir testfoo
 Volume in drive C is System
 Volume Serial Number is 4C20-E54E

 Directory of C:\Users\Ivo\Downloads\testfoo

14/02/2014  02:39 AM    <DIR>          .
14/02/2014  02:39 AM    <DIR>          ..
14/02/2014  02:34 AM    <DIR>          Include
14/02/2014  02:39 AM    <DIR>          Lib
14/02/2014  02:39 AM    <DIR>          Scripts
               0 File(s)              0 bytes
               5 Dir(s)  125,438,689,280 bytes free

C:\Users\Ivo\Downloads>dir testfoo\Scripts
 Volume in drive C is System
 Volume Serial Number is 4C20-E54E

 Directory of C:\Users\Ivo\Downloads\testfoo\Scripts

14/02/2014  02:39 AM    <DIR>          .
14/02/2014  02:39 AM    <DIR>          ..
14/02/2014  02:39 AM            40,448 python.exe
14/02/2014  02:39 AM            40,960 pythonw.exe
               2 File(s)         81,408 bytes
               2 Dir(s)  125,436,813,312 bytes free

So can confirm it seems to be an error.

--no-setuptools seems to work.

C:\Users\Ivo\Downloads>virtualenv --no-pip testfoo2
Using base prefix 'C:\\Python33'
New python executable in testfoo2\Scripts\python.exe
Installing setuptools...
  Complete output from command C:\Users\Ivo\Downloa...2\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'pip'
----------------------------------------
...Installing setuptools...done.
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python33\lib\runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "C:\Python33\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
  File "C:\Python33\lib\site-packages\virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 992, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "C:\Python33\lib\site-packages\virtualenv.py", line 960, in install_wheel
    'PIP_NO_INDEX': '1'
  File "C:\Python33\lib\site-packages\virtualenv.py", line 902, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command C:\Users\Ivo\Downloa...2\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools failed with error code 1

C:\Users\Ivo\Downloads>virtualenv --no-setuptools testfoo3
Using base prefix 'C:\\Python33'
New python executable in testfoo3\Scripts\python.exe

C:\Users\Ivo\Downloads>

Mysterious to me so far.

[pfmoore]

OSError: Command C:\Users\Ivo\Downloa...2\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools failed with error code 1

Where's that "Downloa...2" coming from? Something seems to be abbreviating things here. As far as I know neither pip nor virtualenv would do that...

[Ivoz]

@pfmoore you on IRC? Virtualenv abbreviates lines over 45 chars long in the output.

Downloa...2 == Downloads\testfoo2, basically.

I will note that the wheel uploaded to PyPI has two pips in it - 1.5.1 and 1.5.2. That is very strange. Even after removing the 1.5.1 wheel though, I now get the same error as @pancho-villa

ATM I can't reproduce getting the error

ImportError: No module named 'pip'

Which is what I initially got;
Instead of the one I currently have,

AttributeError: 'module' object has no attribute 'main'

[pfmoore]

@Ivoz nope, sorry not on IRC. I'd forgotten there was that abbreviation going on. Just wondering if it was some strange Windows shortname thing.

I can't reproduce the problem here (Python 3.3, Windows 7 64-bit, virtualenv 1.11.2) so I don't really have much else I can say, sorry. One thing that's given me very odd results before now, you're not running from a directory that has a "pip" subdirectory present, are you?

[Ivoz]

you're not running from a directory that has a "pip" subdirectory present, are you?

Aha, yes I was. Funnily enough, pip creates a ~\pip folder for its log when you kill it / something fails etc...

Am now back to ImportError: No module named 'pip'

[Ivoz]

This is probably windows specific path shenanigans...

anyway, normally in install_wheel(), search_dir, pythonpath and findlinks are resolved to these values, respectively:

['.', 'C:\\Python33\\lib\\site-packages', 'C:\\Python33\\lib\\site-packages\\virtualenv_support', 'C:\\Python33\\lib\\site-packages\\virtualenv_support']
C:\Python33\lib\site-packages\virtualenv_support\setuptools-2.1-py2.py3-none-any.whl;C:\Python33\lib\site-packages\virtualenv_support\pip-1.5.1-py2.py3-none-any.whl
. C:\Python33\lib\site-packages C:\Python33\lib\site-packages\virtualenv_support C:\Python33\lib\site-packages\virtualenv_support

(yes, this is debug by print)

If I amend pythonpath with double backslashes, like so - pythonpath = pythonpath.replace('\\','\\\\') (i.e after L#938) then pip can be found from the wheel module, and the virtualenv will install successfully.

However I can't also achieve the same trick by modifying findlinks the same way. I would like to know what's the underlying cause before submitting any hacky patches...

[pfmoore]

Hmm. It just occurred to me that I never run an installed virtualenv script - I run an uninstalled virtualenv. So maybe there's a logic problem there.

While you're doing debug-by-print, would you amend cmd to print sys.path and os.environ['PYTHONPATH'] before calling pip in the subprocess call? I suspect that there's something screwy going on there - although I don't yet understand why...

BTW, I've just joined IRC if you want to carry this on there.

[pancho-villa]

I installed virtualenv on another Windows 7 x64 machine about a month ago and didn't have this issue. That version is 1.10.1, so I tried installing that on my problem machine, and it worked! Some regression must've been introduced between 1.10.1 and the current build.

[pfmoore]

See #567 for an updated PR that loses an unnecessary pair of double quotes that triggers odd behaviour from subprocess.py (see list2cmdline in subprocess.py for the rules, they are pretty arcane). Essentially it looks like having double quotes triggers the addition of backslashes that are what was causing oddities... There may be a bug in subprocess.py here, but I'm blowed if I can describe it well enough to report it!

[pfmoore]

Looks like this might be a Python 3.3.4 bug - see http://bugs.python.org/issue20621

[pfmoore]

OK. I have reviewed the details of this issue. It seems to me that all of the reported issues can be explained by the bug reported in http://bugs.python.org/issue20621 - every reported traceback is some variation of being unable to find the correct pip module from the wheel on sys.path.

The suggested fix in #566 bears no relation to the import issue, and almost certainly does not fix the issue, unfortunately.

[pfmoore]

For now, I'd recommend downgrading to Python 3.3.3. I've asked on python-dev about what the best way forward is likely to be, I'll report back here when I have news.

[pfmoore]

There will be a 3.3.5 as soon as a fix is available. As there's no viable workaround here, I think I'm just going to have to declare virtualenv broken on 3.3.4. I'll add a test and error into the code so that people get a clean message rather than an obscure bug.

[pfmoore]

I'm going to merge PR #568 to address this. On Python 3.3.4, virtualenv will display an error message and work as if you'd specified --no-pip --no-setuptools. If you can't downgrade to 3.3.3 or upgrade to 3.3.5 (expected to be released in a couple of weeks to address this issue) you can use get-pip.py to manually install pip and setuptools.

I'm going to pick up the various quoting suggestions and simplifications of the pip-calling code in a separate PR. But it'll affect the same code as #568, so I'll wait till that one has landed before submitting it.

[pfmoore]

OK, change of plan. I'm closing this, as the issue is a Python bug, not a virtualenv bug. Users can downgrade to Python 3.3.3, wait a week or so for Python 3.3.5, or use virtualenv --no-pip --no-setuptools`` and then install pip via ``get-pip.py as a workaround. It's not ideal, but blocking use with particular Python versions gains us nothing except better error reporting, for an issue that will go away very soon.