Jython+virtualenv+pip #136

Closed
parlarjb opened this Issue Jun 3, 2011 · 9 comments

Projects

None yet

2 participants

parlarjb commented Jun 3, 2011

I've been playing with Jython recently and ran into an issue using it with virtualenv and pip.

The virtualenv gets created just fine, and puts a 'jython' executable into bin/. The problem is that it doesn't create a 'python' executable symlink from this. Now, this might be the desired behaviour, making sure to never call a Jython interpreter 'python', but if so, it breaks pip.

The current release of pip (1.0.1) builds up its executable path within a virtualenv with:

    if sys.platform == 'win32':
        python = os.path.join(venv, 'Scripts', 'python.exe')
        # check for bin directory which is used in buildouts
        if not os.path.exists(python):
            python = os.path.join(venv, 'bin', 'python.exe')
    else:
        python = os.path.join(venv, 'bin', 'python')
    if not os.path.exists(python):
        python = venv
    if not os.path.exists(python):
        raise BadCommand('Cannot find virtual environment interpreter at %s' % python)

i.e. it's explicitly looking for an executable named 'python', or 'python.exe' on Win32.

Is this a virutalenv issue or a pip one?

Contributor
pnasrat commented Jun 3, 2011

Can you break down your work flow a bit more this seems to work on OS X

  • virtualenv -p jython ve
  • . ve/bin/activate
  • pip install mock
  • jython
  • import mock

This could well be a win32 bug, but I'd like to have a more exact reproducer to be sure.

rm -rf ve; virtualenv -p jython ve
Running virtualenv with interpreter /Users/pnasrat/apps/bin/jython
New jython executable in ve/bin/jython
Installing setuptools......................done.
Installing pip...............done.
pnasrat@dhcp-172-16-18-182:~$ . ve/
Lib/        bin/        cachedir/   jython.jar  registry    
pnasrat@dhcp-172-16-18-182:~$ . ve/bin/activate
(ve)pnasrat@dhcp-172-16-18-182:~$ which python
/usr/bin/python
(ve)pnasrat@dhcp-172-16-18-182:~$ which pip
/Users/pnasrat/ve/bin/pip

Then I can do

(ve)pnasrat@dhcp-172-16-18-182:~$ pip install -U mock
Downloading/unpacking mock
  Downloading mock-0.7.2.zip (938Kb): 938Kb downloaded
  Running setup.py egg_info for package mock
Installing collected packages: mock
  Running setup.py install for mock
Successfully installed mock
Cleaning up...
(ve)pnasrat@dhcp-172-16-18-182:~$ jython
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) 
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_20
Type "help", "copyright", "credits" or "license" for more information.
>>> import mock
>>> mock.__file__
'/Users/pnasrat/ve/Lib/site-packages/mock$py.class'
Contributor
pnasrat commented Jun 3, 2011

Also note for me ve/bin/pip has

#!/usr/bin/env /Users/pnasrat/ve/bin/jython                                                                                                                                                              
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.0.1','console_scripts','pip'

I don't have ready access to a windows box (but can spin up an EC2 instance if need be) so can you confirm what your install looks like.

parlarjb commented Jun 3, 2011

I'm on OS X as well (I only mentioned Win32 because the code above did). Here's the output of an equivalent session:

[~]$ mkvirtualenv -p jython test-ve
Running virtualenv with interpreter /Users/parlarjb/bin/jython
New jython executable in test-ve/bin/jython
Installing distribute.....................................................................................................................................................................................done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/get_env_details
(test-ve)[~]$ pip install mock
Traceback (most recent call last):
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/pip", line 8, in <module>
    sys.exit(
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/__init__.py", line 116, in main
    return command.main(initial_args, args[1:], options)
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/basecommand.py", line 108, in main
    restart_in_venv(options.venv, options.venv_base, site_packages,
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/venv.py", line 50, in restart_in_venv
    proc = subprocess.Popen(
  File "/Users/parlarjb/jython2.5.2/Lib/subprocess.py", line 751, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/parlarjb/jython2.5.2/Lib/subprocess.py", line 1265, in _execute_child
    raise OSError(e.getMessage() or e)
OSError: Cannot run program "/Users/parlarjb/Dropbox/virtualenvs/test-ve" (in directory "/Users/parlarjb"): error=13, Permission denied

With:

(test-ve)[test-ve]$ cat bin/pip
#!/usr/bin/env /Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/jython
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.0.1','console_scripts','pip'
__requires__ = 'pip==1.0.1'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('pip==1.0.1', 'console_scripts', 'pip')()
    )
(test-ve)[test-ve]$ 

The code I pasted in the original comment was from venv.py, and explains that traceback.

Contributor
pnasrat commented Jun 3, 2011

OK just to narrow the components can you try again without using virtualenvwrapper (which provides mkvirtualenv) - that is do

virtualenv -p jython test-ve
. test-ve/bin/activate
pip install mock

Also the version of virtualenv and virtualenvwrapper you are using would be helpful.

Thanks.

parlarjb commented Jun 3, 2011
[test-ve]$ rmvirtualenv test-ve
[virtualenvs]$ virtualenv -p jython test-ve
Running virtualenv with interpreter /Users/parlarjb/bin/jython
New jython executable in test-ve/bin/jython
Installing distribute.....................................................................................................................................................................................done.
Installing pip...............done.
[virtualenvs]$ . ~/Dropbox/virtualenvs/test-ve/bin/activate
(test-ve)[virtualenvs]$ pip install mock
Traceback (most recent call last):
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/bin/pip", line 8, in <module>
    sys.exit(
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/__init__.py", line 116, in main
    return command.main(initial_args, args[1:], options)
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/basecommand.py", line 108, in main
    restart_in_venv(options.venv, options.venv_base, site_packages,
  File "/Users/parlarjb/Dropbox/virtualenvs/test-ve/Lib/site-packages/pip-1.0.1-py2.5.egg/pip/venv.py", line 50, in restart_in_venv
    proc = subprocess.Popen(
  File "/Users/parlarjb/jython2.5.2/Lib/subprocess.py", line 751, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/parlarjb/jython2.5.2/Lib/subprocess.py", line 1265, in _execute_child
    raise OSError(e.getMessage() or e)
OSError: Cannot run program "/Users/parlarjb/Dropbox/virtualenvs/test-ve" (in directory "/Users/parlarjb/Dropbox/virtualenvs"): error=13, Permission denied

And the version:

(test-ve)[virtualenvs]$ virtualenv --version
1.6.1

Not sure how to check which version of virtualenvwrapper I have, as there doesn't seem to be version info in it.

Contributor
pnasrat commented Jun 3, 2011

Sorry but two other data points would be useful - you can either put them in the bug or use gist and link them

  • output of env in the shell you are running this in after you start the virtualenv
  • output of ls -lR /Users/parlarjb/Dropbox/virtualenvs/test-ve after you've re-created it with vanilla virtualenv

Thanks for the update info.

parlarjb commented Jun 3, 2011

https://gist.github.com/1006739

https://gist.github.com/1006760

What it comes down to, from my perspective, is those few lines of code from venv.py I originally posted. With that code, it seems like it'd be impossible for pip to work without a 'python' symlink. And it looks like the copy/paste killed the indent in the original post, if you look again I've corrected it. It wasn't all supposed to be wrapped in the 'win32' check.

Contributor
pnasrat commented Jun 3, 2011

Clearly I see there is a difference between our two setups, which may be a bug.

Looking at the code base I note that git develop branch (for the next release) no longer has pip/venv.py since commit 87dd1579a25779751e4a41b9e9b25ba3e0fe4d5d

As this is working for me as above in a similar environment, I've compared your env output with mine. Looking at the code and your env I think that:

PIP_REQUIRE_VIRTUALENV=true

triggers the buggy code path removed in the commit above. If you unset that in your environment does the issue go away? In which case it's fixed for the next release, and you should be able to work around by configuring virtualenvwrapper not to set that/unset it in a hook.

parlarjb commented Jun 3, 2011

Perfect, that did it. Thanks for all the help.

@parlarjb parlarjb closed this Jun 3, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment