scripts outside a virtualenv that use entry points can't find registrations inside the virtualenv #124

Closed
slinkp opened this Issue Apr 21, 2011 · 3 comments

Comments

Projects
None yet
2 participants
@slinkp

slinkp commented Apr 21, 2011

What the title says. If you have a python script whose shebang line points to an interpreter other than the virtualenv's interpreter, it won't find packages installed in the virtualenv that register things for entry points that it uses.

For detailed example see
http://trac.pythonpaste.org/pythonpaste/ticket/458

Is this actually a virtualenv bug? or a setuptools bug? or what?

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Apr 21, 2011

Contributor

A script with a shebang line that doesn't resolve to the virtualenv's interpreter is not running inside the virtualenv, period. Using the virtualenv's python interpreter is how you use a virtualenv. This is the core of virtualenv's design, and isn't going to change.

The solution is to fix the script so its shebang line uses the conventional "#/usr/bin/env python" - then if your shell has a virtualenv activated, the virtualenv's interpreter will be used.

Contributor

carljm commented Apr 21, 2011

A script with a shebang line that doesn't resolve to the virtualenv's interpreter is not running inside the virtualenv, period. Using the virtualenv's python interpreter is how you use a virtualenv. This is the core of virtualenv's design, and isn't going to change.

The solution is to fix the script so its shebang line uses the conventional "#/usr/bin/env python" - then if your shell has a virtualenv activated, the virtualenv's interpreter will be used.

@carljm carljm closed this Apr 21, 2011

@slinkp

This comment has been minimized.

Show comment
Hide comment
@slinkp

slinkp Apr 21, 2011

I understand what you're saying, thanks. I have to concede that there's really nothing that virtualenv can do to help.

If I could trouble you for insight for another moment...
There's a relevant open pip bug: pypa/pip#17
Does the package installer seem to you like the right place to address this problem?

Also FWIW there are strong arguments against /usr/bin/env as a solution:
http://mail.python.org/pipermail/distutils-sig/2008-April/009283.html
but that's a digression.

Regardless of using /usr/bin/env, I don't think munging the shebang line is practical: the problematic script was likely installed globally by some package manager (eg. apt-get install ...), likely as a dependency of something else, so the user probably doesn't even know it's there; and the script's shebang line was probably generated by setuptools, which by design does not rely on /usr/bin/env. The poor person trying to install my package X and use a script provided by dependency package Y is likely just following some install docs, likely has no understanding of virtualenv, and definitely doesn't understand why Y's scripts are running but not working as expected "even though I have exactly the right version of Y". I've received several problem reports like this. I generally tell them to fix it like easy_install --upgrade Y==1.2.3, but I'm trying to find a solution that doesn't put people in that situation in the first place.

slinkp commented Apr 21, 2011

I understand what you're saying, thanks. I have to concede that there's really nothing that virtualenv can do to help.

If I could trouble you for insight for another moment...
There's a relevant open pip bug: pypa/pip#17
Does the package installer seem to you like the right place to address this problem?

Also FWIW there are strong arguments against /usr/bin/env as a solution:
http://mail.python.org/pipermail/distutils-sig/2008-April/009283.html
but that's a digression.

Regardless of using /usr/bin/env, I don't think munging the shebang line is practical: the problematic script was likely installed globally by some package manager (eg. apt-get install ...), likely as a dependency of something else, so the user probably doesn't even know it's there; and the script's shebang line was probably generated by setuptools, which by design does not rely on /usr/bin/env. The poor person trying to install my package X and use a script provided by dependency package Y is likely just following some install docs, likely has no understanding of virtualenv, and definitely doesn't understand why Y's scripts are running but not working as expected "even though I have exactly the right version of Y". I've received several problem reports like this. I generally tell them to fix it like easy_install --upgrade Y==1.2.3, but I'm trying to find a solution that doesn't put people in that situation in the first place.

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Apr 21, 2011

Contributor

Yeah, I do think that that issue in pip deserves a fix, and is the right (really, only, that I can see) way to address this. I didn't realize the outside-of-virtualenv script you were dealing with was a setuptools/pip installed one.

Re /usr/bin/env, I actually agree with Barry and PJ in that thread, that setuptools (and pip) should install things with hardcoded shebangs pointing to the Python used to install them. (If they didn't, scripts installed in a virtualenv would require activation of the virtualenv to work properly, you wouldn't just be able to run them directly). I just thought if you had your own hand-rolled script and wanted it to respect virtualenv activation, the /usr/bin/env approach would be quickest and easiest.

Contributor

carljm commented Apr 21, 2011

Yeah, I do think that that issue in pip deserves a fix, and is the right (really, only, that I can see) way to address this. I didn't realize the outside-of-virtualenv script you were dealing with was a setuptools/pip installed one.

Re /usr/bin/env, I actually agree with Barry and PJ in that thread, that setuptools (and pip) should install things with hardcoded shebangs pointing to the Python used to install them. (If they didn't, scripts installed in a virtualenv would require activation of the virtualenv to work properly, you wouldn't just be able to run them directly). I just thought if you had your own hand-rolled script and wanted it to respect virtualenv activation, the /usr/bin/env approach would be quickest and easiest.

clebergnu added a commit to clebergnu/avocado that referenced this issue Dec 7, 2015

virtualenv: allow tests to run properly on them
While going through the dependency list (requirements*.txt files) and
performing our self tests out of virtual environments, I noticed that
some tests are run outside the virtual environments.

The reason is that, even though the virtual environment is activated
for the test session (and say, `which python` gives `/venv/bin/python`),
we have hard coded `/usr/bin/python` in most places.

According to the some discussions on the virtualenv project itself[1],
a quick solution is to revert to the also common `/usr/bin/env python`
way of pointing to the Python interpreter.

[1] - pypa/virtualenv#124

Signed-off-by: Cleber Rosa <crosa@redhat.com>

@clebergnu clebergnu referenced this issue in avocado-framework/avocado Dec 7, 2015

Merged

virtualenv: allow tests to run properly on them #910

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment