Skip to content
This repository

Recognise virtualenvs #1171

Closed
takluyver opened this Issue December 17, 2011 · 13 comments

5 participants

Thomas Kluyver Fernando Perez Min RK Julian Taylor
Thomas Kluyver
Collaborator

A recurring issue, which I've just added to the FAQ - a standard installation of IPython won't recognise when you're in a virtualenv. Various workarounds are showing up, but it would be good to work out of the box.

The root of the problem is that the installation sets the shebang line of ipython to #!/usr/bin/python. If it's #!/usr/bin/env python, virtualenvs work as expected. (I don't know what the status is on Windows)

A few possible ways to deal with this:

  • Get an option added to distutils/packaging/setuptools/distribute to control the shebang line written. Long term at best.
  • For Linux distributions, the shebang could be replaced as a step in the packaging.
  • Contribute ipython-awareness to virtualenv, so if IPython is installed, it will copy the script and update the shebang line when you create a virtualenv.
  • Detect when we're in a virtualenv, and add the relevant paths to sys.path. This is suboptimal, because it only affects importable libraries, not the interpreter itself (e.g. if I make a Python 2.6 virtualenv, IPython will still start with the default Python 2.7).
Fernando Perez
Owner

Mmh, how is this problem unique to us? If this is happening, then it happens to any python project that installs executable scripts. Granted, many python projects are indeed just libraries with minimal to no user-facing executable code, but still: nose, cython are two that immediately come to mind with this same situation.

How does everyone else handle this problem? I'm not claiming it's not an issue, just thinking that somebody out there must have figured this one out...

Thomas Kluyver
Collaborator

I think for most things, it makes sense to run it with the system Python, ignoring locally installed libraries (e.g. if I run a mercurial command, a virtualenv shouldn't interfere with it). I suspect nose has the same problem in a standard installation, and I've also run into it with sphinx-build.

Fernando Perez
Owner

OK, I figured it out. The solution (not just for IPython, but for installying any python package that has scripts in a virtualenv) is to do the build step separately:

./setup.py build --executable "/usr/bin/env python"
./setup.py install

This sets the shebang line correctly once the scripts get installed.

Since this is standard distutils, I think we can actually close the issue here. Feel free to put this tip up on the faq, as it's certainly not very obvious. I had an inkling of the solution because I had a vague memory of being able to control that line years ago when I was building RPMs with distutils for a lab where I had a bunch of Fedora machines. But otherwise it's not easy to just stumble upon this solution.

Fernando Perez fperez closed this December 17, 2011
Min RK
Owner

Yes, this has never had anything to do with IPython, it's always affected all distutils-installed scripts, but IPython is often the only such script inherited into a virtualenv.

It should be considered a virtualenv bug that they don't patch the #! line of python scripts.

Thomas Kluyver
Collaborator

So it turns out we can set this without requiring the user to pass an extra command line argument (http://stackoverflow.com/a/1719991/434217), but it only works for distutils style 'scripts', not setuptools 'entry_points'. I've got a branch for that, and I'll look into it more closely for 0.13.

I'm reopening this and assigning it to myself on the grounds that I think we can improve on the current situation.

Thomas Kluyver takluyver reopened this December 18, 2011
Fernando Perez
Owner
Julian Taylor
Collaborator

fyi, the debian packages already use env python as shebang for ipython since a while [0], but for the entry point script used in py3 I have found no other way than to sed the resulting scripts [1].
what I find weird, but probably is a distutils issue, is that the script is only created on install, not on build.
This also makes testing a bit more ackward as iptest3 is only created after installation.

[0] http://anonscm.debian.org/viewvc/python-modules/packages/ipython/trunk/debian/rules?r1=15749&r2=16684
[1] http://anonscm.debian.org/viewvc/python-modules/packages/ipython/trunk/debian/rules?r1=19668&r2=19670

Thomas Kluyver
Collaborator
Julian Taylor
Collaborator

I don't think we need special handling for the new venv behavior. That the system wide ipython is not importable with --no-site-packages is the expected (and likely wanted) behavior.
installing ipython in that venv with easy_install results in the correct shebang.

Thomas Kluyver
Collaborator
Thomas Kluyver takluyver closed this April 23, 2012
Thomas Kluyver
Collaborator

Fixed, as much as it's going to be for now, by PR #1388, so I'm closing this again.

A helpful tip for others on Linux who find the version of IPython shipped in the distro now fails with importerror. Virtualenv changed the default behaviour from --system-site-packages to --no-site-packages. For IPython this means that when you run it from a terminal where the virtualenv is in your path, it will not pick up the ipython libs. Either install IPython in your virtualenv, or make your virtualenv use the system-site-packages.

Thomas Kluyver
Collaborator

If your systemwide IPython is installed with a shebang line like #!/usr/bin/python (not #!/usr/bin/env python), it should basically work, even with the new default.

I like to use --system-site-packages anyway, because it's a pain to install things like numpy and matplotlib repeatedly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.