Recognise virtualenvs #1171

Closed
takluyver opened this Issue Dec 17, 2011 · 13 comments

Projects

None yet

4 participants

@takluyver
IPython member

A recurring issue, which I've just added to the FAQ - a standard installation of IPython won't recognise when you're in a virtualenv. Various workarounds are showing up, but it would be good to work out of the box.

The root of the problem is that the installation sets the shebang line of ipython to #!/usr/bin/python. If it's #!/usr/bin/env python, virtualenvs work as expected. (I don't know what the status is on Windows)

A few possible ways to deal with this:

  • Get an option added to distutils/packaging/setuptools/distribute to control the shebang line written. Long term at best.
  • For Linux distributions, the shebang could be replaced as a step in the packaging.
  • Contribute ipython-awareness to virtualenv, so if IPython is installed, it will copy the script and update the shebang line when you create a virtualenv.
  • Detect when we're in a virtualenv, and add the relevant paths to sys.path. This is suboptimal, because it only affects importable libraries, not the interpreter itself (e.g. if I make a Python 2.6 virtualenv, IPython will still start with the default Python 2.7).
@fperez
IPython member

Mmh, how is this problem unique to us? If this is happening, then it happens to any python project that installs executable scripts. Granted, many python projects are indeed just libraries with minimal to no user-facing executable code, but still: nose, cython are two that immediately come to mind with this same situation.

How does everyone else handle this problem? I'm not claiming it's not an issue, just thinking that somebody out there must have figured this one out...

@takluyver
IPython member

I think for most things, it makes sense to run it with the system Python, ignoring locally installed libraries (e.g. if I run a mercurial command, a virtualenv shouldn't interfere with it). I suspect nose has the same problem in a standard installation, and I've also run into it with sphinx-build.

@fperez
IPython member

OK, I figured it out. The solution (not just for IPython, but for installying any python package that has scripts in a virtualenv) is to do the build step separately:

./setup.py build --executable "/usr/bin/env python"
./setup.py install

This sets the shebang line correctly once the scripts get installed.

Since this is standard distutils, I think we can actually close the issue here. Feel free to put this tip up on the faq, as it's certainly not very obvious. I had an inkling of the solution because I had a vague memory of being able to control that line years ago when I was building RPMs with distutils for a lab where I had a bunch of Fedora machines. But otherwise it's not easy to just stumble upon this solution.

@fperez fperez closed this Dec 18, 2011
@minrk
IPython member

Yes, this has never had anything to do with IPython, it's always affected all distutils-installed scripts, but IPython is often the only such script inherited into a virtualenv.

It should be considered a virtualenv bug that they don't patch the #! line of python scripts.

@takluyver
IPython member

So it turns out we can set this without requiring the user to pass an extra command line argument (http://stackoverflow.com/a/1719991/434217), but it only works for distutils style 'scripts', not setuptools 'entry_points'. I've got a branch for that, and I'll look into it more closely for 0.13.

I'm reopening this and assigning it to myself on the grounds that I think we can improve on the current situation.

@takluyver takluyver reopened this Dec 18, 2011
@takluyver takluyver was assigned Dec 18, 2011
@fperez
IPython member
@juliantaylor

fyi, the debian packages already use env python as shebang for ipython since a while [0], but for the entry point script used in py3 I have found no other way than to sed the resulting scripts [1].
what I find weird, but probably is a distutils issue, is that the script is only created on install, not on build.
This also makes testing a bit more ackward as iptest3 is only created after installation.

[0] http://anonscm.debian.org/viewvc/python-modules/packages/ipython/trunk/debian/rules?r1=15749&r2=16684
[1] http://anonscm.debian.org/viewvc/python-modules/packages/ipython/trunk/debian/rules?r1=19668&r2=19670

@takluyver
IPython member
@juliantaylor

I don't think we need special handling for the new venv behavior. That the system wide ipython is not importable with --no-site-packages is the expected (and likely wanted) behavior.
installing ipython in that venv with easy_install results in the correct shebang.

@takluyver
IPython member
@takluyver
IPython member

Fixed, as much as it's going to be for now, by PR #1388, so I'm closing this again.

@takluyver takluyver closed this Apr 23, 2012
@ghost

A helpful tip for others on Linux who find the version of IPython shipped in the distro now fails with importerror. Virtualenv changed the default behaviour from --system-site-packages to --no-site-packages. For IPython this means that when you run it from a terminal where the virtualenv is in your path, it will not pick up the ipython libs. Either install IPython in your virtualenv, or make your virtualenv use the system-site-packages.

@takluyver
IPython member

If your systemwide IPython is installed with a shebang line like #!/usr/bin/python (not #!/usr/bin/env python), it should basically work, even with the new default.

I like to use --system-site-packages anyway, because it's a pain to install things like numpy and matplotlib repeatedly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment