Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip and virtualenv --system-site-packages #1408

Closed
geertj opened this issue Dec 31, 2013 · 8 comments
Closed

pip and virtualenv --system-site-packages #1408

geertj opened this issue Dec 31, 2013 · 8 comments

Comments

@geertj
Copy link

@geertj geertj commented Dec 31, 2013

If you create a virtualenv with --system-site-packages, then "pip install" will try to install in the system directory, not in the virtualenv.

Previously, it wasn't a problem because the "virtualenv" command would install pip into the virtualenv itself, even if pip is already present on the system. The pip inside the virtualenv does correctly install into the virtualenv. (Not sure how this works exactly)

However, on Python3.3+, virtualenv is merged into Python and shipped as pyvenv. This command does not install pip into the virtualenv. To get pip into the virtualenv, you can either use:

  • python get-pip.py
  • python -m ensurepip # python 3.4

The difference is that both of these will not install pip into the virtualenv, if pip is already present on the system.

Not sure what the best solution is, or whether this is a bug in get-pip/ensurepip, or pip itself. But it seems to me that if pip could install in the virtualenv instead of the system directory, this would be solved.

@qwcode
Copy link
Contributor

@qwcode qwcode commented Dec 31, 2013

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

virtualenv wasn't "merged" per se into Python3.3+. you can still use virtualenv with Python3.3+.

at some point, I think the plan for pyenv is to get pip installed by default, but not sure when that will be? @dstufft ?

@qwcode qwcode closed this Dec 31, 2013
@geertj
Copy link
Author

@geertj geertj commented Dec 31, 2013

@qwcode pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

In a virtualenv with --system-site-packages, sys.prefix = '/path/to/venv'. So how does pip even find out where the real Python is installed?

@qwcode
Copy link
Contributor

@qwcode qwcode commented Dec 31, 2013

pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

ok, you're wording confused me. thought you're saying it wasn't and you had to explicitly run python -m ensurepip

So how does pip even find out where the real Python is installed?

sys.real_prefix, which is determined from a special orig-prefix.txt file in the virtualenv lib dir.
it's not pip so much doing anything, but the custom site.py file in the virtualenv altering the python sys.path.

@geertj
Copy link
Author

@geertj geertj commented Jan 1, 2014

@qwcode there is a bootstrap problem here though..

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

How do you get pip into the virtualenv though? Both get-pip.py and ensurepip exit if a system pip is installed, even if a virtualenv is currently activated.

The only way I found is to use the old "virtualenv", but that has been superseded since Python 3.3 with pyvenv.

@qwcode
Copy link
Contributor

@qwcode qwcode commented Jan 1, 2014

Both get-pip.py and ensurepip exit if a system pip is installed,

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

@geertj
Copy link
Author

@geertj geertj commented Jan 1, 2014

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

Thanks that works.

But wouldn't you agree this is a very common use case that should work a little bit simpler? --system-site-packages is a common requirement with more complicated, non-pip installable packages.

Or should I file a bug against Python3.4? "python -mensurepip" installs pip when not --system-site-packages, but skips it if ---system-site-packages and a system pip is available.

@davidkhess
Copy link

@davidkhess davidkhess commented Jan 8, 2016

For any one else who stumbles across this issue, here's a workaround:

  1. Build the venv without --system-site-packages
  2. After the venv is built, edit pyvenv.cfg and change
include-system-site-packages = false

to

include-system-site-packages = true

Then you will get a venv with pip in it that installs in the venv and uses the system site-packages.

@mlenzen
Copy link

@mlenzen mlenzen commented Nov 11, 2016

I had a the same or a similar issue. pip in a virtual env with --system-site-packages would try to install packages to the system site-packages. pyvenv --system-site-packages env didn't install a new version of pip in env/bin. @davidkhess solution of editing pyvenv.cfg after creating the env without --system-site-packages worked.

Using 3.5.1

@lock lock bot added the S: auto-locked label Jun 4, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.