New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip and virtualenv --system-site-packages #1408

Closed
geertj opened this Issue Dec 31, 2013 · 8 comments

Comments

Projects
None yet
4 participants
@geertj

geertj commented Dec 31, 2013

If you create a virtualenv with --system-site-packages, then "pip install" will try to install in the system directory, not in the virtualenv.

Previously, it wasn't a problem because the "virtualenv" command would install pip into the virtualenv itself, even if pip is already present on the system. The pip inside the virtualenv does correctly install into the virtualenv. (Not sure how this works exactly)

However, on Python3.3+, virtualenv is merged into Python and shipped as pyvenv. This command does not install pip into the virtualenv. To get pip into the virtualenv, you can either use:

  • python get-pip.py
  • python -m ensurepip # python 3.4

The difference is that both of these will not install pip into the virtualenv, if pip is already present on the system.

Not sure what the best solution is, or whether this is a bug in get-pip/ensurepip, or pip itself. But it seems to me that if pip could install in the virtualenv instead of the system directory, this would be solved.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Dec 31, 2013

Contributor

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

virtualenv wasn't "merged" per se into Python3.3+. you can still use virtualenv with Python3.3+.

at some point, I think the plan for pyenv is to get pip installed by default, but not sure when that will be? @dstufft ?

Contributor

qwcode commented Dec 31, 2013

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

virtualenv wasn't "merged" per se into Python3.3+. you can still use virtualenv with Python3.3+.

at some point, I think the plan for pyenv is to get pip installed by default, but not sure when that will be? @dstufft ?

@qwcode qwcode closed this Dec 31, 2013

@geertj

This comment has been minimized.

Show comment
Hide comment
@geertj

geertj Dec 31, 2013

@qwcode pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

In a virtualenv with --system-site-packages, sys.prefix = '/path/to/venv'. So how does pip even find out where the real Python is installed?

geertj commented Dec 31, 2013

@qwcode pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

In a virtualenv with --system-site-packages, sys.prefix = '/path/to/venv'. So how does pip even find out where the real Python is installed?

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Dec 31, 2013

Contributor

pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

ok, you're wording confused me. thought you're saying it wasn't and you had to explicitly run python -m ensurepip

So how does pip even find out where the real Python is installed?

sys.real_prefix, which is determined from a special orig-prefix.txt file in the virtualenv lib dir.
it's not pip so much doing anything, but the custom site.py file in the virtualenv altering the python sys.path.

Contributor

qwcode commented Dec 31, 2013

pyvenv does install pip by default in Python 3.4. This is done through the "ensurepip" module.

ok, you're wording confused me. thought you're saying it wasn't and you had to explicitly run python -m ensurepip

So how does pip even find out where the real Python is installed?

sys.real_prefix, which is determined from a special orig-prefix.txt file in the virtualenv lib dir.
it's not pip so much doing anything, but the custom site.py file in the virtualenv altering the python sys.path.

@geertj

This comment has been minimized.

Show comment
Hide comment
@geertj

geertj Jan 1, 2014

@qwcode there is a bootstrap problem here though..

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

How do you get pip into the virtualenv though? Both get-pip.py and ensurepip exit if a system pip is installed, even if a virtualenv is currently activated.

The only way I found is to use the old "virtualenv", but that has been superseded since Python 3.3 with pyvenv.

geertj commented Jan 1, 2014

@qwcode there is a bootstrap problem here though..

pip will install into the Python it is installed to (whether a real python, or a virtualenv/pyenv Python).
that's how it works. I don't see that changing.

How do you get pip into the virtualenv though? Both get-pip.py and ensurepip exit if a system pip is installed, even if a virtualenv is currently activated.

The only way I found is to use the old "virtualenv", but that has been superseded since Python 3.3 with pyvenv.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Jan 1, 2014

Contributor

Both get-pip.py and ensurepip exit if a system pip is installed,

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

Contributor

qwcode commented Jan 1, 2014

Both get-pip.py and ensurepip exit if a system pip is installed,

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

@geertj

This comment has been minimized.

Show comment
Hide comment
@geertj

geertj Jan 1, 2014

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

Thanks that works.

But wouldn't you agree this is a very common use case that should work a little bit simpler? --system-site-packages is a common requirement with more complicated, non-pip installable packages.

Or should I file a bug against Python3.4? "python -mensurepip" installs pip when not --system-site-packages, but skips it if ---system-site-packages and a system pip is available.

geertj commented Jan 1, 2014

oh.... I see, I just tried this in py33.
just download the pip distribution from pypi, unpack it, and do python setup.py install using the venv python

Thanks that works.

But wouldn't you agree this is a very common use case that should work a little bit simpler? --system-site-packages is a common requirement with more complicated, non-pip installable packages.

Or should I file a bug against Python3.4? "python -mensurepip" installs pip when not --system-site-packages, but skips it if ---system-site-packages and a system pip is available.

@davidkhess

This comment has been minimized.

Show comment
Hide comment
@davidkhess

davidkhess Jan 8, 2016

For any one else who stumbles across this issue, here's a workaround:

  1. Build the venv without --system-site-packages
  2. After the venv is built, edit pyvenv.cfg and change
include-system-site-packages = false

to

include-system-site-packages = true

Then you will get a venv with pip in it that installs in the venv and uses the system site-packages.

davidkhess commented Jan 8, 2016

For any one else who stumbles across this issue, here's a workaround:

  1. Build the venv without --system-site-packages
  2. After the venv is built, edit pyvenv.cfg and change
include-system-site-packages = false

to

include-system-site-packages = true

Then you will get a venv with pip in it that installs in the venv and uses the system site-packages.

@mlenzen

This comment has been minimized.

Show comment
Hide comment
@mlenzen

mlenzen Nov 11, 2016

I had a the same or a similar issue. pip in a virtual env with --system-site-packages would try to install packages to the system site-packages. pyvenv --system-site-packages env didn't install a new version of pip in env/bin. @davidkhess solution of editing pyvenv.cfg after creating the env without --system-site-packages worked.

Using 3.5.1

mlenzen commented Nov 11, 2016

I had a the same or a similar issue. pip in a virtual env with --system-site-packages would try to install packages to the system site-packages. pyvenv --system-site-packages env didn't install a new version of pip in env/bin. @davidkhess solution of editing pyvenv.cfg after creating the env without --system-site-packages worked.

Using 3.5.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment