Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Permission error writing header files in pypy virtualenvs #510

Open
qwcode opened this Issue · 4 comments

3 participants

@qwcode
Owner

for the trace, see: https://travis-ci.org/pypa/pip/jobs/14962187

the error occurs running this pip test:
https://github.com/pypa/pip/blob/1.5.X/tests/functional/test_install_wheel.py#L40

although the test is intended to be a general wheel test, it happens to be using a wheel distribution with headers.

currently, in virtualenv for pypy, the include dir ends up getting linked directly to the global include, if, the global pypy has an include dir already.

e.g. here's an example virtualenv structure on the latest ubuntu using pypy2.2.1 from ppa:pypy/ppa, which does include header files in it's global /usr/lib/pypy/include

drwxrwxr-x 2 ubuntu ubuntu 4096 Dec  6 00:12 bin
lrwxrwxrwx 1 ubuntu ubuntu   21 Dec  6 00:12 include -> /usr/lib/pypy/include
drwxrwxr-x 2 ubuntu ubuntu 4096 Dec  6 00:12 lib_pypy
drwxrwxr-x 3 ubuntu ubuntu 4096 Dec  6 00:12 lib-python
drwxrwxr-x 3 ubuntu ubuntu 4096 Dec  6 00:12 site-packages

this is problematic if the global site is root permissions. header files can't be installed into include (note, I'm assuming that the pip wheel installer had it correct in trying to install to include/site)

for comparison, in linux cpython's, the linking occurs differently due to it following the "normal" distutil's schemes.

bin
include
   python2.7 -> /usr/include/python2.7
lib

it seems the logic needs to be updated to not link "include" dir itself, but the only the global files within.

P.S. as for why this was not a problem in pip's tests with pypy1.9, here's 3 theories:
1) that distribution didn't include a global "include" dir
2) the permissions were different previously for the pypy1.9 install on travis
3) there's some other change in virtualenv (or wheel or pip), that I'm not seeing atm?

cc @dstufft , @alex, @pfmoore

@pfmoore
Owner

I agree with your analysis, linking the directory in the virtualenv to the global dir is clearly wrong. (Surely if it wasn't for the permission error, installing something in the virtualenv would put it in the global include directory, which is obviously wrong).

@qwcode
Owner
@syllog1sm

I recently hit this issue, working on my package cython-murmurhash. Any package which supplies the headers argument to setup should be affected.

This looks to me like the relevant part of the code:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1033

Is this as simple as it looks?

@syllog1sm

The CPython include dir is something like include/python2.7 , while the PyPy include dir is just include/:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1199

So, when this line calls copyfile:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1204

On CPython, the dest arg is something link "venv/include/Python2.7". Since "venv/include" doesn't exist, it's created:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L466

With PyPy, the dest arg has a value like "venv/include". So, "venv" exists, and the include dir itself is symlinked.

I suggest calling os.mkdir(inc_dir) here:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L466

And then copying the contents of include, rather than the directory itself, here:

https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1203

I don't understand the logic in the block starting at L1209 though, so I'm not sure how it should be updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.