Permission error writing header files in pypy virtualenvs #510

qwcode opened this Issue Dec 6, 2013 · 4 comments


None yet

3 participants

qwcode commented Dec 6, 2013

for the trace, see:

the error occurs running this pip test:

although the test is intended to be a general wheel test, it happens to be using a wheel distribution with headers.

currently, in virtualenv for pypy, the include dir ends up getting linked directly to the global include, if, the global pypy has an include dir already.

e.g. here's an example virtualenv structure on the latest ubuntu using pypy2.2.1 from ppa:pypy/ppa, which does include header files in it's global /usr/lib/pypy/include

drwxrwxr-x 2 ubuntu ubuntu 4096 Dec  6 00:12 bin
lrwxrwxrwx 1 ubuntu ubuntu   21 Dec  6 00:12 include -> /usr/lib/pypy/include
drwxrwxr-x 2 ubuntu ubuntu 4096 Dec  6 00:12 lib_pypy
drwxrwxr-x 3 ubuntu ubuntu 4096 Dec  6 00:12 lib-python
drwxrwxr-x 3 ubuntu ubuntu 4096 Dec  6 00:12 site-packages

this is problematic if the global site is root permissions. header files can't be installed into include (note, I'm assuming that the pip wheel installer had it correct in trying to install to include/site)

for comparison, in linux cpython's, the linking occurs differently due to it following the "normal" distutil's schemes.

   python2.7 -> /usr/include/python2.7

it seems the logic needs to be updated to not link "include" dir itself, but the only the global files within.

P.S. as for why this was not a problem in pip's tests with pypy1.9, here's 3 theories:

  1. that distribution didn't include a global "include" dir
  2. the permissions were different previously for the pypy1.9 install on travis
  3. there's some other change in virtualenv (or wheel or pip), that I'm not seeing atm?

cc @dstufft , @alex, @pfmoore

pfmoore commented Dec 8, 2013

I agree with your analysis, linking the directory in the virtualenv to the global dir is clearly wrong. (Surely if it wasn't for the permission error, installing something in the virtualenv would put it in the global include directory, which is obviously wrong).

qwcode commented Dec 8, 2013

I recently hit this issue, working on my package cython-murmurhash. Any package which supplies the headers argument to setup should be affected.

This looks to me like the relevant part of the code:

Is this as simple as it looks?


The CPython include dir is something like include/python2.7 , while the PyPy include dir is just include/:

So, when this line calls copyfile:

On CPython, the dest arg is something link "venv/include/Python2.7". Since "venv/include" doesn't exist, it's created:

With PyPy, the dest arg has a value like "venv/include". So, "venv" exists, and the include dir itself is symlinked.

I suggest calling os.mkdir(inc_dir) here:

And then copying the contents of include, rather than the directory itself, here:

I don't understand the logic in the block starting at L1209 though, so I'm not sure how it should be updated.

@malb malb referenced this issue in sagemath/cysignals Feb 7, 2016

Installing headers #15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment