Cython installation fails with pip - Cython.Runtime.refnanny extension

[VolkerH]

I'm not sure whether this is a Cython or a pip issue, but building Cython using
pip install --upgrade cython
failed for me on two quite different python-2.7 installations during the step of building the Cython.Runtime.refnanny extension.
One machine was Mac OS X Sierra with python installed from homebrew and using virtualenv.
The other machine was on our linux cluster running CentOs. Virtualenv was present in both cases.

bash-4.1$ pip install --upgrade cython
Downloading/unpacking cython
  Downloading Cython-0.25.tar.gz (1.7Mb): 1.7Mb downloaded
  Running setup.py egg_info for package cython
    Unable to find pgen, not compiling formal grammar.

 .... skipping lots ...


    gcc -pthread -shared build/temp.linux-x86_64-2.7/home/xxx/virtualenvs/volker-py27/build/cython/Cython/Compiler/Code.o -L/g/software/linux/pack/python-2.7/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/Cython/Compiler/Code.so
    building 'Cython.Runtime.refnanny' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/g/software/linux/pack/python-2.7/include/python2.7 -c /home/xxx/virtualenvs/volker-py27/build/cython/Cython/Runtime/refnanny.c -o build/temp.linux-x86_64-2.7/home/xxx/virtualenvs/volker-py27/build/cython/Cython/Runtime/refnanny.o
    gcc: /home/xxx/virtualenvs/volker-py27/build/cython/Cython/Runtime/refnanny.c: No such file or directory 

pathnames above slightly anonymized

[robertwb]

Any ideas what would differ between Ubuntu and CentOS, or Sierra and El Capitan, that could cause this? Could you post a bit more of your log? Do you see

[ 9/10] Cythonizing /usr/local/google/home/robertwb/scratch/refnanny/cython/Cython/Runtime/refnanny.pyx

[totaam]

Nope, the file never gets cythonized, it does work if I do it by hand.
FWIW: I have about 50 different build distro VMs (including OSX and win32), and only CentOS 6.x fail out of all of them (all versions from 6.4 to 6.8, and both i386 and x86_64).

[robertwb]

I was able to reproduce this on a fresh CentOS 6.x install. Still scratching my head as to what the difference is though...

[robertwb]

And now I can't get it to fail (and getting virtualenv up and running on CentOS is much more of a pain than I had hoped). Any poking around you could do here would be greatly appreciated.

[robertwb]

I created a fresh CentOS 6.x VM

uname -a
Linux centos 2.6.32-642.6.2.el6.x86_64 #1 SMP Wed Oct 26 06:52:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

installed python and virtualevn according to https://www.digitalocean.com/community/tutorials/how-to-set-up-python-2-7-6-and-3-3-3-on-centos-6-4

In a virtual environment, both pip install --no-use-wheel cython and cloning Cython and doing python setup.py build succeed.

[VolkerH]

Meanwhile I tried by downloading the master branch onto my Mac and building it using python setup.py install. Same error.
When I downloaded the 0.24.x branch the build succeeded without issue.
(I was previously on a very outdated version).
I will try with CentOS on our cluster tomorrow.

[totaam]

Wait. Maybe it's working because of virtualenv?
Is Python 2.6 no longer supported? Why not install directly using the system python?
(Cython 0.24.x worked with Python 2.6 and I didn't see anything in the release announcements to indicate that it had been dropped)

[robertwb]

Python 2.6 is still supported, though in my CentoOS install the system Python did not have development headers which is of course a blocker for using Cython.

[totaam]

yum install python-devel gcc
IIRC, that's all I needed on centos 6.x to build Cython.

[robertwb]

I think I figured this out. It was due to a really old version of setuptools being used.