Skip to content

unable to install on RHEL 5.6 #367

Closed
jseabold opened this Issue Jul 6, 2012 · 13 comments

3 participants

@jseabold
statsmodels member
jseabold commented Jul 6, 2012

I've built Python2.7 and the entire scientific python stack from source. Everything was fine, but when I try to install statsmodels it fails.

python2.7 setup.py build

Succeeds and everything is fine, but when I try to install

[infometrics (js2796a) <statsmodels>] sudo python2.7 setup.py install
Warning: Assuming default configuration (scikits/{setup_scikits,setup}.py was not found)Appending scikits configuration to 
Ignoring attempt to set 'name' (from '' to 'scikits')
compiling '_configtest.c':

/* This file is generated from statsmodels/tools/_build.py to */
void do_nothing(void);
int main(void) {
    do_nothing();
    return 0;
}

C compiler: gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

compile options: '-c'
gcc: _configtest.c
success!
removing: _configtest.c _configtest.o
cython -o /home/js2796a/src/statsmodels/statsmodels/nonparametric/fast_linbin.c /home/js2796a/src/statsmodels/statsmodels/nonparametric/fast_linbin.pyx
Traceback (most recent call last):
  File "setup.py", line 221, in <module>
    cmdclass={'build_py': build_py})
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/core.py", line 152, in setup
    config = configuration()
  File "setup.py", line 179, in configuration
    config.add_subpackage(DISTNAME)
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 1002, in add_subpackage
    caller_level = 2)
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 971, in get_subpackage
    caller_level = caller_level + 1)
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 908, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "statsmodels/setup.py", line 25, in configuration
    config.add_subpackage('nonparametric')
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 1002, in add_subpackage
    caller_level = 2)
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 971, in get_subpackage
    caller_level = caller_level + 1)
  File "/usr/local/lib/python2.7/site-packages/numpy/distutils/misc_util.py", line 908, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "statsmodels/nonparametric/setup.py", line 29, in configuration
    cython(['fast_linbin.pyx'], working_path=cur_dir)
  File "/home/js2796a/src/statsmodels/tools/_build.py", line 110, in cython
    '-o', c_file, pyxfile])
  File "/usr/local/lib/python2.7/subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/local/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/local/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

For some reason the permissions are not getting handled correctly. I've never seen this before, and I didn't have this problem with any of the dozens of other packages I've installed so far. It appears to be some combination of problems with subprocess + permissions and the use relative filenames in nonparametric/setup.py, though I don't have a fix yet or why I'm seeing this.

@josef-pkt
statsmodels member

you could try to install from an sdist that already has the cython created c files

and check if a pure cython call works

@rgommers
statsmodels member
rgommers commented Jul 7, 2012

Looks like subprocess and sudo is not a nice combination, http://stackoverflow.com/questions/567542/running-a-command-as-a-super-user-from-a-python-script

Did you manage to install with sudo on other systems?

@jseabold
statsmodels member
jseabold commented Jul 7, 2012

Yeah I saw similar posts as well. It works on every other system I've ever tried, but not RHEL. The 'easy' fix is to get rid of the cython step during install. Doing from an sdist doesn't work because the Cython call gets made regardless. We don't have a separate custom build command that makes the Cython extensions first if they need to be made, which is what we need to be doing.

@rgommers
statsmodels member
rgommers commented Jul 7, 2012

Another fix is to change permissions on site-packages, or install with --prefix=user. Installing with sudo isn't normally the right thing to do.

@josef-pkt
statsmodels member

But as far as I can see, cython is not used when it is not available, and the extension gets build from the c source.

At least that's what I infer from one of my virtualenv that has a c-compiler but no cython (unless it borrowed cython from another python version which it might be able to do since cython is on the windows path. - bad test setup)

We need to see that cython becomes optional, since we distribute the c sources.

@jseabold
statsmodels member
jseabold commented Jul 7, 2012

Err, I install everything system-wide with sudo every time. The permission issue is in the source folder trying to generate fast_linbin.c, which, incidentally, is already there because it was generated during the build command. We need to move the Cython generation to the main setup.py I think.

Hmm, it might exit gracefully if no Cython, so I misread that, but if it's there, it's called, and this is problematic here.

@josef-pkt
statsmodels member

I prefer if we find a solution with the current decentralized setup, since as you argued initially it's more modular and independent.

If cython is not available, then we just get an extra print and it's skipped from my reading of _build.

maybe as a solution, we need to get more special cases into _build. The advantage of doing it there, is that we have a central location for the build helper functions and platform specific code.

(we are leaving for the airport in 4 hours and I'm just working on closing my open programs.)

@josef-pkt
statsmodels member

another thought: does the setup configuration hand off extra command line arguments, eg. "use_cython" to rebuild the c files with cython, and use existing c files otherwise?

@jseabold
statsmodels member
jseabold commented Jul 7, 2012

I'll think about it. We can check in sys.argv for what command that was given. I've started to do this before, but I don't know distutils well enough to cover all the cases I don't think. I still have the feeling that there's a good reason that other packages aren't doing things like this. I'll look at it next week some time.

@jseabold
statsmodels member
jseabold commented Jul 7, 2012

Though now that I think about it adding a custom build command as others have done in the main setup.py doesn't affect the modularity. It will just generate the C sources. If you wanted, you could do the C code generation by hand or use the sdist.

@josef-pkt
statsmodels member

But I guess, it would still mean that the main setup.py needs to know about where all the cython .pyx files are.

@jseabold
statsmodels member
jseabold commented Jul 7, 2012

Yeah, unless we walk the source tree.

https://github.com/zeromq/pyzmq/blob/master/setup.py#L659

@jseabold
statsmodels member
jseabold commented Apr 6, 2014

Closing this. Not really our issue I guess.

@jseabold jseabold closed this Apr 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.