Bootstraping fails on SLES 11 SP3 #418

omula · 2014-05-22T16:05:24Z

I have tried to follow suggestion on #373 but didn't work

== 2014-05-22 17:38:25,678 bootstrap_eb.environment INFO Environment variable PYTHONPATH set to /home/xadmin/.local/easybuild/software/EasyBuild/1.12.1/lib/python2.6/site-packages:/tmp/tmpXR6Ehy/lib64/python2.6/site-packages/distribute-0.
6.34-py2.6.egg
== 2014-05-22 17:38:26,042 bootstrap_eb.run ERROR EasyBuild crashed with an error (at easybuild/tools/run.py:382 in parse_cmd_output): cmd ". /etc/profile.d/modules.sh &&  python setup.py install --prefix=/home/xadmin/.local/easybuild/sof
tware/EasyBuild/1.12.1 " exited with exitcode 1 and output:
Installing with setuptools.setup...
Installing version 1.12.1.0 (required versions: API >= 1, easyblocks >= 1.12)
running install
Checking .pth file support in /home/xadmin/.local/easybuild/software/EasyBuild/1.12.1/lib64/python2.6/site-packages/
/usr/bin/python -E -c pass
TEST FAILED: /home/xadmin/.local/easybuild/software/EasyBuild/1.12.1/lib64/python2.6/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /home/xadmin/.local/easybuild/software/EasyBuild/1.12.1/lib64/python2.6/site-packages/

and your PYTHONPATH environment variable currently contains:

    '/home/xadmin/.local/easybuild/software/EasyBuild/1.12.1/lib/python2.6/site-packages:/tmp/tmpXR6Ehy/lib64/python2.6/site-packages/distribute-0.6.34-py2.6.egg'

Here are some of your options for correcting the problem:

* You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

* You can add the installation directory to the PYTHONPATH environment
  variable.  (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

* You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:

  http://packages.python.org/distribute/easy_install.html#custom-installation-locations

Please make the appropriate changes for your system and try again.

The text was updated successfully, but these errors were encountered:

boegel · 2014-05-22T17:00:48Z

@omula: can you share the output of the command below on this system?

egrep -RI '/lib/|install_dir.*=' `python -c "import setuptools; print setuptools.__path__[0]"`

omula · 2014-05-22T17:02:11Z

/usr/lib64/python2.6/site-packages/setuptools/dist.py:                dist, args=["x"], install_dir=os.curdir, exclude_scripts=True,
/usr/lib64/python2.6/site-packages/setuptools/command/install_egg_info.py:        self.install_dir = None
/usr/lib64/python2.6/site-packages/setuptools/command/develop.py:        if normalize_path(os.path.join(self.install_dir, self.egg_path)) != target:
/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py:        self.install_dir = self.script_dir = self.exclude_scripts = None
/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py:        self.install_dir = instdir
/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py:            install_dir = '$base/lib/python$py_version_short/site-packages',
/usr/lib64/python2.6/site-packages/setuptools/command/easy_install.py:        install_dir = '$base/Lib/site-packages',
/usr/lib64/python2.6/site-packages/setuptools/command/bdist_egg.py:            self.call_command('install_scripts',install_dir=script_dir,no_ep=1)

boegel · 2014-05-22T17:54:20Z

Hmm, I see the same hardcoding of /lib/ here as is done in the version I have. Then where is the lib64 coming from...

@omula: what does this yield?

grep -RI 'lib64' -C 3 `python -c "import setuptools; print setuptools.__path__[0]"

omula · 2014-05-22T18:32:46Z

It returns nothing.

xadmin@login:$ grep -RI 'lib64' -C 3 python -c "import setuptools; print setuptools.__path__[0]"
xadmin@login:$

omula · 2014-05-22T18:34:55Z

The python command without grep gives something else
xadmin@login:~$ python -c "import setuptools; print setuptools.path[0]"
/usr/lib64/python2.6/site-packages/setuptools

omula · 2014-05-22T18:59:56Z

I have found a pair of links which might be useful

http://serverfault.com/questions/60619/fedora-usr-lib-vs-usr-lib64
http://en.opensuse.org/openSUSE:Packaging_Python

omula · 2014-05-22T19:02:51Z

I think i've finally found were it comes from
/tmp/tmpUfIp0O/lib64/python2.6/site-packages/distribute-0.6.34-py2.6.egg/setuptools/command/easy_install.py: install_dir = '$base/lib/python$py_version_short/site-packages',

boegel · 2014-05-22T19:04:22Z

@omula: but this is still hard-coding /lib/, which is exactly what EasyBuild does (see https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/pythonpackage.py#L58)

boegel · 2014-05-22T21:01:36Z

@omula: can you try this?
https://gist.github.com/boegel/f26b64322eb2a02981f5

one possible way out if for PythonPackage to use --install-lib when installing Python packages, which should enforce the use of /lib/, even on SLES

this needs to be looked into though...

omula · 2014-05-22T21:04:49Z

@boegel This is the output
python setup.py install --prefix=/tmp
running install
/usr/lib64/python2.6/site-packages/setuptools/package_index.py:7: DeprecationWarning: the md5 module is deprecated; use hashlib instead
from md5 import md5
Checking .pth file support in /tmp/lib64/python2.6/site-packages/
/usr/bin/python -E -c pass
TEST FAILED: /tmp/lib64/python2.6/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/tmp/lib64/python2.6/site-packages/

boegel · 2014-05-22T21:23:42Z

@omula: what about:

mkdir -p /tmp/lib/python2.6/site-packages
export PYTHONPATH=/tmp/lib/python2.6/site-packages:$PYTHONPATH
python setup.py install --prefix=/tmp --install-lib=lib

omula · 2014-05-23T06:10:43Z

@boegel: this is the output

xadmin@login:/tmp/tmpr7UyzF$ mkdir -p /tmp/lib/python2.6/site-packages
xadmin@login:/tmp/tmpr7UyzF$ export PYTHONPATH=/tmp/lib/python2.6/site-packages:$PYTHONPATH
xadmin@login:/tmp/tmpr7UyzF$ python setup.py install --prefix=/tmp --install-lib=lib
running install
/usr/lib64/python2.6/site-packages/setuptools/package_index.py:7: DeprecationWarning: the md5 module is deprecated; use hashlib instead
from md5 import md5
Checking .pth file support in lib/
/usr/bin/python -E -c pass
TEST FAILED: lib/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

lib/

and your PYTHONPATH environment variable currently contains:

'/tmp/lib/python2.6/site-packages:'

boegel · 2014-05-23T07:19:20Z

@omula brought this forward, which we used to rely on in PythonPackage:

python -c "import distutils.sysconfig; print distutils.sysconfig.get_python_lib(prefix='$HOME/.local'); "

for me:

Mac OS X 10.9.3: /Users/kehoste/.local/lib/python2.7/site-packages
SL6: /user/home/gent/vsc400/vsc40000/.local/lib/python2.6/site-packages
openSuSE 13.1: $HOME/.lib/python2.7/site-packages

@pforai, @pescobar: can you guys try this too an a couple of your systems, mentioning which system it is?
@omula: can you paste here what this prints on your system(s)?

pescobar · 2014-05-23T07:21:55Z

Ubuntu12.04: /home/escobar/.local/lib/python2.7/site-packages
CentOS6.5: /import/bc2/home/schwede/escobar/.local/lib/python2.6/site-packages
Centos5: /import/bc2/home/schwede/escobar/.local/lib/python2.5/site-packages

boegel · 2014-05-23T07:35:16Z

on JUROPA (SLES 11):

$ cat /etc/SuSE-release 
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1
$ python -c "import distutils.sysconfig; print distutils.sysconfig.get_python_lib(prefix='$HOME/.local'); "
$HOME/.local/lib64/python2.6/site-packages

on RHEL6.5 (JUQUEEN login nodes):

$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.5 (Santiago)
$ python -c "import distutils.sysconfig; print distutils.sysconfig.get_python_lib(prefix='$HOME/.local'); "
$HOME/.local/lib/python2.6/site-packages

boegel · 2014-05-26T09:53:45Z

fixed by #419, thanks for reporting this @omula!

boegel mentioned this issue May 23, 2014

fix det_pylibdir, determine Python lib dir via distutils, which works cross-OS #419

Merged

boegel closed this as completed May 26, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bootstraping fails on SLES 11 SP3 #418

Bootstraping fails on SLES 11 SP3 #418

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 23, 2014

boegel commented May 23, 2014

pescobar commented May 23, 2014

boegel commented May 23, 2014

boegel commented May 26, 2014

Bootstraping fails on SLES 11 SP3 #418

Bootstraping fails on SLES 11 SP3 #418

Comments

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

boegel commented May 22, 2014

omula commented May 22, 2014

boegel commented May 22, 2014

omula commented May 23, 2014

boegel commented May 23, 2014

pescobar commented May 23, 2014

boegel commented May 23, 2014

boegel commented May 26, 2014