mpi4py fails to build - uses system cython

[johannesring]

2014/12/09 18:02:56 - INFO: [package:run_job] running install
2014/12/09 18:02:56 - INFO: [package:run_job] running build
2014/12/09 18:02:56 - INFO: [package:run_job] running build_src
2014/12/09 18:02:56 - INFO: [package:run_job] cythonizing
'mpi4py.MPI.pyx' -> 'mpi4py.MPI.c' 2014/12/09 18:02:56 - INFO:
[package:run_job] Traceback (most recent call last): 2014/12/09
18:02:56 - INFO: [package:run_job]   File "setup.py", line 624, in
<module> 2014/12/09 18:02:56 - INFO: [package:run_job]     main()
2014/12/09 18:02:56 - INFO: [package:run_job]   File "setup.py", line
621, in main 2014/12/09 18:02:56 - INFO: [package:run_job]
run_setup() 2014/12/09 18:02:56 - INFO: [package:run_job]   File
"setup.py", line 493, in run_setup 2014/12/09 18:02:56 - INFO:
[package:run_job]     **metadata) 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/tmp/mpi4py-dps5vsljzl7g-1/conf/mpidistutils.py",
line 441, in setup 2014/12/09 18:02:56 - INFO: [package:run_job]
return fcn_setup(**attrs) 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/core.py",
line 151, in setup 2014/12/09 18:02:56 - INFO: [package:run_job]
dist.run_commands() 2014/12/09 18:02:56 - INFO: [package:run_job]
File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/dist.py",
line 953, in run_commands 2014/12/09 18:02:56 - INFO:
[package:run_job]     self.run_command(cmd) 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/dist.py",
line 972, in run_command 2014/12/09 18:02:56 - INFO:
[package:run_job]     cmd_obj.run() 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/command/install.py",
line 563, in run 2014/12/09 18:02:56 - INFO: [package:run_job]
self.run_command('build') 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/cmd.py",
line 326, in run_command 2014/12/09 18:02:56 - INFO:
[package:run_job]     self.distribution.run_command(command) 2014/12/09
18:02:56 - INFO: [package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/dist.py",
line 972, in run_command 2014/12/09 18:02:56 - INFO:
[package:run_job]     cmd_obj.run() 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/command/build.py",
line 127, in run 2014/12/09 18:02:56 - INFO: [package:run_job]
self.run_command(cmd_name) 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/cmd.py",
line 326, in run_command 2014/12/09 18:02:56 - INFO:
[package:run_job]     self.distribution.run_command(command) 2014/12/09
18:02:56 - INFO: [package:run_job]   File
"/srv/groot/blechta/.hashdist/bld/python/ctzx2dm36c6d/lib/python2.7/distutils/dist.py",
line 972, in run_command 2014/12/09 18:02:56 - INFO:
[package:run_job]     cmd_obj.run() 2014/12/09 18:02:56 - INFO:
[package:run_job]   File "setup.py", line 594, in build_sources
2014/12/09 18:02:56 - INFO: [package:run_job]     wdir='src',
force=cmd.force, VERSION=CYTHON) 2014/12/09 18:02:56 - INFO:
[package:run_job]   File "setup.py", line 573, in run_cython 2014/12/09
18:02:56 - INFO: [package:run_job]     wdir=wdir) 2014/12/09 18:02:56 -
INFO: [package:run_job]   File
"/srv/groot/blechta/.hashdist/tmp/mpi4py-dps5vsljzl7g-1/conf/cythonize.py",
line 9, in cythonize 2014/12/09 18:02:56 - INFO: [package:run_job]
from Cython.Compiler.Main import \ 2014/12/09 18:02:56 - INFO:
[package:run_job]   File
"/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 18, in
<module> 2014/12/09 18:02:56 - INFO: [package:run_job]     import Code
2014/12/09 18:02:56 - INFO: [package:run_job]
ImportError: /usr/lib/pymodules/python2.7/Cython/Compiler/Code.so:
undefined symbol: PyUnicodeUCS4_DecodeUTF8 2014/12/09 18:02:56 - ERROR:
[package:run_job] Command '[u'/bin/bash', '_hashdist/build.sh']'
returned non-zero exit status 1 2014/12/09 18:02:56 - ERROR:
[package:run_job] command failed (code=1); raising

[blechta]

I should add that I had previously my own mpi4py in PYTHONPATH interacting with hashdist so I removed it. Should I remove something from ~/.hashdist to try again from scratch?

[ahmadia]

Are you using a host python or one that hashdist built? Is this using the FEniCS script?

[johannesring]

@blechta - Yes, you can do hit purge mpi4py/<hash>, where <hash> is the build artifact you want to remove. You can find it in ~/.hashdist/bld/mpi4py.

[blechta]

@ahmadia https://bitbucket.org/fenics-project/fenics-install/src/5ce47af8a681cd266aba1b273b22bc65f8e50422/fenics-install.sh?at=master. It builds Python using hashdist.

@johannesring Still the same error. Shouldn't I start from scratch deleting ~/.hashdist?

[johannesring]

No, I don't think deleting ~/.hashdist will help.

[johannesring]

@ahmadia - The script builds its own python using hashdist. Here is the profile for Darwin.

[blechta]

@johannesring Why referring to Darwin script? I'm seeing the issue on Debian (Ubuntu 12.04).

[johannesring]

@blechta - Sorry, may fault! Here is the correct profile.

[ahmadia]

@blechta - The problem you're experiencing is due to Cython getting picked up from:
/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py

In general, this shouldn't happen, but there are a number of ways this can be triggered.

I'm guessing that somehow the HashDist-provided Python that we're building is not being considered executable or otherwise discovered by your system.

You can debug this with something like the following:

git clone https://github.com/hashdist/hashdist.git
git clone https://github.com/hashdist/hashstack.git
cd hashstack
curl https://bitbucket.org/fenics-project/fenics-install/raw/5ce47af8a681cd266aba1b273b22bc65f8e50422/profiles/fenics.Debian.yaml -o default.yaml
../hashdist/bin/hit build --debug 

This will resume the build, dropping you into the failing mpi4py installation before it starts. You should be able to reproduce the problem by executing: /bin/bash _hashdist/build.sh, and with a little work, diagnose how Cython is coming in from the wrong place.

A couple of things to try:

which python
which ${PYTHON}
${PYTHON} -v -c "import Cython"
python -v -c "import Cython"
python -s -v -c "import Cython"
python -m site --user-site

Please let me know if you uncover anything.

[certik]

Hi @blechta, welcome. Thanks for reporting the bug. If you have any questions about hashdist, please ask.

[blechta]

@ahmadia

[HASHDIST DEBUG] python -m site --user-site
/usr/users/blechta/.local/lib/python2.7/site-packages

When I removed ~/.local it seems to work. Do you know where could be a problem?

[ahmadia]

We need to set an environment variable in hashstack to protect you from
your own user directory :). It's our bug. Thanks for the report, I or
another dev will fix it today.

On Thursday, December 11, 2014, Jan Blechta notifications@github.com
wrote:

@ahmadia https://github.com/ahmadia

[HASHDIST DEBUG] python -m site --user-site
/usr/users/blechta/.local/lib/python2.7/site-packages

When I removed ~/.local it seems to work. Do you know where could be a
problem?

—
Reply to this email directly or view it on GitHub
#583 (comment).

[johannesring]

A similar problem was reported today. @ahmadia - have you looked at this? Do you know how to fix it?

[ahmadia]

The idea I had in mind was to add a flag to our Python build recipes (setuptools/distutils) that tells Python not to search in user directories. Per PEP 370

The user site directory can be suppressed with a new option -s or the environment variable PYTHONNOUSERSITE . The feature can be disabled globally by setting site.ENABLE_USER_SITE to the value False . It must be set by editing site.py . It can't be altered in sitecustomize.py or later.

This should probably be set off by default in our Python install.