Installation error

[qmpotential]

Installation of the underworld on Linux cluster.

1. Load necessary modules first: intel/19.0.5, mvapich2/2.3.3/intel-19.0.5, petsc/3.12.3/intel-19.0.5-mvapich-2.3.3, metis/5.1.0/intel-19.0.5, hdf5/1.10.6/intel-19.0.5-mvapich-2.3.3, parmetis/4.0.3/intel-19.0.5-mvapich-2.3.3, hypre/2.18.2/intel-19.0.5-mvapich-2.3.3, superlu-dist/6.1.1/intel-19.0.5-mvapich-2.3.3, mumps/5.2.0/intel-19.0.5-mvapich-2.3.3, fftw/3.3.8/intel-19.0.5-mvapich-2.3.3, suite-sparse/5.6.0/intel-19.0.5, gcc/9.3.0, cmake/3.16.2/gcc-9.3.0

2. Create a conda environment: conda create -n UWORLD python=3.11

3. Activate conda environment: conda activate UWORLD

4. Install ninja package: conda install ninja (if you install it with pip it is not going to see that)

5. Install underworld: pip install -v git+https://github.com/underworldcode/underworld2

Successfully installed h5py-3.10.0 mpi4py-3.1.5 pint-0.23 scipy-1.12.0 typing-extensions-4.9.0 underworld-2.15.1b0

7. Install lavavu: pip install --user lavavu

8. Test the installation:

(UWORLD)[akjhdah]$ python3.11
Python 3.11.0 | packaged by conda-forge | (main, Oct 25 2022, 06:24:40) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import underworld as uw
print (uw.version)
3.15.1b
from underworld import UWGeodynamics as GEO
loaded rc file /site-packages/underworld/UWGeodynamics/uwgeo-data/uwgeodynamicsrc
print (GEO.version)
2.15.1b
from underworld import visualisation as vis
import lavavu
lv = lavavu.Viewer()

That is all!

[julesghub]

Hi @qmpotential,

I can't make out the error from this log, is there more output information available. If so, please send it through.
Also what version of the code are you installing from source? Which git revision?

Another strategy to overcome this is to use the Underworld2's container images. You could use podman (preferred container software) or docker to quickly spin up an Underworld2 instance.
See - https://hub.docker.com/r/underworldcode/underworld2/tags for the available versions. Note the podman / docker commands can be used interchangeably for Underworld usage.

Hope this help.

[julesghub]

Thanks for letting us know pip install underworld command isn't working.

Underworld can't build on platforms like Colab because our dependency code, PETSc doesn't build on colab.
Hence why we have the container solution which works well on other cloud platforms.

Are you wanting to build the code on a local machine? How are you wanting to use it?

Make sure you have petsc installed and the environment variable PETSC_DIR pointing to the installation.
Here's some general information about it https://www.underworldcode.org/setting-up-underworld-dependencies/
Note: The PETSc version is old in the link. The latest underworld runs on petsc-3.19.

[julesghub]

Try install the python development header.

Something like
yum -y install python-devel

That will include "Python.h"

[NTIaN7]

Hi @qmpotential,

Would it be convenient for you to provide complete compilation error information? Sending all installation results to a file instead of a screen may better solve the problem.

N.

[julesghub]

Hi Oleg,

Now you're missing the development header for an MPI implementation.
Try something like
sudo yum install openmpi
to install it.

All Underworld2 build dependencies packages are listed here.
https://github.com/underworldcode/underworld2/blob/master/Installation.rst#build-environment

Also there is this guide to installing most of those dependencies under Ubuntu. Although you're on RHEL it will still be helpful. https://www.underworldcode.org/setting-up-underworld-dependencies/

[julesghub]

I'm not familiar with RHEL8 packages unfortunately.
What openmpi or mpich development packages can you install with RHEL8?

E.g. sudo yum search openmpi

[julesghub]

The python package ninja is not being found

  CMake Error at CMakeLists.txt:7 (project):
    Running

     '/home/olegsupp/.local/bin/ninja' '--version'

    failed with:

     Traceback (most recent call last):
      File "/home/olegsupp/.local/bin/ninja", line 5, in <module>
        from ninja import ninja

    ModuleNotFoundError: No module named 'ninja'

Run
pip install ninja

For a complete list of build requirements, both programs and python packages, for the current docker image see here.

underworld2/docs/development/docker/underworld2/Dockerfile

Line 83 in e62b5de

&& DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \

I'll update the code to ensure ninja is a requirement in the build phase.
We are getting close to an install.

[julesghub]

What is the new error exactly? A log file would be useful.

You can check if you've installed ninja and mpi as follows
ninja --version
mpicc --show

Did you have any luck using the docker solution?

[NTIaN7]

Can I install it via the Conda environment somehow?

Hi @qmpotential
These issues should be caused by environmental dependence. Personally, I would recommend using the Intel MPI suite to compile PETSc and using Conda to manage the Python environment and its dependencies. I do not recommend using OpenMPI or MPICH and PETSc installed by Conda as their performance may be poor.

N.

[NTIaN7]

@qmpotential
Congratulations! The installation seemed smooth this time. But one thing worth noting is that h5py is automatically installed as a dependency of underworld2 and is usually only serial. It can be detected through the following code, True means parallel support:

conda activate xxx
import h5py
h5py.get_config().mpi

You can refer here to install h5py that supports parallel IO.

CC=mpicc HDF5_MPI="ON" HDF5_DIR=/path/to/your/hdf5/install/ pip install --no-binary=h5py h5py

[qmpotential]

@NTIaN7

That is great! Thank you so much for the notice. I will try that and get back to you!