PUMGen compilation and hdf5 on Shaheen

[vyasjc]

Hii All,

I am getting following error in PUMGen compilation on Shaheen. Any suggestion:

---

Currently Loaded Modulefiles:

1. modules/3.2.11.4 12) intel/19.0.5.281 23) alps/6.6.58-7.0.1.1_6.8__g437d88db.ari
2. craype-haswell 13) python/3.8.0-cdl 24) rca/2.2.20-7.0.1.1_4.51__g8e3fb5b.ari
3. craype-network-aries 14) cray-libsci/19.06.1 25) atp/3.6.4
4. craype/2.6.3 15) udreg/2.3.2-7.0.1.1_3.36__g8175d3d.ari 26) perftools-base/7.1.3
5. cray-mpich/7.7.11 16) ugni/6.0.14.0-7.0.1.1_7.38__ge78e5b0.ari 27) PrgEnv-intel/6.0.5
6. slurm/slurm 17) pmi/5.0.15 28) cmake/3.13.4
7. dws/3.0.28-7.0.1.1_6.35__ge55277c.ari 18) dmapp/7.1.1-7.0.1.1_4.52__g38cf134.ari 29) scons/3.0.1
8. eproxy/2.0.24-7.0.1.1_5.10__g45c478a.ari 19) gni-headers/5.0.12.0-7.0.1.1_6.31__g3b1768f.ari 30) cray-netcdf-hdf5parallel/4.6.3.2
9. xalt/1.1.2 20) xpmem/2.2.20-7.0.1.1_4.13__g0475745.ari 31) cray-hdf5-parallel/1.10.5.2
10. darshan/3.1.7 21) job/2.2.4-7.0.1.1_3.40__g36b56f4.ari 32) cray-tpsl/19.06.1
11. ksl/ksl 22) dvs/2.12_2.2.157-7.0.1.1_9.4__g083131db
    simModSuite=yes mpiLib=mpich3 netcdf=yes cc=/opt/cray/pe/craype/2.6.3/bin/cc cxx=/opt/cray/pe/craype/2.6.3/bin/CC logLevel=debug prefixPath=/home/vyasjc/meshing-soft-pumi-pumgen/PUMI/core:/home/vyasjc/meshing-soft-pumi-pumgen/simmetrix/14.0-200719:/home/vyasjc/meshing-soft-pumi-pumgen/parmetis-4.0.3/include:/home/vyasjc/TP-SeisSol/hdf5-1.8.21/build:/home/vyasjc/TP-SeisSol/netcdf-4.6.1/build
    scons: Reading SConscript files ...
    Checking for C++ library gmi_sim... yes
    Checking for C++ library apf_sim... yes
    Checking for C++ library apf_zoltan... yes
    Checking for C++ library gmi... yes
    Checking for C++ library mds... yes
    Checking for C++ library ma... yes
    Checking for C++ library apf... yes
    Checking for C++ library pcu... yes
    Checking for C++ library lion... yes
    Checking for C++ library mth... yes
    Checking whether h5pcc program exists...(cached) yes
    IndexError: list index out of range:
    File "/home/vyasjc/meshing-soft-pumi-pumgen/PUMGen/SConstruct", line 135:
    libs.find(env, 'hdf5', required=True, parallel=True)
    File "/home/vyasjc/meshing-soft-pumi-pumgen/PUMGen/site_scons/libs/init.py", line 43:
    return lib_find(env, **kw);
    File "/home/vyasjc/meshing-soft-pumi-pumgen/PUMGen/site_scons/libs/hdf5.py", line 103:
    if not conf.CheckLibWithHeader(flags['LIBS'][0], 'hdf5.h', 'c', extra_libs=flags['LIBS'][1:]):

---

[Thomas-Ulrich]

Hi, I can have a look if you share /home/vyasjc/meshing-soft-pumi-pumgen on shaheen.

[Thomas-Ulrich]

Hi, so the problem is the output of h5pcc -show -shlib on shaheen:

Shaheen

x_ulricht@cdl2:/project/k1488/meshing-soft-pumi-pumgen-TU/PUMGen> /opt/cray/pe/hdf5-parallel/1.10.5.2/bin/h5pcc -show  -shlib
cc -DpgiFortran -dynamic -Wl,-rpath -Wl,/opt/cray/pe/hdf5-parallel/1.10.5.2/CRAY/9.0/lib

SupermucNG:

di73yeq4@login03:~> h5pcc -show -shlib
/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/intel-mpi/2019.6.154-intel-lmhtdss/bin/mpicc -I/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/zlib/1.2.11-gcc-ajxhsmr/include -I/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/libszip/2.1.1-gcc-v3th2mi/include -fPIC -L/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/hdf5/1.8.20-intel-mhsivcc/lib -lhdf5_hl -lhdf5 -L/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/zlib/1.2.11-gcc-ajxhsmr/lib -L/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/libszip/2.1.1-gcc-v3th2mi/lib -lpthread -lsz -lz -ldl -lm -Wl,-rpath -Wl,/dss/dsshome1/lrz/sys/spack/release/19.2/opt/x86_avx512/hdf5/1.8.20-intel-mhsivcc/lib

@tum: any idea how to fix that?

Thomas.

[Thomas-Ulrich]

So the problem what that hdf5.py was looking for the -L option which is not there

1. A suggested workaround is to unload the hdf5 module and manually set the PKG_CONFIG_PATH. On shaheen this is done with:

export PKG_CONFIG_PATH=$PE_INTEL_FIXED_PKGCONFIG_PATH:$PKG_CONFIG_PATH

(but then another other problem arises at linking: build/src/pumgen.o:pumgen.cpp:function main: error: undefined reference to 'H5open').

2. it turned out that the problem had been fixed before but not properly documented, 2 lines need to be added to hdf5.py:

        utils.pkgconfig.appendPathes(env, flags)
        flags['LIBS'].append('hdf5_hl')
        flags['LIBS'].append('hdf5_hl_parallel')
        if not conf.CheckLibWithHeader(flags['LIBS'][0], 'hdf5.h', 'c', extra_libs=flags['LIBS'][1:]):

3. actually there is an (undocumented yet) cmake builder, so fixing the hdf5.py is not a priority...