[BUG/ISSUE] "-lhdf5_hl -lhdf5" flags break compilation

[JiaweiZhuang]

Describe the bug

During compilation, some commands contain -lhdf5_hl -lhdf5 but don't specify the HDF5 library path via -L:

mpif90 -L/home/centos/tutorial/Code.GCHP/GCHP/Shared/Linux/lib -L/home/centos/tutorial/Code.GCHP/GCHP/Shared/Linux/lib  -o sst_sic_EIGTHdeg.x libNSIDC-OSTIA_SST-ICE_blend.a -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-fortran-4.4.5-uv5xocaoik42r6odukzzhjixymhytovx/lib -lnetcdff -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-4.7.0-5fkurucr6jdlwztewiqmqkxen4vvm7xa/lib -lnetcdf -lnetcdf -lhdf5_hl -lhdf5 -lz -lm -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-4.7.0-5fkurucr6jdlwztewiqmqkxen4vvm7xa/lib -lnetcdf -ldl -lc -lpthread -lrt  -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -lstdc++
mpif90 -L/home/centos/tutorial/Code.GCHP/GCHP/Shared/Linux/lib -L/home/centos/tutorial/Code.GCHP/GCHP/Shared/Linux/lib  -o sst_sic_QUARTdeg.x libNSIDC-OSTIA_SST-ICE_blend.a -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-fortran-4.4.5-uv5xocaoik42r6odukzzhjixymhytovx/lib -lnetcdff -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-4.7.0-5fkurucr6jdlwztewiqmqkxen4vvm7xa/lib -lnetcdf -lnetcdf -lhdf5_hl -lhdf5 -lz -lm -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-4.7.0-5fkurucr6jdlwztewiqmqkxen4vvm7xa/lib -lnetcdf -ldl -lc -lpthread -lrt  -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -lstdc++
ld: cannot find -lhdf5_hl
ld: cannot find -lhdf5
ld: cannot find -lhdf5_hl
ld: cannot find -lhdf5
gmake[13]: *** [sst_sic_EIGTHdeg.x] Error 1
gmake[13]: *** Waiting for unfinished jobs....
gmake[13]: *** [sst_sic_QUARTdeg.x] Error 1

Full compile log: compile_hdf5_error.log

This can be solved by a dirty fix that copies HDF5 libraries into NetCDF directory:

ln -s $(spack location -i hdf5)/lib/* $(spack location -i netcdf)/lib/

Successful compile log: compile_success.log

To Reproduce

• Install libraries via spack -v install netcdf-fortran %intel ^hdf5+fortran+hl ^intel-mpi
• Grab GCHP 12.3.2 code, apply Support Intel-MPI for both ifort and gfortran (allow ESMF_COMM=intelmpi) #35, and run make build_all

The environment config is:

source $(spack location -i intel)/bin/compilervars.sh -arch intel64  # enable icc/ifort
module load intelmpi  # enable mpicc/mpifort

export I_MPI_CC=icc
export I_MPI_CXX=icpc
export I_MPI_FC=ifort
export I_MPI_F77=ifort
export I_MPI_F90=ifort

export CC=icc
export CXX=icpc
export FC=ifort
export F77=$FC
export F90=$FC

export OMPI_CC=$CC
export OMPI_CXX=$CXX
export OMPI_FC=$FC
export COMPILER=$FC
export ESMF_COMPILER=intel

export ESMF_COMM=intelmpi
export MPI_ROOT=/opt/intel/compilers_and_libraries/linux/mpi/intel64

export NETCDF_HOME=$(spack location -i netcdf)
export NETCDF_FORTRAN_HOME=$(spack location -i netcdf-fortran)

export GC_BIN="$NETCDF_HOME/bin"
export GC_INCLUDE="$NETCDF_HOME/include"
export GC_LIB="$NETCDF_HOME/lib"

export GC_F_BIN="$NETCDF_FORTRAN_HOME/bin"
export GC_F_INCLUDE="$NETCDF_FORTRAN_HOME/include"
export GC_F_LIB="$NETCDF_FORTRAN_HOME/lib"

export PATH=${NETCDF_HOME}/bin:$PATH
export PATH=${NETCDF_FORTRAN_HOME}/bin:$PATH

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${NETCDF_HOME}/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${NETCDF_FORTRAN_HOME}/lib

export ESMF_BOPT=O

Environment

• GEOS-Chem/GCHP version: 12.3.2
• Compiler version: ifort 19.0.4
• MPI library and version: Intel MPI 2019.4
• netCDF and netCDF-Fortran library version: 4.7.0 and 4.4.5
• Computational environment: AWS cloud, CentOS 7
• Code modifications: No change to GC-classic part; apply Support Intel-MPI for both ifort and gfortran (allow ESMF_COMM=intelmpi) #35 to GCHP part

[JiaweiZhuang]

I saw no hard-coded -lhdf5_hl -lhdf5 in GCHP Makefile. @jimmielin suspect that the -lnetcdf -lnetcdf -lhdf5_hl -lhdf5 -lz -lm part comes from nf-config --flibs in some ESMF Makefiles. For example ESMF/build/common.mk uses

ESMF_NFCONFIG = $(shell $(ESMF_NETCDF) --prefix)/bin/nf-config

which gives you this annoying -lhdf5_hl -lhdf5:

$ $(spack location -i netcdf-fortran)/bin/nf-config --flibs
-L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-fortran-4.4.5-tuuki3kp5ngtji3m54zjw3bkh7ri2num/lib -lnetcdff -L/home/centos/spack/opt/spack/linux-centos7-x86_64/intel-19.0.4/netcdf-4.7.0-sale32bbd3tize6jgjq6npfvgrfeyw65/lib -lnetcdf -lnetcdf -lhdf5_hl -lhdf5 -lz -lm

[JiaweiZhuang]

Same problem with 12.5.0. See geoschem/GCHP#17 (comment) for a log containing this HDF5 error.

[jimmielin]

I can confirm this will also be the case for GEOS-Chem Classic when generating the final geos executable. This does not impact GEOS-Chem Classic itself as it does not depend on Intel MPI (so this scenario is unlikely to break any users' installs), however there should be no reason GCC depends on -lhdf5 -lhdf5_hl.

This is specified in Makefile_header.mk in the main GEOS-Chem directory:

NC_LINK_CMD      := $(shell $(GC_F_BIN)/nf-config --flibs)

nf-config --flibs will return -lhdf5 -lhdf5_hl. Compiling without those linker flags will not break GCC so there could be a regex to patch them out of the nf-config output, for a quick fix.

[JiaweiZhuang]

In a container environment (with almost no pre-installed libraries), the -lz flag (linking zlib) also breaks the build. Can be fixed by yum install zlib-devel.

[lizziel]

@jimmielin @JiaweiZhuang Have either of you verified if this issue is still an open issue when using CMake? I also wonder about using the latest GCHP which has an updated ESMF.

[lizziel]

I am closing this issue since there has been no responses. The problematic code is in Makefiles that will be retired in the next version (13.0).