Failure to build netcdf fortran 4.6.0 w/ netcdf-c 4.9.2 on Centos 7

[HathewayWill]

@edhartnett

• environmental information (i.e. Operating System, compiler info, java version, python version, etc.)

Environment:

Centos 7 - VM
Libraries:L
zlib - 1.2.13
hdf5 - 1.14.1.2
netcdf-c - 4.9.0 (pass) 4.9.2 (fail)
netcdf-fortran - 4.6.0
libpng - 1.6.39
mpich - 4.1.2
jasper - 1.900.1
pnetcdf - 1.12.3

• a description of the issue with the steps needed to reproduce it

see test scripts and log files attached in zip folder:

Errors.zip

[WardF]

Thanks for providing the error logs! From the failed log, I see the following:

opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: /home/workhorse/WRF_SFIRE/Libs/NETCDF/lib/libnetcdf.a(libnchdf5_la-nc4hdf.o): warning: relocation against `H5T_STD_U16LE_g' in read-only section `.text'
/opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: /home/workhorse/WRF_SFIRE/Libs/NETCDF/lib/libnetcdf.a(libdispatch_la-dinstance.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: final link failed: bad value

Long story short, you'll need to reinstall libnetcdf (netcdf-c); when you do so, specify CFLAGS='-fPIC' (in addition to any other CFLAGS you're setting). It appears that you're setting that when configuring netCDF-fortran. Can you see what happens if you recompile libnetcdf with CFLAGS=-fPIC, or let me know that you're already doing so?

[HathewayWill]

Already doing so. CFLAGS = -O3 -fPIC -fPIE

…

On Wed, Jul 26, 2023, 6:20 PM Ward Fisher ***@***.***> wrote: Thanks for providing the error logs! From the failed log, I see the following: opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: /home/workhorse/WRF_SFIRE/Libs/NETCDF/lib/libnetcdf.a(libnchdf5_la-nc4hdf.o): warning: relocation against `H5T_STD_U16LE_g' in read-only section `.text' /opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: /home/workhorse/WRF_SFIRE/Libs/NETCDF/lib/libnetcdf.a(libdispatch_la-dinstance.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC /opt/rh/devtoolset-11/root/usr/libexec/gcc/x86_64-redhat-linux/11/ld: final link failed: bad value Long story short, you'll need to reinstall libnetcdf (netcdf-c); when you do so, specify CFLAGS='-fPIC' (in addition to any other CFLAGS you're setting). It appears that you're setting that when configuring netCDF-fortran. Can you see what happens if you recompile libnetcdf with CFLAGS=-fPIC, *or* let me know that you're already doing so? — Reply to this email directly, view it on GitHub <#406 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4FUT5RRV3PKEN2GEXPAQQDXSGQ33ANCNFSM6AAAAAA2XXP4UI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[HathewayWill]

@WardF

The shell script attached in the zip files should show the exact steps I took for recreation.

[HathewayWill]

@WardF

So for some reason CFLAGS is either getting removed or over written from autotools maybe?

[WardF]

@HathewayWill I'm revisiting this and the shell scripts you've provided; I am not immediately certain why CFLAGS is being removed. How are you invoking the script, simply running it from the command line vis-a-vis:

$ ./netcdf-4.9.2.sh

Or is it being invoked some other way?

[HathewayWill]

@WardF being built from source tar file.

[HathewayWill]

@WardF @edwardhartnett

This issue also occurs on any ubuntu system less than 22.04.2

Something about netcdf 4.9.1 & 4.9.2 w/ 4.6.1 causes the CFLAGS to be removed.

[WardF]

That is interesting; CFLAGS is an environmental variable, and I'm unable to replicate the issue where the CFLAGS just disappear. When you invoke the script, are you running it directly, are you running it via sudo or su?

[HathewayWill]

That is interesting; CFLAGS is an environmental variable, and I'm unable to replicate the issue where the CFLAGS just disappear. When you invoke the script, are you running it directly, are you running it via sudo or su?

Let me send a file with the installation. @WardF

netcdf_error.zip

this is for ubuntu 20, works fine on ubuntu 22

[HathewayWill]

@WardF

Just wanted to give you an update. I tested netcdf 4.9.0, 4.9.1, 4.9.2 with netcdf fortrtan 4.6.1

and the only one that works is 4.9.0 with 4.6.1. So the error seems to be with netcdf-c.

Tested on these operating systems: MacOS 13, Centos9, Centos7, Ubuntu22, Ubuntu20

[WardF]

@WardF

Just wanted to give you an update. I tested netcdf 4.9.0, 4.9.1, 4.9.2 with netcdf fortrtan 4.6.1

and the only one that works is 4.9.0 with 4.6.1. So the error seems to be with netcdf-c.

Tested on these operating systems: MacOS 13, Centos9, Centos7, Ubuntu22, Ubuntu20

Running the script you provided on a Ubuntu 23.10 machine, I'm seeing a failure although it is unrelated to any changes in CFLAGS; rather, it looks like there is a problem in how the script is trying to build libnetcdf. Investigating, and will follow up; I've added a -e invocation before the compilation step in the script, so that an error will cause the script to stop running.

[HathewayWill]

I havent tried on Ubuntu 23 yet myself. So for all I know it may work on that one. Unfortunately I'm sick with COVID so I can't access my computer

…

On Fri, Jan 5, 2024, 12:51 PM Ward Fisher ***@***.***> wrote: @WardF <https://github.com/WardF> Just wanted to give you an update. I tested netcdf 4.9.0, 4.9.1, 4.9.2 with netcdf fortrtan 4.6.1 and the only one that works is 4.9.0 with 4.6.1. So the error seems to be with netcdf-c. Tested on these operating systems: MacOS 13, Centos9, Centos7, Ubuntu22, Ubuntu20 Running the script you provided on a Ubuntu 23.10 machine, I'm seeing a failure although it is unrelated to any changes in CFLAGS; rather, it looks like there is a problem in how the script is trying to build libnetcdf. Investigating, and will follow up; I've added a -e invocation before the compilation step in the script, so that an error will cause the script to stop running. — Reply to this email directly, view it on GitHub <#406 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4FUT5VWXD3EZIVAFYGTETDYNBDTRAVCNFSM6AAAAAA2XXP4UKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGEZDCMRRGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[WardF]

Take care of yourself, I will keep working on this; I am travelling but have put a pin on this so we can get it sorted out. I'm finding an error in the script provided in which configure for libhdf5 fails, reporting a failure related to -m64 for gfortran. libhdf5 is not generated, which means libnetcdf is not generated, which means netCDF-Fortran will not build. Peculiar. I will stand up an older VM to see if the behavior changes, or if this is something on the latest Ubuntu and not related to the initial issue.