Recommended library stack for mpi, netcdf, parallel-netcdf, & PIO versions

[pwolfram]

Background

Previously, there have been some incompatibilities between different versions of netcdf (and its bindings), parallel-netcdf, and PIO. I've reconciled this by "tagging" the versions to known numbers that work at https://github.com/pwolfram/homebrew-mpas, namely with the following installation recipe.

Installation

Run ./install.sh to install brew and taps needed for this installation.

• netcdf bindings for MPAS-Tools: brew install pwolfram/mpas/netcdf --enable-fortran --enable-cxx-compat
  • netcdf 4.3.3.1
  • netcdf-cxx 4.2
  • netcdf-fortran 4.4.2
• parallel-netcdf 1.4.1: brew install pwolfram/mpas/parallel-netcdf
• pio 1.7.2: brew install pwolfram/mpas/pio

Compile MPAS, e.g.,

export NETCDF=/usr/local/Cellar/netcdf/4.3.3.1/
export PNETCDF=/usr/local/Cellar/parallel-netcdf/1.4.1/
export PIO=/usr/local/Cellar/pio/1.7.2/
make clean gfotran CORE=ocean

Question

What is the preferred library stack, especially for PIO2? In particular, the problem with the
homebrew recipe in that there is an incompatibility with openmpi > 1.6.

Any recommendations on the current best-practices library stack is greatly appreciated. Furthermore, I would recommend that we place this information on the README.md for the project so that new users will have fewer challenges building.

cc @hguo, @tpeterka, @vanroekel

[vanroekel]

@pwolfram why such an old version of pnetcdf? I would recommend 1.8.1

[pwolfram]

@vanroekel, what other versions are you using with pnetcdf 1.8.1?

[xylar]

I just did an Ubuntu upgrade and will eventually need to build MPAS and dependencies again as well. So I'm also keen to find out what the most up-to-date working combination might be. I recall PIO being the biggest problem for me in the past but that was before MPAS supported PIO2 so maybe things are better now?

[pwolfram]

@tpeterka and @hguo tried this recently with PIO2 but didn't have success due to runtime errors.

I'm curious to hear what @mgduda and @mark-petersen use for their software stack.

[tpeterka]

Correct. We built the mpas-o with PIO2, but it crashed during running. Building with PIO 1.7.2 solved that problem. We didn't need quite so old versions of netcdf and pnetcdf, however. We used netcdf 4.4.1.1 and pnetcdf 1.8.1 successfully. This is for the ocean forward model. I don't know anything about the tools or the rest of the code, though.

[mark-petersen]

These are the versions I use: module load intel/17.0.1 openmpi/1.10.5 netcdf/4.4.1 parallel-netcdf/1.5.0 pio/1.7.2 module load gcc/5.3.0 openmpi/1.10.5 netcdf/4.4.1 parallel-netcdf/1.5.0 pio/1.7.2 From: tpeterka <notifications@github.com> Reply-To: MPAS-Dev/MPAS-Model <reply@reply.github.com> Date: Wednesday, May 2, 2018 at 2:36 PM To: MPAS-Dev/MPAS-Model <MPAS-Model@noreply.github.com> Cc: "Petersen, Mark Roger" <mpetersen@lanl.gov>, Mention <mention@noreply.github.com> Subject: Re: [MPAS-Dev/MPAS-Model] Recommended library stack for mpi, netcdf, parallel-netcdf, & PIO versions (#34) Correct. We built the mpas-o with PIO2, but it crashed during running. Building with PIO 1.7.2 solved that problem. We didn't need quite so old versions of netcdf and pnetcdf, however. We used netcdf 4.4.1.1 and pnetcdf 1.8.1 successfully. This is for the ocean forward model. I don't know anything about the tools or the rest of the code, though. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#34 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AD2Y0AwdNnkRthLPSachUylP7pLqHbBbks5tuhixgaJpZM4TudL4>.

[vanroekel]

I have used the following combinations routinely

intel/17.0.1 -- openmpi/1.10.5 (or mvapich2/2.2) -- netcdf/4.3.3.2 (4.4.1 is fine too) -- parallel-netcdf/1.8.1 pio/1.7.2 (pio/1.9.1 is fine too)

gcc/5.3.0 -- openmpi/1.10.5 -- netcdf/4.4.1 -- parallel-netcdf/1.8.1 -- pio/1.7.2 -- These are the versions in my MPAS docker image

I've never tried pio2.

[mgduda]

For what it's worth, I've been recently using:

• MPICH 3.2
• HDF5 1.8.19
• netCDF (C) 4.4.1.1
• netCDF (Fortran) 4.4
• parallel-netCDF 1.8.1
• The latest PIO2 code from https://github.com/NCAR/ParallelIO

[mgduda]

Also, because it took a few iterations to converge on this recipe, here's how I've been installing PIO2:

export PIO_INSTALL_DIR=/somewhere/pio2
git clone https://github.com/NCAR/ParallelIO
cd ParallelIO
export PIOSRC=`pwd`
cd ..
mkdir pio-build
cd pio-build
export CC=mpicc
export FC=mpifort
cmake -DNetCDF_C_PATH=$NETCDF -DNetCDF_Fortran_PATH=$NETCDF -DPnetCDF_PATH=$PNETCDF -DCMAKE_INSTALL_PREFIX=$PIO_INSTALL_DIR -DPIO_ENABLE_TIMING=OFF -DCMAKE_VERBOSE_MAKEFILE=1 $PIOSRC
make
make install
cd ..
rm -rf pio-build ParallelIO
export PIO=$PIO_INSTALL_DIR

Of course, change /somewhere/pio2 to the location you'd like to install PIO2.

[pwolfram]

Thanks everyone for this really helpful feedback.

Has anyone gotten a compile to work with newer versions of open-mpi? It is now in version 3 and when @mark-petersen and I tried this a year ago in July with open-mpi 2.x it did not work.

[pwolfram]

I'm using (following the workflow at https://github.com/pwolfram/homebrew-mpas), tested on macOS Sierra and High Sierra using the nightly test suite:

• gcc 8.2.0
• mpich 3.2.1
• netcdf 4.6.1
• netcdf-cxx 4.3
• netcdf-fortran 4.4.4
• parallel-netcdf 1.7.0
• pio 1.7.2

cc pwolfram/homebrew-mpas#4

[mark-petersen]

This issue has gone stale. See new versioning at this confluence page