Skip to content

ESMF 8.4.0
Compare
Choose a tag to compare
@theurich theurich released this 28 Oct 17:37
· 811 commits to develop since this release
v8.4.0
9935617

Overview

The 8.4.0 release of ESMF implements a number of exciting new features. Some are incremental in nature while others explore new territory. Highlights of the 8.4.0 release are outlined in the following paragraphs. A detailed list of release notes is provided further down.

ESMX, a new layer built on top of the ESMF and NUOPC APIs, is being introduced in this release. ESMX stands for the Earth System Model eXecutable (ESMX) layer. The goals of ESMX are (1) to simplify standing up new NUOPC-based systems, (2) promote hierarchical model component testing, (3) reduce the cost of maintaining NUOPC-based modeling systems, (4) improve alignment and interoperability between different NUOPC-based systems, and (5) provide a fast and coordinated roll-out strategy for new ESMF/NUOPC features.

Exploration and experimentation with ESMX is strongly encouraged. However, please recognize that it is a new development effort that comes with the typical rough edges. For example, no formal reference manual is available yet. A good starting point to learn about the layer is the ESMX/README.md. We expect development of ESMX to continue throughout the next several release cycles.

Progress was made in the area of I/O. Multi-tile Arrays and Fields can now be used in Read and Write operations. A common use case is the six-tile representation of the cubed sphere grid. There are a number of restrictions that apply to the multi-tile I/O support made available in this release. Please see the associated release note below for more information. The plan is to relax restrictions in future releases.

On the ESMF object level a new concept of “Named Aliases” was introduced. This concept builds on the existing ESMF alias concept for deep objects (objects explicitly created and destroyed by Create() and Destroy() methods). Named aliases manage their own private name without affecting the name of the aliased object. This allows the same object to be known under different names inside of different components.

Incremental progress was also made in the areas of XGrid, LocStream, and Grid. Here user feedback identified a number of issues that have been addressed in this release.

With the increasing importance of ARM based architectures, such as Apple’s M1, etc., work was done in this release to support the native development environment on Darwin (Mac OS X). The compiler combination GFortran + Clang is now considered fully supported on Darwin. The release has been fully tested on an Apple M1 system with MPIUNI, MPICH, and OpenMPI.

ESMPy users please note that the Python module has been renamed from "ESMF" to "esmpy" for better alignment with Python Enhancement Proposal (PEP) guidelines. Please check out the respective release note for details and potential impact on user code.

Release Notes

  • The public Fortran API in this release is backward compatible with the last release of ESMF 8.3.0 and patch release ESMF 8.3.1. There were a few minor API changes, none of which require user code changes. The list of API changes is summarized in a table showing interface changes since ESMF 8.3.0, including the rationale and impact for each change.
  • The Python module name for ESMPy was changed in this release from “ESMF” to “esmpy” for better alignment with Python Enhancement Proposal (PEP) guidelines. This change will require user code changes! ESMPy users will have to do a global search and replace on their scripts to adjust to the module name change. Also notice that the internal ESMF source tree directory structure has changed, moving ./src/addon/ESMPy to ./src/addon/esmpy. This might affect anybody keeping an ESMF repository clone, and maintaining scripts that depend on the internal directory naming.
  • No bit-for-bit changes were observed for this release compared to release ESMF 8.3.0. This is based on test runs with the Intel compilers using options “-O2 -fp-model precise”.
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • All of the fixes and improvements released with patch 8.3.1 are also included in release 8.4.0. Of particular note is support for messages above the 2GiB limit by the ESMF VM communication layer.
  • The Earth System Model eXecutable (ESMX) layer was added to ESMF. ESMX removes technical hurdles that impede hierarchical NUOPC model testing. It includes a unified executable capable of driving multiple configurations of model components: single component, one-way forced, and fully dynamic two-way coupled. A simple YAML file is used to list the component build dependencies (CMake based). A standard NUOPC run configuration file is used to specify the processor layout, run sequence, the field dictionary, and model component options. ESMX supports hierarchical testing by allowing decentralized testing at the component level before integration into larger coupling systems, as well as testing on the integration level.
  • The concept of “Named Aliases” has been introduced. Regular aliases of deep ESMF objects continue being created using a simple assignment operator. Changing the name of a regular alias affects the object and thus all other aliases. Named aliases are created using the new ESMF_NamedAlias() function. Named aliases manage their own private name. Changing the name of a named alias does not affect the object or other aliases.
  • Using a Mesh that contains one or more elements with greater than 4 sides when creating an XGrid is now supported. Previously doing so would result in an error when data was transferred into or out of the XGrid.
  • LocStreams can now be created from a Cartesian Mesh file. Previously the attempt to do so would result in an error.
  • Attempting to create a Grid from a GRIDSPEC mosaic file that doesn’t contain variables with standard_name set to "grid_tile_spec", "geographic_latitude", or "geographic_longitude" now triggers clear ERROR log messages before returning with an error. Previously no ERROR messages were logged, making this situation hard to debug.
  • Read and Write operations are now permitted for multi-tile Arrays and Fields. A common use case for this is representing a cubed sphere grid as a six-tile grid. This change applies to ESMF_ArrayRead(), ESMF_ArrayWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_FieldBundleRead(), and ESMF_FieldBundleWrite(). Each of these methods reads or writes to multiple files in the multi-tile case. See the respective API documentation for details of how the tile-specific file names are constructed.
    Current limitations are:
    • For I/O of ArrayBundles and FieldBundles, all Arrays / Fields in the bundle must contain the same number of tiles;
    • I/O is not yet permitted for multi-tile Arrays / Fields with ungridded / undistributed dimensions;
    • I/O is currently only permitted for multi-tile Arrays / Fields with 1 DE per PET.
  • ESMF profiles now include a summary of the full application run time for convenience. Previously user instrumentation was required to introduce an end-to-end profiling level.
  • The configuration for ESMF_COMPILER=gfortranclang now works on Darwin systems (Mac OS X). Note that on Darwin, the system-level g++ invokes clang++, and unless you ensure that you have a true g++ early in your path (or build the MPI compiler wrappers to ensure that they wrap the true g++), you will end up using clang++ even if you think you are using the GNU C++ compiler. Setting ESMF_COMPILER=gfortranclang is correct for this typical situation. Attempting to use ESMF_COMPILER=gfortran when the C++ compiler is actually clang++ now issues an error message.

Known Issues

Platform-specific issues:

  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default, ESMF uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to “mysterious” application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting the environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+mpich
    • Green: Darwin+gfortranclang+mpich/openmpi
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni

Documentation

Tables

Assets 2