[PULL REQUEST] Rebuild fullchem and Hg mechanisms with KPP 2.5.0 #1258

yantosca · 2022-05-19T16:00:01Z

KPP has been recently updated to version 2.5.0 (see https://kpp.readthedocs.io). In this version, all of the features that were hardwired into KPP 2.2.*_gc and KPP 2.3.*_gc have now been added as user options and have been incorporated into the main line of development (along with updates from Rolf Sander in KPP 2.4.0).

We have now rebuilt the fullchem and Hg mechanisms with KPP 2.5.0, and have made related updates in the various driver modules (fullchem_mod.F90, mercury_mod.F90).

This is a zero-diff update. Difference tests (1-hr simulations) w/r/t 13.4.0 (main branch) yield identical results:

Fullchem
Ref = 13.4.0 + KPP 2.3.3_gc on 5 cores
Dev = KPP 2,5.0 on 8 cores

Using configuration file yml/rst.yml
... Printing totals and differences
... Only showing variables with |absolute difference| > 0.0
Variable               Ref=KPP 2.3.3_gc         Dev=KPP 2.5.0 & yaml     Dev - Ref

Hg
Ref = 13.4.0 + KPP 2.3.3_gc on 5 cores
Dev = KPP 2,5.0 on 8 cores

Using configuration file yml/hg.yml
... Printing totals and differences
... Only showing variables with |absolute difference| > 0.0
Variable               Ref=KPP 2.3.3_gc         Dev=KPP 2.5.0 + yaml     Dev - Ref

KPP/Hg/commonIncludeVars.H KPP/fullchem/commonIncludeVars.H - Include file with common variables needed to compile all mechanisms. These variables are now #include-d into gckpp_Global.F90. KPP/Hg/CMakeLists.txt KPP/fullchem/CMakeLists.txt - Added commonIncludeVars.H KPP/Hg/Hg.kpp KPP/fullchem/fullchem.kpp - Add "#MINVERSION 2.5.0" to force exit unless we are using KPP 2.5.0+ - Add "#UPPERCASEF90" on to generate gckpp_*.F90 code (using .F90 instead of .f90 suffix) - Added comments - Added "#include commonIncludeVars.H" in the F90_GLOBAL section KPP/custom/custom.eqn KPP/Hg/Hg.eqn KPP/Hg/fullchem.eqn - Now use "#INCLUDE atoms.kpp" instead of "#INCLUDE atoms" KPP/OHreact_parser.py - Restructured into functions - Now gets the mechanism name as a command-line argument - Set OHreact to zero for the carboncycle mechanism (coming soon!) - Strip out extra characters generated by KPP 2.5.0+ from input KPP/build_mechanism.sh - Remove code that checks for the KPP version (we can use #MINVERSION) - Pass mechanism name to OHreact_parser.py KPP/Hg/README.md KPP/fullchem/README.md - Updated for KPP 2.5.0 (removed "KPP-for-GEOS-Chem") KPP/Hg/gckpp.map KPP/Hg/gckpp_*.F90 KPP/fullchem/gckpp.map KPP/fullchem/gckpp_*.F90 - Rebuilt with KPP 2.5.0 Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>

In the prior commit, we had built the fullchem and KPP mechanisms with the wrong branch of the KPP repository. We have now rebuilt these mechanisms with the main branch (which at the time of this writing points to the KPP 2.5.0 release). KPP/Hg/gckpp_Integrator.F90 KPP/fullchem/gckpp_Integrator.F90 - Whitespace and comment differences only KPP/Hg/gckpp_Rates.F90 KPP/fullchem/gckpp_Rates.F90 - Whitespace differences only Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>

GeosCore/fullchem_mod.F90 GeosCore/mercury_mod.F90 - Remove reference to Nhnew - Remove call to Init_KPP, this only sets C=0 and CFACTOR=1, which we now do manually at the start of the DO loop - Add new local variable !$OMP PRIVATE variable localC for MODEL_GEOS and MODEL_WRF. This replaces VAR and FIX which are no longer public variables. - Add ICNTRL(15) = -1, which prevents calls to Update_SUN, Update_PHOTO, and Update_RCONST from occurring within the integrator - Replace VAR with C(1:NVAR) and FIX with C(NVAR+1:NSPEC), since VAR and FIX are now private to the gckpp_Integrator.F90 file. - Update comments for RCNTRL(3) and RSTATE(3) Only in mercury_mod.F90: - Exit if the integrator fails twice, as we do in fullchem_mod.F90. - Call FUN with the proper arguments for MODEL_GEOS Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>

yantosca · 2022-05-19T16:01:40Z

GeosCore/fullchem_mod.F90

@@ -103,12 +103,11 @@ SUBROUTINE Do_FullChem( Input_Opt,  State_Chm, State_Diag,                 &
    USE fullchem_SulfurChemFuncs, ONLY : fullchem_HetDropChem
    USE GcKpp_Monitor,            ONLY : SPC_NAMES, FAM_NAMES
    USE GcKpp_Parameters
-    USE GcKpp_Integrator,         ONLY : INTEGRATE, NHnew
+    USE GcKpp_Integrator,         ONLY : INTEGRATE


Nhnew is a parameter in the gckpp_Integrator.F90 file that is set to 3. However, not all integrators contain this parameter. We will work to fix this in a future version of KPP. For now, remove the reference to Nhnew.

yantosca · 2022-05-19T16:02:29Z

GeosCore/fullchem_mod.F90

 #ifdef MODEL_GEOS
    REAL(f4)               :: NOxTau, NOxConc, Vdotout(NVAR)
+    REAL(dp)               :: localC(NSPEC)


localC is needed for MODEL_GEOS and MODEL_WRF, in order to restore the prior concentrations.

yantosca · 2022-05-19T16:03:25Z

GeosCore/fullchem_mod.F90

       RCONST    = 0.0_dp                   ! KPP rate constants
       PHOTOL    = 0.0_dp                   ! Photolysis array for KPP
       K_CLD     = 0.0_dp                   ! Sulfur in-cloud rxn het rates
       K_MT      = 0.0_dp                   ! Sulfur sea salt rxn het rates
+       CFACTOR   = 1.0_dp                   ! KPP conversion factor


We now set CFACTOR = 1 directly. This had been done in Init_KPP() but this routine only sets C= 0 and CFACTOR=1, so it's kind of useless. We can now do this in the body of the main parallel loop.

yantosca · 2022-05-19T16:04:38Z

GeosCore/fullchem_mod.F90

+                    RCT     = RCONST,                                        &
+                    Vdot    = Vloc,                                          &
+                    Aout    = Aout,                                          &
+                    Vdotout = Vdotout                                       )


KPP 2.5.0 now automatically adds Aout and Vdotout variables as optional arguments to subroutine Fun(). So the GMAO folks won't have to hack this in going forward.

yantosca · 2022-05-19T16:05:22Z

GeosCore/fullchem_mod.F90

-          RCNTRL(3)  = 0e+0_fp
-          CALL Init_KPP( )
-          VAR = C(1:NVAR)
-          FIX = C(NVAR+1:NSPEC)


In KPP 2.5.0, VAR and FUN are now private to gckpp_Integrator.F90. This is necessary in order to make KPP-generated code thread-safe.

yantosca · 2022-05-19T16:06:03Z

GeosCore/mercury_mod.F90

@@ -104,6 +104,7 @@ MODULE Mercury_Mod
  !--------------------------------------------------------------------------
  ! Scalars
  !--------------------------------------------------------------------------
+  LOGICAL  :: Failed2x


In mercury_mod.F90, we add a similar test so that we can exit gracefully if the integration fails twice.

yantosca · 2022-05-19T16:07:36Z

KPP/Hg/CMakeLists.txt

@@ -19,6 +19,7 @@ target_link_libraries(KPP_FirstPass
 # Add libKPP.a
 #-----------------------------------------------------------------------------
 add_library(KPP STATIC EXCLUDE_FROM_ALL
+  commonIncludeVars.H


To simplify the coding, I have added a new include file commonIncludeVars.H that contain all of the variables that need to be inlned into gckpp_Global.F90. This will decrease the chances that we'll add a variable in e.g. fullchem but forget about it in e.g. Hg.

yantosca · 2022-05-19T16:07:57Z

KPP/Hg/Hg.eqn

@@ -15,7 +15,7 @@ Comment format is
 Equations - Date modified; Reference; Developer initials
 }

-#include atoms
+#include atoms.kpp


We decided to rename the atoms file to atoms.kpp in version 2.5.0.

yantosca · 2022-05-19T16:08:44Z

KPP/Hg/commonIncludeVars.H

@@ -0,0 +1 @@
+../fullchem/commonIncludeVars.H


We keep commonIncludeVars.H in the fullchem folder and link to it elsewhere.

yantosca · 2022-05-19T16:09:28Z

KPP/Hg/gckpp.map

+!        R. Sander,   Max-Planck Institute for Chemistry, Mainz, Germany
+!        M. Long,     Harvard University, USA
+!        H. Lin,      Harvard University, USA
+!        R. Yantosca, Harvard University, USA


Comment headers have been updated in the KPP-generated code. This should be the only difference in most of the gckpp_*.F90 files.

yantosca · 2022-05-19T16:11:08Z

KPP/Hg/gckpp_Global.F90

  !$OMP THREADPRIVATE( VAR )
 ! FIX - Concentrations of fixed species (global)
-  REAL(kind=dp) :: FIX(NFIX)
+  REAL(kind=dp), POINTER :: FIX(:)


KPP 2.5.0 now automatically generates the gckpp_Global.F90 file with C as a target and VAR and FIX as pointers. This gets rid of the non-thread-safe EQUIVALENCE statement, and makes the code thread-safe for OpenMP.

yantosca · 2022-05-19T16:11:37Z

KPP/Hg/gckpp_Global.F90

  !$OMP THREADPRIVATE( FIX )
 ! RCONST - Rate constants (global)
  REAL(kind=dp) :: RCONST(NREACT)
  !$OMP THREADPRIVATE( RCONST )
 ! TIME - Current integration time
  REAL(kind=dp) :: TIME
  !$OMP THREADPRIVATE( TIME )
+! SUN - Sunlight intensity between [0,1]
+  REAL(kind=dp) :: SUN
+  !$OMP THREADPRIVATE( SUN )


We have re-introduced this in order to make KPP more general.

yantosca · 2022-05-19T16:13:06Z

KPP/Hg/gckpp_JacobianSP.F90

@@ -41,7 +44,7 @@ MODULE gckpp_JacobianSP
      23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, & ! index 97 - 108
      24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, & ! index 109 - 120
      25, 25, 25, 26, 26, 26, 26, 26, 26, 27, 27, 27, & ! index 121 - 132
-      27, 27, 27, 27 /)
+      27, 27, 27, 27 /) ! index up to 136


Rolf Sander added this comment for the last line into KPP 2.4.0.

yantosca · 2022-05-19T16:13:53Z

KPP/Hg/gckpp_Monitor.F90

-     'NO2            ','NO             ','O3             ', & ! index 31 - 33
-     'HO2            ','BrO            ','ClO            ', & ! index 34 - 36
-     'CO             ','CH4            ' /)
+  CHARACTER(LEN=32), PARAMETER, DIMENSION(38) :: SPC_NAMES = (/ &


In KPP 2.4.0, the max species name length is now 32 chars (up from 15).

yantosca · 2022-05-19T16:14:39Z

KPP/Hg/gckpp_Rates.F90

+!  Note: the default argument type for rate laws, as read from the equations file, is single precision
+!        but all the internal calculations are performed in double precision
+!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+


Some general rate law functions (not used by GEOS-Chem) have been restored in the gckpp_Rates.F90 file.

yantosca · 2022-05-19T16:16:06Z

KPP/Hg/gckpp_Util.F90

@@ -142,6 +145,67 @@ END SUBROUTINE GenerateMatlab
 ! ****************************************************************


+! ****************************************************************
+!


New routine used internally by the integrators. Determines if the Update_* routines have to be called within the integrator step. For GEOS-Chem we now set ICNTRL(15) = -1 as an input argument to the Integrate routine, which sets all the logicals to FALSE.

yantosca · 2022-05-19T16:17:20Z

KPP/OHreact_parser.py

-# See git history for subsequent updates
-
-# Starts here
+"""


Restructured the OHreact_parser.py into functions. It also accepts the mechanism directory name as an argument (e.g. because for carboncycle we would need to set the reactivity to zero, and thus can skip a lot of the code).

yantosca · 2022-05-19T16:17:51Z

KPP/build_mechanism.sh

@@ -33,16 +33,6 @@
 #------------------------------------------------------------------------------
 #BOC

-#============================================================================
-# Check KPP version


This code to check the KPP version is now supplanted by #MINVERSION

yantosca · 2022-05-19T16:18:33Z

KPP/fullchem/commonIncludeVars.H

@@ -0,0 +1,214 @@
+  !==========================================================================


New include file. All of these variables (even if they aren't used in a mechanism) need to be present at compile time in order to avoid compilation errors.

yantosca · 2022-05-19T16:22:01Z

Integration tests pending

yantosca · 2022-05-19T17:34:02Z

GCHP integration tests passed

==============================================================================
GCHP: Execution Test Results

Number of execution tests: 3
==============================================================================
 
Execution tests:
------------------------------------------------------------------------------
gchp_fullchem_benchmark_merra2_c48...............Execute Simulation.....PASS
gchp_fullchem_standard_merra2_c24................Execute Simulation.....PASS
gchp_TransportTracers_geosfp_c24.................Execute Simulation.....PASS
 
Summary of execution test results:
------------------------------------------------------------------------------
Execution tests passed:        3
Execution tests failed:        0
Execution tests not completed: 0

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%  All execution tests passed!  %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

yantosca · 2022-05-20T22:08:31Z

All GEOS-Chem "Classic" integration tests passed:

Summary of test results:
------------------------------------------------------------------------------
Execution tests passed: 48
Execution tests failed: 0
Execution tests not yet completed: 0

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%  All execution tests passed!  %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

I have merged this up to dev (locally) and will bring this into alpha.4.

yantosca · 2022-05-23T17:24:36Z

New integration tests for the KPP 2.5.0 chemistry merged on top of alpha.4 all passed:

Summary of execution test results:
------------------------------------------------------------------------------
Execution tests passed:        3
Execution tests failed:        0
Execution tests not completed: 0

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%  All execution tests passed!  %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
Summary of test results:
------------------------------------------------------------------------------
Execution tests passed: 48
Execution tests failed: 0
Execution tests not yet completed: 0

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%  All execution tests passed!  %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

yantosca · 2022-05-23T18:14:14Z

Now merged into the 14.0.0 development stream.

yantosca added 3 commits May 18, 2022 17:24

yantosca added category: Feature Request New feature or request topic: Chemical Mechanisms Related to KPP and/or GEOS-Chem chemistry mechanisms no-diff-to-benchmark This update will not change the results of fullchem benchmark simulations labels May 19, 2022

yantosca added this to the 14.0.0 milestone May 19, 2022

yantosca requested a review from msulprizio May 19, 2022 16:00

yantosca self-assigned this May 19, 2022

yantosca commented May 19, 2022

View reviewed changes

yantosca merged commit f833fba into dev May 23, 2022

yantosca deleted the feature/KPP_250_fullchem_Hg_only branch May 23, 2022 17:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PULL REQUEST] Rebuild fullchem and Hg mechanisms with KPP 2.5.0 #1258

[PULL REQUEST] Rebuild fullchem and Hg mechanisms with KPP 2.5.0 #1258

yantosca commented May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca May 19, 2022

yantosca commented May 19, 2022

yantosca commented May 19, 2022

yantosca commented May 20, 2022

yantosca commented May 23, 2022

yantosca commented May 23, 2022

		@@ -0,0 +1,214 @@
		!==========================================================================

		@@ -0,0 +1 @@
		../fullchem/commonIncludeVars.H

[PULL REQUEST] Rebuild fullchem and Hg mechanisms with KPP 2.5.0 #1258

[PULL REQUEST] Rebuild fullchem and Hg mechanisms with KPP 2.5.0 #1258

Conversation

yantosca commented May 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yantosca commented May 19, 2022

yantosca commented May 19, 2022

yantosca commented May 20, 2022

yantosca commented May 23, 2022

yantosca commented May 23, 2022