Fixes CNP Budget

[bishtgautam]

The following updates are made to fix the CNP budget in ELM.

• The value for three fluxes used in the monthly CNP budget is now initialized.
• An MPI_Allreduce (instead of an MPI_Reduce) is used when budgets are computed.
• An additional field is written out in the ELM restart file.
• Long names and units of a few variables in the restart files are corrected.

Fixes #6230
[BFB] (except a new field is added in the ELM restart file)

[github-actions]

PR Preview Action v1.4.7
🚀 Deployed preview to https://E3SM-Project.github.io/E3SM/pr-preview/pr-6233/
on branch gh-pages at 2024-02-27 21:26 UTC

[peterdschwartz]

I did a test merge of this to next and received DIFF for fates_cold_allvars test in CMASS_BALANCE_ERROR. Only cause I can think of is that the gridcell balance checking for FATES is not correct and that will be fixed by PR #6235 . So, this PR may need to be merged after PR #6235 to avoid the extra diff. I'll try to test them together.

[rljacob]

@peterdschwartz we need this merged because it fixes month-long restart tests with the standard v3 F-case. Can't wait for it to work with FATES.

[rljacob]

This caused ERP_Ld15.ne4pg2_ne4pg2.IELMFATES.chrysalis_intel.elm-fates_cold_allvars to diff. Expected?

[peterdschwartz]

@rljacob Did you miss my comment above?

[rljacob]

Ah I missed that. Thanks.

[rljacob]

This fix allows SMS_Lm1.ne4pg2_oQU480.F2010 and SMS_Ly1.ne4pg2_oQU480.F2010 to run and pass. ERP_Lm3.ne4pg2_oQU480.F2010 and ERS_Ld31.ne4pg2_oQU480 no longer have runtime fails but do fail the restart test with "2 had differences in fill patterns" (for ERP) and "1 had differences in fill patterns" (for ERS).

[peterdschwartz]

@rljacob Sorry, I'm not sure where to look for those tests on CDash

[rljacob]

They're in the gcp-12 report for extra-coverage: https://my.cdash.org/viewTest.php?onlyfailed&buildid=2502048

[rljacob]

SMS_Ln9.ne4pg2_oQU480.F2010..eam-outfrq9s has diffs with baselines.
FILLDIF and RMS diff: ER
FILLDIFF only: SOM_C_LEACHED
RMS only: CMASS_BALANCE_ERROR

[rljacob]

Further investigation and discussion with @bishtgautam: any test that outputs an elm h0 file will have a diff like the above. So all prod-suite tests need to be blessed as well.

[rljacob]

@bishtgautam please update the PR description with your other changes.

[bishtgautam]

@rljacob I have updated the PR description.

[kchong75]

Hello, I was running E3SMv3.0.0 with a non-January starting month (e.g., 10), using the corresponding ELM restart file from a previous run (e.g., CASE.elm.r.0001-10-01.nc) as the initial dataset (finidat) for ELM. However, the CNPBudget fails its annual check for carbon state (net change in state >> time integrated flux) at the end of year, due to all annual beginning values being zeros.
Below is the error message:

-----------------------------------------------------------------------|--------------------
                                                  *SUM*   382.70401574 |      3042037680.28
-----------------------------------------------------------------------|--------------------
 
 CARBON STATES (kgC/m2*1e6): period   annual: date =        20101           0
 
                                        beg              end          |        *NET CHANGE*
                    PFT                 0.00       4515692979.23      |      4515692979.23
                    CWD                 0.00          2368343.95      |         2368343.95
           Total litter                 0.00        413913580.76      |       413913580.76
              Total SOM                 0.00        836694905.52      |       836694905.52
     Total wood product                 0.00                0.00      |               0.00
        Truncation sink                 0.00            32479.12      |           32479.12
      Crop seed deficit                 0.00                0.00      |               0.00
         Grid-level Err                 0.00               -0.00      |              -0.00
----------------------------------------------------------------------|-----------------------
           *SUM*                      5768702288.59                   |      5768702288.59
 time integrated flux =    3042037680.27547     
 net change in state  =    5768702288.59410     
 current state        =    72.4916509223762     
 relative error [%]   =    47.2665163135523     
 ENDRUN:
 ERROR in CNPBudgetMod.F90 at line 929

Is there a way to fix this issue, or is it necessary to run the model starting from January? Thank you in advance for your help.

[rljacob]

was the previous run that made the restart file also from v3.0.0?

[kchong75]

was the previous run that made the restart file also from v3.0.0?

Yes. It is a monthly restart file generated from a v3.0.0 run that starts from January.
And I did try a run without specifying the finidat file, it still fails the annual budget check, as long as it has a non-Jan starting month.

[kchong75]

I am not sure how the time integrated flux is calculated for the annual budget check, but it seems that the annual beg values from finidat (no matter if it is specified by user or not) is not updated to the ELM restart files if the model does not start from month 01, causing the annual beg values being all zeros.

[bishtgautam]

@kchong75 Can you please share the commands to reproduce the error? It is quite possible that I had not considered the case when a simulation starts in the middle of the year instead of 1st-Jan.

[bishtgautam]

@kchong75 I have created a new issue (#6337) and we can have a followup discussion on that new issue. Thanks.