More thoroughly investigate answer changes in dev019, dev020 and dev035

[billsacks]

I have looked into what CTSM master tags from ctsm1.0.dev001 through the present could warrant more careful investigation to verify that they haven't changed climate. Other than ctsm1.0.dev001 itself (#690 ), the ones that could use investigation are ctsm1.0.dev019, ctsm1.0.dev020 and ctsm1.0.dev035. I'm thinking it could be worth batching these changes together and running a single set of verifications of ctsm1.0.dev035 against ctsm1.0.dev018.

Here is an annotated list of all tags on master since ctsm1.0.dev001 that have changed answers:

• ctsm1.0.dev002

  • Changes answers for all clm5_0 BGC cases
  • This is on the release branch; no need to do verification of it

• ctsm1.0.dev004

  • Changes answers for year-2000 cases
  • This is on the release branch; no need to do verification of it

• ctsm1.0.dev007

  • Changes answers for startup / hybrid runs that include CISM (even NOEVOLVE) and do not use init_interp. Changes can be larger than roundoff.
  • This is on the release branch; no need to do verification of it

• ctsm1.0.dev008

  • Updates N dep file. Changes answers for 1850 and 20th century cases; expected to be similar climate.
  • This is on the release branch; no need to do verification of it

• ctsm1.0.dev013

  • Fix for transient non-crop cases
  • Significant answer changes for these transient non-crop cases
  • This is on the release branch; no need to do verification of it

• ctsm1.0.dev016

  • Changes answers due to datm update: new presaero and CO2 datasets
  • No need to do verification of this

• ctsm1.0.dev018

  • Roundoff-level changes in sensible heat flux; everything else bfb
  • No need to do verification of this

• ctsm1.0.dev019

  • Reworks initialization of wa and zwt
  • Larger-than-roundoff changes for CLM50 cold start and transient (Hist) cases; expected to be same climate
  • It might be worth doing more thorough testing of this tag

• ctsm1.0.dev020

  • No longer resets wa_col to aquifer_water_baseline every time step
  • Differences only observed for cold start tests or when using a CLM45 initial conditions file in a CLM50 case, but it's possible that other configurations would see differences rarely
  • It might be worth doing more thorough testing of this tag. This doesn't seem too important, but this could be combined with other tag(s) in doing some additional testing.

• ctsm1.0.dev022

  • Roundoff-level changes with groundwater irrigation
  • Roundoff-level changes in the diagnostic field, QIRRIG_FROM_SURFACE
  • No need to do verification of this

• ctsm1.0.dev028

  • Interpolated all out-of-the-box initial conditions
  • Changes answers for cases that require interpolation of out-of-the-box initial conditions
  • I was a bit suspicious of the changes in this tag. However, given that this only affects cases that require interpolation of out-of-the-box initial conditions, this probably won't affect most scientifically-important configurations, and also wouldn't be picked up by standard diagnostics runs.
  • So no need to do verification of this

• ctsm1.0.dev030

  • cime update resulted in differences for f19_g17 (roundoff-level) and T31_g37
  • No need to do verification of this

• ctsm1.0.dev031

  • Large answer changes for runs with transient glaciers
  • For runs with transient vegetation, roundoff-level changes in dynbal fluxes
  • Large changes in diagnostic variables ICE_CONTENT1, LIQUID_CONTENT1 and HEAT_CONTENT1, over glaciers containing some glacier area
  • No need to do verification of this

• ctsm1.0.dev034

  • Just changes answers for clm45
  • No need to de verification of this

• ctsm1.0.dev035

  • Replaces uses of h2ocan with snocan + liqcan
  • Expected to be just roundoff-level, but some differences seen in the test suite raised suspicions
  • We should do more thorough testing of this tag

[olyson]

Dave and I talked and thought that we would re-run a section near the end of a control historical simulation (the control being run with release-clm5.0.01) with a CTSM tag that enforces canopy liquid and snow capacities consistently (see issue#699).
So, our first comparison would be the tag with this change compared to the initial CLM release tag (release-clm5.0.01). We'd run and examine our regular diagnostics. If we saw larger differences than hoped for, we would then back up to the next major tag you've identified.

[olyson]

I've done a GSWP3 BGC historical 1989-2014 run with ctsm1.0.dev039 (includes removing the temperature dependence of canopy liquid and snow capacities) and compared it to a run with release-clm5.0.01.
Diagnostics are here:

http://www.cgd.ucar.edu/tss/clm/diagnostics/ctsm_dev/index.html

Differences in our key flux and state variables are very small. Larger than roundoff but definitely not climate-changing.

[billsacks]

Great, thanks a lot @olyson ! I'll close this issue, and we'll consider redoing something like this again in a couple of months.