compare_namelist.pl fails to find differences and mislabels differences that it does find

[billsacks]

I understand from @jedwards4b that compare_namelist.pl will be replaced by a python script at some point. I'm opening this issue to point out problems in the current operation of this script, which may (or may not) help in the creation of the new script.

Note that I used the cesm-development version of cime for the test I mention below... I'm not sure if this exact same issue occurs in the ESMCI version.

I ran a test suite comparing https://svn-ccsm-models.cgd.ucar.edu/clm2/branch_tags/mountain_glaciers_old_behavior_tags/mountain_glaciers_old_behavior_n11_clm4_5_9_r184 (soon to be clm4_5_9_r185) with clm4_5_9_r184. These clm tags use tag clm4_5_8_r180_cime4.5.14, which is identical to the head of cime in terms of compare_namelist.pl.

I expected namelist differences for all clm45 and clm50 cases. Indeed, that seems to be the case when spot checking a few by hand. But the namelist comparison test is listed as pass for most tests.

It picks up differences for a handful of tests, where there is just one difference, like this:

[yslogin1:/glade/p/cesmdata/cseg/ccsm_baselines]$ test=ERS_D_Ld3.f09_g16_gl5.IG1850CRUCLM50BGCCROP.yellowstone_intel.clm-clm50KitchenSink
[yslogin1:/glade/p/cesmdata/cseg/ccsm_baselines]$ diff -u clm4_5_9_r184/$test/CaseDocs/lnd_in mountain_glaciers_old_behavior_n11_clm4_5_9_r184/$test/CaseDocs/lnd_in
--- clm4_5_9_r184/ERS_D_Ld3.f09_g16_gl5.IG1850CRUCLM50BGCCROP.yellowstone_intel.clm-clm50KitchenSink/CaseDocs/lnd_in    2016-07-02 19:52:37.705217000 -0600
+++ mountain_glaciers_old_behavior_n11_clm4_5_9_r184/ERS_D_Ld3.f09_g16_gl5.IG1850CRUCLM50BGCCROP.yellowstone_intel.clm-clm50KitchenSink/CaseDocs/lnd_in 2016-07-07 01:51:59.551402000 -0600
@@ -148,6 +148,7 @@
 /
 &clm_glacier_behavior
  glacier_region_behavior = 'single_at_atm_topo','virtual','multiple'
+ glacier_region_melt_behavior = 'remains_in_place','replaced_by_ice','replaced_by_ice'
 /
 &ch4par_in
  fin_use_fsat = .false.

But misses differences for the majority of tests, where there are two consecutive differences, like this:

[yslogin1:/glade/p/cesmdata/cseg/ccsm_baselines]$ test=ERP_D_Ld5.f19_g16.ICRUCLM50BGC.yellowstone_intel.clm-default
[yslogin1:/glade/p/cesmdata/cseg/ccsm_baselines]$ diff -u clm4_5_9_r184/$test/CaseDocs/lnd_in mountain_glaciers_old_behavior_n11_clm4_5_9_r184/$test/CaseDocs/lnd_in
--- clm4_5_9_r184/ERP_D_Ld5.f19_g16.ICRUCLM50BGC.yellowstone_intel.clm-default/CaseDocs/lnd_in  2016-07-02 19:53:22.070625000 -0600
+++ mountain_glaciers_old_behavior_n11_clm4_5_9_r184/ERP_D_Ld5.f19_g16.ICRUCLM50BGC.yellowstone_intel.clm-default/CaseDocs/lnd_in   2016-07-07 01:51:50.223256000 -0600
@@ -141,6 +141,8 @@
  ncrit = 1.d-9
 /
 &clm_glacier_behavior
+ glacier_region_behavior = 'single_at_atm_topo','virtual','multiple'
+ glacier_region_melt_behavior = 'remains_in_place','replaced_by_ice','replaced_by_ice'
 /
 &ch4par_in
  fin_use_fsat = .false.

Furthermore, for the tests where it picks up a difference, the difference listed in TestStatus.log is wrong:

FAIL namelist compare: lnd_in differs
NEW: &clm_humanindex_inparm
BASELINE: &cnmresp_inparm
NEW: &cnmresp_inparm
BASELINE: &photosyns_inparm
NEW: &cnfire_inparm
BASELINE: &lifire_inparm
NEW: &clm_canopy_inparm
BASELINE:

[billsacks]

By the way: I personally haven't found a lot of value in compare_namelists. That's not to say that it isn't useful (it seems like some people DO find value in it): I just mean to say that you should NOT take my issue report as suggesting that I want this fixed at high priority :-)

[jgfouca]

@billsacks : ACME has long been using a python version of compare_namelists. This version is now in CIME at scripts/Tools/compare_namelists

[billsacks]

Nice, okay, thanks - I'll close this.