Wrong activity_id warning (should be error?), etc..

[sashakames]

I ran prepare on ssp126 (ScenarioMIP). The activity_id global attribute was mistakenly CMIP. Should PrePARE exit with an error in this case? Otherwise sites may publish these files.

ESC[1;34;47m!!!!!!!!!!!!!!!!!!!!!!!!!
!
! Warning: Your input attribute "activity_id" with value
! "CMIP" will be replaced with value "ScenarioMIP"
! as defined for experiment_id "ssp126".
!
! See Control Vocabulary JSON file.(/usr/local/cmip6-cmor-tables/Tables/CMIP6_CV.json)
!
!
!!!!!!!!!!!!!!!!!!!!!!!!!ESC[0m

[sashakames]

Also the wording in the warning should be corrected: "will be" -> "needs to be"

[sashakames]

Rather than create another issue. Here's another message treated as a warning, but might be an Error:

! In function: _CV_checkParentExpID
! ESC[0m

ESC[1;31;47m!!!!!!!!!!!!!!!!!!!!!!!!!
!
! Error: Your input attribute "parent_variant_label" is not defined
! properly for historical
! Please refer to the CMIP6 documentations.
!
!
!!!!!!!!!!!!!!!!!!!!!!!!!ESC[0m
In my example:
:variant_label = "r1i1p1f2" ;
:parent_variant_label = "r1i1p1f1" ;

[durack1]

@sashakames thanks for this, the first is a problem, we should not allow incorrect activity_id entries, when we know these for each exprriment_id.

For the second, it’s perfectly valid for an r1i1p1f2 simulation to be spawned of an r1i1p1f1 parent, so a warning is correct here, although it should say warning, not error.

@taylor13 correct?

[sashakames]

I should mention this is with v3.5.0. I'm seeing a different error now (checking Ron's UA MCM data) that I haven't with whichever old version, but maybe I'll open a separate issue for that...

[sashakames]

More problem examples that should be caught:

CMIP6/CFMIP/BCC/BCC-CSM2-MR/amip/r1i1p1f1/AERmon/od550aer/gn/v20190801/od550aer_AERmon_BCC-CSM2-MR_amip_r1i1p1f1_gn_197901-201412.nc
CMIP6/CMIP/FIO-QLNM/FIO-ESM-2-0/piControl/r1i1p1f1/Omon/thetao/gn/v20190926/thetao_Omon_PCMDI-test-1-0_piControl-withism_r3i1p1f1_gn_016201-016201.nc

In this case the directory structure incorrectly has CMIP for the activity id, otherwise fine:
/p/css03/esgf_publish/CMIP6/CMIP/CAS/FGOALS-g3/ssp126/r1i1p1f1/Amon/rsus/gn/v20190818/rsus_Amon_FGOALS-g3_ssp126_r1i1p1f1_gn_201501-201912.nc

[durack1]

@sashakames how are these errors occurring? Surely the PrePARE checking would fail with these errors?

[sashakames]

Could be result of an old bug. In the FIO case, it looks like they completely misconfigured CMOR. The path vs global attributes really aught to be checked as they may have specified the path correctly and that would indicate that they had a misconfiguration.

Thanks for taking this Chris @mauzey1 - I think its essential we can state with confidence that PrePARE will not let bad data get through.

All the example files should be rejected. I also have the UA ssp126 data with the error available.

[taylor13]

1. Concerning the activity_id for ssp126. I agree this is a fatal error and I would reword the error message as follows:

! ERROR encountered by CMOR3/PrePARE:
!       For experiment_id "ssp126", the value of "activity_id" specified by 
!       the data provider is "CMIP", 
!       but should be "ScenarioMIP".
!
! See Control Vocabulary JSON file.(/usr/local/cmip6-cmor-tables/Tables/CMIP6_CV.json)

2. Concerning

:variant_label = "r1i1p1f2" ;
:parent_variant_label = "r1i1p1f1" ;

although not the same, this is not an error. (We recommend the labels be the same whenever that makes sense, but we don't require it, and sometimes it would be inappropriate.)
I think CMOR/PrePARE should just ignore the inconsistency, and assume it would not make sense to specify them as the same.

3. Concerning the directory structure, CMOR3 will, by default, correctly generate the directory structure. For PrePARE to check the directory structure (unless the path were required to be passed to it somehow), it would have to issue system commands, and I think we wanted to avoid that. Is there some way the publisher could check the directory structure outside of PrePARE?