Testing: checks for data and metadata should be separated

[jmccreight]

Expected Behavior

For example, #45

In CI when metadata changes are applied to the CHRTOUT and CHANOBS files, the checks on those files fail and it is ambiguous if the failure is data or metadata. One can not tell and to be sure, should reproduce the test locally. I think it's better to be explicit about this rather than cause all that work

[kafitzgerald]

No worries if it’s too much for the log, but on a failure I think it’d be nice to just see the output from nccmp. This would give us a lot to work with before having to go rerun offline.

[tjmills]

Does it not print in the log now? I am pretty sure nccmp outputs are included?

     Variable Group  Count       Sum    AbsSum       Min       Max     Range  \
0  streamflow     /     44  0.303615  1.076170 -0.052830  0.122318  0.175148   
1   q_lateral     /     34  0.268734  0.491489 -0.051165  0.039766  0.090931  

https://travis-ci.org/NCAR/wrf_hydro_nwm_public/builds/379002191

[kafitzgerald]

Ah, maybe just not for metadata.

Was hoping for something like this:

DIFFER : NAME OF GLOBAL ATTRIBUTE : model_total_valid_times : GLOBAL ATTRIBUTE DOESN'T EXIST IN NWM/run_reference/201108281400.CHRTOUT_DOMAIN1
DIFFER : VARIABLE "time" IS MISSING ATTRIBUTE WITH NAME "valid_min" IN FILE "NWM/run_reference/201108281400.CHRTOUT_DOMAIN1"
DIFFER : VARIABLE "time" IS MISSING ATTRIBUTE WITH NAME "valid_max" IN FILE "NWM/run_reference/201108281400.CHRTOUT_DOMAIN1"
DIFFER : NUMBER OF ATTRIBUTES : VARIABLE : time : 3 <> 5

https://travis-ci.org/NCAR/wrf_hydro_nwm_public/builds/378172024?utm_source=github_status&utm_medium=notification

[tjmills]

I have now separated the metadata and data checks but did not closed this issue with the PR as I should have. Closing now