Question about handling of missing values in cmor_variable.c

[ehogan]

In cmor_variable.c, in this part of the code, the missing_value provided by the user is compared with the missing_value provided in the MIP table (see, e.g. the Emon MIP table).

If the missing_value provided by the user is non-standard (i.e. not 1e20), when the comparison above is performed, it will always fail, meaning the missing_value provided by the user is replaced with the missing_value provided in the MIP table, which is then written to the output netCDF files.

Would it be possible to remove this check? Replacing the missing_value if it isn't the standard value isn't the correct thing to do, since the value of missing_value written to the output netCDF files will then not correspond to the missing values in the data.

[doutriaux1]

@ehogan are you saying that the data is actually written out with the "user missing value" but the missing_value attributes is the one from the tables?

[ehogan]

The value of the missing_value attribute in the output netCDF file is the value from the MIP table and not the value I provided via cmor_variable.

[ehogan]

Apologies @doutriaux1 for misunderstand your question; after talking with Jamie and checking our data, I now understand that the missing values in the data, as well as the missing_value, are replaced with the standard missing value. Totally my fault for not reading the documentation correctly. However, would you consider making a slight adjustment to the history that is written to the output netCDF file? Instead of altered by CMOR: replaced missing value flag (-1.07374e+09) with standard missing value (1e+20)., perhaps something like altered by CMOR: replaced missing value flag (-1.07374e+09) and corresponding data with standard missing value (1e+20).?

[taylor13]

Yes this is a good idea. Not terribly urgent I don't think, but let's reword the history message for next release.

[ehogan]

Thanks @taylor13, I agree this isn't urgent :)

[taylor13]

I think we can now reword the "history" text as suggested by @ehogan at the end of #450 (comment)