Deal with salinity unit variants better

[durack1]

The current version of all sos obs store data using the "PSS-78" scale/units (which stands for the Practical Salinity Scale 1978) and variants around this:

[durack1@oceanonly ~/141105_pcmdi_metrics]$ ls /clim_obs/obs/ocn/mo/sos/*/ac/*.nc
sos_pcmdi-metrics_Omon_JPL-Aquarius-v2_201108-201304-clim.nc
sos_pcmdi-metrics_Omon_NODC-WOA09_177201-200812-clim.nc
sos_pcmdi-metrics_Omon_UCSD-ARGO_200401-201212-clim.nc
[durack1@oceanonly ~/141105_pcmdi_metrics]$ ls /clim_obs/obs/ocn/mo/sos/*/ac/*.nc | xargs -n 1 ncdump -h | grep 'sos:units'
                sos:units = "1e-3" ;
                sos:units = "PSS-78" ;
                sos:units = "Practical Salinity Scale 78" ;

Modeling centers often use psu (the case with GFDL), and consequently we we should make sure that the metrics package automatically deals with this variable, which is not the case currently:

OBS SHAPE IS  (12, 72, 144)
 sos   ACCESS1-0   (12, 72, 144)   (12, 72, 144)    default
Error while processing observation sos for variable default:
Could not convert model units (psu) to obs units: (PSS-78)

I suggest that any of the valid variants 1e-3,g kg-1,g/kg,ppt,Practical Salinity Scale 1978,Practical Salinity Scale 78,pss78,PSS78,pss-78,PSS-78,psu,PSU should be dealt with automagically. It's also likely that NCAR will provide data literally offset by 1e-3 (so rather than valid values from ~2 to ~40, they're ~0.002 to ~0.04) and the following durolib code checks and corrects this: https://github.com/durack1/pylib/blob/master/durolib.py#L189-242

As an aside, it appears that https://github.com/PCMDI/cmor can in fact deal with psu:
https://github.com/PCMDI/cmip5-cmor-tables/blob/master/Tables/CMIP5_Omon#L2925-2989

[durack1]

As I expected it doesn't seem that even the latest release of UDUNITS (2.2.17) can deal with any of the salinity "unit" variants: http://www.unidata.ucar.edu/software/udunits/udunits-2.2.17/doc/udunits/udunits2.html#Database

[doutriaux1]

thanks for checking though!

[durack1]

@doutriaux1 did you need a brain dump for me on this? Or do you have enough to kick things off?

[doutriaux1]

i got enough to get started I think, I'll get back to you when I know more precisely what stops me.

[durack1]

Looks like we'll have to go it alone on this one: Unidata/UDUNITS-2#27 (comment)

As we're not adding salinities in the package we won't hit the issue that was raised in the comment above.

[durack1]

@doutriaux1 moving to a 1.1 milestone as 1.0.0 should be tagged now.

[durack1]

@doutriaux1 it would be great to get this sorted out next week.. I really do need to start generating some metrics on the CMIP5 salinity fields..

[durack1]

@doutriaux1 any movement on this?

[jservonnat]

@durack1 @doutriaux1 same probleme on my side:
Error while processing observation sos for variable default:
Could not convert model units (psu) to obs units: (PSS-78)

[durack1]

Yep, this is something that I hope @doutriaux1 can get to early next week.. He's under the pump now finalising the 2.2 release for UV-CDAT..

[jservonnat]

Perfect. I've fixed (at least for the moment) the bounds issue, so now I'm ready to compute metrics on the ORCA grid.

[durack1]

@jservonnat great, you're linked into this issue so you should get an email notification when the status changes (it's fixed)

[doutriaux1]

@durack1 please confirm, I'm going to go with this for now:

1 psu = 1 PSS-78 = 1 Practical Salinity Scale 78 = 1.e-3

please correct me if I'm wrong.

[doutriaux1]

@durack1 getting there:

/lgm/uvcdat/2015-03-20/Library/Frameworks/Python.framework/Versions/2.7/bin/pcmdi_metrics_driver.py:341: 
UserWarning: Model and observation units differed, converted model (psu) to observation unit (1e-3) scaling: 1 offset: 0

[durack1]

@doutriaux1 yep your equality above is correct