Problematic time units from PMEL

[wckoeppen]

Perhaps tangential to the discovery phase (more in the analysis phase), but I've been finding that when accessing climate forecast models from PMEL I have to manually set the time units in order to use the netCDF module.

Apparently PMEL is setting their time units to:
sec since YYYY-MM-DD HH:MM:SS

which causes netCDF4.num2date to choke because it only accepts "days, hours, minutes, seconds, milliseconds or microseconds", not "sec".

Common type of problem?

[wckoeppen]

I guess it is common. Apparently we modify nearly every time unit string we get from data providers (using ncml). Great!

[rsignell-usgs]

"we" meaning Axiom, right? Do you also report these errors back to the data providers (e.g. PMEL)?

[wckoeppen]

I wish I could say that was the regular policy. The reality is that it takes us 5 minutes to change the ncml files and correct problems like this, whereas reporting errors, waiting for a response (if one ever comes), and then accepting new files takes significantly longer. So we generally only send reports back to data providers if something is unintelligible or unusable. For one-off (probably not the right word) models like the ones I'm using from PMEL, AOOS may be the only web-accessible host anyway.

[dpsnowden]

I'm not sure I understand the "problem". The use of time units as described above is the CF convention. See http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#time-coordinate. The udunits.dat library accepts [sec(s), s, second(s)] since YYYY-MM-DD HH:MM:SS. If the netCDF4.num2date function only accepts a subset of those options I don't think it's an error on the data providers part, it just means that netCDF4 didn't implement the full specification.

How should we document this for the final report? Is the issue closed?

[wckoeppen]

Fair enough, as I said, it's probably tangential to data discovery. But it has been problematic for me for actual data analysis. Maybe we can put in a request for the netCDF library to be improved? Meanwhile, we'll just keep fixing the units so they'll work with the available tools.

[dpsnowden]

OK, I'd like remember this for the final report and describe it as a "nice to have" extension to client software. Not sure where the netCDF4 library is developed but I would hope they have a way to request enhancements. @rsignell-usgs ?

[jkupiec]

I have added the Enhancement label so that we will know to list this item in the final report.

[rsignell-usgs]

This is not a problem with NetCDF4. The problem is a provider issue -- "sec" is not an allowable UDUNIT, and therefore this data is not CF compliant. The only acceptable way to specify seconds as a base is as "second" or "s"

From there you can append prefixes such as "milli", or "micro", but "sec" is NOT allowed.