Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pint for units #160

Merged
merged 80 commits into from
Aug 2, 2021
Merged

Use pint for units #160

merged 80 commits into from
Aug 2, 2021

Conversation

TomNicholas
Copy link
Contributor

@TomNicholas TomNicholas commented Jul 8, 2021

Uses pint-xarray to first add units to the dataset during pre-processing, then convert depth to metres. Might help close #31

Doesn't seem to break any tests (I had one fail locally but that also failed before I made any changes).

Not really sure what other changes might be required to use pint in this package properly but this is a start.

@TomNicholas
Copy link
Contributor Author

Installs and uses cf_xarray.units so possibly related to #59

@jbusecke
Copy link
Owner

jbusecke commented Jul 9, 2021

It seems like this actually works quite well. The only thing we need now is to ignore models that provide no unit attrs (there are a few 😭). I feel that should be an upstream option though?
If we want to be more fancy, we could 'guess the units', but that seems risky.

@@ -436,6 +422,8 @@ def combined_preprocessing(ds):
ds = broadcast_lonlat(ds)
# shift all lons to consistent 0-360
ds = correct_lon(ds)
# codify units with pint
ds = ds.pint.quantify()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do this inside correct_units?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would fix your bug, but whether or not you want to conceptually combine "adding codified physical units to the data" and "potentially change what units the data is expressed in" is up to you.

@jbusecke jbusecke self-assigned this Jul 9, 2021
@jbusecke
Copy link
Owner

jbusecke commented Jul 9, 2021

So now that I have put in another if statement to not run on the datasets that are missing units it complains about this:

incompatible units for variable 'lev': cannot use 'centimeters' as a unit

Full error here.

I first thought that centimeters might be a weird spelling and it might not be in the pint registry, but then I checked:

from pint import UnitRegistry
ureg = UnitRegistry()
'centimeters' in ureg

and got True...

not sure what is going on here.

@keewis
Copy link
Contributor

keewis commented Jul 9, 2021

it is a bit difficult to see, but the unit attribute is currently a str. To get the conversion of dimension coordinates to work properly you have to call .pint.quantify first.

@TomNicholas
Copy link
Contributor Author

But we are calling ds = ds.pint.quantify() first?

@keewis
Copy link
Contributor

keewis commented Jul 9, 2021

that does not seem to be the case for the traceback posted in #160 (comment)?

@TomNicholas
Copy link
Contributor Author

Right I see what you mean - in the failing test it only imports correct_units(), without first calling .quantify.

@jbusecke
Copy link
Owner

jbusecke commented Jul 9, 2021

Thanks a lot for the pointers. I think I get it now (still very new to the whole pint thing).

Ill fiddle some more...

@keewis
Copy link
Contributor

keewis commented Jul 9, 2021

just checking: if you want to get the string "meter" after calling dequantify, use "P" as the format. If you want "m" instead, use "~P".

@jbusecke
Copy link
Owner

Ah thanks a lot for checking in @keewis, I misunderstood that one. Lets see if the tests pass this time. Ill try to get this merged today.

@jbusecke jbusecke merged commit 44de62c into jbusecke:master Aug 2, 2021
@jbusecke
Copy link
Owner

jbusecke commented Aug 2, 2021

Thanks a lot @keewis and @TomNicholas for getting this started and all the detailed pointers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unify variable attributes and units
4 participants