WIP: Enhanced xarray function compatibility on input/output

[jthielen]

Description Of Changes

This PR utilizes the wrapper of #1223 across the library where it naturally fit, as well allowing broadcasting of xarray elements in the xarray preprocessing decorator. It also has an attempt at filling in kinematics grid parameters (however, there are still some errors in the code).

Checklist

• Closes #xxxx
• Tests added
• Fully documented

[jthielen]

@dopplershift I'm just now getting the chance to get back to this, and after running into some ...interesting... issues combining xarray, pint, and MetPy in current versions, I wanted to ask about what the target "xarray data structure" should be in v1.0 to guide the effort here. Should it be

• A DataArray with an ndarray inside, and units on the DataArray attribute?
  • Works with older versions of xarray (pre-0.13) and pint (pre-0.10)
  • Does not play nice when taking advantage of pint's NEP-18 compatibility in current versions (i.e., multiplying by a pint Quantity gives a mixed state with incorrect units)
  • Current behavior from Add arbitrary window smoother and refactor existing smoothers #1223/this PR
• A DataArray with a pint Quantity inside?
  • Requires newer xarray (at least 0.13) and pint (at least 0.10)
  • The more "future-proof" option (I think)
  • However, tests that this works across the xarray API are still considered WIP (see support for units with pint pydata/xarray#3594)
  • For end-user ease-of-use, may need re-implementation of/dependency on/vendoring of pint-xarray-type features that MetPy is missing

[dopplershift]

@jthielen 😭 The pain of being at the cutting edge.

First question: Is it possible that both are supported?

I'd like to get to the second, but that seems like a pretty big reach today (feel free to correct me if I'm wrong). I'm ok with updating to the latest versions; I have concerns about how much back-porting might be required.

[jthielen]

First question: Is it possible that both are supported?

Now that you say that, yes, it should be technically possible to support both, they would just become separate options in the wrap_output_like helper.

However, I think it would be rather confusing for end users to have two different kinds of DataArray output from MetPy functions that they would have to interact with differently. For instance, to apply an offset to, say, potential temperature, the first option would look something like

theta = potential_temperature(pressure, temperature)
theta.metpy.unit_array = theta_0.metpy.unit_array + 5 * units.K

(in general, lots of reliance on MetPy's accessor and explicitly handling magnitudes/units, and easy to mess up if you try multiplying a DataArray by something with a unit)

and the second like

theta = potential_temperature(pressure, temperature) + 5 * units.K

(more natural, can be easily conceptualized as a "Quantity with coordinates on top")

Things could also go quite wrong if these two are mixed outside of the relative safety of MetPy's calculation wrappers (e.g., units being dropped from the units-on-attribute DataArray, but not the Quantity-DataArray).

I'd like to get to the second, but that seems like a pretty big reach today (feel free to correct me if I'm wrong). I'm ok with updating to the latest versions; I have concerns about how much back-porting might be required.

I'm glad to hear the versions aren't an issue! Although, I'm not quite sure of what you exactly mean by "pretty big reach." Is it effort in implementation? Reliance on yet-to-be-finalized upstream functionality? Impacts on documentation/downstream code/training materials? Something else?

Implementation-wise, I really don't think it's that big of a deal. Since (in the philosophy of this PR at least) most of the calculation internals remain operating on Quantities, preferring the second option over the first is a simple switch in how the wrapper reconstructs the DataArray (and may even make using DataArrays internally in the calculations easier down the road). The "pint-xarray-type" quantification/dequantification helpers should also be easy to add to the accessors.

While I've not run into any issues recently with having Quantities-in-DataArrays in my own tests (with the exclusion of coordinates/indexes of course), I'm a bit nervous about relying on the technically-WIP compatibility in production code. However, the alternative (how it is now) definitely feels like an ugly API to commit to.

Switching to just using the second would no doubt require adjustments elsewhere. However, I think they would mostly be clean-ups to the unit-compatibility hacks that are required now? Though, any place where da.data is expected to be an ndarray, or da.units/da.attrs['units'] is expected to be available/needing to be set, however, would need to be changed (I think this is mostly in declaritive plotting, cross sections, and derivatives at the moment). Regardless, this likely an easy-to-moderate difficultly task if we standardize around the second option, but could be more complex if needing to support both options.

---

I guess a lot of this is rambling to say that:

• Standardizing on the second option alone seems ideally the best, but feels risky due to WIP upstream support
• Having both is most adaptable, but also most technically complex, and likely confusing to users
• Standardizing on the first option alone has inertia from the past going for it, but feels a shame to abandon the recent NEP-18 niceties, and may still be confusing for users.

[dopplershift]

@jthielen I see what you're saying. I think it's clear that the second is the most desirable option, and I agree having two similar, but different, paths is not a good user experience.

As I understand this, the main challenges we're looking at relying on some WIP support upstream is that things could:

1. Require us to update our wrapper
2. End up being buggy

All we really need right now is to commit to an interface for our users. It sounds to me like we have that, it just might be a slightly rough road. Worst case, people need to drop back to numpy arrays if they run into bugs, right?

[jthielen]

All we really need right now is to commit to an interface for our users. It sounds to me like we have that, it just might be a slightly rough road. Worst case, people need to drop back to numpy arrays if they run into bugs, right?

Sounds good, and I agree. I'll move forward with the Quantity-inside-DataArray option.

[jthielen]

Closing in favor of #1353.