Test against xarray release candidate

[spencerahill]

Xarray has a v0.10 release candidate out. It's a big release. We should check that nothing breaks on our end, both for our own sake and as a service to xarray in the case we find any bugs.

More generally moving forward, I think it would be useful to have a Travis branch that tests against the xarray dev branch; it can be permitted to have failures. Xarray does the same for pandas and dask; e.g. here for the pandas dev branch and the "allow failures" section of their .travis.yml.

@micahkim23, I'm going to put you on this one, as I think it's a good entree into the packaging/release/CI side of things. For the v0.10rc1 part, I would clone that release to your local machine, make sure its tests pass, and then make sure our tests pass using it. For the dev branch/CI part, just mimic the examples I gave above.

Let us know if you have any questions or trouble.

[spencerahill]

@micahkim23 I should have mentioned, this one is more time sensitive than anything else you're working on aospy-wise. So I would put the others on hold and do this first. Want to make sure we catch any bugs before the release candidate gets adopted.

[micahkim23]

@spencerahill One issue that I ran into was I got errors while running our tests. This was on my machine which had old versions of h5py and netCDF4 (h5py-2.7.0 and netCDF4-1.2.4). Once I upgraded to the latest versions of h5py and netCDF4, the tests worked (h5py-2.7.1 and netcdf4-1.3.1).

[spencerahill]

Are the old versions older than the minimum requirements that xarray 0.10 lists? If yes, then no problem. If no, then we should probably dig in further.

[micahkim23]

When I ran the xarray tests, I got 2 failed, 2611 passed, 101 skipped, 11 xfailed, 95 warnings. The 2 failures were both NotImplementedError: h5netcdf does not yet support unlimited dimensions.

[spencerkclark]

@micahkim23 I think you may need to upgrade your h5netcdf installation. h5netcdf and xarray should support this now; see h5netcdf/h5netcdf#33 and pydata/xarray#1637.

[micahkim23]

xarray 0.10 doesn't list minimum versions for netCDF4 or h5py.

[spencerahill]

Oh right, because they're technically optional.

I agree with @spencerkclark's suggestion to try updating h5netcdf. But h5netcdf isn't something that we particularly rely on, so let's just leave it as is whether or not the update fixes the tests.

Once I upgraded to the latest versions of h5py and netCDF4, the tests worked (h5py-2.7.1 and netcdf4-1.3.1).

To clarify, you're saying all of our tests now pass w/ xarray 0.10? If so, that's great!

[micahkim23]

@spencerahill the update for h5netcdf fixed the xarray tests. Yeah, our tests pass with xarray 0.10.

[spencerahill]

Great! So only the Travis xarray-dev bit is left to do on this one.

[micahkim23]

I'm thinking about the approach to adding the Travis xarray-dev bit. Do we want to test the xarray-dev bit for each version of python that we have or just for python 3.6?

[spencerahill]

I think just one Python version should be fine. Since xarray themselves test against all of the Python versions we support, it would be unlikely that an update on their end would yield a problem for aospy on one Python version but not the others.

More what we're after is: if they inadvertently introduce something that didn't account for our particular use case and thus caused a bug for us on all Python versions.

So yes, just 3.6.