PROPOSAL: Add upper level ERA data

[fmaussion]

This adds data from ERA-Interim reanalysis, monthly averages of upper level data. The file is 4,0M large (i.e. smaller than the other ones), highly compressed and in netcdf3 format. here's the dump:

netcdf eraint_uvz {
dimensions:
	longitude = 480 ;
	latitude = 241 ;
	level = 3 ;
	month = 2 ;
variables:
	float longitude(longitude) ;
		longitude:_FillValue = NaN ;
		longitude:units = "degrees_east" ;
		longitude:long_name = "longitude" ;
	float latitude(latitude) ;
		latitude:_FillValue = NaN ;
		latitude:units = "degrees_north" ;
		latitude:long_name = "latitude" ;
	int level(level) ;
		level:units = "millibars" ;
		level:long_name = "pressure_level" ;
	short z(month, level, latitude, longitude) ;
		z:number_of_significant_digits = 5 ;
		z:units = "m**2 s**-2" ;
		z:scale_factor = -1.7250274674968 ;
		z:long_name = "Geopotential" ;
		z:add_offset = 66825.5 ;
		z:_FillValue = NaN ;
		z:standard_name = "geopotential" ;
	short u(month, level, latitude, longitude) ;
		u:number_of_significant_digits = 2 ;
		u:units = "m s**-1" ;
		u:scale_factor = -0.00157270493804553 ;
		u:long_name = "U component of wind" ;
		u:add_offset = 26.96875 ;
		u:_FillValue = NaN ;
		u:standard_name = "eastward_wind" ;
	short v(month, level, latitude, longitude) ;
		v:number_of_significant_digits = 2 ;
		v:units = "m s**-1" ;
		v:scale_factor = -0.000477819996337667 ;
		v:long_name = "V component of wind" ;
		v:add_offset = -1.46875 ;
		v:_FillValue = NaN ;
		v:standard_name = "northward_wind" ;
	int month(month) ;

// global attributes:
		:Conventions = "CF-1.0" ;
		:Info = "Monthly ERA-Interim data. Downloaded and edited by fabien.maussion@uibk.ac.at" ;
}

[fmaussion]

This will allow to show examples like:

import xarray as xr
import matplotlib.pyplot as plt
import cartopy.crs as ccrs

ds = xr.open_dataset('/home/mowglie/eraint_uvz.nc', engine='scipy')
f = plt.figure(figsize=(14, 7))
u = ds.u.sel(level=500, month=7)
v = ds.v.sel(level=500, month=7)
ws = (u**2 + v**2)*0.5
ax = plt.axes(projection=ccrs.PlateCarree())
ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(), density=4,
              color=ws.values, cmap=plt.get_cmap('cool'))
ax.coastlines(color='grey');

[rabernat]

This is a great example dataset. I think it would really enhance to documentation.

[shoyer]

I'd love to add this, but I am somewhat concerned that redistributing this data could violate ECMWF's terms of use.

[fmaussion]

That's a good point. I'll check the terms of use when I get back from EGU.

ERA-Interim data is free to download and this kind of aggregated + compressed data shouln't be a big deal, but you're right that we should go for the safe side.

[shoyer]

Last time I checked you need explicit permission to redistribute. But I really can't see why that would be a problem for ECMWF, so perhaps it's worth asking.

[fmaussion]

OK we have approval from ECMWF. I just wonder if I couldn’t add some more data (e.g. more levels/variables) for even more examples.

What would you say is an acceptable file size for the demo? Right now we have air_temperature at 7.4M and rasm at 17M.

[shoyer]

What would you say is an acceptable file size for the demo? Right now we have air_temperature at 7.4M and rasm at 17M.

Smaller is definitely better -- the goal is for this to be fast enough to be interactive, even if everything in a tutorial session is trying to download the dataset at the same time over poor conference wifi :). There are no hard limits, but I like the file size of the current datasets. We could potentially go a bit bigger for a supplemental dataset if it's clear that it's larger.

A table with names, descriptions and file sizes for each tutorial dataset in the docstring for xarray.tutorial.load_dataset could help.

Note that xarray already supports gzipped netcdf3 (just using the standard library + scipy) so that's another option for further comprehension.

[shoyer]

Shall we go ahead and merge this dataset? It seems like it could be a nice addition.

[fmaussion]

@shoyer thanks for the heads up - I wanted to get back to this and add an example to the docs right away, and I will try to do this before going on holidays by the end of the week. I'll keep you informed by Thursday

[fmaussion]

OK, just too much to do before leaving - if you want to go ahead and merge please do. I'll be able to contribute to the doc again in three weeks, hopefully with less duties and more time for xarray after the break :-(

[keewis]

should we merge this and use it as part of the visualization gallery, to show-case the new streamplot and quiver functions?

[shoyer]

+1 for merging this from my perspective. I think this is a nice example dataset for streamplot and quiver.