how to incorporate zarr's new open_consolidated method?

[rabernat]

Zarr has a new feature called consolidated metadata. This feature will make it much faster to open certain zarr datasets, because all the metadata needed to construct the xarray dataset will live in a single .json file.

To use this new feature, the new function zarr.open_consolidated needs to be called. So it won't work with xarray out of the box. We need to decide how to add support for this at the xarray level.

I am seeking feedback on what API people would like to see before starting a PR. My proposal is to add a new keyword argument to xarray.open_zarr called consolidated (default = False). An alternative would be to automatically try open_consolidated and fall back on the standard open_group function if that fails.

I played around with this a bit and realized that zarr-developers/zarr-python#336 needs to be resolved before we can do the xarray side.

cc @martindurant, who might want to weigh on what would be most convenient for intake.

[shoyer]

My proposal is to add a new keyword argument to xarray.open_zarr called consolidated (default = False)

This sounds like a pretty safe approach to me. (We could imagine later changing this to be the default behavior, if we end up deciding that we usually want to write Zarr datasets with consolidated metadata.)