Recognize nz as the zarr convention implemented by xarray? #11485
Unanswered
dblodgett-usgs
asked this question in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Xarray's mapping of the netCDF data model onto Zarr predates any mechanism for registering conventions — ZEP0004 mentions it as one of four examples of this situation. These xarray conventions have been around for a decade or so and non-Python readers that interoperate with xarray-written stores have aligned with the same behavior by matching xarray's behavior through inspection.
A recently written zarr-convention, nz, states the structure xarray writes as requirements independent of the implementation. Fully populated dimension names, one length per dimension label within a group, dimension coordinates identified by name-to-dimension match,
_FillValueseparated from the storagefill_value. Xarray datasets already follow the convention.Would the xarray development team consider adding NZ-1.0 as the base zarr V3 convention that xarray writes?
I think the NetCDF-C team would be willing to do the same and I would imagine that would be enough to establish the implicit xarray Zarr 3 convention as an explicit convention that the broad community of NetCDF users need to fully adopt a Zarr basis for their domain conventions.
Regarding the concern that this is a domain convention that should not be part of xarray: NZ is based on the (also domain agnostic) NetCDF Users Guide layer of NetCDF. Dimension names, co-extent within a group, name-to-dimension coordinate matching, and semantic fill values are domain agnostic.
The benefit to this would be downstream rather than in xarray. Conventions in this framework advance by count of independent implementations, and NZ is at Proposal. A declaration from xarray would convert the de facto structure into something a zarrs, zarr.js, netCDF-C NCZarr, or GDAL Zarr driver implementer can cite when they need to say what correct means — and gives them a fixed reference to test against instead of a moving library to track.
All reactions