-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use Arrow C data interface format strings? #61
Comments
I don't see any link to endianness in the arrow document, though it seem like we have both in current spec. Do you know know common it is to have different endianness in the same Zarr ? |
Endianness is important given data may be produced on one system and consumed on another. I don't know of any cases where endianness differs between different arrays in the same hierarchy, that would probably be rare. But it would be nice to include this information within the array metadata. |
As you know, both HDF5 and netcdf support setting endianness on a per-array |
WRT format strings. The binary and large binary distinction seem |
WRT the time types, that has been an issue with netcdf for a while. |
Also related to issue ( zarr-developers/numcodecs#227 ) |
Reading The Arrow C data interface I'm wondering if we should consider following any of the approach described there for the zarr v3.0 core protocol spec.
In particular, the format strings for core data types may be easier to handle than the currently used numpy-style format strings. Although unfortunately there is no concept of endianness.
The text was updated successfully, but these errors were encountered: