-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
h5netcdf: Don't assume there are dimensions when there are chunks #10092
h5netcdf: Don't assume there are dimensions when there are chunks #10092
Conversation
Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient. |
I added a somewhat artificial test to not have to include an hsds service in the CI chain. |
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look reasonable. @rho-novatron Is this a speciality of h5pyd/hsds to return strings with chunks? I just want to make sure there is no other issue hidden which could be resolved upstream.
That is a good question, and I don't know the answer. I'll check and get back. |
It really does look as if chunking isn't really supported for scalar datasets, but that they are stored with layout class H5D_CHUNKED and thus having chunk dimensions == (1,). There are several places, both in h5netcdf and h5pyd that verifies that scalar datasets cannot be created with chunks. However, I can't really find anything in the HSDS documentation that says that a scalar dataset couldn't have a chunk. Perhaps it would be better to check for scalar datasets in h5netcdf and return None for scalar datasets? I think that might be the best balance between keeping xarray generic, h5pyd slightly raw and h5netcdf the adaptor logic? Do you have a better idea? |
I added a draft PR that makes concrete what I meant: h5netcdf/h5netcdf#259 |
Thanks for digging into this @rho-novatron. I was about suggesting something along your PR h5netcdf/h5netcdf#259. Responded over there. |
Thanks @rho-novatron for this fast interaction, great experience! |
len(var.chunks) > 0
butlen(var.dimensions) == 0
#10091whats-new.rst
api.rst