Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comparison & possible combination with zarr v3 sharding #17

Open
jstriebel opened this issue May 18, 2022 · 0 comments
Open

Comparison & possible combination with zarr v3 sharding #17

jstriebel opened this issue May 18, 2022 · 0 comments

Comments

@jstriebel
Copy link

Great to see this project! We are currently working on a sharding mechanism for zarr v3, which will be part of the specification and is intended to be available broadly across implementations. You can find the current WIP state of the specification here:
https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/extensions/storage-transformers/sharding/v1.0.html

This is the related spec issue, there are also links to prototype PRs:
zarr-developers/zarr-specs#127

This sharding mechanism is mostly aimed at combining array chunks into different units. My current understanding of this store implementation is that a folder can be relayed onto another store, which seems to tackle other use cases. IMO it would be great to have a clear understanding of the differences and common usage patterns. What's your view on this, @thewtex?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant