-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: don't mix arrays and groups #107
Comments
In the case of labels, I can definitely see correcting the structure as it currently stands. There has been ongoing confusion as seen with the currently open "labels" issues that we could likely address at the same time. I'm a bit more hesitant about saying generally that arrays and groups can't be at the same level. My preference would be to have metadata that clearly defines the semantics.
Similarly, I would avoid trying to encode too much in the names because invariably there will be difficulties getting everything need in a single group or array name.
And maybe relatedly, inspecting a plate today, I did have the impulse to add |
Agreed, but Zarr already provides a set of operational semantics for "list-like thing" (zarr group) and "value-like thing" (zarr array). Life would be easier if the OME data model had a clean mapping onto the zarr semantics, e.g. something like type definitions. On one extreme (most permissive typing), you could define that all elements of an OME-Zarr have the recursive type The current spec is somewhere in the middle: it looks something like And re: naming groups with integers -- what's the motivation for this? zarr uses integers as the names of chunk directories in nested storage, and for chunks themselves, so it seems like integer names should probably be reserved for that purpose. |
Guess I think of zarr groups as being more dictionary like than list like which may explain some of our differing opinion.
This is how I think of it in the general case.
Gotcha. If this issue is "don't mix the multiscale zgroup" then I could see adding that typing info to the spec, even if not necessarily doing it for all zgroups. (The top-level zgroup is of course a special case)
That's what I was getting at as well (though in Zarr v3 chunks themselves also get prefixes) |
No, you're right, groups are definitely more dict-like than list-like, that's just an oversight on my part.
Right, I support this tighter framing of the issue. I wasn't considering cases of zarrays outside of a multiscale group -- those situations should be considered separately. |
I didn't have time to fully read this issue; I will try to have a look later.
These integer named groups are super confusing. They should be avoided in the spec. (As a side-note: having the scale arrays with integer names is also not helping and calling them e.g. |
In the current spec, the zarr group for an image contains arrays (the different scale levels of the image data) as well as (optionally) a group (the
labels
group). E.g.,Storing arrays and groups in the same level of hierarchy can complicate semantics.
I would propose the following structure instead:
Under with this scheme, we can define a "multiscale group" as a collection of arrays with a specific group metadata scheme, and then re-use that definition.
I chose the name "image" arbitrarily, it could also be something like "raw" or "default", and maybe the name of this group could be reserved.
The text was updated successfully, but these errors were encountered: