Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to specify ZarrSeparator #241

Closed
stuarteberg opened this issue Sep 7, 2020 · 4 comments
Closed

How to specify ZarrSeparator #241

stuarteberg opened this issue Sep 7, 2020 · 4 comments

Comments

@stuarteberg
Copy link
Contributor

By default, zarr chunk files use . as the axis separator in their names, e.g. 10.20.30. But in Python, one can use a NestedDirectoryStore to store in a directory tree, effectively using / as the separator, e.g. 10/20/30.

Unfortunately, the metadata in .zarray does not specify which format is used. (This is a known shortcoming of the zarr spec.)

Is there any way to tell neuroglancer which format a zarr array uses? I notice that the source code seems to support both, at least in theory. But I can't figure out how to tell neuroglancer that my zarr array uses the / separator in chunk names instead of the default . separator. Is there a way to do it?

@jbms
Copy link
Collaborator

jbms commented Sep 7, 2020

You can use the key_encoding property:

https://google.github.io/tensorstore/tensorstore/driver/zarr/index.html

@jbms jbms closed this as completed Sep 7, 2020
@stuarteberg
Copy link
Contributor Author

Thanks very much for your reply. Sorry if I'm being dense, but I'm still confused. I'm trying to open the volume in neuroglancer. I can use either zarr or tensorstore to create the array on disk, but either way, the key_encoding setting is not stored in the metadata on disk, as far as I can tell.

Is there a way to somehow provide a tensorstore-style JSON volume configuration to neuroglancer? So far, I'm just providing a source in the form of zarr://http://localhost:9000/path/to/myarray.zarr. Are you saying neuroglancer also accepts tensorstore JSON data in that text control?

Again, sorry if that's a dumb question. I'm clearly missing something here.

@jbms jbms reopened this Sep 7, 2020
@jbms
Copy link
Collaborator

jbms commented Sep 7, 2020

Oh sorry, I didn't notice that you opened this issue on neuroglancer rather than tensorstore. Indeed, there is not at the moment support for specifying the separator. If you want to add support, I'd suggest making it a query string parameter, like

zarr://http://localhost:9000/path/to/myarray.zarr?separator=/

@joshmoore
Copy link

I will be looking at a PR for zarr-python at least which will use one of a small number of options for finding the key. Work on the Java side has taken place in bcdev/jzarr#17:

  • heuristic that loops through chunks "0.0.0", "0/0/0", "0.0.1", "0/0/1" searching for existence... (with the downside being the overheaded of the search if no key_separator was chosen)
  • uses optional metadata in the .zarray (with the downside that we need to agree on this metadata)
  • ...

@jbms jbms closed this as completed in 6b2111e Apr 16, 2021
sandyhider pushed a commit to jhuapl-boss/neuroglancer that referenced this issue Apr 16, 2021
* feat(zarr): support dimension_separator in .zarray file

This implements the spec change described here:
zarr-developers/zarr-python#715

This also adds a query string parameter by the same name.

Fixes google#241.

* feat(multiscale mesh): add additional debugging checks

* fix(multiscale mesh): turn off multiscale mesh fragment debugging by default

This was mistakenly left on by default.

Co-authored-by: Jeremy Maitin-Shepard <jbms@google.com>
hannah-martinez added a commit to jhuapl-boss/neuroglancer that referenced this issue Apr 30, 2021
* feat: add state saving/sharing ui allowing users to send their state to a server that accepts a json encoded version of the neuroglancer state. the "state server" is expected to return a url that will respond that same json string

* feat(zarr): support dimension_separator in .zarray file

This implements the spec change described here:
zarr-developers/zarr-python#715

This also adds a query string parameter by the same name.

Fixes google#241.

* feat(multiscale mesh): add additional debugging checks

* fix(multiscale mesh): turn off multiscale mesh fragment debugging by default

This was mistakenly left on by default.

* simplified state sharing code. replaced dialog with a state server config file and a select element if there is more than 1 server

* moved state share code out of viewer into separate file, renamed state key for the selected state server

* feat(ui): add support for configurable side panels

Previously, only two side panels were supported, the layer side panel
and the selection details side panel.  Only a static configuration was
supported, a single column on the right, with the layer side panel on
top and the selection details below, and the vertical height divided
equally.

This commit adds support for an arbitrary number of side panels, to
the left/right/top/bottom of the main data panels.  Panels can be
moved via drag and drop, and the size can also be adjusted.  The side
panels for multiple layers can now be shown at once, and individual
tabs of side panels can also be dragged to separate panels.

* feat: add layer list panel and support for "archived" layers

Archived layers are shown only in the new layer list panel; they are
not visible and are not shown in the layer bar.  They may be useful in
the case where there are a large number of layers that would otherwise
clutter up the layer bar.

This commit also improves the visual feedback during layer and drop
operations.

* feat(python): add ConfigState options for additional panel buttons

* fix(python): fix ManagedLayer.visible property

* fix(annotation): only bind properties actually referenced by the shader

Annotation rendering maps properties to shader vertex attributes, and
WebGL implementations limit the number of attributes to ~16.
Furthermore, previously all properties were bound even if not
referenced.  With this commit, only properties actually referenced by
the shader are bound.  This allows annotation layers to support data
sources with a large number of properties, provided that only a small
number are actually referenced by the shader at once.

* fix(annotation): fix rendering tab to better handle a large number of properties

* fix(annotation): prevent collapse of annotation property list

* fix(perspective_panel): eliminate separate transparentPickEnabled behavior

Segmentation layers have a "pick" boolean option that previously
controlled two things:

1. When off, double clicking does not select/deselect segments in the
layer.

2. When on, picking of transparent meshes behaves exactly the same as
if the mesh were opaque --- i.e. it is not possible to pick objects
behind/inside the mesh.  When off, picking of transparent meshes has
lower precendence than opaque objects behind.

In practice, the picking behavior for transparent objects with
"picking on", which made it impossible to pick opaque objects behind,
is not very useful.

With this change, the second behavior change no longer applies:
transparent objects now always have lower picking precendence than
opaque objects, making it possible to pick opaque objects behind
transparent objects.

* fix(ui/side_panel): only allow dragging from title bar

This allows slider controls in the body of side panels to work correctly.

Also disables auto-select on focus behavior for the name input field
of the layer side panels.  The previous auto-select on focus behavior
meant that attempting to start a drag from the text area (which takes
up most of the title bar and is the most natural place to drag from)
worked, but showed the text itself being dragged rather than the
entire panel.

Co-authored-by: Chris Jordan <jordanchriss@gmail.com>
Co-authored-by: Jeremy Maitin-Shepard <jbms@google.com>
Co-authored-by: Hannah Gooden <hannah.gooden@jhuapl.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants