OME-Zarr support

[normanrz]

Hi!

I am looking for ways to make the interaction of NRStitcher easier with Webknossos because converting the input and output data has become quite a time sink. I was wondering if you were planning to add support for OME-Zarr. It is becoming a community standard for large images [1] and Webknossos already has good support for it.

I see that you have NN5 support. Is that compatible with this N5? If yes, the Zarr implementation should be straightforward as it is a similar chunked file format.

There is a C++ implementation for Zarr as well: https://google.github.io/tensorstore/

Happy to contribute a PR, if you're interested.

[1] https://link.springer.com/article/10.1007/s00418-023-02209-1

[arttumiettinen]

Hi,

Sure this is interesting, we should make file format conversions unnecessary as much as possible.

The NN5 format (Not N5) is a bit similar to N5, but not compatible. It is designed to support reading and writing of compressed data from multiple compute nodes in parallel, and removes some restrictions like 2 GB limit on chunk size, introduces differences in chunk file format etc. It should work well in a cluster environment, but it is still kind of work in progress and not well documented. Performance is not guaranteed.

I'm open to adding support for new file formats, but I must admit that such large changes might take significant time if I'm doing them myself. I would be happy to accept contributions, though!

Best,
Arttu

[normanrz]

That sounds great. We are currently looking for funding that we could use to develop Zarr support in NRStitcher. I'll keep you updated.