BioFormats 5.2.4 CZIReader is broken for Tile Images

[sebi06]

Hi,

I just tested BioFormats 5.2.4 with a 20x20 Tile Image I just acquired today. In former versions, such an image was recognized by BioFormats as 400 Image Series, no matter if the image in ZEN was with or without a pyramid.

Now BioFormats offers me to open 5 image series, which seem to correspond to the different pyramid levels. But all images except for the last image contain corrupted image information (see screenshot).

When I repeat the image acquisition without generating a pyramid, BF 5.2.4 does not recognize this image as a tile image any more (even it is exactly the same image). It now detects 400 image series again, which is the old (not nice) way to do it. But at least the image data are ok.

This is really nasty for the users, since the default way to acquire images in ZEN is of course with pyramid. Turning this off is a special option only.

In fact with 5.2.4 most of the CZI images cannot be read an more! Any help is therefore really appreciated.

Cheers, Sebi

Testdata: https://dl.dropboxusercontent.com/u/623476/CZI_Test_20x20Tile.czi

[dgault]

Hi Sebi,

Thank you for bringing this to our attention and providing test data.
I have been able to reproduce the issue as described and isolated the PR in which this functionality changed.

I have added a card to our Trello inbox for managing this (https://trello.com/c/AldQjHV2/32-czireader-tiled-images-bug) and shall continue to debug to root cause the problem and work on a solution.

With Thanks,
David Gault

[sebi06]

Hi David,

please let me know when you need more test CZI images from us. I have the feeling that the ongoing JPG-XR integration for reading CZI images seem to break some existing workflows for most of our users.

To be clear: Only a small percentage of our users actually use JPG-XR to compress their CZI images. Most of them use the normal uncompressed way. And 5.2.4 breaks some of their workflows that did run sooth so far.
This feature to read the pyramid levels is cool, but we have to be careful here not to introduce new issues in order to make the CZI JPG-XR work. People using JPG-XR are usuallay acquiring large tiles region (slide scanner), but the vast majority of CZIs are different and must also work with BioFormats / CZIReader.

[melissalinkert]

This should be resolved with the current state of #2648. In ImageJ I now see:

which seems correct based upon your above description and the result of opening the file in Zen.

[sebi06]

Hi,

Is it really correct that all thumbnails look the same even if the
represent different pyramid levels?

Sebi

Am 03.11.2016 12:40 nachm. schrieb "Melissa Linkert" <
notifications@github.com>:

This should be resolved with the current state of #2648
#2648. In ImageJ I now
see:

[image: image]
https://cloud.githubusercontent.com/assets/109998/19975374/ae5acb4c-a1b9-11e6-92e4-142c0ea2fef6.png

which seems correct based upon your above description and the result of
opening the file in Zen.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#2634 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADp9obE7HMosBdXRDmF6OXCl5IdkuxaBks5q6g6XgaJpZM4Kf-Pn
.

[melissalinkert]

Yes, that is expected, as each pyramid level is just a smaller version of the full resolution image.

[sebi06]

Hi Melissa,

i uploaded two more "tricky" CZIs. These should be tested with 5.3.0 to see at least what we can expect here. I am especially concerned that the "new" way BF handles so Tile Images including the pyramids will break a lot of existing workflows for people using BF wrappers for Python (python-bioformats), MATLAB or even in Fiji or Knime.

CZI_Test_2Pos_3x3_20x5_CH=1_A1_A2.czi --> 2 TileRegions in different wells with different size

CZI_Test_2Pos_10x10_10x10_CH=1_A1_A2.czi -->2 TileRegions in different wells with the same size

Links:

https://dl.dropboxusercontent.com/u/623476/CZI_Test_2Pos_10x10_10x10_CH%3D1_A1_A2.czi
https://dl.dropboxusercontent.com/u/623476/CZI_Test_2Pos_3x3_20x5_CH%3D1_A1_A2.czi

[sebi06]

Hi Melissa,

where can I get a testversion of the version, where this bug is fixed. I really wnat to test this because this is very crucial for a lot of our users.

[sbesson]

Hi @sebi06,

Following your recent set of issues, we feel the past and ongoing work on the Zeiss CZI reader requires clarification. We will be using this GitHub issue to discuss our perspective on the problems you raised and propose solutions to address them.

As was announced a few months ago, Carl Zeiss Microscopy GmbH and Glencoe Software Inc. have partnered to deliver JPEG-XR support to the CZI reader in Bio-Formats. The main target of this work are histopathology images acquired by the Axio Scan.Z1 instrument. As part of this work it was essential that we improve the support the CZI Bio-Formats reader provides for some of the imaging modalities supported by the CZI format, namely pyramidal subresolution images. Prior to Bio-Formats 5.2.0, multi-resolution, tiled CZI files were opened as a linear array of individual series with no notion of the subresolution representations or stitching.

As an image collected on an Axio Scan.Z1 instrument can easily have 10 or more subresolutions and contain many 1000s of tiles, the pre-5.2.0 reader was simply unusable for many concrete workflows using Axio Scan.Z1 files, notably importing and annotating histopathology images into an OMERO server or visualizing subresolutions of lower detail in Fiji. These limitations forced us to review the set of requirements to support JPEG-XR compressed CZI images. We concluded that in addition to the compression/decompression support, fixing the data layout presented by the CZI reader to handle subresolutions was a necessity. The first step of this subresolution support has been introduced in Bio-Formats 5.2.0. The work should be completed in Bio-Formats 5.3.0, but we have released it in its current state in 5.2.0, in order to deliver some of this functionality to our community.

The discussions initiated in the recent issues (#2632, #2634, #2558) highlight a series of challenges associated with this work:

• Some analytical workflows have been developed in the community against these types of datasets that rely on the pre-5.2.0 behavior of the CZI reader where tiles in the full resolution image were read as a linear series
• While the changes introduced in 5.2.0 enabled some subresolution functionality, they missed proper documentation explaining their implications and a way for API consumers to upgrade with confidence
• The changes introduced in 5.2.0 had some bugs in how pyramids were detected and how tiles within each pyramid resolution were stitched together. This was especially problematic for datasets that also contained multiple scenes.

We think the best strategy to meet everyone’s needs is to make sure there is some mechanism allowing backwards compatibility in the CZI reader, while also delivering new features. For some generic format concepts, some API features already exist that can aid backwards compatibility. For example, it is possible to hide pyramidal subresolutions in ImageJ using FormatReader.setFlattenedResolutions(false), so that they do not appear as regular series. In other cases, we need options which may be reader-specific, allowing the user to select their desired behavior. The latter case is one of the major objectives of the Options API we will be introducing for Bio-Formats 5.3.0.

However, we have more decisions to make regarding what should be prioritized for Bio-Formats 5.3.0 and how to engage with the community to make these decisions. A couple of specific questions:

• Does the setFlattenedResolutions() API mentioned above resolve some of the backwards-compatibility issues?
• Would adding an option to disable all automatic tile stitching in the CZI reader (so that tiles are returned as separate series) be sufficient?
• Are there other behaviors you are aware of with respect to the CZI reader that should be selectively enabled/disabled via an options mechanism?

Finally, we do not have the resources or even the knowledge to review and test all existing workflows that are important for the community. How, in addition to our contributing documentation which documents the publicly accessible continuous integration builds we use for testing, can we help people who want to test?

[imagejan]

• Would adding an option to disable all automatic tile stitching in the CZI reader (so that tiles are returned as separate series) be sufficient?

As far as the Grid/Collection Stitching plugin is concerned, it relies on tiles being read as separate series, so it would be sufficient if reading resolution levels can be disabled and reading tiles as separate series can be enabled.

The relevant code that reads the files is here:
https://github.com/fiji/Stitching/blob/e3d49c789f54fae4f24d18755e3074043529e2b0/src/main/java/plugin/Stitching_Grid.java#L799-L827

[sebi06]

Hi guys,

We thought about all those topics and I will try to summarize the most important facts, where I think some clarification is needed. Most of the things have nothing to do with the actually JPG-XR compression, but rather are workflow and compatibility questions.

• One main issue in the past was the missing possibility to read CZI images using the JPG-XR compression. It was just not possible to open those images using BioFormats regardless the image pyramid.

Our users and customers always complained about this issue and therefore the decision was made, that Carl Zeiss Microscopy GmbH and Glencoe Software Inc. team up to implement this option as part of the CZIReader within BioFormats.

• JPG-XR has technically nothing to do with the Axio Scan.Z1. It is a general feature of our ZEN Blue image acquisition software and can be used by everybody running ZEB Blue and all of the supported microscopes stands.

• It is correct that mostly Axio Scan.Z1 users so far make use of JPG-XR because their images are typically really large (up-to hundreds of GB). Therefore those people have the most urgent need for a compression.

• There is only one CZI data format for all of our systems, with the exception of some systems that still run on ZEN Black. In the future even those will be migrated one after the other to ZEN Blue. In this respect the AxioScan.Z1 is no special imaging modality since it is using the standard CZI data format from ZEN Blue. What is special are the super-resolution systems Elyra, the LightSheet and the LSM880 systems, when it comes to some aspects of the CZI data format.

• If one looks at all the people that use BioFormats to open their CZI data the number of people using CZI with JPG-XR is really small (yet). Therefore it is absolutely crucial that any new features of BioFormats 5.3.0 are backwards compatible when it comes to those cool new features, like reading pyramid level images and the tiling.

From our point of view the most typical workflows involving BioFormats and CZI are:

• Reading CZI images with Fiji/Image
• ReadingCZI images in Python using python-bioformats
• Reading CZI images with CellProfiler which is also using python-bioformats
• Reading CZI in KNIME and Icy
• Reading CZI images in MATLAB

There are probably more but at least for those workflows we need compatibility features discussed without any doubt. Therefore the idea to use the setFlattenedResolutions() API sounds like what we need. Additionally an option to switch off the tiling could be beneficial, depending on how for instance the Grid/Collection Stitching plugin will handle those tiles.
To clarify the following workflows let’s assume the following general and still quite simple image acquisition using ZEN Blue, that has nothing to do with Axio Scan.Z1 in particular, since it is a general feature of all of our systems.

10x10 tile image with CH=2, Z=5 and T=3 (so nothing fancy, we assume this image already has 3 pyramid levels)

• BioFormats opens 10*10=100 image series with the same pixel dimension in XY containing the TZC dimensions.
• Every image series has the correct stage coordinates etc., so that for instance a plugin like Stitching knows what to do…
• A lot of 3rd party software from above has adapted to this format.

With the new features the situation will be different.

• BioFormats will open 4 image series, where the images are already tiled and where the 4 image series will all have different pixel dimensions in XY due to the pyramid levels.
• This will be unexpected for all those other software packages, which now have to adapt to the fact, that images series no longer represent tiles or positions, but image pyramid levels.
• As an example, will there be still the correct XYZ-values for every tile and can Stitching deal with those?

Let’s assume a slightly different, but also really general example:

5 independent positions with CH=2, Z=5 and T=3 (no pyramid levels here)

• Currently this would open a 5 images series, where all the series will have the same size in XY.

In this case those image series do not represent pyramid levels, therefore those must be marked a “normal” image series. Even trickier will be the following example:

20x15 tile and a 10x10 tile image and with CH=2, Z=5 and T=3 (more fancy, where every scene might have a different number of pyramid levels).

Here I have no clear idea on how this would open with new BioFormats. But from all those things mentioned above I think we really need the following things:

1. setFlattenedResolutions() = False results in images series representing pyramid levels where the images are always already tiled.
2. setFlattenedResolutions() = True and setTiling() = False results in images series without any pyramid levels, where every series is a single tile.
3. setFlattenedResolutions() = True and setTiling() = True results in only one image in full resolution, which is already tiled.

Can you think of more workflows we have to consider?

It is still not really clear to me how the mentioned plugins, like Stitching or the other mentioned software packages (python-bioformats) will be able to deal with those issues.
What has to be adapted inside the python-bioformats library, since those new methods will have probably mapped from the java API to the python library. I know this is not directly your problem, but from a workflow and customer point of view this is exactly the critical point when it comes to use different open source packages in real live.

Cheers,

Sebi

[sbesson]

Hi @sebi06,

Thanks for the detailed answer and the precise user workflows. We just released Bio-Formats 5.3.0 yesterday. In addition to the JPEG-XR support, this release allows to pass key/value options in the Zeiss CZI reader and disable attachments and/or auto stitching when reading tiled CZI images.

We have exposed these options into the ImageJ plugin via the Zeiss CZI configuration interface (see below). They can also be manipulated directly using the new DynamicMetadataOptions class introduced in 5.3.0 and the IFormatReder.setMetadataOptions API.

We expect these additions to solve most of the issues you reported in #2628, #2632 and #2634. Therefore we will be closing these issues and we are looking forward to your testing and your feedback on these improvements. We will also start working on documenting the usage of these options in the context of various CZI modalities inspired by your concrete examples.

Best,
Sebastien