Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reader for Ventana .bif files #3336

Open
wants to merge 27 commits into
base: develop
from

Conversation

Projects
None yet
3 participants
@melissalinkert
Copy link
Member

commented Mar 29, 2019

Backported from private PRs. See https://trac.openmicroscopy.org/ome/ticket/12899 and https://trello.com/c/oCH7PbA5/10-ventana-bif.

Files have been uploaded to inbox/ventana/ for copying to curated - there are 5 files total. With this PR, showinf -noflat on each should detect a pyramid with 9 or 10 resolutions plus two label/overview series.

Ventana .bif files are TIFF files where the first two IFDs correspond to label/overview/macro images and the remaining IFDs form a pyramid. Each IFD in the pyramid stores the image as tiles that are typically at least 1k x 1k.

Not all stored tiles are used; the largest IFD in the pyramid contains XML that defines one or more AOIs (areas of interest) that have been scanned. Any tiles outside scanned AOIs are discarded. Within each AOI, the file defines a list of X and Y overlaps for some pairs of tiles within the AOI (TileJointInfo). These are approximate values, and do not represent all of the overlaps that need to be removed. The TileJointInfo values are filtered by the Confidence value and averaged to produce 3 numbers: X overlap, Y overlap, and Y shift. X and Y overlap is applied to all tiles in the AOI; Y shift moves every other column of tiles in the AOI by the calculated number of pixels. These 3 numbers are used to calculate absolute positions for each tile.

In the largest pyramid resolution, each TIFF tile corresponds to one acquired tile. In smaller resolutions, each TIFF tile is n x n acquired tiles with overlaps embedded. To read smaller resolutions, each TIFF tile must be split apart and reassembled using the absolute positions that were calculated for the largest resolution.

Ventana .tif files are similar to .bif files, but have been processed to remove all overlaps and all scanned area metadata. The IFD ordering and magnification metadata is the same as in .bif, but the tiles can be treated as normal TIFF tiles.

To my knowledge, there is no freely available software that can be used to get a ground truth, so this is a little difficult to thoroughly validate. Comparing the pyramid images to the label/thumbnail image is probably the most reliable check. The pyramid images can also be checked for any really obvious seams between tiles that indicate incorrect overlap calculation.

There is also a ventana.split_tiles option (turned off by default) which, when enabled, will return each tile in the largest resolution as a separate series. This is a fall-back option in case the reader's stitching isn't good enough for a particular file.

This will require a minor release but isn't critical for 6.1.0, so feel free to exclude in favor of other readers.

melissalinkert added some commits Sep 26, 2018

Progress on tile stitching for the full resolution image
Not done yet, as the overlaps are still not quite right.  This is closer
than the previous version with no overlap calculation though.
Average a subset of recorded overlaps instead of using overlap list
A lot of trial and error indicates that the recorded overlap list for
each AOI does not indicate the total number of overlaps to be removed.
Individual overlap values are also sometimes incorrect by a significant
amount.

This takes the approach of averaging high-confidence overlaps to produce
a single value for each of X and Y, which are then applied to all tiles
in the AOI.
Add "ventana.split_tiles" boolean option
The option is false by default; when set to true, each tile in the full
resolution image is returned as a separate series.  Lower resolutions
are then ignored.  This should make it easier to stitch tiles externally
if needed.
Ventana: adjust XY size to match AOI bounds
Conflicts:
	components/formats-gpl/src/loci/formats/in/VentanaReader.java
Adjust XY coordinate calculation for low resolution tiles
Changing the rounding strategy results in smoother stitching,
particularly for resolution levels >= 5.
Ventana: treat as normal TIFF if no scanned areas defined
.bif files may be post-processed to remove overlaps and overlap
metadata.  These files will have the same IFD order and magnification
metadata, but do not need any of the logic to recalculate image sizes
and tile positions.

Conflicts:
	components/formats-gpl/src/loci/formats/in/VentanaReader.java
@sbesson

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

See https://web-proxy.openmicroscopy.org/east-ci/job/BIOFORMATS-test-folder/61459/console, the reader seems to misdetect TIFF files that should be detected as Metamorph.

Check for "iScan" in the XML tag during isThisType
This should prevent incorrect detection of other TIFF files that have tag 700.
@melissalinkert

This comment has been minimized.

Copy link
Member Author

commented Apr 1, 2019

As noted on the config PR, files have now been moved from inbox to curated/ventana/.

@dgault dgault added the reader:new label Apr 11, 2019

melissalinkert added a commit to melissalinkert/ome-codecs that referenced this pull request Apr 17, 2019

@melissalinkert melissalinkert removed the exclude label Apr 26, 2019

@melissalinkert

This comment has been minimized.

Copy link
Member Author

commented Apr 26, 2019

Removing exclude label as discussed with @sbesson. I will check the build tomorrow and fix or re-exclude as needed.

@sbesson sbesson added this to the 6.2.0 milestone May 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.