Support for biomedical imaging formats (WSI, microscopy, multi‑resolution images, OCT etc)

[steffenvogler]

Biomedical imaging (whole‑slide images, fluorescence microscopy, radiology‑adjacent research imaging) are high‑value but structurally hard data types (from ML perspective). Unlike generic ML image datasets (usually PNG format), these formats involve:

• Multi‑resolution pyramids
• Tile‑based access
• Extremely large file sizes
• Acquisition parameters as part of the header information
• Potentially tight coupling to clinical or basic research context

This goes beyond what current Croissant image examples cover and is central to translational biomedical AI workflows.

[steffenvogler]

just tagging: @joshmoore @slobentanzer @stefanches7

Josh said he would be open to tag-team this feature enhancement with a few of us. Instant messaging during code development could be here: https://imagesc.zulipchat.com/#narrow/channel/500081-CroissantML

First question for @joshmoore: what's the best way to get the magic of bioformats into Croissant?

[jo-mueller]

On the metadata: is bioio able to load the metadata only - without transferring anything of the pixel data? Or does it load everything, but it it simply does not put the pixels into the python object? Obviously, loading the metadata to do exploratory data analysis first and only load the pixel data in a second step (with the correct filters in place) would be the absolute best way to do it.

Jep. In terms of API, bioio gives you a BioImage object, which hands you all the metadata in a standardized fashion, as far as bioio is able to retrieve it. The pixel data is only queried if you explicitly call for it.

[lubianat]

Some things on the OME-Zarr croissant writing side are (slowly) going on at MIT-LCP/croissant-baker#108

[steffenvogler]

@lubianat - thanks for the pointer. I will reach out to Rafi on this. If I understand things correctly, then we are currently building quite some redundancy in that Croissant Baker is implementing its own handler logic rather than using the ones from "vanilla" Croissant. But I will talk to Ravi, where and how to add features...

[lubianat]

@steffenvogler thank you for taking a look at it. I add the disclaimer that I am myself getting familiar with the croissant spec and the tooling around, so I don't know much about the redundancy/overlaps

[steffenvogler]

@slobentanzer - link to the Zulip thread on imagesc I mentioned 2 days ago Link