support multi-pos datasets that are not complete multi-well plate acquisitions

[mattersoflight]

There is an apparent gap in data formats specified by OME-NGFF and supported by napari-ome-zarr plugin: they specify/support single FOV ND arrays or multi-FOV ND arrays acquired in a multi-well plate, but not simple multi-pos acquisitions.

We can represent this data as as a multi-well dataset to leverage the napari-ome-zarr plugin for visualization.

This topic requires a discussion among @ziw-liu , @Soorya19Pradeep , @jrbyrum13 , @Christianfoley , @ieivanov, @talonchandler who either acquire or analyze multi-pos datasets.

I see two options:

1. Have a single column, with row names = position names in micro-manager acquisition .
2. Map the data to a square grid to make it easy to visualize with napari plugin, e.g., if we have 100 FOVs, sort them into 10 rows and 10 columns.

Votes & Ideas?

[jrbyrum13]

On note 2: would this require that the data is acquired in multiples of 10? Will it support incomplete square grids?

[ziw-liu]

would this require that the data is acquired in multiples of 10? Will it support incomplete square grids?

Not necessarily. Absent nodes on the grid will be rendered as blank (zero-filled). The ome-zarr-napri plugin will print some annoying warnings in the terminal but it 'works'.

[ieivanov]

Note that even well plate acquisitions may not be perfectly square - it's not uncommon that the sample in one well will die or that the experiment will, for example, have more samples in one row than others. The reader, writer, and viewer should not be picky about that.

Conceptually, all acquisitions start off with a list of positions. These can then be arranged in a grid manually by the user, or automatically based on the position name (e.g. A2 or 000-001, giving the row and column position). The Fiji Image Stitcher plugin has good logic on how the list of positions may be ordered: https://imagej.net/plugins/image-stitching#gridcollection-stitching. We should allow for both of these options - manual sorting or automatic grid sorting based on the file name. If these fail, we can just square off the data - for example, a 7 position acquisition will be arranged in ceil(sqrt(7)) = 3x3 grid with datasets in alphabetical order, i.e. FOV0 goes to (0,0), FOV1 to (0,1), etc.

[ziw-liu]

Note that even well plate acquisitions may not be perfectly square - it's not uncommon that the sample in one well will die or that the experiment will, for example, have more samples in one row than others. The reader, writer, and viewer should not be picky about that.

See #22 (comment)

We should allow for both of these options - manual sorting or automatic grid sorting based on the file name.

Here we are constrained by other community projects (ome/ome-zarr-py#200). To get the nice stitching behavior for visualization, some options are (ascending by amount of effort):

1. Wait for ome-zarr-py and napari-ome-zarr to implement these features or try to contribute.
2. Fork the viewer, modify its logic, and use the fork until it is merged to the main package.
3. Make our own napari reader plugin.

However I would argue that none of the above is within the current scope of iohub. If we decide to invest in the last option, it might better fit in a separate package (e.g. iohub-napari).

[ziw-liu]

Can #31 as of 92fa0db close this?

[ziw-liu]

I think the current API is expressive enough to store arbitrary HCS-conformant layouts.

If there's a specific problem/question about usage, please open a separate issue.