Well metadata

[will-moore]

See https://github.com/ome/omero-ms-zarr/issues/76#issuecomment-721068552

omero zarr export Plate:1 now exports 'well' metadata like

"well": {
    "images": [
        {"path": "0"},
        {"path": "1"}
    ]
}

and 'plate' metadata like:

"plate": {
        "columns": [
            {"name": "1"}, {"name": "2"}, {"name": "3"}
        ],
        "rows": [
            {"name": "A"}, {"name": "B"}
        ],
        "wells": [
            {"path": "0/A/3"},
            {"path": "0/B/2"},
            {"path": "0/A/1"},
            {"path": "0/B/3"},
            {"path": "0/A/2"},
            {"path": "0/B/1"}
        ],
        "field_count": 1,
        "name": "plate1_1_013",
        "acquisitions": [
            {"path": "0"}
        ],
    }

[will-moore]

Data exported with this PR can be found at https://minio-dev.openmicroscopy.org/idr/idr0033-rohban-pathways/41744_illum_corrected/pr_43/5966.zarr and viewed with

https://mystifying-lalande-e12142.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/idr0033-rohban-pathways/41744_illum_corrected/pr_43/5966.zarr

This allows individual Wells to be shown with the fields in a grid:

[will-moore]

To review plate .zattrs generated with this PR, see review https://minio-dev.openmicroscopy.org/idr/idr0094-ellinger-sarscov2/screenB/7825.zarr/.zattrs
and https://minio-dev.openmicroscopy.org/idr/idr0033-rohban-pathways/41744_illum_corrected/pr_43/5966.zarr/.zattrs
(exported before the last commit)

And Well .zattrs e.g. https://minio-dev.openmicroscopy.org/idr/idr0094-ellinger-sarscov2/screenB/7825.zarr/0/A/10/.zattrs

[manics]

Unrelated to this PR, but note that on https://minio-dev.openmicroscopy.org/idr/idr0094-ellinger-sarscov2/screenB/7825.zarr/.zattrs
acquisitions mentioned in this PR description is named plateAcquisitions instead:

        "field_count": 9,
        "name": "ESP0025722",
        "plateAcquisitions": [
            {
                "path": "0"
            }
        ],

which means we have a mix of snake_case and camelCase

[sbesson]

Thanks @manics, I vote for doing plateAcquisitions -> acquisitions as part of this PR

[will-moore]

Thanks @manics for catching that. Fix pushed.

[will-moore]

I think I'll try fixing that by manually editing the exported plate.zattrs above at the same time as updating vizarr to keep everything in sync.
I could try to update the column names to str and fix the version commit too, but those aren't breaking changes.

[will-moore]

Plate metadata exported with current state of this PR:
https://minio-dev.openmicroscopy.org/idr/idr0002-heriche-condensation/plate1_1_013/pr_43/422.zarr/.zattrs
With lists truncated:

{
    "_creator": {
        "name": "omero-zarr",
        "version": "0.1.dev149+g37c48d6"
    },
    "plate": {
        "acquisitions": [
            {
                "path": "Run 422"
            }
        ],
        "columns": [
            {
                "name": "1"
            },
            {
                "name": "2"
            },
        ],
        "field_count": 1,
        "name": "plate1_1_013",
        "rows": [
            {
                "name": "A"
            },
            {
                "name": "B"
            },
        ],
        "wells": [
            {
                "path": "Run 422/B/4/"
            },
            {
                "path": "Run 422/D/2/"
            },
        ]
    }
}

[will-moore]

Export failed for Plate:4287 after about 20 Wells. Don't know which Well/Image it failed on
(partial plate exported at
https://minio-dev.openmicroscopy.org/idr/idr0012-fuchs-cellmorph/HT01/pr_43/4287.zarr/)
with:

File "/tmp/venv/lib64/python3.6/site-packages/omero/gateway/__init__.py", line 4793, in __call__
    return self.f(*args, **kwargs)
  File "/tmp/venv/lib64/python3.6/site-packages/omero_api_RawPixelsStore_ice.py", line 1199, in getPlane
    return _M_omero.api.RawPixelsStore._op_getPlane.invoke(self, ((z, c, t), _ctx))
omero.ApiUsageException: exception ::omero::ApiUsageException
{   
    serverStackTrace = ome.conditions.ApiUsageException: Z '1' greater than sizeZ '1'.
        at ome.services.RawPixelsBean.handleException(RawPixelsBean.java:836)
        at ome.services.RawPixelsBean.getPlane(RawPixelsBean.java:399)

Loading in napari unexpected array size for one of these Wells? All C Wells have only 1 channel.

LOADING tile... 0/C/24/0/4
Failed to load 0/C/23/0/4
Failed to load 0/C/22/0/4
LOADING tile... 0/D/1/0/4
LOADING tile... 0/D/2/0/4
Failed to load 0/C/24/0/4
LOADING tile... 0/D/3/0/4
...data.shape (1, 1, 3, 64, 84)

Others are ...data.shape (1, 3, 1, 64, 84)

[will-moore]

Some URLs to view data from this PR (working with the current state of hms-dbmi/vizarr#43 but not guaranteed to work in future after any breaking changes)

• idr0033: https://mystifying-lalande-e12142.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/idr0033-rohban-pathways/41744_illum_corrected/pr_43/5966.zarr
• idr0002: https://mystifying-lalande-e12142.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/idr0002-heriche-condensation/plate1_1_013/pr_43/422.zarr
• idr0004: https://mystifying-lalande-e12142.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/idr0004-thorpe-rad52/P101/pr_43/1751.zarr
• idr0012 (incomplete - see above) https://mystifying-lalande-e12142.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/idr0012-fuchs-cellmorph/HT01/pr_43/4287.zarr/

These also work in napari with current state of ome/ome-zarr-py#57 using e.g.

$ napari https://minio-dev.openmicroscopy.org/idr/idr0033-rohban-pathways/41744_illum_corrected/pr_43/5966.zarr

[will-moore]

Current status demo: https://www.youtube.com/watch?v=i27bF2j03IU

[joshmoore]

Code looks fine. If there are no objections, I'd vote for getting this in with the other linked PRs. Any changes to the metadata can kick off a round of N other PRs.

[sbesson]

As far as I am concerned this could be tagged immediately to capture the well metadata layout that was extensively tested over the last week but we can also get #44 in the tag assuming it's soon ready.

[sbesson]

No objection raised during today's NGFF call. Merging to make progress on the two next concepts:

• plate + labels
• plate + acquisitions