V05 dev2

[will-moore]

This updates the validator to support v0.5-dev2 data, as specified at ome/ngff#242.

Validating against image schema at https://raw.githubusercontent.com/normanrz/ngff/spec-rfc2/latest/schemas/image.schema and also loading version schema from https://raw.githubusercontent.com/normanrz/ngff/spec-rfc2/latest/schemas/version.schema

Preview build viewing sample data:
https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/idr0044/4007801.zarr

cc @normanrz

TODO:

• Handle Well - e.g. https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD885/0046b0d0-f20b-4482-84b1-4b2b154865fd/0046b0d0-f20b-4482-84b1-4b2b154865fd.zarr/A/1/
• Handle bioformats2raw for 0.5 data, e.g. https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/idr0054/Tonsil%203.zarr
• All Wells: https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https%3A%2F%2Fuk1s3.embassy.ebi.ac.uk%2Fidr%2Fshare%2Fome2024-ngff-challenge%2Fidr0090%2F190621.zarr&well=all

[netlify]

✅ Deploy Preview for ome-ngff-validator ready!

Name	Link
🔨 Latest commit	1d55e98
🔍 Latest deploy log	https://app.netlify.com/sites/ome-ngff-validator/deploys/66f2ccb0ad494700080ee7ac
😎 Deploy Preview	https://deploy-preview-36--ome-ngff-validator.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[joshmoore]

@will-moore @manzt: I've also hard-coded the URL for hms-dbmi/vizarr#172 here as well 👍

[joshmoore]

When using ome_zarr view idr0001-2551.zarr

When using .../idr0001-2551.zarr $ http-server --cors

[joshmoore]

👍 from my side on the additional column:

[will-moore]

@joshmoore Not exactly sure what you're doing there....

$ ls plates/
2783928.zarr	4403.zarr	51.zarr
$ http-server --cors

https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=http://127.0.0.1:8080/plates/4403.zarr
Looks good.

Also e.g.

$ $ http-server plates --cors

Works at https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=http://127.0.0.1:8080/4403.zarr
etc.

[joshmoore]

Sorry, I just wanted to get a screenshot in place. No complaints 😄

[joshmoore]

Initial RO-Crate validation! 🎉

cc: @sherwoodf

[will-moore]

@sherwoodf - I noticed you had key licence at https://github.com/ome/ome2024-ngff-challenge/blob/53cdf2b73dec2a8341998b7a114a16dea3e5424f/dev3/zarr-crate/example_usage/example_ro_crate/ro-crate-metadata.json#L26C7-L26C65 and Josh had the same in the sample above. But we think this should be license as at https://schema.org/license. So the validator is checking for license.

NB: for name, description and license I'm just looking for the first "@graph" item that has those attributes, which is very simplistic. Any suggestions on improvements to this validation logic would be appreciated!

[sherwoodf]

@will-moore On license vs licence - you're definitely correct it should be licen_s_e

I think looking for objects with those fields works for the basic ro-crates that we're expecting to get.

If you wanted to invest in something longer term right now (which might be over the top), I would recommend using some sort of ro-crate/linked data tooling instead because the flattened data structure isn't wonderful for parsing, and we can probably update rules we go.

I've only used the python one, but there is a javascript package that might work with svelte: https://www.npmjs.com/package/ro-crate - it mentions an 'in development' validation promises.

We could write shacl shapes and run on the rdf graph that gets produced (https://github.com/zazuko/rdf-validate-shacl) - again, probably over engineering a solution for what we need right now. I could write us some shapes if we wanted to go down that route (if you're not familiar with them they're essentially schema descriptions for validating rdf graphs)

[sherwoodf]

@will-moore ignore the bit about shacl - i've just remembered that we don't have terms for the fbbi_id yet so it gets dropped from the RDF graph.

[joshmoore]

See https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://data-humerus.webknossos.org/data/zarr3_experimental/scalable_minds/l4_sample/color for a webknossos dataset that is currently erroring because sharding isn't used.

[will-moore]

@joshmoore @normanrz That non-sharded image is fixed now: https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://data-humerus.webknossos.org/data/zarr3_experimental/scalable_minds/l4_sample/color
But I don't see any chunks being loaded. I'm getting 404s when I try to load any chunk, and I'm not seeing anything in vizarr either. It is mostly a sparse image or is something broken?

[normanrz]

The dataset is sparse indeed. Data starts at 3072,3072,512. Unfortunately, OME-Zarr or Zarr don't have metadata for storing data offsets like this.

https://webknossos.org/links/EIyfC3zTaCNnB8Df

[will-moore]

That last commit attempts to look-up Organism and Imaging method. E.g.
https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/0.0.5/idr0010/76-45.zarr

[joshmoore]

Only quick thought is if we can have two more levels of "not there":

• ✅ there
• ❌ missing (license)
• ✖️ missing (organism / modality)
• [ ] or just omit (name / description)

[joshmoore]

Suggested change

	<h1>Ro-crate metadata</h1>
	<h1>RO-Crate metadata</h1>

could potentially have a info (I) with a link to what RO-Crate is.