Schemas v0.1 v0.2

[will-moore]

This adds schemas, tests and samples for JSON schema support of v0.1 and v0.2.

Updates all schemas to NOT require version, name etc, since these are only SHOULD in the spec (noticed this when validating some existing v0.1 files).

Also adds plate samples and plate.schema for the HCS spec (tests for 0.1 only, since this hasn't changed since).

This also adds a way of specifying the SHOULD rules within the schema, using a second schema with allOf to add restrictions to the image.schema (see below - thanks @sbesson).
This more strict schema can be used to validate without failing on the first Error, allowing it to be used for reporting multiple suggestions for rules that SHOULD be obeyed, but aren't actually invalid.

Added tests that check that all valid files (except image_complete.json) give at least one SHOULD warning.

{
  "allOf": [
    {
      "$ref": "http://localhost:8000/image.schema"
    },
    {
      "properties": {
        "multiscales": {
          "items": {
            "required": [
              "version", "metadata", "type"
            ]
          }
        }
      }
    }
  ]
}

[will-moore]

That last commit avoids the need to run httpserver in the test, which will be nice for ome-zarr-py validate use-case where we want to ship the schemas with the library and not load them from a URL each time (or have to serve them from httpserver).
However, it does require that the code has some knowledge of the schema to update appropriate $ref:

schema['allOf'][0]['$ref'] = IMAGE_SCHEMA_KEY

[will-moore]

The last commit uses a better approach of subclassing the RefResolver to always load local schemas instead of remote.

[joshmoore]

In general, love all the examples and the fact that this covers all current versions. One general naming consideration about the URLs since those are something we want to last. I haven't (yet) gone through each example and schema, but I don't think I'll get to that before Thursday. If it helps to get this in here so that ome-zarr-py etc. can build on top of it, no objections. We should just review before we make the URLs live. (I assume at this point they don't actually get built into the spec page)

[joshmoore]

This was dropped from v0.3?

[will-moore]

Fixed to "0.1"

[joshmoore]

The "schema(s)" in this URL are a bit much....

[will-moore]

That is just the path we currently have in this repo. Since we're only supporting json_schema now, we could drop that dir. But the paths are even more painful at ome/ome-zarr-py#142 where I wanted to maintain the same directory structure as here, simply replacing https://ngff.openmicroscopy.org/ with a single path in that repo (expecting that we'll package the schemas as part of the release). So the path in that repo then becomes schemas/0.3/schemas/json_schema/image.schema!
The problem with is that in the NGFF repo, the version is at the top level, so we have a schemas directory under each version, but it's the other way around on ome-zarr-py where I wanted all the schemas to be under a top-level schemas/.
Need to unify the approach between the repos. Do we create top-level version dirs on ome-zarr-py?

[will-moore]

Once we've "published" a schema to a public URL that people can use for validation, can we make any future changes to that schema, and how do we version it?
E.g. if we make a change/fix and then an instance file becomes valid/invalid as a result??

[will-moore]

From the discussion this morning a plan of action:

• Move schemas to e.g. /0.3/schemas/image.schema (remove extra json_schema dir)
• Make ngff pip installable
• Include get_schema(version, strict=False) from ome-zarr-py validation PR
• Update bikeshed job to also publish schemas - see https://github.com/joshmoore/spec-prod

I think only the first point should be done in this PR?

[joshmoore]

Merging as discussed this morning. As mentioned, we can re-review the URLs before going public with them.