RO-Crate profile validation with LinkML — anyone got a full stack working?

[matentzn]

A question came up in the LinkML Slack that probably belongs in a wider venue, so I'm reposting it here for discussion. Cross-posting from a thread started by @cmungall (on slack).

Has anyone (outside the B2AI team) been using RO-Crate with LinkML (or SHACL) to define profiles, with a full validation stack actually up and running?

By "full validation stack" I mean something that covers both:

• the RDF graph constraints (i.e. SHACL), and
• the JSON-LD shape constraints that RO-Crate profiles also impose (cardinality on the flattened JSON, required @type patterns, file/directory descriptors, etc.).

A few partial answers came up on slack:

• François: has used SHACL, but it only covers the graph side. Profiles and the base RO-Crate spec also make restrictions on the JSON itself, which SHACL can't capture. LinkML didn't seem to close the gap either.
• Tiago: pointed at crs4/rocrate-validator, which uses SHACL to validate RO-Crates — e.g. 1_file-descriptor_metadata.ttl. Useful, but again SHACL-only.
• @matentzn (me): the same gap came up recently in the ESIP workspace — different community, same unsolved problem.

Chris also flagged two older threads that are still relevant:

• ResearchObject/ro-crate#264 — "Use LinkML to define schemas" — opened May 2023 by someone working on microscopy metadata profiles over OMERO, who wanted LinkML tooling for the spec + auto-export.
• forum.image.sc — RO-Crate and OMERO — related discussion from the imaging community.

Both are from a couple of years ago and don't seem to have led to a concrete LinkML-based validation stack landing anywhere public. In case you have anything more to add, please do so in this thread!

[dalito]

@elichad from ro-crate helped here with a SHACL issue (#2440). Could be another contact.

[lubianat]

I'm generally interested in the RO-Crate and LinkML interface, as are others at GerBI (cc @joshmoore @gouttegd )

[elichad]

This question comes back to haunt me every couple of months... thanks @matentzn for opening the discussion here.

A couple of years ago a few of us from the RO-Crate community tried to implement LinkML schemas for RO-Crate profiles. We came from two different angles but ultimately both groups ran into similar barriers and came to the conclusion that there are some incompatibilities between LinkML and RO-Crate that are quite challenging to resolve.

Our work is written up in an (as yet unpublished) blog post: https://hackmd.io/@elichad/rJQpnZEwyg . I have been meaning to get somebody from the LinkML community to look at it - I'd be interested to know if any of these issues have been addressed on the LinkML side in recent releases (we did flag them all at the time, either in Slack or in GitHub issues). If it's not too contentious I would like to post it on the RO-Crate blog so we can point to it whenever this question comes up.

We have some upcoming developments in the RO-Crate community that might also improve the compatibility from our side - in particular RO-Crate Machine Actionable Schemes and Profiles (under development) focuses on the kinds of validation that can be defined in a schema and so might be more easy to convert to/from LinkML.

If anyone does want to push forward with trying to develop an integration again, I'd be happy to lend my RO-Crate expertise, though I can't commit any dev time at the moment. I do think it would be a really beneficial thing to have - I get questions about LinkML compatibility pretty regularly when I'm doing RO-Crate outreach.

[sherwoodf]

By "full validation stack" I mean something that covers both:
the RDF graph constraints (i.e. SHACL), and
the JSON-LD shape constraints that RO-Crate profiles also impose (cardinality on the flattened JSON, required @type patterns, file/directory descriptors, etc.).

I would love to have something like this. Perhaps greedily, I would be keen to go even further and say that it would be amazing if linkML could be a generic 'JSON-LD' validator beyond RO-Crate. However, https://linkml.io/linkml/howtos/using-jsonld.html makes it seem like LinkML has some strong opinions on what json-ld documents should look like that make me less hopeful that this will be the case.

My attempts to use linkML for this correspond with the findings in Eli's blog post. More generally, it did not seem possible to use a linkMl generator to generate 2 complementary validation systems that worked together for JSON-LD. It seemed like it might be possible if a very specific context & json-ld 'form' were used - but those definitely didn't correspond to RO-Crate's expectations. It also seemed like it isn't always possible to generate 'open' schemas - while that's not json-ld specific, it added to the reasons I didn't use link-ml in our my particular case.

I keep wondering whether there is a different approach to validating JSON-LD that might be easier to get working by making use of its processing algorithms. For a given json-ld document (with a given context) there's only 1 rdf graph and there's only 1 expanded form. Those feel easier to validate, but since you can't necessarily just reapply the original context and get back to the original json-ld document, it always seems like the original json would still need some validation.

RO-Crate is perhaps in a better position by having a root entity (the ro-crate-metadata.json object) - perhaps using framing to produce a nested json starting at that object, it might be possible to validate the json contents of the document + presence of required term definitions. Though you might still need strict rules about the context (e.g. it explicitly sets container expectations, and doesn't have 2 different terms point to the same IRI).