Add support for Shapes Constraint Language (SHACL)

[abrokenjester]

SHACL is a W3C working draft. It is a language for validating RDF graphs against a set of conditions. These conditions are provided as shapes and other constructs expressed in the form of an RDF graph.

A SHACL implementation, available as a SAIL component, would enable automated validation of business constrains on RDF databases in RDF4J, which until now was only possible using ad-hoc approaches.

Possible approaches for implementation are either by implementing directly from scratch, or by extending/reusing the SPARQL engine. The former has a possible performance benefit while the latter is possibly easier (and would benefit from database query optimizations).

[hmottestad]

I would recommend limiting support to:

• sh:targetClass
• sh:property
• Restrictions
  • sh:path
  • sh:minCount
  • sh:maxCount
  • sh:class
  • sh:datatype

This lets you model simple restrictions such as "Person must have exactly 1 age which is an integer".

[hmottestad]

Probably simplest to implement by overriding the commit() method and implementing the SailConnectionListener to listen on changes.

Getting as close to a stream based approach where all the rules can be checked in one pass would be the most efficient. For insertions, one pass should be sufficient to determine sh:class, sh:datatype and sh:minCount. sh:maxCount is harder, but can at least be optimised to only run on resources where the restricted property has been added in the current commit.

[barthanssens]

So would it make sense to add an abstract ShaclSail listing the various constraints, and leave the actual implementation to e.g. ShaclSparqlSail and/or ShaclRawSail ?

Additional use case: the European Commission (or at least the DCAT-AP group) is "establishing a task group to develop an OWL expression of DCAT-AP, as well as a SHACL file that could be used for validation of DCAT-AP implementations" (quote from the mailing list, not sure if the archives are public, although the work itself of course is...)

[hmottestad]

I actually wrote a SHACL file for the Norwegian profile (DCAT-AP-NO) :)

As for the SHACL Sail. @heshanjse is writing a proposal for doing a Google Summer of Code project developing the SHACL Sail. Hopefully that'll get approved. I'm pushing him towards a database engine style implementation, so that we can optimise which rules need to run and how much data needs to be retrieved from the base sail. Essentially, if you have a native store with 100 million triples, and you add 1 triple. The SHACL Sail shouldn't need to run all the SHACL validation rules on the entire 100 million +1 triples.

[hmottestad]

Abstract class for the SHACL sail will probably be nice to extract once we have an implementation and realise what would potentially be common between two implementations. When I developed the new RDFS reasoner I started off using the abstract RDFS reasoner as a base, but found out in the end that it was not generic enough. So the abstract RDFS reasoner is a superfluous abstraction that just makes understanding the code more difficult and couldn't be reused for other RDFS reasoner implementations anyway.

[barthanssens]

OK, any objections if I already contribute a SHACL vocabulary class (org.eclipse.rdf4j.model.vocabulary.SHACL) ?
I probably won't have time to work on the sail itself, so if would be great if the GSoC-proposal would be accepted :-)

[hmottestad]

Vocabulary class is a great contribution:)

[tonyhammond]

We think this is a really great idea.

Btw, our approach to SHACL shapes in publishing the datasets in the Springer Nature SciGraph project can be seen here:

https://github.com/springernature/scigraph/tree/master/shapes

[barthanssens]

Nice, I've created a pull request for the vocabulary class with constants for SHACL Classes / properties #802

[VladimirAlexiev]

Hi folks! 2 days ago we submitted an EC H2020 Big Data Research proposal, see https://lists.w3.org/Archives/Public/public-rdf-shapes/2017Apr/0047.html for some info. One of the goals is:

• efficient SHACL validation on big data (out of memory repos).
  Hopefully merge AKSW RDFUnit, TQ SHACL, WESO Shaclex, and improve them significantly.
  Implement SHACL core in optimized Java (for Jena and Sesame)

We'll know the outcome in 5 months.

[VladimirAlexiev]

#695, a prerequisite for SHACL list unrolling rdf:rest*/rdf:first, was recently fixed

[abrokenjester]

@hmottestad can we call this issue done and log any further work on followup issues?