[Discussion] PySHACL Alternate Modes

[ashleysommer]

PySHACL was originally built to be a basic (but fully standards compliant) SHACL validator. That is, it uses SHACL shapes to check conformance of a data graph, and gives you the result (True/False, plus a ValidationReport).
PySHACL does that job quite well. It can be called from python or from the command line, and it delivers the results users expect.

Over the last 12 months, I've been slowly implementing more of the SHACL Advanced Features spec, and pySHACL is now almost AF-complete.

The Advanced features add capability to SHACL which extends beyond that of just validating. Eg, the SHACL Rules allow you to run SHACL-based entailment on your data graph. SHACL Functions allow you to execute parameterised custom SPARQL Functions over the data graph. Custom Targets allow you to bypass the standard SHACL node-targeting mechanism and use SPARQL to select targets.

These features can use useful to execute validation in a more customisable way, but their major benefit is in the general use outside of just validating a data graph against constraints.

With these new features I see the possibility of PySHACL operating in additional alternative modes, besides that of just validating. Eg, expansion mode could run SHACL-AF Functions and Rules on the data graph, then return the expanded data graph (without validating).

Related to #20

[ashleysommer]

Another alternate operating mode could be a highly targeted version.
See #53

You would be able to specify a single focus-node in the data graph to target, or use one of the SHACL-AF sh:target methods to select a target or set of targets.
You would also pass in the ID of one or more Shapes from the SHACL graph. PySHACL will then validate only those target nodes against only the given shapes.

This would allow for the ability to programmatically guide the PySHACL operation using your own external logic.

[ashleysommer]

I have another idea for a different tool entirely, called shacl-quickcheck. That will be a heavily cut-down version of pyshacl, it will have zero dependencies.
It will have:

• Removed SHACL-SPARQL features including SPARQLConstraint and SPARQLConstraintComponent
• Removed SHACL-AF Custom Targets including SPARQL-Targets and SPARQL-TargetTypes
• Removed SHACL-AF SPARQLFunction (and other SHACL Functions)
• Potentially remove other features (like owl:imports?)
• Bundled stripped down and customized version of RDFLib 5.1.0 with sparql module and sparqlwrapper removed
• New efficient in-memory store (Memory2) available as the only Graph store backend
• Bundled rdflib-jsonld, so it doesn't need to be imported separately.
• Bundled OWL-RL (or removed OWL/RDFS inferencing).
• If possible, all smashed into a single python file.
• Able to be executed directly from the commandline as a python3 script

This will result in:

• Much faster validator start up
• Faster validating - Quicker turnaround time from execution to result.
• No need to install the module before using it
• No need to mess with virtualenvs, dependency trees, version conflicts etc
• ???

[JaimieMurdock]

First: thank you for all your work on PySHACL. This project and its compliance to the W3 specs has made development of SHACL-based tooling incredibly easy.

This issue is tagged as "help wanted". I've been diving into the code base for the validate function and particularly exception handling throughout the module and would like to offer some development cycles for:

1. Improved documentation.
2. Testing of SHACL Advanced Features, particularly the Node Expressions.

There's a lot of exciting things about the advanced features and your exploration of alternate modes. One thing that intersects with my interests in the project is how many matches a particular rule was evaluated against. There's a whole class of false positives caused by ill-formed shapes that never match anything in the ontology being validated - building an engine that can recognize those cases could be very interesting and possibly spur development in an expansion of the SHACL validation report spec.

Two other notes on the shacl-quickcheck project:

1. Based on past experience with bundling stripped down versions of the usual libraries, I would strongly recommend against it - as a maintainer, it significantly increases your workload. There's very little shame in developing a module that can be run with python -m pyshacl.validate myfile. There's always defining new entry_points as well for getting it as a cleaner script.
2. Is installing-an-environment a barrier for the target audience? (i.e., being so deep into semantic web/RDF world that you're doing SHACL validation) If so, maybe a docker file or an environment.yml for conda-based env management would be a better fit?

[rob-metalinkage]

I was about to add a new issue - but I guess this fits under here... when validating it is tricky to get the graph closure right - too little and you get validation errors for object types where the object is not in the graph closure - too much and you get validation errors for all the stuff in the graph closure and not the graph you are validating.

So, rather than running a specific rule (which is probably a good idea...) I want to be able to target a specific graph in validation. (i.e. ignore the ont_graph contents as focus nodes for validation and entailment.)

I also want (currently do this long-hand in scripting) to be able to do entailment+validation step wise - and be able to generate validation checks as a set of entailments and/or transformations are performed on a series of interoperability goals and identify which rules are failing to entail or validate properly.

[rob-metalinkage]

Another idea is to validate in a Linked Data context - where object references are resolved (via URI or a local catalog of some form) - could output the local catalog if you wish to persist between executions.

[ashleysommer]

Hi @rob-metalinkage
Thanks for the feature requests.

I want to be able to target a specific graph in validation.

PySHACL already has built-in support for validating on multi-graph datasets (using rdflib.ConjunctiveGraph and rdflib.Dataset abstractions), but the only option is to do all of the graphs. So it follows that it should be possible to add a feature to generate focus and value nodes from only one target named graph in that Dataset.

To make it a bit easier for implementation and testing, would you be able to generate small demonstration set of files (shape_graph, ont_graph, and data_graph) which demonstrate both cases:

1. too little and you get validation errors for object types where the object is not in the graph closure
2. too much and you get validation errors for all the stuff in the graph closure and not the graph you are validating.

I also want to be able to do entailment+validation step wise
be able to generate validation checks as a set of entailments and/or transformations are performed on a series of interoperability goals and identify which rules are failing to entail or validate properly.

I don't think I'm quite following what you're asking there. Can you simplify it, or provide an example?

Another idea is to validate in a Linked Data context - where object references are resolved (via URI or a local catalog of some form) - could output the local catalog if you wish to persist between executions.

I think another user asked about a similar feature. Could be worth visiting in the future.

[rob-metalinkage]

ok here are three tiny graphs for the source, validator and ont_graph and the test report.

the extra ont has two things - one a skos:Concept to make the test.ttl valid and an extra invalid reference to show it gets validated as well.

test_validator.zip

[rob-metalinkage]

And its also important that when running advanced features the output graph excludes the content of the ont_graph... I don't think it can be treating as working unless thats the default behaviour. There may be a need to include it bu I can't imagine it - and it would be trivial for users to add after the fact if required.

[ashleysommer]

important that when running advanced features the output graph excludes the content of the ont_graph

Its unclear, are you taking about using SHACL-AF features as they stand now, or talking about potential future alternate modes (as per the discussion in this thread)?

In its current form, PySHACL's output graph is a graph containing a validation report and nothing else. I think you're referring to the input data graph. PySHACL creates a working copy of the input datagraph in memory, which it then uses to mix in the ont_graph, apply rdfs/owl inferencing, and apply SHACL-AF Rules. This is done in a working copy to avoid polluting the datagraph (its also a requirement that SHACL validation engines should not modify the datagraph).

The inplace option was added for users who need skip the working-copy step, and forces PySHACL to operate directly on the input datagraph. This is useful for users to inspect the contents of the datagraph after validation was complete, which is not normally possible, (it is also useful in the case that the datagraph is very large, and its not feasible to create a working copy in memory). The inplace option is undocumented and is an unsupported mode of operation at this time.

When using the inplace option, then it is expected and desired that the contents of the ont_graph is in the datagraph after the validation is run, that is its purpose.

If however you're taking about a future alternate mode of operation, where PySHACL is used as a kind of entailment engine, then yes in that case the output graph would be an inflated closure of the input graph, and I agree, it should probably not include the ont_graph contents by default.

[rob-metalinkage]

Am talking about the entailment engine idea I guess,.

when I'm dealing with a source artefact my in-memory graph is always a working copy, so agnostic about changing it.

[ashleysommer]

A related topic has come up on the SHACL Mailing list today, in relation to the TopBraid validation engine. It was discussed there that TopBraid has a tool called shaclinfer that runs the engine in an alternate mode as discussed at the top of this thread.

I think that is a logical starting point to look at how such a mode would work for PySHACL.