Skip to content

Example illustrating provenance querying

Jump to bottom
Hassaan edited this page Jun 2, 2021 · 16 revisions

The example below illustrates the use of the query client.

After the SPADE server has been started, add the Neo4j storage and DSL reporter using the SPADE controller (invoked with spade control):

-> add storage Neo4j database=spade_database
Adding storage Neo4j... done

-> add reporter DSL /tmp/spade_pipe
Adding reporter DSL... done

To create the provenance metadata needed for the example, use the following commands in terminal or shell to add the vertices and edges:

echo type:Process id:1 name:root\\ process pid:10 >> /tmp/spade_pipe 
echo type:Process id:2 name:child\\ process pid:32 >> /tmp/spade_pipe 
echo type:WasTriggeredBy from:2 to:1 time:5\\:56\\ PM >> /tmp/spade_pipe 
echo type:Artifact id:3 filename:output.tmp >> /tmp/spade_pipe 
echo type:Artifact id:4 filename:output.o >> /tmp/spade_pipe 
echo type:Used from:2 to:3 iotime:12\\ ms >> /tmp/spade_pipe 
echo type:WasGeneratedBy from:4 to:2 iotime:11\\ ms >> /tmp/spade_pipe 
echo type:WasDerivedFrom from:4 to:3 >> /tmp/spade_pipe 
echo type:Agent id:user uid:10 gid:10 name:john >> /tmp/spade_pipe 
echo type:WasControlledBy from:1 to:user >> /tmp/spade_pipe 
echo type:WasControlledBy from:2 to:user >> /tmp/spade_pipe

If the Graphviz storage had been added, the resulting graph would be:

Note that the Open Provenance Model (OPM) convention of using octagons for Agent vertices, rectangles for Process vertices, and ellipses for Artifact vertices is followed. In addition, the graph elements are colored with the following semantics:

Color OPM meaning
pink Agent
light blue Process
yellow Artifact
purple WasControlledBy
green Used
red WasGeneratedBy
dark blue WasTriggeredBy
orange WasDerivedFrom

Using the SPADE query client (invoked with spade query), the graph of the provenance of the root process (which has a pid equal to 10) can be retrieved:

-> $c1_lineage_limit_10 = $base.getLineage($base.getVertex("pid" == '10'), 100, 'ancestors').limit(10)
-> export > /tmp/ancestors.dot
-> dump all $c1_lineage_limit_10
Output exported to file /tmp/ancestors.dot

The file /tmp/ancestors.dot is in Graphviz format. When rendered it looks like this:

Similarly, the graph in which the same vertex is part of the provenance can be retrieved with:

-> $c1_lineage_limit_10_desc = $base.getLineage($base.getVertex("pid" == '10'), 100, 'descendants').limit(10)
-> export > /tmp/descendants.dot
-> dump all $c1_lineage_limit_10_desc
Output exported to file /tmp/descendants.dot

When rendered, it looks like this:

This material is based upon work supported by the National Science Foundation under Grants OCI-0722068, IIS-1116414, and ACI-1547467. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Clone this wiki locally