Nifty library to manage, query and store RDF triples. Make RDF great again!
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cmd/triplestore Fixing and improving triplestore CLI Aug 23, 2017
fuzz Fuzzing binary format decoding also Aug 23, 2017
testdata Fixing benches and storing them for comparison Feb 13, 2018
.gitignore Simplifying ntriples parsing Jan 8, 2018
.travis.yml Triggering CI Mar 30, 2017
LICENSE Initial commit Mar 30, 2017
README.md Triples: escape new lines in String Literal object Feb 13, 2018
codec_test.go Triples: escape new lines in String Literal object Feb 13, 2018
codecbench_test.go Simplifying ntriples parsing Jan 8, 2018
decode.go NTriples: newlines escape includes \r Feb 13, 2018
dsl.go Triples from struct: use bnode for embedeed fields Jan 9, 2018
dsl_test.go Lenient NTriples with bnodes & langtag support Aug 23, 2017
encode.go NTriples: newlines escape includes \r Feb 13, 2018
last-benches.txt Triples: escape new lines in String Literal object Feb 13, 2018
ntparser.go Triples: escape new lines in String Literal object Feb 13, 2018
ntparser_test.go Simplifying ntriples parsing Jan 8, 2018
ntparser_w3c_test.go Lenient NTriples with bnodes & langtag support Aug 23, 2017
rdf.go Adding Bnode on Object interface Jan 9, 2018
rdf_test.go Simplifying ntriples parsing Jan 8, 2018
source.go Decoding/Parsing NTriples format against W3C norms Aug 3, 2017
source_test.go Decoding/Parsing NTriples format against W3C norms Aug 3, 2017
streamcodec_test.go Lenient NTriples with bnodes & langtag support Aug 23, 2017
struct.go Triples from struct: use bnode for embedeed fields Jan 9, 2018
struct_test.go Triples from struct: use bnode for embedeed fields Jan 9, 2018
tree.go RDFGraph as a Tree and basic traversal API Jun 26, 2017
tree_test.go RDFGraph as a Tree and basic traversal API Jun 26, 2017
types.go Decoding/Parsing NTriples format against W3C norms Aug 3, 2017
util_test.go Lenient NTriples with bnodes & langtag support Aug 23, 2017

README.md

Build Status Go Report Card GoDoc

Triple Store

Triple Store is a library to manipulate RDF triples in a fast and fluent fashion.

RDF triples allow to represent any data and its relations to other data. It is a very versatile concept and is used in Linked Data, graphs traversal and storage, etc....

Here the RDF triples implementation follows along the W3C RDF concepts. (Note that reification is not implemented.). More digestible info on RDF Wikipedia

Features overview

  • Create and manage triples through a convenient DSL
  • Snapshot and query RDFGraphs
  • Binary encoding/decoding
  • Lenient NTriples encoding/decoding (see W3C Test suite in testdata/ntriples/w3c_suite/)
  • DOT encoding
  • Stream encoding/decoding (for binary & NTriples format) for memory conscious program
  • CLI (Command line interface) utility to read and convert triples files.

Library

This library is written using the Golang language. You need to install Golang before using it.

Get it:

go get -u github.com/wallix/triplestore

Test it:

go test -v -cover -race github.com/wallix/triplestore

Bench it:

go test -run=none -bench=. -benchmem

Import it in your source code:

import (
	"github.com/wallix/triplestore"
	// tstore "github.com/wallix/triplestore" for less verbosity
)

Get the CLI with:

go get -u github.com/wallix/triplestore/cmd/triplestore

Concepts

A triple is made of 3 components:

subject -> predicate -> object

... or you can also view that as:

entity -> attribute -> value

So

  • A triple consists of a subject, a predicate and a object.
  • A subject is a unicode string.
  • A predicate is a unicode string.
  • An object is a resource (or IRI) or a literal (blank node are not supported).
  • A literal is a unicode string associated with a datatype (ex: string, integer, ...).
  • A resource, a.k.a IRI, is a unicode string which point to another resource.

And

  • A source is a persistent yet mutable source or container of triples.
  • A RDFGraph is an immutable set of triples. It is a snapshot of a source and queryable .
  • A dataset is a basically a collection of RDFGraph.

You can also view the library through the godoc

Usage

Create triples

Although you can build triples the way you want to model any data, they are usually built from known RDF vocabularies & namespace. Ex: foaf, ...

triples = append(triples,
	SubjPred("me", "name").StringLiteral("jsmith"),
 	SubjPred("me", "age").IntegerLiteral(26),
 	SubjPred("me", "male").BooleanLiteral(true),
 	SubjPred("me", "born").DateTimeLiteral(time.Now()),
 	SubjPred("me", "mother").Resource("mum#121287"),
)

or dynamically and even shorter with

triples = append(triples,
 	SubjPredLit("me", "age", "jsmith"), // String literal object
 	SubjPredLit("me", "age", 26), // Integer literal object
 	SubjPredLit("me", "male", true), // Boolean literal object
 	SubjPredLit("me", "born", time.now()) // Datetime literal object
 	SubjPredRes("me", "mother", "mum#121287"), // Resource object
)

or with blank nodes and language tag in literal

triples = append(triples,
 	SubjPred("me", "name").Bnode("jsmith"),
 	BnodePred("me", "name").StringLiteral("jsmith"),
 	SubjPred("me", "name").StringLiteralWithLang("jsmith", "en"),
)

Create triples from a struct

As a convenience you can create triples from a singular struct, where you control embedding through bnode.

Here is an example.

type Address struct {
	Street string `predicate:"street"`
	City   string `predicate:"city"`
}

type Person struct {
	Name     string    `predicate:"name"`
	Age      int       `predicate:"age"`
	Size     int64     `predicate:"size"`
	Male     bool      `predicate:"male"`
	Birth    time.Time `predicate:"birth"`
	Surnames []string  `predicate:"surnames"`
	Addr     Address   `predicate:"address" bnode:"myaddress"` // empty bnode value will make bnode value random
}

addr := &Address{...}
person := &Person{Addr: addr, ....}

tris := TriplesFromStruct("jsmith", person)

src := NewSource()
src.Add(tris)
snap := src.Snapshot()

snap.Contains(SubjPredLit("jsmith", "name", "..."))
snap.Contains(SubjPredLit("jsmith", "size", 186))
snap.Contains(SubjPredLit("jsmith", "surnames", "..."))
snap.Contains(SubjPredLit("jsmith", "surnames", "..."))
snap.Contains(SubjPred("me", "address").Bnode("myaddress"))
snap.Contains(BnodePred("myaddress", "street").StringLiteral("5th avenue"))
snap.Contains(BnodePred("myaddress", "city").StringLiteral("New York"))

Equality

	me := SubjPred("me", "name").StringLiteral("jsmith")
 	you := SubjPred("me", "name").StringLiteral("fdupond")

 	if me.Equal(you) {
 	 	...
 	}
)

Triple Source

A source is a persistent yet mutable source or container of triples

src := tstore.NewSource()

src.Add(
	SubjPredLit("me", "age", "jsmith"),
	SubjPredLit("me", "born", time.now()),
)
src.Remove(SubjPredLit("me", "age", "jsmith"))

RDFGraph

A RDFGraph is an immutable set of triples you can query. You get a RDFGraph by snapshotting a source:

graph := src.Snapshot()

tris := graph.WithSubject("me")
for _, tri := range tris {
	...
}

Codec

Triples can be encoded & decoded using either a simple binary format or more standard text format like NTriples, ...

Triples can therefore be persisted to disk, serialized or sent over the network.

For example

enc := NewBinaryEncoder(myWriter)
err := enc.Encode(triples)
...

dec := NewBinaryDecoder(myReader)
triples, err := dec.Decode()

Create a file of triples under the lenient NTriples format:

f, err := os.Create("./triples.nt")
if err != nil {
	return err
}
defer f.Close()

enc := NewLenientNTEncoder(f)
err := enc.Encode(triples)

Encode to a DOT graph

tris := []Triple{
        SubjPredRes("me", "rel", "you"),
        SubjPredRes("me", "rdf:type", "person"),
        SubjPredRes("you", "rel", "other"),
        SubjPredRes("you", "rdf:type", "child"),
        SubjPredRes("other", "any", "john"),
}

err := NewDotGraphEncoder(file, "rel").Encode(tris...)
...

// output
// digraph "rel" {
//  "me" -> "you";
//  "me" [label="me<person>"];
//  "you" -> "other";
//  "you" [label="you<child>"];
//}

Load a binary dataset (i.e. multiple RDFGraph) concurrently from given files:

path := filepath.Join(fmt.Sprintf("*%s", fileExt))
files, _ := filepath.Glob(path)

var readers []io.Reader
for _, f := range files {
	reader, err := os.Open(f)
	if err != nil {
		return g, fmt.Errorf("loading '%s': %s", f, err)
	}
	readers = append(readers, reader)
}

dec := tstore.NewDatasetDecoder(tstore.NewBinaryDecoder, readers...)
tris, err := dec.Decode()
if err != nil {
	return err
}
...

triplestore CLI

This CLI is mainly ised for triples files conversion and inspection. Install it with go get github.com/wallix/triplestore/cmd/triplestore. Then triplestore -h for help.

Example of usage:

triplestore -in ntriples -out bin -files fuzz/ntriples/corpus/samples.nt 
triplestore -in ntriples -out bin -files fuzz/ntriples/corpus/samples.nt 
triplestore -in bin -files fuzz/binary/corpus/samples.bin

RDFGraph as a Tree

A tree is defined from a RDFGraph given:

  • a specific predicate as an edge
  • and considering triples pointing to RDF resource Object

You can then navigate the tree using the existing API calls

tree := tstore.NewTree(myGraph, myPredicate)
tree.TraverseDFS(...)
tree.TraverseAncestors(...)
tree.TraverseSiblings(...)

Have a look at the godoc fro more info

Note that at the moment, constructing a new tree from a graph does not verify if the tree is valid namely no cycle and each child at most one parent.