rdflib-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Knowledge Representation on the Web -- RDF tutorial\n",
    "\n",
    "In this tutorial we'll learn the basics of interacting with RDF graphs with Python. We'll be using rdflib for this, a widely used Ptyhon library for RDF (all documentation can be found [here](https://rdflib.readthedocs.io/en/stable/index.html))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports\n",
    "These are the main classes and types we'll be using from rdflib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install rdflib\n",
    "\n",
    "from rdflib import Graph, ConjunctiveGraph, Literal, BNode, Namespace, RDF, URIRef\n",
    "from rdflib.namespace import DC, FOAF\n",
    "\n",
    "import pprint\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading data remotely and from files\n",
    "\n",
    "rdflib accepts importing RDF data from a variety of sources, either locally from a file (including an extensive support of serializations), or remotely via a URI (this is a great way of checking practically if URIs return RDF according to the 3rd Linked Data principle).\n",
    "\n",
    "A Graph object is always required to load triples.\n",
    "**Note**: to load quads, and hence supporting named graphs, you'll need to use an instance of ConjunctiveGraph instead\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "h = Graph()\n",
    "result = g.parse(\"http://www.w3.org/People/Berners-Lee/card\")\n",
    "\n",
    "result2 = h.parse(\"https://csarven.ca/\")\n",
    "\n",
    "print(\"Graph has %s statements.\" % len(g))\n",
    "print(\"Graph has %s statements.\" % len(h))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"demo.nt\", \"r\") as f:\n",
    "    print(\"This is an N-Triples file\")\n",
    "    print(\"=========================\")\n",
    "    for line in f:\n",
    "        print(line)\n",
    "    \n",
    "with open(\"demo.xml\", \"r\") as f:\n",
    "    print(\"This is a RDF/XML file\")\n",
    "    print(\"======================\")\n",
    "    for line in f:\n",
    "        print(line)\n",
    "    \n",
    "with open(\"demo.ttl\", \"r\") as f:\n",
    "    print(\"This is a Turtle file\")\n",
    "    print(\"=====================\")\n",
    "    for line in f:\n",
    "        print(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "g.parse(\"demo.ttl\", format='turtle')\n",
    "\n",
    "len(g) # prints 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for stmt in g:\n",
    "    pprint.pprint(stmt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can also parse directly from a string\n",
    "g = Graph()\n",
    "\n",
    "g.parse(data = '<urn:a> <urn:p> <urn:b>.', format='n3')\n",
    "for stmt in g:\n",
    "    pprint.pprint(stmt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving RDF graphs\n",
    "\n",
    "We use the function Graph.serialize(format)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "g.parse(\"demo.nt\", format='n3')\n",
    "# for a,b,c in g:\n",
    "#     print(a,b,c)\n",
    "\n",
    "print(g.serialize(format='nt').decode()) # 'html', 'hturtle', 'mdata', 'microdata', 'n3', 'nquads', 'nt', 'rdfa', 'rdfa1.0', 'rdfa1.1', 'trix', 'turtle', 'xml'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Merging graphs\n",
    "\n",
    "Merging graphs can be done via sequential parsings or by the overloaded operator +\n",
    "\n",
    "**Note:** Set-theoretic graph semantics apply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph = Graph()\n",
    "\n",
    "# Sequential parsings merge *new* triples\n",
    "\n",
    "graph.parse(\"demo.nt\", format='nt')\n",
    "graph.parse(\"demo.xml\", format='xml')\n",
    "\n",
    "print(\"Graph has {} triples\".format(len(graph)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g1 = Graph()\n",
    "g1.parse(\"demo.nt\", format='nt')\n",
    "print(\"g1 has {} triples\".format(len(g1)))\n",
    "\n",
    "g2 = Graph()\n",
    "g2.parse(\"demo.xml\", format='xml')\n",
    "print(\"g2 has {} triples\".format(len(g2)))\n",
    "\n",
    "graph = g1 + g2\n",
    "print(\"g1 + g2 has {} triples\".format(len(graph)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, if we merge graphs with different contents\n",
    "\n",
    "tim_g = Graph()\n",
    "tim_g.parse(\"http://www.w3.org/People/Berners-Lee/card\")\n",
    "print(\"Tim graph has {} triples\".format(len(tim_g)))\n",
    "\n",
    "g3 = g1 + tim_g\n",
    "print(\"g3 has {} triples\".format(len(g3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Creating RDF triples\n",
    "\n",
    "Triples are added to the graph with the function Graph.add()\n",
    "\n",
    "The parameter is a triple given in a Python **tuple** (subject, predicate, object)\n",
    "\n",
    "Notice the namespace convenience syntax!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "\n",
    "# Create an identifier to use as the subject for Donna.\n",
    "donna = BNode()\n",
    "# donna = URIRef(\"http://example.org/donna\")\n",
    "\n",
    "# Add triples using store's add method.\n",
    "g.add( (donna, RDF.type, FOAF.Person) )\n",
    "g.add( (donna, FOAF.nick, Literal(\"donna\", lang=\"foo\")) )\n",
    "g.add( (donna, FOAF.name, Literal(\"Donna Fales\")) )\n",
    "g.add( (donna, FOAF.mbox, URIRef(\"mailto:donna@example.org\")) )\n",
    "\n",
    "for i in range(100):\n",
    "    g.add((donna, FOAF.hadAge, Literal(i)))\n",
    "\n",
    "print(len(g))\n",
    "\n",
    "for s in g:\n",
    "    pprint.pprint(s)\n",
    "    print()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for stmt in g:\n",
    "    print(stmt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(FOAF.Person)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(FOAF.imadethisup)\n",
    "\n",
    "TD - "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Navigating graphs\n",
    "\n",
    "rdflib uses iterators to navigate Graphs. The methods for navigating subjects, predicates and objects are Graph.subjects, Graph.predicates, Graph.objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "g.parse(\"http://www.w3.org/People/Berners-Lee/card\")\n",
    "\n",
    "# Iterate over triples in store and print them out.\n",
    "print(\"--- printing raw triples ---\")\n",
    "for s, p, o in g:\n",
    "    print(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Printing subjects, predicates and objects out of the tuple omits Python datatypes\n",
    "print(\"--- printing raw triples ---\")\n",
    "for s, p, o in g:\n",
    "    print(s, p, o)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"PRINTING SUBJECTS\")\n",
    "for s in g.subjects():\n",
    "    print(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"PRINTING PREDICATES\")\n",
    "for p in g.predicates():\n",
    "    if len(p) > 13:\n",
    "        print(p)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"PRINTING OBJECTS\")\n",
    "for o in g.objects():\n",
    "    print(o)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also filter the subjects, predicates and objects we want to retrieve, and match their values like in a database \"join\" operation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "\n",
    "# Create an identifier to use as the subject for Donna.\n",
    "donna = URIRef('urn:donna')\n",
    "ila = URIRef('urn:ila')\n",
    "# Add triples using store's add method.\n",
    "g.add( (donna, RDF.type, FOAF.Person) )\n",
    "g.add( (ila, RDF.type, FOAF.Person) )\n",
    "g.add( (ila, RDF.type, FOAF.Teacher) )\n",
    "g.add( (donna, FOAF.nick, Literal(\"donna\", lang=\"foo\")) )\n",
    "g.add( (donna, FOAF.name, Literal(\"Donna Fales\")) )\n",
    "g.add( (donna, FOAF.mbox, URIRef(\"mailto:donna@example.org\")) )\n",
    "g.add( (ila, FOAF.mbox, URIRef(\"mailto:ila@example.org\")) )\n",
    "\n",
    "# For each foaf:Person in the store print out its mbox property.\n",
    "# print(\"--- printing mboxes ---\")\n",
    "# for person in g.subjects(RDF.type, FOAF.Person):\n",
    "#     for mbox in g.objects(person, FOAF.mbox):\n",
    "#         print(mbox)\n",
    "\n",
    "# You can reuse matches of subjects to filter further e.g. objects\n",
    "for entity in g.subjects(RDF.type, None):\n",
    "    print(entity)\n",
    "    for objects in g.objects(entity, RDF.type):\n",
    "        print(objects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic triple matching (almost querying!)\n",
    "\n",
    "We use method Graph.triples and a Python tuple that acts as a mask for specifying our criteria"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "g.parse(\"http://www.w3.org/People/Berners-Lee/card\")\n",
    "\n",
    "tim = URIRef(\"https://www.w3.org/People/Berners-Lee/card#i\")\n",
    "\n",
    "if ( tim, RDF.type, FOAF.Person ) in g:\n",
    "   print(\"This graph knows that Tim is a person!\")\n",
    "\n",
    "if ( tim, None, None ) in g:\n",
    "    print(\"This graph contains triples about Tim!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for s,p,o in g.triples( (None, None, None) ):\n",
    "    print(s,p,o)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for s,p,o in g.triples( (tim, RDF.type, None) ):\n",
    "    print(s,p,o)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Namespaces and bindings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mid_uri = URIRef(\"http://purl.org/midi-ld/midi#\")\n",
    "mid = Namespace(mid_uri)\n",
    "\n",
    "print(mid['hello'])  # as item - for things that are not valid python identifiers\n",
    "print(mid.hello)     # as attribute"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = Graph()\n",
    "\n",
    "# Create an identifier to use as the subject for Donna.\n",
    "donna = BNode()\n",
    "\n",
    "# Add triples using store's add method.\n",
    "g.add( (donna, RDF.type, FOAF.Person) )\n",
    "g.add( (donna, FOAF.nick, Literal(\"donna\", lang=\"foo\")) )\n",
    "g.add( (donna, FOAF.name, Literal(\"Donna Fales\")) )\n",
    "g.add( (donna, FOAF.mbox, URIRef(\"mailto:donna@example.org\")) )\n",
    "\n",
    "print(g.serialize(format='turtle').decode())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foaf_uri = URIRef(\"http://xmlns.com/foaf/0.1/\")\n",
    "foaf_namespace = Namespace(foaf_uri)\n",
    "\n",
    "g = Graph()\n",
    "\n",
    "# Bind a few prefix, namespace pairs for more readable output\n",
    "g.bind(\"foaf\", foaf_namespace)\n",
    "\n",
    "# Create an identifier to use as the subject for Donna.\n",
    "donna = BNode()\n",
    "\n",
    "# Add triples using store's add method.\n",
    "g.add( (donna, RDF.type, FOAF.Person) )\n",
    "g.add( (donna, FOAF.nick, Literal(\"donna\", lang=\"foo\")) )\n",
    "g.add( (donna, FOAF.name, Literal(\"Donna Fales\")) )\n",
    "g.add( (donna, FOAF.mbox, URIRef(\"mailto:donna@example.org\")) )\n",
    "\n",
    "print(g.serialize(format='turtle').decode())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}