content/recipes/maturity/isa-json-conversion-to-rdf-linked-data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bdf247b3",
   "metadata": {},
   "source": [
    "# From ISA JSON to ISA JSON-LD (RDF): Dataset Maturity Level 3\n",
    "\n",
    "<br/>\n",
    "<br/>\n",
    "\n",
    "````{panels_fairplus}\n",
    ":identifier_text: FCB065\n",
    ":identifier_link: 'https://w3id.org/faircookbook/FCB065'\n",
    ":difficulty_level: 2\n",
    ":recipe_type: hands_on\n",
    ":reading_time_minutes: 15\n",
    ":intended_audience: principal_investigator, data_manager, data_scientist\n",
    ":maturity_level: 3\n",
    ":maturity_indicator: 20\n",
    ":has_executable_code: yeah\n",
    ":recipe_name: Moving to a semantically typed version - ISA-JSON-LD\n",
    "````\n",
    "\n",
    "## Abstract:\n",
    "\n",
    "The goal of this tutorial is to show how to go from an ISA document to an equivalent RDF representation using python\n",
    "tools, but also to highlight some limitations of existing libraries and point to alternative options to complete\n",
    "a meaningful conversion to RDF Turtle format.\n",
    "\n",
    "This notebook mainly highlights the new functionality coming with ISA-API rc10.3 latest release which allows converting\n",
    "ISA-JSON to ISA-JSON-LD, with the choice of 3 popular ontological frameworks for semantic anchoring.\n",
    "These are:\n",
    "- [obofoundry](http://www.obofoundry.org), a set of interoperable ontologies for the biological domain.\n",
    "- [schema.org](https://schema.org), the search engine orientated ontology developed by companies such as Yandex,\n",
    "Bing,Google\n",
    "- [wikidata](https://wikidata.org), a set of semantic concepts backing wikipedia and wikidata resources.\n",
    "\n",
    "These frameworks have been chosen for interoperability.\n",
    "\n",
    "\n",
    "This notebook has a companion notebook which goes over the exploration of the resulting RDF representations using\n",
    "a set of SPARQL queries.\n",
    "Check it out [here](http://localhost:8888/notebooks/isa-cookbook/content/notebooks/isa-jsonld%20exploration%20with%20SPARQL.ipynb)\n",
    "\n",
    "\n",
    "- support: isatools@googlegroups.com\n",
    "- issue tracker: https://github.com/ISA-tools/isa-api/issues\n",
    "\n",
    "\n",
    "## Let's get started\n",
    "\n",
    "We do so by getting all necessary `ISA tools` and importing the latest module for conversion to JSON-LD from ISA-JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "from json import load\n",
    "import datetime\n",
    "import isatools\n",
    "from isatools.convert.json2jsonld import ISALDSerializer"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 1. Loading an ISA-JSON document in memory with `json.load()` function\n",
    "\n",
    "\n",
    "Prior to invoking the `ISALDserializer` function, we need to do three things.\n",
    "* First, pass an url or a path to the ISA JSON instance to convert to JSON-LD\n",
    "* Second, select the ontology framework used for the semantic conversion. One may choose from the following 3 options:\n",
    "    - [obofoundry.org](https://obofoundry.org) ontologies, abbreviated as `obo`\n",
    "    - [schema.org](https://schema.org) ontology, abbreviated as `sdo`\n",
    "    - [wikidata.org](https://wikidata.org) ontology, abbreviated as `wdt`\n",
    "* Third, choose if to rely on embedding the `@context` file in the output or relying on url to individual contexts.\n",
    "By default, the converter will embed the `all in one` context information. The reason for this is the lack of support\n",
    "for JSON-LD 1.1 specifications in many of the python libraries supported RDF parsing (e.g. RDFlib)\n",
    "\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "instance_path = os.path.join(\"./output/BII-S-3-synth/\", \"isa-new_ids.json\")\n",
    "\n",
    "with open(instance_path, 'r') as instance_file:\n",
    "        instance = load(instance_file)\n",
    "        instance_file.close()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "id": "17796cdc",
   "metadata": {},
   "source": [
    "## 2. Transforming ISA-JSON to ISA JSON-LD with `ISALDserializer` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b699f208",
   "metadata": {},
   "outputs": [],
   "source": [
    "# we now invoke the ISALDSerializer function\n",
    "\n",
    "ontology = \"isaterms\"\n",
    "\n",
    "serializer = ISALDSerializer(instance)\n",
    "serializer.set_ontology(ontology)\n",
    "serializer.set_instance(instance)\n",
    "\n",
    "json_ld_content = serializer.output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "907178a8",
   "metadata": {},
   "source": [
    "Now that the conversion is performed, we can write the resulting ISA-JSON-LD to file:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fd2e196",
   "metadata": {},
   "source": [
    "## 3. Writing ISA JSON-LD to file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0382b1b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "isa_json_ld_path = os.path.join(\"./output/BII-S-3-synth/\", \"isa-new_ids-BII-S-3-ld-\" + ontology + \"-v1.json\")\n",
    "\n",
    "with open(isa_json_ld_path, 'w') as outfile:\n",
    "    json.dump(json_ld_content,  outfile, ensure_ascii=False, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d68913f7",
   "metadata": {},
   "source": [
    "## 4. Converting ISA-JSONLD instance to RDF Turtle using python RDFlib library\n",
    "```{note}\n",
    "Python RDFlib version should be at last 6.0.2\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90f1c325",
   "metadata": {},
   "outputs": [],
   "source": [
    "from rdflib import Graph\n",
    "\n",
    "graph = Graph()\n",
    "graph.parse(isa_json_ld_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27b79db2",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Graph g has {len(graph)} statements.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f619b690",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write turtle file\n",
    "rdf_path=os.path.join(\"./output/BII-S-3-synth/\", \"isa-new_ids-BII-S-3-ld-\" + ontology + \"-v3.ttl\")\n",
    "with open(rdf_path, 'w') as rdf_file:\n",
    "        rdf_file.write(graph.serialize(format='turtle'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1752636",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Conclusion:\n",
    "\n",
    "### What to read next?\n",
    ">\n",
    "> - [packaging as a Research Object (RO)](https://w3id.org/faircookbook/FCB___)\n",
    ">\n",
    "> - [deposition to Zenodo](https://w3id.org/faircookbook/FCB009)\n",
    ">\n",
    "\n",
    "## Authors\n",
    "````{authors_fairplus}\n",
    "Philippe: Writing - Original Draft\n",
    "Dominique: Writing - Original Draft\n",
    "````\n",
    "\n",
    "## License\n",
    "````{license_fairplus}\n",
    "CC-BY-4.0\n",
    "````\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}