examples/notebooks/getting-started/12 - Mapping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-23T18:50:19.036357Z",
     "start_time": "2019-09-23T18:50:19.031896Z"
    }
   },
   "source": [
    "# Mapping\n",
    "\n",
    "[Mappings](https://nexus-forge.readthedocs.io/en/latest/interaction.html#mapping) are pre-defined and declarative rules that encode the logic on how to transform a specific data source into [Resources](https://nexus-forge.readthedocs.io/en/latest/interaction.html#resource) that eventually conform to targeted schemas supported by the configured [Model](https://nexus-forge.readthedocs.io/en/latest/interaction.html#modeling). \n",
    "\n",
    "This notebook specifically demonstrates the `DictionaryMapping` wich is based on a JSON structure that represent a targeted structure along with Python code that will apply desired transformations on the data source."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-23T18:50:20.068658Z",
     "start_time": "2019-09-23T18:50:19.054054Z"
    }
   },
   "outputs": [],
   "source": [
    "from kgforge.core import KnowledgeGraphForge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A configuration file is needed in order to create a KnowledgeGraphForge session. A configuration can be generated using the notebook [00-Initialization.ipynb](00%20-%20Initialization.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "forge = KnowledgeGraphForge(\"../../configurations/forge.yml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from kgforge.core import Resource"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from kgforge.specializations.mappings import DictionaryMapping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "scientists = [\n",
    "    {\n",
    "        \"id\": 123,\n",
    "        \"name\": \"Marie Curie\",\n",
    "        \"gender\": \"female\",\n",
    "        \"middle_name\": \"Salomea\",\n",
    "    },\n",
    "    {\n",
    "        \"id\": 456,\n",
    "        \"name\": \"Albert Einstein\",\n",
    "        \"gender\": \"male\",\n",
    "        \"middle_name\": \"(missing)\",\n",
    "    },\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mapping data to a targeted template"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: DemoModel and RdfModel schemas have not been synchronized yet. The following cell is to be run with DemoModel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Managed entity types:\n",
      "   - Activity\n",
      "   - Contribution\n",
      "   - Dataset\n",
      "   - Entity\n",
      "   - Ontology\n",
      "   - Person\n"
     ]
    }
   ],
   "source": [
    "forge.types()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    id: \"\"\n",
      "    agent:\n",
      "    {\n",
      "        id: \"\"\n",
      "        type: Agent\n",
      "    }\n",
      "    hadPlan:\n",
      "    {\n",
      "        id: \"\"\n",
      "        type: Plan\n",
      "        label: \"\"\n",
      "    }\n",
      "    hadRole:\n",
      "    {\n",
      "        id: \"\"\n",
      "        type: Role\n",
      "        label: \"\"\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "forge.template(\"Contribution\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapping_simple = DictionaryMapping(\"\"\"\n",
    "    type: Contribution\n",
    "    agent:\n",
    "    {\n",
    "        type: Person\n",
    "        name: x.name\n",
    "    }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "resources_simple = forge.map(scientists, mapping_simple)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    type: Contribution\n",
      "    agent:\n",
      "    {\n",
      "        type: Person\n",
      "        name: Marie Curie\n",
      "    }\n",
      "}\n",
      "{\n",
      "    type: Contribution\n",
      "    agent:\n",
      "    {\n",
      "        type: Person\n",
      "        name: Albert Einstein\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(*resources_simple, sep=\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapping_na = DictionaryMapping(\"\"\"\n",
    "    type: Contribution\n",
    "    agent:\n",
    "    {\n",
    "        type: Person\n",
    "        name: x.name\n",
    "        additionalName: x.middle_name\n",
    "    }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    type: Contribution\n",
      "    agent:\n",
      "    {\n",
      "        type: Person\n",
      "        additionalName: (missing)\n",
      "        name: Albert Einstein\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(forge.map(scientists[1], mapping_na))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    type: Contribution\n",
      "    agent:\n",
      "    {\n",
      "        type: Person\n",
      "        name: Albert Einstein\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(forge.map(scientists[1], mapping_na, na=\"(missing)\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### multiple mappings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapping_person = DictionaryMapping(\"\"\"\n",
    "    id: forge.format(\"identifier\", \"persons\", x.id)\n",
    "    type: Person\n",
    "    name: x.name\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapping_association = DictionaryMapping(\"\"\"\n",
    "    type: Contribution\n",
    "    agent: forge.format(\"identifier\", \"persons\", x.id)\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "resources_mapped = forge.map(scientists, [mapping_person, mapping_association])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(resources_mapped) == 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    id: https://kg.example.ch/persons/123\n",
      "    type: Person\n",
      "    name: Marie Curie\n",
      "}\n",
      "{\n",
      "    type: Contribution\n",
      "    agent: https://kg.example.ch/persons/123\n",
      "}\n",
      "{\n",
      "    id: https://kg.example.ch/persons/456\n",
      "    type: Person\n",
      "    name: Albert Einstein\n",
      "}\n",
      "{\n",
      "    type: Contribution\n",
      "    agent: https://kg.example.ch/persons/456\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(*resources_mapped, sep=\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Managing mappings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = \"mappings/scientists-database/DictionaryMapping/Contribution.hjson\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### loading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded = DictionaryMapping.load(filepath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### saving"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded.save(filepath)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7 (nexusforgelatest)",
   "language": "python",
   "name": "nexusforgelatest"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}