greengamma/general/Enrichment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Using ROBOKOP's enrichment service"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "ROBOKOP's enrichment service, takes a list of identifiers, and returns a list of identifiers (of a given type) that are enriched for connections to the original set.  The calculation here is well-known as \"GO Enrichment\" in which given a gene list, the enrichment finds GO terms that are associated with the genes in the set much more frequently than chance would suppose.\n",
    "\n",
    "The following function shows how the enrichment service can be called.  Note that enrichment takes a POST request, so that a list of identifiers can be easily passed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "robokop_server = 'robokop.renci.org'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def enrichment(type1,identlist,type2,threshhold=None,maxresults=None,numtype1=None,include_descendants=None,rebuild=None):\n",
    "    url=f'http://{robokop_server}/api/simple/enriched/{type1}/{type2}'\n",
    "    params = { 'threshhold': threshhold, 'max_results': maxresults, \n",
    "              'num_type1':numtype1, 'identifiers': identlist, \n",
    "              'include_descendants':include_descendants, 'rebuild': rebuild }\n",
    "    params = { k:v for k,v in params.items() if v is not None }\n",
    "    response=requests.post(url, json = params)\n",
    "    print( f'Return Status: {response.status_code}' )\n",
    "    if response.status_code == 200:\n",
    "        return response.json()\n",
    "    return []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Basic Usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's find phenotypes that are enriched for diabetes.  In particular, let's choose two versions of diabetes, type 1 diabetes mellitus (MONDO:0005147) and type two diabetes mellitus (MONDO:0005148).   The return is a list of dicts, which is easily converted to a pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "diabetes = ['MONDO:0005147','MONDO:0005148']\n",
    "type1='disease'\n",
    "type2='phenotypic_feature'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NCIT:C76325</td>\n",
       "      <td>Birth Weight</td>\n",
       "      <td>0.000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NCIT:C81328</td>\n",
       "      <td>Body Weight</td>\n",
       "      <td>0.000004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HP:0005974</td>\n",
       "      <td>Episodic ketoacidosis</td>\n",
       "      <td>0.000020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HP:0011016</td>\n",
       "      <td>Abnormality of urine glucose concentration</td>\n",
       "      <td>0.000021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HP:0006279</td>\n",
       "      <td>Beta-cell dysfunction</td>\n",
       "      <td>0.000024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HP:0030082</td>\n",
       "      <td>Abnormal drinking behavior</td>\n",
       "      <td>0.000026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HP:0005979</td>\n",
       "      <td>Metabolic ketoacidosis</td>\n",
       "      <td>0.000035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HP:0100739</td>\n",
       "      <td>Bulimia</td>\n",
       "      <td>0.000035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HP:0012093</td>\n",
       "      <td>Abnormality of endocrine pancreas physiology</td>\n",
       "      <td>0.000036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HP:0006476</td>\n",
       "      <td>Abnormality of the pancreatic islet cells</td>\n",
       "      <td>0.000037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>HP:0012592</td>\n",
       "      <td>Albuminuria</td>\n",
       "      <td>0.000042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>HP:0011998</td>\n",
       "      <td>Postprandial hyperglycemia</td>\n",
       "      <td>0.000042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>HP:0001952</td>\n",
       "      <td>Abnormal glucose tolerance</td>\n",
       "      <td>0.000044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>HP:0040214</td>\n",
       "      <td>Abnormal insulin level</td>\n",
       "      <td>0.000057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>HP:0040215</td>\n",
       "      <td>Abnormal circulating insulin level</td>\n",
       "      <td>0.000067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>HP:0000847</td>\n",
       "      <td>Abnormality of renin-angiotensin system</td>\n",
       "      <td>0.000067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>HP:0001095</td>\n",
       "      <td>Hypertensive retinopathy</td>\n",
       "      <td>0.000072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>HP:0002919</td>\n",
       "      <td>Ketonuria</td>\n",
       "      <td>0.000088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>HP:0002717</td>\n",
       "      <td>Adrenal overactivity</td>\n",
       "      <td>0.000100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>HP:0006704</td>\n",
       "      <td>Abnormal coronary artery morphology</td>\n",
       "      <td>0.000103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>HP:0001950</td>\n",
       "      <td>Respiratory alkalosis</td>\n",
       "      <td>0.000106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>HP:0004904</td>\n",
       "      <td>Maturity-onset diabetes of the young</td>\n",
       "      <td>0.000114</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>HP:0001993</td>\n",
       "      <td>Ketoacidosis</td>\n",
       "      <td>0.000120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>HP:0001946</td>\n",
       "      <td>Ketosis</td>\n",
       "      <td>0.000128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>HP:0001325</td>\n",
       "      <td>Hypoglycemic coma</td>\n",
       "      <td>0.000144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>HP:0040085</td>\n",
       "      <td>Abnormal circulating aldosterone</td>\n",
       "      <td>0.000151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>HP:0031137</td>\n",
       "      <td>Storage in hepatocytes</td>\n",
       "      <td>0.000187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>HP:0012332</td>\n",
       "      <td>Abnormal autonomic nervous system physiology</td>\n",
       "      <td>0.000207</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>HP:0004943</td>\n",
       "      <td>Accelerated atherosclerosis</td>\n",
       "      <td>0.000211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>HP:0002578</td>\n",
       "      <td>Gastroparesis</td>\n",
       "      <td>0.000238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>HP:0012874</td>\n",
       "      <td>Abnormal male reproductive system physiology</td>\n",
       "      <td>0.000913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>HP:0002270</td>\n",
       "      <td>Abnormality of the autonomic nervous system</td>\n",
       "      <td>0.000917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>HP:0000168</td>\n",
       "      <td>Abnormality of the gingiva</td>\n",
       "      <td>0.000926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>HP:0001278</td>\n",
       "      <td>Orthostatic hypotension</td>\n",
       "      <td>0.000939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>HP:0002958</td>\n",
       "      <td>Immune dysregulation</td>\n",
       "      <td>0.000975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>HP:0002634</td>\n",
       "      <td>Arteriosclerosis</td>\n",
       "      <td>0.000994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>HP:0001454</td>\n",
       "      <td>Abnormality of the upper arm</td>\n",
       "      <td>0.001003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>HP:0002148</td>\n",
       "      <td>Hypophosphatemia</td>\n",
       "      <td>0.001008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>HP:0009800</td>\n",
       "      <td>Maternal diabetes</td>\n",
       "      <td>0.001017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>NCIT:C53652</td>\n",
       "      <td>Acute Coronary Syndrome</td>\n",
       "      <td>0.001020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>HP:0002577</td>\n",
       "      <td>Abnormality of the stomach</td>\n",
       "      <td>0.001045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>HP:0045073</td>\n",
       "      <td>Serositis</td>\n",
       "      <td>0.001083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>HP:0002181</td>\n",
       "      <td>Cerebral edema</td>\n",
       "      <td>0.001107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>HP:0001959</td>\n",
       "      <td>Polydipsia</td>\n",
       "      <td>0.001112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>HP:0008046</td>\n",
       "      <td>Abnormality of the retinal vasculature</td>\n",
       "      <td>0.001112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>HP:0001677</td>\n",
       "      <td>Coronary artery atherosclerosis</td>\n",
       "      <td>0.001156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>HP:0000842</td>\n",
       "      <td>Hyperinsulinemia</td>\n",
       "      <td>0.001156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>HP:0009125</td>\n",
       "      <td>Lipodystrophy</td>\n",
       "      <td>0.001176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>HP:0011772</td>\n",
       "      <td>Abnormality of thyroid morphology</td>\n",
       "      <td>0.001196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>HP:0004360</td>\n",
       "      <td>Abnormality of acid-base homeostasis</td>\n",
       "      <td>0.001294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>HP:0000802</td>\n",
       "      <td>Impotence</td>\n",
       "      <td>0.001294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>HP:0009124</td>\n",
       "      <td>Abnormal adipose tissue morphology</td>\n",
       "      <td>0.001299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>NCIT:C94250</td>\n",
       "      <td>Overweight</td>\n",
       "      <td>0.001312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>HP:0000103</td>\n",
       "      <td>Polyuria</td>\n",
       "      <td>0.001315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>HP:0031466</td>\n",
       "      <td>Impairment in personality functioning</td>\n",
       "      <td>0.001363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>HP:0003117</td>\n",
       "      <td>Abnormality of circulating hormone level</td>\n",
       "      <td>0.001406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>HP:0003074</td>\n",
       "      <td>Hyperglycemia</td>\n",
       "      <td>0.001456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>HP:0000956</td>\n",
       "      <td>Acanthosis nigricans</td>\n",
       "      <td>0.001462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>HP:0100753</td>\n",
       "      <td>Schizophrenia</td>\n",
       "      <td>0.001490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>HP:0002585</td>\n",
       "      <td>Abnormality of the peritoneum</td>\n",
       "      <td>0.001535</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                          name         p\n",
       "0   NCIT:C76325                                  Birth Weight  0.000001\n",
       "1   NCIT:C81328                                   Body Weight  0.000004\n",
       "2    HP:0005974                         Episodic ketoacidosis  0.000020\n",
       "3    HP:0011016    Abnormality of urine glucose concentration  0.000021\n",
       "4    HP:0006279                         Beta-cell dysfunction  0.000024\n",
       "5    HP:0030082                    Abnormal drinking behavior  0.000026\n",
       "6    HP:0005979                        Metabolic ketoacidosis  0.000035\n",
       "7    HP:0100739                                       Bulimia  0.000035\n",
       "8    HP:0012093  Abnormality of endocrine pancreas physiology  0.000036\n",
       "9    HP:0006476     Abnormality of the pancreatic islet cells  0.000037\n",
       "10   HP:0012592                                   Albuminuria  0.000042\n",
       "11   HP:0011998                    Postprandial hyperglycemia  0.000042\n",
       "12   HP:0001952                    Abnormal glucose tolerance  0.000044\n",
       "13   HP:0040214                        Abnormal insulin level  0.000057\n",
       "14   HP:0040215            Abnormal circulating insulin level  0.000067\n",
       "15   HP:0000847       Abnormality of renin-angiotensin system  0.000067\n",
       "16   HP:0001095                      Hypertensive retinopathy  0.000072\n",
       "17   HP:0002919                                     Ketonuria  0.000088\n",
       "18   HP:0002717                          Adrenal overactivity  0.000100\n",
       "19   HP:0006704           Abnormal coronary artery morphology  0.000103\n",
       "20   HP:0001950                         Respiratory alkalosis  0.000106\n",
       "21   HP:0004904          Maturity-onset diabetes of the young  0.000114\n",
       "22   HP:0001993                                  Ketoacidosis  0.000120\n",
       "23   HP:0001946                                       Ketosis  0.000128\n",
       "24   HP:0001325                             Hypoglycemic coma  0.000144\n",
       "25   HP:0040085              Abnormal circulating aldosterone  0.000151\n",
       "26   HP:0031137                        Storage in hepatocytes  0.000187\n",
       "27   HP:0012332  Abnormal autonomic nervous system physiology  0.000207\n",
       "28   HP:0004943                   Accelerated atherosclerosis  0.000211\n",
       "29   HP:0002578                                 Gastroparesis  0.000238\n",
       "..          ...                                           ...       ...\n",
       "70   HP:0012874  Abnormal male reproductive system physiology  0.000913\n",
       "71   HP:0002270   Abnormality of the autonomic nervous system  0.000917\n",
       "72   HP:0000168                    Abnormality of the gingiva  0.000926\n",
       "73   HP:0001278                       Orthostatic hypotension  0.000939\n",
       "74   HP:0002958                          Immune dysregulation  0.000975\n",
       "75   HP:0002634                              Arteriosclerosis  0.000994\n",
       "76   HP:0001454                  Abnormality of the upper arm  0.001003\n",
       "77   HP:0002148                              Hypophosphatemia  0.001008\n",
       "78   HP:0009800                             Maternal diabetes  0.001017\n",
       "79  NCIT:C53652                       Acute Coronary Syndrome  0.001020\n",
       "80   HP:0002577                    Abnormality of the stomach  0.001045\n",
       "81   HP:0045073                                     Serositis  0.001083\n",
       "82   HP:0002181                                Cerebral edema  0.001107\n",
       "83   HP:0001959                                    Polydipsia  0.001112\n",
       "84   HP:0008046        Abnormality of the retinal vasculature  0.001112\n",
       "85   HP:0001677               Coronary artery atherosclerosis  0.001156\n",
       "86   HP:0000842                              Hyperinsulinemia  0.001156\n",
       "87   HP:0009125                                 Lipodystrophy  0.001176\n",
       "88   HP:0011772             Abnormality of thyroid morphology  0.001196\n",
       "89   HP:0004360          Abnormality of acid-base homeostasis  0.001294\n",
       "90   HP:0000802                                     Impotence  0.001294\n",
       "91   HP:0009124            Abnormal adipose tissue morphology  0.001299\n",
       "92  NCIT:C94250                                    Overweight  0.001312\n",
       "93   HP:0000103                                      Polyuria  0.001315\n",
       "94   HP:0031466         Impairment in personality functioning  0.001363\n",
       "95   HP:0003117      Abnormality of circulating hormone level  0.001406\n",
       "96   HP:0003074                                 Hyperglycemia  0.001456\n",
       "97   HP:0000956                          Acanthosis nigricans  0.001462\n",
       "98   HP:0100753                                 Schizophrenia  0.001490\n",
       "99   HP:0002585                 Abnormality of the peritoneum  0.001535\n",
       "\n",
       "[100 rows x 3 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values = enrichment(type1, diabetes, type2)\n",
    "diabetes_enriched_phenotypes = pd.DataFrame(values)\n",
    "diabetes_enriched_phenotypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Limiting results with maxresults and threshhold"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "By default, the enrichment service returns the top 100 results.   That number can be controlled with the maxresults parameter.  If set to 0, all results are returned"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n",
      "Return Status: 200\n",
      "Return Status: 200\n",
      "Calling with no maxresults parameter returned 100 results\n",
      "Calling with maxresults=0 parameter returned 250 results\n",
      "Calling with maxresults=10 parameter returned 10 results\n"
     ]
    }
   ],
   "source": [
    "top_100 = enrichment(type1, diabetes, type2)\n",
    "all_results = enrichment(type1, diabetes, type2, maxresults = 0)\n",
    "top_10 = enrichment(type1, diabetes, type2, maxresults = 10)\n",
    "\n",
    "print(f'Calling with no maxresults parameter returned {len(top_100)} results')\n",
    "print(f'Calling with maxresults=0 parameter returned {len(all_results)} results')\n",
    "print(f'Calling with maxresults=10 parameter returned {len(top_10)} results')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "A node's score in an enrichment calculation is a p-value, in this case calculated from a hypergeometric distribution.  Because it's a p-value, lower is better.  The default (maximum) p-value returned is 0.05.  Note that this p-value is uncorrected for multiplicity at the moment.  The maximum p-value threshold can be specified to control the amount of results that are returned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n",
      "18 results returned with p < 0.0001\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NCIT:C76325</td>\n",
       "      <td>Birth Weight</td>\n",
       "      <td>0.000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NCIT:C81328</td>\n",
       "      <td>Body Weight</td>\n",
       "      <td>0.000004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HP:0005974</td>\n",
       "      <td>Episodic ketoacidosis</td>\n",
       "      <td>0.000020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HP:0011016</td>\n",
       "      <td>Abnormality of urine glucose concentration</td>\n",
       "      <td>0.000021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HP:0006279</td>\n",
       "      <td>Beta-cell dysfunction</td>\n",
       "      <td>0.000024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HP:0030082</td>\n",
       "      <td>Abnormal drinking behavior</td>\n",
       "      <td>0.000026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HP:0005979</td>\n",
       "      <td>Metabolic ketoacidosis</td>\n",
       "      <td>0.000035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HP:0100739</td>\n",
       "      <td>Bulimia</td>\n",
       "      <td>0.000035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HP:0012093</td>\n",
       "      <td>Abnormality of endocrine pancreas physiology</td>\n",
       "      <td>0.000036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HP:0006476</td>\n",
       "      <td>Abnormality of the pancreatic islet cells</td>\n",
       "      <td>0.000037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>HP:0012592</td>\n",
       "      <td>Albuminuria</td>\n",
       "      <td>0.000042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>HP:0011998</td>\n",
       "      <td>Postprandial hyperglycemia</td>\n",
       "      <td>0.000042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>HP:0001952</td>\n",
       "      <td>Abnormal glucose tolerance</td>\n",
       "      <td>0.000044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>HP:0040214</td>\n",
       "      <td>Abnormal insulin level</td>\n",
       "      <td>0.000057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>HP:0040215</td>\n",
       "      <td>Abnormal circulating insulin level</td>\n",
       "      <td>0.000067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>HP:0000847</td>\n",
       "      <td>Abnormality of renin-angiotensin system</td>\n",
       "      <td>0.000067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>HP:0001095</td>\n",
       "      <td>Hypertensive retinopathy</td>\n",
       "      <td>0.000072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>HP:0002919</td>\n",
       "      <td>Ketonuria</td>\n",
       "      <td>0.000088</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                          name         p\n",
       "0   NCIT:C76325                                  Birth Weight  0.000001\n",
       "1   NCIT:C81328                                   Body Weight  0.000004\n",
       "2    HP:0005974                         Episodic ketoacidosis  0.000020\n",
       "3    HP:0011016    Abnormality of urine glucose concentration  0.000021\n",
       "4    HP:0006279                         Beta-cell dysfunction  0.000024\n",
       "5    HP:0030082                    Abnormal drinking behavior  0.000026\n",
       "6    HP:0005979                        Metabolic ketoacidosis  0.000035\n",
       "7    HP:0100739                                       Bulimia  0.000035\n",
       "8    HP:0012093  Abnormality of endocrine pancreas physiology  0.000036\n",
       "9    HP:0006476     Abnormality of the pancreatic islet cells  0.000037\n",
       "10   HP:0012592                                   Albuminuria  0.000042\n",
       "11   HP:0011998                    Postprandial hyperglycemia  0.000042\n",
       "12   HP:0001952                    Abnormal glucose tolerance  0.000044\n",
       "13   HP:0040214                        Abnormal insulin level  0.000057\n",
       "14   HP:0040215            Abnormal circulating insulin level  0.000067\n",
       "15   HP:0000847       Abnormality of renin-angiotensin system  0.000067\n",
       "16   HP:0001095                      Hypertensive retinopathy  0.000072\n",
       "17   HP:0002919                                     Ketonuria  0.000088"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p1m4 = enrichment(type1, diabetes, type2, threshhold=0.0001)\n",
    "print (f'{len(p1m4)} results returned with p < 0.0001')\n",
    "pd.DataFrame(p1m4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Caching and rebuilding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "ROBOKOP maintains cached results.  The cache is built both opportunistically (including the results of all previous queries) and proactively (pre-loading data that expected to be heavily used).  By default, expand only looks in its cache.  If a result has not been previously cached, then this call will not return anything (and may return a status code of 500).\n",
    "\n",
    "If a user wants to force the service to look beyond its local cache, it sends a parameter `rebuild=True`, as seen in the NPC1 examples above.\n",
    "\n",
    "If a user wants to be sure to retreive all relevant data, they should use `rebuild=True`, but this will be at the expense of performance.  In order to increase performance without sacrificing reliability, certain type pairs are preloaded into the cache.  In this case, there will be no difference in results between calling `rebuild=True` and `rebuild=False`, but calling with `rebuild=True` will be noticeably slower.\n",
    "\n",
    "Certain pairs of types are preloaded into ROBOKOP's cache, so there is no point in using rebuild for them. The following list will be updated as the preloaded list is modified.  Note that with the data loaded, it doesn't matter which type is the query and which is the resut.  That is, if a row in this table specifies `disease` and `phenotypic_feature`, then there is no reason to use rebuild for `type1='disease' type2='phenotypic_feature'` or `type1='phenotypic_feature' type2='disease'`.\n",
    "\n",
    "See the \"Expand\" notebook for the most up-to-date list of features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## p-values and numtype1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The calculation of p-value in an enrichment requires the total number of things of type 1 that exist.  So if type 1 is genes, then the p-value requires the total number of genes.  This value is passed in the (optional) parameter `numtype1`.  If numtype1 is not specified, then the ROBOKOP enrichment service will estimate it from the cache.  If the entity type is one that has been cached (_i.e._ it's in the table above), then numtype should be well specified.  If not, then passing in an estimate will give a more robust p-value.\n",
    "\n",
    "The overall effect of changing numtype1 is not to change the order of the results, but the overall scaling.  So if the caller is not concerned about the correct statistical significance of the p-value, it may be more useful to ignore numtype1 and use maxresults to control which results are used.\n",
    "\n",
    "Let's see how changing numtype1 changes the results.  Here's the top 10 genes enriched for Fanconi Anemia."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HGNC:28748</td>\n",
       "      <td>SLX1B</td>\n",
       "      <td>0.000342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HGNC:20922</td>\n",
       "      <td>SLX1A</td>\n",
       "      <td>0.000342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HGNC:28623</td>\n",
       "      <td>FANCD2OS</td>\n",
       "      <td>0.000342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HGNC:10071</td>\n",
       "      <td>RNF8</td>\n",
       "      <td>0.000683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HGNC:30298</td>\n",
       "      <td>UIMC1</td>\n",
       "      <td>0.000683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HGNC:13620</td>\n",
       "      <td>FBH1</td>\n",
       "      <td>0.000683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HGNC:17660</td>\n",
       "      <td>DCLRE1A</td>\n",
       "      <td>0.000683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HGNC:29814</td>\n",
       "      <td>MUS81</td>\n",
       "      <td>0.000797</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HGNC:33499</td>\n",
       "      <td>ATRIP</td>\n",
       "      <td>0.000911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HGNC:20994</td>\n",
       "      <td>ZSCAN2</td>\n",
       "      <td>0.000911</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id      name         p\n",
       "0  HGNC:28748     SLX1B  0.000342\n",
       "1  HGNC:20922     SLX1A  0.000342\n",
       "2  HGNC:28623  FANCD2OS  0.000342\n",
       "3  HGNC:10071      RNF8  0.000683\n",
       "4  HGNC:30298     UIMC1  0.000683\n",
       "5  HGNC:13620      FBH1  0.000683\n",
       "6  HGNC:17660   DCLRE1A  0.000683\n",
       "7  HGNC:29814     MUS81  0.000797\n",
       "8  HGNC:33499     ATRIP  0.000911\n",
       "9  HGNC:20994    ZSCAN2  0.000911"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fa='MONDO:0019391'\n",
    "values = enrichment('disease',[fa],'gene',threshhold=0.2,maxresults=10,rebuild=True)\n",
    "pd.DataFrame(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In this case, the number of diseases is being estimated from the database.  We could enforce a particular number, say 20000, like this, and it will affect the p-values, but not the order of the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HGNC:28748</td>\n",
       "      <td>SLX1B</td>\n",
       "      <td>0.00015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HGNC:20922</td>\n",
       "      <td>SLX1A</td>\n",
       "      <td>0.00015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HGNC:28623</td>\n",
       "      <td>FANCD2OS</td>\n",
       "      <td>0.00015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HGNC:10071</td>\n",
       "      <td>RNF8</td>\n",
       "      <td>0.00030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HGNC:30298</td>\n",
       "      <td>UIMC1</td>\n",
       "      <td>0.00030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HGNC:13620</td>\n",
       "      <td>FBH1</td>\n",
       "      <td>0.00030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HGNC:17660</td>\n",
       "      <td>DCLRE1A</td>\n",
       "      <td>0.00030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HGNC:29814</td>\n",
       "      <td>MUS81</td>\n",
       "      <td>0.00035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HGNC:33499</td>\n",
       "      <td>ATRIP</td>\n",
       "      <td>0.00040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HGNC:20994</td>\n",
       "      <td>ZSCAN2</td>\n",
       "      <td>0.00040</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id      name        p\n",
       "0  HGNC:28748     SLX1B  0.00015\n",
       "1  HGNC:20922     SLX1A  0.00015\n",
       "2  HGNC:28623  FANCD2OS  0.00015\n",
       "3  HGNC:10071      RNF8  0.00030\n",
       "4  HGNC:30298     UIMC1  0.00030\n",
       "5  HGNC:13620      FBH1  0.00030\n",
       "6  HGNC:17660   DCLRE1A  0.00030\n",
       "7  HGNC:29814     MUS81  0.00035\n",
       "8  HGNC:33499     ATRIP  0.00040\n",
       "9  HGNC:20994    ZSCAN2  0.00040"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values = enrichment('disease',[fa],'gene',threshhold=0.2,maxresults=10,rebuild=True,numtype1=20000)\n",
    "pd.DataFrame(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Sharpening enrichment with ontological descendants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The enrichment service is capable of extending the input identifiers by including their descendants.  For instance, if the `include_descendants` parameter is True and a single fanconi anemia term is specified as the input, the service will automatically consult MONDO to find all subtypes of diabetes and include them as input in the original calculation.\n",
    "\n",
    "We just looked at the genes enriched for Fanconi Anemia alone.   If we set include_descendants, we will be enriching on FA, and also on anything that descends from FA in the MONDO ontology."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HGNC:3583</td>\n",
       "      <td>FANCB</td>\n",
       "      <td>0.000003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HGNC:3584</td>\n",
       "      <td>FANCC</td>\n",
       "      <td>0.000007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HGNC:3586</td>\n",
       "      <td>FANCE</td>\n",
       "      <td>0.000010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HGNC:3588</td>\n",
       "      <td>FANCG</td>\n",
       "      <td>0.000010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HGNC:28623</td>\n",
       "      <td>FANCD2OS</td>\n",
       "      <td>0.000015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HGNC:3587</td>\n",
       "      <td>FANCF</td>\n",
       "      <td>0.000017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HGNC:3585</td>\n",
       "      <td>FANCD2</td>\n",
       "      <td>0.000095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HGNC:6764</td>\n",
       "      <td>MAD2L2</td>\n",
       "      <td>0.000321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HGNC:23845</td>\n",
       "      <td>SLX4</td>\n",
       "      <td>0.000321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HGNC:9820</td>\n",
       "      <td>RAD51C</td>\n",
       "      <td>0.000441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>HGNC:1101</td>\n",
       "      <td>BRCA2</td>\n",
       "      <td>0.000481</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>HGNC:12829</td>\n",
       "      <td>XRCC2</td>\n",
       "      <td>0.000737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>HGNC:26144</td>\n",
       "      <td>PALB2</td>\n",
       "      <td>0.001319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>HGNC:20748</td>\n",
       "      <td>FANCL</td>\n",
       "      <td>0.001432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>HGNC:25009</td>\n",
       "      <td>UBE2T</td>\n",
       "      <td>0.004015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>HGNC:9817</td>\n",
       "      <td>RAD51</td>\n",
       "      <td>0.004598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>HGNC:3582</td>\n",
       "      <td>FANCA</td>\n",
       "      <td>0.004598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>HGNC:20473</td>\n",
       "      <td>BRIP1</td>\n",
       "      <td>0.004801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>HGNC:3436</td>\n",
       "      <td>ERCC4</td>\n",
       "      <td>0.004801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>HGNC:25568</td>\n",
       "      <td>FANCI</td>\n",
       "      <td>0.006331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>HGNC:28748</td>\n",
       "      <td>SLX1B</td>\n",
       "      <td>0.006816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>HGNC:20922</td>\n",
       "      <td>SLX1A</td>\n",
       "      <td>0.006816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>HGNC:1100</td>\n",
       "      <td>BRCA1</td>\n",
       "      <td>0.007543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>HGNC:10071</td>\n",
       "      <td>RNF8</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>HGNC:30298</td>\n",
       "      <td>UIMC1</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>HGNC:13620</td>\n",
       "      <td>FBH1</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>HGNC:17660</td>\n",
       "      <td>DCLRE1A</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>HGNC:29814</td>\n",
       "      <td>MUS81</td>\n",
       "      <td>0.015835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>HGNC:33499</td>\n",
       "      <td>ATRIP</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>HGNC:20994</td>\n",
       "      <td>ZSCAN2</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>HGNC:18536</td>\n",
       "      <td>HELQ</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>HGNC:11422</td>\n",
       "      <td>CENPX</td>\n",
       "      <td>0.020315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>HGNC:2737</td>\n",
       "      <td>DDX12P</td>\n",
       "      <td>0.022548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>HGNC:37101</td>\n",
       "      <td>DDX11L8</td>\n",
       "      <td>0.022548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>HGNC:25752</td>\n",
       "      <td>ATAD5</td>\n",
       "      <td>0.024776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>HGNC:26171</td>\n",
       "      <td>FAAP100</td>\n",
       "      <td>0.027000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>HGNC:17641</td>\n",
       "      <td>DCLRE1B</td>\n",
       "      <td>0.029218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>HGNC:9823</td>\n",
       "      <td>RAD51D</td>\n",
       "      <td>0.029218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>HGNC:14060</td>\n",
       "      <td>REV1</td>\n",
       "      <td>0.031432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>HGNC:29170</td>\n",
       "      <td>FAN1</td>\n",
       "      <td>0.031432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>HGNC:23168</td>\n",
       "      <td>FANCM</td>\n",
       "      <td>0.035844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>HGNC:18870</td>\n",
       "      <td>POLN</td>\n",
       "      <td>0.040238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>HGNC:23163</td>\n",
       "      <td>CENPS</td>\n",
       "      <td>0.042428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>HGNC:25539</td>\n",
       "      <td>RFWD3</td>\n",
       "      <td>0.046793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>HGNC:25764</td>\n",
       "      <td>RMI1</td>\n",
       "      <td>0.048969</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            id      name         p\n",
       "0    HGNC:3583     FANCB  0.000003\n",
       "1    HGNC:3584     FANCC  0.000007\n",
       "2    HGNC:3586     FANCE  0.000010\n",
       "3    HGNC:3588     FANCG  0.000010\n",
       "4   HGNC:28623  FANCD2OS  0.000015\n",
       "5    HGNC:3587     FANCF  0.000017\n",
       "6    HGNC:3585    FANCD2  0.000095\n",
       "7    HGNC:6764    MAD2L2  0.000321\n",
       "8   HGNC:23845      SLX4  0.000321\n",
       "9    HGNC:9820    RAD51C  0.000441\n",
       "10   HGNC:1101     BRCA2  0.000481\n",
       "11  HGNC:12829     XRCC2  0.000737\n",
       "12  HGNC:26144     PALB2  0.001319\n",
       "13  HGNC:20748     FANCL  0.001432\n",
       "14  HGNC:25009     UBE2T  0.004015\n",
       "15   HGNC:9817     RAD51  0.004598\n",
       "16   HGNC:3582     FANCA  0.004598\n",
       "17  HGNC:20473     BRIP1  0.004801\n",
       "18   HGNC:3436     ERCC4  0.004801\n",
       "19  HGNC:25568     FANCI  0.006331\n",
       "20  HGNC:28748     SLX1B  0.006816\n",
       "21  HGNC:20922     SLX1A  0.006816\n",
       "22   HGNC:1100     BRCA1  0.007543\n",
       "23  HGNC:10071      RNF8  0.013588\n",
       "24  HGNC:30298     UIMC1  0.013588\n",
       "25  HGNC:13620      FBH1  0.013588\n",
       "26  HGNC:17660   DCLRE1A  0.013588\n",
       "27  HGNC:29814     MUS81  0.015835\n",
       "28  HGNC:33499     ATRIP  0.018078\n",
       "29  HGNC:20994    ZSCAN2  0.018078\n",
       "30  HGNC:18536      HELQ  0.018078\n",
       "31  HGNC:11422     CENPX  0.020315\n",
       "32   HGNC:2737    DDX12P  0.022548\n",
       "33  HGNC:37101   DDX11L8  0.022548\n",
       "34  HGNC:25752     ATAD5  0.024776\n",
       "35  HGNC:26171   FAAP100  0.027000\n",
       "36  HGNC:17641   DCLRE1B  0.029218\n",
       "37   HGNC:9823    RAD51D  0.029218\n",
       "38  HGNC:14060      REV1  0.031432\n",
       "39  HGNC:29170      FAN1  0.031432\n",
       "40  HGNC:23168     FANCM  0.035844\n",
       "41  HGNC:18870      POLN  0.040238\n",
       "42  HGNC:23163     CENPS  0.042428\n",
       "43  HGNC:25539     RFWD3  0.046793\n",
       "44  HGNC:25764      RMI1  0.048969"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values = enrichment('disease',[fa],'gene',threshhold=0.05,maxresults=0,rebuild=True,include_descendants=True)\n",
    "pd.DataFrame(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Identifier prefixes and synonymization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The enrichment service will make an effort to cross-interpret different identifiers.  For instance, Fanconi Anemia has an identifer from MONDO, but it also has an identifier from Disease Ontology (DOID:13636).   The results returned from the enrichment service should be independent of the identifier used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return Status: 200\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HGNC:3583</td>\n",
       "      <td>FANCB</td>\n",
       "      <td>0.000003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HGNC:3584</td>\n",
       "      <td>FANCC</td>\n",
       "      <td>0.000007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>HGNC:3586</td>\n",
       "      <td>FANCE</td>\n",
       "      <td>0.000010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>HGNC:3588</td>\n",
       "      <td>FANCG</td>\n",
       "      <td>0.000010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>HGNC:28623</td>\n",
       "      <td>FANCD2OS</td>\n",
       "      <td>0.000015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>HGNC:3587</td>\n",
       "      <td>FANCF</td>\n",
       "      <td>0.000017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>HGNC:3585</td>\n",
       "      <td>FANCD2</td>\n",
       "      <td>0.000095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>HGNC:6764</td>\n",
       "      <td>MAD2L2</td>\n",
       "      <td>0.000321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>HGNC:23845</td>\n",
       "      <td>SLX4</td>\n",
       "      <td>0.000321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>HGNC:9820</td>\n",
       "      <td>RAD51C</td>\n",
       "      <td>0.000441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>HGNC:1101</td>\n",
       "      <td>BRCA2</td>\n",
       "      <td>0.000481</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>HGNC:12829</td>\n",
       "      <td>XRCC2</td>\n",
       "      <td>0.000737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>HGNC:26144</td>\n",
       "      <td>PALB2</td>\n",
       "      <td>0.001319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>HGNC:20748</td>\n",
       "      <td>FANCL</td>\n",
       "      <td>0.001432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>HGNC:25009</td>\n",
       "      <td>UBE2T</td>\n",
       "      <td>0.004015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>HGNC:9817</td>\n",
       "      <td>RAD51</td>\n",
       "      <td>0.004598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>HGNC:3582</td>\n",
       "      <td>FANCA</td>\n",
       "      <td>0.004598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>HGNC:20473</td>\n",
       "      <td>BRIP1</td>\n",
       "      <td>0.004801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>HGNC:3436</td>\n",
       "      <td>ERCC4</td>\n",
       "      <td>0.004801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>HGNC:25568</td>\n",
       "      <td>FANCI</td>\n",
       "      <td>0.006331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>HGNC:28748</td>\n",
       "      <td>SLX1B</td>\n",
       "      <td>0.006816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>HGNC:20922</td>\n",
       "      <td>SLX1A</td>\n",
       "      <td>0.006816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>HGNC:1100</td>\n",
       "      <td>BRCA1</td>\n",
       "      <td>0.007543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>HGNC:10071</td>\n",
       "      <td>RNF8</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>HGNC:30298</td>\n",
       "      <td>UIMC1</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>HGNC:13620</td>\n",
       "      <td>FBH1</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>HGNC:17660</td>\n",
       "      <td>DCLRE1A</td>\n",
       "      <td>0.013588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>HGNC:29814</td>\n",
       "      <td>MUS81</td>\n",
       "      <td>0.015835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>HGNC:33499</td>\n",
       "      <td>ATRIP</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>HGNC:20994</td>\n",
       "      <td>ZSCAN2</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>HGNC:18536</td>\n",
       "      <td>HELQ</td>\n",
       "      <td>0.018078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>HGNC:11422</td>\n",
       "      <td>CENPX</td>\n",
       "      <td>0.020315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>HGNC:2737</td>\n",
       "      <td>DDX12P</td>\n",
       "      <td>0.022548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>HGNC:37101</td>\n",
       "      <td>DDX11L8</td>\n",
       "      <td>0.022548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>HGNC:25752</td>\n",
       "      <td>ATAD5</td>\n",
       "      <td>0.024776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>HGNC:26171</td>\n",
       "      <td>FAAP100</td>\n",
       "      <td>0.027000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>HGNC:17641</td>\n",
       "      <td>DCLRE1B</td>\n",
       "      <td>0.029218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>HGNC:9823</td>\n",
       "      <td>RAD51D</td>\n",
       "      <td>0.029218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>HGNC:14060</td>\n",
       "      <td>REV1</td>\n",
       "      <td>0.031432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>HGNC:29170</td>\n",
       "      <td>FAN1</td>\n",
       "      <td>0.031432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>HGNC:23168</td>\n",
       "      <td>FANCM</td>\n",
       "      <td>0.035844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>HGNC:18870</td>\n",
       "      <td>POLN</td>\n",
       "      <td>0.040238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>HGNC:23163</td>\n",
       "      <td>CENPS</td>\n",
       "      <td>0.042428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>HGNC:25539</td>\n",
       "      <td>RFWD3</td>\n",
       "      <td>0.046793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>HGNC:25764</td>\n",
       "      <td>RMI1</td>\n",
       "      <td>0.048969</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            id      name         p\n",
       "0    HGNC:3583     FANCB  0.000003\n",
       "1    HGNC:3584     FANCC  0.000007\n",
       "2    HGNC:3586     FANCE  0.000010\n",
       "3    HGNC:3588     FANCG  0.000010\n",
       "4   HGNC:28623  FANCD2OS  0.000015\n",
       "5    HGNC:3587     FANCF  0.000017\n",
       "6    HGNC:3585    FANCD2  0.000095\n",
       "7    HGNC:6764    MAD2L2  0.000321\n",
       "8   HGNC:23845      SLX4  0.000321\n",
       "9    HGNC:9820    RAD51C  0.000441\n",
       "10   HGNC:1101     BRCA2  0.000481\n",
       "11  HGNC:12829     XRCC2  0.000737\n",
       "12  HGNC:26144     PALB2  0.001319\n",
       "13  HGNC:20748     FANCL  0.001432\n",
       "14  HGNC:25009     UBE2T  0.004015\n",
       "15   HGNC:9817     RAD51  0.004598\n",
       "16   HGNC:3582     FANCA  0.004598\n",
       "17  HGNC:20473     BRIP1  0.004801\n",
       "18   HGNC:3436     ERCC4  0.004801\n",
       "19  HGNC:25568     FANCI  0.006331\n",
       "20  HGNC:28748     SLX1B  0.006816\n",
       "21  HGNC:20922     SLX1A  0.006816\n",
       "22   HGNC:1100     BRCA1  0.007543\n",
       "23  HGNC:10071      RNF8  0.013588\n",
       "24  HGNC:30298     UIMC1  0.013588\n",
       "25  HGNC:13620      FBH1  0.013588\n",
       "26  HGNC:17660   DCLRE1A  0.013588\n",
       "27  HGNC:29814     MUS81  0.015835\n",
       "28  HGNC:33499     ATRIP  0.018078\n",
       "29  HGNC:20994    ZSCAN2  0.018078\n",
       "30  HGNC:18536      HELQ  0.018078\n",
       "31  HGNC:11422     CENPX  0.020315\n",
       "32   HGNC:2737    DDX12P  0.022548\n",
       "33  HGNC:37101   DDX11L8  0.022548\n",
       "34  HGNC:25752     ATAD5  0.024776\n",
       "35  HGNC:26171   FAAP100  0.027000\n",
       "36  HGNC:17641   DCLRE1B  0.029218\n",
       "37   HGNC:9823    RAD51D  0.029218\n",
       "38  HGNC:14060      REV1  0.031432\n",
       "39  HGNC:29170      FAN1  0.031432\n",
       "40  HGNC:23168     FANCM  0.035844\n",
       "41  HGNC:18870      POLN  0.040238\n",
       "42  HGNC:23163     CENPS  0.042428\n",
       "43  HGNC:25539     RFWD3  0.046793\n",
       "44  HGNC:25764      RMI1  0.048969"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values = enrichment('disease',['DOID:13636'],'gene',threshhold=0.05,maxresults=0,include_descendants=True)\n",
    "pd.DataFrame(values)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}