tutorials/vector_qoi_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "subject-wagon",
   "metadata": {},
   "source": [
    "# EasyVVUQ - Vector Quantities of Interest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "protective-intervention",
   "metadata": {},
   "source": [
    "**Author**: Vytautas Jancauskas, LRZ (jancauskas@lrz.de)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "elect-tract",
   "metadata": {},
   "source": [
    "It is often the case that simulation outputs are vector valued and represent changes over time in whatever phenomenon that is simulated. Here we examine how to analyse these cases with EasyVVUQ. As an example we use the following model from epidemiology - [SIR](https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology#The_SIR_model). It compartmentalises the population into the following groups: ($S$)usceptible, ($I$)nfected and ($R$)emoved. There are four input parameters to out model. They are initial number of susceptible people $S_0$, initial number of infected people $I_0$, transmission rate $\\beta$ and recovery rate $\\gamma$. The system is governed by three differential equations below. The number of suspectible people is reduced by them getting infected (and thus moving to $I$) at rate specified by $\\beta$. The number of infected people increases similarly. Finally the number of recovered people increases depending on the recovery rate and the number of people currently infected."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "neither-sierra",
   "metadata": {},
   "source": [
    "$${\\frac  {dS}{dt}}=-{\\frac  {\\beta SI}{N}}$$\n",
    "\n",
    "$${\\displaystyle {\\frac {dI}{dt}}={\\frac {\\beta SI}{N}}-\\gamma I}$$\n",
    "\n",
    "$${\\displaystyle {\\frac {dR}{dt}}=\\gamma I}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "scientific-shareware",
   "metadata": {},
   "source": [
    "The simulation is run as shown below, where ```input_file``` is a JSON file of the following format: ```{\"outfile\": \"$outfile\", \"S0\": $S0, \"I0\": $I0, \"beta\": $beta, \"gamma\": $gamma, \"iterations\": $iterations}```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "german-upper",
   "metadata": {},
   "source": [
    "```sir <input_file>```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "religious-problem",
   "metadata": {},
   "source": [
    "Lets try and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "pressed-recipe",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:25.930406Z",
     "start_time": "2021-06-09T08:57:25.534388Z"
    }
   },
   "outputs": [],
   "source": [
    "!echo '{\"outfile\": \"output.csv\", \"S0\": 997, \"I0\": 3, \"beta\": 0.2, \"gamma\": 0.04, \"iterations\": 100}' > input.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bearing-force",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:27.202480Z",
     "start_time": "2021-06-09T08:57:25.931781Z"
    }
   },
   "outputs": [],
   "source": [
    "!./sir input.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "organizational-framework",
   "metadata": {},
   "source": [
    "The simulation code outputs a CSV file with a specified name. We can try opening it up and printing the first 10 lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "according-timer",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:27.563489Z",
     "start_time": "2021-06-09T08:57:27.204218Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "S,I,R,r0,t\r\n",
      "996.4018,3.47784108,0.1391136432,4982.009,1\r\n",
      "995.7087475755611,4.0312978050411745,0.300365555401647,4978.543737877805,2\r\n",
      "994.905980318425,4.672165937353743,0.4872521928957967,4974.5299015921255,3\r\n",
      "993.9763679472093,5.4140230672294996,0.7038131155849767,4969.8818397360465,4\r\n",
      "992.9001871310173,6.27247777694467,0.9547122266627635,4964.500935655087,5\r\n",
      "991.654756898651,7.2654467105291545,1.2453300950839297,4958.273784493255,6\r\n",
      "990.2140324288114,8.413460157063296,1.5818685013664615,4951.070162144058,7\r\n",
      "988.548155936764,9.739995672478827,1.9714683282656145,4942.740779683821,8\r\n",
      "986.6229648025937,11.27183768247915,2.4223418355647803,4933.11482401297,9\r\n"
     ]
    }
   ],
   "source": [
    "!head output.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "japanese-canvas",
   "metadata": {},
   "source": [
    "Finally lets plot the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "moral-booking",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:28.350065Z",
     "start_time": "2021-06-09T08:57:27.565838Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "df = pd.read_csv('output.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "introductory-cambodia",
   "metadata": {},
   "source": [
    "As we can see it shows the number of infected people increasing at first and then decreasing as the number of people who have recovered rises. Number of susceptible and recovered people also changes in a way that makes intuitive sense."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "sudden-boring",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:28.435209Z",
     "start_time": "2021-06-09T08:57:28.350940Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1122f0970>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plt.plot(df['S'], label='S')\n",
    "plt.plot(df['I'], label='I')\n",
    "plt.plot(df['R'], label='R')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "collectible-bread",
   "metadata": {},
   "source": [
    "So let us see what EasyVVUQ can tell us about the distribution of the number of infected people over time. We will not go into the details of the EasyVVUQ API this time. Only where it differs from the [basic concepts](./basic_tutorial.ipynb) tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "flexible-password",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.588513Z",
     "start_time": "2021-06-09T08:57:28.437128Z"
    }
   },
   "outputs": [],
   "source": [
    "import easyvvuq as uq\n",
    "import chaospy as cp\n",
    "import os\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "victorian-dallas",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.592929Z",
     "start_time": "2021-06-09T08:57:30.590057Z"
    }
   },
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"S0\": {\"type\": \"float\", \"default\": 997}, \n",
    "    \"I0\": {\"type\": \"float\", \"default\": 3}, \n",
    "    \"beta\": {\"type\": \"float\", \"default\": 0.2}, \n",
    "    \"gamma\": {\"type\": \"float\", \"default\": 0.04, \"min\": 0.0, \"max\": 1.0},\n",
    "    \"iterations\": {\"type\": \"integer\", \"default\": 100},\n",
    "    \"outfile\": {\"type\": \"string\", \"default\": \"output.csv\"}\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "meaning-berry",
   "metadata": {},
   "source": [
    "Encoder is the same as in the basic concepts tutorial, essentially. However, since the simulation outputs a CSV file with the evolution of output variables over time we want the ```SimpleCSV``` decoder. The arguments to it should be self explanatory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "owned-preliminary",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.598048Z",
     "start_time": "2021-06-09T08:57:30.595407Z"
    }
   },
   "outputs": [],
   "source": [
    "encoder = uq.encoders.GenericEncoder(template_fname='sir.template', delimiter='$', target_filename='input.json')\n",
    "decoder = uq.decoders.SimpleCSV(target_filename='output.csv', output_columns=['I'])\n",
    "actions = uq.actions.local_execute(encoder, os.path.abspath('sir') + ' input.json', decoder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "neutral-marina",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.630175Z",
     "start_time": "2021-06-09T08:57:30.599889Z"
    }
   },
   "outputs": [],
   "source": [
    "campaign = uq.Campaign(name='sir', params=params, actions=actions)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "searching-interpretation",
   "metadata": {},
   "source": [
    "We assume that the infection rate $\\beta$ is uniformly distributed between 0.15 and 0.25 and the recovery rate $\\gamma$ is normally distributed with mean 0.04 and small variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "native-order",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.633520Z",
     "start_time": "2021-06-09T08:57:30.630961Z"
    }
   },
   "outputs": [],
   "source": [
    "vary = {\n",
    "    \"beta\": cp.Uniform(0.15, 0.25),\n",
    "    \"gamma\": cp.Normal(0.04, 0.01),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "engaging-spoke",
   "metadata": {},
   "source": [
    "For this tutorial we will use Polynomial Chaos Expansion method. However, both [SCSampler](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.sampling.html#module-easyvvuq.sampling.stochastic_collocation) and [QMCSampler](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.sampling.html#module-easyvvuq.sampling.qmc) would work as well and might be preferable depending on the case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cosmetic-stake",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.761510Z",
     "start_time": "2021-06-09T08:57:30.634548Z"
    }
   },
   "outputs": [],
   "source": [
    "campaign.set_sampler(uq.sampling.PCESampler(vary=vary, polynomial_order=5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "governing-warrant",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:30.785220Z",
     "start_time": "2021-06-09T08:57:30.762311Z"
    }
   },
   "outputs": [],
   "source": [
    "execution = campaign.execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "junior-proxy",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:31.042069Z",
     "start_time": "2021-06-09T08:57:31.036748Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ready': 28, 'active': 8, 'finished': 0, 'failed': 0}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "execution.progress()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "regulated-semester",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:35.750226Z",
     "start_time": "2021-06-09T08:57:31.043481Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 36/36 [00:07<00:00,  4.72it/s]\n"
     ]
    }
   ],
   "source": [
    "execution.collate(progress_bar=True)\n",
    "result = campaign.analyse(qoi_cols=['I'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "descending-resident",
   "metadata": {},
   "source": [
    "We can now see the results of the analysis. One thing to try would be to plot the first order sobol indices over time. This shows us how much influence the two parameters (```beta``` and ```gamma```) have over the number of people infected over time ```t```. The ```higher orders``` line is meant to represent the influence of the interactions between the input variables. However it is negligible in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "pressing-sister",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:35.908776Z",
     "start_time": "2021-06-09T08:57:35.765529Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='t', ylabel='First Order Sobol Index'>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.plot_sobols_first('I', xlabel='t')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exposed-paraguay",
   "metadata": {},
   "source": [
    "Finally we can try and visualize certain aspects of the distribution for our quantity of interest (number of infected people) over time. In this plot we will plot the mean value, standard deviation and 0.01 and 0.99 quantiles. This is with respect to the input variable distributions we have specified in the ```vary``` dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "corrected-titanium",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.072288Z",
     "start_time": "2021-06-09T08:57:35.917531Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='t', ylabel='I'>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.plot_moments('I', xlabel='t')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "overhead-wright",
   "metadata": {},
   "source": [
    "If you want to access all of this data for processing you can use the following methods. Due to large output I have not evaluated these cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "destroyed-darkness",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.076580Z",
     "start_time": "2021-06-09T08:57:36.073217Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'beta': array([0.89204991, 0.89185266, 0.89149001, 0.89096097, 0.89026466,\n",
       "        0.88940042, 0.88836788, 0.88716713, 0.88579887, 0.88426465,\n",
       "        0.88256715, 0.88071051, 0.87870074, 0.87654612, 0.87425765,\n",
       "        0.87184944, 0.86933898, 0.86674715, 0.86409788, 0.8614171 ,\n",
       "        0.85873111, 0.85606382, 0.85343316, 0.85084649, 0.84829549,\n",
       "        0.84575091, 0.84315795, 0.84043276, 0.83746062, 0.83409584,\n",
       "        0.83016322, 0.82546019, 0.81975914, 0.81280885, 0.80433458,\n",
       "        0.7940368 , 0.78158862, 0.76663314, 0.74878204, 0.72761773,\n",
       "        0.7027023 , 0.67359755, 0.63990145, 0.60130728, 0.55769073,\n",
       "        0.50922621, 0.45652411, 0.40076392, 0.34377714, 0.28801728,\n",
       "        0.23636068, 0.19172748, 0.15659244, 0.13253095, 0.11995903,\n",
       "        0.11815331, 0.12551813, 0.13997798, 0.15935905, 0.18167041,\n",
       "        0.20525921, 0.22885867, 0.25156611, 0.272786  , 0.29216284,\n",
       "        0.30951812, 0.32479749, 0.33802971, 0.34929615, 0.35870914,\n",
       "        0.366397  , 0.37249406, 0.37713427, 0.38044717, 0.38255565,\n",
       "        0.38357479, 0.38361138, 0.38276395, 0.38112306, 0.37877176,\n",
       "        0.37578605, 0.37223548, 0.36818363, 0.36368866, 0.35880377,\n",
       "        0.3535776 , 0.34805468, 0.34227574, 0.33627808, 0.33009582,\n",
       "        0.32376017, 0.31729967, 0.3107404 , 0.30410619, 0.29741875,\n",
       "        0.29069789, 0.28396159, 0.27722619, 0.27050651, 0.26381593]),\n",
       " 'gamma': array([0.10795009, 0.108081  , 0.1082452 , 0.10844475, 0.10868198,\n",
       "        0.10895947, 0.10928011, 0.10964712, 0.11006411, 0.11053511,\n",
       "        0.1110646 , 0.11165763, 0.11231979, 0.11305739, 0.11387752,\n",
       "        0.11478817, 0.11579846, 0.11691886, 0.11816155, 0.11954088,\n",
       "        0.12107397, 0.12278152, 0.12468883, 0.12682701, 0.12923442,\n",
       "        0.13195833, 0.13505678, 0.13860045, 0.14267476, 0.14738191,\n",
       "        0.15284307, 0.15920061, 0.16662051, 0.17529499, 0.18544537,\n",
       "        0.19732504, 0.21122225, 0.22746203, 0.24640622, 0.26844958,\n",
       "        0.29400941, 0.32350472, 0.35731988, 0.39574695, 0.43890161,\n",
       "        0.48661152, 0.53828507, 0.59278447, 0.64834747, 0.70261752,\n",
       "        0.75283613, 0.79620725, 0.8303664 , 0.85381512, 0.86616865,\n",
       "        0.86813485, 0.86125663, 0.84753487, 0.8290625 , 0.80775511,\n",
       "        0.78520285, 0.76262477, 0.74089032, 0.72057421, 0.70202081,\n",
       "        0.68540466, 0.67078107, 0.65812541, 0.64736233, 0.63838646,\n",
       "        0.63107668, 0.62530575, 0.6209464 , 0.61787514, 0.61597451,\n",
       "        0.6151342 , 0.61525151, 0.61623137, 0.61798614, 0.62043516,\n",
       "        0.62350436, 0.62712574, 0.63123693, 0.6357807 , 0.64070456,\n",
       "        0.64596039, 0.65150402, 0.65729496, 0.66329612, 0.66947349,\n",
       "        0.67579593, 0.68223496, 0.68876457, 0.69536099, 0.70200259,\n",
       "        0.70866968, 0.71534439, 0.72201054, 0.7286535 , 0.73526013])}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.sobols_first('I')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "contrary-wound",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.084115Z",
     "start_time": "2021-06-09T08:57:36.077428Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  3.4778336 ,   4.03404206,   4.68171534,   5.43612205,\n",
       "         6.31505352,   7.3392123 ,   8.5326462 ,   9.92322633,\n",
       "        11.54316403,  13.42955585,  15.62493745,  18.17781631,\n",
       "        21.14313804,  24.58262204,  28.56487862,  33.16519322,\n",
       "        38.46483541,  44.54972656,  51.50828773,  59.42829956,\n",
       "        68.39265292,  78.47396641,  89.72820311, 102.18763148,\n",
       "       115.85371859, 130.69076976, 146.62126765, 163.52384337,\n",
       "       181.23458606, 199.55197143, 218.24513987, 237.06470706,\n",
       "       255.75488712, 274.06555177, 291.76296652, 308.63828094,\n",
       "       324.51329838, 339.24349061, 352.71856713, 364.86111453,\n",
       "       375.62389166, 384.98633302, 392.95071792, 399.53834415,\n",
       "       404.78592878, 408.74236162, 411.46586385, 413.02155449,\n",
       "       413.47939812, 412.91249216, 411.39564756, 409.00421782,\n",
       "       405.81313627, 401.89612761, 397.32506563, 392.16945484,\n",
       "       386.49601832, 380.36837781, 373.84681454, 366.98810159,\n",
       "       359.84539945, 352.46820727, 344.90236319, 337.19008678,\n",
       "       329.37005755, 321.47752329, 313.54443255, 305.5995859 ,\n",
       "       297.66880097, 289.77508687, 281.93882404, 274.17794602,\n",
       "       266.50812042, 258.94292658, 251.49402806, 244.17133841,\n",
       "       236.98317926, 229.93642984, 223.0366674 , 216.28829859,\n",
       "       209.69468132, 203.25823762, 196.98055751, 190.86249422,\n",
       "       184.9042513 , 179.10546191, 173.46526081, 167.98234963,\n",
       "       162.6550557 , 157.4813851 , 152.45907035, 147.58561312,\n",
       "       142.85832243, 138.27434872, 133.83071424, 129.52433996,\n",
       "       125.35206945, 121.31068996, 117.396951  , 113.60758069])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe('I', 'mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "biblical-saint",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.092045Z",
     "start_time": "2021-06-09T08:57:36.085146Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9.13081930e-02, 2.11560354e-01, 3.67777061e-01, 5.68492124e-01,\n",
       "       8.24044420e-01, 1.14691458e+00, 1.55210823e+00, 2.05758473e+00,\n",
       "       2.68472560e+00, 3.45882970e+00, 4.40961189e+00, 5.57166667e+00,\n",
       "       6.98483857e+00, 8.69441463e+00, 1.07510228e+01, 1.32100846e+01,\n",
       "       1.61306351e+01, 1.95733000e+01, 2.35972171e+01, 2.82557340e+01,\n",
       "       3.35908224e+01, 3.96263450e+01, 4.63605930e+01, 5.37588577e+01,\n",
       "       6.17471385e+01, 7.02083075e+01, 7.89820280e+01, 8.78693628e+01,\n",
       "       9.66423075e+01, 1.05057574e+02, 1.12873064e+02, 1.19864881e+02,\n",
       "       1.25842595e+02, 1.30660906e+02, 1.34226572e+02, 1.36500409e+02,\n",
       "       1.37494923e+02, 1.37268669e+02, 1.35918592e+02, 1.33571532e+02,\n",
       "       1.30375825e+02, 1.26493586e+02, 1.22094009e+02, 1.17347705e+02,\n",
       "       1.12421962e+02, 1.07476667e+02, 1.02660572e+02, 9.81076004e+01,\n",
       "       9.39330242e+01, 9.02295425e+01, 8.70636263e+01, 8.44728002e+01,\n",
       "       8.24646905e+01, 8.10185006e+01, 8.00890700e+01, 7.96129967e+01,\n",
       "       7.95157804e+01, 7.97187921e+01, 8.01451251e+01, 8.07238318e+01,\n",
       "       8.13924958e+01, 8.20983823e+01, 8.27985372e+01, 8.34592011e+01,\n",
       "       8.40548441e+01, 8.45670392e+01, 8.49833179e+01, 8.52960937e+01,\n",
       "       8.55016939e+01, 8.55995188e+01, 8.55913271e+01, 8.54806402e+01,\n",
       "       8.52722529e+01, 8.49718369e+01, 8.45856236e+01, 8.41201536e+01,\n",
       "       8.35820817e+01, 8.29780275e+01, 8.23144641e+01, 8.15976363e+01,\n",
       "       8.08335031e+01, 8.00277000e+01, 7.91855163e+01, 7.83118844e+01,\n",
       "       7.74113774e+01, 7.64882151e+01, 7.55462733e+01, 7.45890979e+01,\n",
       "       7.36199211e+01, 7.26416791e+01, 7.16570304e+01, 7.06683749e+01,\n",
       "       6.96778727e+01, 6.86874620e+01, 6.76988771e+01, 6.67136650e+01,\n",
       "       6.57332016e+01, 6.47587064e+01, 6.37912566e+01, 6.28318001e+01])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe('I', 'std')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "demanding-quick",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.100836Z",
     "start_time": "2021-06-09T08:57:36.093842Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  3.29852547,   3.62647216,   3.98668024,   4.38225166,\n",
       "         4.81657168,   5.29333157,   5.81655257,   6.39061079,\n",
       "         7.02026301,   7.71067311,   8.46743864,   9.29661703,\n",
       "        10.20475056,  11.19888908,  12.28660919,  13.47602849,\n",
       "        14.77581412,  16.1951858 ,  17.74391638,  19.43233693,\n",
       "        21.27135854,  23.27281793,  25.45579714,  27.81784671,\n",
       "        30.38158209,  33.16280847,  36.17864098,  39.44701466,\n",
       "        42.98560529,  46.81014475,  50.91298283,  55.33142179,\n",
       "        60.05388907,  65.0750717 ,  70.40555395,  75.96874042,\n",
       "        81.88559425,  88.17559439,  94.76253446, 101.7906746 ,\n",
       "       109.13806097, 116.78335595, 124.53367842, 132.57163759,\n",
       "       140.93801848, 149.3901359 , 156.94171257, 165.42669799,\n",
       "       173.68721157, 182.3618398 , 190.73479682, 198.98726743,\n",
       "       206.78659683, 214.04149374, 220.33043481, 226.35521018,\n",
       "       228.79596829, 226.72347854, 221.53235772, 215.16015523,\n",
       "       207.36520066, 198.98675867, 190.26158561, 181.44579311,\n",
       "       172.83328595, 164.59885256, 156.70183258, 149.13567726,\n",
       "       141.8927622 , 134.96464222, 128.3422621 , 122.01613044,\n",
       "       115.97646298, 110.20352917, 104.68933692,  99.47634281,\n",
       "        94.48254206,  89.77077696,  85.28690008,  80.94645478,\n",
       "        76.78996234,  72.81996272,  68.95827451,  65.4722659 ,\n",
       "        62.03300167,  58.71085406,  55.55787107,  52.63004237,\n",
       "        49.92122326,  47.34740766,  44.90213795,  42.57922957,\n",
       "        40.34962849,  38.23677543,  36.22652996,  34.2786497 ,\n",
       "        32.43927534,  30.72164407,  29.10293078,  27.56140991])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe('I', '1%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "distinguished-greene",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-06-09T08:57:36.165232Z",
     "start_time": "2021-06-09T08:57:36.102023Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  3.65473544,   4.45159463,   5.42105695,   6.59995928,\n",
       "         8.03273578,   9.77237069,  11.88245812,  14.43937619,\n",
       "        17.53357297,  21.2718391 ,  25.77933367,  31.20140399,\n",
       "        37.70491606,  45.47868804,  54.73247024,  65.69375369,\n",
       "        78.60154364,  93.73168347, 111.27448377, 131.6344494 ,\n",
       "       154.7857425 , 180.86403334, 209.58681046, 241.43954559,\n",
       "       274.88582043, 309.82691674, 345.72971088, 382.13983461,\n",
       "       418.49281379, 453.56094937, 489.24298587, 517.06485735,\n",
       "       547.05623467, 568.58269584, 589.97745129, 605.24056993,\n",
       "       615.80919014, 624.13853541, 632.23797939, 637.78993505,\n",
       "       643.17971248, 646.1495388 , 647.29732844, 647.22908542,\n",
       "       644.68533053, 640.67413635, 640.04781522, 638.15402871,\n",
       "       636.31941958, 633.30742371, 629.6295947 , 623.59809917,\n",
       "       617.39323537, 612.19873087, 610.48310439, 606.38344966,\n",
       "       602.70792655, 596.66956347, 591.12890481, 586.73621   ,\n",
       "       582.17974134, 575.14420168, 568.66783872, 562.49386021,\n",
       "       557.14404893, 551.10398734, 545.40509088, 538.20147774,\n",
       "       532.15271447, 525.78873229, 517.77018519, 509.80511032,\n",
       "       502.58155003, 495.34759641, 487.36912554, 479.82915848,\n",
       "       471.76860154, 463.73045288, 456.12268072, 448.80372073,\n",
       "       441.58051555, 434.45455213, 427.42694505, 419.80808179,\n",
       "       412.09975428, 405.52141951, 397.96888624, 390.49863826,\n",
       "       383.2239695 , 377.25538331, 371.37972682, 364.39773588,\n",
       "       358.25220422, 351.3360762 , 344.50675675, 338.08728769,\n",
       "       331.76636824, 326.60428765, 321.52509349, 316.52755532])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe('I', '99%')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}