doc/user_guide/evaluating-survival-models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluating Survival Models\n",
    "\n",
    "The most frequently used evaluation metric of survival models is the concordance index (c index, c statistic). It is a measure of rank correlation between predicted risk scores $\\hat{f}$ and observed time points $y$ that is closely related to [Kendall’s τ](https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient). It is defined as the ratio of correctly ordered (concordant) pairs to comparable pairs.\n",
    "Two samples $i$ and $j$ are comparable if the sample with lower observed time $y$ experienced an event, i.e., if $y_j > y_i$ and $\\delta_i = 1$, where $\\delta_i$ is a binary event indicator. A comparable pair $(i, j)$ is concordant if the estimated risk $\\hat{f}$ by a survival model  is higher for subjects with lower survival time, i.e., $\\hat{f}_i > \\hat{f}_j \\land y_j > y_i$, otherwise the pair is discordant.\n",
    "Harrell's estimator of the c index is implemented in [concordance_index_censored](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored).\n",
    "\n",
    "While Harrell's concordance index is easy to interpret and compute, it has some shortcomings:\n",
    "1. it has been shown that it is too optimistic with increasing amount of censoring [[1]](https://dx.doi.org/10.1002/sim.4154),\n",
    "2. it is not a useful measure of performance if a specific time range is of primary interest (e.g. predicting death within 2 years).\n",
    "\n",
    "Since version 0.8, scikit-survival supports an alternative estimator of the concordance\n",
    "index from right-censored survival data, implemented in [concordance_index_ipcw](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw), that addresses the first issue.\n",
    "\n",
    "The second point can be addressed by extending the well known receiver operating characteristic curve (ROC curve) to possibly censored survival times. Given a time point $t$, we can estimate how well a predictive model can distinguishing subjects who will experience an event by time $t$ (sensitivity) from those who will not (specificity). The function [cumulative_dynamic_auc](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.cumulative_dynamic_auc.html#sksurv.metrics.cumulative_dynamic_auc) implements an estimator of the cumulative/dynamic area under the ROC for a given list of time points.\n",
    "\n",
    "The [first part](#Bias-of-Harrell's-Concordance-Index) of this notebook will illustrate the first issue with simulated survival data, while the [second part](#Time-dependent-Area-under-the-ROC) will focus on the time-dependent area under the ROC applied to data from a real study. Finally, [part three](#Time-dependent-Brier-Score) will discuss the time-dependent Brier score, which is an extension of the mean squared\n",
    "error to [right censored data](https://scikit-survival.readthedocs.io/en/latest/user_guide/understanding_predictions.html#what-is-survival-analysis)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn import set_config\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "from sksurv.datasets import load_flchain, load_gbsg2\n",
    "from sksurv.functions import StepFunction\n",
    "from sksurv.linear_model import CoxPHSurvivalAnalysis, CoxnetSurvivalAnalysis\n",
    "from sksurv.metrics import (\n",
    "    concordance_index_censored,\n",
    "    concordance_index_ipcw,\n",
    "    cumulative_dynamic_auc,\n",
    "    integrated_brier_score,\n",
    ")\n",
    "from sksurv.nonparametric import kaplan_meier_estimator\n",
    "from sksurv.preprocessing import OneHotEncoder, encode_categorical\n",
    "from sksurv.util import Surv\n",
    "\n",
    "set_config(display=\"text\")  # displays text representation of estimators\n",
    "plt.rcParams[\"figure.figsize\"] = [7.2, 4.8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bias of Harrell's Concordance Index\n",
    "\n",
    "Harrell's concordance index is known to be biased upwards if the amount of censoring in the test data is high [[1]](https://dx.doi.org/10.1002/sim.4154). [Uno et al](https://dx.doi.org/10.1002/sim.4154) proposed an alternative estimator of the concordance index that behaves better in such situations.\n",
    "In this section, we are going to apply [concordance_index_censored](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored)\n",
    "and [concordance_index_ipcw](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw) to synthetic survival data and compare their results.\n",
    "\n",
    "### Simulation Study\n",
    "\n",
    "We are generating a synthetic biomarker by sampling from a standard normal distribution. For a given hazard ratio, we compute the associated (actual) survival time by drawing from an exponential distribution. The censoring times were generated from a uniform independent distribution $\\textrm{Uniform}(0,\\gamma)$, where we choose $\\gamma$ to produce different amounts of censoring.\n",
    "\n",
    "Since Uno's estimator is based on inverse probability of censoring weighting, we need to estimate the probability of being censored at a given time point. This probability needs to be non-zero for all observed time points. Therefore, we restrict the test data to all samples with observed time lower than the maximum event time $\\tau$. Usually, one would use the `tau` argument of `concordance_index_ipcw` for this, but we apply the selection before to pass identical inputs to `concordance_index_censored` and `concordance_index_ipcw`.\n",
    "The estimates of the concordance index are therefore restricted to the interval $[0, \\tau]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scipy.optimize as opt\n",
    "\n",
    "\n",
    "def generate_marker(n_samples, hazard_ratio, baseline_hazard, rnd):\n",
    "    # create synthetic risk score\n",
    "    X = rnd.randn(n_samples, 1)\n",
    "\n",
    "    # create linear model\n",
    "    hazard_ratio = np.array([hazard_ratio])\n",
    "    logits = np.dot(X, np.log(hazard_ratio))\n",
    "\n",
    "    # draw actual survival times from exponential distribution,\n",
    "    # refer to Bender et al. (2005), https://doi.org/10.1002/sim.2059\n",
    "    u = rnd.uniform(size=n_samples)\n",
    "    time_event = -np.log(u) / (baseline_hazard * np.exp(logits))\n",
    "\n",
    "    # compute the actual concordance in the absence of censoring\n",
    "    X = np.squeeze(X)\n",
    "    actual = concordance_index_censored(np.ones(n_samples, dtype=bool), time_event, X)\n",
    "    return X, time_event, actual[0]\n",
    "\n",
    "\n",
    "def generate_survival_data(n_samples, hazard_ratio, baseline_hazard, percentage_cens, rnd):\n",
    "    X, time_event, actual_c = generate_marker(n_samples, hazard_ratio, baseline_hazard, rnd)\n",
    "\n",
    "    def get_observed_time(x):\n",
    "        rnd_cens = np.random.RandomState(0)\n",
    "        # draw censoring times\n",
    "        time_censor = rnd_cens.uniform(high=x, size=n_samples)\n",
    "        event = time_event < time_censor\n",
    "        time = np.where(event, time_event, time_censor)\n",
    "        return event, time\n",
    "\n",
    "    def censoring_amount(x):\n",
    "        event, _ = get_observed_time(x)\n",
    "        cens = 1.0 - event.sum() / event.shape[0]\n",
    "        return (cens - percentage_cens) ** 2\n",
    "\n",
    "    # search for upper limit to obtain the desired censoring amount\n",
    "    res = opt.minimize_scalar(censoring_amount, method=\"bounded\", bounds=(0, time_event.max()))\n",
    "\n",
    "    # compute observed time\n",
    "    event, time = get_observed_time(res.x)\n",
    "\n",
    "    # upper time limit such that the probability\n",
    "    # of being censored is non-zero for `t > tau`\n",
    "    tau = time[event].max()\n",
    "    y = Surv.from_arrays(event=event, time=time)\n",
    "    mask = time < tau\n",
    "    X_test = X[mask]\n",
    "    y_test = y[mask]\n",
    "\n",
    "    return X_test, y_test, y, actual_c\n",
    "\n",
    "\n",
    "def simulation(n_samples, hazard_ratio, n_repeats=100):\n",
    "    measures = (\n",
    "        \"censoring\",\n",
    "        \"Harrel's C\",\n",
    "        \"Uno's C\",\n",
    "    )\n",
    "    data_mean = {}\n",
    "    data_std = {}\n",
    "    for measure in measures:\n",
    "        data_mean[measure] = []\n",
    "        data_std[measure] = []\n",
    "\n",
    "    rnd = np.random.RandomState(seed=987)\n",
    "    # iterate over different amount of censoring\n",
    "    for cens in (0.1, 0.25, 0.4, 0.5, 0.6, 0.7):\n",
    "        data = {\n",
    "            \"censoring\": [],\n",
    "            \"Harrel's C\": [],\n",
    "            \"Uno's C\": [],\n",
    "        }\n",
    "\n",
    "        # repeaditly perform simulation\n",
    "        for _ in range(n_repeats):\n",
    "            # generate data\n",
    "            X_test, y_test, y_train, actual_c = generate_survival_data(\n",
    "                n_samples, hazard_ratio, baseline_hazard=0.1, percentage_cens=cens, rnd=rnd\n",
    "            )\n",
    "\n",
    "            # estimate c-index\n",
    "            c_harrell = concordance_index_censored(y_test[\"event\"], y_test[\"time\"], X_test)\n",
    "            c_uno = concordance_index_ipcw(y_train, y_test, X_test)\n",
    "\n",
    "            # save results\n",
    "            data[\"censoring\"].append(100.0 - y_test[\"event\"].sum() * 100.0 / y_test.shape[0])\n",
    "            data[\"Harrel's C\"].append(actual_c - c_harrell[0])\n",
    "            data[\"Uno's C\"].append(actual_c - c_uno[0])\n",
    "\n",
    "        # aggregate results\n",
    "        for key, values in data.items():\n",
    "            data_mean[key].append(np.mean(data[key]))\n",
    "            data_std[key].append(np.std(data[key], ddof=1))\n",
    "\n",
    "    data_mean = pd.DataFrame.from_dict(data_mean)\n",
    "    data_std = pd.DataFrame.from_dict(data_std)\n",
    "    return data_mean, data_std\n",
    "\n",
    "\n",
    "def plot_results(data_mean, data_std, **kwargs):\n",
    "    index = pd.Index(data_mean[\"censoring\"].round(3), name=\"mean percentage censoring\")\n",
    "    for df in (data_mean, data_std):\n",
    "        df.drop(\"censoring\", axis=1, inplace=True)\n",
    "        df.index = index\n",
    "\n",
    "    ax = data_mean.plot.bar(yerr=data_std, **kwargs)\n",
    "    ax.set_ylabel(\"Actual C - Estimated C\")\n",
    "    ax.yaxis.grid(True)\n",
    "    ax.axhline(0.0, color=\"gray\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us assume a moderate hazard ratio of 2 and generate a small synthetic dataset of 100 samples from which we estimate the concordance index. We repeat this experiment 200 times and plot mean and standard deviation of the difference between the *actual* (in the absence of censoring) and *estimated* concordance index.\n",
    "\n",
    "Since the hazard ratio remains constant and only the amount of censoring changes, we would want an estimator for which the difference between the actual and estimated c to remain approximately constant across simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdMAAAFLCAYAAACNy2aDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtI0lEQVR4nO3de5wU1Zn/8c9XBPHKGDAjihHWgIrRYJx4A93xsiom3qKumotoLqyJmIhxI675RdxkE2JM1hsbdInRZI2uxiTeSIyyQDTxghiiIl6IEh1BY1TwHkSe3x9VwzRDd09DTXdNT3/fr9e8puvUqaqni2mePlWnzlFEYGZmZutvg7wDMDMzq3dOpmZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGTqZmZmYZbZh3ALU0aNCgGDp0aN5hmJnxxBNPALDjjjvmuo+eoJ7Oxbx58/4WEVt1Lm+oZDp06FAefPDBvMMwM6O1tRWA2bNn57qPnqCezoWkvxQr92VeMzOzjJxMzczMMnIyNTMzy6ih7pmamfVW7777Lm1tbbzzzjt5h7LOzj//fAAWLlyY6z4K9e/fnyFDhtC3b9+K6juZmpn1Am1tbWy++eYMHToUSXmHs0422CC5SJqlJ2537KNdRPDyyy/T1tbGsGHDKjt+5qOamVnu3nnnHQYOHFh3ibQnksTAgQPXqZXvZGpm1ks4kXafdT2XTqZmZtYtNttsszWWr776aiZMmFCVYxXue/LkyfziF7+oeNsHHniA/fffnx133JGddtqJz3/+87z11luZ4vE9UzOzXmjopNu7dX+Lp3ysW/dXzMqVK9lwww1LLneHF198keOPP57rr7+effbZh4jgpptu4vXXX2eTTTZZ7/06mZqZWdXdeuutfOtb32LFihUMHDiQa6+9lubmZiZPnsxjjz3G888/z/bbb8+IESNYsmQJixcvZtCgQVxyySWcdtppPPvsswBcfPHFjB49eo19b7bZZvTp0weASy+9lGnTprHhhhsycuRIrr/++jXqTp06lXHjxrHPPvsAyeXc4447LvP7czI1M7Nu8fbbbzNq1KjVy6+88gpHHnkkAGPGjOG+++5DEtOnT+fCCy/k+9//PgALFizgZz/7GR/+8IeZPHky8+bN45577mHjjTfmk5/8JBMnTmTMmDE8++yzHHrooWs9/nL22WevHpt3ypQpPPPMM2y00UYsW7ZsrRgfffRRxo0b1+3v3cnUrA70ljFYrXfbeOONmT9//urlq6++evV46G1tbZxwwgksXbqUFStWrPHIyYEHHkj//v1XLx955JFsvPHGANx111089thjq9e99tprvP766yVj2G233fjUpz7F0UcfzdFHH91N76xr7oBkZmZVd8YZZzBhwgQeeeQRrrjiijUeO2lPnO023XTT1a9XrVrFvffey/z585k/fz7PP/88m2++ecnj3H777Zx++unMmzePPfbYg5UrV66xfpdddmHevHnd9K465JpMJR0m6QlJiyRNKrJeki5N1z8s6SNpeX9JD0j6k6QFki6offRmZlap5cuXs+222wJwzTXXVLzdIYccwuWXX756ubDl29mqVat47rnnOOCAA7jwwgtZtmwZb7zxxhp1JkyYwDXXXMP999+/uux//ud/eOGFFyqOqZjckqmkPsBUYCwwEjhJ0shO1cYCw9Of8cAP0/K/AwdGxIeBUcBhkvauRdxmZrbuJk+ezPHHH89+++3HoEGDKt7u0ksv5cEHH2S33XZj5MiRTJs2rWTd9957j09/+tPsuuuu7L777kycOJGmpqY16jQ3N3P99ddz9tlns+OOO7Lzzjtz9913s8UWW6zvWwNAEZFpB+t9YGkfYHJEHJounwsQEd8pqHMFMDsirkuXnwBaI2JpQZ1NgHuAL0bE/ZTR0tISns/U6pHvmfY+3T2H58KFC9l5552zB5aDnjo5eLFzKmleRLR0rptnB6RtgecKltuAvSqosy2wNG3ZzgM+CEwtlUgljSdp1dLc3Oz/jKwutfdK9N8vnHnmmUDyiEQ9645/08J9DBgwoGzHnJ6s/b5mlvi7Yx+dvfPOOxX/++SZTIuN1dS5mVyyTkS8B4yS1AT8UtKHIuLRtSpHXAlcCUnLtP2bnFk9ab9U5b/f3nMuuuN9FO5j4cKFZTvm9GTtAzNkib879tFZ//792X333Suqm2cHpDZgu4LlIcCSda0TEcuA2cBh3R6hmZlZBfJMpnOB4ZKGSeoHnAjc0qnOLcDJaa/evYHlEbFU0lZpixRJGwMHA4/XMHYzM7PVcrvMGxErJU0A7gD6AFdFxAJJp6XrpwEzgMOBRcBbwKnp5oOBa9L7phsAN0TEbbV+D2ZmZpDzCEgRMYMkYRaWTSt4HcDpRbZ7GKjsQrZZztwT16z38whIZmaW2eLFi/nQhz60RtnkyZO56KKL1mt/s2fP5pRTTqm4/ksvvcRZZ53FDjvswMiRIzn88MN58skn1+vY68Nj85qZ9UaTB3Tz/pZ37/66UUQwYcIEjjnmGG6/PZl6bv78+bz44ouMGDGiJjG4ZWpmZlXX2trKOeecw5577smIESO4++67geRZznPPPZcjjjiC3XffnVmzZgHQr18/BgxIvhDMmTOHUaNGMWrUKHbfffe1niWdNWsWffv25cQTT1xdNmrUKPbbb78avTu3TM3MrEZWrlzJAw88wIwZM7jgggu46667mDp1KpDMdxoRHHLIITz55JPsu+++7LvvvgBcdNFFTJ06ldGjR/PGG2+sMcMMJNOq7bLLLjV/P4XcMjUzs8ykYmPsrFn+iU98AoA99tiDxYsXA3DPPfdw1FFHAbDTTjux/fbbr3Wvc/To0Zx11llceumlLFu2bPUADT2Jk6mZ1Uxra2vdj1xkxQ0cOJBXX311jbJXXnlljUHtN9poIwD69Omzevi/SsaHnzRpEtOnT+ftt99m77335vHH1xxWYJdddmHBggVZ30ImTqZmZpbZZpttxuDBg5k5cyaQJNLf/OY3jBkzpux2+++/P7feeisATz75JM8+++xag9X/+c9/Ztddd+Wcc86hpaVlrWR64IEHsmLFCm644YbVZXPnzmXOnDnd8dYq0vPaymZmVpd+8pOfcPrpp/PVr34VgPPPP58ddtih7DZf+tKXuPvuuzniiCPYdNNNufrqq1e3YNtdfPHFzJo1iz59+jBy5EjGjh27xnpJXHbZZXznO99hhx12oH///gwdOrSmkyE4mZqZ9UY5PMoycuTI1b1xOysctGTQoEGr75n279+fKVOmAKWnT7vsssu6PHZzczMXX3xxt07Bti58mdfMzCwjJ1MzM7OMnEzNzMwycjK1HsuPUZitm0oeM7HKrOu5dDI1M+sF+vfvz8svv+yE2g0igpdffnmtkZbKcW9eM7NeYMiQIbS1tfHSSy/lHco6e+GFFwBYtWpVrvso1L9/f4YMGVJxfSdTM7NeoG/fvgwbNizvMNbLF7/4RSDbnL/dsY8sfJnXzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEytKjzji5k1EidTMzOzjHJNppIOk/SEpEWSJhVZL0mXpusflvSRtHw7SbMkLZS0QNJXah+9mZlZIrdkKqkPMBUYC4wETpI0slO1scDw9Gc88MO0fCXw1YjYGdgbOL3ItmZmZjWRZ8t0T2BRRDwdESuA64GjOtU5CvhJJO4DmiQNjoilEfEQQES8DiwEtq1l8GZmZu3yTKbbAs8VLLexdkLsso6kocDuwP3dH6KZmVnX8pwcXEXKYl3qSNoMuAk4MyJeK3oQaTzJJWKam5tzmzi20SxbtgzINlFvd+yjJ/C56OBz0cHnokNvOBd5JtM2YLuC5SHAkkrrSOpLkkivjYhflDpIRFwJXAnQ0tISflyjNpqamgAyPR7THfvoCXwuOvhcdPC56NAbzkWel3nnAsMlDZPUDzgRuKVTnVuAk9NevXsDyyNiqSQBPwIWRsQPahu2mZnZmnJrmUbESkkTgDuAPsBVEbFA0mnp+mnADOBwYBHwFnBquvlo4DPAI5Lmp2X/FhEzavgWzMzMgHwv85ImvxmdyqYVvA7g9CLb3UPx+6lmZmY15xGQzMzMMnIyNTMzy8jJtIfxAPFmZvXHydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJtBt5wAUzs8bkZGpmZpaRk6mZmVlGTqZmZmYZlUymkraSNLJI+S6StqpuWGZmZvWjXMv0MqBY0hwCXFKdcMzMzOpPuWS6a0TM6VwYEXcAu1UvJDMzs/pSLpn2Xc91ZmZmDaVcMn1K0uGdCyWNBZ6uXkhmZmb1ZcMy6yYCt0n6Z2BeWtYC7AN8vNqBmZmZ1YuSLdOIeBLYFZgDDE1/5gC7pevMzMyM8i1TIuLvwI9rFIuZmVldKptMzcwsR5MHlF+/+M2u601e3n3xWEkeAcnMzCwjt0wtP/7WbWa9RMlkKukRIEqtjwgP3GBmZkb5lmn74y+np79/mv7+FPBW1SIyMzOrMyWTaUT8BUDS6IgYXbBqkqTfA/+e9eCSDiMZ57cPMD0ipnRar3T94SQJ/JSIeChddxVJwv9rRHwoayxmufIlb7O6VkkHpE0ljWlfkLQvsGnWA0vqA0wFxgIjgZOKzFIzFhie/owHfliw7mrgsKxxmJmZZVVJB6TPAVdJGkByD3U58NluOPaewKKIeBpA0vXAUcBjBXWOAn4SEQHcJ6lJ0uCIWBoRv5M0tBviMDMzy6TLZBoR84APS9oCUER017WkbYHnCpbbgL0qqLMtsLSbYjAzM8usy2QqqRn4NrBNRIxNL8XuExE/ynhsFSnr3Hu4kjrlDyKNJ7lETHNzM7Nnz16XzdfJsmXLADIdozv20RNU9D52vKD8Pja5PNnHjhNKV6qD8+Rz0cGfkQ7+u+jQG/4uKrnMezXJkILnpctPAv8LZE2mbcB2BctDgCXrUaesiLgSuBKgpaUlWltb1znQSjU1NQGQ5RjdsY+eoKL3Mfmo8vt4K+l00/rE+aUrndTzO934XHTwZ6SD/y469Ia/i0qS6aCIuEHSuQARsVLSe91w7LnAcEnDgOeBE4FPdqpzCzAhvZ+6F7A8InyJ18zqwtBJt5dc98LTL3dZZ3H/bg/JqqSSZPqmpIGkl1cl7U3SCSmTNClPAO4geTTmqohYIOm0dP00YAbJYzGLSB6NObV9e0nXAa3AIEltwPndcOnZzPLmx4SsDlWSTM8iaSHukD5fuhVwfHccPCJmkCTMwrJpBa+DjkEjOm97UnfEYGZmllUlyXQB8I/AjiQdgp7AA+SvP3/rNjPrdSpJivdGxMqIWBARj0bEu8C91Q7MzMysXpQb6H5rkmc6N5a0Ox2PqWwBbFKD2MzMzOpCucu8hwKnkDyO8oOC8teBf6tiTGZmZnWl3ED31wDXSDo2Im6qYUxWJ9zt38wsUclwgjdJ+hiwC9C/oDzzrDFmZma9QZcdkCRNA04AziC5b3o8sH2V4zIzM6sblfTm3TciTgZejYgLgH1Yc4g/MzOzhlZJMn07/f2WpG2Ad4Fh1QvJzMysvlQyaMNtkpqA7wEPkQwrOL2aQZnVG3fGMmtslXRA+mb68iZJtwH9u3FOUzMzs7pXyXymfYCPAUPb60siIn5Qbrveyi0QMzPrrJLLvLcC7wCPAKuqG46ZmVn9qSSZDomI3aoeiZn1Cr56Y42okt68v5Z0SNUjMTMzq1OVtEzvA34paQOSx2JEMtXoFlWNzMzMrE5Ukky/TzJQwyPpZN1mZmZWoJLLvE8BjzqRmpmZFVdJy3QpMFvSr4G/txc26qMxZmZmnVWSTJ9Jf/qlP2ZmZlagkhGQLqhFIGZmZvWqZDKVdHFEnCnpVpLxeNcQEUdWNTIzM7M6Ua5l+tP090W1CMTMzKxelUymETEvfTkqIi4pXCfpK8CcagZmZmZWLyp5NGZckbJTujkOMzOzulXunulJwCeBYZJuKVi1BfBytQMzMzOrF+Xumf6B5BnTQSSjILV7HXi4mkGZmZnVk3L3TP8C/EXSwcDbEbFK0ghgJ5Lp2MzMzIzKBm34HbCfpC2BmcCDwAnAp6oZmJmZ2WqTB5Rfv/jNrutNXt598XRSSQckRcRbwCeAyyLiGGBkdxxc0mGSnpC0SNKkIusl6dJ0/cOSPlLptmZmZrVSUTKVtA9JS7R9Rt9KWrRd7bQPMBUYS5KcT5LUOUmPBYanP+OBH67DtmZmZjWhriaDkbQ/cDbw+4j4rqR/AM6MiC9nOnCSoCdHxKHp8rkAEfGdgjpXALMj4rp0+QmgFRja1bbFDBs2LM4///wsYXPf06U7Mq/46zMA9Hv/sJJ19t5gYdn9z3/hPQBGbd2ndKWhY8ruo1Z8Ljr4XHTwuejgc1GZ+fPnAzBq1KjSlRbfU34fNToXp5566ryIaOlcXsnYvL8juW/avvy0pLMyRwTbAs8VLLcBe1VQZ9sKtwVA0niSVi2DBw9m2bJlmYLe6X2l/6EWvZL8/mCZOsv4UNn9r/zboqRe0wdLV8r4HrqLz0UHn4sOPhcdfC4Sb/59Zdn1K959F4DnX/xbyTqbNvXsc1GyZSrpnogYk77+aUR8pmDdQxHxkaIbVnpg6Xjg0Ij4fLr8GWDPiDijoM7twHci4p50eSbwNeAfutq2mJaWlnjwwQezhF1Wa2srALNnz851Hz2Bz0UHn4sOPhcdGulcDJ10e9n1L/ws6fay9SenlKyzeMrHyu6jVudC0jq3TDcteN35K4G6IaY2YLuC5SHAkgrr9KtgWzMzs5oo1wEpSrwutrw+5gLDJQ2T1A84EbilU51bgJPTXr17A8sjYmmF25qZmdVEuZZpk6RjSBJuk6RPpOUCunjgp2sRsVLSBOAOoA9wVUQskHRaun4aMAM4HFgEvAWcWm7brDH1BD39co2Zma2tXDKdAxxZ8PqIgnW/W7v6uouIGSQJs7BsWsHrAE6vdFszM7M8lBtO8NRaBmLWmVvpZlYvKhm0wczMzMpwMjUzM8vIydTMzCyjdUqmkq6sViBmZmb1al1bpmuN+mBmZtbo1jWZ/rUqUZiZmdWxdUqmEXFYtQIxMzOrV5nnJTUzMyuny0Hq7/seALO7qNeTuTevmZlZRm6ZdiOP2GNm1phKtkwlXdg+6Hyn8omSvlvdsMzMzOpHucu8HweKPVd6CVC/F7bNzMy6Wdn5TCNiVZHCVXTP5OBmZma9Qrlk+pak4Z0L07K3qxeSmZlZfSnXAekbwK8lfQuYl5a1AOcCZ1Y5LjMzs7pRbj7TX0s6GvhX4Iy0+FHg2Ih4pAaxmZmZ1YWyj8ZExKPAuBrFYr2IHxMys0biQRvMzMwycjI1MzPLyMnUzMwso5L3TCVdBkSp9RHx5apEZGZmVmfKdUB6sGZRmJmZ1bFyj8ZcU8tAzMzM6lWXs8ZI2go4BxgJ9G8vj4gDqxiXmZlZ3aikA9K1wEJgGHABsBiYW8WYzMzM6kolyXRgRPwIeDci5kTEZ4G9qxyXmZlZ3ahkcvB3099LJX0MWAIMqV5IZmZm9aWSlum3JA0AvgqcDUwHJmY5qKT3SbpT0lPp7y1L1DtM0hOSFkmaVFB+vKQFklZJaskSi5mZWVZdJtOIuC0ilkfEoxFxQETsERG3ZDzuJGBmRAwHZqbLa5DUB5gKjCXp/HSSpJHp6keBTwC/yxiHmZlZZpX05v0xRQZvSO+drq+jgNb09TXAbJIew4X2BBZFxNNpHNen2z0WEQvTsgwhmNUPTxxg1rNVcs/0toLX/YFjSO6bZtEcEUsBImKppPcXqbMt8FzBchuw17oeSNJ4YDxAc3Oz/1Oymlu2bBnghAjdcy56y/n0uejQG85Fl8k0Im4qXJZ0HXBXV9tJugvYusiq8yqMrVizs+TwhqVExJXAlQAtLS3R2tq6rrswy6SpqQkA/+11z7noLefT56JDbzgXlbRMOxsOfKCrShFxcKl1kl6UNDhtlQ4G/lqkWhuwXcHyELK3iM3MzLpdlx2QJL0u6bX2H+BW1r6/ua5uoWPS8XHAzUXqzAWGSxomqR9wYrqdmZlZj1LJZd7Nq3DcKcANkj4HPAscDyBpG2B6RBweESslTQDuAPoAV0XEgrTeMcBlwFbA7ZLmR8ShVYjTzMysS5X05p0ZEQd1VbYuIuJlYK3tI2IJcHjB8gxgRpF6vwR+ub7HNzMz607l5jPtD2wCDEoHVWjvELQFsE0NYjMzW0u991y13qlcy/RfgDNJEuc8OpLpaySDKZiZmRnl5zO9BLhE0hkRcVkNYzIzM6srlYzNu0pSU/uCpC0lfal6IZmZmdWXSpLpFyJiWftCRLwKfKFqEZmZmdWZSpLpBioYBDcdgL5f9UIyMzOrL5WMgHQHyTOh00iG8zsN+E1VozIzM6sjlSTTc0gGiv8iSY/e3wL/Xc2gzMzM6kkl85muiohpEXFcRBwLLCAZfcjMzMyocKB7SaOAk4ATgGeAX1QxJjMzs7pSbgSkESSDy58EvAz8L6CIOKBGsZmZmdWFci3Tx4G7gSMiYhGApIk1icrMzKyOlLtneizwAjBL0n9LOojiE3abmZk1tJLJNCJ+GREnADsBs4GJQLOkH0o6pEbxmZmZ9XiV9OZ9MyKujYiPA0OA+cCkagdmZmZWLyoZAWm1iHglIq6IiAOrFZCZmVm9WadkamZmZmtzMjUzM8uookEbzGz9zZ49O+8QzKzK3DI1MzPLyC1TM6sZt9Ktt3LL1MzMLCMnUzMzs4ycTM3MzDLyPVMzszrle9A9h1umZmZmGTmZmpmZZeRkamZmllEuyVTS+yTdKemp9PeWJeodJukJSYskTSoo/56kxyU9LOmXkppqFryZmVknebVMJwEzI2I4MJMiU7pJ6gNMBcYCI4GTJI1MV98JfCgidgOeBM6tSdRmZmZF5JVMjwKuSV9fAxxdpM6ewKKIeDoiVgDXp9sREb+NiJVpvftI5lk1MzPLRV7JtDkilgKkv99fpM62wHMFy21pWWefBX7d7RGamZlVqGrPmUq6C9i6yKrzKt1FkbLodIzzgJXAtWXiGA+MB2hubvZzWWbWIyxbtgzws6LQPeci7/NZtWQaEQeXWifpRUmDI2KppMHAX4tUawO2K1geAiwp2Mc44OPAQRERlBARVwJXArS0tERra+s6vQ8zs2poamoCwP8ndc+5yPt85nWZ9xZgXPp6HHBzkTpzgeGShknqB5yYboekw4BzgCMj4q0axGtmZlZSXsl0CvBPkp4C/ildRtI2kmYApB2MJgB3AAuBGyJiQbr95cDmwJ2S5kuaVus3YGZm1i6XsXkj4mXgoCLlS4DDC5ZnADOK1PtgVQM0MzNbBx4ByczMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzyyiXge7NzBqdJwXvXdwyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4xySaaS3ifpTklPpb+3LFHvMElPSFokaVJB+TclPSxpvqTfStqmdtGbmZmtKa+W6SRgZkQMB2amy2uQ1AeYCowFRgInSRqZrv5eROwWEaOA24Bv1CRqMzOzIvJKpkcB16SvrwGOLlJnT2BRRDwdESuA69PtiIjXCuptCkT1QjUzMytvw5yO2xwRSwEiYqmk9xepsy3wXMFyG7BX+4Kk/wBOBpYDB1QxVjMzs7Kqlkwl3QVsXWTVeZXuokjZ6hZoRJwHnCfpXGACcH6JOMYD4wGam5uZPXt2hYc3M7NaWLZsGUCm/5+7Yx9ZVC2ZRsTBpdZJelHS4LRVOhj4a5FqbcB2BctDgCVF6v0MuJ0SyTQirgSuBGhpaYnW1tbK3oCZmdVEU1MTAFn+f+6OfWSR1z3TW4Bx6etxwM1F6swFhksaJqkfcGK6HZKGF9Q7Eni8irGamZmVldc90ynADZI+BzwLHA+QPuIyPSIOj4iVkiYAdwB9gKsiYkH79pJ2BFYBfwFOq/k7MDMzS+WSTCPiZeCgIuVLgMMLlmcAM4rUO7aqAZqZma0Dj4BkZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUZOpmZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRoqIvGOoGUkvkcx/mqdBwN9yjqGn8Lno4HPRweeig89Fh55yLraPiK06FzZUMu0JJD0YES15x9ET+Fx08Lno4HPRweeiQ08/F77Ma2ZmlpGTqZmZWUZOprV3Zd4B9CA+Fx18Ljr4XHTwuejQo8+F75mamZll5JapmZlZRk6mZmZmGTmZmpmZZeRkarmR9KW8Y7CeS9JOeceQN39G6seGeQfQKCT1jYh3O5UNioieMKJH1Uk6q3MRcK6k/gAR8YPaR5UPSZsBXwOOBYYAK4A/A9Mi4uocQ+tpfgt8IO8gasWfkcpI+nVEjM07js6cTKtM0gHAT4GNJP0RGB8Ri9PVvwU+kldsNXYBMANYQPKfBEAfYPPcIsrPtcAvgUOBfwY2Ba4Hvi5pRET8W57B1ZKkS0utAppqGEpP4M9ISlKp/xcFjKphKBXzozFVJmkucEpELJB0HPAd4DMRcZ+kP0bE7jmHWBOSPgD8gKQFdkFEvCXp6Yj4h5xDqzlJf4qIDxcsz42Ij0raAHgsIhrm8qak14GvAn8vsvr7ETGoxiHlxp+RDpLeA+bQ8aWi0N4RsXGNQ+qSW6bV1y8iFgBExM8lLQR+IWkS0DDfZCLiWeA4SUcBd0r6z7xjytGbksZExD2SjgBeAYiIVZKK/efRm80FHo2IP3ReIWly7cPJjz8ja1gI/EtEPNV5haTncoinS+6AVH3vStq6fSFNrAcB5wPDc4sqJxFxM/BPwF5AW87h5OU04AeSlgPnAF8GkLQVMDXPwHJwHDC/2IqIGFbbUHoGf0YAmEzp/HRGDeOomC/zVpmkg4GXIuJPncqbgNMj4j9yCczMzLqNk6nVRNo6Px9YBXyD5NvlJ4DHga9ExNIcw6spSf2AE4HnI2KmpE8C+5Jc2rqyc6/v3kzSAOBc4GigfY7IvwI3A1MiYlk+kdWepIeAXwDXRcSf844nb5J2AI4BtgNWAk+RnJvluQZWgi/zVpmkFkmzJP2PpO0k3SlpuaS5kkblHV8NXQ08BjwHzALeBj4O3A1Myy+sXPwY+BhwpqSfAscD9wMfBabnGVgObgBeBVojYmBEDAQOSMtuzDWy2tuSpAfzLEkPSJooaZucY8qFpC+T/L/Qn+RzsTFJUr1XUmt+kZXmlmmVSXqApEXWBFwITEw7Ih0EfCsi9skzvlop7Lks6dmI+EDBuvkRMSq34GpM0sMRsZukDYHngW0i4r2089GfImK3nEOsGUlPRMSO67quN5L0UER8JH29H3ASydWbhSQtsh49a0p3kvQIMCr9XGwCzIiI1rTH88098SkIt0yrr29E/DoirgMiIn5O8mImybeuRlH4t/aTMusawQbppd7NgU2AAWn5RkDf3KLKx18kfU1Sc3uBpGZJ55BcxWhIEXF3RHwJ2Bb4LtAQX7o7aX/aZCPSZ23THs898jPiR2Oq7x1Jh5D8hxmSjo6IX0n6R+C9nGOrpZslbRYRb0TE19sLJX0QeDLHuPLwI5J7xX2A84AbJT0N7E0yeEMjOQGYBMyR9P607EXgFpIBLRrJWp+DiHgP+E3600imA3Ml3QfsT/KFor3H+yt5BlaKL/NWmaQPk1zeXQVMBL4IjCO5vDc+In6fY3iWk/Z7YRGxJO3ZfTDwbEQ8kGtgZj2EpF2AnUmeQ34873i64mRqNVHQg3VJRNzVyD1YrTRJY4A9Sf4D/W3e8eRN0k8i4uS847CuOZnWQDr7xbbA/RHxRkH5YRHREJdvJF1LclthE2AZsBnJYwAHkfwdjssvup5D0m0R8fG846gVSQ9ExJ7p6y8Ap5OMW3wIcGtETMkzvlqSdEvnIpKezf8HEBFH1jyoHqinfkacTKss7eJ9OkkLbBTJM5U3p+tW997r7dyDtTKSBjfYM7eFvbznAodHxEuSNgXui4hd842wdtLnTB8juV8YJMn0OpIrOkTEnPyi6zl66mek0XpR5uELwB4RcTTQCvw/SV9J1zXSOKzuwVqBnvifRJVtIGlLSQNJvty/BBARb5I8qN9IWoB5JJ3SlkfEbODtiJjjRArp30iP/Yy4N2/19Wm/tBsRi9MHjn8uaXsaK5m6B2vKI92sYQBJAhFJb/etI+IFJXO+NtLng4hYBfynpBvT3y/SoP9HS5oCXBQRf5PUQjK4xypJfYGTe+KXC1/mrTJJ/wecFRHzC8o2BK4CPhURffKKrdbcgzUh6RngJpJHP14guZT3vxGxJNfAepD0Qf3miHgm71jyIuljwOhGmt+2naRH2i/xS5oFfC0i5koaAfwsIlryjXBtTqZVJmkIsDIiXiiybrQfjWk8HunGrDxJjwMfioiVku6LiL0L1q1OtD2J75lWWUS0FUuk6TonUpLeeXnHkBePdFNaI/9ddNaA52IqMEPSgcBvJF0saX9JF1Biyr68uWWao57axbvWemrvvGqRdH1EnJh3HD1do/1dlNOI5yLtX/JFYATJvePngF8BP+6Jz6U7meaoET8glpC0J8lYzXMljQQOAx6PiBk5h2aWO0l7AQsj4rX0/vkk4CPAAuDbPXEaNl/mzVEjJVJJW0v6oaSpkgZKmizpEUk3SBqcd3y1JOl84FLgh5K+A1xOMojFJEnn5RpcjUl6SNLXlcxd2dB8LtZwFfBW+vpikkfqpqRlP84pprKcTKtM0mEFrwdI+pGkhyX9rHCmjAZwNWvPZ/oxGnM+0+OA0SQDeJ8OHB0R/w4cSjLweyPxHJ4dfC46bBAR7c8Zt0TExIi4JyIuAP4hz8BKcTKtvm8XvP4+sBQ4ApgLXJFLRPlojojL0uHhmiLiuxHxbERcBmyfd3A1tjIi3ouIt4A/R8RrABHxNsmECI3k1Yg4O53f9qvAcOAhSbMkjc85tlrzuejwqKRT09d/Sp81JX00psfdLwUn01priYivR8RfIuI/gaF5B1RD5eYzbZhnbVMr0vtAAHu0F0oaQOMl09Xcs7mDzwWfB/5R0p+BkcC96SAv/52u63EacnSNGnu/pLNIRnPZQpKio9dXI32ZuVml5zN9Ise48rB/RPwdVo96064vyfR8jcRzeHbwuUilHYxOkbQ5yWXdDYG2iHgx38hKc2/eKks7mxT6r3Qg762BCxtpeiXPnmOV8LRjCXk6urriZJojSadGRI/smdbdJJ0BTKDBZ8+xNXnasQ6ejq6+OZnmSNKzaWeDXk/SI8A+EfGGpKHAz4GfRsQlhdNwWWOR9EeSZwcbftoxT0dX33zPtMokPVxqFdBIj8Z49hwrZg/gKyQzCf1rRMyX9HYjJdECG0jakqQvxRrT0UlqtOno6o6TafU1kzw/+GqncgF/qH04uXlB0qj22XPSFurHSR7O9jfuBuVpx9bg6ejqWKP+0dbSbcBmhVOwtZM0u+bR5OdkOk32nD6UfbKkRnre1oqIiDbg+HTasdfyjicPETG0xKpVwDE1DMXWg++ZmpmZZdRIzzmamZlVhZOpmZlZRk6mZg1MUqukffOOoxYkHSlpUt5xWO/ke6ZmdUbShgUzamTd12TgjYi4qDv211N15zkzK8YtU2tIkoZKelzSdEmPSrpW0sGSfi/pqXTybiRtKukqSXMl/VHSUQXb353OQflQe+subenNlvTzdP/XSlrrsYa0zsWS/pAev6vjnSLpRkm3Ar+VtJmkHyuZE/ZhScem9Q6RdG8a043pYxVIWizpgrT8EUk7pYNnnAZMlDRf0n6SjpB0f3rsu5ROEyhpK0l3pttfIekvkgal6z6tZMqw+em6tSYukPTR9L3+Ka27uaQ+kr6XvteHJf1LV+dQ0hRJj6X1L0rLtpc0My2bKekDafnVkn4gaRbw3fQcXl6w7tI0pqclHZeWbyDpvyQtkHSbpBnt68zKigj/+Kfhfkhm7FlJ8ozrBiTP911F8jzfUcCv0nrfBj6dvm4iGYx8U2AToH9aPhx4MH3dCiwHhqT7vRcYU+T4s4H/Tl/vTzL+arnjnQK0Ae9L130XuLhgf1sCg4DfAZumZecA30hfLwbOSF9/CZievp4MnN1pP+1XrD4PfD99fTlwbvr6MJLRigYBOwO3An3Tdf8FnNzpvfYDngY+mi5vQfJY3njg62nZRsCDwLBS5xB4H8mkCO3xNaW/bwXGpa8/W/BvdzXJo2l90uVTgMsL1t2Y7n8ksCgtPw6YkZZvTfJ8+HF5/736p+f/+DlTa2TPRMQjAJIWADMjIpQMfTg0rXMIcKSks9Pl/sAHgCXA5ZJGAe8BIwr2+0Akz00iaX66r3uKHP86gIj4naQtJDWVOR7AnRHxSvr6YNIh99J9vKpkEIyRwO/Thlw/kkTU7hfp73nAJ0qckyHA/0oanG7/TFo+hvRZx4j4jaT2QUgOIhnFaG56zI2Bv3ba547A0oiYm27/GiStaGC3gpbfAJIvJisofg7vA94Bpku6nSRRQjI9Wfv7+SlwYcGxb4xk5pVifhXJoBGPtbfA0/d5Y1r+QtqqNeuSk6k1sr8XvF5VsLyKjs+GgGMjYo1p4pTca3wR+DBJK+adEvt9j9Kfs84dFtrHpi12vL2ANwuLimwvkoR7UonjtcdVLqbLgB9ExC1KhnycXLDvYgRcExHnllhfKtb28jMi4o41CpPjrnUOI2Jlejn8IJIvEhOAA4vst/BYbxZZ367wGOr022yd+J6pWXl3AGcU3LNrH5B/AElraxXwGdZvgvMT0n2OAZZHModjqeN19luSZEJab0uSlttoJXPEImkTSSNKbN/udWDzguUBwPPp68K5Ve8B/jnd7yEkl4MBZgLHSXp/uu59SsZbLvQ4sI2kj6Z1Npe0Yfpevyipb1o+Qsmg7kWl938HRMQM4EyS2YcgGZazvZX+KYpfBajUPcCx6b3TZpJLzmZdcjI1K++bJJN2Pyzp0XQZknuD4yTdR3KJt1wLqJRXJf0BmAZ8rovjdfYtYEslnZf+BBwQycDopwDXKZlg4T5gpy5iuBU4pr0DEklL9EZJdwN/K6h3AXCIpIeAscBS4PWIeAz4OkmnqIeBO4HBhQeIiBUkXxwuS2O9k+Ty9XTgMeCh9L1eQfmrZZsDt6XHmQNMTMu/DJyaln+GZOD89XUTyb3p9njuJ7l/a1aWH40xy4GScZnPjogH846lEpI2At5LL7XuA/wwIkblHFZVSNoskokYBgIPAKMj4oW847KezfdMzawSHwBukLQBSQehL+QcTzXdlnYG6wd804nUKuGWqZmZWUa+Z2pmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRv8fw2BH4vdxMqMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "hazard_ratio = 2.0\n",
    "ylim = [-0.035, 0.035]\n",
    "mean_1, std_1 = simulation(100, hazard_ratio)\n",
    "plot_results(mean_1, std_1, ylim=ylim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can observe that estimates are on average below the actual value, except for the highest amount of censoring, where Harrell's c begins overestimating the performance (on average).\n",
    "\n",
    "With such a small dataset, the variance of differences is quite big, so let us increase the amount of data to 1000 and repeat the simulation (*this may take some time*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdMAAAFLCAYAAACNy2aDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAr/klEQVR4nO3de7xVdZ3/8ddbFEFRULAjigUZXvCGevKKzlEbFWu8lI5aKdqFscKSxhlxnEfi5DRm1qhJITEGlWmaOaFRhvyEtEQBhwREkZT0JGKp4P0CfH5/rHVgc9h7n43r7L3OPvv9fDzO4+z1Xd+11mcv9uGzv2t91/eriMDMzMzeuy3yDsDMzKzeOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZbRl3gHU0oABA2Lw4MF5h2FmZnVq/vz5f4uIndqXN1QyHTx4MPPmzcs7DDMzq1OS/lys3Jd5zczMMnIyNTMzy8jJ1MzMLKOGumdqZtZdvfvuu7S2tvLWW2/lHUq30KtXLwYNGsRWW21VUX0nUzOzbqC1tZXtttuOwYMHIynvcOpaRPDiiy/S2trKkCFDKtrGl3nNzLqBt956i/79+zuRdgJJ9O/ff7Na+U6mZmbdhBNp59ncc+lkamZmnaJPnz4bLU+ZMoUxY8ZU5ViF+x4/fjxTpkypeNuHH36Yo48+mj333JO99tqLz33uc7zxxhuZ4vE9UzOzbmjwuF916v6WX/XRTt1fMWvWrGHLLbcsudwZVq5cyRlnnMGtt97K4YcfTkRwxx138Oqrr7LNNtu85/06mZqZWdXdddddXHnllbzzzjv079+fm2++maamJsaPH89zzz3H8uXLGTBgAHvsscdGy9dddx0XXHABzzzzDADXXnstRx555Eb77tOnD7179wbg+uuvZ+LEiWy55ZYMGzaMW2+9daO6EyZMYNSoURx++OFAcjn39NNPz/z+nEzNzKxTvPnmmwwfPnz98ksvvcTJJ58MwIgRI5gzZw6SmDx5MldffTXf/va3AZg/fz4PPPAAvXv3Zvz48Rstf/KTn2Ts2LGMGDGCZ555hhNOOIElS5ZsdNyLL754/eurrrqKp59+mq233ppVq1ZtEuOiRYsYNWpUp793J1MzM+sUvXv3ZsGCBeuXp0yZsn489NbWVs4880xWrFjBO++8s9EjJyeffPL6lmX75XvvvZfHHnts/bpXXnmFV199tWQM+++/P5/61Kc49dRTOfXUUzvpnXXMHZDMzKzqLrzwQsaMGcPChQu58cYbN3rsZNttt92obuHyunXrePDBB1mwYAELFizgL3/5C9ttt13J4/zqV7/iS1/6EvPnz+fggw9mzZo1G63fZ599mD9/fie9qw1yTaaSTpT0hKRlksYVWS9J16frH5V0UFreS9LDkv4oabGkK2ofvZmZVWr16tXsuuuuAEydOrXi7Y4//nhuuOGG9cuFLd/21q1bx7PPPssxxxzD1VdfzapVq3jttdc2qjNmzBimTp3KQw89tL7sJz/5Cc8//3zFMRWTWzKV1AOYAIwEhgFnSxrWrtpIYGj6Mxr4flr+NnBsRBwADAdOlHRYLeI2M7PNN378eM444wyOOuooBgwYUPF2119/PfPmzWP//fdn2LBhTJw4sWTdtWvX8ulPf5r99tuPAw88kLFjx9KvX7+N6jQ1NXHrrbdy8cUXs+eee7L33ntz//33s/3227/XtwaAIiLTDt7zgaXDgfERcUK6fClARPxXQZ0bgVkRcUu6/ATQEhErCupsAzwAfCEiHqKM5ubm8HymZtYdLVmyhL333jvvMLqVYudU0vyIaG5fN88OSLsCzxYstwKHVlBnV2BF2rKdD3wImFAqkUoaTdKqpampiVmzZnVK8GZmXUnfvn3LdsyxzffWW29VnDPyTKbFxmpq30wuWSci1gLDJfUD7pS0b0Qs2qRyxCRgEiQt05aWliwxm5l1SUuWLCnbMcc2X69evTjwwAMrqptnB6RWYLeC5UHAc5tbJyJWAbOAEzs9QjMzswrkmUznAkMlDZHUEzgLmNauzjTg3LRX72HA6ohYIWmntEWKpN7AR4DHaxi7mZnZerld5o2INZLGAPcAPYCbImKxpAvS9ROB6cBJwDLgDeD8dPOBwNT0vukWwG0RcXet34OZmRnkPAJSREwnSZiFZRMLXgfwpSLbPQpUdiHbzMysyjwCkpmZZbZ8+XL23XffjcrGjx/PNddc8572N2vWLM4777yK6z///POcddZZ7L777gwbNoyTTjqJpUuXvqdjvxcem9fMrDsa37eT97e6c/fXiSKC0047jVGjRq2fJWbBggWsXLmSPfbYoyYxuGVqZmZV19LSwiWXXMIhhxzCHnvswf333w8kz3Kef/7560ctuu+++wDo2bMnffsmXwhmz57N8OHDGT58OAceeOAmz9Ped999bLXVVlxwwQXry4YPH85RRx1Vo3fnlqmZmdXImjVrePjhh5k+fTpXXHEF9957LxMmTABg4cKFPP744xx//PEsXbqUI444giOOOAKAa665hgkTJnDkkUfy2muv0atXr432u2jRIg4++OCav59CbpmamVlmUrExdjYu//jHPw7AwQcfzPLlywF44IEHOOeccwDYa6+9+MAHPrDJvc4jjzySr371q1x//fWsWrWKLbfseu1AJ1MzM8usf//+vPzyyxuVvfTSSxsNar/11lsD0KNHj/VTo1UyPvy4ceOYPHkyb775JocddhiPP77xsALVmlZtcziZmplZZn369GHgwIHMnDkTSBLpb37zG0aMGFF2u6OPPpqbb74ZgKVLl/LMM8+w5557blTnT3/6E/vttx+XXHIJzc3NmyTTY489lrfffpsf/OAH68vmzp3L7NmzO+OtVcTJ1MzMOsWPfvQjrrzySoYPH86xxx7L5Zdfzu677152my9+8YusXbuW/fbbjzPPPJMpU6asb8G2ufbaa9l333054IAD6N27NyNHjtxovSTuvPNOZsyYwe67784+++zD+PHj2WWXXTr9PZaS2xRsefAUbGbWXXkKts63OVOwuWVqZmaWkZOpmZlZRk6mZmZmGTmZmpl1E43UB6baNvdcOpmamXUDvXr14sUXX3RC7QQRwYsvvrjJSEvldL1hJMzMbLMNGjSI1tZW/vrXv+YdSrfQq1cvBg0aVHF9J1Mzs25gq622YsiQIXmH0bB8mdfMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8so12Qq6URJT0haJmlckfWSdH26/lFJB6Xlu0m6T9ISSYslfaX20ZuZmSVyS6aSegATgJHAMOBsScPaVRsJDE1/RgPfT8vXAP8cEXsDhwFfKrKtmZlZTeTZMj0EWBYRT0XEO8CtwCnt6pwC/CgSc4B+kgZGxIqIeAQgIl4FlgC71jJ4MzOzNnkm012BZwuWW9k0IXZYR9Jg4EDgoc4P0czMrGN5Tg6uImWxOXUk9QHuAC6KiFeKHkQaTXKJmKamJmbNmvWegjUzMyslz2TaCuxWsDwIeK7SOpK2IkmkN0fEL0odJCImAZMAmpubo6WlJXPgZmZmhfK8zDsXGCppiKSewFnAtHZ1pgHnpr16DwNWR8QKSQL+B1gSEd+pbdhmZmYby61lGhFrJI0B7gF6ADdFxGJJF6TrJwLTgZOAZcAbwPnp5kcC5wALJS1Iy/4tIqbX8C2YmZkBoIj2tym7r+bm5pg3b17eYZiZWZ2SND8imtuXewQkMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy6hkMpW0k6RhRcr3kbRTdcMyMzOrH+Vapt8FiiXNQcB11QnHzMys/pRLpvtFxOz2hRFxD7B/9UIyMzOrL+WS6VbvcZ2ZmVlDKZdMn5R0UvtCSSOBp6oXkpmZWX3Zssy6scDdkv4RmJ+WNQOHAx+rdmBmZmb1omTLNCKWAvsBs4HB6c9sYP90nZmZmVG+ZUpEvA38sEaxmJmZ1SUP2mBmZpaRk6mZmVlGTqZmZmYZlbxnKmkhEKXWR4QHbjAzM6N8B6S2x1++lP7+cfr7U8AbVYvIzMyszpR7NObPEfFn4MiI+NeIWJj+jANO6IyDSzpR0hOSlkkaV2S9JF2frn9U0kEF626S9IKkRZ0Ri5mZ2XtVyT3TbSWNaFuQdASwbdYDS+oBTABGAsOAs4vMUjMSGJr+jAa+X7BuCnBi1jjMzKz+tbS00NLSktvxyz5nmvoscJOkviT3UFcDn+mEYx8CLIuIpwAk3QqcAjxWUOcU4EcREcAcSf0kDYyIFRHxO0mDOyEOMzOzTDpMphExHzhA0vaAImJ1Jx17V+DZguVW4NAK6uwKrOikGMzMzDLrMJlKagK+AewSESPTS7GHR8T/ZDy2ipS17z1cSZ3yB5FGk1wipqmpiVmzZm3O5mZmXdZFF10EwLXXXptrHF3BqlWrAHL7P76Sy7xTSIYUvCxdXgr8DMiaTFuB3QqWBwHPvYc6ZUXEJGASQHNzc+R5Td3svWr73PrLoBXq168fQK73CruKvM9FJR2QBkTEbcA6gIhYA6zthGPPBYZKGiKpJ3AWMK1dnWnAuWmv3sOA1RHhS7xmZtalVJJMX5fUn/TyaltSy3rgNCmPAe4BlgC3RcRiSRdIuiCtNp1k7tRlwA+AL7ZtL+kW4EFgT0mtkj6bNSYz6/ry7rVpVkwll3m/StJC3F3S74GdgDM64+ARMZ0kYRaWTSx4HWwYNKL9tmd3RgxmZmZZVZJMFwN/B+xJ0iHoCTymr5mZdZLu0CegkqT4YESsiYjFEbEoIt4lubxqZmZmlB/ofmeSZzp7SzqQDY+pbA9sU4PYzMzM6kK5y7wnAOeRPI7ynYLyV4F/q2JMZmZmdaVkMo2IqcBUSZ+IiDtqGJOZmVldqWQ4wTskfRTYB+hVUP4f1QzMrDt0SjCzxtBhByRJE4EzgQtJ7pueAXygynGZmZnVjUp68x4REecCL0fEFcDhbDzEn5mZWUOrJJm+mf5+Q9IuwLvAkOqFZNa9eMQes+6vkkEb7pbUD/gW8AjJsIKTqxmUmZlZPamkA9LX05d3SLob6NWJc5qamZnVvUrmM+0BfBQY3FZfEhHxnXLbmZm15x7a1l1Vcpn3LuAtYCHpNGxmZma2QSXJdFBE7F/1SMzMzOpUJb15fy3p+KpHYmbWQNzLu3uppGU6B7hT0hYkj8WIZKrR7asamdU13xszs0ZSSTL9NslADQvTybrNrLON71t+/fLXO6433p3szfJSyWXeJ4FFTqRmZmbFVdIyXQHMkvRr4O22Qj8aY5m5NWZm3UQlyfTp9Kdn+mNmZmYFKhkB6YpaBGJmZlavSiZTSddGxEWS7iIZj3cjEXFyVSNrUO4Fa2ZWf8q1TH+c/r6mFoGYmZnVq5LJNCLmpy+HR8R1heskfQWYXc3AzKxBuWOa1aFKOiCNAq5rV3ZekTIzM+tM/mJRN8rdMz0b+CQwRNK0glXbAy9WO7B65PudZmaNqVzL9A8kz5gOIBkFqc2rwKPVDMrMzKyelLtn+mfgz5I+ArwZEesk7QHsRTIdm70XvmxjZtbtVHLP9HfAUZJ2AGYC84AzgU9VMzDr+gaP+1XJdc8/9WKHdZb36vSQzMxyUcnYvIqIN4CPA9+NiNOAYZ1xcEknSnpC0jJJ44qsl6Tr0/WPSjqo0m3NzMxqpZKWqSQdTtIS/exmbNfRTnsAE4C/B1qBuZKmRcRjBdVGAkPTn0OB7wOHVritWc24lW7W2NTRZDCSjgYuBn4fEd+U9EHgooj4cqYDJwl6fESckC5fChAR/1VQ50ZgVkTcki4/AbQAgzvatpghQ4bE5ZdfniVs5jxVuiPzOy88DUDP9w0pWeewLZaU3f+C59cCMHznHqUrDR5Rdh+1Uu1zUZEGORf19LkoZ8GCBQAMHz68dKXlD5Tfh8/Fhn000rmowT4qcf7558+PiOb25ZWMzfs7kvumbctPSfpqJ8S0K/BswXIrSeuzozq7VrgtAJJGA6MBBg4cyKpVqzIFvdeOpT+0y15Kfn+oTJ1V7Ft2/2v+tiyp1+9DpStlfA+dpdrnoiINci7q6XPx+ttrSq575913AfjLyr+VrLNtP5+LNt3lXJQ7D1DhudBbZfex5t1kUrNVL7SWrtSzT9l9ZFHuOdMHImJE+vrHEXFOweqHgYOKb1kxFSlr30wuVaeSbZPCiEnAJIDm5ua46KKLNiPEzdP2nOkNN9xQtWPUC5+LDTrjXNTT+Sx7yfuuqQDs/MmTStZZftVHy+7f52KDejkX5c4DdPK5mPD9zQtuM40dO7ZoebmW6bYFr9t/PSqWzDZXK7BbwfIg4LkK6/SsYFszM7OaKNebN0q8Lrb8XswFhkoaIqkncBYwrV2dacC5aa/ew4DVEbGiwm3NzMxqolzLtJ+k00gSbj9JH0/LBXQw8kDHImKNpDHAPUAP4KaIWCzpgnT9RGA6cBKwDHgDOL/ctlljMuuqPESlWddWLpnOBk4ueP0PBet+t2n1zRcR00kSZmHZxILXAXyp0m3z5v/wzMwaU7nhBM+vZSBmZmb1KvPgC2Zmhcr1umyZ8y0AZnXQM9Os3lQynKCZmZmV4WRqZmaW0WZd5pU0KSJGVysY6z7cGcvMGsnmtkw3GY/QzMys0W1uB6QXqhKFmZl1Wx0OBdgNOqZtVss0Ik6sViBmZmb1yh2QzMzMMvJzpmZmVeJnbhuHk6mZ1RX3FLeuqORlXklXtw063658rKRvVjcsMzOz+lHununHSCfVbuc6wNclzMzMUmXnM42IdUUK19E5k4ObmZl1C+Xumb4haWhEPFlYKGko8GZ1wzLrPnyPz6z7K5dMvwb8WtKVwPy0rBm4FLioynGZmZnVjXLzmf5a0qnAvwAXpsWLgE9ExMIaxGZmZmX4qkfXUfbRmIhYBIyqUSxmZmZ1ySMgmZmZZeRkamZmlpGTqZmZWUYl75lK+i4QpdZHxJerEpGZmVmdKdcBaV7NojAzM6tj5R6NmVrLQMzMzOpVh7PGSNoJuAQYBvRqK4+IY6sYl5mZWd2opAPSzcASYAhwBbAcmFvFmMzMzOpKJcm0f0T8D/BuRMyOiM8Ah1U5LjMzs7pRyeTg76a/V0j6KPAcMKh6IZmZmdWXSlqmV0rqC/wzcDEwGRib5aCSdpQ0Q9KT6e8dStQ7UdITkpZJGldQfoakxZLWSWrOEouZ1c6sWbM8nqx1Sx0m04i4OyJWR8SiiDgmIg6OiGkZjzsOmBkRQ4GZ6fJGJPUAJgAjSTo/nS1pWLp6EfBx4HcZ4zAzM8uskt68P6TI4A3pvdP36hSgJX09FZhF0mO40CHAsoh4Ko3j1nS7xyJiSVqWIQQzM7POUck907sLXvcCTiO5b5pFU0SsAIiIFZLeV6TOrsCzBcutwKGbeyBJo4HRAE1NTb7EZGZdwqpVqwBPowadcy7yPp8dJtOIuKNwWdItwL0dbSfpXmDnIqsuqzC2Ys3OksMblhIRk4BJAM3NzdHS0rK5uzAz63T9+vUDwP8ndc65yPt8VtIybW8o8P6OKkXER0qtk7RS0sC0VToQeKFItVZgt4LlQWRvEZuZmXW6DjsgSXpV0ittP8BdbHp/c3NNY8Ok46OAXxapMxcYKmmIpJ7AWel2ZmZmXUoll3m3q8JxrwJuk/RZ4BngDABJuwCTI+KkiFgjaQxwD9ADuCkiFqf1TgO+C+wE/ErSgog4oQpxmpmZdaiS3rwzI+K4jso2R0S8CGyyfUQ8B5xUsDwdmF6k3p3Ane/1+GZmZp2p3HymvYBtgAHpoAptHYK2B3apQWxmZmYVybtXdLmW6T8BF5EkzvlsSKavkAymYGZmZpSfz/Q64DpJF0bEd2sYk5mZWV2pZGzedZL6tS1I2kHSF6sXkpmZWX2pJJl+PiJWtS1ExMvA56sWkZlZA/Cg/91LJcl0CxUMgpsOQN+zeiGZmZnVl0pGQLqH5JnQiSTD+V0A/KaqUZmZmdWRSpLpJSQDxX+BpEfvb4EfVDMoMzOzelLJfKbrImJiRJweEZ8AFpOMPmRmZmZUONC9pOHA2cCZwNPAL6oYk5mZWV0pNwLSHiSDy58NvAj8DFBEHFOj2MzMzOpCuZbp48D9wD9ExDIASWNrEpWZmVkdKXfP9BPA88B9kn4g6TiKT9htZmbW0Eom04i4MyLOBPYCZgFjgSZJ35d0fI3iMzMz6/Iqmc/0deBm4GZJO5LMPTqO5BEZMzOzTLrDSFCVjIC0XkS8FBE3RsSx1QrIzMys3mxWMjUzM7NNOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUZOpmZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGuSRTSTtKmiHpyfT3DiXqnSjpCUnLJI0rKP+WpMclPSrpTkn9aha8mZlZO3m1TMcBMyNiKDAzXd6IpB7ABGAkMAw4W9KwdPUMYN+I2B9YClxak6jNzMyKyCuZngJMTV9PBU4tUucQYFlEPBUR7wC3ptsREb+NiDVpvTnAoOqGa2ZmVlpeybQpIlYApL/fV6TOrsCzBcutaVl7nwF+3ekRmpmZVWjLau1Y0r3AzkVWXVbpLoqURbtjXAasAW4uE8doYDRAU1NTt5jR3czMupaqJdOI+EipdZJWShoYESskDQReKFKtFditYHkQ8FzBPkYBHwOOi4ighIiYBEwCaG5ujpaWls16H2ZmZh3J6zLvNGBU+noU8MsideYCQyUNkdQTOCvdDkknApcAJ0fEGzWI18zMrKS8kulVwN9LehL4+3QZSbtImg6QdjAaA9wDLAFui4jF6fY3ANsBMyQtkDSx1m/AzMysTdUu85YTES8CxxUpfw44qWB5OjC9SL0PVTVAMzOzzeARkMzMzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMckmmknaUNEPSk+nvHUrUO1HSE5KWSRpXUP51SY9KWiDpt5J2qV30ZmZmG8urZToOmBkRQ4GZ6fJGJPUAJgAjgWHA2ZKGpau/FRH7R8Rw4G7gazWJ2szMrIi8kukpwNT09VTg1CJ1DgGWRcRTEfEOcGu6HRHxSkG9bYGoXqhmZmblbZnTcZsiYgVARKyQ9L4idXYFni1YbgUObVuQ9J/AucBq4JgqxmpmZlZW1ZKppHuBnYusuqzSXRQpW98CjYjLgMskXQqMAS4vEcdoYDRAU1MTs2bNqvDwZmZmlalaMo2Ij5RaJ2mlpIFpq3Qg8EKRaq3AbgXLg4DnitT7KfArSiTTiJgETAJobm6OlpaWyt6AmZlZhfK6ZzoNGJW+HgX8skiducBQSUMk9QTOSrdD0tCCeicDj1cxVjMzs7Lyumd6FXCbpM8CzwBnAKSPuEyOiJMiYo2kMcA9QA/gpohY3La9pD2BdcCfgQtq/g7MzMxSimicjrDNzc0xb968vMMwM7M6JWl+RDS3L/cISGZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUZOpmZmZhk5mZqZmWXUUFOwSforyfyneRoA/C3nGLoKn4sNfC428LnYwOdig65yLj4QETu1L2yoZNoVSJpXbC68RuRzsYHPxQY+Fxv4XGzQ1c+FL/OamZll5GRqZmaWkZNp7U3KO4AuxOdiA5+LDXwuNvC52KBLnwvfMzUzM8vILVMzM7OMnEzNzMwycjI1MzPLyMnUciPpfXnH0BVI6iPpIEn98o4lL5K2KlI2II9YrGuQ9P62vwlJgyWdLmnfnMMqyck0J5J+nXcMtSRpx3Y//YGHJe0gace846slSd8reD0CeAz4NrBQ0km5BZYDScdIagWek/RbSYMLVv82p7ByIalZ0n2SfiJpN0kzJK2WNFfSgXnHV0uSxgGzgTmSPgf8BhgJ/EzSV3MNroQt8w6gO5N0UKlVwPAahtIV/I1Nh3LcFXgECOCDNY8oP4cVvP46cGpEPCLpg8BtwPR8wsrF1cAJEbFY0unADEnnRMQckr+TRvI94HKgH/AHYGxE/L2k49J1h+cYW62dAwwDtgGWAx+MiL9K2hZ4CPhOjrEV5WRaXXNJvl0V+0+hX21Dyd2/Ah8B/iUiFgJIejoihuQbVu62j4hHACLiKUk98g6oxnpGxGKAiPi5pCXAL9KWSaM9t7dVRPwaQNI3I+LnABExU9I1+YZWc2sj4k1J7wBvAi8CRMTrUtf8juVkWl1LgH+KiCfbr5D0bA7x5CYirpF0K/Df6Xu/nMb7z7LNXpIeJfmSNVjSDhHxsqQtgE3uHXZz70raOSKeB0hbqMcBdwO75xtazb0l6XigLxCSTo2I/5X0d8DanGOrtUck/RTYFpgJTJX0G+BYktsiXY6TaXWNp/R96QtrGEeXEBGtwBmS/gGYQXIJpxHt3W759fT3jsDXahxL3sYBTcDzbQUR0ZomkDG5RZWPC0gue68DTgC+IGkK8Bfg8znGlYfPAWeQfOH+OXAI8EngCWBCjnGV5BGQLBeSegO7R8SivGMxM8vKybTKJO0FnELS2SaA54BpEbEk18C6AEk/iohz846j1iT1BS4FTgXa5kV8AfglcFVErMonstqTtD3JuRgE/Doiflqw7nsR8cXcgsuBpN2B04DdgDXAk8BPI+KVXAOrsXr8XPjRmCqSdAlwK8m9sYdJOiQJuCXtYNEwJE1r93MX8PG25bzjq7HbgJeBlojoHxH9gWPSsttzjaz2fkjyN3EHcJakOyRtna47rPRm3Y+kLwMTgV7Ah4HeJEl1jqSW/CLLRd19LtwyrSJJS4F9IuLdduU9gcURMTSfyGpP0iMkHQcmk7TQBdwCnAUQEbPzi662JD0REXtu7rruSNKCiBhesHwZcBJwMjAjIko9XtbtSFoIDI+ItZK2AaZHRIuk9wO/jIiGeda0Hj8XbplW1zpglyLlA9N1jaQZmA9cBqyOiFnAmxExu5ESaerPkv5VUlNbgaSm9EpGQ/XyBrZOezEDEBH/STLV1u+A/rlFlZ+2TqFbA9sBRMQzNF4v77r7XLg3b3VdBMyU9CQb/pN8P/AhGqynYkSsI3ks5vb090oa9/N3Jkkv1tkFQyquBKYB/5hbVPm4i+Rxh3vbCiJiavr5+G5uUeVjMjBX0hzgaOCbAJJ2Al7KM7Ac1N3nwpd5qyz9dnUISQckAa3A3IhotOfGNiLpo8CREfFvecdi1lVI2ofk0alFEfF43vFY5ZxMq0zSlhGxJn3dB9gLeCoiGu2bplVA0vkR8cO846gl93i3Ykr0bL4lIlbnGlgJvmdaRZLOA1ZKWippJPAoyaWbP0o6O9fguhBJd+cdQxdyRd4B1JJ7vFem0f5GJH2F4j2bH+yqPZvdMq2itHfeMSQdCf4IHBgRf0o7nsyIiP1zDbCLkDQwIlbkHUetpEMJFl0F7BERW5dY3+24x3tlGvBvpO56NjdqB5BaWRsRfwP+Jum1iPgTQESs7KqDNdeSpPdFxAuN9J9EqolkuLiX25WLZLaQRtLW4739jEKN2OO9pAb8G4EkP62lXc9mFZn7titwMq2uZyT9F8kH4XFJ3wZ+QTJ7SkP9cWjTOUtFMp/pgSRXSBrpHvLdQJ+IWNB+haRZNY8mXxfhHu9AMp8p8C2SsXgvBW4i6by4FBgdEf+XY3i1Vnc9m32Zt4rSIbG+RNKp4gaS1sj5JN/Cr2ykb5uS1rFp62MQSe/miIhGms/UCrjHe0LSw2yYz/RqkvlMf57OonNlRDTSfKZ117PZydRqQtLFeD5Ts5Ik/V/bvUBJz0TE+4uts67JvXlzIml03jHUUkRcQzKt0tckfUfSdjTufKYlNVqvzXIa8Fy8Jel4SWeQzmcK0KDzmZbUVT8Xvmean4brgeT5TCvSaPNWltNo5+ILJPcGC+cz/SHJc7cN9eW7A13yc+HLvFVWbw8eV1P6cP6uwEMk37R3j4hFkk6MiN/kG51Z1yLpKJJ7yQsj4rd5x2Pl+TJvFZWZUqnLPnhcLem5+CVwIbAIOL5gYvBv5BZYDiT1kfQfkhZLWi3pr5LmpIN8NBSfiw3SDkhtrz8PXA/0AS5vtAEsJDVLuk/STyTtJmlG+vmYmz4B0OW4ZVpF9fjgcbWk5+LwiHhN0mDg58CPI+K6RutcIemXwJ0kg3j/I7AtyShA/w78pZHGK/a52KBdB6S5wEkR8VdJ2wJzImK/fCOsnXrs2exkWkVpAmmOiLcl7QDcGxEHp+sWRcS++UZYO5Iei4hhBct9SBLqY8CxhXMXdneS/hgRBxQsz42ID6ePiDwWEXvlGF5N+VxsIOmPQAvJFcN7IqK5YF2jfeGsu57NvsxbXW0PHk8CHiR51rRLP3hcRc9LGt62EBGvAR8DBgAN84079bqkEQBpZ6yXYP00dY3WMc3nYoO+JHP+zgN2lLQzrP/i2Wjnou56NrtlWmX19uBxtUgaBKyJiOeLrDsyIn6fQ1i5kHQA8ANgD5L7x5+JiKXpl6yzI+L6XAOsoYJzsSewEPhsRDzRiOeilPQWUVNEPJ13LLWSfi6uJunZPJakp/O5pD2bu+L/F06mZjmQdCiwLiLmShoGnAg8HhHTcw4td5J+FBHn5h2H5Sf9+3g8IlZL6k0yvOJBwGLgG13xaQgnU7Mak3Q5MJLkOe8ZwKHALJIRou6JiP/ML7rakjStSPGxwP8DiIiTaxuRdQWSFgMHRMSa9DbZ68AdwHFp+cdzDbAIJ1OzGmvr5U0yG8bzwKCIeCX9Bv5QI03NJ+n/SFobk0lGxBJwC3AWQETMzi86y4ukJRGxd/r6kYg4qGDdgq7YYdEdkMxqb01ErI2IN4A/RcQrABHxJo037djBJJ1uLgNWR8Qs4M2ImO1E2tAWSTo/ff1HJTPqIGkP4N3Sm+XHwwma1d47krZJk+nBbYWS+tJgyTTttfvfkm5Pf6/E/y9ZMo73dZL+HfgbyUA3z5JM0/e5XCMrwZd5zWpM0tYR8XaR8gHAwLZZdRqRpI8CRzbSYA1WWjohxgdJvmC1RsTKnEMqycnUzMwsI98zNTMzy8jJ1MzMLCMnU7MGJqlF0hF5x1ELkk5utNlXrHZ8z9SszkjaMiLWdNK+xgOvRcQ1nbG/rqozz5lZMW6ZWkOSNFjS45ImS1ok6WZJH5H0e0lPSjokrbetpJvSeRT/T9IpBdvfL+mR9OeItLxF0ixJP0/3f7OkTQYpT+tcK+kP6fE7Ot55km6XdBfwWyXzgP5Q0kJJj0r6RFrveEkPpjHdng6SjqTlkq5IyxdK2kvJVHgXAGMlLZB0lKR/kPRQeux7JTWl2++kZE7JRyTdKOnPae9jJH1a0sPpPm6U1KPI+/1w+l7/mNbdTlIPSd9K3+ujkv6po3Mo6SpJj6X1r0nLPiBpZlo2U8kUh0iaIuk7ku4DvpmewxsK1l2fxvSUpNPT8i0kfU/J/Kp3S5rets6srIjwj38a7gcYDKwhmbFmC5KBA24iGYHnFOB/03rfAD6dvu4HLCWZc3MboFdaPhSYl75uAVYDg9L9PgiMKHL8WcAP0tdHk0yEUO545wGtwI7pum8C1xbsbweSGXh+B2ybll0CfC19vRy4MH39RWBy+no8cHG7/bRdsfoc8O309Q3ApenrE0lGKxpAMonDXcBW6brvAee2e689gaeAD6fL25M86jAa+Pe0bGuS2VKGlDqHwI7AEwXx9Ut/3wWMSl9/puDfbgpwN9AjXT4PuKFg3e3p/ocBy9Ly04HpafnOwMvA6Xl/Xv3T9X/8cLQ1sqcjfaZTyVigMyMilAz3NzitczxwsqSL0+VewPtJZq+4Qcm0cmtJZoBp83BEtKb7XZDu64Eix78FICJ+J2l7Sf3KHA9gRkS0Td33EdIh99J9vCzpYySJ4fdpQ64nSSJq84v093yg1Nimg4CfSRqYbt82U8kI4LT0WL+R9HJafhzJwBNz02P2Bl5ot889gRURMTfd/hVIWtHA/gUtv74kX0zeofg5nAO8BUyW9CuSRAlweMH7+THJbCNtbo+IUlN2/W8kg0Y81tYCT9/n7Wn582mr1qxDTqbWyAoHTlhXsLyODX8bAj4REU8UbqjkXuNK4ACSVsxbJfa7ltJ/Z+07LLSNTVvseIeSDPa9vqjI9iJJuGeXOF5bXOVi+i7wnYiYJqmFpOXatu9iBEyNiEtLrC8Va1v5hRFxz0aFyXE3OYeRDHp+CEkCPwsYQzIofnuFx3q9yPo2hcdQu99mm8X3TM3Kuwe4sOCe3YFpeV+S1tY64Bxgk/uEFTgz3ecIknFpV5c5Xnu/JUkmpPV2IGm5HSnpQ2nZNkrGMi3nVWC7guW+wF/S16MKyh8A/jHd7/Ekl4MBZgKnS3pfum5HSR9od4zHgV0kfTits52kLdP3+gVJW6Xle0jatlSg6f3fvpFMU3cRyWQBAH9gQyv9UxS/ClCpB4BPpPdOm0guOZt1yMnUrLyvA1sBj0palC5Dcm9wlKQ5JJd4y7WASnlZ0h+AicBnOzhee1cCOyjpvPRH4JiI+CvJfcFbJD1Kklz36iCGu4DT2jogkbREb5d0P8mYqG2uAI6X9AjJ9HErgFcj4jHg30k6RT1KMqXcwMIDRMQ7JF8cvpvGOoPk8vVk4DHgkfS93kj5q2XbAXenx5lNMmk0wJeB89Pyc4CvdPCey7mD5N50WzwPkdy/NSvLj8aY5UDSLJKOP/PyjqUSkrYG1qaXWg8Hvh9dcBqsziCpT0S8Jqk/8DDJWMHP5x2XdW2+Z2pmlXg/cJukLUg6CH0+53iq6e60M1hP4OtOpFYJt0zNzMwy8j1TMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDL6/xEcrFrYCoz/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mean_2, std_2 = simulation(1000, hazard_ratio)\n",
    "plot_results(mean_2, std_2, ylim=ylim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can observe that Harrell's c begins to overestimate performance starting with approximately 49% censoring while Uno's c is still underestimating the performance, but is on average very close to the actual performance for large amounts of censoring.\n",
    "\n",
    "For the final experiment, we double the size of the dataset to 2000 samples and repeat the analysis (*this may take several minutes to compute*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdMAAAFLCAYAAACNy2aDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAArnklEQVR4nO3de5xVdb3/8ddbEMEbKOiIYkIGKqlhTl7RJjUUKy+VR60M7UKUeI6UJ+l4Hkc8eTpm1vHGCdEMKo90McsLZerPIS0vQKGAoJKiTuIlFbxfgM/vj7WG2Qx7NntYs/eaPfv9fDz2Y9b6ru9a67O/Mn5mrfVd368iAjMzM9t0m+UdgJmZWa1zMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLqHfeAVTToEGDYujQoXmHYWZmNWr+/Pn/iIgd2pfXVTIdOnQo8+bNyzsMMzOrUZKeLFbu27xmZmYZOZmamZll5GRqZmaWUV09MzUz66neffddWlpaeOutt/IOpUfo27cvQ4YMYfPNNy+rvpOpmVkP0NLSwjbbbMPQoUORlHc4NS0iePHFF2lpaWHYsGFl7ePbvGZmPcBbb73FwIEDnUi7gCQGDhzYqat8J1Mzsx7CibTrdLYtnUzNzKxLbL311uutz5gxg4kTJ1bkXIXHnjJlCjNmzCh73wceeIDDDz+cPfbYgz333JMvfelLvPHGG5ni8TNTM7MeaOjkW7v0eMsv+liXHq+Y1atX07t37w7Xu8Jzzz3HSSedxKxZszj44IOJCG644QZeffVVttxyy00+rpOpmZlV3M0338yFF17IO++8w8CBA7nuuutoaGhgypQpPPPMMyxfvpxBgwYxYsSI9dYvu+wyJkyYwFNPPQXApZdeyqGHHrresbfeemv69esHwOWXX860adPo3bs3I0eOZNasWevVnTp1KuPGjePggw8Gktu5n/70pzN/PydTMzPrEm+++SajRo1at/7SSy9x3HHHATB69Gjuu+8+JHHNNddw8cUX8/3vfx+A+fPnc88999CvXz+mTJmy3vpnPvMZJk2axOjRo3nqqac4+uijWbJkyXrnPeecc9YtX3TRRTzxxBNsscUWrFy5coMYFy1axLhx47r8uzuZmplZl+jXrx8LFixYtz5jxox146G3tLRw8skns2LFCt555531Xjk57rjj1l1Ztl+/4447ePjhh9dte+WVV3j11Vc7jGHffffls5/9LCeccAInnHBCF32zjXMHJDMzq7izzjqLiRMnsnDhQq666qr1XjvZaqut1qtbuL527VruvfdeFixYwIIFC/j73//ONtts0+F5br31Vs4880zmz5/P/vvvz+rVq9fb/v73v5/58+d30bdqk2sylXSMpEckLZM0uch2Sbo83f6QpA+m5X0lPSDpQUmLJV1Q/ejNzKxcq1atYpdddgFg5syZZe83ZswYrrzyynXrhVe+7a1du5ann36aj3zkI1x88cWsXLmS1157bb06EydOZObMmdx///3ryn72s5/x7LPPlh1TMbklU0m9gKnAWGAkcKqkke2qjQWGp5/xwA/T8reBIyLiA8Ao4BhJB1UjbjMz67wpU6Zw0kkncdhhhzFo0KCy97v88suZN28e++67LyNHjmTatGkd1l2zZg2f+9zn2Geffdhvv/2YNGkSAwYMWK9OQ0MDs2bN4pxzzmGPPfZgr7324u6772bbbbfd1K8GgCIi0wE2+cTSwcCUiDg6Xf8WQET8d0Gdq4DmiLg+XX8EaIqIFQV1tgTuAb4aEfdTQmNjY3g+UzPriZYsWcJee+2Vdxg9SrE2lTQ/Ihrb182zA9IuwNMF6y3AgWXU2QVYkV7ZzgfeB0ztKJFKGk9yVUtDQwPNzc1dEryZWXfSv3//kh1zrPPeeuutsnNGnsm02FhN7S+TO6wTEWuAUZIGADdK2jsiFm1QOWI6MB2SK9OmpqYsMZuZdUtLliwp2THHOq9v377st99+ZdXNswNSC7BrwfoQ4JnO1omIlUAzcEyXR2hmZlaGPJPpXGC4pGGS+gCnADe1q3MT8Pm0V+9BwKqIWCFph/SKFEn9gKOApVWM3czMbJ3cbvNGxGpJE4HbgF7AtRGxWNKEdPs0YDZwLLAMeAM4I919MDAzfW66GfCLiLil2t/BzMwMch4BKSJmkyTMwrJpBcsBnFlkv4eA8m5km5mZVZhHQDIzs8yWL1/O3nvvvV7ZlClTuOSSSzbpeM3NzZx++ull13/22Wc55ZRT2H333Rk5ciTHHnssjz766Cade1N4bF4zs55oSv8uPt6qrj1eF4oITjzxRMaNG7dulpgFCxbw3HPPMWLEiKrE4CtTMzOruKamJs4991wOOOAARowYwd133w0k73KeccYZ60YtuuuuuwDo06cP/fsnfxDMmTOHUaNGMWrUKPbbb78N3qe966672HzzzZkwYcK6slGjRnHYYYdV6dv5ytTMzKpk9erVPPDAA8yePZsLLriAO+64g6lTpwKwcOFCli5dypgxY3j00Uc55JBDOOSQQwC45JJLmDp1KoceeiivvfYaffv2Xe+4ixYtYv/996/69ynkK1MzM8tMKjbGzvrln/zkJwHYf//9Wb58OQD33HMPp512GgB77rknu+222wbPOg899FC+/vWvc/nll7Ny5Up69+5+14FOpmZmltnAgQN5+eWX1yt76aWX1hvUfosttgCgV69e66ZGK2d8+MmTJ3PNNdfw5ptvctBBB7F06frDClRqWrXOcDI1M7PMtt56awYPHsydd94JJIn097//PaNHjy653+GHH851110HwKOPPspTTz3FHnvssV6dv/3tb+yzzz6ce+65NDY2bpBMjzjiCN5++22uvvrqdWVz585lzpw5XfHVyuJkamZmXeInP/kJF154IaNGjeKII47g/PPPZ/fddy+5z9e+9jXWrFnDPvvsw8knn8yMGTPWXcG2uvTSS9l77735wAc+QL9+/Rg7dux62yVx4403cvvtt7P77rvz/ve/nylTprDzzjt3+XfsSG5TsOXBU7CZWU/lKdi6XmemYPOVqZmZWUZOpmZmZhk5mZqZmWXkZGpm1kPUUx+YSutsWzqZmpn1AH379uXFF190Qu0CEcGLL764wUhLpXS/YSTMzKzThgwZQktLCy+88ELeofQIffv2ZciQIWXXdzI1M+sBNt98c4YNG5Z3GHXLt3nNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMck2mko6R9IikZZImF9kuSZen2x+S9MG0fFdJd0laImmxpH+pfvRmZmaJ3JKppF7AVGAsMBI4VdLIdtXGAsPTz3jgh2n5auAbEbEXcBBwZpF9zczMqiLPK9MDgGUR8XhEvAPMAo5vV+d44CeRuA8YIGlwRKyIiL8ARMSrwBJgl2oGb2Zm1irPZLoL8HTBegsbJsSN1pE0FNgPuL/rQzQzM9u4PCcHV5Gy6EwdSVsDNwBnR8QrRU8ijSe5RUxDQwPNzc2bFKyZmVlH8kymLcCuBetDgGfKrSNpc5JEel1E/Lqjk0TEdGA6QGNjYzQ1NWUO3MzMrFCet3nnAsMlDZPUBzgFuKldnZuAz6e9eg8CVkXECkkCfgQsiYgfVDdsMzOz9eV2ZRoRqyVNBG4DegHXRsRiSRPS7dOA2cCxwDLgDeCMdPdDgdOAhZIWpGX/FhGzq/gVzMzMAFBE+8eUPVdjY2PMmzcv7zDMzKxGSZofEY3tyz0CkpmZWUZOpmZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUZOpmZmZhk5mZqZmWXUYTKVtIOkkUXK3y9ph8qGZWZmVjtKXZleARRLmkOAyyoTjpmZWe0plUz3iYg57Qsj4jZg38qFZGZmVltKJdPNN3GbmZlZXSmVTB+TdGz7QkljgccrF5KZmVlt6V1i2yTgFkn/BMxPyxqBg4GPVzowMzOzWtHhlWlEPArsA8wBhqafOcC+6TYzMzOj9JUpEfE28OMqxWJmZlaTPGiDmZlZRk6mZmZmGTmZmpmZZdThM1NJC4HoaHtEeOAGMzMzSndAan395cz050/Tn58F3qhYRGZmZjWm1KsxT0bEk8ChEfHNiFiYfiYDR3fFySUdI+kRScskTS6yXZIuT7c/JOmDBduulfS8pEVdEYuZmdmmKueZ6VaSRreuSDoE2CrriSX1AqYCY4GRwKlFZqkZCwxPP+OBHxZsmwEckzUOMzOzrEq+Z5r6InCtpP4kz1BXAV/ognMfACyLiMcBJM0CjgceLqhzPPCTiAjgPkkDJA2OiBUR8UdJQ7sgDjMzs0w2mkwjYj7wAUnbAoqIVV107l2ApwvWW4ADy6izC7Cii2IwMzPLbKPJVFID8B1g54gYm96KPTgifpTx3CpS1r73cDl1Sp9EGk9yi5iGhgaam5s7s7uZmdlGlXObdwbJkILnpeuPAj8HsibTFmDXgvUhwDObUKekiJgOTAdobGyMpqamTgdqZmZWSjkdkAZFxC+AtQARsRpY0wXnngsMlzRMUh/gFOCmdnVuAj6f9uo9CFgVEb7Fa2Zm3Uo5yfR1SQNJb6+2JrWsJ06T8kTgNmAJ8IuIWCxpgqQJabXZJHOnLgOuBr7Wur+k64F7gT0ktUj6YtaYzMzMNoWSjrIlKiTvdl4B7A0sAnYAToqIBysfXtdqbGyMefPm5R2GmZnVKEnzI6KxfXk5z0wXAx8G9iDpEPQIHtPXzMxsnXKS4r0RsToiFkfEooh4l+T2qpmZmVF6oPudSN7p7CdpP9peU9kW2LIKsZmZmdWEUrd5jwZOJ3kd5QcF5a8C/1bBmMzMzGpKh8k0ImYCMyV9KiJuqGJMZmZmNaWc4QRvkPQx4P1A34Ly/6xkYGZmZrViox2QJE0DTgbOInluehKwW4XjMjMzqxnl9OY9JCI+D7wcERcAB7P+EH9mVmFNTU14KMyE28K6o3KS6Zvpzzck7Qy8CwyrXEhmZma1pZxBG26RNAD4HvAXkmEFr6lkUGZmZp3Rercir5nByumA9O108QZJtwB9u3BOUzMz20R5JxBrU858pr2AjwFDW+tLIiJ+UGo/MzOzelHObd6bgbeAhaTTsJmZmVmbcpLpkIjYt+KRmJmZ1ahyevP+TtKYikdi1o5fgTCzWlHOlel9wI2SNiN5LUZARMS2FY3MzMysRpSTTL9PMlDDwtjYTOJmZmZ1qJzbvI8Bi5xIzTaNb1eb9XzlXJmuAJol/Q54u7XQr8aYmZklykmmT6SfPunHzMzMCpQzAtIF1QjEzMysVnWYTCVdGhFnS7qZZDze9UTEcRWNzMzMrEaUujL9afrzkmoEYmY9n8eStWJ6wr+LDpNpRMxPF0dFxGWF2yT9CzCnkoFZHZjSv/T25a9vvN4Uz7lgZvkrpwPSOOCydmWnFymzLtAT/kKDnvM9qsZ/WJjVtFLPTE8FPgMMk3RTwaZtgRcrHZiZmVmtKHVl+meSd0wHkYyC1OpV4KFKBmVmdcxX6VaDSj0zfRJ4UtJRwJsRsVbSCGBPkunYzMzMjPKemf4ROEzSdsCdwDzgZOCzlQysFpX1nNB/dZuZ9TjljM2riHgD+CRwRUScCIzsipNLOkbSI5KWSZpcZLskXZ5uf0jSB8vd18zMrFrKSqaSDia5Er01LSvninZjB+0FTAXGkiTnUyW1T9JjgeHpZzzww07sa2bWbXkChJ5FG5sMRtLhwDnAnyLiu5LeC5wdEf+c6cRJgp4SEUen698CiIj/LqhzFdAcEden648ATcDQje1bzLBhw+L888/PEnZJCxYsAGDUqFEdV1p+T+ljPLsmOcZOvTquNHR0JyOrvq5oi7LUSVv430XBMeqpLapwjO6gltrijDPOmB8Rje3Lyxmb948kz01b1x+X9PUuiGkX4OmC9RbgwDLq7FLmvgBIGk9yVcvgwYNZuXJlpqBff3t1h9veefddAP7+3D86rLPVgL1LHn/1P5YBsHLA+zqulPE7dJVKt0VZ6qQt/O+iTV21hd4qefzV7yYTea18vqXjSn22LnmMaijVDtAz2qLUe6b3RMTodPmnEXFaweYHgA8W37NsKlLW/jK5ozrl7JsURkwHpgM0NjbG2Wef3YkQO6f1ls3/Tp1asXPUinpri6GTb+1w27M3zwRgp88c22Gd5Rd9rOTxW9vzyiuv7HxwVea2aFO1tpj6w84HV0Wl2gHKbIu+nyl5jKZbk86bV56+VceVvpm98+akSZOKlpe6Mi2MqP2fisWSWWe1ALsWrA8BnimzTp8y9jUzM6uKUsk0Olgutr4p5gLDJQ0D/g6cQjLiUqGbgImSZpHcxl0VESskvVDGvmZVU+oKoum+7wHQvJGrDDOrXaWS6QBJJ5L0+B0g6ZNpuYCNvCy5cRGxWtJE4DagF3BtRCyWNCHdPg2YDRwLLAPeAM4otW/WmMzMrJva2Pv1zU1pveZKR1JUqWQ6BziuYPkTBdv+uGH1zouI2SQJs7BsWsFyAGeWu6+ZmVkeSg0neEY1A+kJPEOKmVl9KmfQBjMzMysh80hGZlZ5vuth1r35ytTMzCyjTl2ZSpoeEeMrFYz1HL6Sql9+TcjqUWevTDcYj9DMzKzedTaZPl+RKMzMzGpYp27zRsQxlQrEzKwcfoRg3ZE7IJmZmWXkZGpmZpaRk6mZmVlGHSZTSRe3DjrfrnySpO9WNiwzM7PaUerK9OOkk2q3cxngl8TMzMxSJeczjYi1RQrXSuqKycHNzHo0D2CRKNUO0DPaotSV6RuShrcvTMverFxIZmZmtaXUlel/AL+TdCEwPy1rBL4FnF3huMzMzGpGqflMfyfpBOBfgbPS4kXApyJiYRViM+sRPMiAWc9XcgSkiFgEjKtSLGZmZjXJ85mamdUo3/XoPjxog5mZWUZOpmZmZhl1eJtX0hVAdLQ9Iv65IhGZmZnVmFLPTOdVLQozM7MaVurVmJnVDMTMzKxWbbQ3r6QdgHOBkUDf1vKIOKKCcZlZD+Tep9ZTlfNqzHXAz0kGt59A8t7pC5UMyszMrDPy/kOtnN68AyPiR8C7ETEnIr4AHFThuMzMzGpGOVem76Y/V0j6GPAMMKRyIZmZmdWWcq5ML5TUH/gGcA5wDTApy0klbS/pdkmPpT+366DeMZIekbRM0uSC8pMkLZa0VlJjlljMzMyy2mgyjYhbImJVRCyKiI9ExP4RcVPG804G7oyI4cCd6fp6JPUCpgJjSTo/nSppZLp5EfBJ4I8Z4zAzM8usnN68P6bI4A3ps9NNdTzQlC7PBJpJegwXOgBYFhGPp3HMSvd7OCKWpGUZQjAzM+sa5TwzvaVguS9wIslz0ywaImIFQESskLRjkTq7AE8XrLcAB3b2RJLGA+MBGhoacu/xZWYGsHLlSiD/XqjdQU9oi40m04i4oXBd0vXAHRvbT9IdwE5FNp1XZmzFLjs7HN6wIxExHZgO0NjYGE1NTZ09hJlZlxswYAAA/n9Sz2iLTZmCbTjwno1VioijOtom6TlJg9Or0sHA80WqtQC7FqwPIfsVsZmZWZfbaAckSa9KeqX1A9zMhs83O+sm2iYdHwf8tkiducBwScMk9QFOSfczMzPrVsq5zbtNBc57EfALSV8EngJOApC0M3BNRBwbEaslTQRuA3oB10bE4rTeicAVwA7ArZIWRMTRFYjTzKwiavn5oG2onN68d0bEkRsr64yIeBHYYP+IeAY4tmB9NjC7SL0bgRs39fxmZmZdqdR8pn2BLYFB6aAKrR2CtgV2rkJsZmZmNaHUlelXgLNJEud82pLpKySDKZiZmRml5zO9DLhM0lkRcUUVYzIzM6sp5YzNu1bSgNYVSdtJ+lrlQjIzM6st5STTL0fEytaViHgZ+HLFIjIzM6sx5STTzVQwCG46AH2fyoVkZmZWW8oZAek2kndCp5EM5zcB+H1FozIzs7rRE965LSeZnksyUPxXSXr0/gG4upJBmZmZ1ZJy5jNdGxHTIuLTEfEpYDHJ6ENmZmZGmQPdSxoFnAqcDDwB/LqCMZmZmdWUUiMgjSAZXP5U4EXg54Ai4iNVis3MzKwmlLoyXQrcDXwiIpYBSJpUlajMzMxqSKlnpp8CngXuknS1pCMpPmG3mZlZXeswmUbEjRFxMrAn0AxMAhok/VDSmCrFZ2Zm1u2V05v39Yi4LiI+DgwBFgCTKx2YmZlZrShnBKR1IuKliLgqIo6oVEBmZma1plPJ1MzMzDbkZGpmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUZOpmZmZhnlkkwlbS/pdkmPpT+366DeMZIekbRM0uSC8u9JWirpIUk3ShpQteDNzMzayevKdDJwZ0QMB+6kyJRuknoBU4GxwEjgVEkj0823A3tHxL7Ao8C3qhK1mZlZEXkl0+OBmenyTOCEInUOAJZFxOMR8Q4wK92PiPhDRKxO691HMs+qmZlZLvJKpg0RsQIg/bljkTq7AE8XrLekZe19Afhdl0doZmZWpt6VOrCkO4Cdimw6r9xDFCmLduc4D1gNXFcijvHAeICGhgaam5vLPL2ZmVl5KpZMI+KojrZJek7S4IhYIWkw8HyRai3ArgXrQ4BnCo4xDvg4cGREBB2IiOnAdIDGxsZoamrq1PcwMzPbmLxu894EjEuXxwG/LVJnLjBc0jBJfYBT0v2QdAxwLnBcRLxRhXjNzMw6lFcyvQj4qKTHgI+m60jaWdJsgLSD0UTgNmAJ8IuIWJzufyWwDXC7pAWSplX7C5iZmbWq2G3eUiLiReDIIuXPAMcWrM8GZhep976KBmhmZtYJHgHJzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzy8jJ1MzMLCMnUzMzs4ycTM3MzDJyMjUzM8vIydTMzCwjJ1MzM7OMnEzNzMwycjI1MzPLyMnUzMwsIydTMzOzjJxMzczMMnIyNTMzyyiXZCppe0m3S3os/bldB/WOkfSIpGWSJheUf1vSQ5IWSPqDpJ2rF72Zmdn68roynQzcGRHDgTvT9fVI6gVMBcYCI4FTJY1MN38vIvaNiFHALcB/VCVqMzOzIvJKpscDM9PlmcAJReocACyLiMcj4h1gVrofEfFKQb2tgKhcqGZmZqX1zum8DRGxAiAiVkjasUidXYCnC9ZbgANbVyT9F/B5YBXwkQrGamZmVlLFkqmkO4Cdimw6r9xDFClbdwUaEecB50n6FjAROL+DOMYD4wEaGhpobm4u8/RmZmblqVgyjYijOtom6TlJg9Or0sHA80WqtQC7FqwPAZ4pUu//gFvpIJlGxHRgOkBjY2M0NTWV9wXMzMzKlNcz05uAcenyOOC3RerMBYZLGiapD3BKuh+ShhfUOw5YWsFYzczMSsrrmelFwC8kfRF4CjgJIH3F5ZqIODYiVkuaCNwG9AKujYjFrftL2gNYCzwJTKj6NzAzM0spon46wjY2Nsa8efPyDsPMzGqUpPkR0di+3CMgmZmZZeRkamZmlpGTqZmZWUZOpmZmZhk5mZqZmWXkZGpmZpaRk6mZmVlGTqZmZmYZOZmamZll5GRqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmllFdTcEm6QWS+U/zNAj4R84xdBduizZuizZuizZuizbdpS12i4gd2hfWVTLtDiTNKzYXXj1yW7RxW7RxW7RxW7Tp7m3h27xmZmYZOZmamZll5GRafdPzDqAbcVu0cVu0cVu0cVu06dZt4WemZmZmGfnK1MzMLCMnUzMzs4ycTM3MzDLqnXcA9UbSjhHxfN5xdCeS9oyIpXnHUW2SGoFdgdXAY3XaBn2AdyPtvCHpI8AHgYcj4ne5BldlkvaNiIfyjqM7qaXfEXdAqiBJ27cvAuYD+5G0/UvVj6r7kfRURLwn7ziqRdKHge8DK4H9gT8B2wHvAqdFxNP5RVddkh4EmiLiZUn/CpwIzAY+DMyLiG/lGmAVSVoDPAFcD1wfEQ/nHFJuavF3xMm0giStZcPhC4cALUBExHurH1U+JF3e0SZgXERsW8148iTpr8CYiHhB0jDgBxFxoqSPAv8aEWNyDrFqJC2KiL3T5XnAYRHxpqTewF8iYt98I6ye9N/FacCpwMnA6ySJdVZELM8xtKqrxd8RPzOtrG8CjwDHRcSwiBgGtKTLdZNIU2cAi0iuzAs/84B3cowrD70i4oV0+SlgN4CIuB3YJbeo8vGKpL3T5X8AfdPl3tTf/58iIhZFxHkR8T7gy8COwN2S/pxzbNVWc78jfmZaQRFxiaRZwP9Ieho4H6jXWwFzgUURscH/FCRNqX44uZon6UfAncDxQDOApC2BXjnGlYcJwHXp7d7nSdpmDrAv8J1cI6s+Fa5ExAPAA5K+ARyeT0i5qbnfEd/mrRJJnwDOA4ZGxE55x1Nt6fPjtyLijbxjyZukzUmuOkYCDwLXRsQaSf2AHSMi75mNqkpSL2AMMILkD/wW4LaIWJlnXNUm6TMR8X95x9Ed1OLviJNpFaX/EHaPiEV5x2JmVkskDYyIF/OOoyP19kwiVxHxZmsilXRG3vFUk6StJf2npMWSVkl6QdJ9kk7PO7Zqk9Qo6S5JP5O0q6Tb0zaZK2m/vOPrLiTV26sx/SVdJGmppBfTz5K0bEDe8VVT+p0HpcuNkh4H7pf0ZNrTt9vxlWlO6vB1kN8CNwJ3AP8EbAXMAv4d+HtE/FuO4VWVpAdInp8PAC4GJkXEryQdCVwYEQfnGV81SfpgR5uAWyJicDXjyZOk24D/B8yMiGfTsp2AccBREfHRPOOrJkkLI2KfdPku4JsRMVfSCOD/uuO8pk6mFSSpoxewBYyIiC2qGU+eJD0YER8oWJ8bER+StBnJC/p75hheVUn6a0Tsly6v90dV4bZ6kL5bOYd2nW9SB0VEvyqHlBtJj0TEHp3d1hNJWgrsHRGrJd0XEQcVbFuXaLsT9+atrAbgaODlduUC6q2r++uSRkfEPWlnrJcAImKtpGL/I+3J3pI0BugPhKQTIuI36e2rNTnHVm1LgK9ExGPtN6Q94OvJk5K+SXJl+hyApAbgdKDe2mIqMFvSRcDvJV0K/Bo4EliQY1wdcjKtrFuArSNiQfsNkpqrHk2+JgDXpLdpFgFfBJC0A8kvTj2ZQHJ7dy3JH1tflTQD+DswPse48jCFjvtunFXFOLqDk4HJwBxJO6ZlzwE3kTwaqRsRcYWkhcBXaevlPQL4DXBhjqF1yLd5zczMMnJv3gpLh0VrXd467ZnWfszeHk/SgZK2TZf7SbpA0s2Sviupf97xVZukPSUdKWnrduXH5BVTXtwWbQraYqt25XXXFh3prm9COJlWUPrax3OSHpU0FngI+C7woKRTcw2u+q4FWgdsuIzkeeF307If5xVUHiT9M/BbktuYiyQdX7C5rkb9cVu0adcWi+u5LTbigrwDKMbPTCvrG8AewDYko3jsFxF/SzsV3E4yiHW92CwiVqfLjRHR+krEPZIW5BRTXr4M7B8Rr0kaCvxK0tCIuIzivVp7MrdFG7dFaiNvQjRUM5ZyOZlW1pqI+AfwD0mvRcTfACLiufrrwMoiSWdExI9JrswbI2Je2iHp3byDq7JeEfEaQEQsl9RE8j/O3aiz/2nitijktmhTc29C+DZvZT0l6b8lXQkslfR9SYdKOh9YkXdwVfYl4MOS/kYy3ua96agmV6fb6smzkka1rqT/A/04MAjodu/PVZjboo3bok3rmxBPtvssJx30vrtxb94KSjvcnEkyU8yVJH9pnUEyx+mFEVFvCRVJ2wDvJR3QvPV9unoiaQiwunWUm3bbDo2IP+UQVi7cFm3cFrXNydTMzCwj3+bNiaR6ezm/Q5JuyTuG7sJt0cZt0cZt0aa7toWvTHMi6SsRcVXecXQHkgbX4y3vYtwWbdwWbdwWbbprWziZVpik3YETgV2B1cBjwPURsSrXwLoBSTtGxPN5x2FmlpVv81ZQ+hL2NKAv8CGgH0lSvTft9l43JG3f7jMQeEDSdvU2IpQ8t+s6bos2bos2tdgWvjKtICUDNY+KiDWStgRmR0STpPcAv62zqbbWkvRiLjQEaAEiIt5b/ajyIc/tuo7boo3bok0ttoWTaQWlybQxIt6WtB1wR0Tsn25bFBF75xth9Ug6BzgK+NeIWJiWPRERw/KNrPrkuV3XcVu0cVu0qcW28G3eyroGmCtpOnAvybumrdOOvZRnYNUWEZeQDM7wH5J+kL5vWq9/yb0uaTSA2s3tSv2NdOO2aOO2aFNzbeHhBCsoIi6TdAewF/CDiFialr8AHJ5rcDmIiBbgpPSX43Zgy5xDystXgavVNrfrF6Bu53ZtbYs9gIXU9zy3/nfRpv38x92+LXyb16pG0p7ALsD9wBpg94hYJOmYiPh9vtFVl6QDgbURMVfSSOAYYGlEzM45tNxJ+klEfD7vOPKWXpkdACyKiD/kHU81pb8fSyNiVdrf5Fzgg8DDwHe649sQTqZWFWnP5jOBJcAo4F8i4rfptr8UzCLT4ykZm3ksyZ2h24EDScYbPQq4LSL+K7/oqkvSTUWKjwD+H0BEHFfdiPIj6YGIOCBd/hIwkaQTzhjg5oi4KM/4qknSYuADEbE6fUz2BvAr4Mi0/JO5BliEk6lVRdoZ6+DC6aWAn6a3wv9aZz2bF5L8QbEF8CwwJCJekdQPuD8i9s0zvmqS9FdgMUn/giB5HnY9cApARMzJL7rqKvw9kDQXODYiXlAyUfh9EVE3g91LWhIRe6XL6/2xLWlBRIzKLbgOuAOSVct600sBTcBYST+gm3YoqKDVEbEmIt4A/hYRrwBExJvA2nxDq7r9gfnAecCqiGgG3oyIOfWUSFObpe9dDyS50HkBICJeJxnwpZ4sknRGuvygpEYAdeMpG51MrVo8vVSbd9LnQJAkEwAk9afOkmlErI2I/yGZTek8JdMV1mvHyP4kf1jMA7aXtBMkAxhQf39w1tyUjb7Na1UhTy+1jqQtIuLtIuWDgMGt7+HWI0kfAw7tji/l5yX9w6shIp7IO5ZqUw1N2ehkamZmlpFv85qZmWXkZGpmZpaRk6lZHZPUJOmQvOOoBknHSZqcdxzWM/mZqVmNkdQ7IrrkVQlJU4DX0rGTe6yubDOzYnxlanVJ0lBJSyVdI2mRpOskHSXpT5Iek9Q6Es1Wkq6VNFfSXyUdX7D/3ZL+kn4OScubJDVL+lV6/OskbfBaQ1rnUkl/Ts+/sfOdLumXkm4G/qBkvscfS1oo6SFJn0rrjZF0bxrTL9PXKpC0XNIFaflCSXumg2dMACZJWiDpMEmfkHR/eu47JDWk++8g6fZ0/6skPZn2PkbS5yQ9kB7jKkm9inzfD6Xf9cG07jaSekn6XvpdH5L0lY21oaSLJD2c1r8kLdtN0p1p2Z1KpjhE0gwlkyrcBXw3bcMrC7Zdnsb0uKRPp+WbSfpfJfNo3iJpdus2s5Iiwh9/6u4DDCV5EX4fkj8q5wPXkrzPdzzwm7Ted4DPpcsDgEdJ5lbcEuiblg8H5qXLTcAqkrlaNyOZLWh0kfM3A1eny4eTjL9a6nynk8z9un267bvApQXH247knd0/AlulZecC/5EuLwfOSpe/BlyTLk8Bzml3nNY7Vl8Cvp8uXwl8K10+hmS0okEkkzjcDGyebvtf4PPtvmsf4HHgQ+n6tiSvOowH/j0t24Lk/cphHbUhsD3wSEF8A9KfNwPj0uUvFPy3mwHcQjJgCGkbXlmw7Zfp8UcCy9LyTwOz0/KdgJeBT+f979Wf7v+p15ejzQCeiLa5VRcDd0ZEKBnub2haZwxwnJL5WAH6Au8BngGuVDIQxRpgRMFxH4hkhhwkLUiPdU+R818PEBF/lLStpAElzgdwe0S0Tt13FOmQe+kxXpb0cZLE8Kf0Qq4PSSJq9ev053ygo7FNhwA/lzQ43b/13cbRwInpuX4v6eW0/EiSgSfmpufsBzzf7ph7ACsiYm66/yuQXEUD+xZc+fUn+cPkHYq34X3AWySzidxKkigBDi74Pj8FLi449y8jYk0H3/U3kUzp9XDrFXj6PX+Zlj+bXtWabZSTqdWzwoET1hasr6Xtd0PApyLikcIdlTxrfA74AMlVzFsdHHcNHf+ete+w0Do2bbHzHQi8XlhUZH+RJNxTOzhfa1ylYrqCZLrAmyQ1kVy5th67GAEzI+JbHWzvKNbW8rMi4rb1CpPzbtCGkQx6fgBJAj+FZCD4I4oct/BcrxfZ3qrwHGr306xT/MzUrLTbgLMKntm1Dsjfn+Rqay1wGrDBc8IynJweczTJuLSrSpyvvT+QJBPSetuRXLkdKul9admWSsYyLeVVYJuC9f7A39PlcQXl9wD/lB53DMntYIA7gU9L2jHdtr2k3dqdYymws6QPpXW2kdQ7/a5flbR5Wj5CyaDuRaXPf/tHMk3d2SSTBQD8mbar9M9S/C5Aue4BPpU+O20gueVstlFOpmalfRvYHHhI0qJ0HZJng+Mk3Udyi7fUFVBHXpb0Z2Aa6aTYJc7X3oXAdko6Lz0IfCSSgdFPB66X9BBJct1zIzHcDJzY2gGJ5Er0l5LuBv5RUO8CYIykv5BMH7cCeDUiHgb+naRT1EMkU8oNLjxBRLxD8ofDFWmst5Pcvr6GZH7Kv6Tf9SpK3y3bBrglPc8cYFJa/s/AGWn5acC/bOQ7l3IDybPp1njuJ3l+a1aSX40xy4GkZpKOP/PyjqUckrYA1qS3Wg8GfhjdcBqsriBp60imChwIPEAyVvAGY0qbFfIzUzMrx3uAX0jajKSD0JdzjqeSbkk7g/UBvu1EauXwlamZmVlGfmZqZmaWkZOpmZlZRk6mZmZmGTmZmpmZZeRkamZmlpGTqZmZWUb/H+ZNLEqIpHcfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mean_3, std_3 = simulation(2000, hazard_ratio)\n",
    "plot_results(mean_3, std_3, ylim=ylim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The trend we observed in the previous simulation is now even more pronounced.\n",
    "Harrell's c is becoming more and more overconfident in the performance of the synthetic marker with increasing amount of censoring, while Uno's c remains stable.\n",
    "\n",
    "In summary, while the difference between [concordance_index_ipcw](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw)\n",
    "and [concordance_index_censored](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored) is negligible for small amounts of censoring, when analyzing survival data with moderate to high amounts of censoring, you might want to consider estimating the performance using `concordance_index_ipcw` instead of `concordance_index_censored`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Time-dependent Area under the ROC\n",
    "\n",
    "The area under the [receiver operating characteristics curve](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) (ROC curve) is a popular performance measure for binary classification task. In the medical domain, it is often used to determine how well estimated risk scores can separate diseased patients (cases) from healthy patients (controls). Given a predicted risk score $\\hat{f}$, the ROC curve compares the false positive rate (1 - specificity) against the true positive rate (sensitivity) for each possible value of $\\hat{f}$.\n",
    "\n",
    "When extending the ROC curve to continuous outcomes, in particular survival time, a patient’s disease status is typically not fixed and changes over time: at enrollment a subject is usually healthy, but may be diseased at some later time point. Consequently, sensitivity and specificity become [time-dependent measures](http://dx.doi.org/10.1111/j.0006-341x.2000.00337.x).\n",
    "Here, we consider *cumulative cases* and *dynamic controls* at a given time point $t$, which gives rise to the time-dependent cumulative/dynamic ROC at time $t$.\n",
    "Cumulative cases are all individuals that experienced an event prior to or at time $t$ ($t_i \n",
    "\\leq t$), whereas dynamic controls are those with $t_i>t$.\n",
    "By computing the area under the cumulative/dynamic ROC at time $t$, we can determine how well a model can distinguish subjects who fail by a given time ($t_i \\leq t$) from subjects who fail after this time ($t_i>t$).\n",
    "Hence, it is most relevant if one wants to predict the occurrence of an event in a period up to time $t$ rather than at a specific time point $t$.\n",
    "\n",
    "The [cumulative_dynamic_auc](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.cumulative_dynamic_auc.html#sksurv.metrics.cumulative_dynamic_auc) function implements an estimator of the cumulative/dynamic area under the ROC at a given list of time points.\n",
    "To illustrate its use, we are going to use data from a [study](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3538473/) that investigated to which extent the serum immunoglobulin free light chain (FLC) assay can be used predict overall survival. The dataset has 7874 subjects and 9 features; the endpoint is death, which occurred for 2169 subjects (27.5%).\n",
    "\n",
    "First, we are loading the data and split it into train and test set to evaluate how well markers generalize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = load_flchain()\n",
    "\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Serum creatinine measurements are missing for some patients, therefore we are just going to impute these values with the mean using scikit-learn's `SimpleImputer`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_columns = [\"age\", \"creatinine\", \"kappa\", \"lambda\"]\n",
    "\n",
    "imputer = SimpleImputer().fit(x_train.loc[:, num_columns])\n",
    "x_test_imputed = imputer.transform(x_test.loc[:, num_columns])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to Uno's estimator of the concordance index described above, we need to be a little bit careful when selecting the test data and time points we want to evaluate the ROC at, due to the estimator's dependence on inverse probability of censoring weighting.\n",
    "First, we are going to check whether the observed time of the test data lies within the observed time range of the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_events = y_train[y_train[\"death\"]]\n",
    "train_min, train_max = y_events[\"futime\"].min(), y_events[\"futime\"].max()\n",
    "\n",
    "y_events = y_test[y_test[\"death\"]]\n",
    "test_min, test_max = y_events[\"futime\"].min(), y_events[\"futime\"].max()\n",
    "\n",
    "assert (\n",
    "    train_min <= test_min < test_max < train_max\n",
    "), \"time range or test data is not within time range of training data.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When choosing the time points to evaluate the ROC at, it is important to remember to choose the last time point such that the probability of being censored after the last time point is non-zero. In the simulation study above, we set the upper bound to the maximum event time, here we use a more conservative approach by setting the upper bound to the 80% percentile of observed time points, because the censoring rate is quite large at 72.5%.\n",
    "Note that this approach would be appropriate for choosing `tau` of  [concordance_index_ipcw](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw) too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 470.3        1259.         1998.         2464.82428571 2979.\n",
      " 3401.         3787.99857143 4051.         4249.         4410.17285714\n",
      " 4543.         4631.         4695.         4781.         4844.        ]\n"
     ]
    }
   ],
   "source": [
    "times = np.percentile(y[\"futime\"], np.linspace(5, 81, 15))\n",
    "print(times)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We begin by considering individual real-valued features as risk scores without actually fitting a survival model. Hence, we obtain an estimate of how well *age*, *creatinine*, *kappa* FLC, and *lambda* FLC are able to distinguish cases from controls at each time point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdMAAAEyCAYAAACh5jqKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABg10lEQVR4nO3dd3hUVfrA8e/JTJJJrwQSCIReQi8iRUAQwUUEe69rXwu6upZdFF3X3nX9YUd3de2KWECRXqQrVXpLgQDpPTNzfn/cSTKTTJJJmdT38zzzzMw9t5y5DHnndKW1RgghhBB159PUGRBCCCFaOgmmQgghRD1JMBVCCCHqSYKpEEIIUU8STIUQQoh6kmAqhBBC1JO5qTPQkKKjo3VCQkJTZ0MIIUQLtGnTppNa63Z1ObZVBdOEhAQ2btzY1NkQQgjRAimlDtf1WKnmFUIIIepJgqkQQghRTxJMhRBCiHqSYCqEEELUkwRTIYQQop4kmAohhBD1JMFUCCGEqKdWNc5UCCFE2/HNlmSeW7SblMwC4sIDuH9Kb2YO6dgkeZFgKoQQosX5ZksyD321jYISGwDJmQU89NU2gCYJqFLNK4QQosV5btHuskBaqqDExnOLdjdJfiSYCiGEaHFSMgtqtd3bJJgKIYRocWLDLW63x4UHNHJODBJMhRCihflmSzJjnl5C1we/Z8zTS/hmS3JTZ6nRTeoTU2lbgK+J+6f0boLcSAckIYRoURqi401z6gVbF4UlNpb8cYKO4RY0kJpZ2OSfQ4KpEEJ4SV2CVrHVzqm8Ik7kGI+Tuc6vi/l553GKbXaXYwpKbPzti618+3sKIRYzoRZfQixmQsqey7dtOJTOy4v3UmQ1ztHUvWDr4r3VB0nOLODjm0Yyunt0U2cHkGAqhBBe4a4E+cCXW9mZmkWv9qGVguXJ3CJO5BaRmV/i9nwhFjPtQvwrBdJSxTY7x7IK2ZtWQm6hlZxCK1a79iivBSU2/vndTs7sHUNYoG/dPnAjOZFTxBtL9zO5X/tmE0hBgqkQQjQYrTVJGQXsSMlm9vztlYZuFFntvLXiYNn7QD8T7UL8iQ72p3u7YE7vFkV0sD/tQvwd2/3K0i2+JgDGPL2EZDc9VjuGB/DD3We45KWwxE5OYQnZhVZyCkvIKbRyzXvr3eb9VF4xgx7/id7tQxieEGE8ukTSKSIApRTQcNXDVZ2nuvOXppV+9hEJEbW+rjdJMBVCiDoostrYezyXnanZ7EzJZmdqNrtSsskpslZ7nAKW3T+B6GB/gvxr/yf4/im9XUq84L7jjVKKAD8TAX4mYkLLt3cMD3AbjKOD/bh2VAIbDmcw/7cUPlp3BIAOoRaGJ0TgZ/bh+62p9a4erqrNd+PhdL7clOy2LRio9Jlf+nkvMSGWZlM1rbT2rBqgJRg+fLjeuHFjU2dDCNHKZOYXuwTNnSnZ7EvLLatGDfA10Tc2hH5xofSLDSMxLpTb/ruJlKzCSufqGB7A6gcn1is/9SkhVgxmpfl/6oIBZeew2TV/HMtm0+EMNhzKYMPBdI5lV/4sxuexsPrBSR7nffRTv7i9L1UJ8jOhgfxiW6W0hriXzpRSm7TWw+t0rARTIYQwlFbTlgbMHSnZ7ErNdinJxYT4O4JmaNlzl6ggTD7K5VyeBK2mUttgrLWm20M/UFW0uHhYJ8b2jGZsj2iigv3dXuOW8V05kVPMa0v2NdjnUMDBp6c13PkkmBokmDaMlt5tXohS1X2Xi6129qbluJQ2d6Zmk1NoVNMqBd2ig+gXF+YSONuF+DfI9VuaqtpqLb4++Jl8yHbct8S4UNqH+rNq7ym3naUsZh8KrZW3m5TC5iYedXRMwlBVO7GUTL1Agmn9Nedf00LUhrvvsq9JMbhTOLnFNval5VBiK6+m7RMb4hI0e3cIIdBPupWUqu5vw/RBcWxLzmLlnhOs3HeS9QfT3Z4jJsSfh//U1+15LhzW0aXN1Pn8ULnN1Bt/l+oTTOWbIgCjjWRHShaPuOmBWNptflyvdkQG+TVRDoVwz2qzcyK3iNSsQo6VPrIL+c/aw5W+yyU2zaYjGYzt2Y7xvdqVBc6u0ZWraYWr0qBVVUl7cHw4g+PDuXNST7o++L3bKuETOUXVnmd4l8hqS/LNuZQvJdM2SmvN3rRc1uw7yer9p/j1wKmy6q3qdAwPoH/HUPrHhdG/o/GoTbWXELVRZLWRlm0EytSsAo5lFZYHzWzjOS2nkIrDKf3MPhS7qUqEhm9nE5VVN3ynIatlG5qUTIVHjpzKZ83+k6zZf4o1+09xMrcIgPjIAKYNiGVU9yie+uEPt732ooP9uHlcN7YlZ7MjOYtFO46XpbUP9XcJrgM6htE+1L9sbJpoO2rTRphfbC0LjKlZhRzPrhwwT+UVVzouyM9EbHgAHUItjO0ZTWyYhQ5hFmLDLLQPtRAbFkBEoC9jn1nq9g96U02E3pZ4OnynNZFg2oodzy5k7f5TZQE0KcP4w9IuxJ8xPaIY0z2aUd2jiI8MLDtGa/dtE/+Y1s/lj2JOYQk7U7LZnmIE123JWSzdnVZWQogO9iMxzgis/TuGkhgX5jL4W7Q+7sYP/u2Lrazdf5IOYQFGkMwu5LijlJntpiYkItCXDmEBdAj1Z2Cn8LJA2SHUUvY6xOLZDD1t8Q96c1FTlXBr1KqqeSO79NWTH37PZdu5A2O5elQCBcU2rnu/8swfFw3rxMXD40nPK+a2/26qlH7V6V2YPiiOlMwC7vn0t0rpN53RjbP6tWf/iVwedhpgXOrOiT0Z2zOaHSlZPL5gZ6X0v03tzbAukWw6nM6zCysvavvI9H4kxoWxau9JXluyt1L6kxcMoHu7YBbvPM7/Ld9PdkEJ2YUlZBWUUFhiVHOFBfjSJSqQ7IISQgN8CXDMpALwf1cNIzLIj883HuWLTUkAnMwt4mh6AcU2O7FhFh6Y2oecwhK+25pa6fqf3jIKgLdW7OfnncfJL7aRV2Qlr8hKfrGNQqsdmyPCmnwUQX4mgvzNBPmbiQuzMO/60/DxUTyz8A82H85wOXdsmIWXLxsCwGMLdrAzJdslvVu7IJ66YCAAD321lQMn8lzS+8WF8uj0RABmfbKF1Apj24Z2ieCBqX0AuPU/m8jIdy0FjekRzV2TegJw7XvrKazQ/japbww3j+sOwKVvrq10b9rKd++nHcd4bck+dqZml/1bu+NrUviZfPAzG49LhsfTvV0wfxzLZu3+U/iZffBx+rE17/rTCPAz8Z+1h2r87v2yK80lzeJr4oMbTuObLcnMnr+dnEIrfiYf4iMDiA72JyLQj7lXDwOQ714L/u4t3nmct1ceqJT+0qWDiQsPYMHvKfz318OV0t393QP47NbRUs3bFtm1Zt3BU3y64SgLt6dyJN0oefooCLH4EhPiz2Pn9Wdcr3b8sC3V7ZfKnehgY/oycP2DVhMfpQj2NxPsmNXF4mvizauHsftYDq/+spdtyVnkFVk5llWIBval5TLo8Z9IjAulwBGEA/3NLsFeND92rckttPLhmsMcOJnLxoPpFFTRPlnqtK6RVKyTuH5MV8cfNDtbjmR6Ja8zh3TkSHo+q/ed9Mr5hSjVqkqmrb0DUpHVxubDmax1VNv+djQTq13jZ/JhSOdwRnePZkyPKAZ2CsfP3HyXqi222tlzPIcdKUb18PZkY2B86TRlgX4mEuOMquHSNtju7YIwm5rvZ2rNUjIL2Hg4g02H0tl0JINdqTnY7BqloHf7EIZ1MeZxffrHPzieXVTp+Obe6USIUjLO1KG1BVOrzc72lGxW7zvJ2v2n2HAonSKrHR8FAzqFM7q70e45rEsEAX4tuzRntdnZdyKX7cnZbE/OYntyFjtTs8umEPM3+9A3NtSlDbZX+5Bm/aOhJbLa7PxxLIdNhzPKAmjp1G+BfiYGx4czvEsEwxIiGdI5nFCn9ksZoyxaOgmmDi09mNrtmj1pOazed4q1+0+y7kB62aTZfTqEMKp7FKO7RzOyW6TLH7HWymbXHDxZHmC3JWex02kicT+TD707hDh6ERvDdXp3CClbXUPULKewhC1HMo3AeTid345kkuf4ARMbZmFYlwij5Nklkr6xITXWDrSmGX9E2yPB1KGlBVOtNYdP5bNm/ylW7z/Jr/tPlQ0FSIgKZFT3aEZ3j+L0blEyltPBbtccSc83qodTstiRnM225CyyCow1IM0+ip7tQ+gfF8qATmEkOqaCa+kl94ZQOu+sUepMZ9PhTHYfy8aujXb2Ph1CGZ7gCJ4JkWXTuAnRVkgwdWgJwfRYViFr9p8sK32WVqG1D/UvG6oyqnsUnSICaziTKFUaJLY7AmxpSbb0h4mPgh4xwS5jYfvFhZZ1lILWWaIqsdnZlZrNxkMZZQG0tE0z2N/MkM7hZaXOwZ3DXe6HEG2RBFOH5hhM0/OK+fWAY6znvlMcOGl0oY8I9HUETqP02S06SMZgNiCtNceyC9nuKLnucATa0mCiFHSNDqJ/XBigWbjjuMuMOS2xrS+roITNRzLYdMgInL8fzSprv+wYHlDWUWhYlwj6dAiV6fOEqECCqUNzCKY5hSVsOJTOmn3GLEM7U43xaUF+Jk7rGsmYHkbps2+HUHzkj1mjS8spZIdTG+yOlGy3s+SA0elpXK92BPqZCPQzEeBrNp4d742H8zbHa18jLcjfjL/Zp94/ktyVmmcMjuNIej4bDxkdhTYfzmBPWg5aG+N5+8WGugTP2DCpshWiJhJMHZoimBaW2Nh8OMMxRd9Jfk/KwmbX+Jl9GNY5gjE9jNLnwE5h+MrQjmapqkm5wej4VVBiI7/YRkGxjfxia6V5YKujFAT6mghwBFrXYGyuMVD/npTJh2sPu5SafZTRsza3yCh1hljMDO1c2lEogkHx4QRJla0QtdZs5+ZVSk0FXgFMwDta66crpIcB/wU6O/LyvNb6fU+ObSpWm53fk7LKxnpuPJxBsdWOyUcxsFMYt43vzujuUQztEiG9SluIuPCAKiflXjhrnMs2rTVFVjv5jsBqBFhHsC0xZn3KLzLS8ktsrunF1rLX+cVWTuUW1ylQ2zXY7PDEzP4MT4igZ0yIVNkK0cS8FkyVUibg38BkIAnYoJT6VmvtPLfUX4CdWuvpSql2wG6l1EeAzYNjG5y76rTzBsXxx7Gcsvlt1x9MJ9cxNKNvbChXn96FMT2iGJEQ6fGcoaJ5qc0crkopLL4mLL6mBl+Ozl2gPvulFW5LzYUlNq46vUuDXl8IUXfeLJmeBuzTWh8AUEp9AswAnAOiBkKU0agUDKQDVmCkB8c2KHeTdN/72W88/NVW8h1z3HaLDmLG4DhGO3rdytqerUNzmZTbXaCuqtQsK58I0bx4M5h2BI46vU/CCJLOXge+BVKAEOBSrbVdKeXJsQAopW4Gbgbo3LlznTP73KLdlRYStmvQKF64eBCje0RJJ45WbOaQjs2y566sfCJEy+DNHjHuGnEq1lhNAX4D4oDBwOtKqVAPjzU2av2W1nq41np4u3bt6pzZlCp6dBaW2LhwWCcJpKJJzBzSkacuGEDH8AAURjtuSxuyI0Rb4M2SaRIQ7/S+E0YJ1Nn1wNPa6FK8Tyl1EOjj4bENSqrTRHPVXEvNQohy3iyZbgB6KqW6KqX8gMswqnSdHQEmASil2gO9gQMeHtug7p/Su9LSX1KdJoQQwhNeK5lqra1KqTuARRjDW97TWu9QSt3qSJ8L/BOYp5TahlG1+4DW+iSAu2O9lVdoPp1QhBBCtDxeHWeqtf4B+KHCtrlOr1OAsz091tvOTNpM4k8vY01NxRwbS0ziLJBgKoQQogYyTYpD1oIFpM5+BF1oTDxvTUkhdfYjAIRNn96UWRNCCNHMyfx2DmkvvVwWSEvpwkLSXnq5aTIkhBCixZBg6mBNTa3VdiGEEKKUBFMHc2ys+wSTifzNmxs3M0IIIVoUCaYOMffMwu7vOreu3deECg7i8BVXkvrYY9hycpood0IIIZozCaYOqxJ9ePMcH06Egh04EQpvTjNz4N2/EXntNWR++hkHpp1LzuLFTZ1VIYQQzYysZ+pw9hdnk5pXuX00NiiWny76iYJt20j9x2yKdu8mZPJZtP/HbHzbx9Q3y0IIIZqJ+qxnKiVTh2N5x6rdHjBgAF2/+Jx2995L7oqVHJg2jYxPPkHb7W6PE0II0XZIMHXoENShxu3K15fom2+i27fzsfTvz7E5j3H46mso2r+/sbIphBCiGZJg6nD30LuxmCyVtncJ6UKJrcRlm1+XLnR+/z1in3ySon37ODjzfE68/m/sxcWNlV0hhBDNiARTh2ndpjFn9Bxig2JRKGKDYhnXcRy/HvuV6xddT1p+msv+SinCLzif7j98T8jZZ3Py9dc5eP4FMoxGCCHaIOmAVINFhxYxe/VsAs2BvDDhBYa1H+Z2v9wVK0idMwdrSirhl11KzF//iikkpEHzIoQQwnukA5IXTUmYwsd/+phgv2BuXHQjH+36CHc/QILHjaP7ggVEXnstmZ99zoFp55L9889NkGMhhBCNTYKpB3pE9OB/0/7H2E5jeXr90zy86mEKrJUXEvcJCqL9Qw+S8OknmCIjSb7zLpLuvJOS48ebINdCCCEaiwRTD4X4hfDKma9wx+A7+P7A91z9w9UczTnqdt+AAQPo+vlntPtr6TCac8n43/9kGI0QQrRSEkxrwUf5cMugW/j3pH+TkpfCZd9dxqrkVW73Vb6+RN90E90WfItlQH+OPfY4h6+6mqJ9+xo510IIIbxNgmkdnNHpDD6d9ikdgjpw++LbeXvr29i1+1KnX+fOdH7vPWKfeori/fs5cP4FnHjtdRlGI4QQrYgE0zqKD43nP+f8h3O6nsOrW17lnqX3kFuc63ZfpRTh58+k2w/fEzplCif//W8Ozjyf/E2bGjnXQgghvEGCaT0E+gby9BlP88CIB1ietJzLv7+c/ZlVz4Zkjoqi4/PPEf/Wm+jCQg5feRWpj86R1WiEEKKFk2BaT0oprup3FW+f/TbZxdlc8f0V/Hy4+iExwePG0e27BURedx2Zn3/OgT9NI/unnxopx0IIIRqaBNMGMqLDCD479zN6RPTg3mX38tKml7DZbVXu7xMYSPsHHyDh008xRUeTfNfdHL3jDhlGI4QQLZAE0wbUPqg97095n0t6XcJ729/j1sW3klGYUe0xAQP60/WzT4m576/krVzFgT9NI/3jj2UYjRBCtCASTBuYn8mP2aNm8/jox9l8fDOXfncpO07tqPYY5etL1I030m3BtwQMGsjxx//J4SuvkmE0QgjRQkgw9ZLze57Ph+d8iEZzzQ/XMH/f/BqP8evcmfh33yX26acoPnDAGEbz6msyjEYIIZo5CaZelBidyKfnfsqQmCH8Y/U/eOLXJyot51aRUorwmTPp9uMPhE6dysk33uDgjJnkN/AE/kIIIRqOBFMvi7REMnfyXK5PvJ5Pd3/qdjk3d8yRkXR87lni334bXVzM4auuJvWRR7FlZzdCroUQQtSGBNNGYPYxc+/we3l+/PPsydjDJQsuYdNxzyZsCD5jLN0WfEvk9deT+cUXxmo0i35yu3KNEEKIpiHBtBF5upxbRT6BgbR/4G8kfPYZpnbRJN99N0l33EnJsWONkGshhBA1kWDayDxdzs2dgP6JdP3sM2Luv4+81as5MO1c0j/6SIbRCCFEE5Ng2gRqs5xbRcpsJurPfy4fRvPPJzh8xZUU7d3r5VwLIYSoigTTJlK6nNvrk16vcTk3d/zi44l/913innma4kOHOHDBhZx49VXsRUVezLUQQgh3JJg2sXGdxnm8nFtFSinCZswwVqM5Zyon3/g/YzWaDRu8nGshhBDOJJg2A7VZzs0dc2QkHZ91GkZz9TWkzn5EhtEIIUQjqTKYKqXuVUr92c32O5VSs7yaqzaotsu5uVM2jOaGG8j88kv2T5tG9sJFMoxGCCG8rLqS6Q3Af9xsf8uRJhpYXZZzq8gnMJD2f7ufhM8/w9yuHcmzZpH0lzsoSU31Uq6FEEJUF0y11rrSpLBa6yJAeS9LYkSHEXx67qcuy7lZ7dZanSMgsXQYzf3krVljDKP570doW9XLwgkhhKibattMlVLtPdkmGl6HoA4uy7ndtvi2Gpdzq8gYRnODMYxm8GCOP2EMoyncs8dLuRZCiLapumD6HPC9Umq8UirE8ZgALACeb4zMtXV1Wc7N7Xni44l/9x3inn2G4sOHOXjBhaS9/LIMoxFCiAZSZTDVWn8IzAYeBw45Ho8Bj2qtP2iMzAlDXZZzq0gpRdh559Htxx8Im/YnTs19k4MzZpK3fr0XciyEEG2Lak09PYcPH643tuKlytIL0/nb8r+x7tg6Lu19KQ+MeABfk2+dzpW7ajXH5syhJCmJ8IsvIua++zCFhTVwjoUQouVQSm3SWg+v07FVBVOl1GuAc6IGTgJLtdaeT9XTiFp7MAWw2q28uvlV3t/xPoPaDeLFCS8SExhTp3PZ8/M58e9/kz7vA0wREXT4x98JmTIFpaR/mRCi7fFWML3WzeZI4BLgU631y3W5oDe1hWBaatGhRcxePZtAcyAvTHiBYe2H1flchTt3kvqP2RTu3EnwhAkEjhlN+vvzsKamYo6NJeaeWYRNn96AuRdCiObHK8G0mosFAGu01kM82Hcq8ApgAt7RWj9dIf1+4ErHWzPQF2intU5XSh0CcgAbYPXkA7alYAqwL2Mfs5bNIjknmftG3McVfa6oc6lSW62kf/gf0l56CUpKXNKUxULsPx+XgCqEaNXqE0xrPZ2g1tqj9cKUUibg38A5QD/gcqVUvwrnek5rPVhrPRh4CFiutU532uVMR3qdPlxrV5/l3CpSZjNRN1yPOSKiUpouLCTtxZfqm10hhGi1ahVMlVJmpdT1QJIHu58G7NNaH3BM/vAJMKOa/S8H/leb/Ij6LefmjvXECffbU1NJfeRR8tasQVtrN4GEEEK0dtXNzZujlMp2fgDJGCXNWzw4d0fA+a96kmObu2sFAlOBL502a+AnpdQmpdTNHlyvzarvcm7OzLGxbrcri4Ws777jyA1/Zu8Z44zAunatBFYhhKD6caYhWuvQCo/2WutLtNYpHpzbXeNdVQ2004HVFap4x2ith2IE778opca5vYhSNyulNiqlNp6oolTVVlRczu2trW95vJxbqZh7ZqEsFpdtpW2mvdaspuOrrxA0apQRWK+/gb3jxpP66BwJrEKINq1WHZCUUt0xqmMv01r3r2HfUcAcrfUUx/uHALTWT7nZ92vgc631x1Wcaw6Qq7WudualttYBqSr5Jfk8tvYxfjj4A2fGn8m/xv6LEL8Qj4/PWrCAtJderrY3r72ggNyVK8lZuJCcZcvR+fmYIiMJmTyZ0KlTCBwxAmU2N/RHE0IIr/Fqb16lVCxwGUYQHQg8BXyltd5Ww3FmYA8wCaN6eANwhdZ6R4X9woCDQLzWOs+xLQjw0VrnOF7/DDyutV5Y3TUlmJbTWvPRro94fuPzxIfE8/KZL9M9vLtXrmUvKCB3xUpyFi0kZ+kydEFBeWA9ZyqBw4dLYBVCNHveGmd6E0YA7QR85njM11p3rUXG/gS8jDE05j2t9b+UUrcCaK3nOva5Dpiqtb7M6bhuwNeOt2bgY631v2q6ngTTyjYc28B9y++j0FrIE2OfYHKXyV69XmlgzV74I7nLlpcH1rMnEzpVAqsQovnyVjAtBtYCf9Vab3RsO6C17lbnnHqZBFP3juUd46/L/8rWE1u5of8N3DnkTsw+3g9o9oICcpevIHvRwvLAGhVFyOSzjMA6YgTKZPJ6PoQQwhPeCqbRwMUYpdP2GCXT67TW8XXNqLdJMK1asa2YZ9Y/w2d7PuP02NN5dtyzRFgqjyn1luoD6zkEjhjepIH1+wPf88rmVziWd4wOQR24e+jdTOs2rcnyI4RofF6fAUkp1YnydtNA4Gut9cN1uaA3STCt2dd7v+aJX58gKiCKl858icSoxEbPgz0/31EVvJDc5U6B9ezJhE6Z2uiB9fsD3zNnzRwKbYVl2ywmC3NGz5GAKkQb0tjTCfbG6M37WF0u6E0STD2z/eR27ll2D+kF6Twy6hFm9KhuLg3vMgLrCrIXLioPrNHRRonVS4E1vySf9MJ0ThWeIr0gndmrZ5NVnFVpv9igWH666KcGvbYQovlq1GDanEkw9VxDLufWUFwC67Jl6MLC8sA69RwChw9zG1htdhuZRZlGcCxMJ70gvTxYunlfmykXt11bbad1IUQrIsHUQYJp7TTkcm4NLTfrFCeWLCR/0c+otZtRRSUUhwWSPDyePwZFsitecbLYCJIZhRloN/OBmJWZCEsEUQFRRFoiyx7O76MCorhryV2k5ae5zcfI2JHcMvAWRnQY4e2PLIRoYhJMHSSY1k3F5dyO5R1r8M44NruNjKIMo6ToKC2WlRzdvHcuPfoXa4bu15z+h2boPo2/FXJDzBweEsuJUT2xD+pDZGC0S4CMskQR4heCj6p5+umq2kwndp7IutR1nCo8xdCYodwy6BZGxY6S9V6FaKW8PWnDL1rrSTVtaw4kmNZd6XJuR7KPYPIxYbWXTw1YVWec/JJ8j6tW61t6jLJEEWGJwLfIRu7y5eVtrI6q4NCzJxMydSqBw9xXBdekqt68hdZCvtz7Je9tf4+0/DQGRg/k5oE3M67TOAmqQrQy3hoaY8HoubsUmED5XLuhwI9a6751uaA3STCtn5ziHM787EyKbEWV0gLMAYzsMLLK0qOzYN/gaoNjXUqP7tjz8ow21h8XkrtihRFY20UTOvlsQqZOqXNgdafYVsz8/fN5d9u7JOcm0zeyLzcPvJmJnSfWOf9CiObFW8H0bmAWEIcxHWBpMM0G3tZav16XC3qTBNP6G/jBQLclSIA+kX3Kg6ElisiAysExwhKBv8m/kXPtCKylJVYvBtYSewk/HPiBt7e9zeHsw/QI78FNA25iSsIUTD4yAYUQLZm3q3nv1Fq/VqecNTIJpvV39hdnk5qXWml7Sxom4hJYly9HFxWVBdbQc6YSMHRovQOrzW5j0aFFvL3tbfZl7qNLaBduHHAj07pNw9enaXtFCyHqpjEmbRgNJGDMkwuA1vrDulzQmySY1l9rm8CgLLCWVgWXBtazpxA6dUpZYPVkpRy359d2lhxZwptb3+SP9D/oGNyRPw/4MzO6z8DP5NcIn1AI0VC8XTL9D9Ad+A2wOTZrrfVddbmgN0kwbRitdWo9e14eOcuWkVNaFVxUhLldO/x69qRg40Z0cXHZvqVruHoSUMFYpWdF0gre3Pom205uo31ge67vfz0X9rwQi9lS8wmEEE3O28F0F9BPt4AxNBJMhafKA+tCcn5e7HYfc1wcPZf8Uqvzaq1Zm7qWN39/k81pm4kOiOa6xOu4uNfFBPoGNkTWhRBe4u1g+jlwl9a6ckNaMyPBVNTFrr79oIr/BzH330/Q2LH49+pZ66EwG45t4K2tb/Fr6q9E+EdwTeI1XNb7MoL9ghsi20KIBubtYLoUGAysB8rGTGitz6vLBb1Jgqmoi70TJ2FNSamcYDaD1Rhva46JIWjsWILHjiFo9GhM4eEen/+3tN94a+tbrExeSYhfCFf1vYor+15JmH9YA30CIURD8HYwHe9uu9Z6eV0u6E3RvaL19Fdd27imJEzhsj6XUWAt4PbFt1c6ZkaPGczsMZOMwgzuXXZvpfRLe1/K1K5TOZZ3jIdWPlQp/drEa5kQP4GDWQd5fO3jldJvHngzo+JG8Uf6Hzyz/plK6XcPvZvBMYP5Le03Xtn8SqX0B057gD6RfVibspa3tr5VKf2RUY/QNawry44u44MdH1RKf+qMp+gQ1IGFBxfy6e5PK6W/OOFFIiwRfLPvG+bvm18p/Y2z3iDAHMAnf3zCokOLKqW/P/V9AOZtn8fyJNevhL/Zn7lnzQVg7u9zWZe6ziU93D+cl858CYCXN73M7yd+d0lvH9Sep894GoBn1j/DH+l/uKR3Ce3CnNFzAJizZg6Hsw+7pPeJ7MMDpz0AwIMrH+R43nGX9EHtBjFr2CyyFizg8N8fxLfYXpZW4uvD3lsmc96FD5C7ahUrv3qV+D8ysBTYsCs41jkY+2kDGXPBHQQMGMANi2+qdG8qfvfySvJIzUslsygTH+XDmLgxPDH2CRRKvntt9LsHcM/Se8gsynRJHxk7klsH3QrArYtvpcjqOvZ7fKfxXNf/OgCuX3g9Fcnfvbp99+adM6/OwbTGFaK11suVUl2AnlrrxUqpQKBZDqiLPF7AZa/tcNnmMykS+lyGvaByGoDv9E7QYyb2jEy36QEXb4GuU7EfS3Ob7nftVoifgP1wktt08607IG4Ues9Bt+nq3j0waTBq2x636Xr2QTi9D+ZNO7hsbuV0e1wSDOyK39qtXPaBm/SeadC9A4HLt3DZ527SB2VCbAShizdx2QI36WMKICSAqB82cNkvldOZajx1mL+Oy9a4ptv9zXCW8brzl+vousk13RYSCGcar3t+so6+O/e7pJdEJ8EZxuvE/65j8P4k1/SOaTDaeD183gZGJZ9wTe+eBacZr8e+swnfk64rw1j7FcIwCJs+nYzXnyAyOQeTVWMzK7Ii/AhIy8I3NpaIiy/G938vkRYXiH+hDUu+lahj+fh9vobDn6/BJzSUO2z5FASaKQw0YzMb1cFVfffs2p8iWzGL+y9n6vGpXNhuMpe+th2FazWyfPda/3cPYPKbWzDl5LvmbxjgCKYz3tiKT5HVJV2PtoAjmLr7t5O/e3X77s2rtKfnagymSqmbgJuBSIxevR2BuUCzm07QYrbQJ7KPy7aQDsY3NsAcUCkNICxmqPFsCXebHhEzCIB2ge3cpkdGDwCgU3AnzG7So6P6AdA1LIFAN+ntInoC0DOiJ+Fu0tuHJQDQL6ofJ92kdwjuBMCA6AGkuzt/YDsABscMIiNyV6X0MEs4AENjhpIVebhSeoA5AIBhHYaRE3miUnqpER1GkBuZ57JNWSwu6fmRNpd056rSER1GUHDMdXymOaZD2evhHYZTlOHa1ujXPqHs9bD2wyguOOSS7t+h/H4MiRmC1X7MJT2gw+Cy1917jcQWk1n2PhQIdJrcPjGqP7qwfLgQQODpI/Hv1o3clauwL1hAYJ6RriwWTGFhBOdFYC8srPK7lzh0LPO6HOa77d/Rs8RKdEA0scGx+PkYQ2rku9c2vnsD2w3E5pvpkl7Tdy/YKd3dd0P+7tX1u7eh0r6e8qSa9zeM31jrtNZDHNu2aa0H1PmqXiJtpqKpaK0p2ruXvFWryVu1kvwNG9ElJSh/fwJHjCBo7BiCzzgDv27dKnVkOpp9lHe3v8v8/UZ104zuM/jzgD8THxLfFB9FiDbL222m67TWI5VSW7TWQ5RSZmCz1npgXS7oTRJMRXNhLyggf8MGcleuIm/VKooPHgTAHBtrdGIaewZBo07HFBpadkxqbirvbX+Pr/Z+hU3bmNZtGjcOuJGdp3a2ynG/QjQ33g6mzwKZwDXAncDtwE6t9d/rckFvkmAqmqvipGTyVq0ib/Uq8tb+ij03F0wmAgYOLCu1WhITUSYTaflpfLDjAz7f8zkF1gJ8lA92Xd45qiXPSCVEc+btYOoD/Bk4G2Oy+0XAO81xEgcJpqIl0CUlFGzdSu7KleStWk3hjh2gNaawMILGjDZKrWPGkBNm5k9f/YnBv2VzxTJNVDacCoWPJygOnNapxcyVLERLIYuDO0gwFS2RNT2dvDVryVu5ktw1q7GdOAmAf69erLHtYcAh8HPqP1Nohjf/pJh84z+ZED+BqICopsm4EK2Mt5Zg2wZVrMUFSJupEA1Pa03R7t1lpda8desqDJgxZATBrJtNFAWYGNxuMBM7T2RS50l0CunU6HkWorXwVjDt4nj5F8fzfxzPVwL5WuvKI3WbmART0drs7NsXVU3lUVFEEElRsCcsn+QohU9CZ3oNncQZA86ld2TvWk+BKERb5u0209Va6zE1bWsOJJiK1qaqqQ5NERFEXn89xfv3U3TgAIX790F+QVl6rgVOxPhj6tqFDonDSRgwloCePfGNi0P5+DTmRxDNUGOsDNUSV5+qTzCtcdIGIEgpNVZrvcpxsdFAUF0uJoSonZh7ZpE6+xGXQfvKYqH9ww+5LA+ntcZ6/DhF+/aTvnsr6dvW4rdvD8Eb9uC3fA8pfGzs5++Lf7fuWLr3wL97N/y6dTeeO3dG+cn6q21BxTWLU/NSmbNmDkCDBbvGuEZz40nJdBjwHlA6K3cmcIPWerN3s1Z7UjIVrVFdFy4HyC3OZc2uRWzd+ANpu7YQc7yQzuk+dM3wIzi9vCSL2Yxf586uAbZbd/y7dcUnUJaOay1OFZzigm8vIL0wvVJabFBsg/UQP/uLs0nNq7zQWENewxsapTevUirUsX9WjTs3EQmmQlSt2FbMutR1LDm6hKVHlpKbfZLOGSbOsHZjaEEMnU5qOJRE8ZEjYCvvPuwbF4df9+74d+uGX/du+Hfvjl+3bpgjIipdoz6BX3iupirUIlsR+zP3sydjD3sz9pY9nyo8Ve15t127rd5501oz8MOq+6cuvmgxG49vbJZVwN5uM/UHLgQScKoWlg5IQrRcNruNrSe38svhX/jlyC8k5SahUAxqN4izYicwXvUi8ngBxQf2U7T/AEX791N88KBLdbMpKqo8wHbrTklaGhn/+Q+6qHyFE2WxEPvPxyWgNqCKVagAvj6+TOg0AaUUezP3cjj7cNlEH/4mf3qE96BnRE96RfTinW3vuC2ZApzR8QzuHHInfaP61ilvh7MP8/jax1l/bH21+zXURCQN3S7r7WC6EMgCNgFlP1e11i/U5YLeJMFUiNrTWrM3cy+/HPmFpUeWsivdmBi8R3gPJnWexKTOk4zJzrWmJCXF6PS0/wBFB/ZT7Ai09uzsKs/vExpK3NNP49sxDt+4OEwhIY310Vqlsz4/i+P5x92mdQruVBY0e0X0omdETzqHdMbkU77Ql7tgbDFZmBA/gTUpa8guzmZKwhT+MvgvdA3r6lGeSuwlfLDjA+b+Phc/Hz/O6nIWPx78sdI17hhyB3N/n0tuSW6lc9RUBVwxcI7rNI75++ZXukZ9ZgfzdjDdrrXuX6ecNTIJpkLUX3JuMkuOLGHJkSVsTtuMXduJC4pjYueJTOw8kSExQzD7lPdd1FpjO3mSvWeM8+j8PiEh+MbFuT46diwPtpGRMqSngtziXJYcXcKPB39kVfIqt/soFFuv3erR+aoq0WUXZ/Phjg/5cOeHFNmKmNF9BrcOupW44Lgqz7X95HbmrJnD7ozdnNX5LB4a+RAxgTFVXmPgBwPRVUxh8MSYJ/j3b/+udIy7HwBVqU+7rLeD6VvAa1rr+leme5kEUyEaVnphOsuPLueXI7+wNmUtxfZiIvwjGB8/nkmdJ3F67OlYzMZyZ1UN4zG3b0+n116lJCWFkuQUx3Oy8ZySYsxT7ERZLPjGxjoFWkewdbw3x8SgTJ4tqdyS23ALrAUsT1rOwoMLWZm0kmJ7MXFBcWQVZ5FXkldp/4bs3HOq4BTvbn+XT//4FI3m4l4Xc9PAm1iXuq4sQLYPbE/38O6sTV1LtCWah09/mEmda16Zs6rOSWD8IHAOtCZlIjEqkV3puyixl3iU99r8qKh0rJeD6U6gB3AQKMKYn1fLDEhCtC35JfmsSl7FkqNLWHF0BTklOQSYAxjbcSwTO09k2O/5pM95Ap+i8j96dn9fOj3xr2oDmC07uyywliS7BtqSlBRs6RXa98xmfNu3rzrYxsbi4+dH1oIFbocVNec23GJbMauTV/PjoR9ZdnQZBdYC2gW0Y0rCFKZ2ncrA6IH8cPAHt9W03lj84FjeMeb+Ppdv9n2DDz5oNFbtulD56R1O58UzXyTEz7Pq+6qqmU3KRJ618o8EkzJh07ZK26vSnEumXdxt11pXXs23iUkwFaJxlNhK2HBsg9HOenQpJwpOoFCM3WHnsmW2skn5v5joz5Sb/1WvP/L2ggJKUlPdlmpLUlKwpqWBvbwzC0phjo7GlpmJLqlcmjHHxdFzyS91zk9Ds9qtrD+2noUHF7L4yGJyinMI9w9ncpfJnNP1HIbGDHVp84TGnxDhcPZhLvr2IrfVrHUJXu7y/9DKh9xW/yoUHYI6VFmaddas20wdFxgL9NRav6+UagcEa60P1uWC3iTBVIjGZ9d2tp3cxi0/3+K2+tHf5M/MHjPpGNzR5RHmH9YgbaO6pISS48crlWqzvvqqymNC/3QO/r16OR698e0Y55V22qqCnl3b2ZK2hR8P/sjPh38mvTCdIN8gJnWexDldz2Fk7Eh8fXwbPD/1UVVbZ32qVZ1VNzb17qF3uy3NzugxgxVJK5pFb94aZ0BSSj0KDAd6A+8DvsB/gWY3naAQovH5KB8GtRtEfkm+2/QiWxE/HvyR7GLXHr9BvkHEBcfRMbgjnYI70TG4Y/n7kE4E+Xo20Zry9cWvUyf8OrlO8p/3669u23CVvz8FW7eR/cOP5Z8hKAj/nj3x790b/149sTgCrSksrNLxnnI3C9Ajax7hu/3fsSdzD2n5aVhMFsbHj+echHMY22ks/iZ/j87dFG3BVZUOOwR1aJDzVxUwnQNkcxybWsqTat7fgCHAZq31EMe2rdJmKoRwVtOsNznFOSTnJhuPHOM5JTeFpNwkknOTKbAWuBwX5h9WqTTbMbgjHUM6EhcUV9bxqSpZCxaQ9I+/V9mGa8vNo2jvHor27KVo926K9uyhcO9e7Fnl89KYO3Qwgmvv3mWlWP+uCVVOvai1psBaQL41n0sWXMKJghNu95sQP4FzEs5hQvwEAn1rN8NUU7UFV9XW2ZBttU09n6+35+Yt1lprpYy1K5RSMi+vEKKS6koWACF+IfSJ7GOMWa1Aa01mUSbJuclGcM0xAm1ybjJ7M/ay/Ohyiu3FLsdEB0SXlWadS7adgjvRIbgDqxJ9WHSODxctwakN14cpiT5MA0zBQQQOGYLvoP74lORjL8mjqDiXopSjFO/Zh23/AdT+I/ge2o1l9Wp8bEa7rN2kyGgfRFqshZT2vhyJURyIspIcWES+rcBlMoIxO2yVFnZfk2jmtYmveXxf7QUF2DIysGZkYEvP4Pi/nnQJpAC6sJC0F1/yajBtjNLhtG7TmlVpszY8KZneB/QEJgNPATcAH2utPf82NBIpmQrRtLxVsrBrOycLTlZZsj2Wd8ylx6eP8ik7riKzj5kOgR3It+aTV5JHka2o0j4VmWyazhlmeqT7kXBSEX/cTuzxYsIyygN8SaAvufHRFCW0x9q1I5t3LWHK6gL8nTq/Fpph/p/Ceejm/7gESFtmBtb0dGwZmdgyMrClp2PNzMCWkYkuKHCTI/d8u3TGP6ErfgkJ+HVNwC+hK35dE4zhRE5twi15yJA3NUYHpMnA2Y63P2mtf67LxbxNgqkQbZPVbuV4/nEjuOYY1cZvbn2zyv2ndZtGkDmIIF/XR6BvYPlrs/E62DeYIN8gfE2VOwTZsrMp2rvXqCLes4ei3Xso2rOn0thZT/gEBWGKiMAUGYkpIhxzRKTxPiICc2SE43UkybNmGT2YKx4fHEzQ2LEUHzpE8aFDrtXAgYH4JXTBPyEBe3ExectXuPR0bu5DhhpLYwTTDsBpgAY2aK2P1eVi3ibBVAhRqqlWLtFaY01NZd/EqicwiHvhecyR5cHSFBGBj4dL4HnSZqrtdqzHj1N86BBFBw+WBdjiQ4cpOXLE7Xmb25ChpuDt3rw3Ao8ASzAmbHhNKfW41vo9D46dCrwCmIB3tNZPV0i/H7jSKS99gXZa6/SajhVCiOrU1IbrLUopY/KIuDj3M0LFxRE2re5V36UBs7pqWuXjY8wiFRtL0KhRLsfv6tsP3BSirKk1j+MUVfOkzXQ3MFprfcrxPgpYo7XuXcNxJmAPRltrErABuFxrvbOK/acD92itJ9b22FJSMhVCOGvK3qHNdQamqqZ9BIi5/34ir7vW4+kaWxtv9+ZNAnKc3ucARz047jRgn9b6AIBS6hNgBlBVQLwc+F8djxVCiEqasneoJyXIphBzz6zKQd7fH78e3Ul77jlyfv6Z2KeexL+rZyvGVKWtdXLyJJgmA+uUUvMx2kxnAOuVUvcCaK1frOK4jrgG3SRgpLsdlVKBwFTgjtoeK4QQzVXY9OnNLoBUFeRDzz2X7O++49gT/+Lg+RcQc+89RFx1FcrHp9bXqFgqt6akkDr7EZfrtzaeBNP9jkep+Y7nmmY1djc3V1V1ytOB1Vrr0hmtPT5WKXUzcDNA586da8iSEEKIqoJ82PTpBJ42kmOPPMLxJ58i+6efiHvySfxq+bf1+LPPuR8L+9LLbTeYaq0fA2OyBq115Yk3q5YExDu97wS4r6iHyyiv4q3VsVrrt4C3wGgzrUX+hBBCVODbPoZOc/+PrK+/4fiTT3Jgxkxi7vsrEZdfXm0pteT4cbK/+56sBQuwnXA/81Nr7uTkSW/eUcC7QDDQWSk1CLhFa317DYduAHoqpbpiVBVfBlzh5vxhwHjgqtoeK4QQouEppQi/4HyCRo8i9R+zOf7PJ8j56WeCJ04k/YMPyqqHo2+7FWUyk7XgW/J/XQdaYxk4EJ/QUOzZ2ZXO6xMUhC4pQfk2r0n8G4InvXnXARcB3zrNzbtda92/xpMr9SfgZYzhLe9prf+llLoVQGs917HPdcBUrfVlNR1b0/WkN68QQjQsrTWZX3zBsX8+AcXFbvfx7dzZUXV8Ln4JCW57MmMygc1GwJAhhPzpHNLfn9fsOid5ez3TdVrrkUqpLU7B9Het9aC6XNCbJJgKIYR37B033u3MS6boaHquXFFpCTt3vXlRPqQ89BBUWGe2OQwZAu8PjTmqlBoNaKWUH3AXsKsuFxNCCNEyWatoB7WdOuV2LdiqOjmlPftspaDcGjonedLn+VbgLxjDVZKAwY73Qggh2ghzbGyttlelqqDc0jsn1RhMtdYntdZXaq3ba61jtNZXlc6GJIQQom2IuWcWyuK6hqyyWIzq21poqKDc3FRZzauUeo2qx4Witb7LKzkSQgjR7DTUjE5uZ2CqQ1BubqprMy3tyTMG6Ad86nh/MbDJm5kSQgjR/DTEjE4uQdkxR3D7v/+9RbeXQjXBVGv9AZQNXTlTa13ieD8X8N76RUIIIVq10qCct249R669FlN4WFNnqd486YAUh+vUgcGObUIIIUSdBQ4dgk9wMHkrVjR1VurNk6ExTwNblFJLHe/HA3O8liMhhBBtgvL1JWjMGHKXr0Br7XaITUvhSW/e9zFWbPna8RhVWgUshBBC1EfweGMyiKJdLXv6Ao/W1tFaH9NazweGaK2PeTlPQggh2ojgM8YCkNvCq3pru1DdeV7JhRBCiDbJ3K4dlv79yV22vKmzUi+1DaYtt0JbCCFEsxQ8fjwFv/+ONSOjqbNSZ7UNpsO8kgshhBBtVvCE8aA1eatWNXVW6qzGYKqU6qWU+sWx7JpdKTVQKfWPxsicEEKI1s+SmIgpKqpFV/V6UjJ9G3gIKAHQWm/FWKxbCCGEqDfl40PwGWeQu2oV2mpt6uzUiSfBNFBrvb7Ctpb5aYUQQjRLwRPGY8/KomDr1qbOSp14EkxPKqW645j0Xil1EdCy18oRQgjRrASNHg0mU4ut6vUkmP4FeBPoo5RKBmYBt3kzU0IIIdoWU2govp07c+q999jVtx97J04ia8GCps6Wx2qcTlBrfQA4SykVBPhorXO8ny0hhBBtSdaCBZQcPQqONlNrSgqpsx8BaBErytQYTJVS4cA1QAJgLp07UdYzFUII0VDSXnq5LJCW0oWFpL30cusIpsAPwK/ANsDu3ewIIYRoi6yp7rviVLW9ufEkmFq01vd6PSdCCCHaLHNsbNli4S7b27VrgtzUnicdkP6jlLpJKRWrlIosfXg9Z0IIIdqMmHtmoSyWStttBfkU7tzZBDmqHU+CaTHwHLAW2OR4bPRmpoQQQrQtYdOnE/vPxzHHxYFSmOPiaPfXv2IKDuHwVVdz/PkX2DtxUrPt6au01tXvoNR+YKTW+mTjZKnuhg8frjdulDgvhBCtRcnx4xy89DJsx1xX/1QWC2HnzyR3+QqsqamYY2OJuWdWvTorKaU2aa2H1+VYT9pMdwD5dTm5EEIIUR++7du7Xa5MFxaS+b9Pyt439VAaT4KpDfhNKbUUKCrdKENjhBBCNAbr8eMe7deUQ2k8CabfOB5CCCFEo6uqp687TTWUxpMZkD5ojIwIIYQQ7sTcM4vU2Y+gCwtr3NccG9sIOXJz3aoSlFKfaa0vUUptwzHJvTOt9UCv5kwIIYSgvA007aWXyzobBY8fR9bX37gEWGWxEHPPrCbJY3Ul07sdz+c2RkaEEEKIqoRNn16pLTRw6FCXAFvf3rz1UWUw1VqXVjzfrrV+wDlNKfUM8EDlo4QQQojG4S7ANhVPJm2Y7GbbOQ2dESGEEKKlqq7N9DbgdqCbUsp56fMQYLW3MyaEEEK0FNW1mX4M/Ag8BTzotD1Ha53u1VwJIYQQLUh1baZZQBZweeNlRwghhGh5PGkzFUIIIUQ1JJgKIYQQ9STBVAghhKgnCaZCCCFEPUkwFUIIIepJgqmzrZ/BS/1hTrjxvPWzps6REEKIFsCrwVQpNVUptVsptU8p9WAV+0xQSv2mlNqhlFrutP2QUmqbI22jN/MJGIFzwV2QdRTQxvOCuySgCiGEqJEn65nWiVLKBPwbYzrCJGCDUupbrfVOp33CgTeAqVrrI0qpmAqnOVNrfdJbeXTxy+NQUuC6raTA2D7wkkbJghBCiJbJa8EUOA3Yp7U+AKCU+gSYAex02ucK4Cut9REArXVaQ2eipKSEpKQkCmtaB+/0F6pO274VTL4Nm7EWzGKx0KlTJ3x95Z4IIQR4N5h2BI46vU8CRlbYpxfgq5RahjHn7yta6w8daRr4SSmlgTe11m/VJRNJSUmEhISQkJCAUqrqHY/bwVZcRaIN/AIgMBIs4eBjqktWWgWtNadOnSIpKYmuXbs2dXaEEKJZ8GYwdRe5Ki4ybgaGAZOAAGCtUupXrfUeYIzWOsVR9fuzUuoPrfWKShdR6mbgZoDOnTtXumBhYWHNgRQgJNZoJ9V2p5P7QEgcaBvkp0PmEVBJEBABgVHgGwg1nbeVUUoRFRXFiRMnmjorQgjRbHgzmCYB8U7vOwEpbvY5qbXOA/KUUiuAQcAerXUKGFW/SqmvMaqNKwVTR4n1LYDhw4dXDNYANQdSMEqdADmpRgnV5GcE2NLtwe2hONcIqvkZkH8KzBYjqAZEtKlqYI/upxBCtCHeDKYbgJ5Kqa5AMnAZRhups/nA60opM+CHUQ38klIqCPDRWuc4Xp8NPO7FvBoCI8uDZ0VKgX+I8QjrCAWZRkDNTobsFLCEQmC0kS7BRggh2hSvBVOttVUpdQewCDAB72mtdyilbnWkz9Va71JKLQS2AnbgHa31dqVUN+BrRwnIDHystV7orbw6+2ZLMs8t2k1KZgFx4QHcP6U3M4d0dN3JxwxB0cajpMAIqvnpUJgFPr6OoBwFZv/GyLIQQogm5s2SKVrrH4AfKmybW+H9c8BzFbYdwKjubVTfbEnmoa+2UVBiAyA5s4CHvtoGUDmglvINgLBOEBpnBNP8dMg9bjz8giEwipmXX89RR4/iu+++m5tvvpl3332XZ555hri4OHr27Im/vz+vv/46J06c4NZbb+XIkSMAvPzyy4wZM6ZRPr8QQoi68WowbW4eW7CDnSnZVaZvOZJJsc3usq2gxMbfvtjK/9YfcXtMv7hQHp2eaHRWCogwHtZiKEg3SqyZh3nvqfuIjEugQAUwYswEpk2bxj//+U82b95MSEgIEydOZNAg47fD3XffzT333MPYsWM5cuQIU6ZMYdeuXQ13E4QQQjS4NhVMa1IxkNa0vUpmPwjpUNZp6dVXZvP1AqOAfjQplf+883+MH3cGkZFG++zFF1/Mnj17AFi8eDE7d5YPxc3OziYnJ4eQkJA6fCIhhBCNoU0F00enJ1abPubpJSRnFlTa3jE8gE9vGVX7CyrFsrWbWLx6I2vXbyRQFTHh7Gn07hTBru2/Q/pBo21Vl3dCttvtrF27loCAgNpfTwghRJOQie6d3D+lNwG+rhMyBPiauH9K7zqfMysri4iICAKDQ/nj6Cl+3bSVfHMky9f9RsbxZKxpu/nyk/9AcR5Yizj77LN5/fXXy47/7bff6nxtIYQQjUOCqZOZQzry1AUD6BgegMIokT51wYCqOx95YOrUqVitVgYOHMjs2bM5/fTT6dilKw//YzYjZ9zAWVfdQ78+vQiz+EDaTl599G42rlvDwIED6devH3Pnzq35IkIIIZqU0trtPAct0vDhw/XGja4LzOzatYu+ffs2UY6qlpubS3BwMFarlfPPP58brr2G86eMNXoD24pBmcpnWvILbOrsVtJc76sQQtSVUmqT1np4XY5tU22mzcmcOXNYvHgxhYWFnH322cy88CJjsofgDo6Zlk45HifBHOA005L8k3nF1s+MFYKykoyhTpMekdWChBAek7/MTeT55593n+Ay05IVChxTF2YnGbMtWcKNSSFkpqWGU7qWbekSfKVr2YIEVCGERySYNmc+ZghqZzyK8x1jV9OhMMOYOzjAMf2hzLTkOa0h5xic3ON47IXN88Ba5LqfrGUrhKgFCaYthV+g8QiJg8JMx0xLx4yHX4jT8nDSpwwwJs7IOAgndpcHzdLn4pzy/fxCKgfSUllHYftX0G9Gm152TwhRMwmmLY2PT/mE/Nai8tJq5mFjebjACAhonp2WvKIgwylQOgXN9IPG0nmlQjtCdC8YfAVE9zReR/cyJtd4eYAROCvyMcMX10NEVxhzNwy6HHwtjffZhBAthgTTlszsbywTF9wBinKMoJp3CvJOGnMGl3Za8mnh/8x2uxHsKgXN3ZDntK6qyQ+iekD7REg83xEwexrb/KuZQWrSI65tpmDcv3NfMX6UrHwRvpsFy56C02+D4TeAJcxrH1cI0fK08L+ybdOyZcvw8/Nj9OjRAMx9800CAwO55pprwObUaSkrCbKSISCcjbsO8+H/vuDV115r4txXozgf0vc7qmadqmVP7QOrU6ALiIDo3tBrankJs10vCO9St+rY0nbRqnrz9jkXDq2EVS/B4jlGcB3xZxh5G4S0r/fHFkK0fDLOtKJGHiJhtVoxm2v3m2bOnDkEBwdz3333Vb9jcb4RVAsyjCpPk59RPRwQZcwfXA91HmeqtVGarFgte3IPZB4FSr+PCiK6lAdL50dQVL3yXi8pv8Hql2HHN8b9HHIljL4TIrs1XZ6EEA2iPuNMJZg6qzhEAozqvumv1iugfvjhhzz//PMopRg4cCAmk4nIyEi2bNnC0KFDuf322/nLX/7CiRMnCAwM5O2336ZPnz4sWLCAJ554guLiYqKiovjoo48oKCjg9NNPx2Qy0a5dO1577TV++eWXsuA6YcIERo4cydKlS8nMzOTdd9/ljDFjWPbTdzz/4kt8N+9F5rwwlyPHTnHg6DGOJKUwa9Ys7rrLGAry3//+l1dffZXi4mJGjhzJG2+8gclUubRX4321WSHjUOVq2ZN7jKXqyu5voGsbZunryO7Nu33y1H5Y8yr89jHYrUa18phZEDuwqXMmhKgjmbTBUz8+CMe2VZ2etAFsboZIzL8DNn3g/pgOA+Ccp6s85Y4dO/jXv/7F6tWriY6OJj09nXvvvZc9e/awePFiTCYTkyZNYu7cufTs2ZN169Zx++23s2TJEsaOHcuvv/6KUop33nmHZ599lhdeeIFbb73VpWT6yy+/uFzTarWyfv16fvjhBx577DEWL14MllDwC4KYfuAXzB971rH0szfJySuk97iZ3Hbjdew7nMynn37K6p8X4Ft4ktv/9hgfvfEM1/z5VqNE605hdoW2TEfgTD8A9pLy/YI7GIGy/0XlQbNdb6N3ckvsgRzVHaa/AhMegl/fgA3vwfYvocdZMPYe6DJGxgEL0Ya0rWBak4qBtKbtHliyZAkXXXQR0dHRAC7LrplMJnJzc1mzZg0XX3xx2TFFRcb1kpKSuPTSS0lNTaW4uJiuXbt6dM0LLrgAgGHDhnHo0CHXRLM/+IcwbcZF+Mf2xT//FDFRERzfuYZfFq9m08b1jBg9DoCCwiJioiOMzj92K5gtRg9iayHkpsEL50NOavm5fcxGdWd0L+gzzam02aP1dtgJ6QCTH4ex98LGd+HX/4N506DTCCOo9jqnZf5YEELUStsKptWUIAF4qb/7IRJh8XD993W6pNYa5aaEEhQUBBhLroWHh7tdHebOO+/k3nvv5bzzzmPZsmXMmTPHo2v6+xuTOJhMJqxWq/t9LBajtGoJxeQfiDUgGm23c+1F03jqoTsrfAi7MftSKWUy2j67T3Stoo1IAJOvR3lsdQLC4Yy/wum3w28fwepX4ZMrjI5SY2cZJfJ6tlMLIZov+cnsbNIjRhupM98AY3tdTzlpEp999hmnTp0CID093SU9NDSUrl278vnnnwNG8P39998BY/m2jh2NFWs++KC8mjkkJIScnBwaVFA0k2ZcwRffLSbtpJHH9IwsDiellO8T1QPa9zeqtkPaw8w3jNJXn2lGUG2rgdSZbwCMuBHu3AwXvmvck29ug1eHwNo3oCi3qXMohPACCabOBl5idDYKiweU8VzPzkeJiYn8/e9/Z/z48QwaNIh777230j4fffQR7777LoMGDSIxMZH58+cDRq/diy++mDPOOKOsmhhg+vTpfP311wwePJiVK1fWOW8V9UtM5IkH7+Lsy29n4FmXMPny20g9ftJINPkZYzVNvtIW6AmTGQZcBLeugiu/MErtix6Cl/vD0qeMMcFCiFZDevMKV/npRlW3tpdvUz7GDwunTkhyX+vg6HpY9TLs/t7oxTz0Whj1FwiPb+qcCSGoX29eKZkKV4GRRuA0Odr3TH6VAqmoo/jT4PKP4fZ10G8mbHgbXh0MX98GaX80de6EEPXQtjogCc+Uzv0rvCOmD5z/f3Dmw7D237D5A/j9Y+g9zWiDjh/R1DkUQtRS6wqmJ/fC+9Nctw151Hi224yxjxUFRhpz2NqsxiojFQVFG9PXWYuNyeQrCo4xhn2UFLrvCRzc3ug1W5zv2iO2VEgs+AcbHVOch5mUCu1ozA9bmA25xyunh8UbkxsUZhnDVSoK72L0Ii3IMObsrSiiq9G+l3/KfTteZDdjir68E1CQWb49Nw3ev6+8l/PqV2HPItdjfS1w1ZfG6+XPwoHlrumBEXDpf43Xi+fA0Q0VPnscXPi28drdGOGo7nDeq8brb+8yJlJw5jwG+MubIDvFNT1+BJw1x3j96VWQn+Ga3m08jP+b8fq/Fxr/xs56TYExjnVPK37vABJnwmk3Gf/2H11cOX3wFUb+hv8Z/ns+7F1kVAH7hxr/rmf8FQZcaMzG9dUtlY8ffQf0Psf43i+YVTl93H3Q/UxI3QoLH6qcPukR6DwSjqwzZv2qaOpTxiQU+5fCCjfr705/2eh4tvtHWPN65fQL3jRmEdv+pTEOt6JLPjRms9rykTH5RUVXfm5899e/bcw4VZF894zXdf3uDbnSmMv7s2sqp4+4Afq34e9eHbSuYCpESxQYafzoCetkrLWanQxpO2Dxo4CGnLTyCUVM/sY0i0ExTZ1rIYQT6YAk6kTuqxdZi2Hb58YcwCf3AIryOYtpkCkuhaiTRp67vLFJByQhWhOzYwL929dBYDQugRSMKS5/eaxJsiZaia2fGZPUzAk3nrd+5tkxC+5yNGdp43nBXZ4d2wZINa+XHTp0iHPPPZft27c3dVZES+PjY7Rlu5OVBF/8GfqeCz0mG+3uQnii4oIeWUeNdt+CTOg9FWwlxvShdmv5a1sJLHrYdREQcPywe7xVlU7rSoJpBd8f+J5XNr/CsbxjdAjqwN1D72ZaNzcN/EI0hrBO7ju2+QbBgaWw/QujHbX7mca6q73/1LRL1InmqSjXWCc4bScsfLByULQWwI/3G4/aykpqmDy2cBJMnXx/4HvmrJlDoc3oOZeal8qcNXMAGiSgHjhwgAsvvJC5c+dyzz33UFBQQEBAAO+//z69e/dm3rx5fP311xQVFXHw4EGuuOIKHn30UQ4dOsTUqVMZOXIkW7ZsoVevXnz44YcEBgby+OOPs2DBAgoKChg9ejRvvvmm27mARQs16ZEqlgV82ehteeRX+OM72LUA9iw0JtjoPNoosfY5VyaEaK2qaru0FsOpvZC2ywicabvg+A73IxHcOe91Y8EKk6/Ts6/Ro//rW4xe/RWFdWrYz9ZCtakOSM+sf4Y/0qseHL/1xFaK7cWVtvv5+DGwnft1KvtE9uGB0x6o8pyl1bxffvkll112Ge+//z7dunUjMDAQs9nM4sWL+b//+z++/PJL5s2bx0MPPcT27dsJDAxkxIgRzJs3j+joaLp27cqqVasYM2YMN9xwA/369eO+++4jPT29bCWaq6++mksuuYTp06fXeK/qSzogNSJPOn1oDam/OwLrd3Bil7E9drAjsE43lryTH1otn7t1l5XJGKaXd8Koli3dFt0TYvpCTKLjuS98eJ770mRYPNxTTXOUu+uCMRf1tBfq/7maAVnPtIG4C6TVbffUiRMnmDFjBl9++SWJiYkcPXqUa6+9lr1796KUoqSkfN3PyZMnExVlVNNdcMEFrFq1ipkzZxIfH8+YMWMAuOqqq3j11Ve57777WLp0Kc8++yz5+fmkp6eTmJjYKMFUNKKBl9TcJqUUxA02HhP/YYx73LXACK5LnjAeUT2M0mrf6RA3VJaGa0ly0yB5MyRvMhalt1YYd6ptxljyMXcbaxbH9DX+vc3+lc816VH3tR01LehR+h0s/WEXGmdMi7nhHeNH3phZbfrHWpsKptWVIAHO/uJsUvMqT5wQGxTL+1Pfr/N1w8LCiI+PZ/Xq1SQmJjJ79mzOPPNMvv76aw4dOsSECRPK9q1YRVv63t32wsJCbr/9djZu3Eh8fDxz5syhsLDCfzLRNkV1N5Z+GzsLslONySB2LYC1rxtDbkJijdV++pwLCWNlxZ/mpDAbUn8rD54pW8rbzZWP67zZzqxFnq1wVTEo1maIS8UfdtYiY1WkxXMg8wic85wxCUxjaUZDddpUMK3J3UPvdmkzBbCYLNw99O56ndfPz49vvvmGKVOmEBwc7LK02rx581z2/fnnn0lPTycgIIBvvvmG994zZu84cuQIa9euZdSoUfzvf/9j7NixZYEzOjqa3NxcvvjiCy666KJ65VW0QqGxRlXciBuN0sueRUZg3fKRUaqwhEOvqUZ1cPdJxqxDomHU9MfeWgTHtzsC52ZI2Wx0FCodDhXexVhofuQt0HEYxA6Cf4+sYt3lWrRdelLb4QmzP1zwDoR3hlUvQVYyXPRezb3LGyIIuuuVvMAxK1QTBFQJpk5KOxl5ozdvUFAQ3333HZMnT+bcc8/loYce4sUXX2TixIku+40dO5arr76affv2ccUVVzB8+HAOHTpE3759+eCDD7jlllvo2bMnt912G4GBgdx0000MGDCAhIQERoyQOV1FDQIiYNBlxqM4H/YvMaqCd/8IWz8BcwD0mGRUBfeaYuwv6sbtEJQ74fAao3NP8iYjkNoczUiB0UbATLwAOg41quLd9cyuqlNaPdZdrhcfH2NqxLB4+OE+mDcNrvjMWPPYnYYKgr883qyG6rSpDkjN3bx589i4cSOvv+4612RzHKvaku6r8ICtBA6vNjov/fE95KQYf/ATxhpVwX2mGW1kLY23qgHtNijONYacFOU4Xjs9F+XC0ieMObPd8Qs2Ood1HGoE0I5DjWDkaZtjM6redLF7IXxxvfHD4LQbjXmVK+bxpcS6dYCqaE44lSY0AUDBnMw6ZV86IAkh6sfkC90mGI9znjXa6XZ9a5Raf7jPeHQcXt4zOLpHU+e4ZlVNTpB30phIviwQ5lQfFItzoSjb6XUulOTVI2MKHjxiDDepq4aqpm1ovafCdd/DB+fBz04l5ayjMP8vsOHdqsel1na8alVjsJtoqI6UTEWdyH1tI7Q22vD+WGCUWlN/M7a36+PoGXyuUcIqLVE1RomppNBo+y1IN1Y6qvScYTzv/6W8CrU2zBaj5OgfDP4h4Bfi9NrxXPa6mn3eGt8wJbCW6IU+7lfBUj7G/S3Jr5xW2/uy9TNj7Ktzh6x6zlstJVMhhHcoZay/GtMHxt0PmUeNauA/voNVL8LK540/gn2mgV8Q/PqG521hdjsUZjoCY0YVgdH52RFA3f0hLmW2QIBjPd7qAunF86oOgg3Vs7muQ1Bag5xj7rdrDdNfcTNeVcGZf6/dNbqOM87nH2LUFkhvXiFEixEeD6ffajzyThodl/74Dja+bywRV1FJAXx3r7EuZcUAWZhZ9TAPlNH5KTDSeA6Jg/b9HYEyojxgOj8HRLj2RH6pfxXVgPGQeH5D3I3q1WcISktXXRVsxfsSGGnMQZ1dy2rerZ8BGm5cAu161TvL9SXBVAhRN0HRMPRq41GUA09V0VZVnAMHV5QHwbABRuBzFxBLg6clvP6TSjSHXq/NtW3T22q69xXvy+fXwbJnoPc0aN+v5vNrbSzq3WlEswikIMFUCNEQ/EOMEl9VJcGmaCNsyyXDplbbe/+n5+HgSvj4UsBujFet7piULcaUmee+5LWPUFteDaZKqanAK4AJeEdr/bSbfSYALwO+wEmt9XhPj20pgoODyc3Nrfd55syZQ3BwMPfdd1+1+1133XWce+65MoGDaFzNoSRYUVstGTYHtbn3QdHGwg3r3yzfVl2b+28fG6slJV7QMHltAF6bnFMpZQL+DZwD9AMuV0r1q7BPOPAGcJ7WOhG42NNjvSVrwQL2TpzErr792DtxElkLFjTGZYVo+QZeYvSkDIsHlPFcj56Voo3Z/UPlbaWTMDizFhlLD/Y9FwLCGyVrnvDmTNenAfu01ge01sXAJ8CMCvtcAXyltT4CoLVOq8WxDS5rwQJSZz+CNSUFtMaakkLq7EcaLKDm5uYyadIkhg4dyoABA5g/fz5gTMrQp08fbrzxRvr378+VV17J4sWLGTNmDD179mT9+vVl5/j999+ZOHEiPXv25O233wZAa80dd9xBv379mDZtGmlpaWX7P/7444wYMYL+/ftz880305qGQolmaOAlRpXunEzjWQKp8JSn40/3LDR6fw++wvt5qgVvVvN2BJwbUJKAkRX26QX4KqWWASHAK1rrDz08ttaOPfkkRbuqXoKt4Pff0cWu3el1YSGpf/8HmZ997vYY/7596PDwwx5d32Kx8PXXXxMaGsrJkyc5/fTTOe+88wDYt28fn3/+OW+99RYjRozg448/ZtWqVXz77bc8+eSTfPPNNwBs3bqVX3/9lby8PIYMGcK0adP49ddf2b17N9u2beP48eP069ePG264AYA77riDRx4xqtmuvvpqvvvuO1lVRgjR/FTVAzionev73z42Fmrodmbj5MtD3iyZupsXq2KxyAwMA6YBU4DZSqleHh5rXESpm5VSG5VSG0+ccLNwbS1UDKQ1ba/1+bXm4YcfZuDAgZx11lkkJydz/PhxALp27cqAAQPw8fEhMTGRSZMmoZRiwIABHDp0qOwcM2bMICAggOjoaM4880zWr1/PihUruPzyyzGZTMTFxbnM97t06VJGjhzJgAEDWLJkCTt27GiQzyKEEA1q0iNGG7sLBXlpxmxKWz6CF/oaJdPiXNj+ZZNksyreLJkmAfFO7zsBKW72Oam1zgPylFIrgEEeHguA1vot4C0wZkCqLkM1lSD3TpxkVPFWYI6Lo8t/Pqz2WE989NFHnDhxgk2bNuHr60tCQkLZyi/+/uXrDvr4+JS99/HxwWq1lqV5ukQbIEu0CSFaDnc9gMfdD8kbYfUrGGUsx5/4opwmXSHGHW+WTDcAPZVSXZVSfsBlwLcV9pkPnKGUMiulAjGqcnd5eGyDi7lnFspicdmmLBZi7pnVIOfPysoiJiYGX19fli5dyuHDh2t9jvnz51NYWMipU6dYtmwZI0aMYNy4cXzyySfYbDZSU1NZunQpgNsl2oQQotmq2OY+7Fo47zVj4vyKlZPuOic1Ia+VTLXWVqXUHcAijOEt72mtdyilbnWkz9Va71JKLQS2AnaMITDbAdwd6628lgpztCWmvfQy1tRUzLGxxNwzq2x7fV155ZVMnz6d4cOHM3jwYPr06VPrc5x22mlMmzaNI0eOMHv2bOLi4jj//PNZsmQJAwYMoFevXowfPx6A8PBwWaJNCNHy5Z9yv722k+N7kUx0L+pE7qsQotFUNzVkA04IUp+J7r1ZzSuEEELUn7vOSU09IUgFEkyFEEI0by1gQhCZm1cIIUTz18ynhmwTJdPW1C7cHMj9FEIIV60+mFosFk6dOiUBoIForTl16hSWCkOIhBCiLWv11bydOnUiKSmJ+s6OJMpZLBY6dapi7UohhGiDWn0w9fX1pWvXrk2dDSGEEK1Yq6/mFUIIIbxNgqkQQghRTxJMhRBCiHpqVdMJKqVOALWfPb7liwZONnUmWhG5nw1H7mXDkvvZcNzdyy5a63budq5JqwqmbZVSamNd55MUlcn9bDhyLxuW3M+G09D3Uqp5hRBCiHqSYCqEEELUkwTT1uGtps5AKyP3s+HIvWxYcj8bToPeS2kzFUIIIepJSqZCCCFEPUkwbYaUUu8ppdKUUtudtkUqpX5WSu11PEc4pT2klNqnlNqtlJritH2YUmqbI+1VpZRq7M/SHCil4pVSS5VSu5RSO5RSdzu2yz2tJaWURSm1Xin1u+NePubYLveyjpRSJqXUFqXUd473ci/rSCl1yHEfflNKbXRsa5z7qbWWRzN7AOOAocB2p23PAg86Xj8IPON43Q/4HfAHugL7AZMjbT0wClDAj8A5Tf3Zmuh+xgJDHa9DgD2O+yb3tPb3UgHBjte+wDrgdLmX9bqn9wIfA9853su9rPu9PAREV9jWKPdTSqbNkNZ6BZBeYfMM4APH6w+AmU7bP9FaF2mtDwL7gNOUUrFAqNZ6rTa+HR86HdOmaK1TtdabHa9zgF1AR+Se1po25Dre+joeGrmXdaKU6gRMA95x2iz3smE1yv2UYNpytNdap4IRHIAYx/aOwFGn/ZIc2zo6Xlfc3qYppRKAIRglKrmndeColvwNSAN+1lrLvay7l4G/AXanbXIv604DPymlNimlbnZsa5T72eqXYGsD3NXl62q2t1lKqWDgS2CW1jq7mmYQuafV0FrbgMFKqXDga6VU/2p2l3tZBaXUuUCa1nqTUmqCJ4e42Sb30tUYrXWKUioG+Fkp9Uc1+zbo/ZSSactx3FH9gOM5zbE9CYh32q8TkOLY3snN9jZJKeWLEUg/0lp/5dgs97QetNaZwDJgKnIv62IMcJ5S6hDwCTBRKfVf5F7WmdY6xfGcBnwNnEYj3U8Jpi3Ht8C1jtfXAvOdtl+mlPJXSnUFegLrHdUZOUqp0x090a5xOqZNcXz+d4FdWusXnZLkntaSUqqdo0SKUioAOAv4A7mXtaa1fkhr3UlrnQBcBizRWl+F3Ms6UUoFKaVCSl8DZwPbaaz72dS9r+Thtkfa/4BUoATjV9KfgSjgF2Cv4znSaf+/Y/RE241TrzNguOPLtB94HcckHW3tAYzFqKbZCvzmePxJ7mmd7uVAYIvjXm4HHnFsl3tZv/s6gfLevHIv63YPu2H0zv0d2AH8vTHvp8yAJIQQQtSTVPMKIYQQ9STBVAghhKgnCaZCCCFEPUkwFUIIIepJgqkQQghRTxJMhfCAUmqOUuo+L57/f0qprUqpe7x1DW9SSs1TSl3keL1MKTXcy9ebpZQK9OY1hKgNmU5QiCamlOoAjNZad3GTZtZaW5sgW5U0p7wAs4D/AvlNnA8hACmZClElpdTfHescLgZ6O22/SSm1QRlren6plApUSoUopQ46pi1EKRXqWFvRVyl1l1Jqp6Pk+YmbS/0ExDjWYDzDUbJ7Uim1HLhbKTVJGetdblPGWrf+jmsccuy3Vim1USk1VCm1SCm1Xyl1axWf6SplrEf6m1LqTaWUybE9Vyn1L8dn+lUp1d6xfZ5S6kWl1FLgGaXUYEf6VqXU18ppbcgqrperlHrGMfH4YqXUaY7Pd0ApdZ5jH5NS6jnHPd2qlLrFsX2CY98vlFJ/KKU+Uoa7gDhgqSNfQjQ5CaZCuKGUGoYxxdsQ4AJghFPyV1rrEVrrQRjLuf1ZG0u7LcNYTgvHsV9qrUsw1lAcorUeCLgLcucB+7XWg7XWKx3bwrXW44F/A/OAS7XWAzBqk25zOvao1noUsNKx30UY64s+7uYz9QUuxZgMfDBgA650JAcBvzo+0wrgJqdDewFnaa3/irEc1QOOz7INeNTN53EWBCzTWg8DcoAngMnA+U55/DOQpbUegXGfb3JM7wbG/Z+FsfZkN0feX8WYK/VMrfWZNVxfiEYhwVQI984AvtZa52utszHm8SzVXym1Uim1DSMYJTq2vwNc73h9PfC+4/VW4COl1FWAp9WknzqeewMHtdZ7HO8/wFg8vlRpvrYB67TWOVrrE0Bh6Ry6TiYBw4ANylhCbRJGgAIoBr5zvN4EJDgd97nW2qaUCsMI8suryIs7xcBCpzwud/zA2OZ0jbOBaxx5Wocx/VtPR9p6rXWS1tqOMQ2kc76EaDakzVSIqlU11+Y8YKbW+nel1HUY86qitV6tlEpQSo0HTFrr7Y79p2EEnfOA2UqpRA/aHvMcz1WuE+dQ5Hi2O70ufV/x/7cCPtBaP+TmPCW6fG5RW4Vj89zs7ynn85blUWttV0qVXkMBd2qtF7lk1liWzPkzVcyXEM2GlEyFcG8FcL5SKsCxEsV0p7QQINXRPnplheM+xFio4H0ApZQPEK+1XoqxCHQ4EFyLfPwBJCilejjeXw0sr2b/6vwCXKSMtR5RSkUqpSp1eqqK1joLyFBKndEAeXG2CLjNqb25lzJW/ahODsa/gxDNgvzKE8INrfVmpdSnGFWLhzHaJEvNxqiOPIxRXen8R/0jjHbB/znem4D/OqpIFfCSNtYB9TQfhUqp64HPHSW5DcDcOn6mnUqpfwA/OYJ8CfAXx+fw1LXAXMewlAOUV2vXxzsY1beblVIKOAHMrOGYt4AflVKp0m4qmgNZNUaIBuQYazlDa311U+dFCNF4pGQqRANRSr0GnIOxVqoQog2RkqkQQghRT9IBSQghhKgnCaZCCCFEPUkwFUIIIepJgqkQQghRTxJMhRBCiHqSYCqEEELU0/8DwSmjstzujJkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_cumulative_dynamic_auc(risk_score, label, color=None):\n",
    "    auc, mean_auc = cumulative_dynamic_auc(y_train, y_test, risk_score, times)\n",
    "\n",
    "    plt.plot(times, auc, marker=\"o\", color=color, label=label)\n",
    "    plt.xlabel(\"days from enrollment\")\n",
    "    plt.ylabel(\"time-dependent AUC\")\n",
    "    plt.axhline(mean_auc, color=color, linestyle=\"--\")\n",
    "    plt.legend()\n",
    "\n",
    "\n",
    "for i, col in enumerate(num_columns):\n",
    "    plot_cumulative_dynamic_auc(x_test_imputed[:, i], col, color=f\"C{i}\")\n",
    "    ret = concordance_index_ipcw(y_train, y_test, x_test_imputed[:, i], tau=times[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot shows the estimated area under the time-dependent ROC at each time point and the average across all time points as dashed line.\n",
    "\n",
    "We can see that age is overall the most discriminative feature, followed by $\\kappa$ and $\\lambda$ FLC. That fact that age is the strongest predictor of overall survival in the general population is hardly surprising (we have to die at some point after all). \n",
    "More differences become evident when considering time: the discriminative power of FLC decreases at later time points, while that of age increases. The observation for age again follows common sense. In contrast, FLC seems to be a good predictor of death in the near future, but not so much if it occurs decades later.\n",
    "\n",
    "### Evaluating a Model's Predictions\n",
    "\n",
    "Most of the time, we do not want to evaluate the discriminatory power of individual features, but how a predictive model performs, based on many features. To demonstrate this, we will fit a survival model to predict the risk of death from the [Veterans' Administration Lung Cancer Trial](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.datasets.load_veterans_lung_cancer.html#sksurv.datasets.load_veterans_lung_cancer).\n",
    "\n",
    "First, we split the data into 80% for training and 20% for testing and use the `stratify` option to ensure\n",
    "that we do not end up with test data only containing censored death times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sksurv.datasets import load_veterans_lung_cancer\n",
    "\n",
    "va_x, va_y = load_veterans_lung_cancer()\n",
    "\n",
    "va_x_train, va_x_test, va_y_train, va_y_test = train_test_split(\n",
    "    va_x, va_y, test_size=0.2, stratify=va_y[\"Status\"], random_state=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we fit a Cox proportional hazards model to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('onehotencoder', OneHotEncoder()),\n",
       "                ('coxphsurvivalanalysis', CoxPHSurvivalAnalysis())])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cph = make_pipeline(OneHotEncoder(), CoxPHSurvivalAnalysis())\n",
    "cph.fit(va_x_train, va_y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the test data, we want to assess how well the model can distinguish survivors from deceased in weekly intervals, up to 6 months after enrollment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdEAAAEyCAYAAAClE+q3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABDi0lEQVR4nO3de3xU5ZnA8d9D7hcgZCIRAgoqoiAIgljvodaCtiparWhr7fZibdWt25YK7W7XbrerLb1ua4vWtWpbRa2I1KJ4jXcFEYQAosg9QSAJt1zI5PLsH3MmDMNMMpnMycyZPN/PJ5/MvOe857wvE/LkvOc9zyuqijHGGGO6r1+yG2CMMcZ4lQVRY4wxJk4WRI0xxpg4WRA1xhhj4mRB1BhjjImTBVFjjDEmTpnJbkBvKCkp0REjRtDQ0EBBQUGym5Mw1p/UZv1JbenWH0i/PqVSf5YvX16jqkeFl/eJIDpixAjeeecdKioqKC8vT3ZzEsb6k9qsP6kt3foD6denVOqPiGyJVG7DucYYY0ycLIgaY4wxcbIgaowxxsTJgqgxxhgTJwuixhhjTJwsiBpjjDFxsiBqjDHGxKlPPCdq0tPCFVXMXbKe6r1NDC3KY9a00cyYWJaydZPVXmOMeyyIGk9auKKKOQtW09TSBkDV3iZmL1hFW3s7l07oPLgsWlnFDxdWcrCl3dW6re1KS1u7K+ecs2A1gAVSY5LMgqjxpLlL1ncE0KCDLe1897FVfPexVd0+nmt1n33alXM2tbQxd8l6C6LGJJkFUeNJ1Xubom773qdP7LTuL579oFfqbty0ieNGjnTtnJ39GxhjeocFUeNJQ4vyqIoQRMqK8rj5k6M6rfvw0m29Ureioory8lGunXNoUV6n9Ywx7rPZucaTZk0bTV5WxmFleVkZzJo2OiXrJvqcAF8+a0SXdY0x7rIgajxpxsQyfnzpmI73ZUV53HHFuJjuEc6YWMYdV4yjrCgP6aW6iTxn6YAc8rL68djybTQ0t3ZZ3xjjHhvONZ51/OD+APzpS5O5cExpt+rOmFgW96SceOsm8pyvfVjDl+57m1l/f4+7rj0NEYnruMaYnrErUeNZa6r3AXBK2YAkt6T3nTOqhNumn8Ti1R9z9ysbk90cY/osC6LGsyqr9uEryOboAbnJbkpS3HDecXxm/BB+/sz7vPrh7mQ3x5g+yYKo8azKqv2MLRvYZ4cyRYS5V45n1OD+3PLwCrbVNSa7Scb0ORZEjSc1t7bxwc4DnDK07w3lhsrPzuTu6ybR3q7c8JflNPnbuq5kjEkYV4OoiEwXkfUiskFEZkfYPktEVjpflSLSJiLFIjI6pHyliOwXkVudOreLSFXItovd7INJTes/PkBruzKubGCym5J0I0oK+O01E3n/4/3MXrAKVU12k4zpM1wLoiKSAdwFXASMAa4RkTGh+6jqXFWdoKoTgDnAy6pap6rrQ8onAY3AEyFVfx3crqqL3eqDSV2VVfsBOMWCKABTRw/muxeeyJMrq7nv9c3Jbo4xfYabV6JTgA2qulFV/cB84LJO9r8GeDhC+QXAR6q6xYU2Go+qrN7HgNxMhg2yrD1B3yo/gWljS/mfxet446OaZDfHmD7BzSBaBmwLeb/dKTuCiOQD04HHI2yeyZHB9WYRWSUi94nIoEQ01njLmqp9nNKHJxVF0q+f8IurTmWEL5+bH1oRMVWgMSaxxK37JyJyFTBNVb/mvL8OmKKqt0TY92rgi6p6SVh5NlANjFXVnU5ZKVADKPATYIiqfiXCMW8AbgAoLS2dNH/+fOrr6yksLExkN5Oqr/antV258flGPnVMFjNPyu6FlsUnWZ/Pjvp2fvxmE0cX9OMHZ+SSnZGYPzT66s+bl6Rbn1KpP1OnTl2uqpOP2KCqrnwBZwJLQt7PAeZE2fcJ4NoI5ZcBz3ZyjhFAZVdtmTRpkqqqvvTSS5pO+mp/1lbv02Nve0oXrtjuboN6KJmfz7NrPtZjb3tKv/voSm1vb0/IMfvqz5uXpFufUqk/wDsaIb64mfZvGTBKREYCVQSGZa8N30lEBgLnA1+McIwj7pOKyBBV3eG8vRyoTGSjTeqrrApmKrJJRdFcOKaUf71gFP/7woegypsb66je28TQojxmTRsdU/rBhSuqmLtkfUe9zxzTRrn7TTfGU1wLoqraKiI3A0uADOA+VV0jIjc62+c5u15O4GqzIbS+c5/0QuAbYYf+uYhMIDCcuznCdpPmKqv2UZCdwUhfQbKbktJuvWAUz6/9mL+/W9VRVrW3iTkLVgN0GkgXrqhizoLVHQufV+1t4v79MGZFlS0EbkwIVxPQa+Dxk8VhZfPC3t8P3B+hbiPgi1B+XUIbaTynsno/Y4cOpF8/m1TUmX79hD2NLUeUN7W0MWfBal54f1fUus+v3dkRQIP87TB3yXoLosaEsFVcjKe0tStrq/czc8rwZDfFEz7edzBieVNLG2ucYfFo2yOpthm/xhzGgqjxlE019TS1tHHKULsfGouhRXkRH3UpK8rjxe+VR6139p0vRqw3tMieyzUmlOXONZ5imYq6Z9a00eRlZRxWlpeVwaxpo7tdr5/QZT1j+hq7EjWeUlm1j5zMfhx/lE0qikXw/mXoLNtYZueG1yvIyaC+uY3SPrrsnDHRWBA1nrK6ah8nDxlAZoYNosRqxsSyuCYDhdZr8rdx3h1LmL1gFc98+zzysjO6qG1M32C/iYxntDuTimzllt6Xl53BV07JYUttI79+/oNkN8eYlGFB1HjG1rpGDjS3ckpZ315DNFlO9mVwzZRjuPfVjby3bW+ym2NMSrAgajyjsjrwSMZYm5mbNHMuPonB/XP5/t9X4W9tT3ZzjEk6C6LGMyqr9pOVIZxY2j/ZTemzBuRm8dPLT2H9zgP8oWJDsptjTNJZEDWesaZ6H6OP7k92pv3YJtMFJ5dy2YSh3PXSBtZ/fCDZzTEmqey3kfEEVaWyap8lWUgR/3nJWPrnZvH9v79HW7s7yyka4wUWRI0nVO1tYk9jC2NtZm5KKC7I5vZLx/Le9n3c99qmZDfHmKSxIGo8oSNT0VCbmZsqLhk/hE+dXMovn1vP5pqGrisYk4YsiBpPWFO9j4x+wslDLIimChHhv2ecQla/fsxesIp2G9Y1fZAFUeMJlVX7GDW4kNwsy5STSo4emMsPP3Myb22s4+FlW5PdHGN6nQVR4wnBNURN6rn69OGcdbyPOxa/z459tlSa6VssiJqUt2v/QXYfaLZMRSlKRLjzivG0tSs/fKISVRvWNX2HBVGT8oKZimz5s9R1jC+f700bzYvv72LRe9XJbo4xvcaCqEl5q7fvRwSbVJTivnzWCCYeU8Tti9ZQU9+c7OYY0yssiJqUV1m9j5ElBRTm2Mp9qSyjn/Dzz42nobmN2xetSXZzjOkVrv5WEpHpwG+BDOBeVb0zbPss4AshbTkZOEpV60RkM3AAaANaVXWyU6cYeAQYAWwGPq+qe9zsh0muNVX7OH1kcbKbYWIwqrQ/t3zyBH753Ae88dFz7Gnwx7wQuDFe5NqVqIhkAHcBFwFjgGtEZEzoPqo6V1UnqOoEYA7wsqrWhewy1dk+OaRsNvCCqo4CXnDemzRVW99M9b6Dlu7PQ4YW5SJAXYMfJZBtas6C1SxcUZXsphmTcG4O504BNqjqRlX1A/OByzrZ/xrg4RiOexnwgPP6AWBGTxppUtua6kCmorE2M9czfvXch4TPz21qaWPukvVJaY8xbnIziJYB20Leb3fKjiAi+cB04PGQYgWeFZHlInJDSHmpqu4AcL4PTmirTUqxNUS9p3pv5GdFo5Ub42Vu3hOVCGXRHiC7BHg9bCj3bFWtFpHBwHMi8r6qvhLzyQOB9waA0tJSKioqqK+vp6KiItZDpLy+0J8XVxzkqDxhxduvJ6dRPdAXPp9IinOF2oNH/lcvzpWU+vdIt88H0q9PXuiPm0F0OzA85P0wINoDZDMJG8pV1Wrn+y4ReYLA8PArwE4RGaKqO0RkCLAr0gFV9R7gHoDJkydreXk5FRUVlJeX96BLqaUv9OdHS1/i9BMGUF4+KTmN6oG+8PlE8h8Dq5izYDVNLW0dZXlZGfzHZeMoT6HJRen2+UD69ckL/XFzOHcZMEpERopINoFAuSh8JxEZCJwPPBlSViAi/YOvgU8Dlc7mRcD1zuvrQ+uZ9LKvsYWtdY02lOsxMyaWcccV4ygryuso+/50m51r0pNrV6Kq2ioiNwNLCDzicp+qrhGRG53t85xdLweeVdXQtZRKgSdEJNjGh1T1GWfbncCjIvJVYCtwlVt9MMm1ZodlKvKqGRPLmDGxjG11jZw39yX2NbUku0nGuMLV50RVdTGwOKxsXtj7+4H7w8o2AqdGOWYtcEEi22lS0xpbQ9Tzhhfnc84JJTy6bBu3fHIUGf0iTZUwxrssY5E5zMIVVZx954uMnP1Pzr7zxaQ+21dZvY+hA3PxFeYkrQ2m52aefgzV+w7y6oe7k90UYxLOgqjpsHBFYEJI1d6mlHhIvrJqH2NtKNfzLhxTSnFBNvOXbut6Z2M8xoKo6TB3yfrDZlRC8h6Sr29uZWNNg2UqSgPZmf343GllPL9uJ7sPWGJ6k14siJoOVSn0kPy6HftRxdYQTRNXn34Mre3K4+9uT3ZTjEkoC6IGVWXeyx9F3T405FGF3lJZZTNz08kJgwuZMqKYR5Zts0W7TVqxINrHNfnb+Pb8ldz59PtMGD6Q3KzDfyTysjKYNW10r7ersmo/JYU5DO5vk4rSxdWnD2dTTQNvb6rremdjPMKCaB9WvbeJq+5+g3+sqmbWtNE88a2zufOK8fgKsgHwFWRzxxXjkvKQ/JrqfYwrG4DzrLBJAxePG0L/3EzmL92a7KYYkzAWRPuodzbXcenvX2NzTSP3fmkyN009ARFhxsQyXvn+VDL6CddMOSYpAfRgSxsf7qq3odw0k5edwYwJZSyu/Ji9jf5kN8eYhLAg2gc9vHQr1/zpLfrnZrHwprO44OTSw7YX5GRyytABLE3SsNu6Hftpa1dL95eGZk4Zjr+13dYWNWnDgmgf0tLWzo+erGTOgtWceXwJC791NicM7h9x3ykji1m5fS8Hwx556Q2VzhqiNjM3/YwdOpDxwwYy3yYYmTRhQbSPqK1v5rr/e5sH39zCN847jj9/+XQG5mdF3X/KSB/+1nZWbd/Xi60MWFO1j6L8rMMSmJv0cfXpw3n/4wO8l4SfLWMSzYJoH7C2ej+X/v513t26l19ffSpzLj65yxymk48dBMCyzb0/pFtZvY9Thg60SUVp6tJTh5KXlWETjExacDUBvYnfwhVVzF2ynuq9TQwtymPWtNiXkgqtOyg/iwMHW/EV5vDYN87k1OFFMR1jUEE2o0v78/amOm6a2oOOdJO/tZ31Hx/gK+eM7L2Tml7VPzeLz44fwqL3qvn3z46hMMd+DRnvsivRFNSTHLbhdesaW2hV5VtTj4s5gAadPnIQyzfX0drWHlc/4vHBzgO0tCnjbGZuWps55Rga/W089V51sptiTI/Yn4ApKFoO2+88upKfPLW2o8zf4if71ecO229Po5/2sPkaqnD3y5v40pndu7qbMtLHX9/ayrodBxg3rHeC2ppqJ1ORzcxNa6cdU8SJpYU8vGwbM6cck+zmGBM3C6IpKFqu2naFi8YdfWi/qmqGlh192D5/fSvyfaZ48t9OGVEMwNubanstiK6u2kf/nEyOKc7vlfOZ5BARrj79GH7y1FrW7djPyUNsJrbxJguiKWhoUV7EZPBlRXn894xxHe8rKmopLx932D4vvb87Yt148t8ePTCXY4rzWba5jq+de1y368ejsmo/Y4YOoJ8t3pz2rphYxs+efp9Hlm3j9kvHJrs5xsTF7onGoLcXqp41bTR5WRmHlcWaw7YndSOZMrKYpZvqeuWZvrZ2Zd2O/ZapqI8YVJDNtFOOZsG725PyPLIxiWBBtAvJWKh6xsQy7rhiXEcwLCvKizmHbbBuWVEe0s26kUwZUcyexhY27KqPq3537GhQmlvbLclCH3LN6cPZf7CVZyo/TnZTjImLDed2obOFqt3MKztjYhlPrKhiT6OfRTef0+26iWrblJGB+6JLN9cxqjRydqNE2bI/8O9sk4r6jk8c5+NYXz4PL92alDzNxvSUXYl2IdqEnN5YqLquwU+xs6JKshzry2dw/5xeyaO7eX87eVkZHHdUoevnMqmhXz/h85OH8/amOjbudn+0w5hEixpEReQ7IvLVCOW3iMitsRxcRKaLyHoR2SAisyNsnyUiK52vShFpE5FiERkuIi+JyDoRWSMi3w6pc7uIVIXUuzjGvsYl2oSc3lioOhWCqIhwei/dF92yv50xQwd0mU3JpJerJg0jo5/wyDvbkt0UY7qtsyvRrwB/iVB+j7OtUyKSAdwFXASMAa4RkTGh+6jqXFWdoKoTgDnAy6paB7QC31XVk4FPADeF1f11sJ6qLu6qLT2R6Ik6sVJVauqbKSlM/qLUZ4wsZse+g2zf497Vd3u7snV/O6cMtfuhfc3gAbl88qTBPL58O/7W3kvsYUwidBZEVVWPWPRPVZuBWC4VpgAbVHWjc5z5wGWd7H8N8LBzjh2q+q7z+gCwDkjKDZPgRJ2jnGDWWwtVN/rbaG5tT/qVKMDpzvOibg7pbqpt4GAbjLWZuX3SNVOGU1Pv58X3dya7KcZ0S6f3REWkNJayKMqA0PGZ7UQJhCKSD0wHHo+wbQQwEXg7pPhmEVklIveJyKAY2xO3GRPLePybZwEw5+KTe2UCRG194O8XXwoE0dGl/RmQm+laMvqFK6q48o9vAPCLJettrck+6PwTB3P0gFweXmpDusZbOpudOxf4p4h8F3jXKZsE/Bz4RQzHjnS1Gu2m2iXA685Q7qEDiBQSCKy3qup+p/iPwE+cY/0E+CURhpdF5AbgBoDS0lIqKiqor6+noqIihqYf6WBroOlLV62l5MCGuI7RHR/tDcxUrdq4nor6jyLu05P+dNdx/ZWKNdup8CU2kL5R3cL9lX78zijergPNfP+xlaxdt5azhkZfqs0LevPz6Q1u92fKUW3844Pd/P3pFynJc3/OY7p9PpB+ffJCf6IGUVV9UER2A/8FnOIUVwL/qapPx3Ds7cDwkPfDgGjZpmfiDOUGiUgWgQD6N1VdENKunSH7/Al4Kkr77yFw/5bJkydreXk5FRUVlJeXx9D0iMcj9+VnKCodTnn5yXEdozva1u2Et96h/MzJTIiSOL4n/emu9fIRdzz9PmMnnclR/RN3n/aHd77YEUCD/O3wz60Z/ODa8oSdJxl68/PpDW735/jxjfxj7ktsyxzGleUnunaeoHT7fCD9+uSF/nT6556qPq2q56uqz/k6P8YACrAMGCUiI0Ukm0CgXBS+k4gMBM4HngwpE+D/gHWq+quw/YeEvL2cQGB3nYjgK8ihpr65N06XUsO5cOh50UQP6SbzESKTWoYX53POCSU89s422sJXUTAmRUW9EhWR33H48KsCNcBLqvpaVwdW1VYRuRlYAmQA96nqGhG50dk+z9n1cuBZVW0IqX42cB2wWkRWOmU/cGbi/lxEJjjt2Qx8o6u2JEpJYXZHcHNbbYMTRAtTI4ieUjaQvKwMlm6q4+JxQ7quEKNoeYJ74xEik3qOP6qAVz+s4fgfLKasm+voGpMMnd0TfSdCWTEwV0QeUdXfdHVwJ+gtDiubF/b+fuD+sLLXiDIDWFWv6+q8bvEV5rDrwMFeOVddQzO5Wf3Iz06NpFJZGf047diihM/Q/d6nT+Q7j7532F9rvfEIkUk9C1dUMX/ZoYlFwRSbgAVSk7I6uyf6QKRyEZkHvAH8xqU2pSxfQTbv79jf9Y4JUFvvx1eQ/GdEQ00Z4eM3L3zAvqYWBuYlZtLPMb4CFCjKy2JvU4tdffRhc5es52DL4TfIeyPFpjE90e0pcKraZ29W+QpzqGnw98qKJrUN/pQZyg06feQgVOHdLXsSdsyH3t5KYU4mr8/+JPdPL+D12Z+0X5h9VLT74FV7m3hixXaaW22lF5N6uhVERSRTRP6FwMzbPqekMBt/azv1za2unysVUv6Fmzh8EFkZwtsJGtLd19jCU6uquWzCUApyUmPY2iRPtPvgGf2Ef3vkPc6640XmLnk/4j10Y5Kls9y5B0Rkf+gXUEUgjV+vTeZJJcGg1huTi2rrm1NuODcvO4Pxw4oSNkN3wYrtNLe2c82UYxJyPONt0VJs/uJz43nwK1OYeMwg/ljxEef+7EVuePAdXt9Q0yujQsZ0prN7ou6ue+VBPif1X21DMyNKClw7j6qm5HAuBFIA/t9rG2nyt5GXndF1hShUlYfe3sqpwwbaItwGODR5aO6S9VTvbWJo2P3x8048iu17Gvnb21t5ZNk2nl27k+OPKuC6TxzL5yYN44V1u6LW7crCFVVx1Y23ntfOaaLr1hiaiBxPIMftTFU9pav9003wmc0al69EUylvbrgzRhYz7+WPWLFtD2cdXxL3cd7ZsocPd9Xzs8+NS2DrjNd1tRbusEH53Db9JL59wSj+uWoHD761hdv/sZafLl5Hu9LxfGl3ZvYuXFHFnAWrO9YNjrVuvPW8ds7Q+r0d9N+obuGHd77Yq+fsri6DqJPcYCaB4DkeuMN53ecEV1Rxezg31RIthDrt2EGIwLJNPQuiD729lf45mVxy6tAEts70FblZGXxu0jA+N2kYq7bvZeY9b9HoP3ziUVNLG995dCX//c+1EY/h9/vJfu056hr8hOd26KouEHe9ntTtql6wT92p+4MnVrPu4/2UFOTgK8zGV5iDryAbX2E2xQXZ5GRmJC3oh6YE7Y1zxqOzZAtfJxAshwGPAl8DnlTVHye8FR5x6J6ou1mLahsCx0/F4dyBeVmcfPQAlm6uBUbFdYw9DX7+uXoHV08enjLPwRrvGj+siCZ/5Jm77QrTxh4dcVt1dTVDhx7N397e2u26QNz1elK3q3rBPnWnbqO/jT+/thl/W+Rl6PrnZtLobzsii1RTSxuzF6zi2bUfR23vi+/vivjYUlf1gnXDU4L25JxuPSrV2W+wu4A3gWtV9R0AEenTd/GzM/sxIDezI5uQWw5diabWxKKgKSOLeWTZNlra2snK6H6i8MffDawbee0ZNqHIJEa0zFdlRXn89PLItwwqKmopLx9Hxfrd3a4LxF2vJ3W7qhfsU3frvnbbVA40t1Jb76euoZmaej+19X5q65upbfBz/xubIx7zYEs7H+6sj9re8GAWa72e1I1Wz61Uop0F0aHAVcCvnOXPHgW8vaxGApQUup8/t84J0ql4TxQCQfT+NzZTWbWPicd0byU6VeXhpVuZeEwRJw+xBbhNYsyaNvqwITyIPfNVvHXT6ZwiwoDcLAbkZjEywqTJ59bujBqAn/vO+VHPefadL8ZVryd1o9VzK5Vo1MsIVa1R1T+q6nnABcA+YJeIrBOR/3GlNR7g64X8uamWNzdcTxbpXrqpjo92N3CtPdZiEmjGxDLuuGIcZUV5CIFftHdcMS6m4bt46/aVc0L0x49iCfrx1AvWzQ6LUG6fMx4x3ZBS1e0E1hD9hYiMJjDRqE/yFeSwsabzYYieqq1vJi8rI2XvFx7VP4fjSgpYtrmOb5x/fLfqPrR0K/1zM/nseJtQZBKrq5m9btTtS+eE6I8fJbpesO7adWv559aMXjtnPLr9W1pV1wOemly0cXcDV9/9Jnv3NvHH9W8C8NnxQ7juzBE0+dv48p+XHlHnyknDuGrycOoa/Hzzr8s7yjfVNFDX4Ocf71VzyalDqd7bxL89svKI+l8/9zg+NaaUj3bX8wNnZlioWz45inNGlbCmeh//9Y/DZ+N9tLuegpzAX1LLt9Tx82fWH1H/R5eMAeC1D2v43YsfHrH9f64Yx/FHFfL82p386dWNR2z/9dUTGFqUxz/eq+avb205YvsfvziJ4oJsHntnG39ffmSCqtOOKeK5dbt44I3NLF6944jtj3zjTADueeUjXli3C4DWduXdLXsYWpTX8Yzp/77wIa9vqAHo+HwG5Wcz77pJAPzsmfePSDM4ZGAuv5k5EYAf/2MNa6sPz2d83FEF3HHFeADmLFjFxt0Nh20fM3QA/3nJWABunb+CHfsOX1TgtGMHcdv0kwC48S/L2dN4+MjD2SeU8K8XBCZVXX/fUg62HD6p5YKTB3PDeYE/Lq6++80j/m3i/dkL+uInjnXtZw/g+9NHM+nY4iN+9oKfz48uGcPYoQOT9rN3/79MIS87g7+8uZmnVsX2sxeUm5XBA1+ZAsCTG/wdvw+CvP6zNyK7heDym2787M2+6KTDfvYeXnr4ZKVoP3vDBuUxbFBet3/29u5tZdigQoYNCgzFjiotBLr+vVeYk9lxToCHl27l4aVbe/yzF4n7y8enmayMfrS2K+0uZkppadOEJXh3y+QRxexraunWqja7DzSjBH4RGWNMWlDVtP+aNGmSqqq+9NJL2lMPvLFJj73tKd194GCPjxXNZ/73Ff3yfW93uV8i+hOvrbUNeuxtT+kDb2yKaf/29nadOvclveIPr0fdJ5n9cYP1J7WlW39U069PqdQf4B2NEF+6vBIVkRdiKesrgo+duDm5qK7eT3GKPt4SNGxQHkMH5sacjP7NjbVsrLEJRcaY9NJZsoVcIB8oEZFBHFokewCBx1/6pOCM2UDChcSnF9YUzpsbSkQ4fWQxb35Ui6oiEnEN9Q4Pvb2VgXlZfGb8kF5qoTHGuK+zK9FvAMuBk5zvwa8nCSRi6JNKnOBW41LChQYnb24qpvwLN2VkMbsONLOltrHT/Wrqm1my5mOuOK2M3Kz4k9YbY0yq6WwVl98CvxWRW1T1d73YppR2aDjXnYQLdfWpnWgh1BkjnedFN9d1uqrN35dvp6VN+YJlKDLGpJkuH3FR1d+JyFnAiND9VfVBF9uVsgbmZZHRT1y7J5rKeXPDHX9UIcUF2SzdVMfnJw+PuE97eyBD0ZQRxZww2FbXM8akl1hWcfkLcDywEgg+lKRAnwyi/foJxQXZHcEu0VI9b24oEeH0EYM6XaT7zY21bKlt5N8+dWIvtswYY3pHLMkWJgNjnCm+3SIi04HfAhnAvap6Z9j2WcAXQtpyMnCUqtZFqysixcAjBK6MNwOfV9XDn4h2ma8g27U1RVM9b26400cUs2TNTj7ed5CjIzz/+dDbWxmUn8X0Uzpf1cIYY7wolmQLlUC3fwOKSAaBCUgXAWOAa0RkTOg+qjpXVSeo6gRgDvCyE0A7qzsbeEFVRwEvOO97VUlhjmv3RGs8NJwLcMZIHxC4Lxpu94HAhKLPnTbMJhQZY9JSLEG0BFgrIktEZFHwK4Z6U4ANqrpRVf3AfOCyTva/Bng4hrqXAQ84rx8AZsTQloTyFWa7thxaXb0/pfPmhjt5SH8KczJZFuF50ceWb6O1XbnGJhQZY9JULL+pb4/z2GXAtpD324EzIu0oIvnAdODmGOqWquoOAFXdISKDoxzzBuAGgNLSUioqKqivr6eioiK+3oRo2tPMzr2tCTlWuLUbm8nPaI/p2InqT0+N6K+8VLmViqKajrJ2Vf78ShOjB/Vj25p3Dvswo0mV/iSK9Se1pVt/IP365IX+xDI792URORYYparPOwEvlrG5SE/fR7uvegnwuqoGL2e6UzciVb0HuAdg8uTJWl5eTkVFBeXB7Mw9sEY38OyW9Xzi7HMTPkx538alDOvnp7z8nC73TVR/emqNbmDukvVMmHIWRfmBYehXPtjN7qal/Ptlp1I+IbbVE1KlP4li/Ult6dYfSL8+eaE/saT9+zrwd+Bup6gMWBjDsbcDoc89DAOqo+w7k0NDuV3V3SkiQ5y2DQEOX6ahFwQTLrgxpFvX0OyZSUVBwfVFl20+NL/robe3UlyQbROKjDFpLZZ7ojcBZwP7AVT1QyDiEGqYZcAoERkpItkEAuUR91JFZCBwPoFMSLHUXQRc77y+Pqxer3Az4YIX8uaGGz9sINmZ/Vi6qRaAXfsP8ty6nVw5aRg5mTahyBiTvmK5J9qsqv5gblQRySSGoVVVbRWRm4ElBIZ/71PVNSJyo7N9nrPr5cCzqtrQVV1n853AoyLyVWArcFUMfUioQ/lzE3slqqrUNPg7rnS9IjcrgwnDi1jqXIk++s422tqVayzZvDEmzcUSRF8WkR8AeSJyIfAt4B+xHFxVFwOLw8rmhb2/H7g/lrpOeS1wQSznd0tJYeBKsSbBV6IN/jb8re2eG84FmDKimD++/BEHDrbw8NJtnHW8j5GdpAI0xph0EMtw7mxgN7CaQFL6xcC/u9moVOdz6Z6ol/Lmhpsyspi2duW3z39I1d4mrrXHWowxfUAss3PbgT85XwbIz84kLysj4fdEg4kWgle6XlK9twmAe1/bRD+Bg/62LmoYY4z3dbae6Go6ufepquNdaZFH+AqzE35P1KtXogtXVPHjf6zteN+u8B9PriEzox8zJsb2eIsxxnhRZ1ein3W+3+R8/4vz/QtA5wtI9gG+wpyErynqtby5QXOXrKep5fArz6aWNuYuWW9B1BiT1jpbT3QLgIicrapnh2yaLSKvA//lduNSWUlBNh/vP5jQY3otb25QcCg31nJjjEkXsUwsKhCRjvQ5ztqifX7apVvDuV7Kmxs0tCivW+XGGJMuYgmiXwXuEpHNIrIZ+APwFVdb5QG+whxqG5qJY4W4qGob/J67CgWYNW00eWHpD/OyMpg1bXSSWmSMMb0jltm5y4FTRWQAIKq6z/1mpT5fQTYtbcr+g60MzMtKyDFrG/z4PHY/FOi47zl3yXqq9zYxtCiPWdNG2/1QY0za6zKIikgO8DkCi2BnBjMXqWrfvidaeCj1X6KCaF1DM0d58PEWCARSC5rGmL4mluHcJwms4dkKNIR89WluJFyorffj82gQNcaYviiWGSzDVHW66y3xmEQnoVdVzw7nGmNMXxXLlegbIjLO9ZZ4TPBKtCZBM3S9nDfXGGP6qliuRM8Bviwim4BmAgtma1/PWDQoP7EruQSvaG041xhjvCOWIHqR663woOzMfgzMy6K2ITHDucF7qzaca4wx3tHlcK6TuWg48EnndWMs9fqCRCZc8GreXGOM6cu6DIYi8p/AbcAcpygL+KubjfKKkoKchK0pWuvRlH/GGNOXxXJFeTlwKc5jLapaDfR3s1Fe4SvMTtgjLoeGc+2eqDHGeEUsQdSvgdx2CiAifT5vblBgODcxV6LBvLl52Rld72yMMSYlxBJEHxWRu4EiEfk68Dy2QDcQuGrc09hCa1t7j4/l1by5xhjTl8WSO/cXInIhsB84EfiRqj7ness8oMQJenWNfgb3z+3RsSzRgjHGeE+ss2xXA68CrzivYyIi00VkvYhsEJHZUfYpF5GVIrJGRF52ykY7ZcGv/SJyq7PtdhGpCtl2caztSbTgM511CbgvWlvfbM+IGmOMx8QyO/drwFLgCuBK4C0R6XIpNBHJAO4i8JzpGOAaERkTtk8RgaXVLlXVscBVAKq6XlUnqOoEYBKBx2qeCKn66+B2VV3cZS9dErxyTMRjLnUNfnu8xRhjPCaWZAuzgImqWgsgIj7gDeC+LupNATao6kan3nwCiezXhuxzLbBAVbcCqOquCMe5APjIeUY1pQSvHHv6mIvlzTXGGG+KZTh3O3Ag5P0BYFsM9crC9tvulIU6ERgkIhUislxEvhThODOBh8PKbhaRVSJyn4gMiqEtrgjeE+3plWh9cyv+1nabWGSMMR4jgadXOtlB5EFgHIEl0ZTA1eRS4AMAVf1VlHpXAdNU9WvO++uAKap6S8g+vwcmE7jazAPeBD6jqh8427OBamCsqu50ykqBGqctPwGGqOoRw8sicgNwA0Bpaemk+fPnU19fT2FhYQz/LLFpV+XrzzZy0cgsrjwx/gC4q7Gd77/SxNfGZXNOWexrkya6P8lm/Ult1p/Ul259SqX+TJ06dbmqTg4vj2U49yPnK+hJ53tXCRe2E0gXGDSMQEAM36dGVRuABhF5BTgVJ0ATuJ/6bjCAAoS+FpE/AU9FOrmq3gPcAzB58mQtLy+noqKC8vLyLprdPb43nqfQN5jy8vjz8b+7dQ+88gZnTzqV8pMGx1zPjf4kk/UntVl/Ul+69ckL/YnlEZcfQyDJghPsYrUMGCUiI4EqAsOy14bt8yTwexHJBLKBM4Bfh2y/hrChXBEZoqo7nLeXA5XdaFPC+QpzerwcWnA42IZzjTHGW2KZnXumiKwF1jnvTxWRP3RVT1VbgZuBJU7dR1V1jYjcKCI3OvusA54BVhEYIr5XVSud8+QDFwILwg79cxFZLSKrgKnAv8XWVXeUFGb3eCWXOqe+zc41xhhviWU49zfANGARgKq+JyLnxXJw5/GTxWFl88LezwXmRqjbCPgilF8Xy7l7i68gmy21jT06huXNNcYYb4op2YKqhs/GbXOhLZ7kK8zpcf7c2no/+dmWN9cYY7wmliC6TUTOAlREskXkezhDuyZwH7PB30aTP/6/KyzRgjHGeFMsQfRG4CYCz3huByY47w2BNUWBHt0XtUQLxhjjTbHMzq0BvtALbfEkX0jChWGD8uM6Rm19M6UDepbA3hhjTO+LGkRF5Hc4a4hGoqr/6kqLPCaY+q8nV6J1DX5OHjIgUU0yxhjTSzobzn0HWA7kAqcBHzpfE7CJRR2Cw7DxPivakTfXnhE1xhjPiXolqqoPAIjIl4GpqtrivJ8HPNsrrfMAXw/z53bkzbV7osYY4zmxTCwayuEp/gqdMgPkZ2eSn50R92MuwbVIi+0ZUWOM8ZxYki3cCawQkZec9+cDt7vWIg/yFWZ3JEzorhpL+WeMMZ4Vy+zcP4vI0wTy2gLMVtWP3W2Wt/gKcuJeU7SuI1uRBVFjjPGaWDMWfayqTxJYnNsCaJiSwuy474la3lxjjPGumIJoiEtdaYXH+Qpy4n7EpWM41+6JGmOM53Q3iIorrfA4n3Ml2tUC55HUNVjeXGOM8aruBtFJrrTC43yFObS2K/ubWrtd1/LmGmOMd8WynuiJIvKCiFSqaruIjBeRf++NxnlFiTOztiaOId2a+uaOrEfGGGO8JZYr0T8Bc4AWAFVdBcx0s1FeE7yfGc/kojpLPm+MMZ4VSxDNV9WlYWXdH7dMY4eyFnX/StSGc40xxrtiCaI1InI8TjJ6EbkS2OFqqzzG1zGc270rUVWltt7y5hpjjFfFkrHoJuAe4CQRqQI2AV90tVUeU5wf35VofXMr/jbLm2uMMV4VS8aijcCnRKQA6KeqB9xvlrdkZvRjUH5Wt++JHspWZBOLjDHGi7oMoiJSBHwJGAFkigQeFbX1RA9XXJDd7YQLwUQLxTaca4wxnhTLPdHFBALoagLriwa/uiQi00VkvYhsEJHZUfYpF5GVIrJGRF4OKd8sIqudbe+ElBeLyHMi8qHzfVAsbXGbrzCn22uKWt5cY4zxtljuieaq6ne6e2ARyQDuAi4EtgPLRGSRqq4N2acI+AMwXVW3isjgsMNMVdWasLLZwAuqeqcTmGcDt3W3fYlWUpjN+o+7N9IdvIdqz4kaY4w3xXIl+hcR+bqIDHGuAotFpDiGelOADaq6UVX9wHzgsrB9rgUWqOpWAFXdFcNxLwMecF4/AMyIoY7rAvlzu3clWmtXosYY42nSVb5XEbkJ+CmwF+cxF0BV9bgu6l1J4Arza87764AzVPXmkH1+A2QBYwks/P1bVX3Q2bYJ2OOc825Vvccp36uqRSHH2KOqRwzpisgNwA0ApaWlk+bPn099fT2FhYWd9jdeCzf4WbihhXs/nU9mv9hSDD+8rpmK7a3cfWFBXOd0sz/JYP1Jbdaf1JdufUql/kydOnW5qk4OL49lOPc7wAkRhlW7EimShEfsTAL5eC8A8oA3ReQtVf0AOFtVq50h3udE5H1VfSXWkztB9x6AyZMna3l5ORUVFZSXl3ezG7HZlruFhRsqGT/5TAYPyI2pzsKPVzD4wJ642+Rmf5LB+pParD+pL9365IX+xDKcuwZojOPY24HhIe+HAdUR9nlGVRucIP0KcCqAqlY733cBTxAYHgbYKSJDAJzvsQwBu67EGZLtzuSi2gY/xfZ4izHGeFYsQbQNWCkid4vI/wa/Yqi3DBglIiNFJJtAvt1FYfs8CZwrIpkikg+cAawTkQIR6Q/gPJ/6aaDSqbMIuN55fb1zjKQLTg7qzmMuljfXGGO8LZbh3IXOV7eoaquI3AwsATKA+1R1jYjc6Gyfp6rrROQZYBXQDtyrqpUichzwhPNMaibwkKo+4xz6TuBREfkqsBW4qrttc8Oh/LnduBKt9zNmyAC3mmSMMcZlsWQseqCrfTqpu5jAc6ahZfPC3s8F5oaVbcQZ1o1wzFoC91BTSokzLFsTY+o/VQ0kn7dEC8YY41lRg6iIPKqqnxeR1Rw5IQhVHe9qyzxmQF4mmf0k5sdcLG+uMcZ4X2dXot92vn+2NxridSKCrzA75iT0wWFfy5trjDHeFXVikaoGlzv7lqpuCf0CvtU7zfMWX0FOzPdEg1esNpxrjDHeFcvs3AsjlF2U6IakA19hdsxrigbz5pbYlagxxnhWZ/dEv0ngivM4EVkVsqk/8LrbDfOiksIcNtU0xLRvcNjXrkSNMca7Orsn+hDwNHAHgSTvQQdUtc7VVnmUryC728O5NrHIGGO8K2oQVdV9wD7gmt5rjrf5CnNoammj0d9KfnbnTw/V1vspyM4gNyujl1pnjDEm0WK5J2pi1J2EC3UNzTaUa4wxHmdBNIFKCoP5c7t+zMXy5hpjjPdZEE2g4DOfsVyJ1tb7O5LWG2OM8SYLognUMZwbQxL6ugY/xRZEjTHG0yyIJlDHlWgXz4pa3lxjjEkPFkQTKC87g4LsjC6Hcw84eXMt0YIxxnibBdEE8xXmdJk/t84Jsjaca4wx3mZBNMF8hdldDuda3lxjjEkPFkQTzFeQQ00Xw7nBK1UbzjXGGG+zIJpgJTEsh1ZnV6LGGJMWLIgmmK8wm7oGP+3tR6xj3sHy5hpjTHqwIJpgvoIcWtuV/Qdbou5jeXONMSY9WBBNMF9H6r/o90Utb64xxqQHV4OoiEwXkfUiskFEZkfZp1xEVorIGhF52SkbLiIvicg6p/zbIfvfLiJVTp2VInKxm33orpLCYOq/6PdFaxv8HYkZjDHGeFfn63X1gIhkAHcBFwLbgWUiskhV14bsUwT8AZiuqltFZLCzqRX4rqq+KyL9geUi8lxI3V+r6i/cantPHEr9F/1KtLbez5CBub3VJGOMMS5x80p0CrBBVTeqqh+YD1wWts+1wAJV3Qqgqruc7ztU9V3n9QFgHVDmYlsT5lAS+uhXopY31xhj0oObQbQM2BbyfjtHBsITgUEiUiEiy0XkS+EHEZERwETg7ZDim0VklYjcJyKDEtzuHhmUn4VI9HuiqkptQzO+QhvONcYYrxPV6I9i9OjAIlcB01T1a87764ApqnpLyD6/ByYDFwB5wJvAZ1T1A2d7IfAy8FNVXeCUlQI1gAI/AYao6lcinP8G4AaA0tLSSfPnz6e+vp7CwkJX+hvqlhcaOP3oTL409shA2diifOuFRq4enc1FI7N6dJ7e6k9vsf6kNutP6ku3PqVSf6ZOnbpcVSeHl7t2T5TAlefwkPfDgOoI+9SoagPQICKvAKcCH4hIFvA48LdgAAVQ1Z3B1yLyJ+CpSCdX1XuAewAmT56s5eXlVFRUUF5e3uOOdeXod18mZ2Ah5eWTjti2uaYBXqhgyviTKZ80rEfn6a3+9BbrT2qz/qS+dOuTF/rj5nDuMmCUiIwUkWxgJrAobJ8ngXNFJFNE8oEzgHUiIsD/AetU9VehFURkSMjby4FK13oQJ19hdtSVXIJrjfrsERdjjPE8165EVbVVRG4GlgAZwH2qukZEbnS2z1PVdSLyDLAKaAfuVdVKETkHuA5YLSIrnUP+QFUXAz8XkQkEhnM3A99wqw/x8hXmsG7H/ojbgsHVHnExxhjvc3M4FyfoLQ4rmxf2fi4wN6zsNUCiHPO6BDcz4UoKol+JWt5cY4xJH5axyAXFBTnsa2rB39p+xDbLm2uMMenDgqgLgvc79zQeeTVqeXONMSZ9WBB1QUlH/twjEy7U2TOixhiTNiyIusDXkT83wpWoZSsyxpi0YUHUBcH7ncHHWULV1vvtfqgxxqQJC6Iu6PxKtNmeETXGmDRhQdQFA3IzycqQI/LnqqqTfN7uiRpjTDqwIOoCEcFXkHPESi4HmltpaVMbzjXGmDRhQdQlvsLsI9YU7chWZMO5xhiTFiyIusRXeOSVaJ0z0chm5xpjTHqwIOqSkoLsI+6JWt5cY4xJLxZEXRIYzm0mdL3WjpR/NpxrjDFpwYKoS3yFORxsaafR39ZR1pF83oZzjTEmLVgQdUlHwoWQIV3Lm2uMMenFgqhLSpyECzUhWYtqLW+uMcakFQuiLgne9wy9Eq2zvLnGGJNWLIi65FDqv5ArUcuba4wxacWCqEsOJaEPuSdqeXONMSatWBB1SW5WBoU5mR1rilreXGOMST8WRF3kK8zuuCe6/2Agb26JXYkaY0zasCDqIl9BdseaovaMqDHGpB9Xg6iITBeR9SKyQURmR9mnXERWisgaEXm5q7oiUiwiz4nIh873QW72oScC+XMDwdPy5hpjTPpxLYiKSAZwF3ARMAa4RkTGhO1TBPwBuFRVxwJXxVB3NvCCqo4CXnDep6SSwkP5c4PfS+w5UWOMSRtuXolOATao6kZV9QPzgcvC9rkWWKCqWwFUdVcMdS8DHnBePwDMcK8LPeMryKGuoZn2drXhXGOMSUOZLh67DNgW8n47cEbYPicCWSJSAfQHfquqD3ZRt1RVdwCo6g4RGRzp5CJyA3ADQGlpKRUVFdTX11NRUdGjTnXHnh0ttCv88/kK3tnWAkDl8rf4IEMScvze7o/brD+pzfqT+tKtT17oj5tBNFKk0LD3mcAk4AIgD3hTRN6KsW6nVPUe4B6AyZMna3l5ORUVFZSXl3fnMD2y/71q/vb+Ck46dTLvNm+lcOt2Pn3B1IQdv7f74zbrT2qz/qS+dOuTF/rjZhDdDgwPeT8MqI6wT42qNgANIvIKcGoXdXeKyBDnKnQIsIsUVeIM3dbU+y3lnzHGpCE374kuA0aJyEgRyQZmAovC9nkSOFdEMkUkn8CQ7bou6i4CrndeX+8cIyV1pP5raLYgaowxaci1K1FVbRWRm4ElQAZwn6quEZEbne3zVHWdiDwDrALagXtVtRIgUl3n0HcCj4rIV4GtODN6U1FoEvqaej9lRblJbpExxphEcnM4F1VdDCwOK5sX9n4uMDeWuk55LYF7qClvUH42IoEk9HUNzYwrG5DsJhljjEkgy1jkoox+QnF+Nrs77onaM6LGGJNOLIi6zFeYzZbaBsuba4wxaciCqMt8BTl8sLMesEQLxhiTbiyIusxXmN2xHJrPUv4ZY0xasSDqstBcuT67EjXGmLRiQdRloYHThnONMSa9WBB1WegQrgVRY4xJLxZEXRZMuFCYk0luVkaSW2OMMSaRLIi6LDica1ehxhiTfiyIumzF1r0AbK1r5Ow7X2ThiqrkNsgYY0zCWBB10cIVVfzyufUd76v2NjFnwWoLpMYYkyYsiLpo7pL1HGxpP6ysqaWNuUvWR6lhjDHGSyyIuqh6b1O3yo0xxniLBVEXDS3K61a5McYYb7Eg6qJZ00aTF/ZYS15WBrOmjU5Si4wxxiSSq+uJ9nUzJpYBgXuj1XubGFqUx6xpozvKjTHGeJsFUZfNmFhmQdMYY9KUDecaY4wxcbIgaowxxsTJgqgxxhgTJ1eDqIhMF5H1IrJBRGZH2F4uIvtEZKXz9SOnfHRI2UoR2S8itzrbbheRqpBtF7vZB2OMMSYa1yYWiUgGcBdwIbAdWCYii1R1bdiur6rqZ0MLVHU9MCHkOFXAEyG7/FpVf+FW240xxphYuHklOgXYoKobVdUPzAcui+M4FwAfqeqWhLbOGGOM6SE3g2gZsC3k/XanLNyZIvKeiDwtImMjbJ8JPBxWdrOIrBKR+0RkUILaa4wxxnSLm8+JSoQyDXv/LnCsqtY79zYXAqM6DiCSDVwKzAmp80fgJ86xfgL8EvjKEScXuQG4wXlbLyLrgRKgJp7OpCjrT2qz/qS2dOsPpF+fUqk/x0YqdDOIbgeGh7wfBlSH7qCq+0NeLxaRP4hIiaoG/9EuAt5V1Z0h+3W8FpE/AU9FOrmq3gPcE1omIu+o6uQ4+5NyrD+pzfqT2tKtP5B+ffJCf9wczl0GjBKRkc4V5UxgUegOInK0iIjzeorTntqQXa4hbChXRIaEvL0cqHSh7cYYY0yXXLsSVdVWEbkZWAJkAPep6hoRudHZPg+4EvimiLQCTcBMVVUAEcknMLP3G2GH/rmITCAwnLs5wnZjjDGmV7iaO1dVFwOLw8rmhbz+PfD7KHUbAV+E8ut60KR7ut7FU6w/qc36k9rSrT+Qfn1K+f6Ic+FnjDHGmG6ytH/GGGNMnPpEEO0q/WCqE5HhIvKSiKwTkTUi8m2n3LMpEEVks4isdtr9jlNWLCLPiciHzndPPAMcLU2l1z4f57nrXSJSGVIW9TMRkTnO/6n1IjItOa2OLkp/5orI+85z5k+ISJFTPkJEmkI+q3lRD5wkUfoT9WfMo5/PIyF92SwiK53y1P18VDWtvwhMavoIOA7IBt4DxiS7Xd3swxDgNOd1f+ADYAxwO/C9ZLcvzj5tBkrCyn4OzHZezwZ+lux2xtGvDOBjAs+UeerzAc4DTgMqu/pMnJ+/94AcYKTzfywj2X2IoT+fBjKd1z8L6c+I0P1S8StKfyL+jHn18wnb/kvgR6n++fSFK9FEpR9MGlXdoarvOq8PAOuInP3J6y4DHnBePwDMSF5T4ubZNJWq+gpQF1Yc7TO5DJivqs2qugnYQOD/WsqI1B9VfVZVW523bxF4ft0Tonw+0Xjy8wlyHn38PEdmq0s5fSGIxpp+0BNEZAQwEXjbKfJqCkQFnhWR5U52KYBSVd0BgT8cgMFJa138wtNUevXzCYr2maTD/6uvAE+HvB8pIitE5GUROTdZjYpDpJ8xr38+5wI7VfXDkLKU/Hz6QhCNJf2gJ4hIIfA4cKsGsj39ETiewIo3OwgMf3jF2ap6GoGsVDeJyHnJblBPyaE0lY85RV7+fLri6f9XIvJDoBX4m1O0AzhGVScC3wEeEpEByWpfN0T7GfP058ORiXZS9vPpC0G0y/SDXiAiWQQC6N9UdQEEUiCqapuqtgN/IsWGazqjqtXO910ElrmbAuwMZqRyvu9KXgvjcliaSi9/PiGifSae/X8lItcDnwW+oM4NN2fYs9Z5vZzAPcQTk9fK2HTyM+blzycTuAJ4JFiWyp9PXwiiXaYfTHXO/YH/A9ap6q9Cyj2ZAlFECkSkf/A1gckelQQ+l+ud3a4HnkxOC+N22F/PXv18wkT7TBYBM0UkR0RGElg4YmkS2tctIjIduA24VAMJXYLlR0lg7WJE5DgC/dmYnFbGrpOfMU9+Po5PAe+r6vZgQUp/Psme2dQbX8DFBGa0fgT8MNntiaP95xAYilkFrHS+Lgb+Aqx2yhcBQ5Ld1hj7cxyBmYPvAWuCnwmBDFUvAB8634uT3dZu9CmfQN7ngSFlnvp8CPwBsANoIXAl89XOPhPgh87/qfXARcluf4z92UDgXmHw/9E8Z9/POT+L7xFYXeqSZLc/xv5E/Rnz4ufjlN8P3Bi2b8p+PpaxyBhjjIlTXxjONcYYY1xhQdQYY4yJkwVRY4wxJk4WRI0xxpg4WRA1xhhj4mRB1JgecFbR+J6Lx3/YSen2b26dw00icr+IXOm8rhCRyS6f71YRyXfzHMaEykx2A4wxkYnI0cBZqnpshG2ZeiiRelKlUluAW4G/Ao1d7GdMQtiVqDHdJCI/dNZofB4YHVL+dRFZJiLvicjjIpIvIv1FZJOTthERGeCsk5glIv8qImudK835EU71LDDYWT/xXOdK7n9E5GXg2yJygZOQe7WTfDzHOcdmZ783ReQdETlNRJaIyEcicmOUPn1RRJY657o7JDtMvYj81OnTWyJS6pTfLyK/EpGXgJ+JyARne3Cdzk6T7TvH/ZmzAMHzIjLF6d9GEbnU2SdDAut/LnOO+w2nvNzZ9+8SWBv0bxLwr8BQ4CWnXca4zoKoMd0gIpMIpI6cSCC/5+khmxeo6umqeiqB5eq+qoGl6yqAzzj7zAQeV9UWAutzTlTV8UCk4HYpgWXVJqjqq05ZkaqeD9xFILPL1ao6jsCo0jdD6m5T1TOBV539rgQ+AfxXhD6dDFxNYFGACUAb8AVncwHwltOnV4Cvh1Q9EfiUqn4XeBC4zenLauA/I/QnVAFQoaqTgAPAfwMXEkhdF2zjV4F9qno6gX/nrzsp7CDw738rgXUzj3Pa/r8E8sNOVdWpXZzfmISwIGpM95wLPKGqjRpYSSc0D/MpIvKqiKwmEITGOuX3Av/ivP4X4M/O61XA30TkiwRWFIlFMCn3aGCTqn7gvH+AwCLHQcF2rQbeVtUDqrobOCgiRWHHvACYBCwTkZXO++OcbX7gKef1cgKLIwc9pqptIjKQQHB/OUpbIvEDz4S08WXnD4vVIef4NPAlp01vE0hBOMrZtlRVt2sg8frKsHYZ02vsnqgx3RctV+b9wAxVfU9EvgyUA6jq6yIyQkTOBzJUNZgk/DMEgs2lwH+IyNgY7i02ON8jLXUVqtn53h7yOvg+/P+9AA+o6pwIx2nRQ7lB28LqNkTYP1ahx+1oo6q2S2AVj2C7blHVJYc1VqScw/sU3i5jeo1diRrTPa8Al4tInrMSzSUh2/oDO5z7n18Iq/cggYTbfwYQkX7AcFV9Cfg+UAQUdqMd7wMjROQE5/11wMud7N+ZF4ArRWSw07ZiETliMlM0qroP2COHFkruSVtCLQG+GXI/+UQJrPrTmQMEPgdjeoX99WZMN6jquyLyCIEhxC0E7jkG/QeBYcctBIYlQ3+Z/43Afb/gUmkZwF+doVABfq2qe7vRjoMi8i/AY86V2zJgXpx9Wisi/w486wT3FuAmpx+xuh6Y5zxespFDw9c9cS+BYdp3RUSA3cCMLurcAzwtIjvsvqjpDbaKizG9wHlW8jJVvS7ZbTHGJI5diRrjMhH5HXARgTVgjTFpxK5EjTHGmDjZxCJjjDEmThZEjTHGmDhZEDXGGGPiZEHUGGOMiZMFUWOMMSZOFkSNMcaYOP0/VFuOPorZ3xAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "va_times = np.arange(8, 184, 7)\n",
    "cph_risk_scores = cph.predict(va_x_test)\n",
    "cph_auc, cph_mean_auc = cumulative_dynamic_auc(va_y_train, va_y_test, cph_risk_scores, va_times)\n",
    "\n",
    "plt.plot(va_times, cph_auc, marker=\"o\")\n",
    "plt.axhline(cph_mean_auc, linestyle=\"--\")\n",
    "plt.xlabel(\"days from enrollment\")\n",
    "plt.ylabel(\"time-dependent AUC\")\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot shows that the model is doing moderately well on average with an AUC of ~0.72 (dashed line). However, there is a clear difference in performance between the first and second half of the time range. The performance on the test data increases up to 56 days from enrollment, remains high until 98 days and quickly drops thereafter. Thus, we can conclude that the model is most effective in predicting death in the medium-term.\n",
    "\n",
    "### Using Time-dependent Risk Scores\n",
    "\n",
    "The downside of Cox proportional hazards model is that it can only predict a risk score that is independent of time (due to the built-in proportional hazards assumption). Therefore, a single predicted risk score needs to work well for *every* time point. In contrast, a Random Survival Forest does not have this restriction. So let's fit such a model to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('onehotencoder', OneHotEncoder()),\n",
       "                ('randomsurvivalforest',\n",
       "                 RandomSurvivalForest(min_samples_leaf=7, random_state=0))])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sksurv.ensemble import RandomSurvivalForest\n",
    "\n",
    "rsf = make_pipeline(OneHotEncoder(), RandomSurvivalForest(n_estimators=100, min_samples_leaf=7, random_state=0))\n",
    "rsf.fit(va_x_train, va_y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For prediction, we do not call [predict](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.ensemble.RandomSurvivalForest.html#sksurv.ensemble.RandomSurvivalForest.predict), which returns a time-independent risk score, but call [predict_cumulative_hazard_function](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.ensemble.RandomSurvivalForest.html#sksurv.ensemble.RandomSurvivalForest.predict_cumulative_hazard_function), which returns a risk function over time for each patient. We obtain the time-dependent risk scores by evaluating each cumulative hazard function at the time points we are interested in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "rsf_chf_funcs = rsf.predict_cumulative_hazard_function(va_x_test, return_array=False)\n",
    "rsf_risk_scores = np.row_stack([chf(va_times) for chf in rsf_chf_funcs])\n",
    "\n",
    "rsf_auc, rsf_mean_auc = cumulative_dynamic_auc(va_y_train, va_y_test, rsf_risk_scores, va_times)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can compare the result with the predictive performance of the Cox proportional hazards model from above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcsAAAEyCAYAAACVn/uWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABoWUlEQVR4nO2deXiTVfbHP7fpSltaacvSln2HUnYUUCmg4Iai4zAuo/7GUcdt3EYcHTd0ZtQRRx2XGcdZ3GVREdFRcIGioshOW0H2raVAW2hpS/fe3x9vkm5JmqRJs/R8nidPmpv3vu+5SZqTe+8536O01giCIAiCYJ8QXxsgCIIgCP6OOEtBEARBaAVxloIgCILQCuIsBUEQBKEVxFkKgiAIQiuIsxQEQRCEVgj1tQGeJDExUSclJREdHe1rUzxGeXm5jMfPCbYxyXj8GxmP99i4cWOh1jrJ1nNB5Sz79OnDM888Q0ZGhq9N8RiZmZkyHj8n2MYk4/FvZDzeQyl1wN5zsgwrCIIgCK0gzlIQBEEQWsGrzlIpdZ5SaodSardS6n4bz8cppT5WSm1VSv2olPqVs30FQRAEob3wmrNUSpmAl4HzgWHAlUqpYc0Ouw3YprUeCWQAf1VKhTvZVxAEQRDaBW/OLCcAu7XWe7XW1cBC4JJmx2ggVimlgBjgOFDrZF9BEARBaBe86SxTgEONHuea2xrzEjAUOAxkA3dqreud7CsIgiAI7YI3U0eUjbbm9cBmAluAaUB/4Aul1DdO9jUuotRNwE0A3bp1o6ysjMzMTDdN9j9kPP5PsI1JxuPfyHh8gzedZS7Qs9HjVIwZZGN+BTyljaKau5VS+4AhTvYFQGv9KvAqwLhx43RMTIzf5Ox4An/KQfIEbo0nazF89TiU5EJcKkx/BNLneLevC/3kPfJvZDz+TaCMx5vOcj0wUCnVF8gDrgCuanbMQWA68I1SqhswGNgLFDvRV+gIZC2Gj++Amgrjcckh43F9HYy43HHf7Pfhf3e73tdeP3DeSQuCEFR4zVlqrWuVUrcDKwAT8F+t9Y9KqZvNz78C/BF4XSmVjbH0+nutdSGArb7eslXwY756vMFpWaipgKU3GzdXcbdvTYVhizhLQeiQeFXuTmv9KfBps7ZXGv19GJjhbF+hA1KSa/+5aQ857rvyT+71tdev5BBseA1G/BwiYhxfWxCEoCKotGGFICQu1XBSLdp7wtlzHffd+IZ7fe31CwmDT+6Czx+G9J/D2F9Bj3THNgiCEBSI3J3g30x/BEzhTdvCoox2Z/qGRbne116/2X+HX38BQ2fBlnfhn2fBv6bRPf9LqC5v3R5BEAIWcZaCf5M+BwZYVuqVMSuc9YJze4fpc4xj43q61tdRv54T4NJ/wO9+gvOegqoyhux4Ef46FD6dC0e3GUFJz6XBvHjjPmtx214DQRB8jizDCv5PTRl0HwE3f+t63/Q57gXltNYv6jQ44xY4/WY2f/R3RtdtgY2vw7pXQYWArjeOk0haQQgKZGYp+Df1dZC7EVIn+NoS2yhFSfxw+Nm/4J6fIDKuwVFasETSCoIQsIiz9DWyZOeYgp+gutRY/vR3ohOg8qTt5xxF9QqC4PeIs/QlloT7kkOAbliyE4fZwKF1xn3qeN/a4Sxxqa61C4IQEIiz9CX2Eu5lya6B3PXQKQG69PO1Jc5hK5IWIO2y9rdFEASPIQE+vsTe0pws2TVwaB35ndO5/C+rOFxcQXJ8FHNnDmb2aOeK0CzdnMf8FTtc7utuP9LnsH7/CXpumk9XXcgxlUDnyFA6bXgdRl8LiQO8c11BELyKOEtfYjfhXpbsADh1HIp2saB+LHnVxgw8r7iCB5ZkA7TqRJZuzuOBJdlU1NS51Nfdfta+63tTUfM3a1v/uuN8GvUIEQuugBu+hKh4j19XEATvIs7Sl0x/pKlIODifcN8RyN0AwLraprOxipo6frd4K/NX7HDY/UhJJXW6aWU3Z/q62q+yspLItSvt9t1T04Vr637LO9V/Yv382dwX9iD1yuT0deev2CHOUhB8jDhLX5I+BypL4NN7zQ0KLnhG8vEs5K6jVoewtb7lfmWd1pzRL8Fh9w822V7Obq2vq/2OHDlC9+4JDvv+UD+YJT3uYU7+fP4Y/R4fd7/N6eseLq6w2S64SFtKvQkdHnGWviZ1nHE/7tew4T8Q08239vgTh9axJ6QPFUS2eColPoq/zhnpsPvavUXk2XA0rfV1tV9m5gkyMka22nfObx6CT0vIWPcqGWdnwKimVefs9U2OtxEwJLiGvVJvIA5TcAqJhvU1pUeM+xGXQ1gn2LnCt/b4C/V1kLeRsN6nExHa9GMaFWZi7szBrZ5i7szBRIU1Xe50pq+7/ZzqO/MJ6Hs2fHwnHFrfat8QBb87d2Cr1xVaQSLPhTYiztLXlOYb9/G9oV8G7FoBzfatOiTHtkN1Gf1GT+Wi9B6AUfA0JT6KJy8b4dQe3uzRKTx52QhS4qNc6utuP6f6msLg529A52RYdDWU5NntGx8VRr2G7MN2hA4E59DadiAdSOS54DSyDOtrSo8ACmK6wsAZsONTKNgBXYf42jLfkmsWI+g5HrWzlITocDY8dA5KKZdOM3t0ilvBMe72c6pvpy5w5UL49zmw8Cq4frk1N7N538c+/pHX1uxnQNcYrj69t1v2dGhKj5CW82f7z0vkueAkMrP0NaX5EJ1kzDgGmqtr7FzuW5v8gUProVMinNaXrNxi0lPjXHaUfk3XofCzf0P+VvjodrurCQ9eMJQpg5J49KMf+W53YTsbGcBobexTvnw6p53YCiPmtBSLCAmTyHPBacRZ+prSIxDb3fg7LgW6jYBdn/vWJn8gdx30nEB5dR27j5WRnhrva4s8z+DzYfrDkPM+fPuczUNCTSG8eNVo+iZGc8s7m9hbUNbORgYgpUdh4dWw5EZIGsyGcc8bQveNy66FRkJIaMMPVEFoBXGWvqY0H2J7NDweNAMOroWKE76zydecOg5FuyF1PDl5JdRrSE+N87VV3uHMeyDtZ0agyY7PbB7SOTKM/1w3HlOI4oY3NlByqqadjQwQtIbs9+Hvp8PuL2HGn+BXn1HRybysnT4H7s6BecXw68+htgLW/t2nJguBgzhLX1N6tGFmCTBwJug62LPSdzb5mlxzlGjPCWTllgAE58wSQCm4+CXoMRI+uMGYYdqoQtMroROv/HIsh06c4tZ3N1JTV+/4vB2NsgJYfA188Gvo0t+ofTrptxDSUvwBMF7vYZfA93+H8qL2tVUISMRZ+pK6GigvaDqzTB0HUV1gZwdeij20DpQJkkezNbeY5LhIkmIjfG2V9wjvBFe8Cyj48jG7VWgm9O3Ck5els2Z3EfOW/YiWqGmDnCXGbHLn53DOY8asMWlQ6/0y/gDVZfDd31o/VujwSDSsLyk7BmiIbSREEGKCgefC7i+MXMOOSO466J4G4dFk55UE76yyMXEpRgBKdWnTdksuoDlx/vKxqew+VsYrq/cwsGsM/ze5rw+M9TGNlXhCI43l1OQxMPsfrkWRdx1ivK4/vApn3Np0hUcQmiEzS19iESRoPLMEI+jgVBHkbWp/m3xNfZ0x7tQJFJ+q5kDRKdJ7Bul+ZXPKC2y3N8sFvG/mYM4d1o3HP9lG5o5j7WCYH9G8BmxthRHVOuE37qVbZdwPddXwzbMeN1UILsRZ+hKLIEHzX7QDphvLkLs6oJrPsW3G0lij/cqRHWFmCfZz/pSCFQ/CkRwAQkIUz/9iFIO7d+a3725m19FS2/2CEVtKPPU1sOpP7p2vSz8Y/UvY+BoU2xEuEATEWfoWq7NsNrOMOg16nt4x8y0PmcUIUseTlVsMQFpKB5lZ2iocbQqHbunwwyvwymR45Uz4/mWia47z7+vGERFm4vo31nO8vNo3Nrc33qgBe/Zc4/7r+e6fQwh6xFn6ktIjoEIMUYLmDJoBR7IJr+pgkXq5643X47Q+bM0toW9iNHFRYb62qn1In9M0FzCuJ1zyMty8Gn63A85/2sgNXPEH+OsQUv53He+dmc+Jk6X8/YUnyZ83gPpH4zgybwDrl/3T6csu3ZzH5KdW0vf+/zH5qZV8d9iPU1Pszb7bosQT3xPG/go2vw1Fe9w/jxDUSICPLyk9YlQZsRXePnAmfDmPhKKNwM/a3TSfcWgdpE4ApcjKLW61DFfQkT7HdhWM6EQ4/TfG7dhPsHUBZC2m764VbAqLIKSyhlBVDwq6U0DcxodYD4y/+DcOL2er4PTrJ2HY5jz/rKF51r3wyZ1N2zxRA/as38GmN2H1X+CyV9t2LiEoEWfpS8qO2I/A6zoU4nqSULShfW3yJeVFcHwPjLmGoycrOXqyqmNEwrpK1yFw7mOGg9i3mrq3riBcNc27jFLVpGx6mn/Fn+vwVC+u3GV1lBaq6/HfgtNR8cZ9dFcjIMpTdSlju8HpN8GaFwyhiI6uzSy0QJylLyk9AvG9bD+nFAycwWmb3oHaKggN4jxDCxYxgtTGwT0dZL/SHUJM0H8aEbrKKMnSjO66iD9/ut2tU/ttwendX0BkHNyzHUwe/vqafBes/y9kPgFz3vTsuYWAR5ylLynNh9Tx9p8fNBPThv/A/m+NCNlgJ3edsSeXPJqszEOYQhTDk8VZtsYxlUR3WqadHFWJ5Dw202Hfc59dTX5JZYt2vyw4rTXs+hL6TfW8owSjGszEW42l2PythsqPIJiRAB9fUVtl5FI2j4RtTN+zqQsJ7zjC6ofWQbc0CO/E1twSBnaNISrcjlyZYOXQmLlU6PAmbTXaRO6YucREhDq8/f68IS0KToeF4FSh63bnSLaxdTHQ8dJymzjjVoiMh5UOynoJHRJxlr6i7Khx70g1JCyK4vh0I4Uk2KXN6moNMYKeE9Bak5Vb3HHyK9vI+It/Q87YP3GEJOq1opJwQkJCGD91dqt9mxecDlEQHQrnpfmhms3uL4z7Aed47xpR8TD5DiPH2ZLGJAiIs/Qd9tR7mlGUMA5O7IfCXZ67dtZim2LdPuXYNqgph9QJHDpeQfGpGkbIfqXTjL/4N3Sft5uQx4qJ/O1aTEpD5pNO9Z09OoU1909j31MX8tqvJlBcDc+s2OFli91g15fQfYT3ZelOv9lIX1rpptCBEJSIs/QV9tR7mlGUMM74w1NqPs3lwpqJdfuMXPOv+J7j2WoWI5CZpZsk9IdxvzZSIY795FLXKYOSmNYzlP+s2cfavX6U41tRDId+gAFeXIK1EB5tRMTuWw37vvb+9YSAQJylr3ByZlkVmQRdh8FODzlLW3JhFrFuX3JovZEOEN+b7LwSwk0hDO4e61ubApkpvze+9L981OWuvxgcTq8unbj3va2UVvqJQMHeTKN0nTf3Kxsz7nqITTZml8G+BSI4hThLX1F6xIj87ORE0v3AGXDwe6gsaft1vSEX5gly10FPQ4xg66FihiZ3JjxUPp5uE50AZ95t7Hfv+8alrhGhir/+fCSHiyv40yfupZ54nN1fQEScIVjRHoRFwtn3GrPZ3V+2zzUFv0a+jXxF6RGI6Q4hTrwFg2ZCfS3sWdX263a2k2jeFrmwtlJeCMf3Qup46uo1OXklkl/pCc64BTqnwucPQb1rxaLH9enCb6b0Z9GGQ3y1/aiXDHQSrWH3V9A/wzspI/YYfY2RB73yjzK7FMRZ+ozSfOcDFVInGOHsnkgh6ZbWss0TcmFtwSJG0HMCewvKKK+uE+UeTxAWBdMfhvwtkPOBy93vOmcgQ7rH8vsPsn0r1H40x/h/aY/9ysaEhkPGA0bO5U+ftO+1Bb9DnKWvKHUgddccU6ghSrDrc5dnCE3I32osZ/U+0yzWjVEKbNYLbZcLawuHGsQItopyj2cZMQe6p8NXj0FNS/EBR0SEmnh2zihKKqp5aGk22lezq13tkDJijxFzIGGgkXfZUYuxC4A4S9/hyswSDGH18gLI3+ze9epqYdkdxh7pFW/D3Tlw7uNG0ET/ae6d01PkrjdSAsKiyMotJjrcRL+kGN/aFCyEhMCMPxpRz+ucr0RiYVhyZ+4+dxCfZh9h2dbDXjDQCXZ/Cd1GQGfHwXBewRQKUx+Agu0wf4B/pVsJ7Yo4S19QUwGVxa45ywHnGOW83I2K/eEfxnLcBfONepkAKWON+7xN7p3TE9TVQt5Ga+DG1twShqfEYQqxIXYquEe/DGMJ8+u/wqnjLnf/zdn9GdMrnoeX5nDEhjSeV6ksgYNrYaAPZpUW6moBBRXH8at0K6FdEWfpC5xMG2lCdIKhI+uOszy+z1hGGnwhDLukob3HSEDBYR86y2M/Qs0p6DmB6tp6tueflCVYb3Du41BdCqufdrmrKUTx1zmjqKnTzH1/a/sux1pSRtp7v7IxK/8INBuzP6RbCe2KOEtfYHWWLiqRDJxhzA4t/Z1Ba/jkLmNP8MJnjGomFiJiIWmwb2eWFkmx1PHsPFpKdW29BPd4g27DYPQvYf2/jchjF+mbGM0fLhzKN7sKefuHg14w0A67voCIzkZaka/w13QroV0RZ+kLytyYWYKRQgINAQ/OsHWh8ev83HnQObnl8yljjZmlr4I3ctcbBbDje4lyj7eZ+iCYwuDLx9zq/svTe3HWwESe+N929heWe9g4G1hTRqYadvsKe2lVvky3EtodcZa+wJ1lWDDSPjqnOC99V1YAKx6AnmfA2OttH5M82ggcKjnkmi2e4tA6Y3lZKbIOlXBapzB6dvHD8lDBQGx3mPRb2LbULZFwpRRPX55OmEnxu/e2Ulfv5R9YR3+E0sO+XYIFI60qrNln0tfpVkK7I87SF5Tmgym8IdDGWZQy5L72ZEKtE3lvy++H6nK4+AX74gcpY4x7XyzFlhXAiX3WJbatucWMSI1HKQnu8RqT7jBkBT9/yK3VhB5xUTx+SRobD5zg1a9dX851ifaoMuIM6XOM9CpLuhUKLnjGt+lWQrsjztIXWNR73HEKA2cagRoHv3N83M7PIed9OOteY1/SHt3SICTMN0E+FjGC1AlUVNex61iZBPd4m4gYmPoHQ8Zt+8duneKSUclcMKI7z36xg+35Jz1sYCN2+TBlpDnpc4x0q2uXAdr4sSt0KLzqLJVS5ymldiildiul7rfx/Fyl1BbzLUcpVaeU6mJ+br9SKtv83AZv2tnuuJpj2Zh+U8AU4TgqtqoUPrkbkoYa+qCOCI0wchx9MbPMtYgRjGJbfgl19ZoRKeIsvc7oayBxsCGy7swKRTOUUvxp9ggiQkOY9eK39L3/f0x+aiVLN+d5zsbKk3DIxykjtuhzFsT3Niq6CB0KrzlLpZQJeBk4HxgGXKmUGtb4GK31fK31KK31KOABYLXWunEi2FTz8+O8ZadPcEW9pznh0dDnTMfOcuWf4GSesfwa6sQv4JQxcHhL29SB3OHQekNdJiyKrYfMyj0949vXho6IKdQQKji+Fza+5tYpvt5ZQHWtprZeo4G84goeWJLtOYe5N9PQQ/b1fmVzQkJgzDWw/xso2uNra4R2xJszywnAbq31Xq11NbAQuMTB8VcCC7xoj/9QesT14J7GDDoPju+x/c96aD388E+YcKPz4fbJY4yl3SIPFphujbpaY+nXbGNWbjHdOkfQrXNk+9nQkRk4w5glZT7lVjWb+St2UF3X9MdVRU0d8z1VNHq3H6SM2GPU1YZAyOa3fW2J0I5401mmAI1DLHPNbS1QSnUCzgMaqz1r4HOl1Eal1E1es7K9qS6HqpNtq/Y+aIZx33x2WVttKIt0TnYtUs+q5LPRfZtc5WiOIUaQOh6ArNwSya9sT5QyZpcVx+Hb51zufri4wqV2l9Da2K/sl+HblBF7dE42Zrxb3jWr+wgdAW/Wu7EVvWIv/G4WsKbZEuxkrfVhpVRX4Aul1E9a6xZly82O9CaAbt26UVZWRmZmZhtN9x5Rp/I5HdieV8xRJ+y0N57xnVKpWreQrKqGle3e+xfT99g2stMeouh7FxyfruNMUyRH133MrmIbuZgexDKe5Lz/MQj4Pq+e40dWsbfwFKNOq/br984e/v6Zc8TQrlNI+vYFqte9QUT1CaoiEumc/HNaG06XSEVRZct/5y6Rqs2vRXTZfsaXHuan+l4c8cDr6o33JzF8DGllK8j+8FmKEtt39hvInzdbBMp4vOksc4GejR6nAvaUmK+g2RKs1vqw+f6YUupDjGXdFs5Sa/0q8CrAuHHjdExMDBkZGW023mvsXwPrYOj4qQztn9Hq4ZmZmbbHU30p0WtfIWPiWEOJp2AnfPMeDL+MEZfPdcOucaTUHiXFy6+ddTwfvAMx3Zk48+es2VME/MDss0Zz9qAkr17fG9h9jwKByO2wfDWR1cbv1MiqAkYe+A+m0aMdpkY8HJfHA0uyqahpqMQRZlI8fMlIMkbbqZnqLN8+D8CQC29jiC0hDRfxyvtTNxn2/4cRNVsg4z7PnrsVAvrzZoNAGY83l2HXAwOVUn2VUuEYDnFZ84OUUnHAFOCjRm3RSqlYy9/ADCDHi7a2H6X5xn1b9izBSCGprzEHQtQby69hneD8v7h3vpQxcCTbrehIt8hdBz0NMQKLck+6pI20P9+/3KLJVF/Vqu7p7NEpPHnZCFLio1BAaIiiW2wkl4zywMrE7i/NAhzeXeVoE6YwGHUl7FzumvykELB4zVlqrWuB24EVwHZgsdb6R6XUzUqpmxsdeinwuda6sX5WN+BbpdRWYB3wP631cm/Z2q5Y1Xu6te08vc4AUxQs+Q08fhoc/B6GzYaYru6dL2UM1FUbe4nepqwATuy3VhrJzi2hV5dOxHeS3LV2x67u6SF47QL4+E747iVjf7xoT5M9utmmNayJuIN9kVezpfM9jDn5hbUeqdtUnjQ+y74WInCG0dcYIu9bO0ZcYkfHm8uwaK0/BT5t1vZKs8evA683a9sLjPSmbT6jNB9CIyEyvm3n+fFDqK+GukYFabMXQZ/J7imLJJuVfA5valD18Ra5Zqk1ayRsCaN7xXv3moJt4lJtSx2GRRupG9uWmUtTmTGFQ5d+xmf4aI5xDBBTmc9fwv7NB58lMOqme923Z99q45wD/SxlxBaJA6HXJNj0Fky+yz2RESFgEAWf9saSY9nWf6yvzIWbG+Nk2aClm/OY/NTKpsnk8b2gUyLkuVlc2hUOrTNUg3qMorCsirziChFP9xU2dE/rQiJg1vPw68/h9/tg7l64fgVc/BKccQt06d/EUVqIUtVMzfsHJ8rbsJRvrTJyuvvnaE/GXGOkcR1oRVFLCHjEWbY3bc2xtOBm2aClm43AjLziiqbJ5FsOGzPK9kgfyV0PPdIhLJIs2a/0LU10TxXE9WTH4Nuark5EJxjL/mOuMepiXvku1NfZPF0Pinhvo5ui/Fob+5X9pvhnyogthl1iOHdR9Al6xFm2N22RumuMm2WD5q/Y0SSCERolkyePgcIdUFXWdvvsoOprDWk9837l1kMlhChIE5k732HRPZ1XDHfncKzblNb72PmcHTcl8Pbag9S7U5Hk2HZDecrfVHscER4NaT+DbR9BRbGvrRG8iDjL9qbsqGdmlm6WDcpzlEyeMgZ0PeRvbbt9dogu3w+1FUYkLIZyz4CuMURHeHX7XPA0tj5/QFhMArnHy/h6V4Hr5/SXKiOuMuZa4zOd876vLRG8iDjL9qSqFKrLPDOztLF8xqwX7Ab31NVr5q/4ye7pkuOjGoJ8vLgUG1dilkNLnYDWmuw8Ue4JSGx9/kZdTdzJHTwUtYS31x5w/Zy7voCuwyGujXma7U3yaCPVZdNbvrZE8CLyc749cbfosz3S5zgV+VpYVsUdCzbz3Z4izujbhS25xVTWNOh6RoWZmDtzMMQkQVwv75TryloMXz3OgJJDhq7mwe853GsWhWXVsl8ZqNj6/IWEcv2mN1i/sxeHjg+nZ5dOzp2rqhQOroWJt3reTm+jlDG7/Ow+yM8y9uOFoENmlu2JRZAgpo05li6w8cBxLnrhWzYeOMHTl6ez8DcTeeqydFLiG5bQHr5oKLMtqispoz1fritrsSGaUHLI0EDUhohCwRrjl7jMLIOIC+ZT1WMcz4T+gy8yVznfb+9qQ2QjkPYrGzPi50bpvM0yuwxWxFm2J56eWTpAa81ra/bxi3+uJTw0hCW3TmLOOEN9cPboFNbcP43Fv5kIQGJMREPHlLFQfADKCz1nzFePG2ktjampoO/WvxJmUgztEeu5awm+JTSCiCvfpiY0mnOz7qGqtMi5fru/gPBYI+o2EOnUBYbOgqxFLT/rQlAgzrI9sUrdeWDP0gHlVbX8dsFmHvt4GxmDu/Lxb89keHLLpc6RPeOIDAvh+72NvtCs4gQezLe0k84SW32UId07ExFq8ty1BN/TuQf7pr1CN11A8VvX2k0zsWKtMhJAKSO2GHONUe5s+ye+tkTwAuIs25PSI4YySoT3ZlK7jpZy8Uvf8ml2Pr8/bwivXjOWuCjbX0ARoSbG9e7C93saO8tRgPLsUqydNIMjOkH2K4OUkRNn8LeI39Dt2Lew8o+ODy74CU7mBoZqjyP6nA3xvWHTG762RPAC4izbE0uOpZdksZZtPcwlL6+hpKKGt284nVsy+hMS4vhaZ/Trwk9HSjluUV2JiIWkwZ4N8pn+iCGP1oj60Cieqpkjyj1BSkiI4rSzbuSd2ulGvcycJfYP3mVJGQlwZxkSYujF7v8Gju/1tTWCh7EbDauUugco0Vr/p1n7bwGT1vp5L9sWfJS2zLFcujmP+St2cLi4guT4KObOHNwQbNMKjft2ijBRXlXHuN6n8dJVY+geF9n6CYCJ/RMA+GFvEeePMNuWPMbYQ9LaM449fY4hxJ3zPhpQcT3Z2P+3LPsulVt7yswyWLl8bCpnrvgVZ3Y6Ru+PboPEQdA9reWBu7+ArsMCL2XEFqOugswnYPPbrhVgF/weRzPL6wFboV2vmp8TXKWZeo9d6bnNea2eqnnf8qo6TCGKKyf0dNpRghGJ2inc1HTfMmUMlBe0Kp3nEgU/Qco4Vmd8BHfn8Kk6k8iwEAYkxXjuGoJfEd8pnPNH9uKastuoj+gMC6+CU8ebHlRVCgcCpMqIM8SlGDPkLe82qdAiBD6O8iy11rqFIrLWukopkdd3Ga0bRNTN2JOeu3vRFh78MBuAuro6TCtbVic7VV1Hc0GxunrNs1/s4mdje7Y43h5hphDG9Wm+b9moAkm88+eyy5EcQ3j7/PlgDhTMyi0hLTmOUJPsBAQz10zszXsbc/ls2Hwu3PhreP96uPp9MJm/evZ9baSMBPp+ZWPGXAOLfmno3A4+z9fWCB7C4TeVUqpFQqCtNsEJKksMSaxGy7CH7UjPaeDKCb24ckIvpvQMtf7d+GZPedPeOR0xsV8Cu46VUVBaZTR0TzOqgnhKySdrEYSEGhqaQG1dPT8eFuWejkB6ajwjU+N47qc49AXPwN5V8NVjDQfs+gLCY6BngKaM2GLQeRCdJDmXQYajmeV84H9Kqd8BlmiPscDTwDPeNswXtGX/sFWsOZYNM8vk+CibWq0p8VE8dNEwADIzj5GRMazFMZ/lHLHZNzm+pV5na1j2LdfuLWLWyGQIjTAcpiciYuvrIPs9Y2kq2rjOzqNlVNbUM1L2KzsEvzyjN3Pfz2Jt/EVMHLcVvnsBeow0fjzt/hL6ZUBoEBX+NoXByCvh+5fNcQoyvwgG7M4stdZvAg8DjwP7zbfHgEe11kEXG92W/UOnsJFjOXfmYKLCmuYYWqXnWqEtfZuTltyZmIjQlvmWh7dAfb3dfk6x72tj7CN/YW1qKMsV37ZzCwHBrJHJxEWFGXqx5z0FvSbCkpvh6f5G4ekDawyVp2BizLVGvdmtC3xtieAhHC7Daq0/01pP0VonmG9TtNaftZdx7YnD0lWewIZ6z+zRKTx+yXDr45T4KJ68bIRTs9nZo1N48rIRpMRHoVzs25xQUwgT+nZhbeN9y5SxUF0KRbtcPl8TshYb9f4GNezdZOWV0DkylD4JTuqGCgFNZJiJOeNSWfHjEY6eqoe0y0HXQoX581ZxwpBDDCaHmTjQ+FGw+S0jXkEIeByljrwITbbGNFAIrNJaf+ttw9obe3t97uwB2sSOLuzkAYkAPHnZCK6c0MulU84eneKxZeKJ/RJY+dMxjp6spFvnSCMiFoyl2CTXZ6sAVJ+C7ctg+KVNyjll5RaTnhqPxIl1HK4+vTf/+mYfC9cd4s7s56H5rntNhSGL6ERhgIBhzLWw9BY4+D30nuRra4Q24mhmuQHY2Oi2CSgD5iul7vK+ae2Lvb0+d/YAbVJ21JhhRTRNlSgsM4Jqmuiz+gDLvqU1KjZxkKE21BZxgp/+Z5QkG3mFtam6TvNTfikjRLmnQ9EnMZqzByXx7roDaHspSZ5MVfIHhl1i6N1uetPXlggewNGe5Rs2bs8B04Dr2s/E9sGTe4A2aZZjaaGozMjOSYjxbYDD0B6d6RwZ2uAsQ0xGnb62RMRmLTTqHPZq+FV9qLSe2nrNSHGWHY5rzujN0ZNVVHayU0jAjixiwBIeDSMuhx+XGtHwQkDjcpKb1jooJfUte4AWh9mWPUCbNMuxtFBgnlkm+XhmaQpRnN4vgbX7Gu9bjoYj2VDbIt22dcqOwZ6VRumikIaP2b4SI2BIgns6HtOGdCUlPor/RlzTZFkeMB4Ho+JN5x5GythTveC5tODal+1guOQslVKhSqlfAUG2XmIwe3QKV53ei07hJtbcP81zjhLMM8uWv6j9ZWYJcEa/BA4UnWrYp00eA3XVcOxH10+W/b5Rt7LREiwYzjIxJoIeLqgMCcGBKURx1em9mJ8/kiNTnjZWHVDG/awXgmu/EgzH+O1zDY9LDgVfIFMHwq6zVEqVKqVONr4BecD5wG/azcJ2JjEmglPVdZRXeVCqyqLeY6Poc2FZFZ3CTXQKd5Ty2j5M7Nds37JxkI+rZC2EHqOswUFLN+cx+amVrDlcS1llDR9tOewBi4VAY864noSZFP88MRbuzoF5xcZ9sDlKsFvHla8e9409QptwtGcZq7Xu3OzWTWs9R2sdtN90SbHGcqgl8MYjVJwwZmg2Z5ZVfjGrBBjSPZbTOoU15FvG94ZOCa47y2M/Qf5WSDdyKxvnsAJU1tZ7NodVCBiSYiM4P60H72/M5VR1kGundpRApg6Cq8uw/ZVSDymlcrxlkK9JNDsujzpLB0WfC8uqfR4JayEkRHF634SGmaVSZnECF51l1iJQJiO4gXbIYRUCimsm9qa0spZlwb66YC9gKdgCmToIrTpLpVQPpdTdSql1wI+ACbjS65b5CIvjKih1I6jFHlZn2XJmWVhWRUK0fzhLMFJI8oorOHT8lNGQMsaoGFJV5twJ6usNebv+0yCmK9AOOaxCQDGu92n06BzBQ0tz6Hv//5j81MrgXGWY/kjLQCalgjOQqQPgaM/yRqXUSmA1kADcAORrrR/TWme3l4HtStZihi6cxN6IqzjzkwzPbcSXHjXu7cwsk2L9YxkWbORbpow1AnXytzp3ggNrjECGRoE9Xs9hFQKKj7YcprC8mtp67R1pSX8hfY4RuGQJZIrqYsQvhPrPj2PBeRzNLF/GmEVepbV+SGudRQvZjSAiazF8fAem0lxCFMRU5nsucs3OMmx9veZ4uX/NLAd2jSExJrxh37JxuS5nyFpoVJEYfIG1ae7MwYSGNFXr8WgOqxBQzF+xg5q6pl8lQbssnz6nIZBp7m5IHAwr/yS1LgMQR84yGVgIPKuU2qGU+iMQ1j5m+QBvRq6VHoHI+BZLMidOVVOvG/ZJ/QGljHzL7/cUobWGmCTjl7EzQT41FbBtGQy9GMIbdF/PS+tOmEkRGWZ83DyewyoEFPaW3/OKK9hXWN7O1rQjISaY9hAU7jR+VAoBhaNo2EKt9T+01mcD04ES4JhSartS6ol2s7C98Gbkmr0cy3JLjqX/zCzBSCE5crKS/UWN9i2dUfLZ8RlUnWxSYQRg2ZbDVNTU8/qvJvD6edGez2EVAgpHy+9Tn8nkylfXsmzrYapq6+weF7AMnWWs1mQ+BbUeDCIUvI5T0bBa61yt9TNa67HAbCD43mVvRq6VHrFZ066w1D90YZvTYt8yeQwUH4DyIge9MKJgY5Ohz1nWJq01r3+3n8HdYjm9bxdvmSwEEPakJR+/eBhzZw4mt/gUdyzYzBlPfMWfPtnG7mNOBpcFApYAn5JDsOG/vrZGcAGXM+G11jsw6loGF9MfMfYoGy/FekqCq/QIJJ7VornQPLP0p2VYgH6J0XSNjeD7vUVcdXqvBnGCw5th4Dm2O5UXGoV8z7jVWG4ys/HACbbln+TPl6ZJlREBwLqqYK/Q+i1T+rNmTyEL1h3k9e/28+9v9zGhTxeuPL0n56f1YHnOEbeKtLeluLu7fW33mwp9z4avn4HRv4SIWL+xV7CP72Vj/AWLgsjnD0HZUU7oWE6b9WzblUXq66HMti6sv84slVJM7J/Amt3GvqXqMQpQRpCPPWeZswTqa1vI273x/QFiI0O5VP5RhUY4Ki8XEqI4a2ASZw1MoqC0ig825bJw3UHuXrSVPyzJprpOU1dvBAjlFVdw/5IsyqtquWCEbYH2smrNO2sP8Mf/baOypt7pfhY+zc53q6+jfrMmP0jnt2dS8fVLVE76nUt946o1J8rtp7bZ6vvAEiOBoTWH6QsH/d3hGh58aqXf/ygQZ9mY9Dkw/DJq/pzK0upJXDHkZ7Q5ueFUkeFEbO5ZVmEKUcRF+V/c1MR+CXy05TB7CsoZ0LWzUbLL0b5l1kLoNgK6NRSzPnqyks+y87luUh+/kPMTAo+k2AhuntKfm87qx9q9Rfz6jQ3U1dc3Oaaypp4Hl+bw4FJHWiktn3Oun23c7WvtB7wSNp5J3/6NaV/14QSdne4LwMovXLpuRU0d972fxf+y80mIDqeLjduG/cd5esUOt53sA0uyreIjzvZdujmP13OqqTa/pe1xTXdp9RtMKfWV1np6a21BgymUE6elMaZgF4VlVfTs0qn1Po4oO2Lc25xZVpMQHU5IiP8tT55h0YndW8SArjHGUuzur4w8sebLqYW7DEc6409Nmt/94SB1WnPNGb3by2whSAkJUUwakEhljf2gn3mzhtls37V7N+9stz8Ts9fP+vzH29zq21q/irL7ifn+57w1+Ds2DLrH6b5XDw1n4IABLl+3uq6eQ8dPseVQMSfMea6tUVFTxz2Lt/DkZ9sdHldQWkXz0znT191+jvrOX7GjfZ2lUioS6AQkKqVOAyzfkJ0x0kqCloquYxhW+G+yT5S03VmWWpyl7Zmlv0XCWuid0IkecZGs3VNkOLvkMbB1AZzMaxn0lLUIVAikXW5tqq6t5911B8kYlESfxOh2tl4IVpLjo6waw41JiY/i/yb3tdkns+YAmfkml/tZ+Nc3+9zq23q/vnDqCtJyFpF22f0Ql+JU33N7h5Dh5nWX33U2YATenays5Xh5tfV245sbbJ6vXsPUwV3tXg9g4fpDbvV1t5+jvt5SBnM0s/wNcBeGY9xIg7M8iSFYELz0HE/Y9n9Sk7sR+l/UtnM50IUtKKv2u+AeC0opJvZLYPXOAmPfMmWs8UTexqbOUmvDWfadYtTuM/NZTj4FpVVcN6lP+xouBDVzZw5usvQGzglcuNvP69fMuN+Qh1z9F7j4Bef6luxq83WVMrZ/4qLC6Gv+MZvi4IfIUz9Ld3jNb3YVutXX3X6O+npLGcxRnuXftNZ9gXu11v201n3Nt5Fa65e8Yo2fENX3DADCDtv+peUSlpmljfJcRWVVfhfc05gz+idQVF7NzqNl0D0NQsJaihMcXAvFB1sG9ny3n76J0Zw9MKkdLRaCHUuR9pT4KBTOC1y428/r1zytN4y7Hja/DYW7fWqvvZQeZ39QuNN37szBhDfzQt6+pru0umeptX5RKTUJ6NP4eK31m16xyA84rWsKB+q70rlwS9tPVppvlLlqpgeptaawrMpvZ5bQuL5lIYO79zWCd5rL3mUthLBOMKRhBp6dW8Kmg8U8ctEwv9yPFQIbR5G03ujn9Wuefa/hLFf9GX7+Wpuv6W7f1lJ6vNF39ugUtm3fxv8Omtrtmu7iTIDPW0B/YAtgmddrIGidZZgphJyQwZxVkmU7oMUVSo9ATMsl2FPVdVTW1PvtniVAzy6dSD0tiu/3Fhl7LCljjSWj+noICYGaSvjxQ8NRRsRY+735/X46hZu4fJyUIhKEVonpCmfcAt88A2feBT1G+swUX/ygmJQcxh+uymjXa7qDMwo+44DJWutbtda/Nd/u8LZhvmZP5DA61xYZShttoTTfTrUR/8yxbM7Efgn8sO849fXaiIitOglF5uWiXZ9DZUkTebvj5dV8tPUwl41JoXOk/6XECIJfMum3hn70V3/0tSWCHZxxljlAy2/7ICc/Js34I3d9205UesROHUuLLqz/LsOCIX1XfKqG7UdOtqxAkrXI2Ivtm2E9ftH6Q1TX1nPtxD7tbaogBC5R8XDWPbD7Czjwna+tEWzgjLNMBLYppVYopZZZbt42zNec6jKUSsLhUBucZX0dlB1zOLNM8veZZWOd2KTBEBZtRMSeOg47VxjpIiZjNb+2rp631x5gYr8EBnWzLeElCIIdJtxk/LD+8jFj+0fwK5yRVZnnbSP8kS6xncjW/Rmfu879k5QXgq6z6SyLAmRm2SMuij4JnVi7t4gbzuoHyaOMiNgfl0B9TZMl2K9+OkZecQUPX+Q4yVsQBBuERcGU++CTu40tjkEzfW2R0IhWZ5Za69XAfiDM/Pd6wMlKwIFLYkwEG+oGoPOzjEAWd7DmWNpahjVmlv5U+NkeE/sb+5Z19RqSR8ORbNj8DiQNhe4NuVBvfLef5LhIzhnqOJlYEAQ7jL4GuvQz6ug2k/UTfEurzlIpdSPwPvBPc1MKsNSLNvkFSTERbK4fgKqvgfyt7p3EkXpPWRWdI0MJD3WqSppPOaNfAqWVtfx4uATqqozb4U1QetiIjgV2HS3luz1F/HJib0JN/j8mQfBLTGEw9UE4mgM5H/jaGqERznyr3QZMxlDuQWu9C3Bq6qCUOk8ptUMptVspdb+N5+cqpbaYbzlKqTqlVBdn+nqbxNhwNtUPMh64uxTrQL2nsKyaxFj/n1VCQ75lwXdvwaa3Gp6oLDHKmmUt5o3v9xMeGsIV43v5yEpBCBKGX2YUJVj1Z6ir8bU1ghlnnGWV1tqqQqyUCsXIs3SIUsqEIYt3PjAMuFIp1WQzS2s9X2s9Sms9CngAWK21Pu5MX2+TGBNBIXGcik6FQ+46yyOAMvKomlFYVkViACzBAnTtHEm/pGhG7ngBapstSddUUP/lYyzZlMfFI5PpEu3fe7CC4PeEhMD0h+HEPnhmIMyLh+fSIGuxry3r0DjjLFcrpf4ARCmlzgXeAz52ot8EYLfWeq/Z2S4ELnFw/JXAAjf7epwk86zvWOcRkOum7F1pPkQnGksrzSgsqyIxNnAcy8R+CXSpLbD5nDqZx6nqOq6TdBFB8AyVJUZxgooTgDbyvc2rOIJvcMZZ3g8UANkY4uqfAg850S8FaJzRn2tua4FSqhNwHmBZpHe6r7ewBN7sjxpu7M2V5Lp+klLbRZ8BisqrAyK4x8LE/gkc1gk2nzuqEhnTK54RqXHtbJUgBClfPQ66WYBPTYXRLvgEZ7Rh64F/mW+uYEsjzt7y7Sxgjdb6uKt9lVI3ATcBdOvWjbKyMjIzM1001TbRYfD1iQQygB9XvE5B1zNd6j82fzfV4fFkN7Ontl5TfKqGssLDZGYWOjyHJ8fTFuqrNE/XzuGZiH8T3rAqT42K4ImqnzM+vsIpO/1lPJ4k2MYk4/E9U0pybX8JluQG5HgcESjjcVTPMhsHe5Naa8f1U4zZYM9Gj1OBw3aOvYKGJViX+mqtXwVeBRg3bpyOiYkhIyOjFdOco/vGTAoSxkBpJMM7nwJXz7uhDPpMbmHP0ZOV8PlXjE0bTEYrhZEzMzM9Np628tK2EP5tiuXWuneNmXZcKv8N+yXfFY/juznTnIrs9afxeIpgG5OMxw/YnGpTalPFpeLJ7zh/IFDeH0ffbhdhzPiWm29Xm2+fYqSStMZ6YKBSqq9SKhzDIbZQ/lFKxQFTgI9c7ettEmMiOFpmzi10NSK2rtas3tMybaSgNDB0YZszsV8CLxaMofq3WTCvmAPX/sBTeSO46vReAZECIwgBw/RHDJGCxoRGGu2CT3BUz/KA1voAhoj6fVrrbPPtfqBVaQmtdS1wO7AC2A4s1lr/qJS6WSl1c6NDLwU+11qXt9bXnQG2hcTYCArKqiB1vJFrWVvlfOfyY4C2rd5Tbixj+nN5LltM7J9ARU0dWbnFALz1/QFMSnH16ZIuIggeJX0OzHoB4npi3ZXqNclo9yZZi43IW4nAbYEzcnfRSqkztdbfAphrW0Y7c3Kt9acYM9HGba80e/w68LozfdubpJgICkuroOcE+O4FyM+CnuOd6+xIvSdAZ5an901AKUMndlhyZxZvOMR5ad3p1jnS16YJQvCRPqfBOX58p6GadWK/966XtdiIuK2pMB5bInAttnRwnFk7+zXwslJqv1JqP/B34HqvWuUnJMVGUFpVS2U3c7UNV5ZiLeo9Md1aPFVUbpa6C7CZ5WnR4Qzp3pnv9xaxdPNhTlbWct2kPr42SxCCnym/hxATZD7lvWt89XiDo7QgEbhWnNGG3ai1HgmkAyPNIgJBrw0LDcukheo0iOvlmjiBQ13YaiJCQ4iJcGZi7190jQnnuz1F/OHDbEJDFLnHT/naJEEIfjonw4QbYetCOpUf9M417KXHuZM2F4S0+m2tlIoAfgb0AUKVMtbPtdZB/3PDskxaWFZNas/xcPAH5zuXHjGSiqOTWjxVWFZFYkwEltcyUFi6OY/v9x63Pq6t1/zhwxyUUu1WrVwQOixn3gMb36DvvreBaz1//jjbEbjEpXr+WgGIM8uwH2Go59QC5Y1uQY/VWZaag3xO5sJJe9kvzSg9AtFdrbUeG1NYVh1wwT0A81fsoLquaaJ0RU0d81fs8JFFgtCB6NQFJt1BUuEP7quKOWLMdS3bTOESgWvGmXXAVK31eV63xA+xCJ0bEbETjMZD62D47NY7O1LvKasKyKCYw8UVLrULguBhzriF6m9fJPzLeXDdx+Cp1anaKvjxQ4joDBGxxqTAFGakqwy+wDPXCHCcmVl+p5Qa4XVL/JAEsyh4YWkVdB8BpgjIXe9c59IjNvcrwbIMG3gzy+T4KJfaBUHwMBExHOj9c9j/Dexd5bnzrv4LHPsRfvZvuGcbzCuG6z6BqpOw5m+eu04A44yzPBPYaC6XlaWUylZKZXnbMH8gMsxE58hQo1BzaDgkj3LBWebbnFlqrSkqqyYhwNJGAObOHExUmKlJW1SYibkzB/vIIkHoeBxOPs8IOPzqcdCtFoBqnbyN8O1zMOqXMKhRCn2v041yYd+9KEE+OOcszwcGAjMwFH0syj4dgsTYCArLzFqoqePh8BaorXbYh9pqOFVoc2ZZUlFDbb0OuBxLgNmjU3jyshGkxEehgJT4KJ68bIQE9whCO6JDwiDjfji8Gba3UdisphI+vMX4rjrviZbPn/uYIegu6SNOpY4cwNBpnWb++5Qz/YKFxBizig8Y4gR1VXCklYl12VHjPrZljqXF8QbiMiwYDnPN/dPY99SFrLl/mjhKQfAFI6+AxMGw8k+GtKa7rPozFO6Ai1+ESBtVg+J7wcTbIGsR5G50/zpBQKtOTyn1KPB7jOLMAGHA2940yp9IiokwlmGhIcintaVYiyCBzRzLwFTvEQTBjwgxwbSHoHAnZC107xwHfzCWWMf+HwyYbv+4M+82UuBW/MEzy74BijMzxEuBizGni2itDwOx3jTKn0iMCbcKn9O5B3RObV2cwCpIYEMX1jqzFGcpCEIbGDoLkscYqj6u6FYDVJ+CpbcY2rMz/uT42MjOhmM+tNaImO2gOOMsq7XWGnO5LqWUU7qwwUJiTASllbVU1tQZDT3Htz6ztC7D2p9ZBprUnSAIfoZScM6jhpDAhv+61nflH+H4HrjkJSNVpDVGXwPd0uDLR419zg6IM85ysVLqn0C8UupG4EtcLwQdsCSZcy0tlUJInWB8OE/m2+9Umg/KBJ0SWzxVVFZFiILTOomzFAShjfTLgL5T4Ov5UFXqXJ/9a2DtP2D8jdBvinN9Qkww889QfBB++Ifb5nqcdqyS4kyAzzMY9Ss/AAYBj2itX/SaRX5GExUfMCJiwfHs0iJIENLy5S0oq6ZLdDimkMCSuhMEwU+Z/iicKoLv/976sdXl8NGtcFpvOGeea9fplwGDzoev/2rU6vU1liopJYcA3VAlxUsO09mo1mzgG+Br898dBouKjzXIp0e6IQHlqAKJnRxLMGaWsl8pCILHSB0LQy4ygnXKixwf++U8OHEAZv8DImJcv9aMP0FthRFF62vauUqKM9GwNwDrgMuAy4G1SqkOUaILGlUesTjL0AjoMcqxNmMr6j2yXykIgkeZ9jDUlMO3z9o/Zu9qWPcqnHEL9J7k3nUSBxjLt5vehKM/uncOT9HOVVKcmVnOBUZrrf9Pa30dMBYjlaRDYJkFWiNiwSxOsNm+OIGjmWV5tcwsBUHwLF2HQPoVsO5fUJLX8vmqUvjodujS33CsbWHKfYaGrC9TSXZ8Zv85L1VJccZZ5gKNd45LARt1XIKTyDATsRGhDSo+YETE1lbCURsr0jWVUHECYmw7y8LSKhKixVkKguBhMu431HZW/6Xlc58/bFRNuvQVCO/Utut06mJca28m7FzRtnO5itbw/cuw4EpDMCG0WUGKsCivVUlxxlnmAT8opeaZBQrWAruVUvcope7xilV+RlJsIxUfaFSBxEaQT5lFkKCls6yorqO8uo7EWFmGFQTBw5zWG8ZdD5vfhsLdDe27v4KNr8HE2w0VMk8w/gZIGACfPwR1NZ45Z2vU1cD/7jFmtENnwa1rDeWhuJ6AMu5nvQDpc7xyeWec5R5gKeY8S4z6lvkYwgQdQpwgMSaiIRoWIC4FOqfYjogtbT3HMlFmloIgeIOz7zVmW6vMQgOVJbDst4Y03tQHPXcdU5gR7FO0y/UcT3eoLIF35xjXOvNu+Pkbxgw5fQ7cnWNUSbk7x2uOEpyoZ6m1fgwMMQKtdYco+tycxNhwdhxplsOUOs52RKwj9R5zrqbMLAVB8AoxXWHirUbe5YHvGgRSpv4BwjxcQ3fQeUaOZ+aThpOKOs2z57dwYj+8+wso2g2XvAyjf+md67SCM9GwE5VS24Dt5scjlVJOJPQED4kxEU33LMFYii0+2DCTtOBIF9Y8O5U9S0EQvIYlwKWs0XfTt895Pv9QKZj5hDHrW/20Z89t4dA6+Nd0YxJyzYc+c5Tg3DLs88BMoAhAa70VONuLNvkdiTERlFTUUFVb19DY046oemk+hIQZm+DNsC7DxoqzFATBS3z9TMs2b+Ufdk8zpPDWvdp0n9QTZL8Pr19kyPHd8BX09a3bcUqUQGvdPPq1zuaBQYol1aOo8eyye7rhFJsvxVpyLFVLhR7LMmxCtCzDCoLgJdo5/5CpDxr7pF+0MSXFgtbGTPWDX0PKWMNRJg70zLnbQKt7lsAhpdQkQCulwoE7MC/JdhQaCxMkx0cZjWGR0GNky4hYBzmWBaVVxEaEEhlm8qa5giB0ZOJSzRJwNtq9QWw3OOseY+Y6fwCUFxrXmv6IUwE3XY+uhuduN5x5XIq5stNaI2/04hcMIRg/wJmZ5c3AbUAKRs7lKPPjDkNSc8k7Cz0nGOIEjUOnS4/YLPoMxsxS1HsEQfAq0x8x8g0b48X8Q8CcV66gvACXdFqzFjN4x8uN9F1zDUc5dLaRE+onjhKci4YtBK5uB1v8lgYx9eZBPuNg7d/haA4kjzbaSo/YVfIvLBVdWEEQvIxlNvfV4+bZmvOzPLfJfJKG7EIzNRWw7A746RP7/XZ+jqneRi3OwxttbmX5ErvOUin1Ii1G34DW+g6vWOSHWGaWBc1nlo3FCZJHGwVVq0ocSN1V0TexQ5UDFQTBF6TP8a5zbI69/dDaCijYYb9fbYXtdm/tr7YBRzNLi1L4ZGAYsMj8+OfARm8a5W9EhpmIiQhtuQwbl2oE8+Suh9NvaqTeY09EvZrxfVpGyQqCIAQ0dvdJe8JtP9jv91xa++6vtgG7e5Za6ze01m8AA4GpWusXzXUsp2PsW3YoEmPCm4qpg7FMkDq+ISK21L7UXW1dPSdOVZMgy7CCIAQb7u6TTn+EupBm34ne3l91E2cCfJJpKmsXY27rUBjCBDbW1lPHGwoTZQWN1HtaziyPn6pGa0iSAB9BEIKN9DmGLqurOq3pc9gx+LZ203dtC86kjjwFbFZKrTI/ngLM85pFfkpSbAS7jpW1fMIqTrDO4czSkqMpM0tBEIISN/dJj3WbwrBfPOoFgzyLM9GwrymlPgNONzfdr7U+4l2z/I/EmAi+32ujCnmPkWZxgvVQX2sk50bGtzjMqt4jzlIQBCHgcFbB54jW+iOMItAdzlGC4eSKT9VQU1ff9ImwKOg+woiILT0CMd1sq/dYZ5ayDCsIghBoOOUsG3GxV6wIACyVQoqaC6qDWZxgkxHubDcSVmaWgiAIgYqrztK/skTbEYuTaxERC0aQT80pyN1gN8eysKyacFMInSOd2SYWBEEQ/AlXneVYr1gRAFhVfOxFxALU1zicWSbEhKP8TJVCEARBaB1n6lkOUkp9pZTK0VrXK6XSlVIPtYdx/kRXeyo+AId+AGV+Kbe+a1MPscjsLAVBEITAw5mZ5b+AB4AaAK11FnCFN43yR+zOLLMWG4LB2hz4U1liU0C4sKxa9isFQRACFGecZSetdbOijdR6wxh/JircRHS4qaWY+lePG4LBjbFRaLWorIqEaHGWgiAIgYgzzrJQKdUfs6i6UupyIN+rVvkpibE2VHycKLSqtTZmlrGyDCsIghCIOBOaeRvwKjBEKZUH7AN+6VWr/JTEmIiW0bBOFFotraqluq6eRJlZCoIgBCStziy11nu11ucAScAQrfWZWuv9XrfMD0mMCW85s3RCQLjQ7GBlZikIghCYtDqzVErFA9cCfYBQS+pDR6pnaSEpNoJ1+443bXSi0GpRuVm9R2aWgiAIAYkzy7CfAmuBbKC+lWODmsSYCE6YJe/CTI0m5a0ICFtnlhINKwiCEJA44ywjtdb3eN2SAMDi7I6XV9Otc6TT/QrNM8tEybMUBEEISJyJhn1LKXWjUqqHUqqL5eZ1y/wQh5J3DrDMLLtEi7MUBEEIRJxxltXAfOB7YKP5tsGZkyulzlNK7VBK7VZK3W/nmAyl1Bal1I9KqdWN2vcrpbLNzzl1PW+TZA7Qsani44Ci8ipO6xRGqMlVdUFBEATBH3BmGfYeYIDWutCVEyulTMDLwLlALrBeKbVMa72t0THxwN+B87TWB5VSXZudZqqr1/UmVhUfl2eWot4jCIIQyDgz1fkROOXGuScAu82pJ9XAQuCSZsdcBSzRWh8E0Fofc+M67UaD5J2NMl0OKCoXXVhBEIRAxpmZZR2wRSm1CrBOqZxIHUkBGmfr5wKnNztmEBCmlMoEYoG/aa3ftFwC+FwppYF/aq1fdcJWrxIdEUqncJPtyiMOKCyrZnhyZy9ZJQiCIHgbZ5zlUvPNVWzVotI2rj8WmA5EAd8rpdZqrXcCk7XWh81Ls18opX7SWn/d4iJK3QTcBNCtWzfKysrIzMx0w1zniDbV8+Oeg2RmOj8JPlJcTv9OVW7Z5e3xtDfBNh4IvjHJePwbGY9vaNVZaq3fcPPcuUDPRo9TgcM2jinUWpcD5Uqpr4GRwE6t9WHz9Y8ppT7EWNZt4SzNM85XAcaNG6djYmLIyMhw0+TW6bltDaZwExkZZzh1fGVNHRXLlzNqSD8yMga6fL3MzEyvjqe9CbbxQPCNScbj38h4fIPdPUul1GLzfbZSKqv5zYlzrwcGKqX6KqXCMcp6LWt2zEfAWUqpUKVUJ4xl2u1KqWilVKz5+tHADCDH9eF5Hpv6sA44blHvkQAfQRCEgMXRzPJO8/1F7pxYa12rlLodWAGYgP9qrX9USt1sfv4VrfV2pdRyIAtDHejfWuscpVQ/4EOztF4o8K7Werk7dniaxNgINhw44fTxlv1NiYYVBEEIXOw6S621pQzXrVrr3zd+Tin1F+D3LXu1OMenGHJ5jdteafZ4PkYeZ+O2vRjLsX6HIXlXTW1dvVN5k0VllpmlRMMKgiAEKs6kjpxro+18TxsSKCTFRqB1w/Jqa1gEDJJkZikIghCw2J1ZKqVuAW4F+jXbo4wF1njbMH8lKaZBxaerE/qwMrMUBEEIfBztWb4LfAY8CTSWqivVWh+33SX4cVWYoLCsik7hJjqFO5OlIwiCIPgjjvYsS4AS4Mr2M8f/cVVMvahM1HsEQRACHVH2dpHEWMvM0jlnWVgmurCCIAiBjjhLF4kONxEZFuK0mHphWRUJ0eIsBUEQAhlxli6ilCIpNsKlmaWltJcgCIIQmIizdIPEmAinAnzq6zXHy2VmKQiCEOiIs3QDw1m2PrM8caqaeg2JEuAjCIIQ0IizdANn9WGLRBdWEAQhKBBn6QZJMeEcN0veOcISBCTRsIIgCIGNOEs3SLRI3p1yvG9ZaJ5ZyjKsIAhCYCPO0g0sOq+Fpa04S5lZCoIgBAXiLN3AWWGCovIqTCGKuKiw9jBLEARB8BLiLN2gQR/WsbMsLK0mITqckBDVHmYJgiAIXkKcpRtY9iBbi4gtKq+SSFhBEIQgQJylG8REhBIRGtLqzLKgrFqCewRBEIIAcZZuoJRySsWnqKxKij4LgiAEAeIs3SSxFX1YrbUhoi4zS0EQhIBHnKWbJLWi4nOquo7KmnpJGxEEQQgCxFm6SVJsuMNlWMusUwJ8BEEQAh9xlm6SGBPB8fIq6uq1zectjlQCfARBEAIfcZZukhgTQb2G4+W2Z5eWmaUswwqCIAQ+4izdpDVhgiLrzFKcpSAIQqAjztJNLMur9pylpb1LtCzDCoIgBDriLN0kqRV92KKyKuKiwggPlZdYEAQh0JFvcjexiqnbqTxSWFYtOZaCIAhBgjhLN4mNCCU8NIQCB8uwsl8pCIIQHIizdBOlFEkxEdaalc0xnKXMLAVBEIIBcZZtIDEm3MHMslpmloIgCEGCOMs2YE9Mvbq2npKKGhKixVkKgiAEA+Is20CSHTF1i1BBYqwswwqCIAQD4izbgCF5V91C8s6qCyszS0EQhKBAnGUbSIwJp65ec+JU06VYi7NMkpmlIAhCUCDOsg0k2hEmsEjdycxSEAQhOBBn2Qas+rCltmeWFmcqCIIgBDbiLNuAPTH1ovJqIkJDiA43+cIsQRAEwcOIs2wD9vRhC0sN9R6llC/MEgRBEDyMOMs20DkylHBTS8m7wvJqUe8RBEEIIsRZtgGllKHiU2p7ZikIgiAEB+Is20hibEsVn6LyKqk4IgiCEESIs2wjic3E1OvrNUWiCysIghBUhPragEAnMSacnLwS6+OTlTXU1msSxFm6RU1NDbm5uVRWVvraFKeJi4tj+/btvjbDY8h4XCcyMpLU1FTCwsK8eh3Bd4izbCOJMREUlVdTX68JCVENOZayDOsWubm5xMbG0qdPn4CJJi4tLSU2NtbXZngMGY9raK0pKioiNzeXvn37eu06gm+RZdg2khQbQV29priiBsC6fynLsO5RWVlJQkJCwDhKQVBKkZCQEFCrIYLriLNsIxanaImIbZhZirN0F3GUQqAhn9ngR5xlG2mu4mPVhZVlWEEQhKBBnGUbsVQWsTjLwrIqQhSc1kmcZXuwdHMek59aSd/7/8fkp1aydHNem8955MgRrrjiCvr378+wYcO44IIL2Llzp8vnmTdvHikpKYwaNYq0tDSWLVtmbX/mmWeaHNunTx8KCwtbnENrzbRp0zh58qR7g/EwtbW1JCYm8sADDzRpb25/ZmYmF110kfXxZ599xrhx4xg6dChDhgzh3nvvbbMtGzduZMSIEQwYMIA77rgDrXWLY9555x0mT57MqFGjGDVqFCEhIWzZssVh/wMHDjB9+nTS09PJyMggNzcXgIKCAs4777w22y0EJl51lkqp85RSO5RSu5VS99s5JkMptUUp9aNSarUrff2Blsuw1XSJDscUIssy3mbp5jweWJJNXnEFGsgrruCBJdltcphaay699FIyMjLYs2cP27Zt44knnuDo0aNune/uu+9my5YtvPfee1x//fXU19e71P/TTz9l5MiRdO7c2a3re5rPP/+cwYMHs3jxYpvOyRY5OTncfvvtvP3222zfvp2cnBz69evXZltuueUWXn31VXbt2sWuXbtYvnx5i2Ouvvpq1qxZw5YtW3jrrbfo06cPo0aNctj/3nvv5dprryUrK4tHHnnE+sMgKSmJHj16sGbNmjbbLgQeXouGVUqZgJeBc4FcYL1SapnWelujY+KBvwPnaa0PKqW6OtvXX4iLCiPMpKyBPYVlot7jKR77+Ee2HbY/o9p8sJjquqbOp6Kmjvvez2LBuoM2+wxL7syjs4bbPeeqVasICwvj5ptvtrZZvly11tx333189tlnKKV46KGH+MUvfsHcuXNJTk7mkUceYcWKFfz5z38mMzOzyXmHDh1KaGiozdmjI9555x1uuukmAPbv3895553HmWeeydq1axk5ciS/+tWvePTRRzl27BjvvPMOEyZMoLy8nN/+9rdkZ2dTW1vLvHnzuOSSS9i/fz/XXHMN5eXlALz00ktMmjSJzMxM5s2bR2JiIjk5OaSnp7No0SKb+3ALFizgzjvv5B//+Adr165l4sSJrY7h6aef5sEHH2TIkCEAhIaGcuutt7r0OjQnPz+fkydPWq9/7bXXsnTpUs4//3y7fRYsWMCVV17Zav9t27bx3HPPATB16lRmz55tPcfs2bOts1WhY+HNmeUEYLfWeq/WuhpYCFzS7JirgCVa64MAWutjLvT1CwzJu4hGe5ai3tNeNHeUrbU7Q05ODmPHjrX53JIlS9iyZQtbt27lyy+/ZO7cueTn5/PYY4+xaNEiVq1axR133MFrr71GSEjTf60ffviBkJAQkpKSAHjuueesS4OjRo3i8OHDNq+5Zs2aJvbs3r2bO++8k6ysLH766Sfeffddvv32W5555hmeeOIJAP785z8zbdo01q9fz6pVq5g7dy7l5eV07dqVL774gk2bNrFo0SLuuOMO63k3b97M888/z7Zt29i/f7/N2VNFRQVfffUVF110EVdeeSULFixo82vamFWrVjV5TSy3SZMmtTg2Ly+P1NRU6+PU1FTy8hyvKCxatMjqLB31HzlyJB988AEAH374IaWlpRQVFQEwbtw4vvnmm1bHIgQf3syzTAEONXqcC5ze7JhBQJhSKhOIBf6mtX7Tyb4AKKVuAm4C6NatG2VlZS1+1XubCF3NjgP5ZGae4FDBKfrHh3jMBl+Mx5u0Np64uDhKS0sBuCejl8NzzXjxB/JPVrVo79E5gn9flWa3n+X8tqisrKS6utrmMStXruTSSy/l1KlTdOrUiUmTJvH1118zc+ZM/va3v3Huuefy5JNP0rVrV0pLS6mqquLVV1/lzTffJCYmhtdee42ysjKqqqq49dZbmzirtLQ0ysrKiIhouipx/Phxq81lZWX07t2bPn36UF5ezqBBg5g0aRJlZWX07duXvXv3UlpayvLly1m6dClPP/00YDi57du30717d+69916ys7MxmUzs3r2b0tJSTp06xdixY4mLi6O8vJy0tDS2b9/OyJEjm9jy4YcfcuaZZ1JXV8eMGTN4/PHHefzxxzGZjFJ0je0/deoUdXV1lJaWUl9fT3l5ucPXHRw7ouZ9y8rKrOdvfr3m1NXVsXLlSiIjI+ndu7f1tbTXf968edx7773897//ZdKkSSQnJ1NRUUFpaSlRUVHk5eXZvE5lZWW7/K92tO8Ef8GbztLWpl3zTY5QYCwwHYgCvldKrXWyr9Go9avAqwDjxo3TMTExZGRkuGuzW/Tdt45jpVVkZJzFqZXLGdavFxkZwzxy7szMzHYfjzdpbTzbt293OoH89+cP5YEl2VTU1FnbosJM/P78oW4noY8dO5ZPPvnEZv+wsDAiIyOtz4WFhREVFYXJZGLPnj0kJCRw/Phx6/MRERHcc889LYJZIiIiiIiIaHINpRQxMTEtrhsaGkp0dDQhISHExMQQFRXV5Pzx8fHExsbSuXNn6uvriY2NRSnFhx9+yODBg5uca968eaSmpvLuu+9SX19vHUunTp3o1KmT9byhoaGEhYW1sGXp0qWsWbOGESNGAIYj37BhA+eccw6JiYnU1NRY+1RWVtK9e3diY2MZMWIEP/30k80ZYmNWrVrF3Xff3aK9U6dOfPfdd03aBg8eTH5+vvV6x48fp1evXjbft9LSUj7++GOuvvpq6/OO+sfGxlqDscrKyvj444+ts9DS0tImr1VjIiMjGT16tMMxeoKO9p3gL3hzGTYX6NnocSrQfK0pF1iutS7XWhcCXwMjnezrN1iWYSuq6yivriMxVpZh24PZo1N48rIRpMRHoYCU+CievGwEs0enuH3OadOmUVVVxb/+9S9r2/r161m9ejVnn302ixYtoq6ujoKCAr7++msmTJjAwYMH+etf/8rmzZv57LPP+OGHHzwwOoPBgwezd+9el/rMnDmTF1980RqAs3nzZgBKSkro0aMHISEhvPXWW9TV1Tk6TRNOnjzJt99+y8GDB9m/fz/79+/n5Zdfti7FZmRk8NZbbwHGTO7tt99m6tSpAMydO5cnnnjCGlFcX1/Ps88+2+IaU6dOZcuWLS1uzR0lQI8ePYiNjWXt2rVorXnzzTe55BLbOzX19fW89957XHHFFU71LywstAZiPfnkk1x//fXWfjt37iQtzf6qhRC8eNNZrgcGKqX6KqXCgSuAZc2O+Qg4SykVqpTqhLHUut3Jvn5DYmwERWXV1ojYxGgJ8GkvZo9OYc3909j31IWsuX9amxwlYJ2VffHFF/Tv35/hw4czb948kpOTufTSS0lPT2fkyJFMmzaNp59+mm7dunH77bfzzDPPkJyczH/+8x9uuOEGj6m5XHjhhS4vUT388MPU1NSQnp5OWloaDz/8MAC33norb7zxBmeccQY7d+4kOjra6XMuWbKEadOmNVkmvuSSS1i2bBlVVVU8/PDD7N69m5EjRzJ69GgGDBjAL3/5SwDS09N5/vnnufLKKxk6dChpaWnk5+e7NCZb/OMf/+CGG25gwIAB9O/f3xrcs2zZMh555BHrcWvWrCE1NbVFBK69/pmZmQwePJhBgwZx9OhRHnzwQWufVatWceGFF7bZdiEA0Vp77QZcAOwE9gAPmttuBm5udMxcYBuQA9zlqG9rt7Fjx+pVq1bp9ubf3+zVvX//iV7501Hd+/ef6K+2H/HYuX0xHm/S2ni2bdvWPoZ4kJMnT3rt3IcPH9bnnHOO185vC2+Oxxd4cjxnnXWWPn78uM3n2uuz29G+E9oTYIO241+8KqSutf4U+LRZ2yvNHs8H5jvT119JijV+bf+Ub2z6J8jMUvAQPXr04MYbb+TkyZN+k2vZUSkoKOCee+7htNNO87Upgg+QqiMewFJh5KcjRk5gYqw4S8FzzJkzx9cmCBiiBI1zLoWOhcjdeYCkmOYzSwnwEQRBCCbEWXoAi2LPnoIyYiNCiQwz+dgiQRAEwZOIs/QAcVFhhIYoauu1qPcIgiAEIeIsPUBIiLI6SdGFFQRBCD7EWXoIS0SszCzbmazF8FwazIs37rMWt/mUJpPJWlZr1qxZFBcXA0Zy+x133EFaWhojRoxg/Pjx7Nu3DzBKVI0YMcKqZ2orkb6iooIpU6a4JAbgTQoKCggLC+Of//xnk/YePXo0efz6669z++23Wx+/+eabpKWlMXz4cIYNG9ai3Jg7LF++nMGDBzNgwACeeuopm8fMnz/f+vqmpaVhMpk4fvw4lZWVTJgwgZEjRzJ8+HAeffRRa5/33nuPCRMmEBISwoYNG6zt2dnZ/N///V+b7RY6DuIsPYRlRikzy3YkazF8fAeUHAK0cf/xHW12mFFRUWzZsoWcnBy6dOnCyy+/DBhC3IcPHyYrK4vs7Gw+/PBD4uPjrf1WrVplVZ2xJe323//+l8suu8yqpepr3nvvPc444wynBdHBqEv5/PPP8/nnn/Pjjz+yadMm4uLi2mRHXV0dt912G5999hnbtm1jwYIFbNvWssDQ3Llzra/vk08+yZQpU+jSpQsRERGsXLmSrVu3smXLFpYvX87atWsBQ3P3nXfe4eyzz25yrhEjRpCbm8vBg7ar0whCcyR1xENYnGSCOEvP8dn9cCTb/vO566GumZB6TQV8dDtsfMN2n+4j4HzbMxdbTJw4kaysLMAo62SRiwOa6IU6wzvvvMO7774LGCoxjz76KN26dWPLli1cdtlljBgxgr/97W9UVFSwdOlS+vfvT0FBATfffLP1S/35559n8uTJrFu3jrvuuouKigqioqJ47bXXGDx4MK+//jrLli3j1KlT7Nmzh0svvdQqqN6cBQsW8Ne//pWrrrqKvLw8UlJaVz968sknrWpFYOih3njjjU6N3x7r1q1jwIABVoWdK664go8++ohhw+zrKzcut2XR1QWoqamhpqbGWl5s6NChdt+fWbNmsXDhQu6777422S90DGRm6QGWbs5jec4RAN74bn+big8LLtDcUbbW7urp6+r46quvuPjiiwEj3/Hjjz9m1KhR/O53v7NqrlqYOnUqo0aN4vTTWxbIqa6uZu/evfTp08fatnXrVv72t7+RnZ3NW2+9xc6dO1m3bh033HADL774IgB33nknd999N+vXr+eDDz7ghhtuAGDIkCF8/fXXbN68mccff5w//OEP1vNu2bKFRYsWkZ2dzaJFizh06BDNOXToEEeOHGHChAnMmTOHRYsWOfWaOFtu65133rFZbuvyyy9vcWxeXh49ezZIQbdWbuvUqVMsX76cn/3sZ9a2uro6Ro0aRdeuXTn33HNtvgfNkXJbgivIzLKNLN2c16TyRUlFDQ8sMWZDbdUp7fC0NgN8Ls28BNuMuJ7wq/+5fdmKigpGjRrF/v37GTt2LOeeey5gfInv2LGDlStXsnLlSqZPn27dEwNjGTYxMdHmOQsLC5ss2QKMHz/euj/Yv39/ZsyYARhLhKtWrQLgyy+/bLIkefLkSUpLSykpKeG6665j165dKKWoqamxHjN9+nTr0uiwYcM4cOBAE2cEsHDhQqvYwRVXXMGvf/1r7rnnHruvia1C0I64+uqrufrqq506VuuWBYUcXe/jjz9m8uTJdOnSxdpmMpnYsmULxcXFXHrppeTk5LQqeN61a1e7dUQFoTkys2wj81fsaFIiCqCipo75K3b4yKIOxPRHICyqaVtYlNHeBix7lgcOHKC6utq6ZwlGWazzzz+f+fPn84c//IGlS5c6fc7m4uqNRclDQkKsj0NCQqitrQWMoKLvv//euleXl5dHbGwsDz/8MFOnTiUnJ4ePP/64ybkbn9dkMlnP1ZgFCxbw+uuv06dPHy6++GK2bt3Krl27AGNptbq62nrs8ePHrT8Chg8fzsaNG1sdryszy9TU1Caz39zcXOsyry0WLlxoXYJtTnx8PBkZGSxfvrxVGysrK4mKimr1OEEAcZZt5nBxhUvtggdJnwOzXjBmkijjftYLRrsHiIuL44UXXuCZZ56hpqaGTZs2WWci9fX1ZGVl0bt3b6fOddppp1FXV+dyNZIZM2bw0ksvWR9v2bIFMMptWfYYX3/9dZfOuWPHDsrLy8nLy7OW23rggQdYuHAhAGeeeSZvv/02YMyyFy9ebC239cADD3Dfffdx5Iix7VBVVcULL7zQ4hpXX321zXJb77//fotjx48fz65du9i3bx/V1dUsXLjQuvTdnJKSElavXt2kHFdBQYE1YrmiooIvv/ySIUOGtPo6SLktwRXEWbaR5Hjbv0zttQseJn0O3J0D84qNew85SgujR49m5MiRLFy4kGPHjjFr1izS0tJIT08nNDS0SUpFa8yYMYNvv/3Wpeu/8MILbNiwgfT0dIYNG8Yrrxh1CO677z4eeOABJk+e7HIqyoIFC7j00kubtP3sZz+zRsX+5S9/YcmSJYwaNYozzjiDn//859Zo0gsuuIDbbruNc845h+HDhzN27FibM1dXCA0N5aWXXmLmzJkMHTqUOXPmMHz4cABeeeUV65gBPvzwQ2bMmNGkvFh+fj5Tp04lPT2d8ePHc+6553LRRRdZjx8yZAjff/89F154ITNnzrT2k3JbgisoW/sFgcq4ceP0M888065Vt5vvWQJEhZnaXITYQqBUEXeW1sazfft2hg4d2n4GeYDS0lJiY2NbPW7z5s08++yz1iLJ/oqz4wkUbI2nqqqKKVOm8O233xIa6pnQjfb67Ha074T2RCm1UWs9ztZzEuDTRiwOcf6KHRwuriA5Poq5MwdLcI/QgtGjRzN16lTq6ur8Jteyo3Lw4EGeeuopjzlKIfiRT4oHmD06RZyj4BTXX3+9r00QgIEDBzJw4EBfmyEEELJnKfgdwbQ1IHQM5DMb/IizFPyKyMhIioqK5MtHCBi01hQVFREZGelrUwQvIsuwgl+RmppKbm4uBQUFvjbFaSorK4Pqi1LG4zqRkZFW+UMhOBFnKfgVYWFh9O3b19dmuERmZiajR4/2tRkeQ8YjCC2RZVhBEARBaAVxloIgCILQCuIsBUEQBKEVgkrBRylVAJQDhb62xYMkIuPxd4JtTDIe/0bG4z16a62TbD0RVM4SQCm1wZ5cUSAi4/F/gm1MMh7/RsbjG2QZVhAEQRBaQZylIAiCILRCMDrLV31tgIeR8fg/wTYmGY9/I+PxAUG3ZykIgiAIniYYZ5aCIAiC4FGCylkqpc5TSu1QSu1WSt3va3tcRSnVUym1Sim1XSn1o1LqTnP7PKVUnlJqi/l2ga9tdRal1H6lVLbZ7g3mti5KqS+UUrvM96f52k5nUEoNbvQebFFKnVRK3RVI749S6r9KqWNKqZxGbXbfD6XUA+b/px1KqZm+sdoxdsY0Xyn1k1IqSyn1oVIq3tzeRylV0ei9esVnhtvBznjsfsb8/T2yM55FjcayXym1xdzuv++P1jooboAJ2AP0A8KBrcAwX9vl4hh6AGPMf8cCO4FhwDzgXl/b5+aY9gOJzdqeBu43/30/8Bdf2+nGuEzAEaB3IL0/wNnAGCCntffD/NnbCkQAfc3/XyZfj8HJMc0AQs1//6XRmPo0Ps4fb3bGY/MzFgjvka3xNHv+r8Aj/v7+BNPMcgKwW2u9V2tdDSwELvGxTS6htc7XWm8y/10KbAeCsar0JcAb5r/fAGb7zhS3mQ7s0Vof8LUhrqC1/ho43qzZ3vtxCbBQa12ltd4H7Mb4P/MrbI1Ja/251rrW/HAtEDAlQey8R/bw+/fI0XiUUgqYAyxoV6PcIJicZQpwqNHjXALY0Sil+gCjgR/MTbebl5T+GyjLlmY08LlSaqNS6iZzWzetdT4YPxCArj6zzn2uoOk/eKC+P2D//QiW/6nrgc8aPe6rlNqslFqtlDrLV0a5ga3PWKC/R2cBR7XWuxq1+eX7E0zOUtloC8hQX6VUDPABcJfW+iTwD6A/MArIx1i2CBQma63HAOcDtymlzva1QW1FKRUOXAy8Z24K5PfHEQH/P6WUehCoBd4xN+UDvbTWo4F7gHeVUp19ZZ8L2PuMBfp7dCVNf3T67fsTTM4yF+jZ6HEqcNhHtriNUioMw1G+o7VeAqC1Pqq1rtNa1wP/ws+WWRyhtT5svj8GfIhh+1GlVA8A8/0x31noFucDm7TWRyGw3x8z9t6PgP6fUkpdB1wEXK3NG2Lm5coi898bMfb4BvnOSudw8BkL2PdIKRUKXAYssrT58/sTTM5yPTBQKdXX/Mv/CmCZj21yCfP6/X+A7VrrZxu192h02KVATvO+/ohSKlopFWv5GyPoIgfjfbnOfNh1wEe+sdBtmvwaDtT3pxH23o9lwBVKqQilVF9gILDOB/a5jFLqPOD3wMVa61ON2pOUUibz3/0wxrTXN1Y6j4PPWMC+R8A5wE9a61xLg1+/P76OMPLkDbgAI4J0D/Cgr+1xw/4zMZZQsoAt5tsFwFtAtrl9GdDD17Y6OZ5+GJF6W4EfLe8JkAB8Bewy33fxta0ujKkTUATENWoLmPcHw8nnAzUYs5JfO3o/gAfN/087gPN9bb8LY9qNsZdn+T96xXzsz8yfxa3AJmCWr+13cjx2P2P+/h7ZGo+5/XXg5mbH+u37Iwo+giAIgtAKwbQMKwiCIAheQZylIAiCILSCOEtBEARBaAVxloIgCILQCuIsBUEQBKEVxFkKghOYqz7c68XzLzBLmd3trWt4E6XU60qpy81/Zyqlxnn5encppTp58xqC0JhQXxsgCB0dpVR3YJLWureN50J1gyC4T/EnW4C7gLeBU60cJwgeQWaWgmAHpdSD5hqBXwKDG7XfqJRar5TaqpT6QCnVSSkVq5TaZ5YrRCnV2VynL0wpdYdSapt55rjQxqU+B7qa6/edZZ6ZPaGUWg3cqZSabhaWzjaLaEeYr7HffNz3SqkNSqkxSqkVSqk9Sqmb7Yzpl0qpdeZr/bORWkqZUurP5jGtVUp1M7e/rpR6Vim1CviLUmqU+XlLnUiHovHm8/7FLKT/pVJqgnl8e5VSF5uPMSmj/uR683l/Y27PMB/7vjJqU76jDO4AkoFVZrsEweuIsxQEGyilxmJIJo7G0K8c3+jpJVrr8VrrkRhl1H6tjZJqmcCF5mOuAD7QWtdg1IgcrbVOB2w5sYsxyn2N0lp/Y26L11pPAV7GUDr5hdZ6BMZq0C2N+h7SWk8EvjEfdzlwBvC4jTENBX6BIW4/CqgDrjY/HQ2sNY/pa+DGRl0HAedorX8HvAn83jyWbOBRG+NpTDSQqbUeC5QCfwLOxZBss9j4a6BEaz0e43W+0SzdBsbrfxdG3cZ+ZttfwNA/naq1ntrK9QXBI4izFATbnAV8qLU+pY3KL411htOUUt8opbIxnM1wc/u/gV+Z//4V8Jr57yzgHaXULzEqYDiDRVx6MLBPa73T/PgNjGK6Fix2ZQM/aK1LtdYFQKVSKr7ZOacDY4H1yqhMPx3DAQFUA5+Y/96IUYTXwnta6zqlVByGE19txxZbVAPLG9m42vwDIrvRNWYA15pt+gFDfm+g+bl1WutcbQiIb2lmlyC0G7JnKQj2sacF+TowW2u9VSn1f0AGgNZ6jVKqj1JqCka1eovY9YUYTuVi4GGl1HAn9v7Kzfe2SjA1psp8X9/ob8vj5v/fCnhDa/2AjfPU6Abty7pmfcttHO8sjc9rtVFrXa+MqhMWu36rtV7RxFilMmg6puZ2CUK7ITNLQbDN18ClSqkoc+WUWY2eiwXyzfuTVzfr9yaGcPRrAEqpEKCn1noVcB8QD8S4YMdPQB+l1ADz42uA1Q6Od8RXwOVKqa5m27oopVoEFdlDa10CnFANBXnbYktjVgC3NNrvHaSMKjWOKMV4HwShXZBfaYJgA631JqXUIoylvwMYe4IWHsZYLjyAsZzY+Ev7HYx9OUsJLxPwtnkJUwHPaa2LXbCjUin1K+A980xsPfCKm2PappR6CPjc7MRrgNvM43CW64BXzGkbe2lYdm4L/8ZYXt2klFJAATC7lT6vAp8ppfJl31JoD6TqiCB4EHOu4SVa62t8bYsgCJ5DZpaC4CGUUi8C52PUIBUEIYiQmaUgCIIgtIIE+AiCIAhCK4izFARBEIRWEGcpCIIgCK0gzlIQBEEQWkGcpSAIgiC0gjhLQRAEQWiF/weZN9SsL6ZNlQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 518.4x345.6 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(va_times, cph_auc, \"o-\", label=f\"CoxPH (mean AUC = {cph_mean_auc:.3f})\")\n",
    "plt.plot(va_times, rsf_auc, \"o-\", label=f\"RSF (mean AUC = {rsf_mean_auc:.3f})\")\n",
    "plt.xlabel(\"days from enrollment\")\n",
    "plt.ylabel(\"time-dependent AUC\")\n",
    "plt.legend(loc=\"lower center\")\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indeed, the Random Survival Forest performs slightly better on average, mostly due to the better performance in the intervals 25–50 days, and 112–147 days. Above 147 days, it actually is doing worse. This shows that the mean AUC is convenient to assess overall performance, but it can hide interesting characteristics that only become visible when looking at the AUC at individual time points.\n",
    "\n",
    "## Time-dependent Brier Score\n",
    "\n",
    "The time-dependent Brier score is an extension of the mean squared\n",
    "error to [right censored data](https://scikit-survival.readthedocs.io/en/latest/user_guide/understanding_predictions.html#what-is-survival-analysis). Given a time point $t$, it is defined as:\n",
    "\n",
    "$$\n",
    "\\mathrm{BS}^c(t) = \\frac{1}{n} \\sum_{i=1}^n I(y_i \\leq t \\land \\delta_i = 1)\n",
    "        \\frac{(0 - \\hat{\\pi}(t | \\mathbf{x}_i))^2}{\\hat{G}(y_i)} + I(y_i > t)\n",
    "        \\frac{(1 - \\hat{\\pi}(t | \\mathbf{x}_i))^2}{\\hat{G}(t)} ,\n",
    "$$\n",
    "\n",
    "where $\\hat{\\pi}(t | \\mathbf{x})$ is a model's predicted probability of\n",
    "remaining event-free up to time point $t$ for feature vector $\\mathbf{x}$, and $1/\\hat{G}(t)$ is an inverse probability of censoring weight.\n",
    "\n",
    "*Note that the time-dependent Brier score is only\n",
    "applicable for models that are able to estimate a survival function.\n",
    "For instance, it cannot be used with* [Survival Support Vector Machines](https://scikit-survival.readthedocs.io/en/latest/api/svm.html#survival-support-vector-machine).\n",
    "\n",
    "The Brier score is often used to assess calibration.\n",
    "If a model predicts a 10% risk of experiencing an\n",
    "event at time $t$, the observed frequency in the data\n",
    "should match this percentage for a well calibrated model.\n",
    "In addition, the Brier score is also a measure of\n",
    "discrimination: whether a model is able to predict risk scores\n",
    "that allow us to correctly determine the order of events.\n",
    "The concordance index is probably the most common measure\n",
    "of discrimination. However, the concordance index disregards\n",
    "the actual values of predicted risk scores\n",
    "– it is a ranking metric –\n",
    "and is unable to tell us anything about calibration.\n",
    "\n",
    "Let's consider an example based on data from the\n",
    "[German Breast Cancer Study Group 2](http://ascopubs.org/doi/abs/10.1200/jco.1994.12.10.2086)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "gbsg_X, gbsg_y = load_gbsg2()\n",
    "gbsg_X = encode_categorical(gbsg_X)\n",
    "\n",
    "gbsg_X_train, gbsg_X_test, gbsg_y_train, gbsg_y_test = train_test_split(\n",
    "    gbsg_X, gbsg_y, stratify=gbsg_y[\"cens\"], random_state=1\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to train a model on the training data and assess\n",
    "its discrimination and calibration on the test data.\n",
    "Here, we consider a [Random Survival Forest](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.ensemble.RandomSurvivalForest.html#sksurv.ensemble.RandomSurvivalForest)\n",
    "and [Cox's proportional hazards model with elastic-net penalty](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.linear_model.CoxnetSurvivalAnalysis.html#sksurv.linear_model.CoxnetSurvivalAnalysisl)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CoxnetSurvivalAnalysis(fit_baseline_model=True, l1_ratio=0.99)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rsf_gbsg = RandomSurvivalForest(max_depth=2, random_state=1)\n",
    "rsf_gbsg.fit(gbsg_X_train, gbsg_y_train)\n",
    "\n",
    "cph_gbsg = CoxnetSurvivalAnalysis(l1_ratio=0.99, fit_baseline_model=True)\n",
    "cph_gbsg.fit(gbsg_X_train, gbsg_y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's start with discrimination as measured by the\n",
    "concordance index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RSF       0.688\n",
       "CPH       0.688\n",
       "Random    0.500\n",
       "Name: c-index, dtype: float64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_cindex = pd.Series(\n",
    "    [\n",
    "        rsf_gbsg.score(gbsg_X_test, gbsg_y_test),\n",
    "        cph_gbsg.score(gbsg_X_test, gbsg_y_test),\n",
    "        0.5,\n",
    "    ],\n",
    "    index=[\"RSF\", \"CPH\", \"Random\"],\n",
    "    name=\"c-index\",\n",
    ")\n",
    "\n",
    "score_cindex.round(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result indicates that both models perform equally well,\n",
    "achieving a concordance index of 0.688, which is significantly\n",
    "better than a random model with 0.5 concordance index.\n",
    "Unfortunately, it doesn't help us to decide which model we should\n",
    "choose. So let's consider the time-dependent Brier score\n",
    "as an alternative, which asses discrimination *and* calibration.\n",
    "\n",
    "We first need to determine for which time points\n",
    "$t$ we want to compute the Brier score for. We are going to use a data-driven\n",
    "approach here by selecting all time points between the 10% and 90%\n",
    "percentile of observed time points.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "lower, upper = np.percentile(gbsg_y[\"time\"], [10, 90])\n",
    "gbsg_times = np.arange(lower, upper + 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This returns 1690 time points, for which we need to estimate the\n",
    "probability of survival for, which is given by the survival function.\n",
    "Thus, we iterate over the predicted survival functions on the test data\n",
    "and evaluate each at the time points from above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "rsf_surv_prob = np.row_stack([fn(gbsg_times) for fn in rsf_gbsg.predict_survival_function(gbsg_X_test)])\n",
    "cph_surv_prob = np.row_stack([fn(gbsg_times) for fn in cph_gbsg.predict_survival_function(gbsg_X_test)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, we want to have a baseline to tell us how much better\n",
    "our models are from random. A random model would simply predict 0.5\n",
    "every time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "random_surv_prob = 0.5 * np.ones((gbsg_y_test.shape[0], gbsg_times.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another useful reference is the Kaplan-Meier estimator that does not consider\n",
    "any features: it estimates a survival function only from `gbsg_y_test`.\n",
    "We replicate this estimate for all samples in the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "km_func = StepFunction(*kaplan_meier_estimator(gbsg_y_test[\"cens\"], gbsg_y_test[\"time\"]))\n",
    "km_surv_prob = np.tile(km_func(gbsg_times), (gbsg_y_test.shape[0], 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of comparing calibration across all 1690 time points, we'll be\n",
    "using the\n",
    "[integrated Brier score](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.integrated_brier_score.html#sksurv.metrics.integrated_brier_score)\n",
    "(IBS) over all time points, which will\n",
    "give us a single number to compare the models by.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>c-index</th>\n",
       "      <th>IBS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>RSF</th>\n",
       "      <td>0.688</td>\n",
       "      <td>0.194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CPH</th>\n",
       "      <td>0.688</td>\n",
       "      <td>0.189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Random</th>\n",
       "      <td>0.500</td>\n",
       "      <td>0.247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kaplan-Meier</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0.217</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              c-index    IBS\n",
       "RSF             0.688  0.194\n",
       "CPH             0.688  0.189\n",
       "Random          0.500  0.247\n",
       "Kaplan-Meier      NaN  0.217"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_brier = pd.Series(\n",
    "    [\n",
    "        integrated_brier_score(gbsg_y, gbsg_y_test, prob, gbsg_times)\n",
    "        for prob in (rsf_surv_prob, cph_surv_prob, random_surv_prob, km_surv_prob)\n",
    "    ],\n",
    "    index=[\"RSF\", \"CPH\", \"Random\", \"Kaplan-Meier\"],\n",
    "    name=\"IBS\",\n",
    ")\n",
    "\n",
    "pd.concat((score_cindex, score_brier), axis=1).round(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Despite Random Survival Forest and Cox's proportional hazards model\n",
    "performing equally well in terms of discrimination (c-index), there seems to be\n",
    "a notable difference in terms of calibration (IBS), with\n",
    "Cox's proportional hazards model outperforming Random Survival Forest."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Metrics in Hyper-parameter Search\n",
    "\n",
    "Usually, estimators have hyper-parameters that one wants to optimize. For example, the maximum tree depth for tree-based learners. For this purpose, we can use scikit-learn's [GridSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV) to search for the hyper-parameter configuration that on average works best. By default, estimators' performance will be evaluated in terms of Harrell's concordance index, as implemented in [concordance_index_censored](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored).\n",
    "For other metrics, one can wrap an estimator with one of the following classes:\n",
    "\n",
    "- [as_concordance_index_ipcw_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_concordance_index_ipcw_scorer.html#sksurv.metrics.as_concordance_index_ipcw_scorer)\n",
    "- [as_cumulative_dynamic_auc_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_cumulative_dynamic_auc_scorer.html#sksurv.metrics.as_cumulative_dynamic_auc_scorer)\n",
    "- [as_integrated_brier_score_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_integrated_brier_score_scorer.html#sksurv.metrics.as_integrated_brier_score_scorer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV, KFold\n",
    "\n",
    "from sksurv.metrics import (\n",
    "    as_concordance_index_ipcw_scorer,\n",
    "    as_cumulative_dynamic_auc_scorer,\n",
    "    as_integrated_brier_score_scorer,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To illustrate this, we are going to use the Random Survival Forest and the\n",
    "[German Breast Cancer Study Group 2](http://ascopubs.org/doi/abs/10.1200/jco.1994.12.10.2086) from above.\n",
    "\n",
    "First, we define that we want to evaluate the performance of each hyper-parameter configuration by 3-fold cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv = KFold(n_splits=3, shuffle=True, random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define the set of hyper-parameters to evaluate. Here, we search for the best value for ``max_depth`` between 1 and 10 (excluding). Note that we have to prefix ``max_depth`` with ``estimator__``, because we are going to wrap the actual ``RandomSurvivalForest`` instance with one of the classes above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv_param_grid = {\n",
    "    \"estimator__max_depth\": np.arange(1, 10, dtype=int),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can put all the pieces together and start searching for the best hyper-parameters that maximize [concordance_index_ipcw](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcv_cindex = GridSearchCV(\n",
    "    as_concordance_index_ipcw_scorer(rsf_gbsg, tau=gbsg_times[-1]),\n",
    "    param_grid=cv_param_grid,\n",
    "    cv=cv,\n",
    "    n_jobs=4,\n",
    ").fit(gbsg_X, gbsg_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same process applies when optimizing hyper-parameters to maximize\n",
    "[cumulative_dynamic_auc](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.cumulative_dynamic_auc.html#sksurv.metrics.cumulative_dynamic_auc)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcv_iauc = GridSearchCV(\n",
    "    as_cumulative_dynamic_auc_scorer(rsf_gbsg, times=gbsg_times),\n",
    "    param_grid=cv_param_grid,\n",
    "    cv=cv,\n",
    "    n_jobs=4,\n",
    ").fit(gbsg_X, gbsg_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While [as_concordance_index_ipcw_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_concordance_index_ipcw_scorer.html#sksurv.metrics.as_concordance_index_ipcw_scorer)\n",
    "and [as_cumulative_dynamic_auc_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_cumulative_dynamic_auc_scorer.html#sksurv.metrics.as_cumulative_dynamic_auc_scorer)\n",
    "can be used with any estimator,\n",
    "[as_integrated_brier_score_scorer](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.as_integrated_brier_score_scorer.html#sksurv.metrics.as_integrated_brier_score_scorer) is only available for estimators that provide the ``predict_survival_function`` method, which includes ``RandomSurvivalForest``. If available, hyper-parameters that maximize the **negative** integrated time-dependent Brier score will be selected, because a lower Brier score indicates better performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcv_ibs = GridSearchCV(\n",
    "    as_integrated_brier_score_scorer(rsf_gbsg, times=gbsg_times),\n",
    "    param_grid=cv_param_grid,\n",
    "    cv=cv,\n",
    "    n_jobs=4,\n",
    ").fit(gbsg_X, gbsg_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can visualize the results of the grid search and compare the best performing hyper-parameter configurations (marked with a red dot)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAF0CAYAAAAJn79lAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA7YklEQVR4nO3deZxcdZ3v/9enq/c93Z3u7OkEkkAAidCCLEJLYERcGMYZlWVcxjEyI4qO44g681Pn/pjLjI7KXBXIIAIaQUUdGS/DbkB2EhYhCSERsie9pNP7XvW5f5zTnUqnk3SnqlLV3e/n49FUnXO+55xPNZ161/l+T51j7o6IiMjRykp3ASIiMrEpSEREJCEKEhERSYiCREREEqIgERGRhChIREQkIdnpLiAdqqqqvLa2Nt1liIhMKGvXrm129+kj50/JIKmtrWXNmjXpLkNEZEIxs62jzVfXloiIJERBIiIiCVGQiIhMER+65Wk+dMvTSd+ugkRERBKSEUFiZheb2UYz22xm142y/Itm9lL486qZRc2sYizriohIaqU9SMwsAnwfeDewFLjczJbGt3H3b7r7MndfBnwZeMzdW8ayrojIsZSq7qNMlvYgAc4ANrv7G+7eD9wNXHqY9pcDdx3luiIikmSZ8D2S2cD2uOkdwJmjNTSzQuBi4JqjWHcFsAKgpqaG1atXJ1S0iMhoWlt7ADLyPSZVtWVCkNgo8w51t633AU+6e8t413X3lcBKgLq6Oq+vrx9nmSKSSYa6j372qbPSXMmBbtoY1FVfn1l1Qepqy4SurR3A3LjpOcCuQ7T9MPu7tca7roiIpEAmBMnzwCIzW2BmuQRhce/IRmZWBpwP/Ga864rI0ZuKg8cyPmnv2nL3QTO7BngAiAC3ufs6M7s6XH5z2PQy4EF37zrSusf2FYiITG1pDxIAd78PuG/EvJtHTN8O3D6WdUUmokzt8xc5kkzo2hIRkQlMQSIiIglRkIiISEIUJCIikhAFiYiIJERBIiIiCVGQiIhIQhQkMuXom9oiyaUgERGRhChIREQkIQoSERFJiIJEREQSoiCRlNGgtsjUoCAREZkKVq3ie1+5jLuuPgdqa2HVqqRtOiMuIy8iIim0ahWsWMH07u5geutWWLEieH7llQlvXkckIiKTkLvT1jPAluYu1n77Vh6eeTI/P+Ui9hRXBg26u+GrX03KvpJ6RGJm+e7eO2Jelbs3J3M/IiJTzUA0xr7uflq6xvazr7ufgagHK1/098Pbue0XX2dG595gYtu2pNSW7K6t583sk+7+DICZfQD438DiJO9HZEJyd3oHYnT0DdDRO0hn72Dw2DdAU0cfUXdWPbuVnKwsIllGdsTIiWSRnRU+RozsrKHHA+flRIzssG12VvA8J26ZmaX75UvI3enujx7wxr+3q5994WNLVx8tXQO0dPWxr3uAvZ19tPcOHnJ7ZQU5VBblMq0ol7kVhZw6p5yK4txgXmEuFdesoGLLZiq626juatm/4rx5SXk9yQ6SK4DbzGw1MAuoBC440kpmdjFwI8F912919xtGaVMPfBfIAZrd/fxw/ueBvwYceAX4+MijIpFkGIjGht/823sH6OzbHwIdYSDET3f2DtLRd/C8wZgfdj9f/fWrKak/MhQwI0ImPqwiceGUEy57bU8HBvz1HWvIMsgyIysLzCx4PjQv/vnw8v3LzCBiRlZW8Dy+/YFtISvLDrvczGhs7wWDnz0ffKo2wqA88OGAAN0/b8RjuGRk1g6tO3K90dYZWtTS1Y8Dq57dSktnGBAjjiT2dvXTPxgb9f9TTsSoCAOgsjiXWeUFVBblUlGUR0VRDhVFeUwryqGyKI+KolzKC3PIiRxhlOKaK4MxkaExEoDCQrj++sOvN0ZJDRJ3f8XMrgd+DHQA57n7jsOtY2YR4PvARcAOgqOae919fVybcuAHwMXuvs3MqsP5s4HPAkvdvcfMfg58mFHu7T6Z6V7fYxccEUTpHYhy78u7wiOC/aHQETc9cl7fIf7hx8uJGCX5ORTnZVOSn01xXjazywsoyS/ZPy8/m5L8HEri2hTnZ/OVX71CJMu46arTGYjGGIw6g7EYA1Effj4Y84OWRUeZNxgN2g7GgucDQ+tHPVwnxkC4bDDq+5/HPQ5EY/QPxhiMxXCHXa09xNxxh5j78PNo+DwWC36/seHlQ9PhvFjc8xHbOUK2HtaXfvnK0a+cQkMfCErysplWlEtFUS4zSvM5cWbp8NFDRVHu8POhx5K87OQfPYYD6k2f/QKVLY1kzZ8XhEgSBtoh+WMkPwSOA95C0J3132b2PXf//mFWOwPY7O5vhNu4G7gUWB/X5grgV+6+DcDdG+OWZQMFZjYAFAK7kvV6RtIb9sTSPxhjU2MH63e1s25XO+t3t7NhVzsdfUEXwWfvenG4bZYRvtHnDL+5VxblMr+yiJL8bErysuOCIGd4Xkl+ThgMwfK87KyjfhPIz4kAUFOan/iLT6Jj9Xc/FELRmI8aNAeGlPOpO9fiOD+48nQ8bhvB42jbDx/xEdMj1h3RHg7e5sg28dv8h3teBuA/P/I2phXlkJcdGf8vIxWuvJJrOhcCyf9/meyurVeBv/bg/8ibZvZ24NtHWGc2sD1uegdw5og2i4GcsMusBLjR3e90951m9i1gG9ADPOjuDybhdcgE09E7wIbdHazf1ca6MDg2NXYMDzYW5kY4cWYpf/rW2TyxuYmCnAj/cflbh48eCnMjGkNIMzMjYkEX3FjkZgfdObPKC1JZ1rgV5gZvqzPKMusDQSolu2vrO2Y238wWufvDQD/wuSOsNtpfzcjPE9nA6cByoAB42syeAZoIjl4WAK3AL8zsKnf/yUE7MVsBrACoqalh9erVY31Zw1pbewCOat1Umkp1uTttfc7Wjhjb2mNsbY+xrSNGY/f+P5nSXJhXGuGiednML81iXkkWNUVGlvUDzTwX7SUahR3r1yatrmSYSv8fk0F1jV+qakt219YnCd6sKwi6uOYANxMEwKHsAObGTc/h4O6pHQQD7F1Al5k9DpwaLnvT3ZvC/f8KOBs4KEjcfSWwEqCurs7r6+vH9doAbtoYHOLX12dW19ZkrSsWc7bs7Ro+wli/u531u9po7uwfbjO/spC640pZOrOUk2aVsXRWKdUleYc9upisv69UUV3jk6l1QepqS3bX1qcJxjyeBXD3TUMD44fxPLDIzBYAOwkGy68Y0eY3wPfMLBvIJej6+g5QBLzdzAoJuraWA2uS9FrkGOodiLKpoZN1YdfU+t3tbNjdTnd/FAgGsRdVl/DOJdUsnRWExgkzSyjNz0lz5SKS7CDpc/f+4VPmgjf+w56P4e6DZnYN8ADB6b+3ufs6M7s6XH6zu28ws/uBPwAxglOEXw33cQ/wAjAIvEh41CGZq617gHW721i/qz342d3O5sbO4dNii/OyWTqzlA/WzQ1Do5RF1SXDfeIiklmSHSSPmdlXCM6iugj4W+C/j7SSu98H3Ddi3s0jpr8JfHOUdb8GfC2RoiU13J3+aIyH1jeEZ061sX53Ozv29Qy3qS7J46RZpSw/sZqTZpVx0qxS5k4rJGuMA64ikn7JDpLrgE8QfDHwUwThcGuS9yEZaiAaY/2udtZs3cfarS28uL2VgajzyTvXYAYLKotYNrecK86cF4xnzCxlekleussWkQQl+6ytGPCf4Y9Mcm09A7ywbR9rt+xjzdYWXt7eRs9AMKYxu7yA0vA7Ft/+4KmcMKOUojxdbFpkMkrKv2wze4XDjIW4+1uSsR9JH3dne0sPa7a2BEccW/bxemMH7sF5/0tnlvKht82lrnYadfMrmFGWP/xFttPnV6S5ehFJpWR9RHxv+Pjp8PHH4eOVQPfBzSXTDURjrNvVzpotLazduo81W/fR1NEHBJd8eOv8abznLTOpmz+NU+eW62hDZApLyr9+d98KYGbnuPs5cYuuM7MngX9Oxn4kddq6g26qNVtbWLNlHy/vaKV3ILi21JxpBZxzXCWn11ZQN38ai2tKxvztYxGZ/JL9MbLIzM519ycAzOxsgu96SAZxd7a1dLNmy77hgfHXGzqBoJvqpFmlXH7GPOrmV1BXOy3jrv0kIpkl2UHyCYLLyJeF063AXyV5HzJO/YMx1u1qC7qotuxj7ba4bqr8bE6bN433vWUWp9dOY9nc8uFrBYmIjEWyz9paC5xqZqWAuXtbMrcvY9PWPcDabS3DRxx/iOummltRwLnHV3H6/GnU1U5jUfXU66bS1ZtFkivZ19rKAz4A1ALZQ99wd3eNkaRQ/2CMtp4BvvyrVw7qpjp5VilXnDGfutppnD5f3VQiknzJ7sP4DdAGrAX6krxtCbk7rzd08tD6PTy0voGXdwQHfk2dfZw+X91UInJsJftdZo67X5zkbQowGI3x/JZ9PLS+gYc3NLCtJTiretnccuZMK6C8IId7rzlXlxYRkWMu2UHylJmd4u6Zee/LCaarb5DHX2/iofUNPLqxkdbuAXKzszjnuEquPv84LjyxmurS/V/8U4hMbBq7kYkq2UFyLvAxM3uToGvLANc328eusb2Xhzc08tD6PTz5x730D8YoL8zhgiXVXLS0hvMWT9eX/0QkoyT7HendSd7epOfubGrs5KH1DTy4voGXt7cCwdlVV505n4uW1vC22mlkR3QJdRHJTMm61lapu7cDHcnY3mQ3GI2xdmsw3vHQhga27g3GO06dU8bf/8liLlo6g8U1xbqHuGQEdbnJkSTriOSnBNfbWktw8cb4d0AHFiZpPxNWd38w3vHg+gZ+91oj+7oHyI1kcdZxlXzyHQu58MQaZpTp1FwRmXiSda2t94aPC5KxvcmisaOXRzY08tD6Bp7Y3Ez/YIyyghwuOGH/eEfxJB7v0CdZkakhZe9iZvZ1d/96qrafidydPzZ18uD6Bh5a38BL21txDy56eOWZ88LxjgpyNN4hkjB9UBm/VP3OUvlx+P3A11O4/YwQjXk43rGHhzc08mZzFwCnzC7j8xcu5qKlNZwwo0TjHSIyaaUySMb8zmlmFwM3AhHgVne/YZQ29cB3gRyg2d3PD+eXE9zO92SC8Zi/cvenEyv98Lr7B/n9pubg+x2vNdLS1U9OxDjruCr+6twFXHhiNTPLClJZgohIxkj2tbbuAK5191bgdDObBvy7ux/yCsBmFgG+D1wE7ACeN7N73X19XJty4AfAxe6+zcyq4zZxI3C/u/+5meUChcl8TfFauvpp6uzjrf/8EH2DMUrys4fHO85fPJ2S/JxU7VpEJoip2OWW7COSt4QhMnT/9n1m9tYjrHMGsNnd3wAws7uBS4H1cW2uAH7l7tvCbTeGbUuB84CPhfP7gf5kvZiROvsG6e6LckU43nHGAo13iIgk+10wKzwKAcDMKjhyWM0GtsdN7wjnxVsMTDOz1Wa21sw+Es5fCDQBPzKzF83sVjNL2Y205pQXsGxuGV9//0mcc3yVQkREhOQfkfw7wfW27iEYr/ggcP0R1hltLMVHTGcDpwPLgQLgaTN7Jpx/GvAZd3/WzG4ErgP+6aCdmK0AVgDU1NSwevXqsb6mYe3tPQBHtW4qtbZmZl0iMjUk+8ZWd5rZGuACgoD4s/ixjkPYAcyNm54D7BqlTbO7dwFdZvY4cCrwe2CHuz8btruHIEhGq20lsBKgrq7O6+vrx/y6hty0MRjDr6/PrD7QTK1LRKaGpJ+1FQbHkcIj3vPAIjNbAOwEPkwwJhLvN8D3zCwbyAXOBL7j7nvMbLuZLXH3jQRHLOPZt4iIJCjtX6t290EzuwZ4gOD039vcfZ2ZXR0uv9ndN5jZ/cAfgBjBKcKvhpv4DLAqPGPrDeDjx/5ViIhMXWkPEgB3vw+4b8S8m0dMfxP45ijrvgTUpbK+TDcVTzcUkcyh045ERCQhChIREUmIgkRERBKiIBERkYQoSEREJCEKEhERSYiCREREEqIgERGRhChIREQkIQoSERFJiIJEREQSoiAREZGEKEhERCQhChIREUmIgkRERBKiIBERkYQoSEREJCEKEhERSYiCREREEpIRQWJmF5vZRjPbbGbXHaJNvZm9ZGbrzOyxEcsiZvaimf322FQsIiJDstNdgJlFgO8DFwE7gOfN7F53Xx/Xphz4AXCxu28zs+oRm7kW2ACUHpuqRURkSNqDBDgD2OzubwCY2d3ApcD6uDZXAL9y920A7t44tMDM5gDvAa4H/i6Vhf7sU2elcvMiIhNSJgTJbGB73PQO4MwRbRYDOWa2GigBbnT3O8Nl3wX+IZx/SGa2AlgBUFNTw+rVqxOtW0REyIwgsVHm+YjpbOB0YDlQADxtZs8QBEyju681s/rD7cTdVwIrAerq6ry+/rDNRURkjDIhSHYAc+Om5wC7RmnT7O5dQJeZPQ6cCpwGvN/MLgHygVIz+4m7X3UM6hYRETLjrK3ngUVmtsDMcoEPA/eOaPMb4B1mlm1mhQRdXxvc/cvuPsfda8P1HlWIiIgcW2k/InH3QTO7BngAiAC3ufs6M7s6XH6zu28ws/uBPwAx4FZ3f/Vo97l27dpmM9t6lKtXAc1Hu+8UUl3jo7rGR3WNT6bWBYnVNn+0meY+cjhCDsfM1rh7XbrrGEl1jY/qGh/VNT6ZWhekprZM6NoSEZEJTEEiIiIJUZCM38p0F3AIqmt8VNf4qK7xydS6IAW1aYxEREQSoiMSERFJiIJEREQSoiAREZGEKEhERCQhChIREUmIgkRERBKiIBERkYQoSEREJCEKEhERSYiCREREEqIgERGRhGREkJjZxWa20cw2m9l1oyz/opm9FP68amZRM6sIl33ezNaF8+8ys/xj/wpERKautF+00cwiwOvARQT3Zn8euNzd1x+i/fuAz7v7BWY2G3gCWOruPWb2c+A+d7/92FQvIiJpv9UucAaw2d3fADCzu4FLgVGDBLgcuCtuOhsoMLMBoBDYdaQdVlVVeW1tbSI1i4hMOWvXrm129+kj52dCkMwGtsdN7wDOHK2hmRUCFwPXALj7TjP7FrAN6AEedPcHD7HuCmAFQE1NDd/61reS9gJERKaCd77znVtHm58JQWKjzDtUf9v7gCfdvQXAzKYRHL0sAFqBX5jZVe7+k4M26L6S8IYudXV1Xl9fn3jlIiKSEYPtO4C5cdNzOHT31Ic5sFvrQuBNd29y9wHgV8DZKalSRERGlQlB8jywyMwWmFkuQVjcO7KRmZUB5wO/iZu9DXi7mRWamQHLgQ3HoGaZwD50y9N86Jan012GyKSR9q4tdx80s2uAB4AIcJu7rzOzq8PlN4dNLyMYA+mKW/dZM7sHeAEYBF4ks++VLCIy6aQ9SADc/T7gvhHzbh4xfTtw+yjrfg34WgrLExGRw8iEri0REZnAFCQiIpIQBYmkjAa1x0e/L5moFCQiclgKODkSBYmIiCREQSIiE5KOlDKHgkREJIkyOeBSVZuCREREEqIgERGRhChIJoFMPpQWkclPQSIiIglRkIiISEIUJCIikhAFiUgmWLWK733lMu66+hyorYVVq9JdkciYZcRl5EWmInenbzBG109/Rvd1X6UrK5/dM46jsLOXwr/7MkVRo/DKD5MT0ec9yWwKEpExGIjG6O6P0t0/SFffiMf+KD0jprv7wsf+wWC9vihd4fOuvsHhbcUcoAw++v2Dd7oe+Or/kJudRVFuhMLcbIryIhTlZVOUm01hbvC8MDdCcV728PLhdrnZFIaP++cH7RVOkkwKEpky2nsH2NLcxd7OPvoGY3z7wY3Db/ZDIdDdHx0OguD5IN19UfqjsTHvJydiwZt2boTCvOzhEJhZlk9BbvYBoVCYm03RF66lsL+HwoE+cgcH6M7Npzsnn67cArpv+CZdYfgM1TIUSM2dfQfM6x0Ye43x4VSctz9wRgun3W09RLKyWLt1H4triinJzzmaX79MYgoSmVSGwuLN5i627u1mS3MXW/Z2sWVvNy1d/Qe0/T+/20xRbjYFuZED3tzLC3KYXZ5PYfjGelAohI/BenGBEG4rN3ucn/b3vgpbtx48f/58WL5ozJuJxvzAsAkfu/pGHCX1DdJ5iHaHC6cP3PQUALPLC1hcU8ziGSUsqSlhcU0Jx1cXk58TGd/rlklDQTIOQ1/6+9mnzkpzJVNbW88AW/eOLSxmleUzv7KId500g9rKQmqrivjeo5vIy47wi6vPwszS9CriXH89rFgB3d375xUWBvPHIZJllObnUJrEI4bBaIwPrXyawajzmQsWsbGhg9cbOti4p4MnNjczEHUAsgxqq4qGg2XJjOCxtrKQbHWjTXoKEslIbT0D+wOiuTsIjr1BcIwlLBZUFTGvonDUT8m3PfEmQGaECMCVVwLQ9NkvUNnSSNb8eUGIhPPTKTuSRXZWFtlZcOHSGi5cWjO8bCAaY0tzVxAuezrY2NDBa3s6uH/dHjzIF3Kzszh+evFwsCyZUczimhJmlxdkzu9fEqYgkbQ5VFhsae5iX/fAAW3jw2JBVSHzKw8fFhPOlVdyTedCYOIc8eZEslhUU8KimhJ4y/75Pf1R/tjUyWt79h+9PPPGXn794s7hNsV52SyqKWZJePSypKaExTNKqCrOS8MrkUQpSCSlBqMxXt7eOhwWQRfUocOitqqIi0+eOTnDYoooyI1w8uwyTp5ddsD8tp4BNjV0DB/BDB293P389uE2lUW5B3SNBY8a4M90ChJJKndnw+4OHli3h1d2ttHdH+XS7z85vFxhMXWVFeRQV1tBXW3F8Dx3p6mzj9f3dB7QRfbzNdvp7o8OtxttgD8Wc7Ky1D12KO5Oc2c/O1t72Lmvh52twQe5mtL8pO9LQSIJi8WcF7bt44F1e3hgXQPbWroxg+LcbOZMK+Cf3rtUYSGjMjOqS/KpLsnn3EVVw/NjMWdnaw8bw2AZbYAfIC87i8tXPsOMsnyqS/OoKcmnpjSfmtI8akrzmV6SN2n/5gajMfa094Yh0bP/Me553+CBp4RHzCgvyE16LRkRJGZ2MXAjEAFudfcbRiz/IjA08pgNnAhMd/cWMysHbgVOBhz4K3fXNdVTrH8wxlN/bOaBdQ08tL6B5s4+ciNZnHN8JX9bfxwXLq3h06teAOBdJ81Ic7Uy0WRlGXMrCplbUXjQAP/WvV1s3NPJv9y3np7+4Ds+z29pobG9b9Tv+5QX5lBTEgTNjNL9QVMd97yqOC/jvqTZOxA9MCBGPO5p7yUa8wPWqSrOZXZ5ASfMLGH5idXMLi9g9rTC8LGAFXeuSUmtaQ8SM4sA3wcuAnYAz5vZve6+fqiNu38T+GbY/n3A5929JVx8I3C/u/+5meUChcf0BUwhXX2DPPZ6E/e/uoffvdZIR98gRbkR6k+o5l0nzeCdS6arL1tSKieSxfHVJRxfXcKdT28B9p+c4O60dg/Q0NFLQ3sfDW29NLT3Dk83tveyqaGTps6+g96AzaCyKG/4SGb/Yxg64ZFOZVFu0rrT2noG4sKh+6CjiebOA89OjGQZM0rzmV1ewBkLKobDIf4xXUdfaQ8S4Axgs7u/AWBmdwOXElwgYjSXA3eFbUuB84CPAbh7P9B/iPXkKLR09fPwhgYeXLeHxzc10z8Yo6Iol0tOmcm7Tq7h7OOqJm3XgUwsZsa0olymFeVywmEOgqMxZ29XH43tfexpOzBoGtp72dPWyx92tB70Rg6QnWVMLwmPZkqCsJlRlk91yf7gGYzGiGQZje297DjEEcWu1h46+gYP2HZedtZwKJw4s/SgoJhRmp+x38nJhCCZDWyPm94BnDlaQzMrBC4GrglnLQSagB+Z2anAWuBad+9KXbmT367WnnC8Yw/PvdlCzIPBzivPnMe7TppB3fxpGfsHLXIkkaz94zIjzyyL1z8Yo7mzjz3tvWHI9AVHOO19NHb0smVvF8++2UJbz8Co65/xL48cMF2an83saUF33VnHVR4QFLPKC6gqzp2w363JhCAZ7Tfno8wDeB/wZFy3VjZwGvAZd3/WzG4ErgP+6aCdmK0AVgDU1NSwevXqcRfa2toDcFTrplIy6trVGWNtwyAvNER5sz3oZ55dbLxnYQ6nV0eYX2qYNdG7rYknth27ulJBdY2P6oJ8YD4wPz+cqB5akkt/NIfWPmdfr9Pa59zzej9Rdy5ZkEtVgVFZkEVVgVGQbUAU6Ax+YsBe2LcX9qX8FQRS9TvLhCDZAcyNm54D7DpE2w8TdmvFrbvD3Z8Np+8hCJKDuPtKYCVAXV2d19fXj7vQmzYGY/j19Zn1hbGjqcvdeXlH2/CRxxtNwR/YsrnlfOnsGbzrpBoWTi8+5nUdC5la11H8SR4Tmfr7ytS6XggvpfT/fzSz6oLU/c4yIUieBxaZ2QJgJ0FYXDGykZmVAecDVw3Nc/c9ZrbdzJa4+0ZgOYceW5nyBqMxnnuzhQfW7eHB9Q3sbuslO8t4+8JKPn52LRctncGMsuSfYy4ik1vSgsTM3gZUufv/jJj/fmCnu68dbT13HzSza4AHCE7/vc3d15nZ1eHym8OmlwEPjjL+8RlgVXjG1hvAx5P1miaD3oEoj7/exAPrGnjktQZauwfIz8nivEXT+eK7lrD8hBrKCnWmlYgcvWQekXyT8OypEdYTdCldcKgV3f0+4L4R824eMX07cPso674E1I2z1kmtrWeA373WyAPr9rB6YxM9A1FK87O58MQa/uSkGZy/eDoFuTrTSsZmolz7S9InmUFS6e5bRs50981mVpnE/cgo+gdj/OSZrTywbg9P/3EvgzGnuiSPD5w+m4tPmsmZCysy7gtXIjI5JDNICg6zrCiJ+5FQNOb87rVGXtvTTlvPIC9ub2VBVRGfeMcC3nXSDJbNKde1iEQk5ZIZJA+b2fXAP7r78Om7ZvYN4NEk7mfKa+se4OdrtvPjZ7ayraWbnIgxuzyfH338DBZVF0/Yc9FFZGJKZpB8geCaV5vN7KVw3qnAGuCvk7ifKWvjng5uf2oL//XiTnoGorytdhr/cPES7nhqC1lmLK4pSXeJIjIFJS1IwrOpLjezhcBJ4ex1Q5c+kaMzGI3x8IZGbn/qTZ55o4W87CwuXTaLj55dy0mzgm/l/vjpUe73LYekwWOR5Erm6b+nxU3uJPh2+ujXDpAjaunq5+7nt7HqmW3sbO1hdnkBX7r4BD78trlMK0r+ZaBFRI5WMru2/n2UeRXh9zsuD0/TlSN4dWcbdzy1hXtf3kXfYIyzFlbyT+9dyoUnVuv6ViKSkZLZtfXO0eabWR3wHwRX6ZVRDERj3P/qHu54agtrtu6jICfCB06fw0fPqmXJDI17iEhypKpbN+WXSHH3NWaW2EWbJqmmjj7uem4bq57dSkN7H/MqCvnH95zIX5w+V982F5EJI+VBYmY1HPpqvlPSS9tbueOpLfzfP+ymPxrjHYuq+JfLTqF+STURfe9DRCaYZA62/x8ODowK4Gzg2mTtZ6LqG4xy3yu7uf2prby8vZWi3AiXnzGXj5xdy3EJXmVXZCrS2XeZI5lHJCNvBuzAXuDv3L0xifuZUBrae1n1zFZ++tx2mjv7WFhVxNfft5QPnD5n0t+WVv/QRaaGZA623zHafDOba2ZfDO+7PiW4Oy9s28ePntzC/a/uIerOO5dU89Gza3nH8VW6bImITCopGSMxsyrgLwjurz4b+HUq9pNpegei3PvyLu58eguv7mynJD+bj55dy0fOms/8Sl1uTEQmp2SOkZQQ3DPkCmAxQXgsdPc5ydpHptrZ2sNPntnK3c9tY1/3AItrirn+spO57K2zKczNhHuHiYikTjLf5RqB54B/BJ5wdzezy5K4/Yzi7jzzRgt3PLWFB9fvAeDCE2v42Nm1nHVcpS6cKCJTRjKD5CsEt8m9Cfipmf0sidvOGNGY89Nnt3Hn01t4bU8H5YU5fPK8hfzl2+czZ1phussTkTSbiieZJHOw/TvAd8KLNl4O/Bcwy8y+BPza3V9P1r7SZXdbLztbe1izdR8nzizlXz9wCpcum01+ju42KCJTV9I78MOr/V4PXG9mpxCMmfwPcFyy93WsZRmUFeRwy1+eTt38aeq+EhEBUnoVQHd/xd2/7O4TPkQAakrzWVRdzNtqKxQiIiKhZJ619YS7n2tmHRz4DXcD3N1Lk7UvERHJHMkcIzk3fNTlakVEphDd4EJERBKiIBERkYRkRJCY2cVmttHMNpvZdaMs/6KZvRT+vGpmUTOriFseMbMXzey3x7ZyERFJe5CYWQT4PvBuYClwuZktjW/j7t9092Xuvgz4MvCYu7fENbkW2HCMShYRkThpDxLgDGCzu7/h7v3A3cClh2l/OXDX0ISZzQHeA9ya0ipFRGRUmXBFwdnA9rjpHcCZozU0s0LgYuCauNnfBf4BmLJni03FSzKISObIhCAZ7Zt9h7o17/uAJ4e6tczsvUCju681s/rD7sRsBbACoKamhtWrV4+70NbWHoCjWldEZLLKhCDZAcyNm54D7DpE2w8T160FnAO838wuAfKBUjP7ibtfNXJFd18JrASoq6vz+vr6cRd608anAaiv1xGAiMiQTBgjeR5YZGYLzCyXICzuHdnIzMqA84HfDM0LL78yx91rw/UeHS1EREQkddJ+ROLug2Z2DfAAEAFuc/d1ZnZ1uPzmsOllwIPu3pWmUkVEZBRpDxIAd78PuG/EvJtHTN8O3H6YbawGVie9OBEROaxM6NoSEZEJTEEiIiIJUZCIiEhCFCQiIpIQBYmIiCREQSIiIglRkIiISEIUJCIikhAFiYiIJERBIiIiCVGQiIhIQhQkIiKSEAWJiIgkREEiIiIJUZCIiEhCFCQiIpIQBYmIiCREQSIiIgnJiFvtThQ/+9RZ6S5BRCTj6IhEREQSoiAREZGEKEhERCQhChIREUmIuXu6azjmzKwJ2HqUq1cBzUksJ1lU1/iorvFRXeOTqXVBYrXNd/fpI2dOySBJhJmtcfe6dNcxkuoaH9U1PqprfDK1LkhNberaEhGRhChIREQkIQqS8VuZ7gIOQXWNj+oaH9U1PplaF6SgNo2RiIhIQnREIiIiCVGQiIhIQhQkIiKSEAWJiIgkREEiIiIJUZCIiEhCFCQiIpIQBYmIiCREQSIiIglRkIiISEIUJCIikhAFiYiIJERBIiIiCclOdwHpUFVV5bW1tekuQ0RkQlm7dm3zaLfanZJBUltby5o1a9JdhojIhGJmW0ebr64tERFJiIJEREQSoiARkcP60C1P86Fbnk53GZLBFCQiIlNEqj4UKEhERJJoKh7BKUhEZEKaim/YmUpBIpIh9MYoE5WCREREEqIgERGRhChIREQkIQoSERFJiIJEphwNaoskl4JEREQSoiAREZGEKEgkZdSFJDI1KEhERCQhChIREUmIgkRERBKiIBERkYQoSEREJCEKEhERSYiCREREEqIgERGRhChIREQkIQoSERFJiIJkEtClSCRlVq3ie1+5jLuuPgdqa2HVqnRXJBkorUFiZhVm9pCZbQofpx2i3W1m1mhmr46Yv8zMnjGzl8xsjZmdcWwqF5kCVq2CFSuY3tJAFg5bt8KKFQoTOUh2mvd/HfCIu99gZteF018apd3twPeAO0fM/zfgG+7+P2Z2SThdn7pyRSaPnv4ozZ19NHf2sbezn71dfTR39u+ffmgPzR/+N/YWltGRW0hVdxvVnS3U3Pcm1cWvUlOaT3VJXvBYmkdNST7lhTmYWbpfmhxj6Q6SS9n/xn8HsJpRgsTdHzez2lHWd6A0fF4G7Ep6hSITRCzmtPYMHBAO8SHR1BE8Ds3v7o+Oup3ivGwqi3OpGowxv303p+3cQHF/D3sLymgsqeCPxZU89eJO2nsHD1o3N5IVhMooIRP/vLQgW4EziaQ7SGrcfTeAu+82s+pxrv854AEz+xZBN93ZSa5PJK16B6IHHjF09NPcdWBIBMHRz77ufqIxP2gbWQYVRXlUFedSVZzHvHmFVBXnBWFRlEdVSS6VReF0cR75OZFgxdqPB91ZI82fD1u20DsQpbG9j4aOXhrae4efN7b30dDey+sNHTyxuZmOUQInLzvr4LApzaemNI/qkvCxNJ+SPAXOWLk7/dEYvQMx+gai9A7E6B2M0jsQpW8wRu9AlH1d/RTnJ/9tP+VBYmYPAzNGWfTVJGz+b4DPu/svzeyDwA+BCw9RxwpgBUBNTQ2rV69Owu4zQ2trD0DGvaapVJe7MxCD/ij0xzx4jIaPsf3P+6JBu74oDMTi5kVhQ+MgMYcLb/gf2vud9j6nd/SDBvIiUJprwU+esbTcKKvOHp6OX1aUA1lmQAzoCX9CXcHPPoKfzXH7qL7qKpZ861tE+vqG50Xz8th41VU0jvjdFYc/CwuAAqBmuFL6BnNp7XNa+5x9fU5bn7Ov12nt66O1o5cdza209o7+WnMjUJ5nwz/T8ozy/CzK8oxdTf1EsuCu3z6KGRgwlDlD0RM/f3he+B/D9i+Paw9B+A613d/+4O3E728o8Ib+vh5+9Hf7/yaiQ38fwePQ8/4YDMT9nQyE8/rDv4/4dUbbzlC7YN2gi+ZIZhdb0v9NmvtYdp0aZrYRqA+PRmYCq919ySHa1gK/dfeT4+a1AeXu7hb8X2xz99LR1o9XV1fna9asSc6LyABDZ2z97FNnpbmSA2ViXf2DMf7i5qfoH4zxpXefQO9AlJ7w01tPf/C8L37ewIHzevrDT3oD0eF1h9oejUiWUZATIT8nQkfvAFlmvHVeOZXF+48gKoty9x9BhI+FuceoM2HVKpo++wUqWxrJmj8Prr8errwyJbvq7Buksb2XhvY+GuOObBo6+mhs76Wxo489bb30DBwiXTOAGSTjLTUvO4v8nAj5OeFjdvA8L/xbGV4+sl3csv3b2L/8G/euIy8nwi//5ug6b8xsrbvXjZyf7q6te4GPAjeEj78Z5/q7gPMJxlYuADYlsziZeNydvV39bGvpZnv4sy382d7Sw+62HoZ6fz72o+dH3YYZ5GdHKMiNUJATIS8na/jNvjA3m4qi4B/m0LyC3P3/WIfn5Ywyb5Tt5UT2nzg5FLw//eTbU/57GrMrr+SazoVA6j8QFOdlUzy9mIXTiw/Zxt2DwOno49OrXmAwGuMzyxcBEHPHPXgj97Dt8OPwPHA8+BsYXr6/bSx8zoi2Q8+HQsL94PlD+/rl2h0AXH7GvOG/gUMHwIFhkZeTRV52Vsq684ryUvOWn+4guQH4uZl9AtgG/AWAmc0CbnX3S8LpuwgG5avMbAfwNXf/IfBJ4EYzywZ6CbuuZHLrHYgGIbGvm217u9nW0sO2lm527AsCY+Qg8vSSPOZVFHLGggrmTivgt3/YTW52Fv/yZ6cc8MY/9Cafyn/IkhgzoyQ/h5L8HMoKcgC4dNnsNFd1oOfebAEYDripIK1B4u57geWjzN8FXBI3ffkh1n8COD1lBUpaxGJOQ0cv28OAGHl00djRd0D7gpwI8yoKmVtRyFnHVTKvonD4Z860QgpyIwe0fzb8h37avFG/tiQi43TEIDGztwHb3X1POP0R4APAVuDr7t6S2hJlMuroHRgOiqGA2B4eUezY10P/4P4xBzOYVVbA3IoCzl88PQiJyiA45k4rpKo4V0cQImk0liOSWwjPhDKz8wi6oz4DLANWAn+equJkYhuMxejqi3LXc9sOOqrY1z1wQNuS/GzmVxaypKaEi06sYW54RDG3opDZ5QXkZutqPiKZaixBEok76vgQsNLdfwn80sxeSlllMuG4O+t3t/PY602s3tjEC1tbceDLv3qF7CxjzrQC5lYU8u5TZg53Pc2dFjyWFeaku3wROUpjChIzy3b3QYLxjPgB7XQP1kuatXUP8PvNQXA89noTTeH4xUmzSplZlk9pQQ63frSOmWUFRLLU/SQyGY0lCO4CHjOzZoJvMv0ewMyOB9pSWJtkoFjMeXVX23BwvLhtHzGHsoIc3rGoivol1Zy3qIrq0vzh01nnTCtMc9UikkpHDBJ3v97MHgFmAg/6/m8wZhGMlcgk19LVz+83BUcdj7/exN6ufszgLbPLuOadx3P+kmqWzS3XEYfIFDWmril3fyZ+2syqgE1xoSKTSDTmvLyjdfio4w87WnGHiqJczguPOt6xqIrK4rx0lyoiGWAsp/++neBMrRbgfwE/BqqALDP7iLvfn9oS5Vho6ujj8debWP16E7/f1ERr9wBZBsvmlvO55YupXzKdU2aXkaWjDhEZYSxHJN8DvkJwmfZHgXe7+zNmdgLB+ImCZAIajMZ4cXsrqzc28tjrTby6sx2AquI8lp9QQ/2S6Zx7fBXTinLTXKmIZLqxBEm2uz8IYGb/PNTN5e6v6UtgE8uett7wqKOR328KLu8dyTJOnzeNL75rCecvns7SmaU66hCRcRlLkMRf1rRnxDKNkWSw/sEYa7fuY/XrjTy2sYnX9nQAMKM0n0tOnkn9kumcfXzV8DWLRESOxliC5FQzaye45H5B+JxwOj9llWWgTLws+kg7W3t4bGMTqzc28tQf99LZN0hOxKibX8F17z6B+iXTWVJTokuKiEjSjOX038iR2kh6dfcP0tTRz0XffoxNjZ0AzC4v4P3LZlG/ODjqKE7R5aNFRPTuMoG93tDBjQ9v4pWd7RiwqKaKD71tLvVLpnPc9GIddYjIMTGW0387CMZC4t+VhqZ9LHcklOTa3NjBjY9s5rd/2EVhToRZZfnMLMvnx584M92licgUNJaurZJjUYgc2RtNnfzHI5v4zcu7KMiJ8DfnH8cn37GQq3+yNt2licgUNu6uLTN7n7v/dyqKkdFtae7iPx7ZxH+9tJO87AgrzlvIincs1DfLRSQjHM0YyfWAguQY2La3m/94dBO/fnEnORHjE+cu4FPnH0eVAkREMsjRBIlGcFNse0s333t0M/e8sIPsLOOjZ9Vydf1Cqkum1NnWkiEy+XR3yQxHEyT6EmKK7Gzt4XuPbuYXa7aTlWX85dvn87f1x1FdqgBJJr0xiiSXTv/NALvbevj+7zbzs+e3YxhXnDmPv60/nhllChARyXwKkjRqaO/lB7/bzF3PbcdxPlg3l0+/83hmlRekuzRJAx0pyUR1NEHSkPQqppjG9l5+sPqP/PS5bcRizl/UzeHT7zxedxIUkZRK1YeVcQeJu1+UikKmgqaOPm5+7I/85JmtDMacD5w2m89csIi5FQoQEZm41LV1DDR39rHy8Te48+kt9A/GuOytc/js8uOZX1mU7tJEJix1BWYOBUkKtXT1s/LxN7jjqS30DUb502Wz+czyRSyoUoCITFZTMeAUJCmwr6uf//x9ECDdA1Hef+osPrt8EcdNL053acfUVPwHJTIVKUiSqK17gFufeIMfPbmFrv5B3nPKTK5dvohFNbpcmYhMXgqSJGjrGeC2J97ktifepKNvkEtOmcG1yxezZIYCREQmPwVJAjp6B/jRk1u49fdv0N47yLtOquFzFy7mxJm6sr6ITB1pDRIzqwB+BtQCW4APuvu+EW3mAncCMwjuH7/S3W8c6/qp0Nk3yB1PbWHl42/Q1jPAhSfW8LkLF3Hy7LJU71pEJONkpXn/1wGPuPsi4JFweqRB4AvufiLwduDTZrZ0HOsnTTTm7Grt4R3/+ijffGAjdfOn8d/XnMutH61TiIjIlJXurq1Lgfrw+R3AauBL8Q3cfTewO3zeYWYbgNnA+rGsn0xvNnext6uf+iXT+dyFi1k2tzxVuxIRmTDSHSQ1YVDg7rvNrPpwjc2sFngr8OzRrJ+oWeUF1JTmc/vHz0jlbkREJpSUB4mZPUwwvjHSV8e5nWLgl8Dn3L39KOpYAawAqKmpYfXq1ePdBP3dPQBHtW4qtbZmZl0iMjWkPEjc/cJDLTOzBjObGR5NzAQaD9EuhyBEVrn7r+IWjWn9sI6VwEqAuro6r6+vH/druWnj0wDU12fWF+0ytS4RmRrS3bV1L/BR4Ibw8TcjG5iZAT8ENrj7t8e7/lSgb5CLSDql+6ytG4CLzGwTcFE4jZnNMrP7wjbnAH8JXGBmL4U/lxxufREROXbSekTi7nuB5aPM3wVcEj5/gkPcJ/5Q64uIyLGT7iMSERGZ4BQkIiKSEAWJiIgkREEiIiIJUZCIiEhCFCQiIpIQBYmIiCREQSIiIglRkIiISEIUJCIikhAFiYiIJERBIiIiCVGQiIhIQhQkIiKSEAWJiIgkREEiIiIJSfetdicU3dJWRORgOiIREZGEKEhERCQhChIREUmIuXu6azjmzKwJ2HqUq1cBzUksJ1lU1/iorvFRXeOTqXVBYrXNd/fpI2dOySBJhJmtcfe6dNcxkuoaH9U1PqprfDK1LkhNberaEhGRhChIREQkIQqS8VuZ7gIOQXWNj+oaH9U1PplaF6SgNo2RiIhIQnREIiIiCVGQjJGZ3WZmjWb2arpriWdmc83sd2a2wczWmdm16a4JwMzyzew5M3s5rOsb6a4pnplFzOxFM/ttumsZYmZbzOwVM3vJzNaku54hZlZuZveY2Wvh31narxVkZkvC39PQT7uZfS7ddQGY2efDv/lXzewuM8tPd00AZnZtWNO6ZP+u1LU1RmZ2HtAJ3OnuJ6e7niFmNhOY6e4vmFkJsBb4U3dfn+a6DChy904zywGeAK5192fSWdcQM/s7oA4odff3prseCIIEqHP3jPr+gZndAfze3W81s1yg0N1b01zWMDOLADuBM939aL8flqxaZhP8rS919x4z+zlwn7vfnua6TgbuBs4A+oH7gb9x903J2L6OSMbI3R8HWtJdx0juvtvdXwifdwAbgNnprQo80BlO5oQ/GfGpxczmAO8Bbk13LZnOzEqB84AfArh7fyaFSGg58Md0h0icbKDAzLKBQmBXmusBOBF4xt273X0QeAy4LFkbV5BMImZWC7wVeDbNpQDD3UcvAY3AQ+6eEXUB3wX+AYiluY6RHHjQzNaa2Yp0FxNaCDQBPwq7Am81s6J0FzXCh4G70l0EgLvvBL4FbAN2A23u/mB6qwLgVeA8M6s0s0LgEmBusjauIJkkzKwY+CXwOXdvT3c9AO4edfdlwBzgjPDwOq3M7L1Ao7uvTXctozjH3U8D3g18OuxOTbds4DTgJnd/K9AFXJfekvYLu9reD/wi3bUAmNk04FJgATALKDKzq9JbFbj7BuBfgYcIurVeBgaTtX0FySQQjkH8Eljl7r9Kdz0jhV0hq4GL01sJAOcA7w/HI+4GLjCzn6S3pIC77wofG4FfE/Rnp9sOYEfc0eQ9BMGSKd4NvODuDekuJHQh8Ka7N7n7APAr4Ow01wSAu//Q3U9z9/MIuumTMj4CCpIJLxzU/iGwwd2/ne56hpjZdDMrD58XEPwDey2tRQHu/mV3n+PutQRdIo+6e9o/MZpZUXiyBGHX0Z8QdEeklbvvAbab2ZJw1nIgrSdyjHA5GdKtFdoGvN3MCsN/m8sJxi3Tzsyqw8d5wJ+RxN+b7pA4RmZ2F1APVJnZDuBr7v7D9FYFBJ+w/xJ4JRyPAPiKu9+XvpIAmAncEZ5RkwX83N0z5lTbDFQD/Dp47yEb+Km735/ekoZ9BlgVdiO9AXw8zfUAEPb1XwR8Kt21DHH3Z83sHuAFgq6jF8mcb7n/0swqgQHg0+6+L1kb1um/IiKSEHVtiYhIQhQkIiKSEAWJiIgkREEiIiIJUZCIiEhCFCQiIpIQBYlIhgovK191lOt+zMxmJWNbIkeiIBGZnD5GcK0nkZRTkIgcgZnVhjd1ujW8MdAqM7vQzJ40s01mdkb481R4hdynhi4pYmZ/Z2a3hc9PCdcvPMR+Ks3swXAbtwAWt+yq8EZhL5nZLeEVAzCzTjP7dzN7wcweCS9N8+cE91pZFbYvCDfzmbDdK2Z2Qip/ZzK1KEhExuZ44EbgLcAJwBXAucDfA18huI7YeeEVcv8/4F/C9b4LHG9mlwE/Aj7l7t2H2MfXgCfCbdwLzAMwsxOBDxFcHXgZEAWuDNcpIrho4WkE95j4mrvfA6wBrnT3Ze7eE7ZtDtvdFNYtkhS61pbI2Lzp7q8AmNk64BF3dzN7BagFygiuLbaI4L4iOQDuHjOzjwF/AG5x9ycPs4/zCC6mh7v/XzMbuhbScuB04PnwWlwFBPd4geCeKj8Ln/+E4GqzhzK0bO3QfkSSQUEiMjZ9cc9jcdMxgn9H/wv4nbtfFt5gbHVc+0UEt2key5jFaBe/M+AOd//yUa4/ZKjmKPq3L0mkri2R5CgjuG84BAPdAJhZGUGX2HlAZTh+cSiPE3ZZmdm7gWnh/EeAP4+7DHiFmc0Pl2UBQ9u8guB+4QAdQEkCr0dkzBQkIsnxb8D/NrMngUjc/O8AP3D314FPADcMBcIovkFwO9QXCO5Hsg3A3dcD/0hwG94/ENzlbma4ThdwkpmtBS4A/jmcfztw84jBdpGU0GXkRSYwM+t09+J01yFTm45IREQkIToiETnGzOzjwLUjZj/p7p9ORz0iiVKQiIhIQtS1JSIiCVGQiIhIQhQkIiKSEAWJiIgkREEiIiIJ+X8KIDPUu7scOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x432 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_grid_search_results(gcv, ax, name):\n",
    "    ax.errorbar(\n",
    "        x=gcv.cv_results_[\"param_estimator__max_depth\"].filled(),\n",
    "        y=gcv.cv_results_[\"mean_test_score\"],\n",
    "        yerr=gcv.cv_results_[\"std_test_score\"],\n",
    "    )\n",
    "    ax.plot(\n",
    "        gcv.best_params_[\"estimator__max_depth\"],\n",
    "        gcv.best_score_,\n",
    "        \"ro\",\n",
    "    )\n",
    "    ax.set_ylabel(name)\n",
    "    ax.yaxis.grid(True)\n",
    "\n",
    "\n",
    "_, axs = plt.subplots(3, 1, figsize=(6, 6), sharex=True)\n",
    "axs[-1].set_xlabel(\"max_depth\")\n",
    "\n",
    "plot_grid_search_results(gcv_cindex, axs[0], \"c-index\")\n",
    "plot_grid_search_results(gcv_iauc, axs[1], \"iAUC\")\n",
    "plot_grid_search_results(gcv_ibs, axs[2], \"$-$IBS\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When optimizing for the concordance index, a high maximum depth works best, whereas the other metrics are best when choosing a maximum depth of 5 and 6, respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "I hope this notebook helped you to understand some of the pitfalls when estimating the performance of markers and models from right-censored survival data. We illustrated that [Harrell's estimator](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored) of the concordance index is biased when the amount of censoring is high, and that [Uno's estimator](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.concordance_index_ipcw.html#sksurv.metrics.concordance_index_ipcw) is more appropriate in this situation. Finally, we demonstrated that the [time-dependent area under the ROC](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.cumulative_dynamic_auc.html#sksurv.metrics.cumulative_dynamic_auc) is a very useful tool when we want to predict the occurrence of an event in a period up to time $t$ rather than at a specific time point $t$. Finally, the [integrated time-dependent Brier score](https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.metrics.integrated_brier_score.html#sksurv.metrics.integrated_brier_score) is an extension of the mean squared error that can measure discrimination *and* calibration."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}