tutorials/tutorial-08-tabular_classification_and_regression.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "import os\n",
    "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\";\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"\n",
    "\n",
    "import numpy as np\n",
    "import random\n",
    "import tensorflow as tf\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_columns', None)\n",
    "\n",
    "seed_value = 0\n",
    "os.environ['PYTHONHASHSEED']=str(seed_value)\n",
    "random.seed(seed_value)\n",
    "np.random.seed(seed_value)\n",
    "tf.random.set_seed(seed_value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ktrain\n",
    "from ktrain import tabular"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification and Regression on Tabular Data in `ktrain`\n",
    "\n",
    "As of v0.19.x, *ktrain* supports classification and regression on \"traditional\" tabular datasets. We will cover two examples in this notebook:\n",
    "- **Part I: Classification**:  predicting which [Titanic passengers survived](https://www.kaggle.com/c/titanic)\n",
    "- **Part II: Regression**:  predicting the age of people from [census data](http://archive.ics.uci.edu/ml/datasets/Census+Income)\n",
    "\n",
    "Let's begin with a demonstration of tabular classfication using the well-studied Titatnic dataset from Kaggle.\n",
    "\n",
    "## Part I:  Classification for Tabular Data\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solving the Titanic Kaggle Challenge in `ktrain`\n",
    "\n",
    "This notebook demonstrates using *ktrain* for predicting which passengers survived the Titatnic shipwreck.\n",
    "\n",
    "The dataset can be [downloaded from Kaggle here](https://www.kaggle.com/c/titanic/overview). There is a `train.csv` with labels (i.e., `Survived`) and a `test.csv` with no labels.  We will only use `train.csv` in this notebook.\n",
    "\n",
    "Let's begin by loading the data as a pandas DataFrame and inspecting it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = pd.read_csv('data/titanic/train.csv', index_col=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "\n",
       "                                                          Name     Sex   Age  \\\n",
       "PassengerId                                                                    \n",
       "1                                      Braund, Mr. Owen Harris    male  22.0   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "3                                       Heikkinen, Miss. Laina  female  26.0   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "5                                     Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "PassengerId                                                          \n",
       "1                1      0         A/5 21171   7.2500   NaN        S  \n",
       "2                1      0          PC 17599  71.2833   C85        C  \n",
       "3                0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "4                1      0            113803  53.1000  C123        S  \n",
       "5                0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll drop the `Name`, `Ticket`, `Cabin` columns, as they seem like they'll be less predictive.  These columns are largely unique or near-unique to passengers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = train_df.drop('Name', 1)\n",
    "train_df = train_df.drop('Ticket', 1)\n",
    "train_df = train_df.drop('Cabin', 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*ktrain* will automatically split out a validation set if given only a training set.  But, let's also manually split out a test set that we can evaluate later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(42)\n",
    "p = 0.1 # 10% for test set\n",
    "prop = 1-p\n",
    "df = train_df.copy()\n",
    "msk = np.random.rand(len(df)) < prop\n",
    "train_df = df[msk]\n",
    "test_df = df[~msk]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(799, 8)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(92, 8)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 1: Load and Preprocess the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing train: 717 rows x 8 columns\n",
      "\n",
      "The following integer column(s) are being treated as categorical variables:\n",
      "['Pclass', 'SibSp', 'Parch']\n",
      "To treat any of these column(s) as numerical, cast the column to float in DataFrame or CSV\n",
      " and re-run tabular_from* function.\n",
      "\n",
      "processing test: 82 rows x 8 columns\n"
     ]
    }
   ],
   "source": [
    "trn, val, preproc = tabular.tabular_from_df(train_df, label_columns=['Survived'], random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Automated Preprocessing\n",
    "*ktrain* automatically preprocesses the dataset appropriately.  Numerical columns are automatically normalized, missing values are handled, and categorical variables will be vectorized as [entity embeddings](https://arxiv.org/abs/1604.06737) for input to a neural network. \n",
    "\n",
    "##### Auto-generated Features\n",
    "*ktrain* will auto-generate some new features.  For instance, if `Age` is missing for a particular individual, an `Age_na=True` feature will be automatically added.\n",
    "\n",
    "New date features are also automatically added.  This dataset does not have any **date** fields.  If it did, we could populate the `date_columns` parameter to `tabular_from_df` in which case they would be used to auto-generate new features (e.g., `Day`, `Week`, `Is_month_start`, `Is_quarter_end`, etc.) using methods adapted from the **fastai** library.\n",
    "\n",
    "##### Manually-Engineered Features\n",
    "\n",
    "In addition to these auto-generated features, one can also optionally add manually-generated, dataset-specific features to `train_df` **prior** to invoking `tabular_from_df`. For instance, the `Cabin` feature we discarded earlier might be used to extract the **deck** associated with each passenger (e.g., **B22** --> **Deck B**)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 2: Create a Model and Wrap in `Learner`\n",
    "\n",
    "*ktrain* uses multilayer perceptrons as the model for tabular datasets. The model can be configured with arguments to `tabular_classifier` (e.g., number and size of hidden layers, dropout values, etc.), but we will leave the defaults here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mlp: a configurable multilayer perceptron with categorical variable embeddings [https://arxiv.org/abs/1604.06737]\n"
     ]
    }
   ],
   "source": [
    "tabular.print_tabular_classifiers()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Is Multi-Label? False\n",
      "done.\n"
     ]
    }
   ],
   "source": [
    "model = tabular.tabular_classifier('mlp', trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 3: Estimate the Learning Rate\n",
    "\n",
    "Based on the plot below, we will choose a learning rate of `1e-3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "simulating training for different learning rates... this may take a few moments...\n",
      "Train for 22 steps\n",
      "Epoch 1/5\n",
      "22/22 [==============================] - 1s 50ms/step - loss: 0.6882 - accuracy: 0.5985\n",
      "Epoch 2/5\n",
      "22/22 [==============================] - 0s 18ms/step - loss: 0.6819 - accuracy: 0.6263\n",
      "Epoch 3/5\n",
      "22/22 [==============================] - 0s 20ms/step - loss: 0.6495 - accuracy: 0.6584\n",
      "Epoch 4/5\n",
      "22/22 [==============================] - 0s 19ms/step - loss: 2.1039 - accuracy: 0.6569\n",
      "Epoch 5/5\n",
      " 3/22 [===>..........................] - ETA: 0s - loss: 25.0747 - accuracy: 0.5455\n",
      "\n",
      "done.\n",
      "Visually inspect loss plot and select learning rate associated with falling loss\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3xdVZn/8c+Te5uk16T3W3qjVC4CoUCxiBewooIOCEVAUKHCCOMw6vzAcRgG8eVtGGcUBCsgAgMVQbFCpaCACLbQAC290ZKmFNILSUsvuV/OeX5/nJ1ymp62Kc3O2Sf5vl+v8+rea699zrMSOE/WXnuvZe6OiIhIZ1npDkBERKJJCUJERFJSghARkZSUIEREJCUlCBERSUkJQkREUsoJ883NbDbwv0A2cKe7/6DT8XHAr4FBQZ3r3H2hmZ0B/ADIA1qBb7n70wf6rJKSEp8wYUL3N0JEpBd7+eWXt7l7aapjoSUIM8sGbgPOAKqBpWa2wN1XJ1X7DvCQu99uZtOBhcAEYBvwGXffbGZHAYuA0Qf6vAkTJlBRURFCS0REei8z27i/Y2FeYpoBVLp7lbu3AvOBczrVcWBAsD0Q2Azg7q+6++agfBXQz8zyQ4xVREQ6CTNBjAbeTtqvZt9ewI3AxWZWTaL3cE2K9zkXeMXdWzofMLO5ZlZhZhW1tbXdE7WIiADpH6S+ELjH3ccAZwH3mdmemMzsA8APga+mOtnd57l7ubuXl5amvIQmIiLvU5gJYhMwNml/TFCW7CvAQwDuvhgoAEoAzGwM8Hvgi+6+PsQ4RUQkhTATxFJgipmVmVkeMAdY0KnOW8DHAMzsSBIJotbMBgGPk7ir6YUQYxQRkf0ILUG4eztwNYk7kNaQuFtplZndZGZnB9W+AVxhZsuBB4HLPDG97NXAZOAGM1sWvIaFFauIiOzLest03+Xl5a7bXEWkr1mzZTet7XGOHTvofZ1vZi+7e3mqY+kepBYRkfepsqaei+98kW/+djmxePf/sa8EISKSgd7a3shFdy7BDO645ASys6zbPyPUqTZERKT7bd7ZxBfuXEJLe5z5c09mUmlRKJ+jHoSISAaprWvh4jtfZFdjG/d+eQbTRgw4+Envk3oQIiIZoqGlnUvvfoktu5q57yszOGbM+xuY7ir1IEREMkA87vzzb5bx+tbd/Pzi4ymfMCT0z1SCEBHJAD9atJanVr/Dv396Oh85omceC1OCEBGJuIdfruaOv67nopPGcdnMCT32uRqDEBGJkO8+tprXt+6mrKSQspIi+udl8x9/WMXMSUO58ewPYNb9t7PujxKEiEiE/P7VTcTdWVG9i93N7QCUlRTy84uOJze7Zy/6KEGIiERIU2uMS04Zz/WfnMaOxjbe3N7AlGFFFBfk9ngsShAiIhERjztNbTEKcrMxM4YU5jGkMC9t8WiQWkQkIlra4wD0y81OcyQJShAiIhHR2JoYc+ifpwQhIiJJmtpigHoQIiLSSXNHgugLPQgzm21ma82s0syuS3F8nJk9Y2avmtlrZnZW0rHrg/PWmtknwoxTRCQKGluj1YMI7S4mM8sGbgPOAKqBpWa2wN1XJ1X7DomlSG83s+nAQmBCsD0H+AAwCvizmU1191hY8YqIpFtTkCD6whjEDKDS3avcvRWYD5zTqY4DHXPVDgQ2B9vnAPPdvcXdNwCVwfuJiPRajcElpoI+kCBGA28n7VcHZcluBC42s2oSvYdrDuFczGyumVWYWUVtbW13xS0ikhbNEbvElO5B6guBe9x9DHAWcJ+ZdTkmd5/n7uXuXl5aWhpakCIiPaExYpeYwnySehMwNml/TFCW7CvAbAB3X2xmBUBJF88VEelV+tJtrkuBKWZWZmZ5JAadF3Sq8xbwMQAzOxIoAGqDenPMLN/MyoApwEshxioiknZRu801tB6Eu7eb2dXAIiAbuNvdV5nZTUCFuy8AvgH80syuJTFgfZm7O7DKzB4CVgPtwNd0B5OI9HYdl5gKItKDCHWyPndfSGLwObnshqTt1cCp+zn3e8D3woxPRCRKmtpi5GZbj0/rvT/RiEJERGhqjUVm/AGUIEREIqOpNRaZ8QdQghARiYymthj986KzTI8ShIhIRDS2xiIzQA1KECIikdHcFovMQ3KgBCEiEhmNre0apBYRkX01tcV1iUlERPbV1NquS0wiIrKvpjY9ByEiIinoOQgREUmpqU0JQkREOmmLxWmLuS4xiYjI3jrWgtAgtYiI7KU5YlN9gxKEiEgkqAchIiIpdSwW1GfGIMxstpmtNbNKM7suxfGfmNmy4LXOzHYmHfuRma0yszVm9lMzszBjFRFJp6aILTcKIa4oZ2bZwG3AGUA1sNTMFgSryAHg7tcm1b8GOC7YnklipbljgsPPAx8Gng0rXhGRdGrqYz2IGUClu1e5eyswHzjnAPUvBB4Mth0oAPKAfCAXeCfEWEVE0mpPgohQDyLMBDEaeDtpvzoo24eZjQfKgKcB3H0x8AywJXgtcvc1Kc6ba2YVZlZRW1vbzeGLiPQcDVLv3xzgYXePAZjZZOBIYAyJpPJRM5vV+SR3n+fu5e5eXlpa2qMBi4h0p6Y+dpvrJmBs0v6YoCyVObx3eQngc8ASd69393rgT8ApoUQpIhIB7/Ug+saSo0uBKWZWZmZ5JJLAgs6VzGwaMBhYnFT8FvBhM8sxs1wSA9T7XGISEekt+tRtru7eDlwNLCLx5f6Qu68ys5vM7OykqnOA+e7uSWUPA+uBFcByYLm7/zGsWEVE0q2jB1GQG5Ur/yHe5grg7guBhZ3Kbui0f2OK82LAV8OMTUQkSpqC5Uaj9MhXdFKViEgfFrWpvkEJQkQkEppa45EafwAlCBGRSGhqa1cPQkRE9tXUGovUQ3KgBCEiEgmNrbFIPSQHShAiIpHQ3BbTGISIiOyrqU2XmEREJIXGVvUgREQkhWY9ByEiIqmoByEiIvtwd41BiIjIvlra47hDgRKEiIgki+J61KAEISKSdlFcbhSUIERE0q4xgsuNQsgJwsxmm9laM6s0s+tSHP+JmS0LXuvMbGfSsXFm9qSZrTGz1WY2IcxYRUTSpTmCy41CiAsGmVk2cBtwBlANLDWzBe6+uqOOu1+bVP8a4Likt7gX+J67P2VmRUA8rFhFRNIpisuNQrg9iBlApbtXuXsrMB845wD1LwQeBDCz6UCOuz8F4O717t4YYqwiImnTMQbRLy9aV/3DjGY08HbSfnVQtg8zGw+UAU8HRVOBnWb2OzN71cx+HPRIOp8318wqzKyitra2m8MXEekZ793FFK1LTFFJV3OAh4O1qCFx6WsW8E3gRGAicFnnk9x9nruXu3t5aWlpT8UqItKtmtraAfrUVBubgLFJ+2OCslTmEFxeClQDy4LLU+3Ao8DxoUQpIpJmTa2JIda+dJvrUmCKmZWZWR6JJLCgcyUzmwYMBhZ3OneQmXV0Cz4KrO58rohIb9DYmuhB9JnbXIO//K8GFgFrgIfcfZWZ3WRmZydVnQPMd3dPOjdG4vLSX8xsBWDAL8OKVUQknZoj+qBcqCMi7r4QWNip7IZO+zfu59yngGNCC05EJCIaW2PkZBm52VEZFk6IVjQiIn1QUwSXGwUlCBGRtIviYkGgBCEiknaNrUoQIiKSQlMEV5MDJQgRkbRr0iUmERFJRT0IERFJKYrrUYMShIhI2jW1xiL3FDUoQYiIpJ16ECIiklKjxiBERCSVxF1M0VoLApQgRETSKhZ3Wtvj6kGIiMjeorrcKChBiIik1Z7lRnWJSUREkr23HrUuMYmISJKmiC4WBCEnCDObbWZrzazSzK5LcfwnZrYseK0zs52djg8ws2ozuzXMOEVE0qVjudEo9iBCu+hlZtnAbcAZQDWw1MwWuPuetaXd/dqk+tcAx3V6m+8Cz4UVo4hIunX0IPrak9QzgEp3r3L3VmA+cM4B6l8IPNixY2YnAMOBJ0OMUUQkraK6HjV0MUGY2deDyz1mZneZ2StmduZBThsNvJ20Xx2UpXr/8UAZ8HSwnwXcAnzzIHHNNbMKM6uora3tSlNERCKlcc9dTBmaIIAvu/tu4ExgMHAJ8INujGMO8LC7x4L9fwQWunv1gU5y93nuXu7u5aWlpd0YjohIz4jyXUxdHYOw4N+zgPvcfZWZ2YFOADYBY5P2xwRlqcwBvpa0fwowy8z+ESgC8sys3t33GegWEclk7z0ol7kJ4mUze5LEZaDrzawYiB/knKXAFDMrI5EY5gBf6FzJzKaR6JUs7ihz94uSjl8GlCs5iEhv1NGDiOIYRFcTxFeADwJV7t5oZkOALx3oBHdvN7OrgUVANnB30PO4Cahw9wVB1TnAfHf399cEEZHM1TEGUZCTuQniFGCZuzeY2cXA8cD/Huwkd18ILOxUdkOn/RsP8h73APd0MU4RkYzS3BYjPyeLrKyDXbXveV0dpL4daDSzY4FvAOuBe0OLSkSkj4jqYkHQ9QTRHlwCOge41d1vA4rDC0tEpG+I6mJB0PVLTHVmdj2J21tnBc8p5IYXlohI35BYLCiaCaKrPYgLgBYSz0NsJXHL6o9Di0pEpI9oas3wBBEkhf8DBprZp4Fmd9cYhIjIYWpqjdE/N3prQUDXp9o4H3gJ+DxwPvCimZ0XZmAiIn1BU1uMgoj2ILqatv4NONHdawDMrBT4M/BwWIGJiPQFTa0xhg/IT3cYKXV1DCKrIzkEth/CuSIish+J21yjeYmpq1E9YWaLeG867gvo9ACciIgcusbWWCTXgoAuJgh3/5aZnQucGhTNc/ffhxeWiEjf0NTaHtkH5brcr3H3R4BHQoxFRKRPaW6L0dAaY0hhXrpDSemACcLM6oBUk+gZ4O4+IJSoRET6gJrdLQCUFkdzkPqACcLdNZ2GiEhIauqaARgW0QShO5FERNKkpi7RgxhWXJDmSFJTghARSZOa3UEPIsOfgxARkW5WW99CTpYxpH80B6lDTRBmNtvM1ppZpZnts2Somf3EzJYFr3VmtjMo/6CZLTazVWb2mpldEGacIiLpULO7hZKi/EguFgSHcJvroTKzbOA24AygGlhqZgvcfXVHHXe/Nqn+NcBxwW4j8EV3f8PMRpFYE3uRu+8MK14RkZ5WU9cS2ctLEG4PYgZQ6e5V7t4KzCex4ND+XEjwpLa7r3P3N4LtzUANUBpirCIiPa6mriWydzBBuAliNPB20n51ULYPMxsPlAFPpzg2A8gjscxp52NzzazCzCpqa2u7JWgRkZ5SW9dMaUTvYILoDFLPAR5291hyoZmNBO4DvuTu8c4nufs8dy939/LSUnUwRCRztMfibG9ojexDchBugtgEjE3aHxOUpTKH9yYCBMDMBgCPA//m7ktCiVBEJE221bfiHt2H5CDcBLEUmGJmZWaWRyIJLOhcycymAYOBxUllecDvgXvdXWtOiEivE/WnqCHEBOHu7cDVwCJgDfCQu68ys5vM7OykqnOA+e6ePOfT+cBpwGVJt8F+MKxYRUR6Wsc8TMMGRHcMItRVKtx9IZ3WjXD3Gzrt35jivPuB+8OMTUQknWrrO6bZ6IM9CBER2b+OHkRJkRKEiIgkqalrZkhhHnk50f0ajm5kIiK9WNQfkgMlCBGRtKipa4n0MxCgBCEikha1u5uVIEREZG/uTm19S2QXCuqgBCEi0sN2NLbRFnONQYiIyN72PEUd4am+QQlCRKTH1UZ8LeoOShAiIj1szzQbusQkIiLJajp6ELrEJCIiyWrqminKz6F/XqjT4R02JQgRkR6WCU9RgxKEiEiPq93dQokShIiIdFZT16wehJnNNrO1ZlZpZtelOP6TpAWB1pnZzqRjl5rZG8Hr0jDjFBHpSYlLTNG+xRVCXDDIzLKB24AzgGpgqZktcPfVHXXc/dqk+tcAxwXbQ4D/AMoBB14Ozt0RVrwiIj2hvqWdxtZY5O9ggnB7EDOASnevcvdWYD5wzgHqXwg8GGx/AnjK3d8NksJTwOwQYxUR6RHvPSTXtxPEaODtpP3qoGwfZjYeKAOePtRzRUQySc3uYJqNDLjEFJVB6jnAw+4eO5STzGyumVWYWUVtbW1IoYmIdJ9MeUgOwk0Qm4CxSftjgrJU5vDe5aUun+vu89y93N3LS0tLDzNcEZHw1egSEwBLgSlmVmZmeSSSwILOlcxsGjAYWJxUvAg408wGm9lg4MygTEQko9XUNZOXncXAfrnpDuWgQruLyd3bzexqEl/s2cDd7r7KzG4CKty9I1nMAea7uyed+66ZfZdEkgG4yd3fDStWEZGeUrs7sdSomaU7lIMKdSIQd18ILOxUdkOn/Rv3c+7dwN2hBScikgaZsBZ1h6gMUouI9AmZ8hQ1KEGIiPSomrqWjLiDCZQgRER6THNbjJ2NbQzPgGcgQAlCRKTHVNU2AFBWWpjmSLpGCUJEpIdUbasHYGJJUZoj6RolCBGRHrK+pgEzKCtRD6LP2LCtgbffbUx3GCIScVXb6hk1sB/98rLTHUqXRHtB1B7g7vz0L5V8+IhSjh0zsMsPr7S0x3hi5Vb+b8lbvPRm4hm+SaWFfOSIYXx02jAK8rJZuWkXr1XvYuWmXWRnGSdPHMopE4cyY+IQBhTs/ynKeNyJu5OTrfwt0pusr61n0rDMuLwEShC8/W4Ttz1TyU/+vI7xQ/tzzrGj+MyxoygqyKFmdws1dS3U1rVQ19xGY2uMprYYuxrb+Mvr77CtvpXxQ/vz7bOmkZudxdOv13Dv4o3c+fyGPe8/tDCPo0YPpLU9zn1LNnLX8xvIMhg7pD9F+TkU5udQmJeNA9vqE5+1rb4VAyYPK2LaiGKmjRzA2MH9aYvFaW6L0dIeJ+5OcUEuAwpyGNAvl4H9chlWnM/g/nlkZe2b5Nyd5rY4u5vb2N3URkNrjOKCHIYW5jGwX+6exNgei1PX3E59SzvFBTl7HeuK1vY4b73byMbtDbTFnAEFORQV5FCUn0NOVhbt8TixuNMeJMFkAwpyGT2oX8r4m9ti7GhsJcuMLDNysox+edkU5GbGX2Ii7k5VbQPl44ekO5Qu6/MJYtzQ/iz9zsdZtGorC5Zt5tZnKvnp05X7rd8vN5v+edkcP34wl5w8ng9NLtnzhfalU8toaGln8frtxNw5evRARg4s2PMF29wW49W3drK4ajsbtjXQ2JL4It5W34rjlBblM33kAEqL84k7rN1ax4sb3uXRZZu73J7cbKO0KJ/BhXm0tMdpbGmnoTVGY2s7bTFPeU5OljGwXy7NbTEaWmP7vF9JUT4lRfmUFudTUpRHSVE+Q4vyaW6Lsb2+lW31LWyrb+HtHY1s2tFEPPXHdEm/3GwmDytiyrAi+udn8+a2RjZsa2DzriY65RPMYMzgfkwZVsyU4UVMHVbMUaMHMqm0UL0viZytu5tpbI1lVA/CvPP/dRmqvLzcKyoqDvt9anY38+c1NUBitsXS4DWwXy79crNT/nUbtp2NrWzZ1Ux+ThYFudnk52SRZUZ9Szu7mhI9gh2NbdTWNfNOXQs1u1vY0di6J5n1z8umf34OAwpyGdAv8W//vGzqmtvZVt/Cuw2t7Gxqo19u9p46hfk51DW3Bz2aRE9qW10L2xta2F7fSnuQBQrzsikpzmdIYR5jBvenbGh/JpQUMqGkkPycrERvpLmdupY2YvFEMsrJTvQAOvdMtte3UllTzxs1dbzxTj1NbTEmlBRSNrQ/ZSVFDBuQT9ydeNyJxZ1dTe1U1tbzxjt1VNU20BqLA1CQm8WRIwdwzOiBzJpSyszJQ+mf1+f/FpI0e6FyGxfd+SIPXH4SMyeXpDucPczsZXcvT3VM/9d0MmxAAV84aVy6w9jLoP55DOqft0/54MK8veZE7ynxuLO7uY38nOzIDLa1x+Js2NbAys27WLlpNys37eKhimp+vXgjedlZzCgbwimThiYudQUJqjA/h1mTSxnYP/qzakrmq6pN3OKaST0IJQg5ZFlZljJhpVNOdhZThhczZXgxnzsuUdbSHqPizR08u7aGZ9fW8uNFa/c5Ly8nizOnD+e8E8Ywa0op2WnoIUrfsL62gcK87IyZhwmUIKQXy8/J5tTJJZw6uYR/+1RisfjW9jjtsThtceed3c0sWLaZR5dt4rHXtlBanM+xYwYxdXgRU4cXc8SIYqaNKM6IaZkl+tbX1jOxtCij/ntSgpA+oyg/B5L+eBs9qB/HjxvM9WdN45nXa1i4Yiuvb93Ns2tr9oyxTB1exGUzy/jccaMjczlNMlNVbQMnThic7jAOiRKE9Hn5OdnMPmoks48aCSRu1X1zewOvbNzBfUs28u3fr+CHT7zOBSeO5YTxgxk9qB9jBvc75FuApe9qao2xaWcTF5SmY9Tw/Qs1QZjZbOB/Sawod6e7/yBFnfOBGwEHlrv7F4LyHwGfIvG091PA17233HIlkZaXk8XU4cVMHV7MBSeOpWLjDn71wgbu/FsV8557r15hXjaf+MAIrjhtIkeOHJC+gCXyOuZgmlSaOQPUEGKCMLNs4DbgDKAaWGpmC9x9dVKdKcD1wKnuvsPMhgXlM4FTgWOCqs8DHwaeDStekVTMjBMnDOHECUPY1djGxncb2LyzieodTVTW1LNg+WZ+9+omTptayldPm8jMSUPVq5B9dMziOjFDZnHtEGYPYgZQ6e5VAGY2HzgHWJ1U5wrgNnffAeDuNUG5AwVAHmBALvBOiLGKHNTA/rkc038Qx4wZtKfsuk9O4/4lG7nn729y0Z0vMnJgAbOmlDBrSikfmlzC4MJo3e0l6bG+tj6jJunrEGaCGA28nbRfDZzUqc5UADN7gcRlqBvd/Ql3X2xmzwBbSCSIW919TecPMLO5wFyAceOi9eyC9A2D+udx9UencPmsiTz22haefv0dnli5lYcqqjGD48YO4uPTh3Pm9OFMyrA7WKT7VNU2MHpQv4ybGibdg9Q5wBTgdGAM8JyZHQ2UAEcGZQBPmdksd/9b8snuPg+YB4knqXsqaJHOCnKzOe+EMZx3whjaY3Fe27SLv66t5enXa/jRE2v50RNrmTC0P2MG96e5LTGnV3NbjA9NLuH6s47MuC8OOTTra+szbvwBwk0Qm2CvB33HBGXJqoEX3b0N2GBm63gvYSxx93oAM/sTcArwN0QiLic7i+PHDeb4cYO59oypbNnVxJ/X1PD0mnfY1dRGQW42A/vlEnfn14s3sqx6F7+4+ARGDMyMZSjl0Lg7G7Y1MKMscybp6xBmglgKTDGzMhKJYQ7whU51HgUuBH5lZiUkLjlVAROBK8zs+yQuMX0Y+J8QYxUJzciB/bjk5PFccvL4fY49sXIr33hoGZ/+2fPcfvHxnDgh875E5MA6JumbmIE9iNCmvHT3duBqYBGwBnjI3VeZ2U1mdnZQbRGw3cxWA88A33L37cDDwHpgBbCcxO2vfwwrVpF0mX3UCB792qkUF+Rw4bwl3PHX9TS3xQ5+omSM9TWJO5gmZdgdTKDZXEUiYVdTG9/87XKeWv0OpcX5XPXhSXzhpHEam+gF7l38Jjf8YRUvfvtjDB8QvcuIB5rNVZPmi0TAwH65/PKL5cyfezKTS4u46bHVnPajZ/jdK9XpDk0O0/qaeoryczJqkr4O6b6LSUSSnDxxKCfPHcri9dv58aLX+ZeHlhN3OO+EMQc/WSKpalsDE0sLM/IWZ/UgRCLolElDeXDuyZw6eSj/75HXeGq1nhPNVOtrMvMWV1CCEIms/JxsfnFJOUeNGsDXHniFJVXb0x2SHKKaumY272pmYoY9Qd1BCUIkworyc7jnSzMYN6Q/l/+6gpWbdoXyOXXNbdzx1/V84ZdLeGjp27QHy7fK4bll0Tpys41PHTMy3aG8L7qLSSQDbNnVxHm3L6a2voXzThjDladNYtzQ/of9vrV1LfzqhQ3ct2Qjdc3tjBxYwJZdzUwsLeQbZxzBJ48aQVaWsa2+hdWbd/NGTT0G5OdmkZ+TTWFeNqdNLaUwf9/hzE07m/jPBauYMryIr3xoIkP62LxUK6p3cfZtz3PFrIl8+6wj0x3Ofh3oLiYlCJEMsWVXEz97upKHK6ppj8f5zLGjuGLWRD4wasAhD4C6O/cv2cjNj6+hNRbnk0eN4MoPT+Lo0QN5cvU73PLkWta9U09ZSSENLe3U1LXs971GD+rH9//haE6bWrqn7Jm1NVz7m2U0t8VoaY/TLzebS04ez+WzJlKagXfzHCp357w7FrNxewNPf/N0BhREd91zJQiRXuSd3c3c9fwG7l+yMfGEbkkhnzx6BJ88amSXkkVzW4x/f3Qlv325mtOPKOXfPz19n0HUWNxZsHwTv62oZuTAfkwfNYDpIwdwxIhiss1oaU988W/Y1sCNf1xFVW0D550whm+fdSS/emEDP3u6kmkjirn94hNoj8W57ZlKFizfTF5OFp8/YSxfOnXCXk8Wt8fiPL5iC3c9vwEz41NHj+BTx4xi9KB+ofwMw/aHZZv4+vxl/OjcYzj/xGgvEqQEIdIL7Wxs5fEVW/jTiq0srtpOLO4MH5DPESMGMHVYEVOGFzGptIgJJYUMLczDzNiyq4kr73uZ5dW7+KePTeGfPzaFrKzDu/2yuS3GT//yBr94roosg7aYc375GG4656i9HvTbsK2Bnz9TyR+WbaYtHudj04Zx2cwy3tzewLznqnjr3UamDCuiX142r1UnxlqOGzeIqz48iTM/MOKwYuxJja3tfPS//kppcT5/+Nqph/3zDZsShEgvt6OhlSdXb2Xx+u1U1tZTWVNPc9t7A82FedmMH1rI1t3NtLbHueX8Y/lEN3/prty0i588tY7ZR43g8+X7/6u5pq6Z+5e8xf8t2cj2hlYAjh07iK+dPomPHzmcrCxj4/YGHnttC4+8Us3G7Y3c+cVyPjJtWLfG213cHXcwSyww9V+L1nLrM5U8ctUpnDA++nNrKUGI9DHxuLNpZ2LVu43bG3hzeyNvvdtIe9y54dNHMnlYcbpDpLktxpOr36G0KJ+TJw5JeWmsvqWdOfMWs76mgQfnnswHxw5K8U7psauxjZ//tZJf//3NvZIxwGc/OIr/mXNcmiI7NEoQIpKxauqaOff2v9PQEuN3V81kQpqfKWhui3Hf4o3c+kwlu5vb+Mwxo5hYWoh7sOKu90AAAA1LSURBVBRmbhYXzRjPwP7RHZhOpgQhIhltw7YGzr397xTl5/DIVTPTdifU+tp6Lr37Jap3NHHa1FKumz2N6aMGpCWW7qLJ+kQko5WVFHL3ZSdSW9fCV++rIBbv+T9sdzW2ccWvK2hqjXH/V07i3i/PyPjkcDBKECKSET44dhA/OPdoXnlrJ3c9XxXa57SleIq8PRbnmvmv8vaORu645AQ+NKUktM+PEiUIEckYZx87ijOnD+eWJ9exvra+29//7uc3MP2GJ/j3R1eyrf69hwN/8KfXeW5dLTd/9qg+tepfqAnCzGab2VozqzSz6/ZT53wzW21mq8zsgaTycWb2pJmtCY5PCDNWEYk+M+PmzyWer/jXh1/r1ktNf11Xy82Pr2b80EIeeOktTv/xs9z69Bvcv2Qjdz6/gctmTuCCE8d12+dlgtDWgzCzbOA24AygGlhqZgvcfXVSnSnA9cCp7r7DzJJvdL4X+J67P2VmRYBmDxMRhhUXcOPZ07n2N8v51QsbuHzWxMN+zw3bGrjmgVeYOryYR66aydbdzfzwT6/zX0+uA+DUyUP5zqeiO59SWMJcMGgGUOnuVQBmNh84B1idVOcK4DZ33wHg7jVB3elAjrs/FZR3f19SRDLWZz84msdf28KPF63lY0cOp+wwbn3d3dzG5b9eSk52Fr/8YjmF+TlMKi1i3hfLeWnDu/xp5Rb+6aNTyMnue1fkw0wQo4G3k/argZM61ZkKYGYvANnAje7+RFC+08x+B5QBfwauc/e9VnM3s7nAXIBx4/pW10+kLzMzvve5oznjv//K5+9YzLFjBjKxtJCJpUWcVDZkr3mekr1QuY1fvbCBUYP6Mak0MRXJXc9XsXF7I/dffhJjh+w9Q+6MsiHMKOs7Yw6dpXvJ0RxgCnA6MAZ4zsyODspnAccBbwG/AS4D7ko+2d3nAfMg8RxETwUtIuk3fEABd1xyAvct3khVbQN/q9xGa3uc/JwsHpx7MsePG7xX/fW19Vx538vk5WTxYtW71LW07zl282eP4uSJQ3u6CZEXZoLYBCRPyDImKEtWDbzo7m3ABjNbRyJhVAPLki5PPQqcTKcEISJ928xJJcyclLjlNBZ33tzewJfvWcrlv67g9/84k/FDE5ee6lvaufK+l8nNyWLBNR9i1MACautaqKytJ8tMyWE/wryothSYYmZlZpYHzAEWdKrzKIneA2ZWQuLSUlVw7iAz65hg/qPsPXYhIrKX7CxjUmkR93xpBu7OZb9ayrsNrbg7//rwctbX1nPrhccxelA/zIxhAwqYOalEyeEAQksQ7t4OXA0sAtYAD7n7KjO7yczODqotArab2WrgGeBb7r49GGv4JvAXM1sBGPDLsGIVkd6jrKSQOy8tZ9POJq64t4Kf/qWShSu2cv0nj2Tm5L7xgFt30VxMItIrLVyxha898Aru8OljRvKzC4875JX3+oIDzcWU7kFqEZFQnHX0SG7+7FE883oNPzz3GCWH90EJQkR6rYtOGs9FJ41PdxgZq+89+SEiIl2iBCEiIikpQYiISEpKECIikpIShIiIpKQEISIiKSlBiIhISkoQIiKSUq+ZasPMaoGNwEBg136qpTp2sLKubJcA295X4AeO41DqdbX8QPthte9w23agY4fbvij87vZ3rCtlvbl9XWlrb2lfOr9bxrt7acoj7t6rXsC8Qzl2sLIubleEGXdX6nW1/ED7YbXvcNsWZvui8Lvb37GulPXm9nWlrb2lfVH4bkn16o2XmP54iMcOVtaV7e7Q1ffbX72ulh9oP6z2HW7bDnSsN7evK2W9uX1dbevhikL7ovDdso9ec4kpncyswvczG2Jv0Jvb15vbBmpfpkt3+3pjDyId5qU7gJD15vb15raB2pfp0to+9SBERCQl9SBERCQlJQgREUlJCUJERFJSghARkZSUIEJmZllm9j0z+5mZXZrueLqTmZ1uZn8zszvM7PR0xxMGMys0swoz+3S6Y+luZnZk8Lt72MyuSnc83c3MPmtmvzSz35jZmemOp7uZ2UQzu8vMHg7rM5QgDsDM7jazGjNb2al8tpmtNbNKM7vuIG9zDjAGaAOqw4r1UHVT2xyoBwqIUNug29oH8P+Ah8KJ8v3rjva5+xp3vxI4Hzg1zHgPVTe171F3vwK4ErggzHgPVTe1r8rdvxJqnLrNdf/M7DQSX4D3uvtRQVk2sA44g8SX4lLgQiAb+H6nt/hy8Nrh7r8ws4fd/byeiv9Auqlt29w9bmbDgf9294t6Kv6D6ab2HQsMJZEAt7n7Yz0T/cF1R/vcvcbMzgauAu5z9wd6Kv6D6a72BefdAvyfu7/SQ+EfVDe3L7TvlZww3rS3cPfnzGxCp+IZQKW7VwGY2XzgHHf/PrDPZQgzqwZag91YeNEemu5oW5IdQH4Ycb5f3fS7Ox0oBKYDTWa20N3jYcbdVd31+3P3BcACM3sciEyC6KbfnwE/AP4UpeQA3f7/X2iUIA7daODtpP1q4KQD1P8d8DMzmwU8F2Zg3eCQ2mZm/wB8AhgE3BpuaN3ikNrn7v8GYGaXEfSWQo3u8B3q7+904B9IJPeFoUbWPQ71/71rgI8DA81ssrvfEWZw3eBQf39Dge8Bx5nZ9UEi6VZKECFz90Yg1OuE6eLuvyORAHs1d78n3TGEwd2fBZ5NcxihcfefAj9NdxxhcfftJMZXQqNB6kO3CRibtD8mKOsNenPbQO3LdGpfD1OCOHRLgSlmVmZmecAcYEGaY+ouvbltoPZlOrWvhylBHICZPQgsBo4ws2oz+4q7twNXA4uANcBD7r4qnXG+H725baD2ofZFWqa0T7e5iohISupBiIhISkoQIiKSkhKEiIikpAQhIiIpKUGIiEhKShAiIpKSEoSkjZnV98BnnN3Fab278zNPN7OZ7+O848zsrmD7MjOLxPxWZjah87TUKeqUmtkTPRWT9AwlCMl4wTTJKbn7Anf/QQifeaB5zE4HDjlBAN8mQ+cOcvdaYIuZRWpdCTk8ShASCWb2LTNbamavmdl/JpU/amYvm9kqM5ubVF5vZreY2XLgFDN708z+08xeMbMVZjYtqLfnL3Ezu8fMfmpmfzezKjM7LyjPMrOfm9nrZvaUmS3sONYpxmfN7H/MrAL4upl9xsxeNLNXzezPZjY8mML5SuBaM1tmZrOCv64fCdq3NNWXqJkVA8e4+/IUxyaY2dPBz+YvZjYuKJ9kZkuC9t6cqkdmiRXxHjez5Wa20swuCMpPDH4Oy83sJTMrDj7nb8HP8JVUvSAzyzazHyf9rr6adPhRIDJrgkg3cHe99ErLC6gP/j0TmAcYiT9aHgNOC44NCf7tB6wEhgb7Dpyf9F5vAtcE2/8I3BlsXwbcGmzfA/w2+IzpJObeBziPxHTXWcAIEutbnJci3meBnyftD+a92QguB24Jtm8EvplU7wHgQ8H2OGBNivf+CPBI0n5y3H8ELg22vww8Gmw/BlwYbF/Z8fPs9L7nAr9M2h8I5AFVwIlB2QASMzv3BwqCsilARbA9AVgZbM8FvhNs5wMVQFmwPxpYke7/rvTqvpem+5YoODN4vRrsF5H4gnoO+Ccz+1xQPjYo305i8aVHOr1Px9TjL5NY5yCVRz2xrsNqS6yEB/Ah4LdB+VYze+YAsf4maXsM8BszG0niS3fDfs75ODDdzDr2B5hZkbsn/8U/Eqjdz/mnJLXnPuBHSeWfDbYfAP4rxbkrgFvM7IfAY+7+NzM7Gtji7ksB3H03JHobwK1m9kESP9+pKd7vTOCYpB7WQBK/kw1ADTBqP22QDKQEIVFgwPfd/Rd7FSYWtPk4cIq7N5rZsySW/wRodvfOK/S1BP/G2P9/2y1J27afOgfSkLT9MxJLrS4IYr1xP+dkASe7e/MB3reJ99rWbdx9nZkdD5wF3GxmfwF+v5/q1wLvkFhqNQtIFa+R6KktSnGsgEQ7pJfQGIREwSLgy2ZWBGBmo81sGIm/TncEyWEacHJIn/8CcG4wFjGcxCBzVwzkvfn6L00qrwOKk/afJLG6GQDBX+idrQEm7+dz/k5i6mdIXOP/W7C9hMQlJJKO78XMRgGN7n4/8GPgeGAtMNLMTgzqFAeD7gNJ9CziwCUk1kLubBFwlZnlBudODXoekOhxHPBuJ8ksShCSdu7+JIlLJIvNbAXwMIkv2CeAHDNbQ2Jt4SUhhfAIieUdVwP3A68Au7pw3o3Ab83sZWBbUvkfgc91DFID/wSUB4O6q0mxCpi7v05iaczizsdIJJcvmdlrJL64vx6U/zPwL0H55P3EfDTwkpktA/4DuNndW4ELSCyFuxx4isRf/z8HLg3KprF3b6nDnSR+Tq8Et77+gvd6ax8BHk9xjmQoTfctAnSMCVhind+XgFPdfWsPx3AtUOfud3axfn+gyd3dzOaQGLA+J9QgDxzPc8A57r4jXTFI99IYhEjCY2Y2iMRg83d7OjkEbgc+fwj1TyAxqGzAThJ3OKWFmZWSGI9RcuhF1IMQEZGUNAYhIiIpKUGIiEhKShAiIpKSEoSIiKSkBCEiIin9f02+erH2M6voAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find(show_plot=True, max_epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 4: Train the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "begin training using onecycle policy with max lr of 0.005...\n",
      "Train for 23 steps, validate for 3 steps\n",
      "Epoch 1/10\n",
      "23/23 [==============================] - 1s 58ms/step - loss: 0.6388 - accuracy: 0.6597 - val_loss: 0.5437 - val_accuracy: 0.7561\n",
      "Epoch 2/10\n",
      "23/23 [==============================] - 1s 23ms/step - loss: 0.5855 - accuracy: 0.6876 - val_loss: 0.4851 - val_accuracy: 0.7805\n",
      "Epoch 3/10\n",
      "23/23 [==============================] - 0s 22ms/step - loss: 0.5259 - accuracy: 0.7448 - val_loss: 0.4044 - val_accuracy: 0.8659\n",
      "Epoch 4/10\n",
      "23/23 [==============================] - 1s 23ms/step - loss: 0.4985 - accuracy: 0.7713 - val_loss: 0.3639 - val_accuracy: 0.8902\n",
      "Epoch 5/10\n",
      "23/23 [==============================] - 1s 22ms/step - loss: 0.4762 - accuracy: 0.7894 - val_loss: 0.3364 - val_accuracy: 0.8659\n",
      "Epoch 6/10\n",
      "23/23 [==============================] - 1s 23ms/step - loss: 0.4626 - accuracy: 0.7908 - val_loss: 0.3174 - val_accuracy: 0.9146\n",
      "Epoch 7/10\n",
      "23/23 [==============================] - 1s 24ms/step - loss: 0.4444 - accuracy: 0.8061 - val_loss: 0.3126 - val_accuracy: 0.9024\n",
      "Epoch 8/10\n",
      "23/23 [==============================] - 1s 23ms/step - loss: 0.4279 - accuracy: 0.8159 - val_loss: 0.2599 - val_accuracy: 0.9146\n",
      "Epoch 9/10\n",
      "23/23 [==============================] - 1s 25ms/step - loss: 0.4030 - accuracy: 0.8243 - val_loss: 0.2721 - val_accuracy: 0.9024\n",
      "Epoch 10/10\n",
      "23/23 [==============================] - 1s 23ms/step - loss: 0.3990 - accuracy: 0.8257 - val_loss: 0.2686 - val_accuracy: 0.9024\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f67e009cb70>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.fit_onecycle(5e-3, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we don't appear to be quite overfitting yet, we could try to train further.  But, we will stop here.\n",
    "\n",
    "\n",
    "**Let's evaluate the validation set:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "not_Survived       0.89      0.96      0.92        49\n",
      "    Survived       0.93      0.82      0.87        33\n",
      "\n",
      "    accuracy                           0.90        82\n",
      "   macro avg       0.91      0.89      0.90        82\n",
      "weighted avg       0.90      0.90      0.90        82\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[47,  2],\n",
       "       [ 6, 27]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.evaluate(val, class_names=preproc.get_classes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make Predictions\n",
    "\n",
    "The `Predictor` for tabular datasets accepts input as a dataframe in the same format as the original training dataframe. \n",
    "\n",
    "We will use `test_df` that we created earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = ktrain.get_predictor(learner.model, preproc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = predictor.predict(test_df, return_proba=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(92, 2)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test accuracy:\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.8478260869565217"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('test accuracy:')\n",
    "(np.argmax(preds, axis=1) == test_df['Survived'].values).sum()/test_df.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Our final results as a DataFrame:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Survived</th>\n",
       "      <th>predicted_Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>66.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>28.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>82.1708</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>41.5792</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Pclass     Sex   Age  SibSp  Parch     Fare Embarked  Survived  \\\n",
       "PassengerId                                                                   \n",
       "2                 1  female  38.0      1      0  71.2833        C         1   \n",
       "12                1  female  58.0      0      0  26.5500        S         1   \n",
       "34                2    male  66.0      0      0  10.5000        S         0   \n",
       "35                1    male  28.0      1      0  82.1708        C         0   \n",
       "44                2  female   3.0      1      2  41.5792        C         1   \n",
       "\n",
       "             predicted_Survived  \n",
       "PassengerId                      \n",
       "2                             1  \n",
       "12                            1  \n",
       "34                            0  \n",
       "35                            0  \n",
       "44                            1  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = test_df.copy()[[c for c in test_df.columns.values if c != 'Survived']]\n",
    "df['Survived'] = test_df['Survived']\n",
    "df['predicted_Survived'] = np.argmax(preds, axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explaining Predictions\n",
    "\n",
    "We can use the `explain` method to better understand **why** a prediction was made for a particular example.  Consider the passenger in the fourth row above (`PassengerID=35`) that did not survive.  Although we classified this passenger correctly here, this row tends to get classified differently across different training runs.  It is sometimes classified correctly (as in this run), but is also often misclassifeid.  \n",
    "\n",
    "Let's better understand why.\n",
    "\n",
    "The `explain` method accepts at minimum the following three inputs:\n",
    "1. **df**: a pandas DataFrame in the same format is the original training DataFrame\n",
    "2. **row_index**: the DataFrame index of the example (here, we choose PassengerID=35)\n",
    "3. **class_id**: the id of the class of interest (we choose the **Survived** class in this case)\n",
    "\n",
    "One can also replace the `row_index=35` with `row_num=3`, as both denote the fourth row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation for class = Survived (PassengerId=35): \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABGoAAAEACAYAAADr4JCdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXhU1f3H8feZZLJn2MIWlrDvoCAIKCLuqKCg1lK1rfvWqlWruFWtdUNttb+6tda1WnEBBMviDiKIsrgg+yIRiCwBsu+Z+/vjTDITSEJIJplJ8nk9zzy5d+bOnTNz7rm593u/51zjOA4iIiIiIiIiIhJ6rlAXQERERERERERELAVqRERERERERETChAI1IiIiIiIiIiJhQoEaEREREREREZEwoUCNiIiIiIiIiEiYUKBGRERERERERCRMKFAjIiIiIiIiIhImFKgREREREREREQkTCtSIiIiIiIiIiIQJBWpERERERERERMKEAjUiIiIiUn88U1qEuggiIiKNiQI1IiIiIg3JM2UbnimXVPHaXXimvH8E63oFz5R/B6to9eQ9PFP+Uts3p0xLW5gyLe2eYBZIREQknEWGugAiIiIi4pM1/eFQF6EeXAF8hmcKZE3/U6gLIyIiEu4UqBERERERyzPFTdb04lq8bxzw2WGWugfPlLlkTV9Wm6KFUsq0NHfq1OQj/11ERERqQYEaERERkYbXFc+UT4CRwDbgarKmL8Uz5X5gDFnTTwXAM6UD8AIwFtgNTAP+DXQna/o237qi8Ux5AfgFkAs8QNb0f5Z/kmfKCcAjwADgAPAs8Deypju+AMvHwGXAn4G2QGItvs8S33sr0xGYC7wPfFWLdZdLmZY2BHgKGIr9Li8Bj6ROTS5NmZb2f0BM6tTkq33Lfg6kpE5NTvHN3w6MS52afJZvfhLwJ6An8DPwYOrU5Dd8r10K3AP8E7gJyAQG1qXsIiIiNaVAjYiIiEjDuxw4F1gPPAG8CvSuZLk3gAygCxADvF3JMhcAvwSuASYBb+GZsoCs6al4pgwA5gGXAP/zfcZ8YC/wmu/9EcBZ2OBH7bJGbBZOeqWveaa8A8wha/rva7Vun5RpaS2Aj4CngTOBHtgAUCHwODbg9JRv2QR8wZyUaWl9UqcmbwROw353UqalnQa8iP29lgDDgQ9SpqVtT52a/LnvI7sBydjfzNSl7CIiIkdCgRoRERGRhvdPsqavAfANBvyHQ+6O5JnSGTgZ6EnW9Cwgyzco74kHretTsqbP8U3PxDMlAzgaSAWuB94ha/ps3+vr8Ux5GvgN/kANwFSypmfW9UulTEtzAfGpU5OzA57+FVnTd9V13cDZQBE288UB1qVMS5sG3IIN1CwEuqRMS+sB9AeWA5uA01KmpaUCxwO3+tZ1E/D31KnJi33zX6dMS3sd+7uUBWqKgTtSpyYXBqHsIiIiNaZAjYiIiEjD+zlgOtf39+AuR518f38KeC71MOsqW1/ZuroDJ+OZcl7A6y5ge8C896D5WvEFaf7tW/+l5S8EJ0gDNqso1RekKbPF9zypU5OzUqalLQdOxQZqPgI2AxdjM5eygdW+93UHTkqZlnZLwLoigMUB8z8rSCMiIqGgQI2IiIhIeNrp+9sV2BowfSRSgZfImv67apZxyJruVPP6YQUEaYZjs4Dqw3YgJWVamgkI1vSgYpDpY/yBmsuAH4F/ARuBTwLelwq8kjo1+fFqPs8bzMKLiIjUlAI1IiIiIuEoa/oOPFMWAo/imXIFdoyae45wLc8Ci/BMWQAsABygD9CWrOmLglja87CBEYC9KdPSqlqud+rU5M21/Iy52DFo7kqZlvY4NitmKnbA3zIfA3/AdpFalTo12ZsyLe1H7Pg9Nwcs9xTwSsq0tGXAUmw2zWDApE5NXlHL8omIiASFK9QFEBEREZEqXQTEATuwg96+43u+Zl1ysqb/AEzABi9+BvYAr1D1HZpqazYwC9u1qIdv/ZU9tla1gsNJnZqcCZyOzZjZDXyAHWfnbwGLfYk9vv00dWpyWUbMx4DH97dsXR8CV2HHtknH/jZPAgm1LZ+IiEiwGMepU6ariIiIiDQUz5QzsEGR2Lp2Vwq2lGlpbmA6kJU6Nfmywy0vIiIilVOgRkRERCRceaYcjR0rZTW2q89bwFqypv82pOWqQsq0tEigZerU5Mpv1S0iIiKHpTFqRERERMJXK+AFoCOQCczHf4vpsJM6NbkE25VIREREakkZNSIiIiIiIiIiYUKDCYuIiIiIiIiIhAl1fRIRkcZkF9A+1IVoBHYDHUJdCKkXagPBp/YiIiJhRV2fRESkMdE/rZozoS6A1Au1gfqh9iIiImFDXZ9ERERERERERMKEAjUiIiIiIiIiImFCgRoREWnUnnrqKbZu3RrqYtTKwoULmTlzZqiLIdIovPLKK6xatSrUxRAREal3CtSIiIiINFP3338/+/fvD9v1iYiINEcK1IiIiIiIiIiIhAndnltERBq9nTt3Mn/+fLKzs+nXrx8TJkwgMjKS/Px8Zs2axY4dO/B6vXTt2pUJEybg8XgA+Pbbb1m0aBG5ubnExcVx8sknM2TIEAC++eYblixZQk5ODp06dWLixIm0bNnykM9+/fXX6dOnD8cee2z5c8899xzjxo2jf//+zJ8/n3Xr1lFYWEjr1q0ZP348KSkph6xn27ZtzJw5k1tuuaX8uaeeeopzzjmHHj164DgOS5YsYeXKlRQUFNCjRw8mTJhAbGxssH9OaWT27t3L3Llz2bVrF4mJiZx66qn07dsXsN2FhgwZwrBhwwC7za9atYrLL7+cl19+GbDbqzGGc845h4SEBGbOnMmIESP48ssviYqKqtAujnR9gwYNKi9nSUkJTzzxBJdffjnt2rUDIDc3lyeffJKbb74Zl8tVbXsNtHDhQvbv3895550HQEZGBk899RT33nsvLpeLgoICPvjgAzZt2oQxhqFDhzJu3DhcLl2jFBGR8Kf/ViIi0uitXr2aSy65hJtuuol9+/bx+eefA+A4DkcffTQ333wzN998M5GRkcybNw+AoqIi5s+fz8UXX8xdd93FFVdcQYcOHQBYv349ixcv5pe//CW33347KSkpzJgxo9LPHjx4MKtXry6f37t3L5mZmfTu3RuATp06ce211zJ16lQGDx7MO++8Q0lJyRF/x6+++or169dz2WWXceuttxITE8PcuXOPeD3StJSWlvLmm2/Ss2dPbrvtNs466yxmzJhBenr6Yd972WWXAXDddddx1113lQdVcnJyyMvL45ZbbmHSpEm8//77dVpfmcjISPr371+hvaxZs4Zu3boRHx9fbXs9Uu+99x4ul4sbb7yRa6+9li1btmh8GxERaTQUqBERkUbv2GOPpUWLFsTGxjJ27NjyE8G4uDgGDBiA2+0mOjqasWPHsm3btvL3GWPYs2cPxcXFJCYmll/lX7FiBWPGjKFt27a4XC5OOOEEdu3aRUZGxiGf3a9fvwqvff/99/Tv35/ISJu0OmTIEOLi4nC5XBx33HGUlJTU6KT3YCtWrODkk0/G4/EQGRnJuHHjWLt2LV6v94jXJU3Hjh07KCoqYsyYMURERNC9e3f69OnDDz/8UKf1nnTSSURGRtKtWzf69OnDmjVrglLewYMHVyjb6tWrGTx4MHD49lpTOTk5bNq0ifHjxxMVFUV8fDyjRo2q828iIiLSUNT1SUREGr3ArhEtWrQgOzsbgOLiYhYsWMDmzZspKCgAoLCwEK/XS1RUFBdccAFLly5lzpw5dOnShTPOOIOkpCQyMzNZsGABH374Yfl6HcchOzv7kO5P0dHR5SfGY8aM4YcffmDixInlry9dupRVq1aRnZ2NMYbCwkLy8vKO+DtmZmby1ltvYYwpf87lcpGTk1Np1xBpHrKzs/F4PBW2i5YtW5KVlVXrdcbExBAVFVU+H9im6qpbt24UFxezY8cOEhIS2LVrF/369QOqb69H0mUpMzMTr9fLX//61/LnHMdROxERkUZDgRoREWn0Ak9KMzMzSUxMBGyQZN++fVx11VXlJ4XPP/98+bK9evWiV69eFBcX8+mnnzJnzhwuv/xyPB4PJ5xwQvm4HIczaNAgFi1aREpKCiUlJXTv3h2A1NRUlixZwm9+8xvatWuHMYZHH3200nW43W6Ki4vL571eL7m5ueXzHo+Hc889l65du9b8h5EmLzExkaysLBzHKQ/WZGZm0qZNG+DQ7SonJ+ew6ywoKKCoqKg8WJOZmVmebVab9QVyuVwMHDiQH374gfj4ePr06UN0dDRw+PYaqLpyeDweIiIiuP322zUmjYiINEr67yUiIo3e119/TVZWFvn5+SxevLh8bIyioiIiIyOJiYkhPz+fhQsXlr8nJyeH9evXly8TFRVVfqI7fPhwvvjiC/bs2QPYE9fqun707t2bjIwMPvvsMwYOHFi+nqKiIlwuF/Hx8Xi9XhYtWkRhYWGl62jTpg0lJSVs3LiR0tJSPv/8c0pLS8tfHz58OJ9++ml5F6vc3FzWr19f+x9NmoTOnTvjdrtZsmQJpaWlbNu2jQ0bNpS3gQ4dOrBu3TqKi4vZv3//IeO0JCQkcODAgUPWu3DhQkpLS0lNTWXjxo0MHDiwTusLVNb9KbDbE1TfXg/WoUMHUlNTyczMpKCggMWLF5e/lpiYSM+ePfnggw8oLCzEcRz2799fq25UIiIioaCMGhERafQGDx7Mf/7zH7Kzs+nbty9jx44FYNSoUcyYMYPHHnuMxMRERo8eXR7ccByHL7/8klmzZmGMoUOHDkyYMAGA/v37U1RUxLvvvktmZibR0dH07Nmz/GT1YGWDpH7zzTeccsop5c/37NmTXr168Y9//AO3283o0aNp0aJFpeuIiYnh7LPPZs6cOTiOw/HHH1+hq8aoUaMAyr9nfHw8gwYNKu82Is1TREQEv/rVr5g7dy6LFy/G4/EwefJkkpKSABg9ejRpaWk8/vjjtG/fniFDhrB169by948bN45Zs2ZRUlLCxIkTiY+PJyEhgZiYGP7617/idruZMGFCrddXWZvp3LkzUVFRZGdnlw+6DdW314OVtcfnnnuOuLg4jj/+eDZs2FD++uTJk/n444955plnKCwspFWrVowZM6ZuP7aIiEgDMY7jhLoMIiIiNaV/WjVnDr+INEL12gYqu018M6H2IiIiYUNdn0REREREREREwoQCNSIiIiIiIiIiYUJdn0REpDHRP62aU1eOpkltoH6ovYiISNhQRo2IiIiIiIiISJhQoEZEREREREREJEwoUCMiIiIiIiIiEiYUqBERkcZkd6gL0Ejod2q6VLfBp99URETCigYTFhEREWneGtvBoAb+FRGRJk0ZNSIiIiIiIiIiYUKBGhERERGp1PTp0xk2bBixsbG0bt2aCy64gM2bN1f7nksvvRRjzCGPzp07V1guOzubm2++mc6dOxMVFUWPHj24//77KS4urs+vJCIiEvYiQ10AEREREQk/L774IldeeSUA3bt3Z9++fcyYMYPFixfz3Xff0aFDh2rf36lTpwrBmXbt2pVPe71eJk6cyKJFi3C73fTo0YNNmzbx5z//ma1bt/Laa6/Vz5cSERFpBJRRIyIiIiIVFBUVcccddwBw/vnns3XrVtatW0diYiJ79uzh4YcfPuw6rrzySpYtW1b+mDNnTvlr7733HosWLQJg5syZrF+/nqeeegqA//znP6xataoevpWIiEjjoECNiIiIiFSwfPly0tPTARuoAUhOTmbUqFEALFiw4LDreOqpp4iOjqZLly5MmTKFLVu2lL82f/58AGJjYznrrLMqfE5N1y8iItJUKVAjIiIiIhVs3769fDqwy1L79u0B+Omnn6p9f1RUFB07dqRz587s2LGDt956ixEjRrBz584K62/Tpg0ul6vCumuyfhERkaZMgRoRERERqRHHOfydvP/4xz+yb98+1q1bx5YtW3j++ecBOHDgAC+//HKd1i0iItIcKFAjIiIiIhV06dKlfHrPnj2HTHft2rXK9w4aNIiEhITy+Ysvvrh8uixTpmz96enpeL3eQz6nuvWLiIg0dQrUiIiIiEgFI0aMoE2bNgDMmDEDgLS0NJYtWwbA+PHjAejXrx/9+vXj6aefLn/vfffdx969e8vnp0+fXj7drVu3Cu8vKChg3rx5FT4n8HUREZHmyCjNVERERKRZq/Rg8F//+hfXXHMN4L89d1ZWFklJSXz33XckJydjjAFscOb+++8HwBiDy+WiR48eOI5TPohwhw4d+O6772jXrh2lpaWMGzeOL774ArfbTc+ePdm4cSNer5eLLrqIN954o7rymqB9cxERkTCkjBoREZFays7OdrKzs3XFQ5qkq6++mtdff52jjz6atLQ0jDFMnjyZJUuWkJycXOX7HnroIY477jiysrLYuXMnvXr14tprr2XFihXlAxNHREQwd+5cbrzxRtq2bcuWLVvo2rUrf/rTn3jllVca6BuKiIiEJ2XUiIiI1FJZkCYxMVFX+KUxa2wHg2pvIiLSpCmjRkREREREREQkTChQIyIiIiIiIiISJhSoEREREREREREJEwrUiIiIiIiIiIiECQVqRERERERERETChAI1IiIiIiIiIiJhQoEaERERkeZtd6gLcAQaU1lFRERqJaiBGmPMeGPMBmPMZmPMHZW8fosxZq0x5ntjzCfGmJSA135rjNnke/w2mOWSytWxvkqNMd/6HnMatuTNTw3q6lpjzGpffXxhjBkQ8NqdvvdtMMac0bAlb55qW1/GmG7GmPyAtvV8w5e++TlcfQUsd74xxjHGDK/kNbWvBlDbulLbOqwOgAnmIzs7m+zsbIwxG40xW4wxdx68jDHmMmNMujHmO9/jqoDXLvXV82ZjzKUB7+vQED9Ic1OD/1uXGmP2BrShKwNe0zFhA6vJvtAYc6HvOH6NMea/Ac/rnKsB1bGu1LYaWA32hU8G1MlGY0xGwGvBbVuO4wTlAUQAW4AeQBTwHTDgoGVOAuJ809cBb/mmWwNbfX9b+aZbBatsegS3vnzzOaH+Ds3lUcO68gRMnwMs8E0P8C0fDXT3rSci1N+pKT/qWF/dgB9C/R2a06Mm9eVbLhH4HFgGDPc9NyArK8vJyspy1L7Cvq7Uthr4EdA2qtsXXgo8XUkd6riwAR81/L9VaV35XtMxYfjVV2/gm7J2A7Tz/VXbaiR15ZtW2wqz+jpo+RuAl3zTQW9bwcyoORbY7DjOVsdxioDpwLmBCziO85njOHm+2WVAZ9/0GcBHjuPsdxznAPARMD6IZZND1aW+pGHVpK6yAmbjAcc3fS4w3XGcQsdxfgQ2+9Yn9acu9SUN77D15fMXYBpQEPBc+XJqXw2iLnUlIVKD+qqMjgsbVk3bloSHmtTXVcAzvvaD4zh7fM+rbTWsutSVNLwj3Rf+CnjTNx30thXMQE0nYHvA/A7fc1W5Aphfy/dK3dWlvgBijDErjDHLjDGT6qOAUq5GdWWM+Z0xZgvwGHDjkbxXgqou9QXQ3RjzjTFmkTHmhPotqlCD+jLGDAO6OI4zt5L3BlL7ql91qStQ2wq1qtrH+cZ2sX7XGNPF95z+dzWsmv7eldUV6JiwodWkvvoAfYwxS3z1Mv4I3ivBU5e6ArWthlbj9mHskCDdgU+P9L01FVmXN9eWMeYSYDhwYig+X45MFfWV4jjOTmNMD+BTY8xqx3G2hKaEAuA4zjPAM8aYi4B7APU7DmNV1NfPQFfHcfYZY44B3jPGDDwoA0cakDHGBfwNm/YvYewwdaW2FZ7eB950HKfQGHMN8CpwcojLJJWrrq50TBh+IrFdasZhM+I/N8YMDmmJpCqV1pXjOBmobYWzKcC7juOU1tcHBDOjZicQGF3v7HuuAmPMqcDdwDmO4xQeyXslqOpSXziOs9P3dyuwEBhan4Vt5o60fUwHyqLualsNr9b15euits83vRLbT7ZPPZVTrMPVVyIwCFhojNkGjALmGDtI7cH1qvZVv2pdV2pbYeGQ9uE4zr6AY4t/A8f4pvW/q2Ed9veupq50TNjwatI+dgBzHMcp9nXN3YgNBqhtNay61JXaVsM7kvYxBX+3pyN9b80EcfCdSOygOd3xD74z8KBlhmIPjnof9Hxr4EfswDutfNOtg1U2PYJeX62AaN90ErCJagZa0qNB6qp3wPREYIVveiAVBxPeigY7Def6altWP9iBzHZqXxj6+jpo+YX4B6gdeNBgwmpf4VtXalsN/DiobVS1L+wYMD0ZWOab1nFhAz5q+H+rqrrSMWF41td44NWAetkOtFHbalR1pbYVhvXlW64fsA0wAc8FvW0FreuT4zglxpjfAx9gR0x+yXGcNcaYB7AnIXOAx4EE4B1jDMBPjuOc4zjOfmPMX4DlvtU94DjO/mCVTQ5Vl/oC+gP/NMZ4sVlZjzqOszYkX6QZqGFd/d6X/VQMHMDX7cm33NvAWqAE+J1Tjyl6Urf6AsYCDxhjigEvcK32hfWrhvVV1XvXZGdnl80uQO2rXtWlrlDbCqXq6utGY8w52P9P+/F1W9NxYcOqYduqtK7QMWGDq2F9fQCcboxZC5QCtzm+rEK1rYZTl7oyxhyH2laDOoLjjCnYm7U4Ae8N+v8tE7B+EREROQLZ2dn2ftCJiSbUZREJJ2obIiIitRfMMWpERERERERERKQOFKgREREREREREQkTCtSIiIiIiIiIiIQJBWpERERERERERMKEAjUiIiIiIiIiImEiJIEaY8zVofhcOXKqq8ZF9dV4qK4aF9VX46G6alxUX42L6qvxUF01LqqvxqOh6ipUGTXaEBsP1VXjovpqPFRXjYvqq/FQXTUuqq/GRfXVeKiuGhfVV+PRpAM1IiIiIiIiIiJyEOM4TpUvjh8/3klPTw/6h+7du5e2bdsGfb0SfKqrxkX11XiorhqXqurL6/UC4HLpuke4UNsKDzVtG6qvxkX11XiorhoX1VfjEcy6Wrly5QeO44yv7LVqAzVAtS+KiIg0Z9nZ2QAkJiaGuCQi4UVtQ0RE5LBMVS/oEqCIiIiIiIiISJhQoEZEREREREREJEwoUCMiIiIiIiIiEiYUqBERERERERERCRORoS6AiIhIY6WBUkVEREQk2JRRIyIiIiIiIiISJhSoEREREREREREJEwrUiIiIiIiIiIiECQVqRERERERERETChAI1dVVcAkUloS5F45FfBF4n1KWQhlDqhYKiUJdCxCophcLiUJdCREREBICiEgev0zTPiwqKm+b3aki661Nt5RdB6h74+QB89yN8+n2oSxTeOraCEwdBlySYuwLWbg91iaS+xEbB8N4wtDvsOgBvLwl1iaQ5S4iBY/vAkBTYsgveXx7qEok0D2/cBMBF09NDXBARkfD066HxtI51sXJnEct3FlLYRK79J8W5uHJEIkWlDr3bRNIiVrkhtaFATW0VFEF6FrgjYEAXeHQG5BWGulTh68Ix0LWtne7fGZ6dH9rySP1p1wKuPwsiXNCtPWzbYx8iodCjvd0eAfp0gjUzYG9WaMsk0owsSVVmpYjIwfomRdLJY0/FR3aJ5p9f55BR0DSyUG4YnQBAVIQho8CrQE0t6VerrZbxEBdtp+Oi4YyhoS1PuJu/Egp83Q56doTBKaEtj9SfPZmwdL1/ftLI0JVFZOtum/UINng48djQlkdERESavUkDYsunF/1Y2GSCNC1jDON6xJTPJ8Ur3FBb+uVqyxhIbu2fP3ckuEzoyhPusvPhk+/885NHha4sUv9mLfNPnzQYWsWHriwigdvjmcNs9zwRERGREGgX72J01+jy+dlr80JYmuA6s28sURH2nDjWbYhz6/y4thSoqYt2LWzXp7LpMf1DW55w995X/ulRfaFT66qXlcZt/Q5Yt8NOuyPh7BGhLY80b8s3wQ7fOBnxMXD60aEtj4iIiDRbE/vHEuG7wP9NWhHbMkpDXKLgiIqAs/r4M4XaxrswRoGa2lKgpi5cLugYEGyYPDp0ZWkMdu6Drzb6589Vl5gmbdaX/umzh0OUhsSSEHGoGChWBqSIiIiEQJzbcHovf9eg2WvzQ1ia4BrXPYaWvvFo3BHQIkbHWnWhQE1ddWhlu0EB9O1kBxaWqgV2QTj1aEiMrXpZadyWrofdGXa6RRycPCS05ZHm7ZPvIMuXWtyhFYzuG9ryiIiISLNzWq8Y4qLsKfhPGSWsSms6A66fGzDuTlKcsmnqSoGauoqKtN2eymjslep9vw22/GynY9xw5jEhLY7UI68DswOyGCaNBO2vJVQKS2DeSv+8MiBFRESkAbkMnNPfH8yYvS6fpjGEMAxLdtO1pc2edxloHacwQ13pFwyGwEGFR/ezV2ulaoFZNeeMgMiI0JVF6tcH3/hvW9+1LRzTK7Tlkebt/eVQ7OsHPqCLzYIUERERaQDHdY2mXYI978ks8LJwa0GISxQ8kwbElU+3jnOVj8EjtadATTDEx/jvauMycK5u/1qtz9fAvmw73ToRxg4MbXmk/uQXwYJV/nllnEkoHciBhav989oeRUREpIEE3pJ73oZ8iprGGMKktIxgaLL/jppJyqYJCv2KwZLcxj99+lCIj6562eauxAvvf+2f18lS0zbnayj12umhPaB7+9CWR5q3wEGFj+9fseuqiIiISD3o1zaSvm3dABSXOszb0HQGEQ4cm6ZFjCEqUtk0waBATbC0jIc4X3AmNgrGDwttecLdvJVQUGyne3aAo7qFtDhSj/ZkwtJ1/vlJutuXhNCPu+HbH+10hAvOUQakiIiI1K/ArkELfywgo6BpjE7TMsYwrrv/LlZJ8QovBIt+yWAxpuJYNRN18F+tnAL47Hv/vE6WmrbZARlU4waDJ67qZUXq25yArJozhkK0bh0vIiIi9SMpzsWoLv6uQf9b33TGpjmjdyzuCJtBExtpbz8uwaFATTCVDZoKsHNf6MrRWHRO8k9v1+/VpHUO6Bp4IAdym84/KGmEAvc9ezPtHaFERERE6kFWoZfsQn8GTbKn6dxIZWeWf6CdIi9N5i5W4UCBmmApKYXdGf75wDsbyaF6d4TBKXa6pLTimDXS9ASOQxQ4Zo1IQzu4u5P21SIiIlKPikph3kb/mDSTA8Z0aeyW/lTInlsQBrwAACAASURBVBwbrCn1woF8hWqCRYGaYNmd4T/5/GkvrNwc2vKEu0kBJ+6Bd4GSpmdYT0hpZ6fzCuGDVdUvL1KfThgASR47vT8HPvshtOURERGRJm/+hnyKSm0Qo0+Sm/5tm0a3a68Dc9b5g1DpuaU4joI1waBATTA4DqTt98/PWqa8r+okeezJUpnAu7BI0xOYTfPhN5BbWPWyIvUtcHucu9xm9ImIiIjUo4wCh4Vb/V3/AwcXbuw+2lxAXpFNWCgsoUI3L6k9BWqCYV82FPruYJSZC5+tDm15wt3EERDp65v5/TbY/HNIiyP1KKUtHNPTTpd6bbcnkVAZ1BV6J9vpwmKYuzK05REREZFmY/Zaf+bJqK5RdEhoGqfiecUOH272B6HSczXEQTA0ja0j1AIHDp67Eoo0MGWVYtxw5jH++fc0PkSTFtjFbdkG2JVR9bIi9S0wm+bT7yErL3RlERERkWblp8xSVqUVAeAyhon9m85YNe+vy6fUazNpcooc8ouVVVNXCtTUVVYeZPuio8UlNpVeqnbq0ZAQY6d37oOvN4W2PFJ/WsbDSYP98xq0VUKpYysY2dc/ry6XIiIi0sDeW+u/SHRqrxjim8jtrPfkevnyp6Ly+fRcdS2vKwVq6ipwbJrPVsOB3NCVJdy5DEwa6Z+f/ZUdgUqaprOHQ5RvoLQNO2Ht9tCWR5q3c0fafRDA8k2wPT205REREZFm55u0Yn7KsL0v4twuTu8dE+ISBU9gECoj36G4VOd5daFATV0UFEF6ln9eV2irN7IPJLe209n58NF3oS2P1J+oSBuoKTPry9CVRSQhBk4/2j+v7C4REREJkfcCxqqZ2D+WiKaRVMOG9BLW77XjtjrAvjyNVVMXCtTURWA2zaotsG1P6MrSGASODzF/pX8AZml6Thpsuz4B7MmAL9aFtjzSvI0fBjFRdvrH3fDtj6Etj4iIiDRbC7cWkJFvgxht4yM4LiU6xCUKnsCsmn25Xry6VXetKVBTWyWlsDtgYFRdoa1e72QYlGKnS0rhfY3l06QFBuVmf60ubhI6kS4451j/vPbVIiIiEkLFXpi7wZ9VM3lA0xlU+MufitidY8enKXXgQL7OAWpLgZra2p9ju++UeiF1D6zcEuoShbeTBvmnP19jb2kuTVPvjtC1rZ3OK4QPvglteaR5G9wNkjx2en8OLPwhpMURERERmb8xnyLfGC49WkfSuUVEiEsUHF7H3gEKIKfQy4E8DSpcW8apPh1JIbDKOI4NzLyzBHp1AHck7M0MdanCX5LHBmlG9ILcwlCXRupTbBQs2wgThmvQVgm9uGi7PY4fWrHLqojUm+xrTwPgldUhLoiISJjq2ToSDPz321xGdmk63Z8iDAxu7+ajzQV09LgY1TWa/u2iiIlsIoPxBFeVP0pkQ5aiSdibCTO/hJ37YVgPaOEbh6NDq9CWq7EwBnp0DHUppCGs2gotE+xDJNRWboE2HvsQkXqXmJbF1hc+Z8C1k0NdFBGRsNYq1sWAdu5QFyOoirwwtns06XlePttayNz1BRzdMYphnaLomOjCGAVtDkeBmpoqLoHFa+GT76BzEhzf3wYd5MgYAy71uGsWVNcSTrQ9ijQsY9TsRERqoOnuKw3tEyNonxhBYYnD9owSXllZRKtYFyO7RDGgXRQxbp1PV0WBmpr4cTfMWApFJTC6L8Q2ndS0BucyNJl70En1VNcSTrQ9ijQsY/O5m+S5h4hIEDWHfWVspKFPkpvebRzS87ws3lbI/I0FDOkQxbBObpITI5RlcxAFaqqTVwgfrLIp8307Qac2oS5R06BG2HyoriWcaHsUaThlGTVqdiIi1TI0o32lMbRPiKB9gi/LJrOE/6wqxhNjfFk2bmLdTT1sVTMK1FTGceCHn+C9ZZAYCycOAnfTGIk75HTU1nyoriWcaHsUaVjGYDC4FCAVEamWMc1zXxnrNvRJiirPslmaWsgHGwsY1N7N0E5RdPY07ywbBWoOtj8HZn8FP+2FId2gTWKoS9S0NN1OmHIw1bWEE22PIg3Ll1HTjI+xRURqxNC895XGGNolRNAuIYKiEoefMkv477d5JEYZju0SxcD2zTPLRoGaMqVe+HI9fPQtdEmCkwbr6mt9aFa5fc2c6lrCibZHkYZl1OxERGpCSb9+MW7/WDb78rx8vb2IDzcXMLCdm6HJUXRp0XyybBSoAdiRbm+5nV9k7+aUEBPqEjVd2hM1H6prCSfaHkUalsaoERGpEQW1KxGQZVNY4vBTRglvfZ9HvNsworPNsomLatpZNs07UFNQBJ9+D19tgn6doFu75p131hDU/aD5UF1LONH2KNKwjNHJh4hIDSioXb1Yt6FvWzd9kmyWzcq0Ij7eUkD/tm6GdWq6WTbNM1DjOLAxDWYtg4RoOHUIRLtDXarmQUdtzYfqWsKJtkeRhmXsoykePIuIBJv2lYdnjKFtgou2CZEUlTqkHijhndV5xET6s2zim1CWTfML1GTmwbwVsPlnGNoD2rcMdYmaF40s2HyoriWcaHsUaVjKqBERqRFl1By5mMiyLJtI9uV5+e7nIj7dUkjftpEMS46ia8vGn2XTfAI1Xi+s3AILVkGnNnDGUIjQLbcbnLofNB+qawkn2h5FGpZLY9SIiNSEgtp1EHjHKF+Wzcw1eURFGI7pFMXgDo03y6Z5BGp2Z8B7yyArD8YMgJbxoS5R86U9UfOhupZwou1RpIEpo0ZEpCYU1A6Og7Ns1u4pZtGPhfRuE8mwTlGkNLIsm6YdqCkugUVrYOk66N8ZRvZR6nuoaU/UfKiuJZxoexRpWEY9DkVEasKgfWUw2bFsImibEEFxqcO2AyXMWZuPywXHdHIzuH0UCdHhn2XTdAM1W3bB7K8gxg1nDIO46FCXSEDdD5oT1bWEE22PIg3L5cuoCXU5RETCnPaV9Sc6wtA3yU3fJNiXV8qGvSUs/rGIXm0iGZocRUqrCFxhGiVreoGa3AI7Ds36nXBMT+iSFOoSSSDlQTcfqmsJJ9oeRRqWMRhjcKndiYhUS/vKhtE2IbLCHaPmrs8HYFgnN4M7RJEYZlk2TSdQ4zjw3Y8wfxV0ag0Th4O76Xy9JkPdD5oP1bWEE22PIiGhVicicnjaVzac6AhDnyQ3fZLc7MsrZcv+EpakFtGjVSRDO0XRLUyybJpGJGNfFry/HNKzYOwASPKEukRSFV3Vbj5U1xJOtD2KNCzdnltEpEZ0LSl02sZH0DbeP5bNgg35eIFhyW4GdYjCE8Ism8YdqCkphaXrYfEa6N8FRvfVGAThTuNENB+qawkn2h5FGpZuzy0iUiMKaodetO+OUX3butmf52Xz/hKWpubQrZUdy6Z764bPsmncgZoVm2HVZjhrOCTGhro0UiO6BUTzobqWcKLtUaRBmbKTD7U7EZHq2KC29pXhIik+gqT4CIqTbZbN/9bnM65HNEd1jGrQcjTuQE1RCSS3gRZxoS6J1JRCxs2H6lrCibZHkYZllFEjIlITOkQJT2VZNtlFDoUlToN/fuMO1IDdqpXO3nio+0HzobqWcKLtUaRh+cao0UViEZHqGSX9hrVQVU3jD9Tock3jovpqPlTXEk60PYo0LGXUiIjUiDJqwluo6qbxB2pcOgpoVLQnaj5U1xJOtD2KNCwDBqNxF0REDsMY7SvDWaiqpvEHapTO3riovpoP1bWEE22PIg1LGTUiIjWia0nhTV2faksZNY2LjtqaD9W1hBNtjyINy3cJUheJRUQOQ2PUhDVl1NSatuxGRSMLNh+qawkn2h5FGpZRfFREpCaUURPeNEZNbUUYiGjgdPb3voKZS+G1m+18j6vh0wehW7uGLUdjZEJQXzV128vQoRXcOik463tqDqTuhSevqPu6TrgTHvk1jBlQ93U1lHCua2l+tD1KXS3bALe8CEsfC3VJGgffXZ908lFzZ7+yl3tP9jCya3SoiyIiDUhB7fCmrk+1VV9b9vJN8MgM2JRm19+rI9z7SziqO5w3yj4CVdUFq6gEHpsJ/1sOWfnQKgFOPxrumxL8Mldlw0548G344Sc4kAPbXqh++TU/wdRXYfMu6NUBpv0WBnYNTlmCXV/H3wHpWRVPwC44Dh64qHbrC+ZRpTHBXV99dPNb9AM8M8/WebQbeiXDVafBaUfXfd1N9b9OZdvcZw9C+5ahK1N1HAcemwUzlkJ+kW3LD15s92kAD7wFn3xnv1OHVnDD2TBpVOXr2nUA7n4dvk+FvZnw5TTo2Nr/+sn3wK4M/3xhMZwyBP71Ozv/QypMfQ227II+HWHapdC/s32toBjufxM++hZKSmFEb3jokuD9rk11e5TaCWzHcVFw4iD7fyM+pur3lGVkhft29Oqn8O5S+79/4gj46+XVL//vj+D5BVBQBGceY/cP0e66lyNIY9R8k1bEk1/ksHV/CS4D3VtHctvYRAa1D0IZw5B2VSLNT7CD2lfM2M/G9BI+uaItUZHhs0MpKnF4eGEWX20vIrPAoXOLCG48LoEx3fzB6Q82FvD8VznszvHSIcHF749L4OSelf9vLipxeGhhFh9vKiTGbbh0WBy/HhYf9HKr61Nt1ceN57Pz4Yp/wIOXwIQRUFwCX2+yBy5VfVZVafXPzYfVqTDnHmjXAnbsg683NmyNuyPt9/jNSXDVM9V/dlEJXP0MXH4q/Pok+O8iO7/wYYgKwuYS7O4HBnjphuBkmhhD0LrSlZT6w6/BWF/Z7xbM327uCrj9FbjnQnjxBkiIsdv5rGVw+tC6r7+pdjUJ1jZX6m2YDI/ZX8N7y2DGHZDc2gaOb33J7pPAnpi+fBN0bwff/AiXPgXd28PRPQ5dl8sFJw2G686EC6Yduk1+9pB/2uu1J8Nnj7DLFBbb/c814+GisfDaZ3bf8tmDdh/14kd2X/nhn+22eNsrNoj07LXB+R2a6vYotRPYjncdgF8/CU/PhTsuqP49jaG7dfuWcMME+PwHGwCtrryLfoDn58Obf7Tvu/oZmw1a3e9QU75rFaYO1yJzCr3cOCeDu0/ycHrvGIq9sGpnEdERpk7rDVdlv1dT/G4iUrW67isD7cwq4Zu0YhKiDIt+LOL03tVcgGhgpY5Dh4QIXjy/NR0TI1i8rZDb52fy7sVt6OSJZHdOKXd/mMnfJ7Ti+JQoFm8r5Lb5Gcy7NIo2cRGHrO/5r3L4KaOUBZe1JT3Py5Uz99OztZvjuwU3KzFUd+RqAoEaV/Dv5LFtj/07ebT9646EcYP9r7/9BUz/HGbe5X9u4Q/w4pOQXQAXjoG7LrDl+n4bjB/mv+qc0s4+yoy+DS4eZ7tS7cm0J8gP/wZignilqHeyffy4285X93t9tRFKvHDVGfbg7orT4V8fwpcb7AlaXdXHnVeqWufbX8Cbn9ssqHe+gJbx8PerYesueGKWDUrdfSH84viyFUFGLlzyN1i1BQalwFNXQuck+/J9/4X5K20gr3t7uO9XMLKPfe1v79mrl9FumxFw7xT/SazLZYN9f/g3FJfC09dApMsG8f77OWTlwZj+tt5bJdj1zVgKj8+C3AK46vTg/3aOY7Osbppot78yx/W3j2BoynfZqey7eb1w3XM2G6+wGAZ0hYd/bdsewI3/gsRY2x3u643w6h9gaE+YNgPmLrfbxpnH2G0nmO1/Rzoc2we6tbfz5422QZKy8t9+nn/ZEb3hmF42YDOs16Hr6tgafnuKPfkDu46q6njpettWzjzGLvPlBvv3itPs61efAS98aIODJwy0Qexxg6GdL4PmnJHw2IzgbUNNeXuU2inbJpLbwElDYGOanT+QA395ywYxCothZF8bzC7bfsr+PjPX7sP3Zdkg6G3n2e0d7P/b216GtdshMgKO7w/PXWf3vX+eboOnhcXQqY39n9Cvc/C+19kj7N/VqfDzgeq3+xlfwi/HQr8udv6mc+y+6q4L616OIHR92p5ZCsDZ/WIBcEdQ4corwKw1eby8Mpf0PC+D27u575QWJHsi+DatiBveP8A7FyXRITGCDXuLuWzGfl6/sA09Wgfn8HdnVgnjX07ngVM9PLMsh7xihz8cl8iAdm7u/TiTXdmlnN0vlrtP8tjvk1HC/Z9ksSG9GAMclxLN3Sd58ET766gso8brOLy0IpcZa/LJLvQysks0957soUWM9mMiTU0wM+n+t76AIR3cDO7g5v11+Yzv4w/UZOR7ueejTFbsLKJbywiOS4lmxY4iXruwDQBb95fwyMIs1u4ppnWsi9+NTmB8n9jgFAxIiHLxu9GJ5fMn9YihkyeH9XtK6NIikr25pXiiXYztbvfz43rEEOs27MwqpW38oYGa99fn8+BpLWgZ66JlrIvzB8UyZ30+J3QPcvdRZdTUkovg54j26mgPbG7+N5w7Eob1tCf5ZcoyLwI/d8E3MO8+yCuEXz5uuwxddKJ97wsfQpQbRva2B2MHR+XeWwZv3Apx0XDp3+Ef78PU8w8t19cb4bdPVV3uV/9gT8iqUlbe6n6vTWkwoEvFK/39u9jnTxlS9ftqqj5yeqtapzHwzVZ79f6+p21w5nfP2249S6bZ8QauegYmDLdZBQabTfLaH2BoD3jobbjxBXjPF5A7ujvcfC54Ym2q+HXPwrIn7Em1MfDht/DP6+H/roLCEnh2nl1nYbG9Stkm0R6QR7jg3x/CB9/YLIc2ifCnN+zj2Wth40646z/wn5ttOR551x5sV/U9n55rTxiqsu7ZQ5/bvAvS9ttMq/rKsW7K+duVfjcDpx5txySKjIS/TLfBufn3lb/M7K/s2FbDetjAzF/esnX78V/sdnH9c/D0/yoGT8p8uR4u/7+qy/T6rXBMz0OfnzwKrnkWUvfYk8J3l9qT0srqJr/QBpevPqP6uit7rbo6fnepr235/lluSrPdnMqXN3Z+U5rtdnLRWPt77M20Aa3Zy2xwOJhdEZvq9ii1U7ZN7NwHn33vCyoa227jo22GWHw0rNhsnz+461O39jDrTpst+/5yuOkFGN7LZqb89T27Xb87FYpK4fsf7fsW/mD/ly9+1P4v2fwzeOIq3zbvfM0eH1SmUxu736j2+3H4fPqNaXDGUP8yg7rC3ix70aJ1QvXrP5wgdH3q3ioClwvu/jCDs/rEMqSju0Kg4tMtBbywPJdnzm1FSssI/r08l6kLMnjjl20Y1imKCwfHcfeHmTw3qRV3fpDJDaMT6NWm8kPfya+n83NWaaWvndUvhntPbnHI82Ul+WF3MfMvbcuKnUX8fs4BxqRE8+L5rSjxwgVv7GN8nxhGdI4CA1eNiGd4pyhyihz+MPcAzy3L4c5xHv86fb/ZG9/m8dnWQl69oDWtY108vDCLhz7L4omzwrSbrYjUWjC7Pr2/Lp/fDotnSAc3F721j/15pST5ghwPL8wizm1YdFU7dmaVcs2s/XT0ROAykFfs5ZpZ+/n96AT+ObkVm9JLuHLmfvokuSvdbz7waSbz1hdUWoaOnghmXZJ02LKm55aSmlFC76RIXAYGt3fTo3UEC7cWcGL3aBZuLSQqwtCvbeQhv09mgZe9ud4Kr/Vr6+azLYVBP9wLVXi88Qdq6uPgu0UczL7bnvze/orNdDllCDxxGbRt4a+twM/9/Vn2hLtNIlx9uj0hu2SczVhoFQ+zvoQ/v2kzJu66wGbdlLnsFOji25j/MNGOAXFnJWnHo/rChudq/71qEqjJK7QHj4HLeGIhryA4v3Mw90RlrviHzVAp86df2t/eBXRtC78aa5+fNBL+739w67kQG2VPAqMi7AnsoBRbtlOHwHH97PJ3XgB9roOf99uD4vLMG+D6M+H/3ocfd9kxPwz2JPks39XUyAj7XE6BzdAZ2AX+crH/QP8/C222RWcbwea2yTD8VpuVMW8lnHaUvxx3nA+vfFL1GDU3TrCPI5GRa/92aFmPgRqa7olx4DY3uh+8chO4IuBXJ/iX+eNkGHSDHfchLhow9kTw2N72dZexXQsXP+o/IbphAtzykq3zgx3fv3btP7k1jOgFY+6wwaDObeCdqYfWjePA7a/a4OC4QdWvM3AfWFkd5xbY7LP/3up/Pa/w0BNSTxzkFtrnene0J7jDbrblHNAFHr80iIEamu72KLVT1o4T4+DUo+z/672Z8NlqWPu0/wLN8b4sQxcVt6Nzj/Wva/Ioe8zw3Y82i9YdYQNAezJtGxzV1y4XFWnbx9afbVvr26nq8k37rX3UVk3GScsrsMc8Zcu0iLN/8wvAlVj1+2r0+XVvdp4YF29c2IZ/r8jhvk8ySc/1MrZ7NA+c6iEpPoK3V+dx9bHx9PadRFw7Mp4Xlufwc3YpnTwR/H50AlOm72PKm/ton+DikqPjquwJNvvXhz+pOFjZd7t+VAKxbsMJ3aKJdRvO7hdTfvX3mE5u1u8tZmSXKLq3iqR7K1tWO55CPM8uy6kQvy77zd5encc9J3lI9tj1/H50Aqe8uBev4xCpfZlIkxKs09mVO4v4ObuUs/rG0CrWRZcWEczbWMClw+Ip9Tp8tLmAOb9OIj7K0CcpknMHxLJ8RxEuA5//WEgnTwTnD7L/Bwa2d3N67xg+2lRAn6RDA/f3n9KC+085NIBdU8WlDlMXZDJpQGx5IMgVYTh3QCxTF2RSWOLgjoAnz25JQtShoZKCEgeAFjGu8t/OE23ILfYGPycguKurscYfqKku9b4u+nWGf1xjpzel2UyMe9+0GRPG5ftPGvC5nZP8813awu4Mf9muON0+8otsV5ybX7SZNn062ZZZ1XuD7eC07cokxNrgQuAyOQX2QDYYZQp29wNj7EnyiZWcWBoXtPX4Py/Ol/rXvpV/mZgoWy8ul11Xpzb+5RPjbGBtT5atl2fn2RPrXRl22ex8OJBb+XvLyrZyix2v5p/XQ0RAyt7OfTY7InD5CBfsy7b1H7iuhFhbjmD+dm18B+B7syGlbXDWebCm2tWkqm2u1GuzsN5fDvuz/d/9QK6tw4O3kT2ZNvPq1Hv963DsP52g/m5PvGcHEv/u75DkgemL4RePwecP2+2/zJ/egK27YcbUw39+4L6ksmXnrrRZBqP7+Z9LiKli3xJrn5v6GjjYYFRsFPz9ffj1U/C/P9X6q1fQVLdHqZ2q2vGGnfbiSutKghTmoP+hb39hB+Hdnm7ncwv8/xPumwKPzoCzHrABn2vH2yzbsYPg8tPgrtdtt8SzhsP9v7LtoD6+4+G2+/gYX7DUt0xuof0bjP/5vkBRXbv290qK5NHxNotk6/4SbpufwSOLsvnb2S1JyyrlkYXZPPZ5dvnyjgN7ckvp3CKCqEjD5IGxPPRZNneMa4Wrno7ek+Jd5d8zJtIcMp9f7GCMvXr88MJsVuwoIrfYwXHsiUXgb1RWbWlZpdzwfkaFEw6Xgf15XtonHtoFQEQat2AMgzJ7bT7Hp0TTOs7uvyf0i2X22nwuOyaeAwVeSrw226Xsszr6AsHGwM/ZpXy/q5hjn91dvr5SL5zTPyboQ7N5HYc7PsgkKgL+dLKnfP1LUwt5YnE2r/6iNQPbR7JmdwnXzz7AvyZH0L9dxWEB4qPsm3KLvcS47ffILXKId7uCXl7dnru2GuIqad9OMGUsvPapL/3Z98GBn/vzAXsFGCBtn717ysHlio+GK0+Dx2fCpp/9fdJ/3u9fNm1/5e8FO8bDlMerLuf022B036pfr8kdK/p3tgMLmoDl122340oEJaOmHjKgqrqqf3A9VfX9A8uUFlAXOQV2rILkVnbsnmfmwow7oV8newDb6xr/9le2jsB1G2OzdgZ0sQOvvneXPXkFe4X171f5x7gJ1KGlb6yEgEyEAzlVf88n59jBH6uS+u9Dn+uTbIMG85bD786u+r110ZS7mlRWF9OX2LsnzbrTZsjtz4F+1wdsIwe9r31Le3X9y8f820V1lqyDi/5a9evvTrVjzBxszU92XJpk3zhZvznJBmW27ILBKfa5h96BL9babdQTd/iyBGbnVVbHb38BvxxT8bV+neGlj/3POY7dt1x/pn1uzU/w54v82UVXnW67j2TnQYsgjODflLdHqZ3Ktt/OSTbYUtl2F7jdb0+3g3LPuMO2uwgXjLvb/3qHVnaMM7DdbC+YZsf/6tEerjnDPvZmwpVP24sAlWXR/vFleGdJ5WXvkgRfPFr99yv7atVt9/0623F0Jvvu9LZuu80cTvJU/Z6aMnZQ3GAOwtirjZvzBsbx1vd5uIyhY2IE141K5Jz+lQe6dmWX8syXOZw/MJbHFmUz4+LoKu+ActYre0mrouvTOf1jeeC0Sro++b6by1T8noHz5fEyY3hqSQ4G+N9v29Iy1sVHmwp44NNM/7IYjO+9HRMjePiMlhzTKeqQzxWRpsWYuu8rC4od5m8swOvAmOfteKtFpQ5ZhQ4b9pbQJymSSBfsyfHS3TdO1+5sL2D3T8mJkYzoHMUrv2hTo8+796NM5qzLr/S1ZE8E8y6t/EKw4zjc82EW+/K8vDC5NdEB++QNe0sY0TmKozra/d5RHaMY0sHNsp+KGNi+4r6wVWwE7eJdbNxbyvHd7PfZmF7WjSrYQfnQHD82/kBNfdyyeGOaHRB20kh7Mrtzn+26NLyX/6T84ADRM3Nt94LcAjv4btnJx/MLbJ/vY3rZVOh3ltgAwFHd/O9/6WPbRzwu2p5wTxpZ+Xc6vh9sf/HIv4/j2HFSSnwHIEW+u0BUdvvNEwb4x1C59BQ76CjAiQPDt+tTdWPUBH5eZV3WyubL6vXj7+z4AcN8g7wO72UPiNfvsBkxbT3gdeDJ92xGTdlnV3ZAXDZ500Q7mPD5j8Kcu202y2WnwCPvwDPX2vWnZ9lBVc86xo6LdPp9/nI8NtN+ZlXf89Zz7ePIfjR7C9YbX7BXjieO8N/16a0v7DgrddUQQdRQqawucgtsm2qTaLs7PfKOfT5wH1Vhe4ywXfT+9Do88hv7vrT9dv9T2cDdJwyoXfsf1sN2xZw00n7Gm4ttObq3s2V5Yha8/7XNXGlTg64Ol9NxywAACN1JREFUBUX+fUlxiX0E7ku2p9sT06evrvgbnTjQXpp56WP49Th4+RM7UPuY/na5oT3g7cVwXF+bUfPKJ7abVqs6jpNRpilvj1I7lbXj5Fa2C+zUV+GxS+0FluWbbVfUwP18fpF9f1uPnX/zc/t/omzcvNlf2f8fndrY4KPBdrP6dqvdnx/Vze5zY9z2f25l2+bfLrePI1VSah9exz6Kim133MhKsjCmjIHf/8t27e3YEv42x3bhDNKFmbrGR7fsK+GzrQWc3S+WjokRpGWVMnd9Pkd3dOMycNHR8Tz5RTYD2kXSJ8lNdqGXxdsKOatvLI7jcOcHGVw4OI7bxiZy2Yz9/H1pNlNPrDwIteCyI88urSpmffDupmw+t8ghMdrQIsawJ6eUF1fkVFhP4Lp+dVQcT36RzeNntqBTi0j25ZWyKq2Y03qFzx1cRCQ4gnGI8smWAiJcMO83bXEH7O5vfD+D2Wvzuct397ynv8zm4TNsRuJ7a/NI9o1Rc3KvaJ5YnM3stXlM8A3gvm5PMXFRhl5tDj1nfPD0Fjx4+pF3ffrTx1ls2V/Ca79oTVxUxS89pKObfy3PYf3eYga0c7NmdzErdxZxydD4Sn+fSQNjefarHIZ0dJOe6+Xt1Xk8Or5FveQEhIICNZXxxNq7/jw3HzLzbJ/t04fCA7/yfV7AZ5c5+xg4+R7IyoeLTrBXrV3GBl/u/a/tUmCMHWT41ZvsVbUyFxwHv5gGP2fYE/XbJgf3O/2UDkf9wT/f6XIbHPj+777Pn2a7KNx6rj1ofOMWewL/wFu2e9YbtwTvLjT1cVX74r9WHPx43GB4/eZD6+ngv+DPHCoLtlxwnL3b0vJNMKQb/Ot6+9qpR9nHyD/aLlTXjfd1YwkI8hz83QKfu/08e5ep8x6xwZrrxttlLphmbw2b5LFXNCcM94/NcfWzNpvm+jNtNkSwt/VJI+2Jwl9n20ErY6Ls1dUbzg7f7KlwUVldXHKivUvMoBtscOHO8+HVz6rfRh66GB6dCafdBweybT1fcVpwBu4uc+sk2P86nHi3HY+iRwc7YHbLeBs4eWSGzewZfov/PbdNtgHGUi+kXAkz77Rj65SU2v1HmeG32raX/h//c29/YTP7ugXc3Q7svvC/t9p9y33/tdvaG7f4gzwPX2K7P434o20rA7vYAbWDtQ015e1Raqeqfeo/r7ddk0bdZrfFEwb4A4plR9MDOsPvzoLxf7bzU06wGZJl29k3W+14c1l5NkPlkd/Y//vb99p1p+6x2/7JQ+wYY8HcNv82G6bN9M+/swSmnmfHvtqeDqNvt5l8XZLs4Po3ToBJD9kg7MRjfXetDM6FmbqefCRGG77fVczLK3PJKnTwRBtO7hHDHeMScRkY3yeG/GIvf/hfBmlZpSRGG45PiWZCv1heXpXHvjwvt5yQSITL8PiZLTn7lXRO6RXDsZ2Dk6VSWaDGVDJftlncdHwCf5yXwdB/7CalZQSTB8by0orcSoM6lw+3GV2XvrufPTle2sS5OLtfDGeE0a12RSQ4gnGIMmtNPhcMiqNzi4pB+d8MjePPn2Rxx7hE/nyqh9vmZzL6ud10bx3JxP6xrN5VjMuAJ9rFaxe25sHPsnhkYTZex6F/Ozd3j/ME7V/UzswS3vwuj6gIGP3cnvLnHzy9BZMGxDK6azQ3HZfIDXMOkJ7rpXWci/9v795Zm4rjOA7/UkI7VHFRxEEEJxXEySJaKC4i+C4ELQgd+gKci4uCCIIvQRzETVykdHAVL5ODm6iIqC1Fi41D7NYmtNacb9LnWc+BnJNzIfmcy//GuX0183cUp8dvV+v+i+V6erUb1ucv7K+bz77VzINPNdFu1ezUZF08vvvnyKZ+PbY6G+9E2FzPiY1bfNP95qZPNb0kO3d6ruru9c2voI+ihUeb3+LN6LGtSWJ/hIHrOO4A+rqz9KPmp//xBe47sPD8e31eWa/bV4wm18vS+5810W7V1NFdHva7a8sONPx31GxcpR5mo7AO27GX1nWvs61JYn+EgWpVOe4A+uj+Ffz/58p3X9Zq7XfViUPtevlhrR6+Wq1blw8M5LOHWVPfz/CHGgAAAGBLK786Nffka31cXq+Dk2N17exkXfI4ZSyhpmmv7zW9BAAAAIywM0fGa3H2cP8ZiTDWfxYAAAAABkGoAQAAAAgh1AAAAACEEGoAAAAAQgz3y4SnTxr2EQAAANh154+NVxPFodXpdHpN7zkRAAAAgG3bsgF59AkAAAAghFADAAAAEEKoAQAAAAgh1AAAAACEEGoAAAAAQgg1AAAAACGEGgAAAIAQQg0AAABACKEGAAAAIIRQAwAAABBCqAEAAAAIIdQAAAAAhBBqAAAAAEIINQAAAAAhhBoAAACAEEINAAAAQAihBgAAACCEUAMAAAAQQqgBAAAACCHUAAAAAIQQagAAAABCCDUAAAAAIYQaAAAAgBBCDQAAAEAIoQYAAAAghFADAAAAEEKoAQAAAAgh1AAAAACEEGoAAAAAQgg1AAAAACGEGgAAAIAQQg0AAABACKEGAAAAIIRQAwAAABBCqAEAAAAIIdQAAAAAhBBqAAAAAEIINQAAAAAhhBoAAACAEEINAAAAQAihBgAAACCEUAMAAAAQQqgBAAAACCHUAAAAAIQQagAAAABCCDUAAAAAIYQaAAAAgBBCDQAAAEAIoQYAAAAghFADAAAAEEKoAQAAAAgh1AAAAACEEGoAAAAAQgg1AAAAACGEGgAAAIAQQg0AAABACKEGAAAAIIRQAwAAABBCqAEAAAAIIdQAAAAAhBBqAAAAAEIINQAAAAAhhBoAAACAEEINAAAAQAihBgAAACCEUAMAAAAQQqgBAAAACCHUAAAAAIQQagAAAABCCDUAAAAAIYQaAAAAgBBCDQAAAEAIoQYAAAAghFADAAAAEEKoAQAAAAgh1AAAAACEEGoAAAAAQgg1AAAAACHafaa3BrIUAAAAALijBgAAACCFUAMAAAAQQqgBAAAACCHUAAAAAIQQagAAAABCCDUAAAAAIf4A5tU467Uglh8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1440x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predictor.explain(test_df, row_index=35, class_id=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot above is generated using the [shap](https://github.com/slundberg/shap) library. You can install it with either `pip install shap` or, for *conda* users, `conda install -c conda-forge shap`.  The features in red are causing our model to increase the prediction for the **Survived** class, while features in blue cause our model to *decrease* the prediction for **Survived** (or *increase* the prediction for **Not_Survived**).  \n",
    "\n",
    "From the plot, we see that the predicted softmax probability for `Survived` is **50%**, which is a comparatively much less confident classification than other classifications. Why is this?\n",
    "\n",
    "We see that`Sex=male` is an influential feature that is pushing the prediction lower towards **Not_Survived**, as it was women and children given priority when allocating lifeboats on the Titanic. \n",
    "\n",
    "On the other hand, we also see that this is a First Class passenger (`Pclass=1`) with a higher-than-average `Fare` price of *82.17*.  In the cell below, you'll see that the average `Fare` price is only *32*.  (Moreover, this passenger embarked from Cherbourg, which has been shown to be correlated with survival.)  Such features suggest that this is an upper-class, wealthier passenger and, therefore, more likely to make it onto a lifeboat and survive. We know from history that  crew members were ordered to close gates that lead to the upper decks so the first and second class passengers could be evacuated first. As a result, these \"upper class\" features are pushing our model to increase the classification to **Survived**.   \n",
    "\n",
    "**Thus, there are two opposing forces at play working against each other in this prediction,** which explains why the prediction probability is comparatively nearer to the border than other examples.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32.23080325406759"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df['Fare'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: We choose `class_id=1` in the example above because the **Survived** class of interest has an index position of 1 in the `class_names` list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "['not_Survived', 'Survived']"
      ],
      "text/plain": [
       "['not_Survived', 'Survived']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preproc.get_classes()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now look at the examples for which we were the most wrong (highest loss)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing test: 92 rows x 8 columns\n",
      "----------\n",
      "id:27 | loss:3.31 | true:Survived | pred:not_Survived)\n",
      "\n",
      "----------\n",
      "id:53 | loss:2.84 | true:not_Survived | pred:Survived)\n",
      "\n",
      "----------\n",
      "id:19 | loss:2.52 | true:Survived | pred:not_Survived)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "learner.view_top_losses(val_data=preproc.preprocess_test(test_df), preproc=preproc, n=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example with the highest losses are `row_num={27, 53, 19}`. Why did we get these so wrong?  Let's examine `row_num=53`.  Note that these IDs shown in the `view_top_losses` output are the raw row numbers, not DataFrame indices (or PassengerIDs). So, we need to use `row_num`, not `row_index` here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation for class = Survived (row_num=53): \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABJ4AAAEACAYAAADyR7x+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU1eH//9cNYSdBFgUjEBZBQKCiqFg3am1LFResbdH6bat20bbqR6yidrPaVql1+X1qP2oXq62tWEWt1rpUBbWodcENBRSRVIiKgCZhCwm5vz/OJBkwJCGTm8nyej4e85g7c5c5czJzc+97zjk3iuMYSZIkSZIkqbnlZLsAkiRJkiRJap8MniRJkiRJkpQIgydJkiRJkiQlwuBJkiRJkiRJiTB4kiRJkiRJUiIMniRJkiRJkpQIgydJkiRJkiQlwuBJkiRJkiRJiTB4kiRJkiRJUiIMniRJkiRJkpQIgydJkiRJkqRsyZ/RO9tFSJLBkyRJkiRJaj3yZ6wgf8YpO5h3Mfkz7tuJbd1M/ozfN1fREnIP+TMua8qKhbOL5xfOLv5hcxeoOeVmuwCSJEmSJEmNUjrnF9kuQgJOB+aRPwNK5/wo24VpbgZPkiRJkiRJjZE/ozOlcyqasN4UYF4DS/2Q/Bn3UzrnmaYULVsKZxd3LppVsMM6MXiSJEmSJEmtzRDyZzwKHAisAL5F6ZynyJ9xCXAIpXOOBCB/xkDgd8BhwPvAbOD3wDBK56xIbasr+TN+B3wR2ABcSumcG2teKX/GocDlwFjgQ+D/gKspnROnAqNHgFOBnwK7AnlNeD8LUuvWZXfgfuA+4D9N2DYAhbOLJwDXAhMJ7+Mm4PKiWQVbC2cX/y/QrWhWwbdSyz4BFBbNKihMPb4AmFI0q+Co1OPjgR8BI4B3gZ8VzSr4S2re14EfAjcC5wAlwN47KpfBkyRJkiRJam1OA44DlgC/Am4BRtax3F+Aj4DBQDfgb3UscyLwZeDbwPHA7eTPeJDSOUXkzxgL/BM4BfhH6jUeAD4A/pRavxNwFCHQ2fnWTkCqldSaOuflz7gDuJfSOd9r0raBwtnFvYF/AdcBnweGE8KscuBKQnh2bWrZXqTCqcLZxaOKZhW8AXyG8L4pnF38GeAPhLpaAEwCHiqcXfxO0ayCJ1IvORQoINRXVF/ZDJ4kSZIkSVJrcyOlc14DSA0O/j8fu/pb/oxBwBHACErnlAKlqUG6D99uW49ROufe1PRd5M/4CNgHKAK+A9xB6Zy/p+YvIX/GdcBXqQ2eAGZROqck0zdVOLs4B+hZNKugLO3pkyid816Gmz4a2EJomRQDiwtnF88GZhKCp/nA4MLZxcOBMcBzwJvAZwpnFxcBBwPnpbZ1DvD/Fc0qeDL1+NnC2cW3EuqkOniqAC4smlVQ3lDBDJ4kSZIkSVJr827a9IbU/fZd3PZI3f837bmiBrZVvb3qbQ0DjiB/xglp83OAd9IeV233uElSodPvU9v/es2MzEMnCC2+ilKhU7W3Us9TNKugtHB28XPAkYTg6V/AMuArhFZlZcCrqfWGAZ8qnF08M21bnYAn0x6/25jQCQyeJEmSJElS27QqdT8EWJ42vTOKgJsonfPdepaJKZ0T1zO/QWmh0yRCK63m9g5QWDi7OEoLn4azbWD2CLXB06nA28BvgTeAR9PWKwJuLppVcGU9r1fV2IIZPEmSJEmSpLandM5K8mfMB64gf8bphDGefriTW/k/4HHyZzwIPAjEwChgV0rnPN6MpT2BEPYAfFA4u3hHy40smlWwrAnbv58whtPFhbOLryS0WppFGAC82iPA/xC65C0smlVQVTi7+G3C2Ffnpi13LXBz4eziZ4CnCK2dxgNR0ayC53e2YDlNeDOSJEmSJEmtwclAD2AlYSDsO1LPN6obGKVzFgHTCIHMu8Bq4GZ2fAW6pvo7cDehO9vw1Pbrui3f0QbqUzSroAT4LKFF0/vAQ4Qxqq5OW+xpQg70WNGsguoWS48A+an76m09DHyTMDbUGkK9XAP0akrZojjOqLWYJEmSJElS65A/43OEkKd7pt3jmlvh7OLOwBygtGhWwakNLd9eGDxJkiRJkqS2KX/GPoTxhl4ldC+7HXid0jlfy2q5dqBwdnEusEvRrII12S5LS3GMJ0mSJEmS1Fb1AX4H7A6UAA8A52W1RPUomlVQSei+1mHY4kmSJEmSJEmJcHBxSZIkSZIkJaI9drV7DxiQ7UK0Ae8DA7NdCEmSJEmSmpGZQPPKODtoj13t2t0bSlCU7QJIkiRJktSMzASaX0bZgV3tJEmSJEmSlAiDJ0mSJEmSJCWiQwRP1157LcuXL892MZpk/vz53HXXXdkuhiRJkiRJasVuvvlmFi5cmO1ifEyHCJ4kSZIkSZJ21iWXXMK6deta7fbaAoMnSZIkSZIkJSI32wVoKatWreKBBx6grKyM0aNHM23aNHJzc9m0aRN33303K1eupKqqiiFDhjBt2jTy8/MBeOmll3j88cfZsGEDPXr04IgjjmDChAkAvPjiiyxYsID169ezxx57cMwxx7DLLrt87LVvvfVWRo0axQEHHFDz3PXXX8+UKVMYM2YMDzzwAIsXL6a8vJy+ffsydepUCgsLP7adFStWcNdddzFz5sya56699lqOPfZYhg8fThzHLFiwgBdeeIHNmzczfPhwpk2bRvfu3Zu7OiVJkiRJahM++OAD7r//ft577z3y8vI48sgj2WuvvYDQPW3ChAnsu+++QMgAFi5cyGmnncYf//hHIJy/R1HEscceS69evbjrrrvYf//9efrpp+nSpcs2OcHObm/cuHE15aysrORXv/oVp512GrvtthsAGzZs4JprruHcc88lJyen3vwi3fz581m3bh0nnHACAB999BHXXnstP/7xj8nJyWHz5s089NBDvPnmm0RRxMSJE5kyZQo5Oc3fPqnDtHh69dVXOeWUUzjnnHNYu3YtTzzxBABxHLPPPvtw7rnncu6555Kbm8s///lPALZs2cIDDzzAV77yFS6++GJOP/10Bg4cCMCSJUt48skn+fKXv8wFF1xAYWEhc+fOrfO1x48fz6uvvlrz+IMPPqCkpISRI0cCsMcee3DGGWcwa9Ysxo8fzx133EFlZeVOv8f//Oc/LFmyhFNPPZXzzjuPbt26cf/99+/0diRJkiRJag+2bt3KbbfdxogRIzj//PM56qijmDt3LmvWrGlw3VNPPRWAM888k4svvrgmJFq/fj0bN25k5syZHH/88dx3330Zba9abm4uY8aM2SY/eO211xg6dCg9e/asN7/YWffccw85OTmcffbZnHHGGbz11luJjQ/VYYKnAw44gN69e9O9e3cOO+ywmj9kjx49GDt2LJ07d6Zr164cdthhrFixoma9KIpYvXo1FRUV5OXl1aSOzz//PIcccgi77rorOTk5HHroobz33nt89NFHH3vt0aNHbzPvlVdeYcyYMeTmhgZnEyZMoEePHuTk5PDJT36SysrKRn1ot/f8889zxBFHkJ+fT25uLlOmTOH111+nqqpqp7clSZIkSVJbt3LlSrZs2cIhhxxCp06dGDZsGKNGjWLRokUZbfdTn/oUubm5DB06lFGjRvHaa681S3nHjx+/TdleffVVxo8fDzScXzTW+vXrefPNN5k6dSpdunShZ8+eTJ48OeM62ZEO09UuvelZ7969KSsrA6CiooIHH3yQZcuWsXnzZgDKy8upqqqiS5cunHjiiTz11FPce++9DB48mM997nP079+fkpISHnzwQR5++OGa7cZxTFlZ2ce623Xt2rXmg33IIYewaNEijjnmmJr5Tz31FAsXLqSsrIwoiigvL2fjxo07/R5LSkq4/fbbiaKo5rmcnBzWr19fZ9M7SZIkSZLas7KyMvLz87c5T95ll10oLS1t8ja7detGly5dah6nZwyZGjp0KBUVFaxcuZJevXrx3nvvMXr0aKD+/GJnusiVlJRQVVXFVVddVfNcHMeJ5QYdJnhK/1CVlJSQl5cHhNBn7dq1fPOb36z5o95www01y+65557sueeeVFRU8Nhjj3Hvvfdy2mmnkZ+fz6GHHlrTj7Mh48aN4/HHH6ewsJDKykqGDRsGQFFREQsWLOCrX/0qu+22G1EUccUVV9S5jc6dO1NRUVHzuKqqig0bNtQ8zs/P57jjjmPIkCGNrxhJkiRJktqpvLw8SktLieO4JnwqKSmhX79+wMfPs9evX9/gNjdv3syWLVtqwqeSkpKa3lFN2V66nJwc9t57bxYtWkTPnj0ZNWoUXbt2BRrOL9LVV478/Hw6derEBRdckMiYTtvrMF3tnn32WUpLS9m0aRNPPvlkTV/KLVu2kJubS7du3di0aRPz58+vWWf9+vUsWbKkZpkuXbrUfFAnTZrEv//9b1avXg2ED159TetGjhzJRx99xLx589h7771rtrNlyxZycnLo2bMnVVVVPP7445SXl9e5jX79+lFZWckbb7zB1q1beeKJJ9i6dWvN/EmTJvHYY4/VdOnbsGEDS5YsaXqlSZIkSZLUhg0aNIjOnTuzYMECtm7dyooVK1i6dGlNJjBw4EAWL15MRUUF69at+9g4R7169eLDDz/82Hbnz5/P1q1bKSoq4o033mDvvffOaHvpqrvbpXezg/rzi+0NHDiQoqIiSkpK2Lx5M08++WTNvLy8PEaMGMFDDz1EeXk5cRyzbt26JnXba4wO0+Jp/Pjx/PnPf6asrIy99tqLww47DIDJkyczd+5cfvnLX5KXl8dBBx1UE9bEcczTTz/N3XffTRRFDBw4kGnTpgEwZswYtmzZwp133klJSQldu3ZlxIgRNR+27VUPEvbiiy/y6U9/uub5ESNGsOeee/LrX/+azp07c9BBB9G7d+86t9GtWzeOPvpo7r33XuI45uCDD96mKdzkyZMBat5nz549GTduXE2zPEmSJEmSOpJOnTpx0kkncf/99/Pkk0+Sn5/P9OnT6d+/PwAHHXQQxcXFXHnllQwYMIAJEyawfPnymvWnTJnC3XffTWVlJccccww9e/akV69edOvWjauuuorOnTszbdq0Jm+vrgxh0KBBdOnShbKyspqLkkH9+cX2qvOJ66+/nh49enDwwQezdOnSmvnTp0/nkUce4Te/+Q3l5eX06dOHQw45JLPK3oEojuNENpxF7e4NJShqeBFJkiRJktqMRDOBFStWcNdddzFz5swkX6a1ySg76DBd7SRJkiRJktSyDJ4kSZIkSZKUCLvadWx2tZMkSZIktSdmAs3PrnaSJEmSJElqfQyeJEmSJEmSlAiDJ0mSJEmSJCWiPQZP72e7AG2E9SRJkiRJam88121eGddnexxcXJIkSZIkqbm0peCk1V1ErD22eJIkSZIkSVIrYPAkSZIkSZLUBHPmzGHfffele/fu9O3blxNPPJFly5bVu86aNWs477zzGDlyJN26dWPQoEGcddZZlJSU1Ln8ypUr6du3L1EUEUUR//jHP5J4K4mxq50kSZIkSdKO1Rmc/OEPf+Ab3/gGAMOGDWPt2rWUlpay22678fLLLzNw4MCPrVNeXs7EiRNZvHgxnTt3ZuzYsbz11lusX7+eAw88kH//+9/k5ubWLF9VVcWRRx7JvHnzap677777mDZt2o7Kalc7SZIkSZKktmzLli1ceOGFAHzhC19g+fLlLF68mLy8PFavXs0vfvGLOtd79NFHWbx4MRBaS7300ku88MILAPznP//hzjvv3Gb5K6+8knnz5vGlL30pwXeTLIMnSZIkSZKknfDcc8+xZs0aIARPAAUFBUyePBmABx98sM71qqqqaqajKNrmHuDhhx+umV64cCE/+tGPOOaYYzjzzDOb9w20IIMnSZIkSZKknfDOO+/UTO+222410wMGDADgv//9b53rHXrooQwaNAiAGTNmMHHiRPbbb7+a+atWrQJg48aNnHzyyfTv35+bbrqp2cvfkgyeJEmSJEmSmkFD42j37t2bRx99lBNOOIHevXuzfPlyDj/8cEaMGAFA586dAbjooot44403uOWWW+jfv3/i5U5SbsOLSJIkSZIkqdrgwYNrplevXv2x6SFDhuxw3VGjRjF37tyax5s2baoZiHz06NEAvPzyywBMnz4dgK1bt9Ys/8UvfpHjjz+e2267LdO30SJs8SRJkiRJkrQT9t9/f/r16wdQEyIVFxfzzDPPADB16lQgBEmjR4/muuuuq1n3mWeeYfPmzQBUVlYyc+ZMSktLgdD9rlocx2zYsIENGzbULA+wefNmNm3alOC7a14GT5IkSZIkSTuhS5cuNVeumzt3LsOHD2fMmDGUlZXRv3//miveLV26lKVLl9YMRA5wxRVX0L9/fyZMmMCAAQO44YYbAPj+97/PpEmTAJg/fz5xHNfc5s2bV7P+fffdxz333NNSbzVjBk8JKysri8vKyurv5ClJkiRJktqUb33rW9x6663ss88+FBcXE0UR06dPZ8GCBRQUFOxwvcMOO4yCggKWLVvGpk2bOPDAA7nlllu48sorW7D0LSdqaOArZaY6dMrLy4saWlaSJEmSJLU6bSk4aXXZgy2eJEmSJEmSlAiDJ0mSJEmSJCXC4EmSJEmSJEmJMHiSJEmSJElSIgyeJEmSJEmSlAiDJ0mSJEmSJCXC4EmSJEmSJGnH3s92ARqpVZbT4KkRoiiaGkXR0iiKlkVRdGE9y30hiqI4iqJJdcxbGkXR55ItafvU1PqPomhoFEWboih6KXW7oeVK3X40VP9RFH09iqIP0ur5G2nzvhZF0Zup29datuTtQ4b1vzXt+XtbtuTtQ2P2P1EUfSmKotejKHotiqK/pj3v5z9DGda/n/9m0Ih90DVp9fxGFEUfpc3zO5ChDOvf70CGGlH/Q6IomhdF0YtRFL0SRdFRafMuSq3nOUATNbX+PQdoHo2o/8Ioih5N1f38KIoGpc1rj/v/gUDUXLeysjLKyspozm2mbgMTq4FMxHHsrZ4b0Al4CxgOdAFeBsbWsVwe8ATwDDAp9dzY0tLSuLS0NAaGpbbTKdvvqS3dMqz/ocCibL+HtnxrTP0DXweuq2PdvsDy1H2f1HSfbL+ntnTLpP5T89Zn+z205Vsj638k8GL1ZxvYLXXv5z+L9Z+a9vPfAn+D7ZY/C7gpNe13IIv1n3rsdyDh+gd+C5yZmh4LrEibfhnoiucA2aj/oXgO0BL1fwfwtdT0EcCfU9Pu/xtxq84Jsl2OlrrZ4qlhBwDL4jheHsfxFmAOcFwdy10GzAY2pz1Xs1wcx28Dy1LbU+NlUv/KXGPrvy6fA/4Vx/G6OI4/BP4FTE2onO1VJvWvzDWm/r8J/Cb1GSeO49Wp5/38Zy6T+lfz2Nl90EnAbalpvwOZy6T+lbnG1H8M5KemewPFqenjgDlxHJd7DtBkmdS/MteY+h8LPJaanpc23/2/PsbgqWF7AO+kPV6Zeq5GFEX7AoPjOL6/jnXTfWxdNSiT+gcYlmp++3gURYcmWM72qsH6T/lCqpntnVEUDd7JdbVjmdQ/QLcoip6PouiZKIqOT7Sk7VNj6n8UMCqKogWpep66E+uqfpnUP/j5bw6N/hxHUVRIaNlRfRLidyBzmdQ/+B3IVGPq/xLglCiKVgL/JLQ6a+y6ql8m9Q+eA2SqMfX/MnBCano6kBdFUb9GrqsOxuApQ1EU5QBXA+dluywdUQP1/y4wJI7jicBM4K9RFOXXsZwycx8wNI7jCYRfNG7Jcnk6mvrqvzCO40nAycC1URSNyEYB27lcQnevKYTWBr+LomiXrJaoY6mv/v38t6wZwJ1xHG/NdkE6qLrq3+9A8k4Cbo7jeBBwFPDn1LGpWsaO6t9zgJbxfeDwKIpeBA4HVgH+D1Cd3DE2bBWQ3oJgUOq5annAOGB+FEUrgMnAvVEY4Dp9ubrWVcOaXP+p5s1rAeI4foHQT3lUi5S6/Wio/onjeG0cx+Wph78H9mvsumpQJvVPHMerUvfLgfnAxCQL2w415jO8Erg3juOKVHeKNwhBiJ//zGVS/37+m8fOfI5nsG03L78Dmcuk/v0OZK4x9X868DeAOI6fBroB/Ru5rurX5Pr3HKBZNOYYtDiO4xNSAd8PUs991Jh11fEYPDXsOWBkFEXDoijqQvjHXnNlkDiOS+I47h/H8dA4jocSBrc+No7j59OXi6JoGOFg+NkWLX3b1+T6j6Jo1yiKOgFEUTScUP/LW/4ttGn11j9AFEW7pz08Flicmn4I+GwURX2iKOoDfDb1nBqvyfWfqveuqen+wMHA6y1S6vajwfoH7iG0tqmu51GE/Yyf/8w1uf79/DebxvwNiKJoNGEA2afTnvY7kLkm17/fgWbRmPr/L/BpgCiKxhCCjw9Sy82Ioqir5wBN1uT69xygWTTmGLR/Wgu/i4CbUtPu//UxudkuQGsXx3FlFEXfI3xZOhGuFvJaFEWXAs/HcbzDy9PGcfxa6hKJAA8C37UJ+s7JpP6Bw4BLoyiqAKqAM+I4Xpd8qduPRtb/2VEUHQtUAusIV1kjjuN1URRdRvjHBXCp9b9zMql/YAxwYxRFVYQfGa6I49iTjp3QyPqvPrh6ndC8/PzqX1n9/Gcmk/qPouiT+PnP2E78D55BGEg5TlvX/wEZyqT+8X9AxhpZ/+cRuvieSxjo+uupv8NrURT9jRD2VeI5wE7LpP6jKPIcIEONrP8pwOVRFMWEq4t/N7Wu+399TLTt/yg1t7KyshggLy8vynZZJEmSJElSdnW0nMCudpIkSZIkSUqEwZMkSZIkSZISYfAkSZIkSZKkRBg8SZIkSZIkKREGT5IkSZIkSUqEwVMziqLoW9kuQ0dm/WeX9Z9d1n92Wf/ZZf1nl/WfXdZ/dln/2WX9Z5f1r8YyeGpefvGyy/rPLus/u6z/7LL+s8v6zy7rP7us/+yy/rPL+s8u61+NYvAkSZIkSZKkRERxHO9w5tSpU+M1a9a0YHHatg8++IBdd911m+eqqqoAyMkx40taXfWvlmP9Z5f1n13Wf3ZZ/9ll/WeX9Z9d1n92Wf/ZZf03XVVVFX379uWRRx6Jsl2WllBv8ATUO1MNKysrAyAvLy/LJZEkSZIkSdmWlhN0iODJZjiSJEmSJElKhMGTJEmSJEmSEmHwJEmSJEmSpEQYPEmSJEmSJCkRudkuQHvnoOKSJEmSJKmjssWTJEmSJEmSEmHwJEmSJEmSpEQYPEmSJEmSJCkRBk+SJEmSJElKhMFTS6iKw03NJ46hqirbpZCk7NrqflCSJEmtm1e1S9KaUlj+HpRtgn++AG8WZ7tEbd+wAbDfCCjoC4+9CouKsl0iSWpZPbrCgaNgZAFs2gJ/npftEkmSJGln/OWcbJegRRk8JWn9JthSCV07Q9dcmL8o2yVq+/rnw/CBYbpnV+tUUsfToyuc+XnolAP5PWDhcijdmO1SSZIkSXWyq12S+vSqnd53RPbK0Z688Fbt9MThkBNlryySlA0by2HJyjCdE8HEYdktjyRJklQPg6ck5fUIv0gDDOwTuocpM2+/D+vKwnR+Dxixe3bLI0nZsDAthPeHDUmSJLViBk9Jyomgd4/ax54cNI+Fy2un97NOJXVA6ftB/7dIkiSpFTN4Stouad3tDEmah7/0S+ro3iyuHdepXx4M3S275ZEkSZJ2wOApaenjPE0YCrlWecZeTPulf8wg6N4le2WRpGyoiuGlt2sfG8JLkiSplTIFSVr3LtCtc+30mMHZLU97ULIx/NoPYQytfRxYV1IHlH6xBVvUSpIkqZUyeGoJdrdrfo5vIqmjS+92vPcQ6JqbvbJIkiRJO2Dw1BL69KydNiRpHi84zpOkDm5tGRStDtNdcmH80KwWR5IkSaqLwVNL6N0TotT0nrvDLj3rXVyNsOQd2FgepnfvAwV9s1seScqGbVp/Ds9eOSRJkqQdMHhqCbmdIK9H7WPHJMpcZRW8sqL2sSdckjoiW39KkiSplTN4ainprZwmGpI0i/Sr21mnkjqiRUVQsTVMD9kV+uVltzySJEnSdgyeWsqWytrpD9dnrxztSd+0E6x11qmkDii/B3TuFKa3VML6TdktjyRJkrQdg6eWEMfbhk3pXSPUdOnd6xZap5I6oPT94Gv/hfLKHS8rSZIkZYHBU0vYXAHlFWF60xZY/E52y9Me5PeAkQVhemsVvPR2dssjSdmQPq6TP2pIkiSpFTJ4agnprZ1eWREGxlZm0n/lX7wyBHqS1JHkRNuOb2fLT0mSJLVCBk8t4SO72TW79F/5PdmS1BGNLIC87mF6bRmsWJ3d8kiSJEl1MHhKWlUMJRtrHxuSNI/0Fk+GeZI6Ise5kyRJUhtg8JS0so1hDCKAdz+E4nXZLU97MGxA7RXtSjbCW+9mtzySlA22/JQkSVIbYPCUtPTxnV70xKBZ7Jd2svXS8tCqTJI6kh5dYfSgMF0Vw4teYEGSJEmtk8FT0j7cUDttl7Dm4WC6kjq6fYZBp9S/8LfehdKN9S8vSZIkZYnBU5IqKmHD5jC9tQpeXpHV4rQLXTvDuCG1jxcuz15ZJClbHOdOkiRJbYTBU5LSWzstXgkby7NXlvZifCF0zg3TK94PV3KSpI7G8Z0kSZLURhg8Jemj9bB+M8Sx4zs1l22u4mRrJ0kdUEFfGNgnTG8shyUrs1seSZIkqR5RHNc7MLOjNjfV+s0w/1V45GWYMBQ2ldviqTnkRLBLL3jsFThkLKzflO0SSVLL694FnloCx+wPK9dmuzSSJEnaCWVnfAaAvLy8KMtFaRG52S5Au1NeAU8vgUdfhvweMGUcdOuS7VK1P107w7AB2S6FJGXPwuXQJy/cJEmS1GbkLXgTvvTLn1I6J9tFaREGT81la1UYZ+OBhdClE0zcE3p1y3ap2q8oghx7ikrqwNwPSpIkqQ0weMpUHMPr78D9z8OWChg7CPr0ynap2r+cCDp1iFaJklQ394OSJElqAwyemiqOYfn78M8XYF0pjNoDBuyS7VJ1LJEnXJI6OPeDkiRJauUMnpqieB08tBDefh/23B3GDfbgv6VFUfi1X5I6KveDkiRJagMMnnbG2rIwaPirRWFg6yM+4UF/tji2iaSOzv2gJEmS2gCDp8Yo2wSPL4Jn34RBfQ2ShWcAABJ8SURBVOHTEyC3U7ZL1bFFGPpJ6tjcD0qSJKkNMHiqz+Yt8PRSmL8IdsuHKeOgW+dsl0pgFxNJcj8oSZKkNsDgqS6VW+GFZfCvlyGvGxw2Fnp2y3aplM4uJpI6OveDkiRJagMMntJVVcFr78CDCyEH2H9P6NMr26VSXexiIqmjcz8oSZKkNsDgCSCOYdm78NCLsH4zjBsCA3bJdqlUnyjySoKSOjb3g5IkSWoDDJ5WrQ0tnN77EMYOgcH9PZBvC+xiIqmjcz8oSZKkNqDjBk9rS8MYTm+sgr0GwdRhHsC3JXYxkdTRuR+UJElSG9DxgqfSjfD4Ili4HEYMgKMnQW6nbJdKO8urOUnq6NwPSpIkqQ3oOMHTpi3w1GJYsCR0pzt6EnTtnO1SqansYiKpo3M/KEmSpDag/QdPFZXw/DKYvwj65cHUidCzW7ZLpUzZxURSR+d+UJIkSW1A+w2eqqrg1SL410vQvSscvjf06ZXtUqm52MVEUkfnflCSJEltQPsLnuIY3iyGh1+EyirYfyQM3CXbpVJz85d+SR2d+0FJkiS1Ae0reHpnTQicPlwPE4eHsZwiD8rbJcc2kdTRuR+UJElSG9B+gqeSjXDzo7DfCDh0rAfj7V5kqCipg3M/KEmSpNav/QRPFZXQvQuMHpTtkqgl2MVEUkfnflCSJEltQPsJnsBuBx2Jf2tJHZ37QUmSJLUB7Sx4wl9/Owqv5iSpo3M/KEmSpDagfQVPeBDeYRgySuro3A9KkiSpDWhfwZPdDjoO/9aSOjr3g5IkSWoD2lnwhL/+dhR2MZHU0bkflCRJUhvQzoInLy3dYUT4t5bUsbkflCRJUhvQvoIngE52O+gQosi/tdTePLMUZv4BnvpltkvSNrgflCRJ7dlDL8Klc6BkI9x+Puw9pGVe12PSZte+gqekutodfCGsKd32AH/ez2DALs3/Ws3h9Xfg8jvh1SIo2wRv3bjt/BNnwysrILdTeLxHX/jXpWH6vQ/hB7fCK0XwQQk8PRt277vj15p8Pny4obZu9h8Jt5wTpuc8CRf/Gbp1qV3+5rPhgFGZv0e7mEitW/p+s0cXOHwcXHoy9Oy243WqW++09u/2LY/BnU/B0lVwzP5w1Wn1L//7f8END8LmLfD5/eBnX4GunTMvh/tBSZKUiefehMvnwpvF4Zhiz93hx1+GTwzLdsmCy++En54Mn92nZV+3rRyTtiHtLHhKqKtdBNx0FhwyNrPtbK1qmV+nu+TCMQfAVw6H795Yd5384v/BFw/++PM5OfCp8XDm50NA1WCdRiFoOmh03bMPGAm3X9Ckt1Evu5hIrVv6fvO9D+H/XQPX3Q8Xnlj/OrSBLtMDdoGzpsETi2BzRf3lfXwR3PAA3Pb9sN63fgPX3lt/PTSW+0FJktRUZZvg9F/Dz06BaftDRSU8+2b4cay1HF+sWgt7FbR8edrKMWkb0r6CJ0juCj91XT2oqgrOvD4kxeUVMHZICHRGFoT5Z/8W8rpD0Qfw7Btwy//AxBEwey7c/xxUbA2/fv94BnRrhl+/q+01KNyWvRseb1/u6pOVuupq977wtU+Hk6nqdRuq0x0tk5MTXiyJv4lXc5Jav+rvaUE/+NQEeKM4PP5wPVx2ewhlyivgwL3gD2fVfqer739zP/z1CVhbCgV94fwTwj4T4O334fw/hhaeuZ3g4DFw/ZkQx/DTOXDPM2Hbe/SD674Nowc13/s6ev9w/2oRvPth/fuiuU/Dlw+D0YPD43OODf8bLv5S5uVwPyhJkppqxepwP/2gcN85F6aM33aZOU/CjQ+GnjCfGAazvwaD+sPzy+C0/4UHLwnHaK//F770S7jnB6HVVKbKK2DC2aHhxtSfwq69YcHs8GPmj/8K/1kaWtF/4zNw2mfCOlffE441u+TCwy+Gcv72u/DPF+D3D4fnrzw1tMIHuP3J0CL93XXQLw/OPApOmRLmbX9MWt/rqlHaV/CUZLeDOrcdwZH7wDWnQ24uXDYH/uf38MBPambz9//An86FfYeHoOmy28OJyiOXhdZP37kervsHXHDCx1/z6SXhC70jt54H+43Y8fzq8tZV7p//DX72t7BjuPALMHmvutdtTJ1+9waoimHcEPjRl2HM4JqX4eW3w06jTy848ZPwvaObp9WXXUyk1q/6e7pqLcx7JYRGOVHYT/bsCvN+Hu6fXxae375Z89ABcPdFsFtvuO85OOd3MGnP0HLoqnvCgcOds2DLVnjl7bDe/EUh6H/yCsjvHgL4/B517y8u+lMIqOqyR7+wn673/dFwF+83iuFzE2uXGTcEPiiFjzZA3171b78h7gclSVJT7bl7CFbO/T0cdyDsOwJ26Vk7/6GF4UfAm8+BYQNCy/Xv3Qj3/jD0ajllSlj3T+eGY7Tzp8Oogrpf68gfhePBuhw/GS7/6rbPde8Cb94Ae5wahoQZNiA0+jjtf8Nx1f+dEQKjGb8K72PK+HBc9MhLcNPZcO03YOZNcMrVcNJh8MLV8LcF4djvmSvDa+zaOzQMKdw1jOl0yjUwcRiMH7rtMWlDr6tGMXhqrNN/DbmpwOSg0eELmNMJTjq0dpnvT4dxZ4VxPHp0BaJwonXAyDA/J4K/Ph5OiKpPOM6aFr4UF37h46958BhYen3Ty1xdFdvXyU++HH79z+0Edz0NX70W5l0Gg3etXSYnbd366vSGM+ETQ0PwdMOD8JWr4YlfhBO9Q8bC4z8PJ3BLVoUuJl1z4TtHNf09pb83T7ik1q16v5nXA478BJxzTPjFbN6r8Pp1tQc3B48J96lGkjXf7eMOqN3W9Mnh4Oflt2HqvtC5UziAWV0SfmmrDs+75MKGzbD8XZg4HPbaY8flm/21cGuqKGp4X7RxM/ROC7569wj3mzZDTl7TXxvcD0qSpKbr3QP+/oNwfHXBzeGY6tMT4FenhlDmz/PhrKNrj6X+5xj49f3h+Gtwfzj/eDj6Mph2GQzsA6cdueOuaY/9rOnlrD4ffWkFrC2D844Pzw8bEIaWufdZOGJCOC46cK8wDXDs/vDAC3D2tNDwYfqB4X2WbYTePbcdN+rgMXD43qGr4SeGbXtM2tDrqlHaWfBEct26bj6ntlleta1VoeXQfc/BurLa1/5wA/TqHtbbo1/t86tLoLwSjvxx7TbiONwnUe7tmwhW2z9tcO+vTIG7n4F5i+Drn6573frKlj620/enwx0L4IW34NOfgOEDa+eNK4SZx4VBdr83rUlvZxt2MZFatx3tN5eugj49oW8doUu03T7rb/8OgfY7a8LjDZvD/jUnB34yA66YC0ddGgKsM6bCyYfDYeNC0+eLb4WVa+CoSXDJSaHbcxLvsaF9Uc9usKG8dpkN5eE+r0fm+zD3g5IkKROjB8Gvvx2m3ywOPVl+fBvc+J0QMP3or/DT29NWiMM5beFu0LULzDg0XJjq0pOhU6dkylh9PrpqHbz/Eez1ndp5W6tg8qgwP4pCYFZ9bNS9a+hC1zkVefRIXeBmUwX0yYFHX4Zf3QPL3wuNKDaVw9jBqW2lHZM29LpqlPYVPEFyv/7W1fJnzoLwgb37opD6rlsPo79Tm45G2603YJfwa/zTvwxdRxqyYDGcfNWO5985K1xFrr4yp9/vzHLpz+1MndZ3BYDGlqexr+Mv/VLrVtf+Y1D/EB5V/9q0/fLV9++sgfNugrkXhv1cpxyY8oPa+QP7hGbUEJpHnzgbPjkGhg+Ab38u3D4ogW9cB//3T7iojsG8v//HEJbXZXB/+PcV9b+/HbUqTTd6UBiHavrk8HjxO+GgqH9+/dtuDPeDkiSpuey1B8w4DP70WDi+KOgH5x4LJ9ZxQSoIXc6uuid0ZbvkNnj4pzu+au8hF9b+kLi9Lx4cWlntSPXxzqB+MGRXePZXO14uvTV4ffcVlaFl/nXfhs/vG8Kpr15TOz992YZeV43SvoKnlh7jacPm8OXqlxe6111+R3g+/cO6zYe/U+gL+6NbQz/WfnlQvC6MAfKpOvqHHjoW3vnDzpc1jsOAbJVbw+MtFSGN7ZIbBvV96W04aK9wIjf36TC+yq9OrS3n5i2161ZUhltdO5H/fhAGWps4PKS+NzwI6zfBgaPCth55OXTD27V36Gp3zb1hx9UswROecEmtXV37zYI+cOQEmHUL/PLrYYyn55bBJ0dvG+Rs2pL65So/PL7tCViyMjR9zonC+HmT9gytSvv2Cuvm5sBLy8OvVp8YCr26hQs3dMqpe39x9WnhtrMqt4ZbVRxuWypC1+XcOn7pm3EIfO+34aBq913g6ntDF233g5IkKZveKIZ/vQTHHxiOp1athbufDsdXORGcegRcPhcmDA0/pJVuDMMlHHdgON8863eh98xPvgxf/GW4gNYlJ9X9Wk/Nbno5q4/9Ju0Jed3g1/+Ab30unNu+sSocM+474uM/CG7fICL9/Lxyazhf3jU/bOfRV8I4oWMG1zYgoZGvq0ZpZ8ETLdvi6ZTDw1WZxp0VBs++6Atwy7zaZWu6YaSt9/OvwBV3wWd+Ah+WhbFJTv9M6E/bXJa/D/udV/t4j9NCX9SFV4eA6Gd/CwPu5uSEZPsvM2uvPlC5NSxfbdJ54aRtzZ/D47N/F0KoK78egreZN4UAqkvncKJ3xwW141fNfxW+eyNsLA8tvGYcAuceY4snqaPYUYvJG78TusJNPh+2VIaQ/ZAx27YUHTsIvntUuJJJThSach84qva7/+Ly0LS7dGMIty//amjt9M4HYdtFq8O+6ogJoW9/c+4vrv47zL6r9vEdC2DWCWGsvnfWwEEXhJatg/vDZ/YJr3/8z0Oof8wBcPGJ7gclSVJ25XeHhW/B9Q9AycYw5tNnJ8KlJ4Xji2MPCOdx37guDF+Q1yM0lpg+GW54CNaUwg+/GM4Vf/NtOPSiMA7nJ0c3/No7o/p4MqcTzDkffvgX2PfcEByN3B1+8KW6z73ru+/dA674amj1VF4JUyeGsqf3WKqZbuB11ShRXD3GUN3qndmqrCmFuU+FFFLt3+V31t11RpI6CveDkiRJbVfvk35K6ZxLsl2MltC+WjzBjkfSV/vj31pSR+d+UJIkSa2cw7BLkiRJkiQpEQZPkiRJkiRJSoTBkyRJkiRJkhJh8CRJkiRJkqREGDxJkiRJkiQpEQZPkiRJkiRJSkT7CZ765cE3P5vtUkiSJEmSJCkliuO4vvn1zpQkSZIkSdJOi7JdgJbSflo8SZIkSZIkqVUxeJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIiDJ4kSZIkSZKUCIMnSZIkSZIkJcLgSZIkSZIkSYkweJIkSZIkSVIichuYH7VIKSRJkiRJktTu2OJJkiRJkiRJiTB4kiRJkiRJUiIMniRJkiRJkpQIgydJkiRJkiQlwuBJkiRJkiRJiTB4kiRJkiRJUiL+fx32mrRhmHuqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predictor.explain(test_df, row_num=53, class_id=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a wealthy First Class (`Pclass=1`) female passenger with a very high `Fare` price of 151.55. As mentioned above, such a passenger had a high chance for survival, which explains our model's high prediction for **Survival**. Yet, she did not survive. Upon further investigation, we can understand why.  This particular passenger is **Bess Allison**, a wealthy married 25-year old mother to two toddlers.  When the collision occurred, her and her husband could not locate their nanny (Alice Cleaver) and son (Trevor).  So, Bess, her husband, and her 3-year-old daughter Loraine stayed behind to wait for them instead of evacuating with other First and Second Class passengers with children.  They were last seen standing together smiling on the promenade deck.  All three died with her daughter Loraine being the only child in 1st class and 2nd class who died on the Titanic.  Their son and nanny successfully evacuated and survived.\n",
    "\n",
    "REFERENCE: [https://rt.cto.mil/stpe/](https://rt.cto.mil/stpe/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving and Reloading the Tabular Predictor\n",
    "\n",
    "It is easy to save and reload the predictor for deployment scenarios."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.save('/tmp/titanic_predictor')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "reloaded_predictor = ktrain.load_predictor('/tmp/titanic_predictor/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "['Survived', 'Survived', 'not_Survived', 'not_Survived', 'Survived']"
      ],
      "text/plain": [
       "['Survived', 'Survived', 'not_Survived', 'not_Survived', 'Survived']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reloaded_predictor.predict(test_df)[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating Test Sets Automatically\n",
    "\n",
    "When we evaulated the test set above, we did so manually.  To evaluate a test set automatically,\n",
    "one can invoke the `learner.evaluate` method and supply a preprocessed test set as an argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing test: 92 rows x 8 columns\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "not_Survived       0.85      0.91      0.88        57\n",
      "    Survived       0.84      0.74      0.79        35\n",
      "\n",
      "    accuracy                           0.85        92\n",
      "   macro avg       0.85      0.83      0.83        92\n",
      "weighted avg       0.85      0.85      0.85        92\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[52,  5],\n",
       "       [ 9, 26]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.evaluate(preproc.preprocess_test(test_df), class_names=preproc.get_classes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `learner.evaluate` method is simply an alias to `learner.validate`, which can also accept a dataset as an argument.  If no argument is supplied, metrics will be computed for `learner.val_data`, which was supplied to `get_learner` above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part II:  Regression for Tabular Data\n",
    "\n",
    "We will briefly demonstrate tabular regression in *ktrain* by simply predicting the `age` attribute in the Census dataset available from te UCI Machine Learning repository.  This is the same example used in the [AutoGluon regression example](https://autogluon.mxnet.io/tutorials/tabular_prediction/tabular-quickstart.html#regression-predicting-numeric-table-columns). Let's begin by downloading the dataset from the AutoGluon website."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/tmp/train.csv\r\n"
     ]
    }
   ],
   "source": [
    "import urllib.request\n",
    "urllib.request.urlretrieve('https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv', \n",
    "                           '/tmp/train.csv')\n",
    "!ls /tmp/train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 1: Load and Preprocess Data\n",
    "\n",
    "Make sure you specify `is_regression=True` here as we are predicting a numerical dependent variable (i.e., `age`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing train: 35179 rows x 15 columns\n",
      "\n",
      "The following integer column(s) are being treated as categorical variables:\n",
      "['education-num']\n",
      "To treat any of these column(s) as numerical, cast the column to float in DataFrame or CSV\n",
      " and re-run tabular_from* function.\n",
      "\n",
      "processing test: 3894 rows x 15 columns\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Task is being treated as REGRESSION because either class_names argument was not supplied or is_regression=True. If this is incorrect, change accordingly.\n"
     ]
    }
   ],
   "source": [
    "trn, val, preproc = tabular.tabular_from_csv('/tmp/train.csv', label_columns='age', \n",
    "                                             is_regression=True, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We used `tabular_from_csv` to load the dataset, but let's also quickly load as DataFrame to see it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25</td>\n",
       "      <td>Private</td>\n",
       "      <td>178478</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Tech-support</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>White</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>23</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>61743</td>\n",
       "      <td>5th-6th</td>\n",
       "      <td>3</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Transport-moving</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>46</td>\n",
       "      <td>Private</td>\n",
       "      <td>376789</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Other-service</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>55</td>\n",
       "      <td>?</td>\n",
       "      <td>200235</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>?</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36</td>\n",
       "      <td>Private</td>\n",
       "      <td>224541</td>\n",
       "      <td>7th-8th</td>\n",
       "      <td>4</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>El-Salvador</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age   workclass  fnlwgt   education  education-num       marital-status  \\\n",
       "0   25     Private  178478   Bachelors             13        Never-married   \n",
       "1   23   State-gov   61743     5th-6th              3        Never-married   \n",
       "2   46     Private  376789     HS-grad              9        Never-married   \n",
       "3   55           ?  200235     HS-grad              9   Married-civ-spouse   \n",
       "4   36     Private  224541     7th-8th              4   Married-civ-spouse   \n",
       "\n",
       "           occupation    relationship    race      sex  capital-gain  \\\n",
       "0        Tech-support       Own-child   White   Female             0   \n",
       "1    Transport-moving   Not-in-family   White     Male             0   \n",
       "2       Other-service   Not-in-family   White     Male             0   \n",
       "3                   ?         Husband   White     Male             0   \n",
       "4   Handlers-cleaners         Husband   White     Male             0   \n",
       "\n",
       "   capital-loss  hours-per-week  native-country   class  \n",
       "0             0              40   United-States   <=50K  \n",
       "1             0              35   United-States   <=50K  \n",
       "2             0              15   United-States   <=50K  \n",
       "3             0              50   United-States    >50K  \n",
       "4             0              40     El-Salvador   <=50K  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv('/tmp/train.csv').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 2: Create a Model and Wrap in `Learner`\n",
    "\n",
    "We'll use `tabular_regression_model` to create a regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mlp: a configurable multilayer perceptron with categorical variable embeddings [https://arxiv.org/abs/1604.06737]\n"
     ]
    }
   ],
   "source": [
    "tabular.print_tabular_regression_models()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "done.\n"
     ]
    }
   ],
   "source": [
    "model = tabular.tabular_regression_model('mlp', trn)\n",
    "learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 3: Estimate Learning Rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "simulating training for different learning rates... this may take a few moments...\n",
      "Train for 274 steps\n",
      "Epoch 1/1024\n",
      "274/274 [==============================] - 8s 29ms/step - loss: 1681.9281 - mae: 38.6405\n",
      "Epoch 2/1024\n",
      "274/274 [==============================] - 7s 25ms/step - loss: 1650.8196 - mae: 38.2378\n",
      "Epoch 3/1024\n",
      "274/274 [==============================] - 7s 26ms/step - loss: 677.0598 - mae: 20.7480\n",
      "Epoch 4/1024\n",
      "274/274 [==============================] - 7s 26ms/step - loss: 123.2551 - mae: 8.7116\n",
      "Epoch 5/1024\n",
      "274/274 [==============================] - 7s 26ms/step - loss: 229.2279 - mae: 11.2846\n",
      "Epoch 6/1024\n",
      " 67/274 [======>.......................] - ETA: 6s - loss: 384.0570 - mae: 12.9530\n",
      "\n",
      "done.\n",
      "Visually inspect loss plot and select learning rate associated with falling loss\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxdVbn/8c+TuRmbqemYpnSCDkBLCwVkUGRGqoIIcqUq2isOoHL1Al4vjtcBvf5EBC3DBRQZBJQyFkQQEApNgc60lI7pmDad0szJ8/vj7JbTkjRJc052cvJ9v17ndfZee529n9XT5Mnea++1zN0RERE5lKSwAxARkZ5PyUJERNqlZCEiIu1SshARkXYpWYiISLuULEREpF0pYQcQD0VFRV5WVhZ2GCIivcr8+fO3uXtxa9sSMlmUlZVRXl4edhgiIr2Kma1ta5suQ4mISLuULEREpF1KFiIi0i4lCxERaZeShYiItEvJQkRE2qVkEUNVextYX1UTdhgiIjGXkM9ZdLfyNVX84pnlrNq2l23V9Rw3PJ9rzxzDSaOKwg5NRCQmdGbRRZt31XHx71/jjTVVbKuuB2D+2h185o7XeeCNdSFHJyISGzqzaIO787U/v8Xk4flc+aERB2xbX1VDTkYKVXsbuGnOclKSjJNHFTG1LJ+vnD6KZ5Zs5r8fW8J1jy7i2aVb+I+zxrJlTx2jB2QzND+zw8evb2ohIzX5kPWWbdrNdY8sZEHFLoYXZvLU1aeQlR6br7W5JTKLYnKSxWR/ItJ7WSJOqzplyhQ/3OE+Nu+qoyQ3nQ/9/AU27KwF4PUbzqAkN4PahmbueHkVv3puxQGfufJDI/jeBeMOKFtfVcP5N7/M7rqmA8rv++IJnNzO5an6pma+8qc3ef6drfzskxP5yFEDGJCT8YF6j729gWseePuAso8cOYCTRxWR1y+Vi48bCsDuukYyUpLZWdtAcXY6ZsYzizczd9V27n51DQBHFGext76JyaX5VNc38YWTR/D5u+eRmmzccN5RpCQnceExg8nNSOHqB97mqUWbGNK/H985ZywXHD24/X9YEenxzGy+u09pdZuSxft21jRw7A+fY0j/fvsTRbS05CQamlv2rx8zNI/PnljGhccMJi3lg1f0quub+P7sJayvqsEd3lhTtX8/k4f3Jzs9lco9dQzNz+TciQMZlJfBrJdWMWfJlg/sKz0liQuOHsyoAdlMGJLLEws28WD5epIM7v788Zw8qohfPruc2158r912lhZksq6LHfFHDcpl2abd+9cH52Xwpy+ewLCCTDbsqGV4YSZmOiMR6U2ULDpod10jR3//2f3rV50+kj11jfxp7oF9D1d/ZBSfmjKMYQUdu6QEkctKD85bz4+eWMrehmYA8jNT2VHT2Gr9SaX9ueaM0dz6wnss3bSb6vqmVuv95csnMrWsYP/63FXb+eNra6lpaOKF5ZUA5Gak0NTi1ATHBTh/4iCmHzuY3H6p7NjbQGlhJrkZqeypa6IoO42fPf0On546jIKsNH769DucPKqIddv3sqBiF0cUZ/Gj6RPYUdPAtx5awBurq/bvNysteX/7br5sEudPHKTLWCK9hJJFJ3zn4QU8VF4BwFvfO5P8rDTcnYodtby5bgcbd9bxpVNGkJJ8ePcGtLQ4Le6YGclJxpbddTy3dAu/enY5Rw7M5fJppUwYnEdpQSZJUb9kX125jRdXVJKdnsLyzXuYMCSPkcVZnDV+YJvHamhqoamlhcy0SB/G9up6lmzcTXLQxxJLr7y7jf/62yLWbD/wjOU/zhrD1z4yOibHaG5xHl+wETOYXJrfqWQtIu1TsuiEusZm/jR3LccNz2dSaX6MI0tsLS2OE+kQv+PlVfz4yWVA5Exs7uoqBuVl8KnjhvFg+XpuOO9IcjNSyUpPobG5hXlrqvj50+9w1KBcvnfBOLLSU/jnikr+/PpajhyYy7bqeu57/cAzvEeuOpHjhhe0EomIHA4lCwnFrppGjvnhs+1X7ISpZfmcNqaYXz67ggE56Zw9fiCnjinmzHElMT2OSF+kZCGhWbFlD5+76w0uPm4oN/9jJQAnjCjg9ah+DoBvnz2WU0YXsWTjbm5/aRXrqmq47twjOXfiIH7/4nv0S0vmxJGFfHjsACDyLMtFt726//NPXX0KowZkt3qjgYh0jJKFJKS/L93Ctx56+4Dbk08bU8wPp49neGFWiJGJ9E6HShZx+zPMzO4ys61mtvig8q+b2TtmtsTMfhFVfr2ZrTSz5WZ2dlT5OUHZSjO7Ll7xSu/z0XElLPz+2Tw4cxqnjC7ihBEF/HNFJafd9CK3v7SKpuaW/Q8WikjXxO3MwsxOBaqBe919QlD2YeC7wPnuXm9mA9x9q5mNA+4HjgcGA38HxgS7WgGcCVQA84DL3H3poY6tM4u+qam5hR88vpQ/zv3gNMI/+cQELj9heAhRifQeoZxZuPtLQNVBxVcBP3P3+qDO1qB8OvCAu9e7+2pgJZHEcTyw0t1XuXsD8EBQV+QDUpKT+NHHJ7Dix+dy0eShRD8T+N2/LuaOl1eFF5xIL9fdvYFjgFPM7HUz+6eZTQ3KhwDro+pVBGVtlYu0KS0liV9dcgzv/eQ8Hpg5jbnXnwHAj59cxvy1B//9IiId0d3JIgUoAKYB3wYeshiNCWFmM82s3MzKKysrY7FL6eWSkoxpRxQyMC+DxT84m8KsyJPpiXhTh0i8dXeyqAAe9Yg3gBagCNgADIuqNzQoa6v8A9x9lrtPcfcpxcXFcQleeq/s9BSu+eho5q3ZwU1zllPX2Nz+h0Rkv+5OFn8DPgxgZmOANGAbMBu41MzSzWwEMBp4g0iH9mgzG2FmacClQV2RTrv4uKGMGpDNrS++x3m/eVlnGCKdEM9bZ+8HXgPGmlmFmV0J3AUcEdxO+wAwIzjLWAI8BCwFngG+6u7N7t4EfA2YAywDHgrqinRaZloKz37jVC6aPJRV2/Zy8e9fY28bAzSKyIH0UJ70OS0tzjUPvs3jCzZy5rgSbr+i1TsFRfqcUG6dFempkpKM3142iX8/9QieW7qFF97Z2v6HRPo4JQvps64+YzRjS3K46r75Shgi7VCykD4rKz2F335mEi0OP3xiKS0aGkSkTUoW0qeNKcnhZ5+cyOpte/fPRy4iH6RkIX3eeRMHMbwwkx8+sZTyNXrCW6Q1ShbS52WkJvPoVSeRm5HCj3Q5SqRVShYiQGF2Ov/9sfEsqNjFU4s3hR2OSI+jZCES+MSkIYwtyeGXc5bT2NwSdjgiPYqShUggOcn49tljWbO9hofK17f/AZE+RMlCJMoZRw1gcml/fjlnOcs27Q47HJEeQ8lCJIqZcePHxlPf1MLHfvsKr763LeyQRHoEJQuRgxwzrD8v/MfpJCcZs9/eGHY4Ij2CkoVIK0pyMzh1TDGvvrc97FBEegQlC5E2nDammHVVNSzZuCvsUERCp2Qh0oZzJgwE4J8rNE2viJKFSBuKstM5cmAOzy/TiLQi8Zwp7y4z2xrMinfwtmvNzM2sKFg3M7vZzFaa2UIzmxxVd4aZvRu8ZsQrXpHWnDthEG+u28HOmoawQxEJVTzPLO4Gzjm40MyGAWcB66KKzyUy7/ZoYCZwW1C3ALgROAE4HrjRzPLjGLPIAU4cWYg7PKa7oqSPi1uycPeXgNaG8Pw18B0gerS26cC9wXzcc4H+ZjYIOBt4zt2r3H0H8BytJCCReJkyPJ9jh/Vn1kurSMQpiEU6qlv7LMxsOrDB3RcctGkIED2+QkVQ1lZ5a/ueaWblZlZeWakOSYmNpCTjMyeUsmFnLe9s3hN2OCKh6bZkYWaZwA3Af8dj/+4+y92nuPuU4uLieBxC+qhTRhcB8Mq7eppb+q7uPLMYCYwAFpjZGmAo8KaZDQQ2AMOi6g4NytoqF+k2g/L6MbI4i1dWKllI39VtycLdF7n7AHcvc/cyIpeUJrv7ZmA2cEVwV9Q0YJe7bwLmAGeZWX7QsX1WUCbSrU4aWcS8NVU0NGnocumb4nnr7P3Aa8BYM6swsysPUf0pYBWwErgd+AqAu1cBPwLmBa8fBmUi3erkUYXUNDSzoGJn2KGIhCIlXjt298va2V4WtezAV9uodxdwV0yDE+mkaUcUAjD3ve1MLSsIORqR7qcnuEU6oH9mGiOLs3RmIX2WkoVIBx0zrD8LKnbpeQvpk5QsRDro2GH9qdxTz5rtNWGHItLtlCxEOujMcSWYwWNv6+5t6XuULEQ6aFBePyYOydOESNInKVmIdMIJIwp4e/1O6hqbww5FpFspWYh0wvEjCmloamHBet0VJX2LkoVIJ0wti4yQP3/djpAjEeleShYindA/M43SgkwWb9C83NK3KFmIdNLEoXksrFCykL5FyUKkk44ekkfFjlp27NVUq9J3KFmIdNLEoXkALNKlKOlDlCxEOmnCECUL6XuULEQ6KTcjlRFFWSzUoILShyhZiByG8YNzWbJxd9hhiHQbJQuRwzB+cKSTe2eNOrmlb4jnTHl3mdlWM1scVXaTmb1jZgvN7K9m1j9q2/VmttLMlpvZ2VHl5wRlK83sunjFK9IZ4wfnArBUZxfSR8TzzOJu4JyDyp4DJrj70cAK4HoAMxsHXAqMDz5zq5klm1ky8DvgXGAccFlQVyRURw2KJIt3Nu8JORKR7hG3ZOHuLwFVB5U96+5NwepcYGiwPB14wN3r3X01kbm4jw9eK919lbs3AA8EdUVCVZSdRkFWGiu2KFlI3xBmn8UXgKeD5SHA+qhtFUFZW+UfYGYzzazczMorKyvjEK7I+8yMMSXZLFeykD4ilGRhZt8FmoD7YrVPd5/l7lPcfUpxcXGsdivSpjElOby7pVrTrEqf0O3Jwsw+B1wAXO7v/5RtAIZFVRsalLVVLhK6MSU5VNc3sXFXXdihiMRdtyYLMzsH+A5wobtHT2Q8G7jUzNLNbAQwGngDmAeMNrMRZpZGpBN8dnfGLNKWsQNzANRvIX1CPG+dvR94DRhrZhVmdiVwC5ADPGdmb5vZ7wHcfQnwELAUeAb4qrs3B53hXwPmAMuAh4K6IqEbMyBIFrojSvqAlHjt2N0va6X4zkPU/wnwk1bKnwKeimFoIjGRl5lKSW66OrmlT9AT3CJdsK+TWyTRKVmIdMGYkhze3bqHlhbdESWJTclCpAvGluRQ19jC+h017VcW6cWULES6YExwR9RydXJLglOyEOmC0QOyAd0+K4lPyUKkC7LSUxia348V6uSWBKdkIdJFY0pydGYhCU/JQqSLxpTksKpyL43NLWGHIhI3ShYiXTR6QDYNzS2sq9IdUZK4lCxEuqisKBOAdduVLCRxKVmIdFFpQRYAa7fvDTkSkfhRshDpoqLsNDLTklmry1CSwJQsRLrIzBhemMVaXYaSBKZkIRIDwwsydRlKEpqShUgMDC/MZH1VLc0aUFASlJKFSAwML8yiobmFzbs1xaokpnjOlHeXmW01s8VRZQVm9pyZvRu85wflZmY3m9lKM1toZpOjPjMjqP+umc2IV7wiXTG8MHL7rC5FSaKK55nF3cA5B5VdBzzv7qOB54N1gHOJzLs9GpgJ3AaR5ALcCJwAHA/cuC/BiPQkpQV61kISW9yShbu/BFQdVDwduCdYvgf4eFT5vR4xF+hvZoOAs4Hn3L3K3XcAz/HBBCQSusH9+5GabKxRspAE1d19FiXuvilY3gyUBMtDgPVR9SqCsrbKP8DMZppZuZmVV1ZWxjZqkXYkJxnD8jNZV6XLUJKYQuvgdncHYnbriLvPcvcp7j6luLg4VrsV6bDSwkw9ayEJq7uTxZbg8hLB+9agfAMwLKre0KCsrXKRHmd4QSbrttcQ+TtIJLF0d7KYDey7o2kG8FhU+RXBXVHTgF3B5ao5wFlmlh90bJ8VlIn0OMMLs9hT30TV3oawQxGJuQ4lCzO7xsxyg1/md5rZm2Z2VjufuR94DRhrZhVmdiXwM+BMM3sX+GiwDvAUsApYCdwOfAXA3auAHwHzgtcPgzKRHmf/7bMaI0oSUEoH633B3X9jZmcD+cBngT8Cz7b1AXe/rI1NZ7RS14GvtrGfu4C7OhinSGj2JYt122uYXKo7vCWxdPQylAXv5wF/dPclUWUiAgzNz8QM1ujBPElAHU0W883sWSLJYo6Z5QCaQ1IkSkZqMoNyM/RgniSkjl6GuhI4Fljl7jXBk9Wfj19YIr1TaWGm+iwkIXX0zOJEYLm77zSzfwP+C9gVv7BEeqeywiyNDyUJqaPJ4jagxsyOAa4F3gPujVtUIr1UaWEm26obqK5vCjsUkZjqaLJoCu5Ymg7c4u6/A3LiF5ZI7zQ8mI9b/RaSaDqaLPaY2fVEbpl90sySgNT4hSXSO2mocklUHU0WnwbqiTxvsZnIsBs3xS0qkV6qVA/mSYLqULIIEsR9QJ6ZXQDUubv6LEQOkpuRSkFWmgYUlITT0eE+LgHeAD4FXAK8bmYXxzMwkd6qtCBTl6Ek4XT0OYvvAlPdfSuAmRUDfwcejldgIr1VWWEm89bsCDsMkZjqaJ9F0r5EEdjeic+K9CmlhVls2lVLfVNz2KGIxExHzyyeMbM5wP3B+qeJjBQrIgcZXpBJi0PFjlpGFmeHHY5ITHQoWbj7t83sIuDkoGiWu/81fmGJ9F5lRe+PPqtkIYmio2cWuPsjwCNxjEUkIZQGD+apk1sSySH7Hcxsj5ntbuW1x8x2H+5BzeybZrbEzBab2f1mlmFmI8zsdTNbaWYPmllaUDc9WF8ZbC873OOKdIei7DQy05JZo9tnJYEcMlm4e46757byynH33MM5oJkNAa4Gprj7BCAZuBT4OfBrdx8F7CAy0i3B+46g/NdBPZEey8woLchknR7MkwQS1h1NKUA/M0sBMoFNwEd4/1bce4CPB8vTg3WC7WeYmSZekh5No89Koun2ZOHuG4BfAuuIJIldwHxgp7vvG6qzAhgSLA8B1gefbQrqF3ZnzCKdNbwwk/VVtTS3eNihiMREtycLM8sncrYwAhgMZAHnxGC/M82s3MzKKysru7o7kS4pLcykobmFzbvrwg5FJCbCuAz1UWC1u1e6eyPwKJFbcvsHl6UgMlDhhmB5AzAMINieR+ShwAO4+yx3n+LuU4qLi+PdBpFDKivUHVGSWMJIFuuAaWaWGfQ9nAEsBV4A9o03NQN4LFieHawTbP9HMLeGSI9VWrBvqHJ1cktiCKPP4nUiHdVvAouCGGYB/wl8y8xWEumTuDP4yJ1AYVD+LeC67o5ZpLMG9+9HarIpWUjC6PBDebHk7jcCNx5UvAo4vpW6dURGuxXpNZKTjGH5Gn1WEocGAxSJk5EDslm5tTrsMERiQslCJE5GD8hm9ba9NDa3hB2KSJcpWYjEyeiSbJpaXJeiJCEoWYjEyegBOQC8u0WXoqT3U7IQiZORxdmYwQolC0kAShYicdIvLZmh+f1YsXVP2KGIdJmShUgcjS3JYflmJQvp/ZQsROJo/OA8VlVWU9PQ1H5lkR5MyUIkjsYPzqXFYdkmnV1I76ZkIRJHE4bkAbBk466QIxHpGiULkTgalJdBQVYaSzYc9izEIj2CkoVIHJkZ4wfnslhnFtLLKVmIxNn4wXms2LKHhiYN+yG9l5KFSJyNH5xLY7OzYos6uaX3UrIQiTN1cksiULIQibPhBZlkp6ewZKM6uaX3CiVZmFl/M3vYzN4xs2VmdqKZFZjZc2b2bvCeH9Q1M7vZzFaa2UIzmxxGzCKHKynJGDcol8UbdGYhvVdYZxa/AZ5x9yOBY4BlRKZLfd7dRwPP8/70qecCo4PXTOC27g9XpGvGD8ll2aY9NLdo+njpnbo9WZhZHnAqwRzb7t7g7juB6cA9QbV7gI8Hy9OBez1iLtDfzAZ1c9giXTJxSB61jc3q5JZeK4wzixFAJfB/ZvaWmd1hZllAibtvCupsBkqC5SHA+qjPVwRlBzCzmWZWbmbllZWVcQxfpPOmlhUAUL52R8iRiByeMJJFCjAZuM3dJwF7ef+SEwDu7kCnztfdfZa7T3H3KcXFxTELViQWhub3oyQ3nXmrq8IOReSwhJEsKoAKd389WH+YSPLYsu/yUvC+Ndi+ARgW9fmhQZlIr2FmTCkroHyNkoX0Tt2eLNx9M7DezMYGRWcAS4HZwIygbAbwWLA8G7giuCtqGrAr6nKVSK8xdXg+G3fVsWFnbdihiHRaSkjH/Tpwn5mlAauAzxNJXA+Z2ZXAWuCSoO5TwHnASqAmqCvS60zZ12+xpoohx36g202kRwslWbj728CUVjad0UpdB74a96BE4uyoQblkp6cwb00V05UspJfRE9wi3SQ5yZg8PJ95q3VHlPQ+ShYi3Wjq8HyWb9nDrprGsEMR6RQlC5FutK/fYv463RUlvYuShUg3OnZYf9KSk3h15fawQxHpFCULkW7ULy2Z40cU8M8VGmVAehclC5FudvrYYt7dWq3nLaRXUbIQ6Wanj40MR/Pi8q3t1BTpOZQsRLrZyOJshvTvx4vLdSlKeg8lC5FuZmacNraYV1duo6GpJexwRDpEyUIkBKePKWZvQzPla3ULrfQOShYiIThpVBGpycYL76jfQnoHJQuREGSnp3DSyCKeXryZyPBnIj2bkoVISM4/ehAVO2pZtGFX2KGItEvJQiQkZ48bSGqy8eRCTc8iPZ+ShUhI8jJTOXlUEU8u2qRLUdLjKVmIhOj8iZFLUW+v3xl2KCKHFFqyMLNkM3vLzJ4I1keY2etmttLMHgxm0cPM0oP1lcH2srBiFom1sycMJCstmXteXRN2KCKHFOaZxTXAsqj1nwO/dvdRwA7gyqD8SmBHUP7roJ5IQsjNSOWy40uZvWAjW3fXhR2OSJtCSRZmNhQ4H7gjWDfgI8DDQZV7gI8Hy9ODdYLtZwT1RRLCZSeU0uLw2Nsbww5FpE1hnVn8P+A7wL6xDgqBne7eFKxXAPsmKR4CrAcItu8K6h/AzGaaWbmZlVdWaswd6T1GFmdz7LD+/GX+enV0S4/V7cnCzC4Atrr7/Fju191nufsUd59SXFwcy12LxN2lU4exYks189dqfm7pmcI4szgZuNDM1gAPELn89Bugv5mlBHWGAhuC5Q3AMIBgex6gacYkoXzsmMFkp6fw59fXhR2KxNiD89ZRdt2TlF33JM8s3hx2OIet25OFu1/v7kPdvQy4FPiHu18OvABcHFSbATwWLM8O1gm2/8N1ri4JJis9hU9MGsITizaxvbo+7HAkhv7zkUX7l6996G029tJJr3rScxb/CXzLzFYS6ZO4Myi/EygMyr8FXBdSfCJxNeOkMhqbW7jzldVhhyIxsmLLHgAuPm4oL3/nwzS78/3ZS+J2vG899Da/e2FlXPq+Qk0W7v6iu18QLK9y9+PdfZS7f8rd64PyumB9VLB9VZgxi8TLqAHZnDdxEPe+tpadNQ1hhyMx8PdlWwD4zAmlDCvI5JozxvDs0i08viA+d749+uYGbpqznJY4XHvpSWcWIn3e1z48iur6Ju7WQ3oJYc22vRTnpDO5NB+AL54ygkml/bnh0UWsr6qJ+fHSUpL4wskjSE6K/dMFShYiPchRg3I5c1wJd72ymj11jWGHI120eXc9A3Mz9q+nJidx86WTaHHnqvvms3b73pgez93JSI3Pr3UlC5Ee5usfGcXuuibuemVN2KFIF7g7b63dwVGDcg4oH1aQyXfPH8fiDbu58JZ/xfSGhqYWJyUOZxWgZCHS4xw9tD/njB/IH156j827NARIb7V1Tz176ps4cmDuB7Z95oRSfv9vk9lV28iFt/yLf/9jOT98fGmXzjRaWhx3SE7SmYVIn3HDeUfR1OL8+MmlYYcih+m2F98DYOSA7Fa3nzNhED/5xAQ27KxlzpIt3PWv1Zx204vc+uLh3c3UFPRqpyTrzEKkzygtzOSrp4/iiYWbePldDV/T2zy/bAt3v7qGssJMTh1d1Ga9zxxfygkjCg4o+8Uzy7n+0UWdThjNQbKIR+c2QEr7VUQkDP9+2hH89a0KbvjrIp68+hRyM1LDDkk66Nd/XwHArZcfx6HGPTUz7v/SNOqamslMS6G+qZkbHl3MA/PWk5ORwnfPH9fhYza1RIbaU5+FSB+TkZrMry45ho076w7rL00Jx+MLNrJ4w26+d8E4xg3+YH/FwZKSjMy0yN/t6SnJ/PJTR3PFicO5/eXV/O2tDe18+n3xPrNQshDpwY4bXsC1Z43hyYWb+JPGjeoV7nxlNUcUZTHjxOGH9Xkz43sXjOP4sgKue3QhSzfu7tDn9vdZKFmI9E1fPnUkp48t5gezlzBvTVXY4Ug7KnbUMqUsn5Tkw//1mpqcxC2XTyKvXypX3jOPddvbf4Dv/TML3Q0l0iclJRm/uXQSwwoyuepP83vtQHR9wcqt1WyrrueoQe1ffmrPgJwM7rhiKnvrm7jirtd5aN6h5zvRmYWIkNcvlduvOI66xha+eE+5xo7qod6rrAbguOH5MdnfxKF5zLpiCklJxnceWcj3Zy+hJWrgp8bmlv1nFGu3RZ7R0N1QIn3cqAE5/O7yyXzpnnL+7c7Xue+L08jrpzukepJ94z2VFmTGbJ/Tjijk7988jZ8+vYzbX17NI29uoLq+iWOG9WfV1mr21DdxRHEWqyojyaIgKy1mx46mMwuRXuS0McX84bPHsXzzHq646w2NH9XDrKuqISc9JeZJPCnJuOG8o/iv84+iX1oyAKsqqxmQm86YkmwG5/XjhBEF/M8nJnLamPjMFGqJeDvelClTvLy8POwwROLm2SWb+cp9bzJhSB53zJhCUXZ62CEJcNFtr5Jk8JcvnxTX49Q1NpORmhzz/ZrZfHef0to2nVmI9EJnjR/I7y6fzDubd3PRba/GfPRSOTwbdtRSVpgV9+PEI1G0p9uThZkNM7MXzGypmS0xs2uC8gIze87M3g3e84NyM7ObzWylmS00s8ndHbNIT3T2+IH8+UvT2FnTyKWz5rJk466wQ+rT3J3te+spTNCzvDDOLJqAa919HDAN+KqZjSMyXerz7j4aeJ73p089FxgdvGYCt3V/yCI90+TSfO7/0jSaW5xP/O5Vbn9p1QF3y0j3WV9VS2OzM6R/RvuVe6FuTxbuvsnd33Bwrq4AAA4rSURBVAyW9wDLgCHAdOCeoNo9wMeD5enAvR4xF+hvZoO6OWyRHmvc4FyevuYUThtbzE+eWsZn7pjLBj2L0e2eC6ZQPWlU2wMH9mah9lmYWRkwCXgdKHH3TcGmzUBJsDwEWB/1sYqg7OB9zTSzcjMrr6zUKJ3StxRmpzPrs8fxi4uOZlHFLs759Us8Mr9C40l1o/teX8voAdkcURT/PoswhJYszCwbeAT4hrsfMPiJR/6Hd+p/ubvPcvcp7j6luDg+t46J9GRmxiVTh/HMN07lyEE5XPuXBXzytld5a92OsENLeG+u28Gqyr1MHJp3yFFme7NQkoWZpRJJFPe5+6NB8ZZ9l5eC961B+QZgWNTHhwZlItKKYQWZPDDzRH76yYls2FHLJ297lW888JYuTcVBc4uzaVctM++N3Kr/ldNHhhxR/HT7E9wWSbt3Asvc/X+jNs0GZgA/C94fiyr/mpk9AJwA7Iq6XCUirUhOMi47vpTzJg7i1hdX8n+vrGHOki189sThfOq4oYwuyWl/J3KA5hbnrXU7WLppN2+srmL55j1s2FlLTUMzSQYPzpzGqAGJ++/a7Q/lmdmHgJeBRUBLUHwDkX6Lh4BSYC1wibtXBcnlFuAcoAb4vLsf8ok7PZQncqANO2v5nyeXMWfJZppanPGDczlv4iAuPGYww2I4NEWicneufWgBjwbzS2SlJXPiyCKG9M9gQG4GRw7M4YyjStrZS893qIfy9AS3SB+yvbqeh+dX8OzSLcxfG+nLmFTan+nHDOa0sQMoK8xM2Gvuh2tnTQPXP7qIpxdv5pTRRVx71liOSdC+CSULEfmAih01PL5gE7MXbGTZpsg9JqUFmZw7YSCnjS1m0rD8/eMQ9TXbq+t5evFmZi/YyBurI3OIXHD0IG6+dBJJcRrVtSdQshCRQ1q5dQ9zV1XxxMLIL8cWj/R7jCnJYfzgXHbXNjIwL4PRJTkMzstgTEkOQ/r36zW/ON0d98icD1V7G9iyu468fqn7h/Neumk3qyr3sruukRfe2co7m/cAkJacxKljivj8ySM4OUGfn4imZCEiHVZd38Rr721nYcVO3l6/kwXrd5KUZDQ0tVDT0Ly/XkZqEkcUZTNqQOQ1sjjyXlqQSb+0ZNydhuYWon/FtDamUUuLs7O2kX15p3JPPbWNzQzIyaC+qZm99c28u3UPTc1Oc4vT7JH3FneamiPv7lBamEldYzO7axup2FnL2m01rNpWTXVdE5XV9TQ2d+x33dSyfE4cWcSJRxRy3PB80lL6zhB6ShYictjcHTOjucXZVl3P+qoaVmypZuXWat6rjLwffFtuSpKRkmzUNbYcUF6UnUZpQSYFWWls2lVHkhkrt1ZT29hMrJXkpnPkwFzyM1Mpyk4nKz2F5CQjMy2Z4YVZ7KlrjCSfFmd4YRZHD80jLSWJ1C5Mh9rbHSpZaPIjETmkfR25yUlGSW4GJbkZTCkrOKBObUMz71VGkseGnbVsr26gvqmZgbkZmBlJZjQ1t7BxVy3vbd3L+qpaMtOTSU9J4mPHDGL0gBySkgx3pzgnnbTkJLbuqadfajJmMLokh4LMNJKSInEkm0Xek4ykJKOp2Vm7fS8ZqclkpaUwIDc9lJFZE5mShYh0Wb+0ZCYMyWPCkLzQYojXDHES0XfPt0REpMOULEREpF1KFiIi0i4lCxERaZeShYiItEvJQkRE2qVkISIi7VKyEBGRdiXkcB9mVklkTgyAPGDXIZajy4qAbYdxyOh9dLZOa+UHl7XXhujlw23DoWLsSJ1YtEPfRfsxdqROT/kuDhVje9s724bodX0XnYsxent/d299XurIaIyJ+wJmHWr5oLLyrh6js3VaKz+4rL02HNSew2pDT2iHvoue045YfBcdaUes2nCI2PVddPG72PfqC5ehHm9nObosFsfobJ3Wyg8ua68NHY2hPWG3IxHa0NEY2hN2O2LRho7sJ1ZtiF7Xd9HxWDq8PSEvQx0uMyv3NkZc7C0SoQ2QGO1IhDZAYrQjEdoA4bajL5xZdMassAOIgURoAyRGOxKhDZAY7UiENkCI7dCZhYiItEtnFiIi0i4lCxERaZeShYiItEvJogPM7BQz+72Z3WFmr4Ydz+EysyQz+4mZ/dbMZoQdz+Ews9PN7OXg+zg97Hi6wsyyzKzczC4IO5bDYWZHBd/Dw2Z2VdjxHC4z+7iZ3W5mD5rZWWHHczjM7Agzu9PMHo7XMRI+WZjZXWa21cwWH1R+jpktN7OVZnbdofbh7i+7+5eBJ4B74hlvW2LRDmA6MBRoBCriFWtbYtQGB6qBDEJoA8SsHQD/CTwUnygPLUY/F8uCn4tLgJPjGW9bYtSOv7n7l4AvA5+OZ7ytiVEbVrn7lXGNM9HvhjKzU4n8crnX3ScEZcnACuBMIr9w5gGXAcnATw/axRfcfWvwuYeAK919TzeFv18s2hG8drj7H8zsYXe/uLviD+KNRRu2uXuLmZUA/+vul3dX/PvEqB3HAIVEkt42d3+ie6KPiNXPhZldCFwF/NHd/9xd8e8T45/vXwH3ufub3RQ+wXFj2Ya4/VynxGOnPYm7v2RmZQcVHw+sdPdVAGb2ADDd3X8KtHpJwMxKgV1hJAqITTvMrAJoCFab4xdt62L1XQR2AOnxiLM9MfouTgeygHFArZk95e4t8Yw7Wqy+C3efDcw2syeBbk8WMfouDPgZ8HR3JwqI+c9F3CR8smjDEGB91HoFcEI7n7kS+L+4RXR4OtuOR4HfmtkpwEvxDKwTOtUGM/skcDbQH7glvqF1Sqfa4e7fBTCzzxGcLcU1uo7p7HdxOvBJIkn7qbhG1jmd/bn4OvBRIM/MRrn77+MZXAd19rsoBH4CTDKz64OkElN9NVl0mrvfGHYMXeXuNUSSXq/l7o8SSXoJwd3vDjuGw+XuLwIvhhxGl7n7zcDNYcfRFe6+nUifS9wkfAd3GzYAw6LWhwZlvU0itCMR2gCJ0Y5EaAMkRjt6XBv6arKYB4w2sxFmlgZcCswOOabDkQjtSIQ2QGK0IxHaAInRjp7XhsMdG723vID7gU28f7volUH5eUTuNngP+G7YcfaFdiRCGxKlHYnQhkRpR29pQ8LfOisiIl3XVy9DiYhIJyhZiIhIu5QsRESkXUoWIiLSLiULERFpl5KFiIi0S8lCegQzq+6GY1zYwaHDY3nM083spMP43CQzuzNY/pyZ9YhxsMys7OChtFupU2xmz3RXTNI9lCwkoQRDO7fK3We7+8/icMxDjbF2OtDpZAHcQC8dr8jdK4FNZhbKHBcSH0oW0uOY2bfNbJ6ZLTSzH0SV/83M5pvZEjObGVVebWa/MrMFwIlmtsbMfmBmb5rZIjM7Mqi3/y90M7vbzG42s1fNbJWZXRyUJ5nZrWb2jpk9Z2ZP7dt2UIwvmtn/M7Ny4Boz+5iZvW5mb5nZ382sJBh2+svAN83sbYvMuFhsZo8E7ZvX2i9UM8sBjnb3Ba1sKzOzfwT/Ns8HQ+djZiPNbG7Q3h+3dqZmkZn5njSzBWa22Mw+HZRPDf4dFpjZG2aWExzn5eDf8M3Wzo7MLNnMbor6rv49avPfgG6fa0TiKOxHyPXSy90BqoP3s4BZgBH5Y+YJ4NRgW0Hw3g9YDBQG6w5cErWvNcDXg+WvAHcEy58DbgmW7wb+EhxjHJG5AwAuJjLcdhIwkMi8GRe3Eu+LwK1R6/m8P5nYF4FfBcvfB/4jqt6fgQ8Fy6XAslb2/WHgkaj16LgfB2YEy18A/hYsPwFcFix/ed+/50H7vQi4PWo9D0gDVgFTg7JcIqNRZwIZQdlooDxYLgMWB8szgf8KltOBcmBEsD4EWBT2/yu9YvfSEOXS05wVvN4K1rOJ/LJ6CbjazD4RlA8LyrcTmcjpkYP2s28Y8/lE5lxozd88Mo/EUovMvAfwIeAvQflmM3vhELE+GLU8FHjQzAYR+QW8uo3PfBQYF5lvB4BcM8t29+gzgUFAZRufPzGqPX8EfhFV/vFg+c/AL1v57CLgV2b2c+AJd3/ZzCYCm9x9HoC774bIWQhwi5kdS+Tfd0wr+zsLODrqzCuPyHeyGtgKDG6jDdILKVlIT2PAT939DwcURiba+ShworvXmNmLRKYkBahz94Nn/qsP3ptp+/95fdSytVHnUPZGLf+WyDSvs4NYv9/GZ5KAae5ed4j91vJ+22LG3VeY2WQiA9T92MyeB/7aRvVvAluITP+aBLQWrxE5g5vTyrYMIu2QBKE+C+lp5gBfMLNsADMbYmYDiPzVuiNIFEcC0+J0/H8BFwV9FyVEOqg7Io/35xuYEVW+B8iJWn+WyMxsAAR/uR9sGTCqjeO8SmS4aoj0CbwcLM8lcpmJqO0HMLPBQI27/wm4CZgMLAcGmdnUoE5O0GGfR+SMowX4LJG5nw82B7jKzFKDz44JzkggciZyyLumpHdRspAexd2fJXIZ5TUzWwQ8TOSX7TNAipktIzJf8tw4hfAIkWGilwJ/At4EdnXgc98H/mJm84FtUeWPA5/Y18ENXA1MCTqEl9LK7Gbu/g6RKT5zDt5GJNF83swWEvklfk1Q/g3gW0H5qDZingi8YWZvAzcCP3b3BuDTRKbbXQA8R+Ss4FZgRlB2JAeeRe1zB5F/pzeD22n/wPtncR8GnmzlM9JLaYhykYPs60OwyLzGbwAnu/vmbo7hm8Aed7+jg/UzgVp3dzO7lEhn9/S4BnnoeF4Cprv7jrBikNhSn4XIBz1hZv2JdFT/qLsTReA24FOdqH8ckQ5pA3YSuVMqFGZWTKT/RokigejMQkRE2qU+CxERaZeShYiItEvJQkRE2qVkISIi7VKyEBGRdilZiIhIu/4/z61WnQB1gjAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find(show_plot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP 4: Train the Model\n",
    "\n",
    "According to our final validation MAE (see below), our age predictions are only off about **~7 years**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "early_stopping automatically enabled at patience=5\n",
      "reduce_on_plateau automatically enabled at patience=2\n",
      "\n",
      "\n",
      "begin training using triangular learning rate policy with max lr of 0.001...\n",
      "Train for 275 steps, validate for 122 steps\n",
      "Epoch 1/1024\n",
      "275/275 [==============================] - 11s 39ms/step - loss: 411.0144 - mae: 14.8399 - val_loss: 108.4525 - val_mae: 8.2879\n",
      "Epoch 2/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 116.3624 - mae: 8.4576 - val_loss: 102.6719 - val_mae: 8.0353\n",
      "Epoch 3/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 112.9161 - mae: 8.3066 - val_loss: 100.8348 - val_mae: 7.9844\n",
      "Epoch 4/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 111.2987 - mae: 8.2026 - val_loss: 97.9699 - val_mae: 7.7425\n",
      "Epoch 5/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 109.3430 - mae: 8.1120 - val_loss: 95.7590 - val_mae: 7.6947\n",
      "Epoch 6/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 107.6256 - mae: 8.0252 - val_loss: 95.1659 - val_mae: 7.5768\n",
      "Epoch 7/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 107.1517 - mae: 8.0267 - val_loss: 94.3338 - val_mae: 7.5559\n",
      "Epoch 8/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 106.7320 - mae: 7.9814 - val_loss: 94.3334 - val_mae: 7.5357\n",
      "Epoch 9/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 106.7105 - mae: 7.9865 - val_loss: 94.0436 - val_mae: 7.5332\n",
      "Epoch 10/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 105.0565 - mae: 7.9049 - val_loss: 94.0949 - val_mae: 7.5426\n",
      "Epoch 11/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 105.6540 - mae: 7.9441 - val_loss: 93.6455 - val_mae: 7.5383\n",
      "Epoch 12/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 105.2223 - mae: 7.9144 - val_loss: 93.8997 - val_mae: 7.5404\n",
      "Epoch 13/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 105.0021 - mae: 7.9089 - val_loss: 93.5568 - val_mae: 7.5250\n",
      "Epoch 14/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 105.1489 - mae: 7.9176 - val_loss: 94.2954 - val_mae: 7.5771\n",
      "Epoch 15/1024\n",
      "273/275 [============================>.] - ETA: 0s - loss: 104.6181 - mae: 7.8825\n",
      "Epoch 00015: Reducing Max LR on Plateau: new max lr will be 0.0005 (if not early_stopping).\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 104.7387 - mae: 7.8875 - val_loss: 93.6825 - val_mae: 7.4777\n",
      "Epoch 16/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 103.6717 - mae: 7.8581 - val_loss: 92.8922 - val_mae: 7.4872\n",
      "Epoch 17/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 103.1032 - mae: 7.8318 - val_loss: 92.6652 - val_mae: 7.4591\n",
      "Epoch 18/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 103.3571 - mae: 7.8300 - val_loss: 92.6492 - val_mae: 7.4712\n",
      "Epoch 19/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 102.9795 - mae: 7.8306 - val_loss: 92.6980 - val_mae: 7.4230\n",
      "Epoch 20/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 102.9318 - mae: 7.8292 - val_loss: 92.5345 - val_mae: 7.4105\n",
      "Epoch 21/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 103.0119 - mae: 7.8332 - val_loss: 92.7922 - val_mae: 7.4064\n",
      "Epoch 22/1024\n",
      "269/275 [============================>.] - ETA: 0s - loss: 102.2146 - mae: 7.7910\n",
      "Epoch 00022: Reducing Max LR on Plateau: new max lr will be 0.00025 (if not early_stopping).\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 102.1557 - mae: 7.7870 - val_loss: 93.0830 - val_mae: 7.5391\n",
      "Epoch 23/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 102.1588 - mae: 7.7912 - val_loss: 92.6078 - val_mae: 7.4737\n",
      "Epoch 24/1024\n",
      "272/275 [============================>.] - ETA: 0s - loss: 101.5359 - mae: 7.7678\n",
      "Epoch 00024: Reducing Max LR on Plateau: new max lr will be 0.000125 (if not early_stopping).\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 101.7744 - mae: 7.7765 - val_loss: 92.8352 - val_mae: 7.5266\n",
      "Epoch 25/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 101.2561 - mae: 7.7520 - val_loss: 92.2433 - val_mae: 7.4054\n",
      "Epoch 26/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 101.2565 - mae: 7.7492 - val_loss: 92.1415 - val_mae: 7.4383\n",
      "Epoch 27/1024\n",
      "275/275 [==============================] - 10s 36ms/step - loss: 101.9846 - mae: 7.7632 - val_loss: 92.1260 - val_mae: 7.4596\n",
      "Epoch 28/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 101.0920 - mae: 7.7495 - val_loss: 91.9819 - val_mae: 7.4022\n",
      "Epoch 29/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 100.7677 - mae: 7.7300 - val_loss: 91.7970 - val_mae: 7.3984\n",
      "Epoch 30/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 100.7598 - mae: 7.7330 - val_loss: 91.9531 - val_mae: 7.4084\n",
      "Epoch 31/1024\n",
      "269/275 [============================>.] - ETA: 0s - loss: 101.6460 - mae: 7.7700\n",
      "Epoch 00031: Reducing Max LR on Plateau: new max lr will be 6.25e-05 (if not early_stopping).\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 101.8179 - mae: 7.7705 - val_loss: 91.9712 - val_mae: 7.4199\n",
      "Epoch 32/1024\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 100.8309 - mae: 7.7345 - val_loss: 91.9763 - val_mae: 7.3991\n",
      "Epoch 33/1024\n",
      "272/275 [============================>.] - ETA: 0s - loss: 100.7709 - mae: 7.7300\n",
      "Epoch 00033: Reducing Max LR on Plateau: new max lr will be 3.125e-05 (if not early_stopping).\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 100.8522 - mae: 7.7294 - val_loss: 91.8345 - val_mae: 7.4091\n",
      "Epoch 34/1024\n",
      "269/275 [============================>.] - ETA: 0s - loss: 100.5609 - mae: 7.7150Restoring model weights from the end of the best epoch.\n",
      "275/275 [==============================] - 10s 35ms/step - loss: 100.5432 - mae: 7.7158 - val_loss: 91.8488 - val_mae: 7.3933\n",
      "Epoch 00034: early stopping\n",
      "Weights from best epoch have been loaded into model.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f67e06dbeb8>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.autofit(1e-3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "[('mae', 7.398410168683522)]"
      ],
      "text/plain": [
       "[('mae', 7.398410168683522)]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.validate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See the [House Price Prediction notebook](https://github.com/amaiya/ktrain/blob/master/examples/tabular/HousePricePrediction-MLP.ipynb) for another regression example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}