examples/tips_and_tricks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Poutyne's Tips and Tricks\n",
    "\n",
    "Poutyne offers a variety of tools for fine-tuning the information generated during the training, such as colouring the training update message, a progress bar, multi-GPUs, user callbacks interface and a user naming interface for the metrics' names. \n",
    "\n",
    "Let's install the latest version of Poutyne and colorama (if they are not already), and import all the needed packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade poutyne\n",
    "%pip install --upgrade colorama\n",
    "%matplotlib inline\n",
    "import os\n",
    "import math\n",
    "import shutil\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.metrics import roc_auc_score\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torch.utils.data import random_split, DataLoader\n",
    "from torchvision import transforms, utils\n",
    "from torchvision.datasets.mnist import MNIST\n",
    "\n",
    "from poutyne import (\n",
    "    set_seeds,\n",
    "    Model,\n",
    "    ModelCheckpoint,\n",
    "    CSVLogger,\n",
    "    Callback,\n",
    "    ModelBundle,\n",
    "    Metric,\n",
    "    SKLearnMetrics,\n",
    "    plot_history,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameters, Dataset and Network\n",
    "\n",
    "In this section, we setup the hyperparameters, dataset and network we will use throughout these tips and tricks. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training Constants\n",
    "\n",
    "Now, let's set our training constants. We first have the CUDA device used for training if one is present. Second, we set the `train_split` to 0.8 (80%) to use 80% of the dataset for training and 20% for testing the trained model. Third, we set the number of classes (i.e. one for each digit). Finally, we set the batch size (i.e. the number of elements to see before updating the model), the learning rate for the optimizer, and the number of epochs (i.e. the number of times we see the full dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "cuda_device = 0\n",
    "device = torch.device(f\"cuda:{cuda_device}\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "train_split_percent = 0.8\n",
    "\n",
    "num_classes = 10\n",
    "\n",
    "batch_size = 32\n",
    "learning_rate = 0.1\n",
    "num_epochs = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Poutyne, as we will see in the following sections, you can define your own loss functions and optimizers. However, we can also pass magic strings to use PyTorch's standard optimizers and loss functions. Furthermore, for the optimizer, we can also use a dictionary to set other parameters as the learning rate, for instance, if we don't want the default learning rate.\n",
    "\n",
    "Here, we initialize the dictionary for our optimizer as well as the string for our loss function. We thus use SGD with the specified learning rate and the cross-entropy loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = dict(optim='sgd', lr=learning_rate)  # Could be 'sgd' if we didn't need to change the learning rate.\n",
    "loss_function = 'cross_entropy'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the Dataset\n",
    "\n",
    "The following code helps load the MNIST dataset and creates the PyTorch DataLoaders that split our datasets into batches. Then, the train DataLoader shuffles the examples of the training dataset to draw the examples without replacement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_train_dataset = MNIST('./datasets', train=True, download=True, transform=transforms.ToTensor())\n",
    "test_dataset = MNIST('./datasets', train=False, download=True, transform=transforms.ToTensor())\n",
    "\n",
    "num_data = len(full_train_dataset)\n",
    "train_length = int(math.floor(train_split_percent * num_data))\n",
    "valid_length = num_data - train_length\n",
    "\n",
    "train_dataset, valid_dataset = random_split(\n",
    "    full_train_dataset,\n",
    "    [train_length, valid_length],\n",
    "    generator=torch.Generator().manual_seed(42),\n",
    ")\n",
    "\n",
    "train_loader = DataLoader(train_dataset, batch_size=batch_size, num_workers=2, shuffle=True)\n",
    "valid_loader = DataLoader(valid_dataset, batch_size=batch_size, num_workers=2)\n",
    "test_loader = DataLoader(test_dataset, batch_size=batch_size, num_workers=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(48000, 12000)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_dataset), len(valid_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initializing the Network\n",
    "\n",
    "We create a fonction to initialize a simple convolutional neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_network():\n",
    "    return nn.Sequential(\n",
    "        nn.Conv2d(in_channels=1, out_channels=16, kernel_size=3, padding=1),\n",
    "        nn.ReLU(),\n",
    "        nn.MaxPool2d(2),\n",
    "        nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, padding=1),\n",
    "        nn.ReLU(),\n",
    "        nn.MaxPool2d(2),\n",
    "        nn.Dropout(0.25),\n",
    "        nn.Flatten(),\n",
    "        nn.Linear(32 * 7 * 7, 128),\n",
    "        nn.ReLU(),\n",
    "        nn.Dropout(0.5),\n",
    "        nn.Linear(128, num_classes),\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vanilla Usage\n",
    "\n",
    "The following code trains our network in the simplest way possible with Poutyne. We use the accuracy and F1-score metrics so that we can see the performance during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.06s \u001b[35mloss:\u001b[94m 0.378701\u001b[35m acc:\u001b[94m 87.714583\u001b[35m fscore_macro:\u001b[94m 0.876223\u001b[35m val_loss:\u001b[94m 0.098711\u001b[35m val_acc:\u001b[94m 96.983333\u001b[35m val_fscore_macro:\u001b[94m 0.969770\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.11s \u001b[35mloss:\u001b[94m 0.133642\u001b[35m acc:\u001b[94m 95.958333\u001b[35m fscore_macro:\u001b[94m 0.959268\u001b[35m val_loss:\u001b[94m 0.068319\u001b[35m val_acc:\u001b[94m 97.941667\u001b[35m val_fscore_macro:\u001b[94m 0.979237\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m7.98s \u001b[35mloss:\u001b[94m 0.096036\u001b[35m acc:\u001b[94m 97.106250\u001b[35m fscore_macro:\u001b[94m 0.970866\u001b[35m val_loss:\u001b[94m 0.052802\u001b[35m val_acc:\u001b[94m 98.358333\u001b[35m val_fscore_macro:\u001b[94m 0.983536\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m7.95s \u001b[35mloss:\u001b[94m 0.081676\u001b[35m acc:\u001b[94m 97.493750\u001b[35m fscore_macro:\u001b[94m 0.974809\u001b[35m val_loss:\u001b[94m 0.048693\u001b[35m val_acc:\u001b[94m 98.566667\u001b[35m val_fscore_macro:\u001b[94m 0.985642\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.41s \u001b[35mloss:\u001b[94m 0.073082\u001b[35m acc:\u001b[94m 97.731250\u001b[35m fscore_macro:\u001b[94m 0.977137\u001b[35m val_loss:\u001b[94m 0.042849\u001b[35m val_acc:\u001b[94m 98.700000\u001b[35m val_fscore_macro:\u001b[94m 0.986997\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.97s \u001b[35mtest_loss:\u001b[94m 0.034079\u001b[35m test_acc:\u001b[94m 98.910000\u001b[35m test_fscore_macro:\u001b[94m 0.988960\u001b[0m            \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],  # Displayed for each batch and at the end of the epoch.\n",
    "    epoch_metrics=['f1'],  # Displayed only at the end of the epoch.\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "history = model.fit_generator(train_loader, valid_loader, epochs=num_epochs)\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training methods in Poutyne returns a list of dictionaries containing the metrics output during training. Using Poutyne, it is then possible to plot this training history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlR0lEQVR4nO3deXiU9b338fc3C4SsJCFhSYAAQZBdCFZBqbvUvcpiT2177GJbW9dTu5w+T33anqW2PdbaY09rj+fY7VgCCmrdj9qK1iUTCDsVkGWSAAkQAgESsvyeP2aCgQ6QhJncs3xe18VlMnNn5pvR6+PNb373Z8w5h4iIxL4krwcQEZHwUKCLiMQJBbqISJxQoIuIxAkFuohInEjx6okHDRrkSkpKvHp6EZGYVFlZucc5VxDqPs8CvaSkBJ/P59XTi4jEJDPbfrL7tOQiIhInFOgiInFCgS4iEic8W0MPpbW1lerqapqbm70eJaLS0tIoLi4mNTXV61FEJI5EVaBXV1eTlZVFSUkJZub1OBHhnGPv3r1UV1czatQor8cRkTgSVUsuzc3N5Ofnx22YA5gZ+fn5cf+3EBHpe1EV6EBch3mnRPgdRaTvRV2gi4jEq44Oxz8/t551tY0ReXwFehf79+/n5z//OQC1tbXMmzfP44lEJJ68u3Ufv1q+lfd3H4zI4yvQu+ga6MOGDWPJkiUeTyQi8WSxz09WWgofmzQ0Io8fVbtcvPbNb36TLVu2MG3aNMaOHcuGDRtYu3Ytjz/+OMuWLePQoUNs2rSJr33taxw9epTf/va39O/fn+eff568vDy2bNnCV77yFerr60lPT+dXv/oV48eP9/rXEpEocKC5lefX7uSm6cWkpSZH5Dm6Fehmdg/wecABa4BbnXN/s03DzG4ClgAznXNnVNTy3WfXsb72wJk8xN+YMCyb+6+deNL7f/CDH7B27VqqqqrYtm0b11xzzbH71q5dy8qVK2lubqa0tJQHHniAlStXcs899/Cb3/yGu+++m9tuu41f/OIXjB07lnfffZfbb7+d1157Lay/g4jEpmdX1dLc2sHCmcMj9hynDXQzKwLuBCY4546YWTlwM/D4CcdlAXcB70ZgTs9dfPHFZGVlkZWVRU5ODtdeey0AkydPZvXq1TQ1NfGXv/yF+fPnH/uZlpYWr8YVkShTXuFn/JAsJhflROw5urvkkgIMMLNWIB2oDXHM94EHgPvCMdipzqS90L9//2NfJyUlHfs+KSmJtrY2Ojo6GDhwIFVVVR5NKCLRauOuA6yqbuQ710yI6Lbl074p6pyrAX4M7AB2Ao3OuZe7HmNm04HhzrnnTvVYZnabmfnMzFdfX38GY0dGVlYWBw/27t3n7OxsRo0axeLFi4HAFaGrVq0K53giEqPKK6pJTTZuOKcoos9z2kA3s1zgemAUMAzIMLNbutyfBDwI/MPpHss596hzrsw5V1ZQELKf3VP5+fnMnj2bSZMmcd99Pf+Lxu9//3see+wxpk6dysSJE3n66acjMKWIxJKjbR0sXVnNFROGkJfRL6LPZc65Ux9gNh+Y65z7XPD7TwPnOeduD36fA2wBmoI/MgTYB1x3qjdGy8rK3IkfcLFhwwbOPvvsXv4qsSWRfleRRPb8mp3c/vsVPH7rTC4aV3jGj2dmlc65slD3dWcNfQdwnpmlA0eAS4FjSeycawQGdXmyPwFfO9NdLiIi8WBRhZ+hOWlcODbyqxLdWUN/l8BWxBUEtiwmAY+a2ffM7LoIzyciErNq9x/hjU31zJtRTHJS5DucurXLxTl3P3D/CTd/5yTHXnQmAznn4r686nTLXCISH56srMY5mD8jcnvPu4qqS//T0tLYu3dvXAdeZx96Wlqa16OISAR1dDgWV1Zz/uh8RuSn98lzRtWl/8XFxVRXVxONWxrDqfMTi0Qkfr2zdS879h3m3svP6rPnjKpAT01N1af4iEhcWOyrJisthbmThvTZc0bVkouISDxoPNLK82t2cv20YREr4gpFgS4iEmbPrqqlpa2DhWUj+vR5FegiImFW7gsUcU0qyu7T51Wgi4iE0YadB1hd3cjCmcP7fAu2Al1EJIzKfX76JSdxw7TIFnGFokAXEQmTlrZ2lq2s4fKJg8mNcBFXKAp0EZEw+d/1dTQcbmVBWd9cGXoiBbqISJgs8vkZlpPGBaWDTn9wBCjQRUTCoHb/EZb3YRFXKAp0EZEwWNJZxOXRcgso0EVEzligiMvPrDH5DM/rmyKuUBToIiJn6J0P9uLfd4SFM707OwcFuojIGSv3+clOS+HKiX1XxBWKAl1E5Aw0HmnlhbW7uH5aUZ8WcYWiQBcROQPPdBZxebzcAgp0EZEzUl7h5+yh2Uwc1rdFXKEo0EVEeml97QHW1DSysKw4Kj4LWYEuItJLnUVc13tQxBWKAl1EpBda2tpZVlXDFR4VcYWiQBcR6YVX1u9mv4dFXKEo0EVEemFRhZ+igQOY7VERVygKdBGRHqrZf4Q3N+/hJg+LuEJRoIuI9NASXzUA82cUezzJ8RToIiI90FnENXvMIE+LuEJRoIuI9MDbH+yluuEI88ui6+wcFOgiIj0SLUVcoSjQRUS6qfFwoIjrhnO8L+IKRYEuItJNz6yq4WhbR1TtPe9KgS4i0k2LfH4mDM1mUlGO16OEpEAXEemGdbWNrK05EBU1uSejQBcR6YbFvmr6pSRx/bRhXo9yUt0KdDO7x8zWmdlaM3vCzNJOuP9LZrbGzKrM7E0zmxCZcUVE+l5zaztLV9Zw5cQhDEyPjiKuUE4b6GZWBNwJlDnnJgHJwM0nHPY/zrnJzrlpwA+BB8M9qIiIV15Zv5vGI60siMK95111d8klBRhgZilAOlDb9U7n3IEu32YALjzjiYh4r9wXLOIaEz1FXKGcNtCdczXAj4EdwE6g0Tn38onHmdlXzGwLgTP0O0M9lpndZmY+M/PV19ef2eQiIn2guuEwb27ew7wZxSRFURFXKN1ZcskFrgdGAcOADDO75cTjnHOPOOfGAN8A/k+ox3LOPeqcK3POlRUUFJzZ5CIifWBJZbCIK8qXW6B7Sy6XAVudc/XOuVbgKWDWKY7/A3BDGGYTEfFUR4djsa+aC0oHUZwbXUVcoXQn0HcA55lZugU+BfVSYEPXA8xsbJdvrwY2hW9EERFv/GXLXmr2H2F+lF4ZeqKU0x3gnHvXzJYAK4A2YCXwqJl9D/A5554BvmpmlwGtQAPwmQjOLCLSJ8p9fnIGpHLFhMFej9Itpw10AOfc/cD9J9z8nS733xXOoUREvNZ4uJUX1+3iEzOHR2URVyi6UlREJISnO4u4ovhS/xMp0EVEQlhU4WfisGwmDovOIq5QFOgiIidYW9PIutroLuIKRYEuInKCxT5/oIhrapHXo/SIAl1EpIvm1naWVdUyd+IQctJTvR6nRxToIiJdvHysiCu2lltAgS4icpzyikAR16wx+V6P0mMKdBGRIP++w7y1ZQ/zy6K/iCsUBbqISNCHRVyxt9wCCnQRESBQxLWkMlDEVTRwgNfj9IoCXUQEeGvLHmr2H4nJN0M7KdBFRIByXzUD01O5YmJsFHGFokAXkYS3//BRXlq3ixumFdE/JTaKuEJRoItIwnu6qjZQxBXDyy2gQBcRYVGFn0lF2UwYlu31KGdEgS4iCW1tTSPrdx5gYYyfnYMCXUQSXHmwiOu6GCviCkWBLiIJq7m1nWUra/jYpNgr4gpFgS4iCeuldbs40NwW82+GdlKgi0jCKvf5GZ43gPNHx14RVygKdBFJSP59h3lr817mzxgek0VcoSjQRSQhLa6sxgxumlHs9Shho0AXkYTT3uFY4vNz4diCmC3iCkWBLiIJ563Ne6htbGZBWfycnYMCXUQSULnPz8D0VC6fELtFXKEo0EUkoTQcOsrL63bHfBFXKAp0EUkoy6pqONoe+0VcoSjQRSRhOOdYVOFnclFOzBdxhaJAF5GEsbbmABt3HWTBzPg7OwcFuogkkHKfn/4pSVw3dZjXo0SEAl1EEkJzazvLqoJFXANiv4grFAW6iCSEl9bt4mAcFXGFokAXkYSwqCJQxHVenBRxhaJAF5G45993mL9s2cuCOCriCqVbgW5m95jZOjNba2ZPmFnaCfffa2brzWy1mb1qZiMjM66ISM8t9vnjrogrlNMGupkVAXcCZc65SUAycPMJh60M3j8FWAL8MNyDioj0RnuHY0llNXPGFjAsjoq4QunukksKMMDMUoB0oLbrnc65151zh4PfvgPE9/8GRSRmvHmsiCt+3wztdNpAd87VAD8GdgA7gUbn3Mun+JHPAS+EusPMbjMzn5n56uvrezOviEiPlPv85KanctmEQq9HibjuLLnkAtcDo4BhQIaZ3XKSY28ByoAfhbrfOfeoc67MOVdWUFDQ+6lFRLqh4dBRXlm3mxvOib8irlC6s+RyGbDVOVfvnGsFngJmnXiQmV0GfBu4zjnXEt4xRUR6bunKQBHXwji91P9E3Qn0HcB5ZpZuZgZcCmzoeoCZnQP8kkCY14V/TBGRnnHOUe7zM6U4h/FD4q+IK5TurKG/S2DnygpgTfBnHjWz75nZdcHDfgRkAovNrMrMnonUwCIi3bGmpjFQxJUAb4Z2SunOQc65+4H7T7j5O13uvyycQ4mInKnOIq5r47SIKxRdKSoicae5tZ2nq2q5avLQuC3iCkWBLiJx58W1gSKu+XH2IdCno0AXkbizqMLPiLx0zhsVv0VcoSjQRSSu7Nh7mLc/2MuCsuK4LuIKRYEuInFlcaWfpAQo4gpFgS4iceNYEddZBQzNie8irlAU6CISN5ZvqmdnghRxhaJAF5G4Ue7zk5fRj8vOHuz1KJ5QoItIXNh36CivrN/NDdOK6JeSmNGWmL+1iMSdpStraG13CVPEFYoCXURinnOOxT4/U4tzGDcky+txPKNAF5GYt7o6WMSVwGfnoEAXkThQ7vOTlppYRVyhKNBFJKYdOdrOM1W1XDVpKNlpiVPEFYoCXURi2ovrdnKwpS3hl1tAgS4iMW5RhZ+R+el8ZFSe16N4ToEuIjFr+95DvPPBPhaUDSfwCZmJTYEuIjFrsa86UMQ1PfGKuEJRoItITOos4vroWQUMyUnzepyooEAXkZj0xqZ6dh1I3CKuUBToIhKTyisCRVyXJmgRVygKdBGJOXubWvjfDbv5+DmJW8QVil4JEYk5nUVcWm45ngJdRGKKc45yn5+pwwcmdBFXKAp0EYkpq6obeX93Ewt1dv43FOgiElM6i7iumTrU61GijgJdRGLGkaPtPFtVy1WTVcQVigJdRGLGC2sDRVxabglNgS4iMWNRhZ+S/HTOVRFXSAp0EYkJ2/Yc4t2t+5ivIq6TUqCLSExYXOlXEddpKNBFJOq1tXewpLKai8YVqojrFBToIhL1lm/aw+4DLSwo09n5qSjQ49zRtg72NrV4PYbIGVlU4Sc/ox+XjFcR16mkdOcgM7sH+DzggDXArc655i73zwEeAqYANzvnloR/VOmOg82trNixH9+2fVRs20eVfz/NrR187oJR3HflONJSk70eUaRHOou4/n5WiYq4TuO0gW5mRcCdwATn3BEzKwduBh7vctgO4O+Br0VgRjmFXY3N+Lbvw7etgfe27mPjrgN0OEhOMiYOy+bvzh3J4aNtPPbmVpZvqucnC6cxcViO12OLdNvSlTW0dTh9CHQ3dOsMPXjcADNrBdKB2q53Oue2AZhZR1ink+N0dDi21DdRsa0hcAa+fR/+fUcAGJCazPSRA7njkrHMLMlj2oiBZPb/8F/v3ElD+PqS1dzwyFvce/k4bpszmuQkbf2S6OacY1GFn2nDB3LWYBVxnc5pA905V2NmPyZwFn4EeNk593JvnszMbgNuAxgxYkRvHiKhtLS1s7am8ViA+7Y3sP9wKwCDMvszsySXv581ipkluZw9NJvU5JP/dfSicYW8dPcc/nHpGh54cSOvb6zj3xZMZXheel/9OiI9VuXfz6a6Jv71xslejxITurPkkgtcD4wC9gOLzewW59zvevpkzrlHgUcBysrKXE9/Pt41HmllxfYGKrYFllCqqvdztC3wl57RBRlcOWEIZSW5zCzJY2R+eo8vrsjN6MfPPzmdpStruP/pdXzsp8u5/9oJzJtRrAs1JCqV+6oZkJrMNVNUxNUd3VlyuQzY6pyrBzCzp4BZQI8DXY5Xu/8IFcE3L33bGvjr7oM4BylJxsSiHD593khmjspjxshcBmX2D8tzmhk3Ti/m3FF53Fu+ivuWrOZ/N+zmXz4+mfwwPYdIOBw+2sazqwJFXFkq4uqW7gT6DuA8M0snsORyKeCL6FRxqKPD8X7dwQ+XT7Y1ULM/sP6d0S+Z6SNzuWryUMpKcpk2fCDp/br79kbvFOem88QXzuOxNz/gxy+9z5UPLedH86Zw8fjCiD6vSHe9sGYXTS1tLNSbod1mzp1+5cPMvgssBNqAlQS2MH4b8DnnnjGzmcBSIBdoBnY55yae6jHLysqczxe//19obm1ndXVj8Ox7H5XbGzjQ3AZAYVZ/Zo7KY+bIXMpK8hg/JIuUU6x/R9qGnQe4Z1EVG3cd5JMfGcG3rz474v9DETmdBb98m/qDLbz2Dx/VkmAXZlbpnCsLeV93Aj0S4i3Q9x8+im9bAxXBLYRrqhs52h5Y/y4tzGRmSR4zg+vfxbkDou4/0ObWdh585X1+tfwDSvIzeHDBVM4Zkev1WJKgtu45xMU//hNfnzuO2y8q9XqcqHKqQNdpWC8456huOIJv+75jSyjv724CIDXZmFyUw62zSygrCax/52X083ji00tLTeYfrzqbi8cV8rXFq5j3i7f56sWlfPWS0lPunhGJhMU+FXH1hgK9G9o7HBt3HQicgQfXv3cdCFwom9U/hRkluVw/rYiykblMHT4wpq/GPH9MPi/cfSH/7+l1/PTVTfzpr3U8uHAaYwoyvR5NEkRnEdfF4woZnK0irp5QoIdw5Gg7Vf7g5fPbG1ixvYGmlsD699CcNM4dFVg+KSvJ46zBWXF3gU52WioPLpzGpWcP5tvL1nD1w8v59tUTuOUjI6JuqUjizxub6qk72MJ8fSpRjynQCXRF+LYHr77c1sDamkbaOhxmMG5wFjecM4yZJXmUleRRNHCA1+P2maunBHbd3LdkNf932Vpe3bCbH940hUKdNUkELarwMyizH5eerR1XPZVwge6cY8e+wx9ePr9tH1vqDwHQLzmJqcNz+MKc0cwsyWXGiDxy0hN7/+vg7DR+fetMfvvOdv7l+Q1c+dAb/OuNk5k7SRd6SPjtaWrh1Q113Dq7RO/d9ELcB3pbewcbdh4MrH0H38SsPxiok80ZkErZyFzmzRjOzJJcJhXlxPT6d6SYGZ8+v4RZYwZxb3kVX/rdCm6aXsz9103QJ69LWC1dESzi0nJLr8RdoB9qaaPKv//Ym5crdjRw+Gg7AMW5A7igdNCxy+dLCzJJirP170gqLczkyS/P4mevbuLfX9/MOx/s5cEFU/nI6HyvR5M44Jyj3OfnnBEDGasirl6J+UCvP9hCZfDMu2LbPtbVHqA9uP49fkg282YUUxbcAz40J3HWvyMlNTmJe68Yx0XjC7l3URU3/+odbrtwNPdecRb9U/S3G+m9lcEirh+oiKvXYi7Qa/Yf4a1Ne4JLKA1s3RNY/+6fksS04QP58kfHUFaSy/SRuVoOiKDpI3J57s4L+afnNvDLNz7gz+/X89DN0xg/JNvr0SRGLfb5A0VcU4d5PUrMirlAf6aqlgde3EhueiplJXl84tzhlJXkMWlYjj7NpI9l9E/hX2+czGVnF/KNJ1dz3c/e4r4rx/G5C0ZpKUt6JFDEtZOrpww9rsdfeibmXrmbphdx+YRCxhRkak90lLj07MG8dPccvvXUGv75+Q28unE3/7ZgWkJt8ZQz87yKuMIi5k5pC7PTKC3MUphHmfzM/vzyUzP44bwprKluZO5P3mDpymq86gqS2FJe4Wf0oAzKRqo/6EzEXKBL9DIzFpQN54W75jBuSBb3LFrFV/9nJfsPH/V6NIliH9Q38d62fcwvG64TtTOkQJewG5GfzqIvns/X547j5fW7uPKhN3jj/Xqvx5IotbiymuQk46bpRV6PEvMU6BIRyUnG7ReVsvT22WSnpfLp/3qP+59ey5HgNQEiELjw78nKai4eV6BKiTBQoEtETSrK4dk7LuCzs0fx67e3c83PlrOmutHrsSRK/Pl9FXGFkwJdIi4tNZnvXDuB33/+IxxqaefjP3+Ln726ibbgB4BI4uos4rpEH30YFgp06TOzSwfx0t1zuGryUP7tlfdZ8Mu32b73kNdjiUfqD7bw2sY6bpxerCKuMNGrKH0qJz2Vhz9xDj+9eRqb65r42E+X88R7O7S9MQEtXVkdLOLSpxKFiwJdPHH9tCJeumcO54wYyLeeWsPnf+071oIp8S9QxFXN9BEDKS1UEVe4KNDFM0NzBvDbz36E71wzgeWb9zD3oTd4Zf1ur8eSPrBix3421zXpytAwU6CLp5KSjM9eMIrn7riAwdlpfOE3Pr6xZPWxj/yT+LTY5ye9XzJXT1ERVzgp0CUqjB2cxbKvzOb2i8awuNLPVT9dTuX2fV6PJRFwqKWNZ1fVcvVkFXGFmwJdoka/lCS+Pnc8i754Pg7H/F+8zY9e2sjRNm1vjCfPr9nJoaPtWm6JAAW6RJ2ZJXm8cNcc5s0o5pHXt/Dxn7/Fpt0HvR5LwqTc52d0QQYzVMQVdgp0iUqZ/VP44byp/PJTM9jZ2Mw1P3uT/35rKx0d2t4Yy7bUN1GxrYEFKuKKCAW6RLUrJw7hpbvnMLt0EN99dj2f/q/32Nl4xOuxpJcW+wJFXDeqiCsiFOgS9Qqy+vPYZ8r4l49PpnJ7A1f+5A2eXVXr9VjSQ23tHTy5opqLxxVSmKUirkhQoEtMMDP+7iMjeOGuCxlTmMkdT6zkrj+spPFwq9ejSTf96a/11B9s0ZWhEaRAl5hSMiiDxV88n3svP4s/rt7J3J++wVub93g9lnTDIp+fQZn9uVhFXBGjQJeYk5KcxJ2XjuWpL89iQL9kPvmf7/L9P66nuVVd69Gq7mAzr22s46bpRSriiiC9shKzpg4fyHN3XMhnzh/JY29u5bp/f5N1tepaj0ZLV9TQ3uHUex5hCnSJaQP6JfPd6yfx68+ey/7DrdzwyFv8x5+20K7tjVEjUMTlp2xkLqWFmV6PE9cU6BIXPnpWAS/dPYfLJwzmgRc3cvOjb+Pfd9jrsQRYsaOBLfWHWKCz84jrVqCb2T1mts7M1prZE2aWdsL9/c1skZltNrN3zawkItOKnEJuRj8e+bvpPLhgKht3HmTuQ29Q7vOra91j5RXVwSKuoV6PEvdOG+hmVgTcCZQ55yYBycDNJxz2OaDBOVcK/AR4INyDinSHmXHj9GJeuPtCJhXl8PUlq/nS7yrZ26SudS8camnjj6truWbKUDJUxBVx3V1ySQEGmFkKkA6ceFXH9cCvg18vAS41XdcrHirOTeeJL5zHt686m9c31nPlQ8t5baO61vvac6tVxNWXThvozrka4MfADmAn0Oice/mEw4oAf/D4NqARyD/xsczsNjPzmZmvvr7+TGcXOaWkJOMLc0bzzB2zGZTZj88+7uPbS9dw+Ki61vtKZxHX9BEq4uoL3VlyySVwBj4KGAZkmNktvXky59yjzrky51xZQUFBbx5CpMfGD8nm6a/O5otzRvM/7+3gqp8uZ+WOBq/Hinub65rwbW9goYq4+kx3llwuA7Y65+qdc63AU8CsE46pAYYDBJdlcoC94RxU5Ez0T0nmW1edzRNfOI/Wdse8X7zNg6+8T2u7utYjZXGln+Qk4+Mq4uoz3Qn0HcB5ZpYeXBe/FNhwwjHPAJ8Jfj0PeM1pa4FEofNG5/PC3Rdy/bRhPPzqJub9x1/YUt/k9Vhxp7W9gycra7hkvIq4+lJ31tDfJfBG5wpgTfBnHjWz75nZdcHDHgPyzWwzcC/wzQjNK3LGstNSeXDBNH7+yels33eYqx9ezm/f3qbtjWH0p7/Ws6epRXvP+5h59R9xWVmZ8/l8njy3SKe6A83ct2Q1f36/no+eVcCP5k2hMFtnlGfq87/2sap6P29/8xJS1N0SVmZW6ZwrC3WfXmlJaIXZaTx+60y+f/1E3t26lyseCnStq+ir9+oONvP6X+u4cXqRwryPaae/JDwz41PnlzCrdBD3LqrijidWYgYj8tIpLcikdHAmpQWZjB2cxZiCDLLSUr0eOao9FSzi0nJL31OgiwSNKchkyZdn8eqG3WzcdZBNdU1sqWti+aY9HO2yG2ZoThqlhZnH/owtzKK0MJO8jH4eTh8dOou4ZpbkMqZARVx9TYEu0kVqchJzJw1l7qQPe0fa2jvYse8wm+uajoX8promFlX4OXz0w6WZ/Ix+jCnMZOwJQT84u3/C7MOu3N7AB/WH+NJHx3g9SkJSoIucRkpyEqMLMhldkMkVEz+8vaPDUdt4hM11Tcf+bKpr4tlVtRxo/vBq1Kz+KV2WbT4M+6KBA0hKiq+gL/f5yeiXzNWTVcTlBQW6SC8lJRnFuekU56Zz0bgPP1bNOUd9U8vxQb+7iT+9X8/iyupjx6WlJjGmoDPgO5dwshiZnx6Tn+rT1NLGH1fv5Nopw1TE5RG96iJhZmYUZqVRmJXGrDGDjruv8XArm+sPsml34Gx+c10Tvm0NPF31Yd9darJRkp9xLOjHBM/oRxdkkJaa3Ne/Trc9t7qWw0fbWaAiLs8o0EX6UE56KjNG5jFjZN5xtx9qaWNL/YfLNpvrmti46yAvrdtF54cvJRkMz0s/LuQ735jNjIIz4nJfNWMKMpg+YqDXoyQs7/8rEBEy+qcwpXggU4oHHnd7c2s72/YeOrZss7m+ic27m3jj/ZPvvOkM+rGFmeT20c6bzXUHqdzewD9eNT5h3gCORgp0kSiWlprM+CHZjB+SfdztnTtvNnVZp99c18Qf3vNzpPX4nTcfBn1gjX7s4EwKs8K782axr5qUJOPj5xSH7TGl5xToIjGo686bK0PsvDm2vTJ4Vv83O2/SUo57M7bzrL43O29a2zt4ckU1l4wvpCCrf7h+RekFBbpIHOm68+biUDtvggEfeFP2IK9trKfc97c7b7ruuiktzDzlzpvXN9axp+morgyNAgp0kQRw3M6b0uN33uw/fPS4N2M31TVRsa2BZSF23owN7qcvHZzF2MJMRg3KoNznpyCrPxeN04fWeE2BLpLgBqb3o6wkj7KS0DtvNnU5q9+w8yAvrj1+502Hgy99dIyKuKKAAl1EQjrdzpvOvfS7Go/w2dklnswox1Ogi0iPnGznjXhPf0cSEYkTCnQRkTihQBcRiRMKdBGROKFAFxGJEwp0EZE4oUAXEYkTCnQRkThhzjlvntisHtjeyx8fBOwJ4zjxTq9Xz+j16jm9Zj1zJq/XSOdcyOIczwL9TJiZzzlX5vUcsUKvV8/o9eo5vWY9E6nXS0suIiJxQoEuIhInYjXQH/V6gBij16tn9Hr1nF6znonI6xWTa+giIvK3YvUMXURETqBAFxGJEzEV6Gb2X2ZWZ2ZrvZ4lFpjZcDN73czWm9k6M7vL65mimZmlmdl7ZrYq+Hp91+uZYoGZJZvZSjP7o9ezRDsz22Zma8ysysx8YX/8WFpDN7M5QBPwG+fcJK/niXZmNhQY6pxbYWZZQCVwg3NuvcejRSUzMyDDOddkZqnAm8Bdzrl3PB4tqpnZvUAZkO2cu8breaKZmW0DypxzEbkIK6bO0J1zbwD7vJ4jVjjndjrnVgS/PghsAIq8nSp6uYCm4LepwT+xc8bjATMrBq4G/tPrWSTGAl16z8xKgHOAdz0eJaoFlw+qgDrgFeecXq9Tewj4OtDh8RyxwgEvm1mlmd0W7gdXoCcAM8sEngTuds4d8HqeaOaca3fOTQOKgXPNTEt7J2Fm1wB1zrlKr2eJIRc456YDHwO+ElxGDhsFepwLrgU/CfzeOfeU1/PECufcfuB1YK7Ho0Sz2cB1wXXhPwCXmNnvvB0pujnnaoL/rAOWAueG8/EV6HEs+CbfY8AG59yDXs8T7cyswMwGBr8eAFwObPR0qCjmnPuWc67YOVcC3Ay85py7xeOxopaZZQQ3J2BmGcAVQFh37MVUoJvZE8DbwDgzqzazz3k9U5SbDXyKwJlTVfDPVV4PFcWGAq+b2WqggsAaurbiSbgMBt40s1XAe8BzzrkXw/kEMbVtUURETi6mztBFROTkFOgiInFCgS4iEicU6CIicUKBLiISJxToEnfMrL3LNs0qM/tmGB+7RG2fEq1SvB5AJAKOBC/fF0koOkOXhBHsov5hsI/6PTMrDd5eYmavmdlqM3vVzEYEbx9sZkuD/eirzGxW8KGSzexXwc70l4NXlWJmdwa751eb2R88+jUlgSnQJR4NOGHJZWGX+xqdc5OBfyfQFAjwM+DXzrkpwO+Bh4O3Pwz82Tk3FZgOrAvePhZ4xDk3EdgP3BS8/ZvAOcHH+VJkfjWRk9OVohJ3zKzJOZcZ4vZtwCXOuQ+CpWW7nHP5ZraHwAeBtAZv3+mcG2Rm9UCxc66ly2OUEKgEGBv8/htAqnPun8zsRQIfwLIMWNalW12kT+gMXRKNO8nXPdHS5et2Pnwv6mrgEQJn8xVmpveopE8p0CXRLOzyz7eDX/+FQFsgwCeB5cGvXwW+DMc++CLnZA9qZknAcOfc68A3gBzgb/6WIBJJOoOQeDQg+KlDnV50znVuXcwNtim2AJ8I3nYH8N9mdh9QD9wavP0u4NFgq2c7gXDfeZLnTAZ+Fwx9Ax4OdqqL9BmtoUvCiPQH9Ip4TUsuIiJxQmfoIiJxQmfoIiJxQoEuIhInFOgiInFCgS4iEicU6CIiceL/AzXeeQDMvsiGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEGCAYAAABrQF4qAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAApcUlEQVR4nO3deXiU5b3/8fc3eyAhYUnYEvaQiCCoAXcUN7BW7XIUtbZqWz1atyNq9XT5nR5/9moL/qi1pbUej9rFjWrt4dgWXAgiCkrAIGsw7GExCZCwZr9/f8wDDCGQCUzyTCaf13XNlZlnmfnOXPC5n7mfZ+7bnHOIiEj0ivG7ABERaVsKehGRKKegFxGJcgp6EZEop6AXEYlycX4X0FSvXr3coEGD/C5DRKRDWbJkSYVzLqO5dREX9IMGDaKwsNDvMkREOhQz23S8deq6ERGJcgp6EZEop6AXEYlyEddHLyKdU11dHaWlpVRXV/tdSkRLSkoiKyuL+Pj4kPdR0ItIRCgtLSU1NZVBgwZhZn6XE5Gcc+zcuZPS0lIGDx4c8n7quhGRiFBdXU3Pnj0V8idgZvTs2bPV33oU9CISMRTyLTuZzyhqgn5vdR3T5qxhY8V+v0sREYkoURP01XWNPL9gI9PfWet3KSLSQaWkpPhdQpuImqDPSE3k2xcOYtaybazcVuV3OSIiESNqgh7gzvFDSUuO58k5xX6XIiIdmHOORx55hJEjRzJq1Chee+01ALZv38748eMZM2YMI0eO5IMPPqChoYHbbrvt8La//OUvfa7+WFF1eWVacjx3XTyUX8xew+KNuxg7qIffJYnISfjP/13Jqm17wvqcI/p14z+uOT2kbf/6179SVFTEsmXLqKioYOzYsYwfP56XX36ZiRMn8sMf/pCGhgYOHDhAUVERW7duZcWKFQBUVlaGte5wiKojeoDbzh9EZmoiU2evQfPhisjJWLBgATfddBOxsbH07t2biy++mMWLFzN27FheeOEFfvKTn7B8+XJSU1MZMmQI69ev57777mP27Nl069bN7/KPEVVH9ADJCbHcd1kOP/7bCuYVlzMhL9PvkkSklUI98m5v48ePZ/78+fz973/ntttuY8qUKXzrW99i2bJlzJkzh2eeeYaZM2fy/PPP+13qUaLuiB5gcn42A3p0YeqcYhobdVQvIq1z0UUX8dprr9HQ0EB5eTnz589n3LhxbNq0id69e3PHHXfw3e9+l6VLl1JRUUFjYyNf//rXeeKJJ1i6dKnf5R8j6o7oARLiYphyxXD+7bUi3lq+nWtH9/O7JBHpQL761a+ycOFCRo8ejZkxdepU+vTpwx/+8AemTZtGfHw8KSkp/PGPf2Tr1q3cfvvtNDY2AvCzn/3M5+qPZZHWj52fn+/CMfFIY6PjS09/QHVdA+9MuZj42Kj88iISNVavXs1pp53mdxkdQnOflZktcc7lN7d91KZfTIzxyMRcNu48wF8KS/0uR0TEN1Eb9ACX5mVy9sDu/Oq9tVTXNfhdjoiIL6I66M2M70/M5Ys9Nfxx4Ua/yxER8UVUBz3AOUN6cvHwDH47bx17quv8LkdEpN1FfdADPDIxl8oDdTw3f73fpYiItLuQgt7MJplZsZmVmNljzay/y8yWm1mRmS0wsxHe8kFmdtBbXmRmz4T7DYRiZP80rj6jL88t2EDFvho/ShAR8U2LQW9mscAM4CpgBHDToSAP8rJzbpRzbgwwFZgetG6dc26Md7srTHW32kNXDKemvpEZBSV+lSAi4otQjujHASXOufXOuVrgVeC64A2cc8GjD3UFIuvifGBIRgrXn53FS4s2U7r7gN/liEgHd6Kx6zdu3MjIkSPbsZoTCyXo+wNbgh6XesuOYmb3mNk6Akf09wetGmxmn5rZ+2Z2UXMvYGZ3mlmhmRWWl5e3ovzWeeDyHDD41buft9lriIhEmrANgeCcmwHMMLObgR8BtwLbgQHOuZ1mdjbwNzM7vck3AJxzzwLPQuCXseGqqam+acl869yBPP/hBv714iEMy0xtq5cSkVPxz8dgx/LwPmefUXDVz4+7+rHHHiM7O5t77rkHgJ/85CfExcVRUFDA7t27qaur44knnuC666477nM0p7q6mrvvvpvCwkLi4uKYPn06EyZMYOXKldx+++3U1tbS2NjIG2+8Qb9+/bjhhhsoLS2loaGBH//4x0yePPmU3jaEdkS/FcgOepzlLTueV4GvADjnapxzO737S4B1wPCTqjRMvjdhGF0S4vh/b2vKQRE5YvLkycycOfPw45kzZ3Lrrbfy5ptvsnTpUgoKCnjooYdaPfz5jBkzMDOWL1/OK6+8wq233kp1dTXPPPMMDzzwAEVFRRQWFpKVlcXs2bPp168fy5YtY8WKFUyaNCks7y2UI/rFQI6ZDSYQ8DcCNwdvYGY5zrlD/SFXA597yzOAXc65BjMbAuQAvl7j2KNrAt+9aDBPvfs5y7ZUMjo73c9yRKQ5JzjybitnnnkmZWVlbNu2jfLycrp3706fPn148MEHmT9/PjExMWzdupUvvviCPn36hPy8CxYs4L777gMgLy+PgQMHsnbtWs477zx++tOfUlpayte+9jVycnIYNWoUDz30EI8++ihf/vKXueiiZnu7W63FI3rnXD1wLzAHWA3MdM6tNLPHzexab7N7zWylmRUBUwh02wCMBz7zlr8O3OWc2xWWyk/Bdy8aQo+uCUzTlIMiEuT666/n9ddf57XXXmPy5Mm89NJLlJeXs2TJEoqKiujduzfV1dVhea2bb76ZWbNmkZyczJe+9CXmzp3L8OHDWbp0KaNGjeJHP/oRjz/+eFheK6Q+eufcP4B/NFn2f4LuP3Cc/d4A3jiVAttCSmIc37tkKE/8fTUflVRw/rBefpckIhFg8uTJ3HHHHVRUVPD+++8zc+ZMMjMziY+Pp6CggE2bNrX6OS+66CJeeuklLr30UtauXcvmzZvJzc1l/fr1DBkyhPvvv5/Nmzfz2WefkZeXR48ePbjllltIT0/nueeeC8v7isrx6ENxy7kDeX7BBn4xp5i/De2Jmfldkoj47PTTT2fv3r3079+fvn378o1vfINrrrmGUaNGkZ+fT15eXquf83vf+x533303o0aNIi4ujhdffJHExERmzpzJn/70J+Lj4+nTpw8/+MEPWLx4MY888ggxMTHEx8fzu9/9LizvK2rHow/Fa4s38+gby/n9N89m4umh97mJSPhpPPrQaTz6Vvj6WVkMyejKk3OKadCUgyISpTp10MfFxvDQFbl8XraPv316oitGRUSOtXz5csaMGXPU7ZxzzvG7rGN02j76Q64a2YeR/bvxy3fXcs3ofiTEdeq2T8RXzrkOdb5s1KhRFBUVtetrnkx3e6dPtZgY4/sT8yjdfZBXPtnsdzkinVZSUhI7d+48qSDrLJxz7Ny5k6SkpFbt1+mP6AEuyunFuUN68Ou5JVyfn0WXBH0sIu0tKyuL0tJS2nK8q2iQlJREVlZWq/ZRouFNOTgpj6/99iNe+HAj90wY5ndJIp1OfHw8gwcP9ruMqNTpu24OOWtAdy4/rTfPvL+OygO1fpcjIhI2Cvogj0zMZV9NPc+8rykHRSR6KOiD5PZJ5Stj+vPiRxso2xOe8SxERPymoG/iwcuHU9/geHquJicRkeigoG9iQM8u3DRuAK9+soVNO/f7XY6IyClT0DfjvkuHERdr/PIdTU4iIh2fgr4Zmd2SuP2CwfzPsm2s2bGn5R1ERCKYgv447ho/lNTEOJ7U5CQi0sEp6I8jrUs8/3rxUN5dXcaSTb5PiiUictIU9Cdw+wWD6JWSyC9mF2v8DRHpsBT0J9AlIY77LxvGJxt28f5ajb8hIh2Tgr4FN44dQHaPZKbNKaZRk5OISAcUUtCb2SQzKzazEjN7rJn1d5nZcjMrMrMFZjYiaN2/e/sVm9nEcBbfHhLiYnjw8uGs3LaHf6zY7nc5IiKt1mLQm1ksMAO4ChgB3BQc5J6XnXOjnHNjgKnAdG/fEcCNwOnAJOC33vN1KNeN6c/w3ilMf3st9Q2NfpcjItIqoRzRjwNKnHPrnXO1wKvAdcEbOOeCLzbvChzq47gOeNU5V+Oc2wCUeM/XocTGGA9fmcv6iv28vqTU73JERFollKDvD2wJelzqLTuKmd1jZusIHNHf35p9O4IrRvTmzAHp/Oq9z6mua/C7HBGRkIXtZKxzboZzbijwKPCj1uxrZneaWaGZFUbq7DJmgSkHt1dV8+dFm/wuR0QkZKEE/VYgO+hxlrfseF4FvtKafZ1zzzrn8p1z+RkZGSGU5I/zhvbkopxezCgoYW91nd/liIiEJJSgXwzkmNlgM0sgcHJ1VvAGZpYT9PBq4NAYv7OAG80s0cwGAznAJ6detn++PzGP3QfqeO6DDX6XIiISkhaD3jlXD9wLzAFWAzOdcyvN7HEzu9bb7F4zW2lmRcAU4FZv35XATGAVMBu4xznXoTu4R2Wl8aVRfXjug/Xs3FfjdzkiIi2ySPtpf35+vissLPS7jBMqKdvHlb98n9svGMyPv9z0SlMRkfZnZkucc/nNrdMvY0/CsMwU/uXsLP60aBPbKg/6XY6IyAkp6E/SA5cPBwe/eldTDopIZFPQn6T+6cnccu5A/rJkC+vK9/ldjojIcSnoT8E9E4aSHB/L9Lc15aCIRC4F/SnomZLIdy4awt+Xb2fF1iq/yxERaZaC/hTdcdFguneJZ6qmHBSRCKWgP0WpSfF875JhzF9bzsJ1O/0uR0TkGAr6MPjmeQPp0y2JqXPWaMpBEYk4CvowSIqP5YHLc/h0cyXvri7zuxwRkaMo6MPk+rOzGNyrK0/OKaZBUw6KSARR0IdJXGwMU64YTvEXe5m17ESDe4qItC8FfRhdPaovI/p2Y/o7a6mt15SDIhIZFPRhFBNjPDIply27DvLa4s1+lyMiAijow+6S4RmMG9yDp+eWcKC23u9yREQU9OFmZjw6KZfyvTW8+NFGv8sREVHQt4WzB/bgsrxMnpm3jqoDmnJQRPyloG8jD0/MZW9NPb+fv87vUkSkk1PQt5HT+nbj2tH9eOHDjZTtrfa7HBHpxBT0bWjKFcOpa2jkN3NL/C5FRDoxBX0bGtizK5PHZvPKJ5vZsuuA3+WISCeloG9j91+WQ2yM8ct3NDmJiPgjpKA3s0lmVmxmJWb2WDPrp5jZKjP7zMzeM7OBQesazKzIu80KZ/EdQe9uSdx6/iDeLNpK8Y69fpcjIp1Qi0FvZrHADOAqYARwk5mNaLLZp0C+c+4M4HVgatC6g865Md7t2jDV3aHcffFQUhLjePJtTU4iIu0vlCP6cUCJc269c64WeBW4LngD51yBc+5QJ/QiICu8ZXZs6V0S+NfxQ3hn1Rcs3bzb73JEpJMJJej7A1uCHpd6y47nO8A/gx4nmVmhmS0ys680t4OZ3eltU1heXh5CSR3P7RcMpldKAlNna3ISEWlfYT0Za2a3APnAtKDFA51z+cDNwFNmNrTpfs65Z51z+c65/IyMjHCWFDG6JsZx74RhLFq/iw8+r/C7HBHpREIJ+q1AdtDjLG/ZUczscuCHwLXOuZpDy51zW72/64F5wJmnUG+HdtM5A+ifnsy0OcU6qheRdhNK0C8GcsxssJklADcCR109Y2ZnAr8nEPJlQcu7m1mid78XcAGwKlzFdzSJcbE8eMVwlm+t4p8rdvhdjoh0Ei0GvXOuHrgXmAOsBmY651aa2eNmdugqmmlACvCXJpdRngYUmtkyoAD4uXOu0wY9wFfP7E9OZgpPvl1MfYMmJxGRtmeR1oWQn5/vCgsL/S6jTc1esYO7/ryEqV8/gxvGZre8g4hIC8xsiXc+9Bj6ZawPJp7em9HZ6Tz17lqq6xr8LkdEopyC3gdmxvcn5rKtqpqXPtaUgyLSthT0PrlgWC8uHNaLGQUl7KvRlIMi0nYU9D56ZGIuu/bX8t8fbPC7FBGJYgp6H43OTmfS6X34rw/Ws2t/rd/liEiUUtD77OGJwzlQW8/v5mlyEhFpGwp6nw3LTOVrZ2Xxh4Wb2F510O9yRCQKKegjwL9dngMOnn7vc79LEZEopKCPAFndu3DzOQOYWVjK+vJ9fpcjIlFGQR8h7r10GIlxMUzXlIMiEmYK+gjRKyWR71w4mLc+286KrVV+lyMiUURBH0HuGD+E9C7xmnJQRMJKQR9BuiXFc/fFQ5lXXM4nG3b5XY6IRAkFfYS59fxB9O6WqCkHRSRsFPQRJik+lvsvy6Fw027mrilreQcRkRYo6CPQDfnZDOrZhWlzimls1FG9iJwaBX0Eio+N4cErhrNmx17+97NtfpcjIh2cgj5CXXNGP07r243p76ylTlMOisgpUNBHqJgY45GJw9m08wCvLd7idzki0oEp6CPYhNxM8gd25+n3PudgraYcFJGTE1LQm9kkMys2sxIze6yZ9VPMbJWZfWZm75nZwKB1t5rZ597t1nAWH+3MjO9PyqNsbw1/WLjR73JEpINqMejNLBaYAVwFjABuMrMRTTb7FMh3zp0BvA5M9fbtAfwHcA4wDvgPM+sevvKj37jBPZiQm8Hv5q2j6mCd3+WISAcUyhH9OKDEObfeOVcLvApcF7yBc67AOXfAe7gIyPLuTwTecc7tcs7tBt4BJoWn9M7j4Ym5VB2s47/mr/e7FBHpgEIJ+v5A8NnAUm/Z8XwH+Gdr9jWzO82s0MwKy8vLQyipczm9XxrXjO7H8x9uoHxvjd/liEgHE9aTsWZ2C5APTGvNfs65Z51z+c65/IyMjHCWFDWmXDGcmvpGZhRoykERaZ1Qgn4rkB30OMtbdhQzuxz4IXCtc66mNftKywb36soN+dm89PEmtuw60PIOIiKeUIJ+MZBjZoPNLAG4EZgVvIGZnQn8nkDIBw/QMge40sy6eydhr/SWyUl44LIcYsx46l1NOSgioWsx6J1z9cC9BAJ6NTDTObfSzB43s2u9zaYBKcBfzKzIzGZ5++4C/i+BxmIx8Li3TE5Cn7Qkbj1/EG9+WsrnX+z1uxwR6SAs0obCzc/Pd4WFhX6XEbF2769l/NQCzh/Wk99/M9/vckQkQpjZEudcs6GgX8Z2MN27JnDH+CHMWfkFRVsq/S5HRDoABX0H9O0LB9OzawLT5qzxuxQR6QAU9B1QSmIc90wYxoclO/mwpMLvckQkwinoO6hvnDuA/unJmnJQRFqkoO+gEuNieeDyHJaVVjFn5Q6/yxGRCKag78C+dmZ/hmZ05cm319KgKQdF5DgU9B1YXGwMD1+ZS0nZPv66tNTvckQkQinoO7hJI/twRlYaT737OTX1mpxERI6loO/gzIxHJuaytfIgL3+82e9yRCQCKeijwIXDenHekJ78Zm4J+2vq/S5HRCKMgj4KmBmPTMpl5/5anl+wwe9yRCTCKOijxFkDunPliN48O389u/fX+l2OiEQQBX0UeXhiLvtq63nm/XV+lyIiEURBH0WG907lq2f258WPNrKjqtrvckQkQijoo8yDlw+n0TmenqvJSUQkQEEfZbJ7dOHmcQOYuXgLGyv2+12OiEQABX0UuvfSHOJjY5j+zlq/SxGRCKCgj0IZqYl8+8JBzFq2jVXb9vhdjoj4TEEfpe4cP5S05HiefLvY71JExGcK+iiVlhzPXRcPZe6aMgo3aj52kc5MQR/Fbjt/EJmpiUydXazJSUQ6sZCC3swmmVmxmZWY2WPNrB9vZkvNrN7M/qXJugYzK/Jus8JVuLQsOSGW+y7L4ZONu5i3ttzvckTEJy0GvZnFAjOAq4ARwE1mNqLJZpuB24CXm3mKg865Md7t2lOsV1ppcn42A3p0YdrsYho1OYlIpxTKEf04oMQ5t945Vwu8ClwXvIFzbqNz7jOgsQ1qlFOQEBfDlCuGs2r7Ht5avt3vckTEB6EEfX9gS9DjUm9ZqJLMrNDMFpnZV5rbwMzu9LYpLC9XF0O4XTu6H3l9Upn+djF1DWqLRTqb9jgZO9A5lw/cDDxlZkObbuCce9Y5l++cy8/IyGiHkjqXmBjj4Stz2bjzAH8p1JSDIp1NKEG/FcgOepzlLQuJc26r93c9MA84sxX1SZhcdlomZw1I51fvraW6TlMOinQmoQT9YiDHzAabWQJwIxDS1TNm1t3MEr37vYALgFUnW6ycPDPj+5Py+GJPDX9cuNHvckSkHbUY9M65euBeYA6wGpjpnFtpZo+b2bUAZjbWzEqB64Hfm9lKb/fTgEIzWwYUAD93zinofXLukJ6MH57Bb+etY091nd/liEg7sUj7IU1+fr4rLCz0u4yotWJrFV/+9QLuv3QYU67M9bscEQkTM1vinQ89hn4Z28mM7J/G1Wf05bkFG6jYV+N3OSLSDhT0ndBDVwynpr6RGQUlfpciIu1AQd8JDclI4fqzs3hp0WZKyvb5XY6ItDEFfSf1wOU5xMYYl09/n2t+vYDpbxfz6ebdNGiYBJGoo5OxndiGiv38Y/l2CtaUsXTzbhod9OiawCXDM7gkL5PxOb1I75Lgd5kiEoITnYxV0AsAu/fXMv/zcgrWlPH+2nJ2H6gjxuDsgd25JDeTS/MyyeuTipn5XaqINENBL63S0OhYVlpJwZoyCorLWLE1MB1h37QkLsnNZEJuBhcM60XXxDifKxWRQxT0ckq+2FPN+8XlzF1TxoKSCvbV1JMQG8M5Q3owITeTCXmZDO7V1e8yRTo1Bb2ETW19I4WbdnlH++WHr9oZ3Ksrl+RmcGleJuMG9yAxLtbnSkU6FwW9tJnNOw8wb20Zc9eUsXDdTmrqG+mSEMsFw3p5R/sZ9E1L9rtMkainoJd2cbC2gYXrKyhYE+jm2Vp5EIC8PqlMyAuc0D0zO524WF3VKxJuCnppd845Ssr2Mdc7oVu4cTf1jY605HjGD89gQm4GFw/PoGdKot+likQFBb34bk91HQs+r2DumjLmFZdTsa8GMxidlc6leZlMyM3k9H7diInR5ZsiJ0NBLxGlsdGxYlsVBWvKKSguY1lpJc5BRmoilwwPnNC9IKcX3ZLi/S5VpMNQ0EtEq9hXw/y1gX79+WvL2VNdT1yMMXZQDybkBYJ/aEaKfqwlcgIKeukw6hsaWbq5koLiMgrWlLFmx14AsronM8H7he55Q3uSFK/LN0WCKeilw9pWedAL/XI+LKngYF0DiXExnD+0JxO8vv3sHl38LlPEdwp6iQrVdQ18smHX4aP9jTsPADAsM4VL8zK5JDeDsYN6EK/LN6UTUtBLVNpQsd+7iqeMj9fvorahkdTEOC7M6cUEL/gzU5P8LlOkXSjoJertr6nnw5KKw908O/ZUAzCqfxoTcjOYkJfJGVnpxOryTYlSpxz0ZjYJ+BUQCzznnPt5k/XjgaeAM4AbnXOvB627FfiR9/AJ59wfTvRaCno5Vc45Vm/fe7iLR2PtS2dwSkFvZrHAWuAKoBRYDNzknFsVtM0goBvwMDDrUNCbWQ+gEMgHHLAEONs5t/t4r6egl3CrPFDL/M8rKPC6eTTWvkSjEwV9KAOKjwNKnHPrvSd7FbgOOBz0zrmN3rrGJvtOBN5xzu3y1r8DTAJeaeV7EDlp6V0SuHZ0P64d3e/wWPvz1pQxt7iMaXOKmTanWGPtS1QL5V9zf2BL0ONS4JwQn7+5ffs33cjM7gTuBBgwYECIT91EQx386avQ70wYcB4MOBe69Di555KoFRtjnDWgO2cN6M6UK3Mp21PNvOLAL3T/d9k2Xvlks8bal6gTEYctzrlngWch0HVzUk+yvxwaamHR7+CjpwPLMvICgX8o+NMHgr6eS5DMbkncMDabG8ZmHzPW/uNvreLxt1YdHmt/Qm4mZ2SlkZYcr24e6VBCCfqtQHbQ4yxvWSi2Apc02XdeiPu2Trd+8J23oe4gbPsUNi+ETQthxZuw5MXANqn9jg7+3qdDjH5hKQEJcTGcP7QX5w/txQ+vhi27DlBQHBhr/+WPN/PChxsBSIqPoW9aMn3TkuiTlkS/tOTA3/Qk+nRLpl96khoDiSihnIyNI3Ay9jICwb0YuNk5t7KZbV8E3mpyMnYJcJa3yVICJ2N3He/1wn4ytrEBylYHgn/zosDfPV47ldgNsscdCf/+Z0O8JsmQYx2sbWDRhp2sK9vHjqpqtldVs73qIDuqqvlibw0NjUf/P2qpMeiblkR6FzUGEj7huLzySwQun4wFnnfO/dTMHgcKnXOzzGws8CbQHagGdjjnTvf2/TbwA++pfuqce+FEr9UuV91Ubj4S+psXQZl3XjkmHvqN8YL/fMg+B7r2bNtapMOrb2ikYl8t27zg31YZ+Lt9TzXbK1vfGPRNSzq8XI2BhEo/mGrJgV2w5ZMjwb9taaC/H6BX7tHdPd0HqZ9fWq1pY7C9KtAItLYxCG4E1BhIMAV9a9VVH+nn37wQNn8MNVWBdal9m/Tzj1Q/v4RFQ6OjfG8N26sOel1DrWsM+nRLom+6GoPOSkF/qhoboTyon3/TQthTGliXkOr183vB3/9sSNBoitI22qIxONRtpMagY1PQt4XKLc308zuIiYO+YwKhP/B8yD5X/fzSrk7UGOzw7ofaGPRJS6afGoMOQUHfHg7uPrqff+uSoH7+4UHdPeepn19811xjsKPqINuqWtEYpAUaBDUGkUFB74e6athedOR6/i2LoNrr50/pc2w/f2xE/HZN5LDgxmBHVbXXCLTcGCTGxRzuGuqZkkBacnyzt27J8aR3CdxPSYxT43CKFPSRoLERytccfT1/lTc6REIqZI8N6ufPVz+/dAgNjY6KfTWHLykNbgy2Vx5k1/5aqg7Wsae6/pgGIVhsjNEtKe6oRuB4DcRR67vEk5IQR4yGn1bQR6zKLbDl4yPh/8VKjvTzjz7S1TPgXOjay+9qRU6ac459NfVUHaw7fNsTdP/Irb7Z9SdqJGKMYxqGlhqKQ9ukJkZPI6Gg7ygOVjbTz18TWNcz50h3z8DzoPtg9fNLp+CcY39tQyD0D7TUUBy7rr6FRiI16QTfGE5wS02KrEZCQd9R1dfAtiLY/JHX3bMIqisD61J6N+nnH6V+fpEmnHMcONRInKAxOOp2ILRGwgxSE+NI69L6hiI1KT7ss52d6nj04pe4RBhwTuAGgX7+iuKjr+df9T+BdQkpkBXUz5+VDwkaXlc6NzOja2IcXRPj6JfeunGsTqaR2F5VfXhdXcOJG4mUxLhjGoC8Pt144PKcU33bx1DQdyQxMZB5WuCW/+3AsqrSI0f7mxfBvJ9xbD//uYHr+VMyfC1fpCM51UbiYF1Ds98SjtdQfLGn+oTnIk7pvajrJsocrITSxUeO+ksLg/r5hx19grfHEPXzi0QJdd10JsnpkHNF4AaBfv7ty2CT18+/5i349E+BdV0zISMX0gdAWhakZUN6duBvWlag60hEOjwFfbSLSwyMxZM9LvC4sREq1gaO+Ld8DLvWw7oC2LudwPztQVJ6Hwn99GxIG+D99RqF5PT2fjcichIU9J1NTAxk5gVu+bcfWV5fG5iQpao08EOuyi1QtTnweMdyKP7nkS6gQxK7Nf9NIH1A4H5K78DriYivFPQSEJcAPQYHbs1pbAzMy1tVGmgAKrcc3SgED/FwSEw8pPX3GgJ1D4n4RUEvoYmJgdTegVvW2c1vU73nSPgf/lbg/W1N99Dh++oeEgkHBb2ET1I3SBoBvUc0v/5w95D3beCkuoeygr4hqHtIJBQKemk/J9U9FNQotNQ9FNwtpO4hkcMU9BI5Wts9VLn56PME6+epe0ikGQp66VhOtnuocou6h6TTCinozWwS8CsgFnjOOffzJusTgT8CZwM7gcnOuY1mNghYDRR7my5yzt0VptpFjtXW3UPJ6RCXFHRLPPZ+fNN1Tf82s68aEmlDLQa9mcUCM4ArgFJgsZnNcs6tCtrsO8Bu59wwM7sR+AUw2Vu3zjk3Jrxli5ykU+0eqiiB+uqgWw3UHeSY7qJW1xUfQiORfPzGo8X9TtDwxMZrKIwoF8oR/TigxDm3HsDMXgWuA4KD/jrgJ97914HfmOYFk46qpe6hppyDxvqjg7++5sjj4EahaSPR9G+z+9YEGp/6cqhvZv2huYlPlsWEqXFpZt/4pMAoqgkpEN/Fu98VYhPUuLSjUIK+P7Al6HEpcM7xtnHO1ZtZFdDTWzfYzD4F9gA/cs590PQFzOxO4E6AAQMGtOoNiPjOLHBUHBsPiant//qNjYHzDvXVgbmKj9eINNdIBN8/0b7VlcdvmE7m20xMHMR3PRL8CV2ObQwO3eKPt02Ktyxom7hENSDNaOuTsduBAc65nWZ2NvA3MzvdObcneCPn3LPAsxAYvbKNaxKJLjExEJMM8cnQutF0T92hbzPH+xZTdwBqD0DtPu/+/qNvdYfue+sOVEDlpqO3aawLvR6LOUGD0eXoxqG57Q5v06RhiU/u0A1IKEG/FcgOepzlLWtum1IziwPSgJ0uMAZyDYBzbomZrQOGAxqHWCQaBH+baSv1tV6DcKihaKbRqPMak8ONRpNtDuyC2i1B2x049uqrE7ITfMvo2uTbSSsalrjkdjkRH0rQLwZyzGwwgUC/Ebi5yTazgFuBhcC/AHOdc87MMoBdzrkGMxsC5ADrw1a9iES/uITALbl7eJ+3ob7JN4p9TRqNA02+dTTTsFTvgT3bj96uvrp1dcQHNRj9zoLrXwjv+ySEoPf63O8F5hC4vPJ559xKM3scKHTOzQL+G/iTmZUAuwg0BgDjgcfNrA5oBO5yzu0K+7sQEWmt2DiITYOktPA+b2NDUGOwP/RGo/ZA4HccbUAzTImIRIETzTClX2mIiEQ5Bb2ISJRT0IuIRDkFvYhIlFPQi4hEOQW9iEiUU9CLiEQ5Bb2ISJSLuB9MmVk5sOkUnqIXUBGmcjoDfV6to8+rdfR5tc6pfF4DnXMZza2IuKA/VWZWeLxfh8mx9Hm1jj6v1tHn1Tpt9Xmp60ZEJMop6EVEolw0Bv2zfhfQwejzah19Xq2jz6t12uTziro+ehEROVo0HtGLiEgQBb2ISJSLmqA3s+fNrMzMVvhdS6Qzs2wzKzCzVWa20swe8LumSGZmSWb2iZkt8z6v//S7po7AzGLN7FMze8vvWjoCM9toZsvNrMjMwjr7UtT00ZvZeGAf8Efn3Ei/64lkZtYX6OucW2pmqcAS4CvOuVU+lxaRzMyArs65fWYWDywAHnDOLfK5tIhmZlOAfKCbc+7LftcT6cxsI5DvnAv7D8yi5ojeOTefwHy10gLn3Hbn3FLv/l5gNdDf36oilwvY5z2M927RcYTURswsC7gaeM7vWiSKgl5OjpkNAs4EPva5lIjmdUMUAWXAO845fV4n9hTwfaDR5zo6Ege8bWZLzOzOcD6xgr4TM7MU4A3g35xze/yuJ5I55xqcc2OALGCcmal78DjM7MtAmXNuid+1dDAXOufOAq4C7vG6o8NCQd9JeX3NbwAvOef+6nc9HYVzrhIoACb5XEokuwC41utzfhW41Mz+7G9Jkc85t9X7Wwa8CYwL13Mr6Dsh7+TifwOrnXPT/a4n0plZhpmle/eTgSuANb4WFcGcc//unMtyzg0CbgTmOudu8bmsiGZmXb0LIzCzrsCVQNiuIIyaoDezV4CFQK6ZlZrZd/yuKYJdAHyTwJFWkXf7kt9FRbC+QIGZfQYsJtBHr0sGJZx6AwvMbBnwCfB359zscD151FxeKSIizYuaI3oREWmegl5EJMop6EVEopyCXkQkyinoRUSinIJeOg0zawi6nLTIzB4L43MP0sipEqni/C5ApB0d9IYxEOlUdEQvnZ43DvhUbyzwT8xsmLd8kJnNNbPPzOw9MxvgLe9tZm9649MvM7PzvaeKNbP/8sasf9v7FS1mdr839v9nZvaqT29TOjEFvXQmyU26biYHratyzo0CfkNg5EWAXwN/cM6dAbwEPO0tfxp43zk3GjgLWOktzwFmOOdOByqBr3vLHwPO9J7nrrZ5ayLHp1/GSqdhZvuccynNLN8IXOqcW+8N9rbDOdfTzCoITNBS5y3f7pzrZWblQJZzriboOQYRGBohx3v8KBDvnHvCzGYTmBTnb8Dfgsa2F2kXOqIXCXDHud8aNUH3GzhyDuxqYAaBo//FZqZzY9KuFPQiAZOD/i707n9EYPRFgG8AH3j33wPuhsMTkqQd70nNLAbIds4VAI8CacAx3ypE2pKOLKQzSfZmiTpktnPu0CWW3b3RKWuAm7xl9wEvmNkjQDlwu7f8AeBZb4TUBgKhv/04rxkL/NlrDAx42hvTXqTdqI9eOr22nJRZJBKo60ZEJMrpiF5EJMrpiF5EJMop6EVEopyCXkQkyinoRUSinIJeRCTK/X8PGYbQ8SFJsAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmiElEQVR4nO3de3RV9Z338fc3d3IhQBIuAiFcgqJgBCOiGFQQp7XO2NqqM2PvrfRiLWqnHR/btTqzpvM8085Tb63jDDN1Rp86dar2MtOuWrkpUBUNFpSLJiAg4ZoECATI7eT7/HEOIQkJnEBO9jk5n9daWfucvc/e55sjfPzy27+zt7k7IiKSeFKCLkBERM6NAlxEJEEpwEVEEpQCXEQkQSnARUQSVNpAvllhYaGXlJQM5FuKiCS8devW1bl7Uff1AxrgJSUlVFZWDuRbiogkPDPb2dN6DaGIiCQoBbiISIJSgIuIJCgFuIhIglKAi4gkKAW4iEiCUoCLiCSoAZ0HLiKSkNrbIdQMbU3QFlm2NnV9ftqy27qyP4eCyf1algJcROKfO4Raog/Ms4ZrT8sTvW8PtZznL2Aw/koFuIgEKNQGLY3Qciy64Ott2RGwfdiH87z5TGompGVBWi/LrGHhZXrWmV/X67LzT7dtqelg1h//BbpQgIsMVu7h8Gs5Bi1Hw8vmxm6PI9s6HjeGf5obTwV189GuoX2uUtLPHIAZOZBd0HV9+pCzBOaZQrXTvqkZkDL4TvkpwEXihTu0Hu8Uno19DNlj3dY3QntbdO+dkgYZueGfzMgyIwdyisLLLusj284Yrj2EbEpqbD+/JKQAFzlX7aHTQ7ajs+1DyHbeJ9phgpMda0YuZOaFH2flw9ALIs8jIZuZCxl5nR7nhJ93fpyREw7ZGPwTX2JLAS7Jqb0dThyEo3vh6H44VhsJ1qNn6HK7DT+0nYj+/dJzuna2mXmQOxIyJp3e2Z4Wut0DODc8pipJTwEug0t7KBzGR/eFfxr3hQO6cV+ndfvDP70NL1hKt9CMBOew8d1CtvuQQ+7pnW1mbji8B+H4qwRPAS6JIdQaDt2OMN7b6fH+8PPGSCft7afvn10AuaMhbxQUXQR5o8M/uaPCy5wiyBwaDty0LA0nSEJQgEuwWpu6dcmdwvhkt3x0Lxyv72FnCwfvyTAeU9YpmDsFdO4oSMsY8F9NJNYU4BIbLcfOPoxxdB80HT59X0uNdMajYFgxjLuiU7c8Jrw+N9I1p+qPsCSvqP70m9li4C7AgH9190fM7DLgn4EsoA34qru/EatCJQ64Q/OR3sP4ZLd8dH/4ZF93qRmnhjEKpkBJxakw7tw5ZxdozFgkCmcNcDObTji8ZwMtwItm9hvgB8DfuvvvzOymyPPrYlirxIo7nDjUtVs+bRgjEtY9zbxIG3IqgEdNhyk3nN4t542GIcM1tizSj6LpwKcBa939OICZvQLcSnjC6tDIa/KBPTGpUM5de3t47LhLGHcP6Eg33dO1HjLywgGcNwbGXt71pF/HOPOo8Mk/BbPIgIsmwDcCf29mBcAJ4CagErgX+L2Z/V/Cl6W9uqedzWwRsAiguLi4H0qWHjU3wu51sOsN2LUWDmzufapc1rBTYTzh6k5dcrfhjIycAf81RCR65n72b36Z2ReArwLHgE1AM+HQfsXdXzCz24FF7n7DmY5TXl7ulZWV5191snOHwx+cCutda2H/xlPT54qmhWdkDL2g2zBGZJmeFWz9ItInZrbO3ctPWx9NgHc70P8GaoD/AwxzdzczAxrcfeiZ9lWAn6O2Ztj79qmw3vVGeNgDwl8SGVcevlTl+Cth3OXhsWYRGTR6C/BoZ6GMdPcDZlZMePx7DnAPcC3wMjAfqO6/cpNc44FO3fUbsOeP4YvJAwybABPnwfjZ4cAeebGm0onEsZa2dhqb28jJTCUzrX8v6BXt3/wXImPgrcDd7n7YzO4CHjWzNKCJyDi39FF7CA5sORXWu9bCoe3hbSnpcMFlMPuuU4GdNzrQckWSRVuonWPNIY40tdLY3EZjcxtHm1o52tTG0abw88amyLqOx6deF1620dwWHtp8+vOzmTe1qF9rjCrA3b2ih3VrgMv7tZpk0NQANZWnwrqm8tSc6ZyicEiXfy68HHOZxqtF+qi93WlsCQdq59A9Gai9hu7J4I2sO9EaOut7pRjkZqaRl5VOXlYauZlpFOZmUFKYQ25mGkMj63Kz0phY2P+TAvRv71hyh4Pvdx0OObCZ8AxMg1GXwKW3RcavZ8PwiZqOJ0nL3TneEuoI2s5dbGNT14DteE0P6xqbo7sGel4kWMMBnEb+kHTGDRvSEcS5WZFgjmw/9dpTYZ2dkYoF+HdWAd6fWk/AnvVdh0OO14W3ZQ4Nn2y8+M/CYT22HLLOeM5XJCG4O81t7V1Ct7GpjSOdhxMij490DD107YpP7tcexZyK7IzUjtDNjQTs6KFZUYduXlYaORlppKQkfrOkAD8fR/Z2Deu9G6C9NbxtxCQoXXhq7LroIt2RRBJCe7tz6HgLdY0t1B5tpq6xucuytrGZ+sYWjjafCubW0NmTNzMt5bQgLc7JJjcrjaFZ6Z1COS0y/JDepUPOy0wnJzOVtFRdZuEkBXi0Qm3hudadh0MaPghvS82EsbPgqq9GpvLNhtz+PVkhcj7cncPHW8Mh3BHIPQd0/bEWQj20whlpKRTlZlKYl8mY/CwuHJLXqetNi3S96d3WnQrhjDQFb39TgPfmxKHIycbI3OuaddB6LLwtdzQUXwlzvhwO7NGX6nKlMuDcnSNNbT2GcF23kK4/1txjl5yeahTmZlKUl8mooVlMvyCfwryMjqA+uSzMzWRoVlqg471yOgU4hE821m/t+kWZ2nfD2ywVRk+HmXeeOtmYP14nGyUm3J3G5rYzdsidw7kldPrNK9JSjILcDIoiwXvR6LyOEA6vy2Bk5Hn+kHSFcgJLzgBvOQ573uo6fn3iUHhb1rBwSM/4RDiwL5gVvkuLyHk41txDp9xLQJ+cN9xZikFB7qkQnjwyl6JIh1zUJZwzGTYkfVCcoJOzS44Ab6jpGtb73jl1kafCqXDRR059Fb2gVNeilqicaAlR19jMgSiGMHqaU2wGBTkZHeE7sTCno0M+GcYntw3PziBVoSzdDL4AD7XCvre7nmw8sju8LW1IeCrf3MWRk41XQPaIYOuVuNLUGjotfHsL52MtPX/RY0RORkcIzywe1qU77hzQI7IzNKNCzkviB/ixeqjpFNa73zp104H88VA859TY9ajpkJoebL0SN2qPNvOHrXWsqq5l/a7D1B5t5mhTz18CyR+S3hG+M8YN69Ildx7KGJGTQbpCWQZIYgV4ezvUvdd1OKR+a3hbSlr4EqrlnwuH9bjZkD822HolrjS1hnhzx0FWV9exurqOLXuPADAsO53ZJSOYV1rUEdKdu+aC3Ix+vwiRSH9IjADf8F/wzs9h15vQ3BBel10Q7qxnfjJysnEmpA8Jtk6JK+3tzrv7jrK6upY1W+t4Y/tBmtvaSU81Lp8wnG/+yYVUlBZyyQX5Gl+WhJQYAV5fDUf2wPSPnTrZOGKSpvLJafYfaWJ1dR1rIqFd1xi+VdzUUbnceeUEKkoLuXLSCLIzEuOPvsiZJMaf4uu/DfO/E3QVEoeOt7SxdvtB1lTXsbq6lqr9jQAU5mYwd0ohFaVFXDOlkNH5uqqjDD6JEeDqtCWivd3ZtOcIq7fWsrqqjnU7D9ESaicjLYXZJSP4+KxxXFNayLTRQzUXWga9xAhwSWp7Dp9gTXV4tsir2+o5eCw8LHLR6Dw+O7eEa6YUMnviCLLSdaJRkosCXOJOY3Mba9+vj8wWqWVbbfgaNEV5mVx3YREVpYXMnVLIyDwNi0hyU4BL4ELtzju7G1hdVcvqrXW8tfMQbe1OVnoKV04s4C9mF1NRWsTUUbm6bodIJwpwCcSug8fDs0W21vKHrfU0nAhfR3362KF8sWIS80oLmTVhuIZFRM5AAS4D4khTK69tq++YLbKj/jgAY/KzuPHiUVRMLWLu5AIKcjMDrlQkcSjAJSbaQu1sqDnc8a3H9bsOE2p3sjNSmTOpgM9cXUJFaSGTizQsInKuFODSL9ydnfXHWb21jtVVtby2rZ6jzW2YwaXjhvGVaydTUVrIzOLhujOLSD9RgMs5azjeyqvb6lgVGcvedTB8EbGxw4Zwc9kYKkqLuHpyAcOydbcikVhQgEvUWkPt/PGDw6yurmV1dR1v1xym3SE3M42rJhewqGIS15QWUVKQrWERkQGgAJdeuTvv1x1jdVX4uiKvbavnWEuIFIPLxg/jnvmlVJQWUjZ+mC6hKhIABbh0cfBYC3/YWtcxW2RPQxMAEwqy+dissVwzpYirJheQP0TXVRcJmgI8yTW3hVi381AksOvYuKcBdxialcbcKYXcPb+QiilFFBdkB12qiHQTVYCb2WLgLsCAf3X3RyLr7wHuBkLAb939WzGqU/qJu1N9oLHja+pr3z/IidYQaSnGrOLh3HfDVCpKC5kxNl+3+xKJc2cNcDObTji8ZwMtwItm9htgPHALUObuzWY2MqaVyjmra4zcOqwqPFtk/5FmACYV5XB7+TgqSouYM7mA3Ez9g0wkkUTzN3YasNbdjwOY2SvArUA58A/u3gzg7gdiVqX0SXu78+q2+o7ZIps73Tps7pRC5pUWck1pEWOH6Q5GIoksmgDfCPy9mRUAJ4CbgEpgKlBhZn8PNAF/5e5vdt/ZzBYBiwCKi4v7q245g3986T2eeHmbbh0mMsidNcDdfYuZfR94CTgGrCc85p0GjADmAFcAPzezSe7u3fZfAiwBKC8v77JN+t/+I008uWY7N186hh984lLdOkxkEIvqLJW7/8TdL3f3ecAhoAqoAX7hYW8A7UBh7EqVaPzTyq2E2p1v/clFCm+RQS7aWSgj3f2AmRUTHv+eQziwrwdWmtlUIAOoi1mlcla7D5/gZ2/s4rbycZr2J5IEom3RXoiMgbcCd7v7YTN7EnjSzDYSnp3yme7DJzKwfryiGoCvzS8NuBIRGQhRBbi7V/SwrgX4ZL9XJOdkZ/0xnqus4c4rizW7RCRJ6Jsag8Sjy6tJTTHuvn5K0KWIyABRgA8CWw808qs/7uZTcyYwcqhu9CuSLBTgg8Cjy6vJSk/ly9dNDroUERlACvAE9+6+I/zPhj189uoSCnU/SZGkogBPcA8vrSIvM41F8yYFXYqIDDAFeALbuLuB32/azxcqJuq2ZSJJSAGewB5aWkX+kHQ+f83EoEsRkQAowBPUup2HWPHuARbNm8TQLN0dRyQZKcAT1MNLqyjIyeCzV5cEXYqIBEQBnoBef7+eNVvr+Mp1k8nRTRhEkpYCPMG4Ow+9VMXIvEw+OWdC0OWISIAU4AlmzdY63thxkK/Nn0JWemrQ5YhIgBTgCcTd+eFLVVyQn8UdV4wPuhwRCZgCPIGsePcA63cd5p4FpWSmqfsWSXYK8ATh7jy0tIriEdl84vJxQZcjInFAAZ4gfr9pH5v2HGHxglLSU/WfTUQU4Akh1B7uvicV5fDRmWODLkdE4oQCPAH85u09VO1v5L4bppKaYkGXIyJxQgEe59pC7Ty6rJqLRufxkRljgi5HROKIAjzO/Wr9Ht6vO8a9N0wlRd23iHSiAI9jraF2Hl1exfSxQ/mTS0YFXY6IxBkFeBx7rrKGXQdP8I2FF2Km7ltEulKAx6mm1hA/WlHNzOJhXHdhUdDliEgcUoDHqWff+IC9DU3qvkWkVwrwOHSiJcTjL2/jyokjmDulIOhyRCROKcDj0E9f30nt0Wa+caO6bxHpnQI8zjQ2t/HEK9uoKC1k9sQRQZcjInEsqgA3s8VmttHMNpnZvd22fcPM3MwKY1Jhknnq1R0cPNbC/QunBl2KiMS5swa4mU0H7gJmA2XAzWY2JbJtPHAj8EEsi0wWDSda+ZdXtrHgopHMLB4edDkiEuei6cCnAWvd/bi7twGvALdGtj0MfAvwGNWXVH6yZjtHmtq4T923iEQhmgDfCFSYWYGZZQM3AePN7BZgt7tvONPOZrbIzCrNrLK2trYfSh6cDh1r4ck12/nw9NFMH5sfdDkikgDOektzd99iZt8HXgKOAeuBTOBBwsMnZ9t/CbAEoLy8XJ16L5asfp9jLeq+RSR6UZ3EdPefuPvl7j4POARsAiYCG8xsBzAOeMvMRses0kGsrrGZ//jDDv700guYOiov6HJEJEFEOwtlZGRZTHj8+yl3H+nuJe5eAtQAs9x9X8wqHcSeeHkbzW0h7r2hNOhSRCSBnHUIJeIFMysAWoG73f1w7EpKLvuPNPHT13dy66xxTCrKDbocEUkgUQW4u1ecZXtJv1SThB5fuZVQu7N4gbpvEekbfRMzQDWHjvOzNz7g9ivGM35EdtDliEiCUYAH6McrtmIYX7t+StCliEgCUoAHZEfdMZ5bV8NfXlnMBcOGBF2OiCQgBXhAHlteTVqK8dXrJgddiogkKAV4ALYeaORX63fzmatLGDk0K+hyRCRBKcAD8MiyKrLSU/nSvElBlyIiCUwBPsC27D3Cb97ey+fmllCQmxl0OSKSwBTgA+zhpVXkZaWxqEJj3yJyfhTgA+idmgZe2ryfL14zifzs9KDLEZEEpwAfQA8tfY9h2el8/pqSoEsRkUFAAT5A1u08xMr3avnSvMnkZan7FpHzpwAfIA8tfY/C3Aw+c/WEoEsRkUFCAT4AXttWzx+21vPlayeTnRHtBSBFRM5MAR5j7s5DS99j1NBMPjlH3beI9B8FeIytrq7jzR2H+Nr1U8hKTw26HBEZRBTgMeTu/HBpFWOHDeH2K8YHXY6IDDIK8BhavuUAG3Yd5usLppCZpu5bRPqXAjxG2tudh5ZWMaEgm1tnjQu6HBEZhBTgMfL7TfvYvPcIixeUkp6qj1lE+p+SJQZCke57clEOt1w2NuhyRGSQUoDHwG/e3kP1gUbuWziV1BQLuhwRGaQU4P2sLdTOI8uquWh0HjdNHxN0OSIyiCnA+9kv/7ib7XXHuG/hVFLUfYtIDCnA+1FLWzuPLq9mxth8brx4VNDliMggpwDvR8+t20XNoRPcf+NUzNR9i0hsKcD7SVNriB+v2Mqs4mFcN7Uo6HJEJAkowPvJz974gL0NTfzVjReq+xaRARFVgJvZYjPbaGabzOzeyLp/NLN3zextM/ulmQ2LZaHx7ERLiMdXbmPOpBFcPaUw6HJEJEmcNcDNbDpwFzAbKANuNrMpwFJgurtfClQB/yuWhcaz//f6Duoam/nGjRcGXYqIJJFoOvBpwFp3P+7ubcArwK3u/lLkOcDrQFJe8KOxuY0nXt7GvKlFXFEyIuhyRCSJRBPgG4EKMysws2zgJqD7tVE/D/yup53NbJGZVZpZZW1t7flVG4f+4w/bOXS8lfsXTg26FBFJMmcNcHffAnwfeAl4EVgPhE5uN7NvA23AM73sv8Tdy929vKhocM3OaDjRypJV73PDtJFcNn5Y0OWISJKJ6iSmu//E3S9393nAIcJj3pjZZ4GbgTvd3WNWZZz6yer3OdLUxn3qvkUkAFHdYdfMRrr7ATMrBm4F5pjZh4BvAde6+/FYFhmPDh1r4ck/7OCmGaO55IL8oMsRkSQU7S3SXzCzAqAVuNvdD5vZj4FMYGlk3vPr7v7lGNUZd/5l1fsca2nj3hvUfYtIMKIKcHev6GHdlP4vJzHUHm3mqVd3cEvZBUwdlRd0OSKSpPRNzHPwxMvbaAm1s1jdt4gESAHeR/samvjp2p3cOnMsEwtzgi5HRJKYAryPHl+5FXfn6wtKgy5FRJKcArwPag4d59k3P+D28vGMH5EddDkikuQU4H3wo+VbMTO+Nj9pz9+KSBxRgEdpR90xnn+rhr+cXcyY/CFBlyMiogCP1qPLq0lPNb56/eSgSxERARTgUdl64Ci/Wr+bz1xVwsi8rKDLEREBFOBReXhZNdnpqXzpWnXfIhI/FOBnsXnPEX779l4+f81ERuRkBF2OiEgHBfhZPLysirysNL54zaSgSxER6UIBfgZv1xxm6eb93FUxifzs9KDLERHpQgF+Bg8trWJ4djqfm1sSdCkiIqdRgPdi3c6DvPxeLV+6djJ5Weq+RST+KMB78cOXqijMzeDTV00IuhQRkR4pwHvw6rY6Xt1Wz1evm0J2RrT3vBARGVgK8G7cnYdeqmL00Cz+8srioMsREemVArybVdV1VO48xN3zp5CVnhp0OSIivVKAdxLuvt9j7LAh3FE+PuhyRETOSAHeybItB9hQ08DiBaVkpOmjEZH4ppSKaG93HlpaRUlBNrfOGht0OSIiZ6UAj3hx0z627D3C4htKSUvVxyIi8U9JBYQi3feUkbn8WZm6bxFJDApw4H827GHrgUbuu2EqqSkWdDkiIlFJ+gBvC7XzyLIqLhqdx4enjw66HBGRqCV9gP/ird3sqD/ON268kBR13yKSQJI6wFva2nl0eTVl4/K5YdrIoMsREemTqALczBab2UYz22Rm90bWjTCzpWZWHVkOj2mlMfDzyl3sPnyC+xZOxUzdt4gklrMGuJlNB+4CZgNlwM1mNgV4AFju7qXA8sjzhNHUGuLHK7ZSPmE4104tCrocEZE+i6YDnwasdffj7t4GvALcCtwCPBV5zVPAR2NSYYz859oP2HekiftvVPctIokpmgDfCFSYWYGZZQM3AeOBUe6+N/KafcConnY2s0VmVmlmlbW1tf1S9Pk60RLin17exlWTCrh6cmHQ5YiInJOzBri7bwG+D7wEvAisB0LdXuOA97L/Encvd/fyoqL4GKp4+rUd1DU2840bpwZdiojIOYvqJKa7/8TdL3f3ecAhoArYb2ZjACLLA7Ers/80Nrfxz69s49qpRZSXjAi6HBGRcxbtLJSRkWUx4fHv/wT+G/hM5CWfAX4diwL727+v2c6h463cv1Ddt4gktmjvF/aCmRUArcDd7n7YzP4B+LmZfQHYCdweqyL7S8PxVpasfp+FF4+ibPywoMsRETkvUQW4u1f0sK4eWNDvFcXQv615n6NNbeq+RWRQSJpvYh481sKTa7bzkRljmDZmaNDliIict6QJ8H9ZtY0TrSHuW1gadCkiIv0iKQL8wNEmnnp1B7dcNpYpI/OCLkdEpF8kRYA/8fI2WkPO4gXqvkVk8Bj0Ab634QTPrP2Aj88aS0lhTtDliIj0m0Ef4I+v3Iq7c898dd8iMrhEOw88Ie06eJz/enMXd1wxnvEjsoMuRyTptba2UlNTQ1NTU9ClxKWsrCzGjRtHenp6VK8f1AH+oxXVmBlfu17dt0g8qKmpIS8vj5KSEl0FtBt3p76+npqaGiZOnBjVPoN2CGV73TFeeGs3n7xyAqPzs4IuR0SApqYmCgoKFN49MDMKCgr69K+TQRvgjy6rIiM1ha9cNznoUkSkE4V37/r62QzKAK/ef5Rfb9jDp6+eQFFeZtDliIjExKAM8EeWVZOTkcaX56n7FpHBa9AF+KY9Dfz2nb18fm4Jw3Mygi5HRCRmBt0slIeXVjM0K40vVEwKuhQROYO//Z9NbN5zpF+PefEFQ/nun15yxtd89KMfZdeuXTQ1NbF48WIWLVrEiy++yIMPPkgoFKKwsJDly5fT2NjIPffcQ2VlJWbGd7/7XT7+8Y/3a73na1AF+IZdh1m2ZT9/deNU8odEN49SRJLLk08+yYgRIzhx4gRXXHEFt9xyC3fddRerVq1i4sSJHDx4EIC/+7u/Iz8/n3feeQeAQ4cOBVl2jwZVgP9waRXDs9P57Nzo5lCKSHDO1inHymOPPcYvf/lLAHbt2sWSJUuYN29ex9zrESPCt1pctmwZzz77bMd+w4cPH/hiz2LQjIFX7jjIqqpavnztZHIzB9X/l0Skn7z88sssW7aM1157jQ0bNjBz5kwuu+yyoMs6Z4MmwH/4UhWFuZl8+qqSoEsRkTjV0NDA8OHDyc7O5t133+X111+nqamJVatWsX37doCOIZSFCxfy+OOPd+wbj0MogyLAX91ax2vv13P39ZMZkpEadDkiEqc+9KEP0dbWxrRp03jggQeYM2cORUVFLFmyhFtvvZWysjLuuOMOAL7zne9w6NAhpk+fTllZGStXrgy4+tMl/FiDu/PDpVWMyc/iL2YXB12OiMSxzMxMfve73/W47cMf/nCX57m5uTz11FMDUdY5S/gO/JWqWtbtPMTX5k8hK13dt4gkj4QOcHfnoaVVjBs+hNsuHx90OSIiAyqhA3zp5v28XdPA1xeUkpGW0L+KiEifJWzqtbeHu++JhTncOnNs0OWIiAy4hA3w323cx7v7jnLvDaWkpSbsryEics4SMvlC7c7Dy6ooHZnLzZdeEHQ5IiKBSMgA/+8Nu9l6oJH7Fk4lNUUXhxeR5BRVgJvZfWa2ycw2mtnPzCzLzBaY2Vtmtt7M1pjZlFgXC9AaaufRZdVcPGYoH7pk9EC8pYgkqdzc3KBLOKOzfpHHzMYCXwcudvcTZvZz4M+BB4Fb3H2LmX0V+A7w2VgWC/CLt2rYUX+cf/t0OSnqvkUS1+8egH3v9O8xR8+AD/9D/x4zjkU7hJIGDDGzNCAb2AM4MDSyPT+yLqZa2tp5bPlWysYPY8G0kbF+OxEZZB544IEu1zf5m7/5G773ve+xYMECZs2axYwZM/j1r38d1bEaGxt73e/pp5/m0ksvpaysjE996lMA7N+/n4997GOUlZVRVlbGq6++ev6/kLuf9QdYDDQCtcAzkXUVQD1QA2wGhvay7yKgEqgsLi728/H0azt8wl//xl9578B5HUdEgrF58+ZA3/+tt97yefPmdTyfNm2af/DBB97Q0ODu7rW1tT558mRvb293d/ecnJxej9Xa2trjfhs3bvTS0lKvra11d/f6+np3d7/99tv94Ycfdnf3trY2P3z4cI/H7ekzAiq9h3yNZghlOHALMBE4DDxnZp8EbgVucve1ZvZN4CHgiz38D2IJsASgvLzcz/V/NE2tIX68oporSoZTUVp4rocRkSQ2c+ZMDhw4wJ49e6itrWX48OGMHj2a++67j1WrVpGSksLu3bvZv38/o0ef+Rybu/Pggw+ett+KFSu47bbbKCwM59TJ64uvWLGCp59+GoDU1FTy8/PP+/eJ5mJWNwDb3b0WwMx+AcwFytx9beQ1/wW8eN7VnMEzaz9g/5FmHrljJmYa+xaRc3Pbbbfx/PPPs2/fPu644w6eeeYZamtrWbduHenp6ZSUlNDU1HTW45zrfv0pmjHwD4A5ZpZt4eRcQHjIJN/MpkZesxDYEqMaOd7SxhMvb2XulAKumlwQq7cRkSRwxx138Oyzz/L8889z22230dDQwMiRI0lPT2flypXs3LkzquP0tt/8+fN57rnnqK+vB05dX3zBggU88cQTAIRCIRoaGs77dzlrgEe67OeBt4B3IvssAe4CXjCzDcCngG+edzW9ePq1ndQ1tnD/wgtj9RYikiQuueQSjh49ytixYxkzZgx33nknlZWVzJgxg6effpqLLrooquP0tt8ll1zCt7/9ba699lrKysq4//77AXj00UdZuXIlM2bM4PLLL2fz5s3n/btYeHx8YJSXl3tlZWWf93t+XQ1vbK/nB58oi0FVIjJQtmzZwrRp04IuI6719BmZ2Tp3L+/+2oS4ocMnLh/HJy4fF3QZIiJxJSECXEQkKO+8807HXO6TMjMzWbt2bS97DBwFuIgMKHdPqJlkM2bMYP369QPyXn0d0k7Ii1mJSGLKysqivr6+z0GVDNyd+vp6srKyot5HHbiIDJhx48ZRU1NDbW1t0KXEpaysLMaNi/58nwJcRAZMeno6EydODLqMQUNDKCIiCUoBLiKSoBTgIiIJakC/iWlmtUB0Fxo4XSFQ14/lDHb6vPpGn1ff6PPqu/P5zCa4e1H3lQMa4OfDzCp7+iqp9EyfV9/o8+obfV59F4vPTEMoIiIJSgEuIpKgEinAlwRdQILR59U3+rz6Rp9X3/X7Z5YwY+AiItJVInXgIiLSiQJcRCRBxX2Am9mTZnbAzDYGXUsiMLPxZrbSzDab2SYzWxx0TfHMzLLM7A0z2xD5vP426JoSgZmlmtkfzew3QdcS78xsh5m9Y2brzazvtyQ707HjfQzczOYBjcDT7j496HrinZmNAca4+1tmlgesAz7q7ud/A75BKHKj7hx3bzSzdGANsNjdXw+4tLhmZvcD5cBQd7856HrimZntAMrdvd+/+BT3Hbi7rwIOBl1HonD3ve7+VuTxUWALMDbYquKXhzVGnqZHfuK7qwmYmY0DPgL8W9C1JLu4D3A5d2ZWAswEgr/3UxyLDAesBw4AS91dn9eZPQJ8C2gPuI5E4cBLZrbOzBb154EV4IOUmeUCLwD3uvuRoOuJZ+4ecvfLgHHAbDPTUF0vzOxm4IC7rwu6lgRyjbvPAj4M3B0ZFu4XCvBBKDKW+wLwjLv/Iuh6EoW7HwZWAh8KuJR4Nhf4s8i47rPAfDP7abAlxTd33x1ZHgB+Cczur2MrwAeZyEm5nwBb3P2hoOuJd2ZWZGbDIo+HAAuBdwMtKo65+/9y93HuXgL8ObDC3T8ZcFlxy8xyIpMJMLMc4Eag32bUxX2Am9nPgNeAC82sxsy+EHRNcW4u8CnCndH6yM9NQRcVx8YAK83sbeBNwmPgmhon/WUUsMbMNgBvAL919xf76+BxP41QRER6FvcduIiI9EwBLiKSoBTgIiIJSgEuIpKgFOAiIglKAS6DgpmFOk2bXG9mD/TjsUt0NUyJR2lBFyDST05Evg4vkjTUgcugFrkW8w8i12N+w8ymRNaXmNkKM3vbzJabWXFk/Sgz+2Xk+uAbzOzqyKFSzexfI9cMfynyrU3M7OuRa6+/bWbPBvRrSpJSgMtgMaTbEModnbY1uPsM4MeEr6QH8CPgKXe/FHgGeCyy/jHgFXcvA2YBmyLrS4HH3f0S4DDw8cj6B4CZkeN8OTa/mkjP9E1MGRTMrNHdc3tYvwOY7+7vRy7ytc/dC8ysjvCNL1oj6/e6e6GZ1QLj3L250zFKCH/FvjTy/K+BdHf/npm9SPiGI78CftXp2uIiMacOXJKB9/K4L5o7PQ5x6vzRR4DHCXfrb5qZzivJgFGASzK4o9PytcjjVwlfTQ/gTmB15PFy4CvQcaOH/N4OamYpwHh3Xwn8NZAPnPavAJFYUbcgg8WQyF11TnrR3U9OJRweudpgM/AXkXX3AP9uZt8EaoHPRdYvBpZErnoZIhzme3t5z1Tgp5GQN+CxyDXFRQaExsBlUIvlDWVFgqYhFBGRBKUOXEQkQakDFxFJUApwEZEEpQAXEUlQCnARkQSlABcRSVD/Hw6X07x+rBz8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEGCAYAAABrQF4qAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAt2UlEQVR4nO3deXxU9b3/8dcnO1kIZGENkAAJhRCWEFllKYhLtS5YrnqVK9Rfrd7a4l68vfe29XbzdnG3lrZurY/SSmvrbb1VQLiAFQUUVFQSCEgCCFkgZCHLTD6/P+YkDCGBhExyJpPP8/HIY86cc+bkM4O++fCdc75HVBVjjDGhK8ztAowxxnQtC3pjjAlxFvTGGBPiLOiNMSbEWdAbY0yIi3C7gJZSUlI0PT3d7TKMMaZH2b59e6mqpra2LeiCPj09nW3btrldhjHG9Cgi8mlb22zoxhhjQpwFvTHGhDgLemOMCXEW9MYYE+Is6I0xJsRZ0BtjTIizoDfGmBAXdOfRG2NMj+X1gKcWPHXgOek81vqtcx4bTp7+vOkxfgDkLQt4WRb0xpjQodp6gDaHbRvrG9ra1jKwWzy2fJ16O1d/2gUW9MaYHsLbAHWVThieJSjbDN+WXW8HwrezIvpARDRExJz+GOmsj05oZbv/z1le2+p2v9eFhXe+/tbeUpcc1RjTczR1wfVVvnCur/Yt11dBnfNYX+1sa1qugvpKv+Vq3/OmZW/d+dcTFtF22EbEQHQ8xKW0sf0cgRp5ljCO6APhkSASuM82SFjQG9PTqEJDjV8IV7Wx3Fo4++9TfSqsGz3t+91hkb6gjXJ+ouMhKs43thyd4FtuXh8PkbGtB+sZgessh0dDuMVSoNknakxXa/SepRNuGbwtuuq29qGd93qOiHFCOc4J4niI6QeJaRDlBPMZwd1i/+blOF8Ymx7Hgt6Ys2n0QnUJVB6Gys/g5PHWu+LTgtovnOurfd13e0XGnRm28QMgamTrnXRUQhvLzn7WHRss6E1v1dgINaWnArz58bPTn1cfBW1s/RgS1krwxkO/YX5h21pnHO/XSft11ZFxEGaXtpjAs6A3oaWxEU6WnzvAq460fipcbAokDIaEQTAox1ke6HuMHwSx/U+Fe2SfkPzizoQeC3rTM6jCyWNOUPuH+JHTw7zqs9a/WOyTdCrAB4w7Fd4Jg049xg2AiKjuf2/GdDELeuMuVag9fvbuuynAvfVnvj6m36mgTslqPcDjB9qXiKZXa1fQi8ilwKNAOPArVf1Ri+0jgGeAVKAcuElVi51t/w1cjm9enTXAclVt5ykDpsdShboT5wjww74hlNYucolOdMJ6EIyY2XaAR/bp/vdmTA9zzqAXkXDgSWAhUAxsFZFXVPUjv91+Arygqs+LyHzgh8ASEZkJzAImOPttBuYCGwL3Fky3q6tsX4C3drZJVMKpAB82rY0AHwRRsd3/vowJUe3p6KcCe1S1EEBEVgFXAf5BPw6421leD/zZWVYgBogCBIgEjnS6atM16qtbCfAWy1VHnPO4W4iMdYJ6MAzNbT28Ewb6zjoxxnSr9gT9UKDI73kxMK3FPjuBRfiGd64BEkQkWVXfEpH1wGF8Qf+Eqn7c8heIyK3ArQDDhw/v8JswHdDohaMfQ/E7cHA7HPvUF96Vn/mGWlqK6HMqsAdPaCPAB/kC3M5AMSYoBerL2HuBJ0RkKbAROAh4RWQ0MBZIc/ZbIyKzVXWT/4tVdSWwEiAvL8/G7wOpphyKt/mCvegdOPiu7wIfgNhkSB4NA8bCqPmtB3hMogW4MT1ce4L+IDDM73mas66Zqh7C19EjIvHAtap6XES+AmxR1Spn2/8CM4DTgt4EiH+3XrTV91i2x7dNwmFgNky8DtKmwrALoH+GhbgxvUB7gn4rkCkiGfgC/nrgn/13EJEUoFxVG4EH8J2BA3AA+IqI/BDf0M1c4JHAlG583fpWX6de3NStO+PnsSkwbCpMutH3OGSy7wpMY0yvc86gV1WPiNwBvIbv9MpnVHWXiDwIbFPVV4B5wA9FRPEN3XzNeflqYD7wAb4vZv+uqv8T+LfRCzR64ehHTqhv9f34d+uDxsPEG3w3LrBu3Zhu5/E2Ul3npbKugao6D1W1HirrPFQ7y1V1Hiqdx+bndR6qak/tnzUogeeWTQ14bRJsp7Tn5eXptm3b3C7DfdVlpwK9rW497QLr1o3ppDqP9xxB7KGqrqE5uJu2+Yd5Va2Hkw3tu7tUfHSE7yfG95jgPMZHRzBqQDy3zR11Xu9DRLaral5r2+zK2GDQslsvegfK9/q2+XfrTeHeP926ddOrqSo19d7TAvf0oG5oEdSeNoO63tvGpHV+wsPktEBOiIkgKS6K4UmxfusjiY+JIMEvxFs+j4uKICys+//ftaB3Q3O37ncmTEO1b1tcqu/L0twlvschk6xbNyHD26gtwrnhzC76HMMbTcMhje0YjIiOCDsVxM7jkH59zljnH+K+cI48bVt0RBjSg5srC/qu5vX4uvXmM2G2tujWc2Dyjb5O3bp104OoKsdrGiipqqO0so6SqjpK/B8r6zhxsuG0Lrqmvn3DG3FR4X5dcSQJ0REMSIg5M5hPex552ra46AiiImzaZ7CgD7zqMl+oNw3BnLVbn2yX+pugoqpU1nl8we2Etn+Il1bVN4d4WXUdDd4z2+qo8DBSE6JJiY+if1wUw5Ji2xiXjmw1uOOiIgh3YXgjlFnQd8YZ3fo7UF7o2xYWAQPHO926c956vxHWrRtXnKz3OsFdS0llvV9wn+q+m5brPGeOWYeHCSnxUaTER5OaEM3nBiWQkhBNqvO8aX1qQjR9YyJ69DBHKLKg74jqUr/z1re26NYH+L4szb3Z9zh4knXrpkvVebyUVtWfpfs+FeLVrQyZiEBSbFRzQGekxPmW46NJSYgiNT7mVGceG+XKl4gmMCzo2+L1wNFdp5+37t+tD8qByTedOm/dunUTAB5vI+XV9Rxtc9iktnm54mRDq8dI7BPZHNA5af38gvtU9z0gIZqkuCgiwm0MuzewoG9SXXp6qFu3bgKksVE5VlPvBHe9M3xy+nh3U/ddXlNPa5e2xEdHkBLv677HDEpgVnzrwybJ8VFER4R3/5s0Qa13Bn3Lbr3oHTi2z7fNv1tvOm+933Dr1s1pVJUTJz2nnWnS1rBJWXU93lbOBYyOCGsO6GFJseSO6O9036dCvKkbj43qnf+rmsDoHf/1VJX4nbe+FQ69e+qmGPEDfWGet8z3aN26aeHQ8ZNsKijhzT1lfFpW3dyJt3ahTUSYNHfZA/vGMH5Iot+wSUxzV56aEE18tH1pabpH6AW91wNHPjx9sq9j+33bwiJg0ASYvMS6ddOm6joPWwrL2FRQyqaCEvaW+IbwUhOiGTu4L6MGxDd326ktzjxJ7BNpX1qaoBM6QX/iMPzx/7XRrX/51FWmdo9R04K3UfnwYAWbCkrYWFDKeweO0eBVYiLDmJaRzA1ThzM7M5WsgfHWgZseKXSCPjYZGhsg919OTfaVOMy6ddOq4mM1bCooZXNBKZv3lDafwZI9pC+3XDiSOZkp5I7oT0ykfbFper7QCfqIKLjldberMEGqsraBLYXlbCooYVNBKftKfcMxg/rGsHDcQGZnpjBrdAop8dEuV2pM4IVO0Bvjx+Nt5P2DFWx2xtnfO3AcT6PSJzKc6SOTWDJ9BLMzUxg9wIZjTOizoDcho6i8ho0FJWwuKOXNPaWcqPUgAuOHJHLrnJHMzkwld0Q/O8/c9DoW9KbHOlHbwFt7y9jkhPv+Mt+X8IMTY7h0/CBmZ6Yya3QKSXFRLldqjLss6E2P4fE2srP4uHPaYyk7io7jbVRio8KZMTKZm2emMzszlVGpcTYcY4wfC3oT1D4tq2ZjQSmb8kt4a28ZlXW+4ZgJQxO5fe4oZmemMHl4f5t33JizsKA3QaXiZANv7S31hXtBCUXlJwEY2q8PV0wczIWjU5k5Kpn+NhxjTLtZ0BtXNXgb2VF0vPkq1J1Fx2lU3x2GZoxK4SuzR3Lh6BQyUmw4xpjzZUFvupWqsr+sxncVan4pWwrLqKrzECYwIa0fd3x+NLOzUpk0rB+RNoWuMQFhQW+63PGaev7hnB2zMb+Ug8d9wzFp/fvwxYlDmJOZwsxRKSTGRrpcqTGhyYLeBFy9p5H3DhzzDcfsKeX94uOoQkJ0BDNGJXPbXN857SOSY204xphuYEFvOk1VKSytZlO+b3qBLYVlVNd7CQ8TJqYl8o35mczJSmFiWj+7o5ExLrCgN+elvLqeN/eUNk8xcKiiFoARybFckzuUC0enMmNUMol9bDjGGLe1K+hF5FLgUSAc+JWq/qjF9hHAM0AqUA7cpKrFzrbhwK+AYYACX1DV/YF6A6Z71Hsa2f7pMd9VqHtK+eBghW84JiaCWaNS+Nr8FGaPTmV4st20xZhgc86gF5Fw4ElgIVAMbBWRV1T1I7/dfgK8oKrPi8h84IfAEmfbC8D3VXWNiMQDZ96WxwQdVWVvSRUb830d+9v7yqlxhmMmD+vHnQuymJ2VwoShiTYcY0yQa09HPxXYo6qFACKyCrgK8A/6ccDdzvJ64M/OvuOACFVdA6CqVYEp23SFsqo6NjcPx5Ty2QnfcExGShxfmpLGhaNTmD4qmb4xNhxjTE/SnqAfChT5PS8GprXYZyewCN/wzjVAgogkA1nAcRH5E5ABrAVWqKrX/8UicitwK8Dw4cPP422Y81Hn8bJ9/zE2FpSyeU8JHx48AUDfmAguzExhdmYqF45OYViSDccY05MF6svYe4EnRGQpsBE4CHid488GJgMHgN8DS4Ff+79YVVcCKwHy8vI0QDWZs6iq83DZoxspKj9JRJiQO6I/9yzMYnZWKjlDEwm3+54aEzLaE/QH8X2R2iTNWddMVQ/h6+hxxuGvVdXjIlIM7PAb9vkzMJ0WQW+637Ob91FUfpKf/dNELs4eRHy0nYBlTKhqz7doW4FMEckQkSjgeuAV/x1EJEVEmo71AL4zcJpe209EUp3n8zl9bN+4oKKmgZWbClk4biCLctMs5I0JcecMelX1AHcArwEfA39Q1V0i8qCIXOnsNg/YLSL5wEDg+85rvfiGddaJyAeAAL8M+LswHfKrzYVU1nq4e2GW26UYY7pBu1o5VX0VeLXFuv/0W14NrG7jtWuACZ2o0QRQeXU9z2zex+UTBjN2cF+3yzHGdAM7AbqX+cXGvZxs8HLXRZlul2KM6SYW9L3I0cpanv/Hfq6aNJTRAxLcLscY000s6HuRn2/YS4NXWb7AunljehML+l7icMVJXtxygC/lppGeEud2OcaYbmRB30s88cYeFOXrC0a7XYoxpptZ0PcCReU1/H5rEddfMJy0/jadgTG9jQV9L/DYugLCw4Q75ls3b0xvZEEf4gpLqvjju8XcNH0EA/vGuF2OMcYFFvQh7tF1BURHhHP7vFFul2KMcYkFfQjLP1LJKzsPsXRWOinx0W6XY4xxiQV9CHt4TT5xURHcOnuk26UYY1xkQR+iPjxYwf9++Bm3XJhB/7got8sxxrjIgj5EPbwmn8Q+kdwyO8PtUowxLrOgD0HvHTjGuk+OcuuckXZ/V2OMBX0o+tmafJLjolg6M93tUowxQcCCPsS8XVjGpoJSbp83iji7c5QxBgv6kKKq/HRNPgMSorlp+gi3yzHGBAkL+hDy5p4y3tlXzh3zRxMTGe52OcaYIGFBHyJ83fxuhiTGcN0Fw9wuxxgTRCzoQ8T63Ud578Bxvr4gk+gI6+aNMadY0IcAVeWnr+czPCmWL01Jc7scY0yQsaAPAa/t+oxdh06wfEEmkeH2R2qMOZ2lQg/nbVR+tiafUalxXD15qNvlGGOCkAV9D/fX9w+Rf6SKOy/KIjxM3C7HGBOELOh7MI+3kUfXFvC5QQlcnjPY7XKMMUHKgr4He/m9gxSWVnPXwizCrJs3xrTBgr6HavA28tgbBeQMTeTicQPdLscYE8TaFfQicqmI7BaRPSKyopXtI0RknYi8LyIbRCStxfa+IlIsIk8EqvDe7qVtxRSVn+Tui7MQsW7eGNO2cwa9iIQDTwKXAeOAG0RkXIvdfgK8oKoTgAeBH7bY/l/Axs6XawBqG7w8/kYBucP7MS8r1e1yjDFBrj0d/VRgj6oWqmo9sAq4qsU+44A3nOX1/ttFZAowEHi98+UagFXvHOBwRS33XjzGunljzDm1J+iHAkV+z4uddf52Aouc5WuABBFJFpEw4KfAvWf7BSJyq4hsE5FtJSUl7au8lzpZ7+WJ9XuZPjKJmaNT3C7HGNMDBOrL2HuBuSLyHjAXOAh4gX8FXlXV4rO9WFVXqmqequalptpQxNn8Zst+SqvquOfiMW6XYozpIdpzZ4qDgP90iGnOumaqeginoxeReOBaVT0uIjOA2SLyr0A8ECUiVap6xhe65tyq6jz8fMNe5mSlckF6ktvlGGN6iPYE/VYgU0Qy8AX89cA/++8gIilAuao2Ag8AzwCo6o1++ywF8izkz99zb+7jWE0D9yzMcrsUY0wPcs6hG1X1AHcArwEfA39Q1V0i8qCIXOnsNg/YLSL5+L54/X4X1dtrVZxsYOXGQi4aO5CJw/q5XY4xpgdp101FVfVV4NUW6/7Tb3k1sPocx3gOeK7DFRoAfr2pkBO1Hu62bt4Y00F2ZWwPUF5dzzNv7ufynMGMG9LX7XKMMT2MBX0P8IuNe6mu93DnRZlul2KM6YEs6IPc0cpanv/Hfq6eNJTMgQlul2OM6YEs6IPczzfspcGrLF9g3bwx5vxY0AexwxUnefHtA3wpN430lDi3yzHG9FAW9EHsiTf2oKp8fcFot0sxxvRgFvRBqqi8hj9sK+L6C4aT1j/W7XKMMT2YBX2QemxdASLC1z5v3bwxpnMs6IPQvtJq/vTeQZZMH8GgxBi3yzHG9HAW9EHo0bX5RIWHcfu8UW6XYowJARb0QSb/SCV/2XmIpbPSSYmPdrscY0wIsKAPMo+szScuKoJbZ490uxRjTIiwoA8iuw5V8OoHn/HlCzPoHxfldjnGmBBhQR9EHl6TT2KfSG65MMPtUowxIcSCPki8d+AYaz8+yq1zRpLYJ9LtcowxIcSCPkj8bE0+SXFRLJ2Z7nYpxpgQY0EfBN7ZV86mglJunzuKuOh23QvGGGPazYLeZarKT17fzYCEaG6aPsLtcowxIciC3mVv7injnX3lfO3zo+kTFe52OcaYEGRB7yJV5adrdjMkMYbrpw5zuxxjTIiyoHfRht0lvHfgOF9fkEl0hHXzxpiuYUHvkqZufnhSLF+akuZ2OcaYEGZB75LXdh3hw4MnWL4gk8hw+2MwxnQdSxgXNDYqD6/JZ2RqHFdPHup2OcaYEGdB74K/fnCY3UcqueuiLMLDxO1yjDEhzoK+m3m8jTyyJp/PDUrg8pzBbpdjjOkFLOi72Z93HKKwtJq7FmYRZt28MaYbtCvoReRSEdktIntEZEUr20eIyDoReV9ENohImrN+koi8JSK7nG3XBfoN9CQN3kYeXZdPztBELh430O1yjDG9xDmDXkTCgSeBy4BxwA0iMq7Fbj8BXlDVCcCDwA+d9TXAv6hqNnAp8IiI9AtQ7T3OS9uKKSo/yd0XZyFi3bwxpnu0p6OfCuxR1UJVrQdWAVe12Gcc8IazvL5pu6rmq2qBs3wIOAqkBqLwnqa2wcvjbxSQO7wf87J65UdgjHFJe4J+KFDk97zYWedvJ7DIWb4GSBCRZP8dRGQqEAXsbfkLRORWEdkmIttKSkraW3uPsuqdAxyuqOWei8dYN2+M6VaB+jL2XmCuiLwHzAUOAt6mjSIyGPgNsExVG1u+WFVXqmqequalpoZet3uy3suTG/YyfWQSM0cln/sFxhgTQO2Z/Pwg4D/jVpqzrpkzLLMIQETigWtV9bjzvC/wN+BbqrolADX3OL/Zsp+SyjqeujHXunljTLdrT0e/FcgUkQwRiQKuB17x30FEUkSk6VgPAM8466OAl/F9Ubs6cGX3HFV1Hp7+v0LmZKVyQXqS2+UYY3qhcwa9qnqAO4DXgI+BP6jqLhF5UESudHabB+wWkXxgIPB9Z/0/AXOApSKyw/mZFOD3ENSee3Mf5dX13L0wy+1SjDG9lKiq2zWcJi8vT7dt2+Z2GQFRcbKB2Q+9wdSMZH51c57b5RhjQpiIbFfVVoPGroztQr/eVMiJWo9188YYV1nQd5Fj1fU88+Z+Ls8ZzLghfd0uxxjTi1nQd5FfbCykut7DnRdlul2KMaaXs6DvAiWVdTz/j/1cNXEImQMT3C7HGNPLWdB3gZ9v2Eu9t5HlF9nYvDHGfRb0AXa44iS/fftTrs0dSkZKnNvlGGOMBX2gPbl+D6rK1+fb2LwxJjhY0AdQUXkNv99axHUXDGNYUqzb5RhjDGBBH1CPv1GAiHDH562bN8YEDwv6ANlXWs0f3z3ITdNGMCgxxu1yjDGmmQV9gDy6Np+o8DBunzfK7VKMMeY0FvQBUHCkkr/sPMTNM9NJTYh2uxxjjDmNBX0APLw2n7ioCL46Z6TbpRhjzBks6Dtp16EKXv3gM758YQb946LcLscYY85gQd9JD6/Jp29MBLdcmOF2KcYY0yoL+k7YUXSctR8f5atzR5HYJ9LtcowxplUW9J3w09d3kxQXxdKZ6W6XYowxbbKgP0/v7CtnU0Ept88dRVx0e+6xbowx7rCgPw+qyk9f301qQjQ3TR/hdjnGGHNWFvTn4R97y3h7Xzl3fH40faLC3S7HGGPOyoK+g5q6+SGJMVw/dZjb5RhjzDlZ0HfQht0lvHvgOHfMzyQ6wrp5Y0zws6DvAFXlp2t2MzwplsV5aW6XY4wx7WJB3wGv7TrChwdP8I0FmUSG20dnjOkZLK3aqbFReXhNPiNT47h60hC3yzHGmHazoG+nv35wmN1HKrnzoiwirJs3xvQg7UosEblURHaLyB4RWdHK9hEisk5E3heRDSKS5rftZhEpcH5uDmTx3cXjbeSRtfmMGZjAFTmD3S7HGGM65JxBLyLhwJPAZcA44AYRGddit58AL6jqBOBB4IfOa5OAbwPTgKnAt0Wkf+DK7x5/3nGIwpJq7lqYRViYuF2OMcZ0SHs6+qnAHlUtVNV6YBVwVYt9xgFvOMvr/bZfAqxR1XJVPQasAS7tfNndp8HbyGPrChg/tC+XZA90uxxjjOmw9gT9UKDI73mxs87fTmCRs3wNkCAiye18LSJyq4hsE5FtJSUl7a29W6zeXsyB8hruWTgGEevmjTE9T6C+VbwXmCsi7wFzgYOAt70vVtWVqpqnqnmpqakBKqnz6jxeHl9XQO7wfswbEzx1GWNMR7Qn6A8C/tf6pznrmqnqIVVdpKqTgW85646357XBbNU7RRyqqOWei62bN8b0XO0J+q1ApohkiEgUcD3wiv8OIpIiIk3HegB4xll+DbhYRPo7X8Je7KwLeifrvTyxfg/TMpKYOSrZ7XKMMea8nTPoVdUD3IEvoD8G/qCqu0TkQRG50tltHrBbRPKBgcD3ndeWA/+F7y+LrcCDzrqg99stn1JSWWfdvDGmxxNVdbuG0+Tl5em2bdtcraGqzsOc/15P9pC+/OaWaa7WYowx7SEi21U1r7VtdolnK57/x37Kq+u55+IxbpdijDGdZkHfQsXJBn7xf3u5aOwAJg3r53Y5xhjTaRb0Lfx68z5O1Hq4a2GW26UYY0xAWND7OVZdzzOb9/GFnEFkD0l0uxxjjAkIC3o/v9hYSHW9h7susm7eGBM6LOgdJZV1PP+P/Vw1cQiZAxPcLscYYwLGgt7x8w17qfc2sty6eWNMiIlwu4Bg8FlFLb99+1OuzR1KRkqc2+UYc94aGhooLi6mtrbW7VJMF4mJiSEtLY3IyMh2v8aCHnhifQGqytfnZ7pdijGdUlxcTEJCAunp6XZFdwhSVcrKyiguLiYjI6Pdr+v1QzdF5TX8fmsR110wjGFJsW6XY0yn1NbWkpycbCEfokSE5OTkDv+LrdcH/eNvFCAi3PF56+ZNaLCQD23n8+fbq4N+X2k1f3z3IDdNG8GgxBi3yzHGmC7Rq4P+0bX5RIWHcfu8UW6XYowxXabXBn3BkUr+svMQN89MJzUh2u1yjAkZjz32GGPHjuXGG290uxTj6LVn3TyytoC4qAi+Omek26UY0yW++z+7+OjQiYAec9yQvnz7i9ln3eepp55i7dq1pKWlBfR3ezweIiKCJ7KCrZ6z6ZUd/a5DFfztg8N8eVY6/eOi3C7HmJBx2223UVhYyGWXXcZ3v/tdJk2axKRJk5g8eTKVlZUAPPTQQ+Tk5DBx4kRWrFgBwI4dO5g+fToTJkzgmmuu4dixYwDMmzePO++8k7y8PB599FG2b9/O3LlzmTJlCpdccgmHDx9us5Z58+Zx1113kZeXx9ixY9m6dSuLFi0iMzOTf//3f2/e7+qrr2bKlClkZ2ezcuXK5vV///vfyc3NZeLEiSxYsACA73znOyxZsoRZs2axZMkS9u/fz/z585kwYQILFizgwIEDAf9MA0JVg+pnypQp2tVueW6r5nz773q8pr7Lf5cx3emjjz5yuwQdMWKElpSU6BVXXKGbN29WVdXKykptaGjQV199VWfMmKHV1dWqqlpWVqaqqjk5ObphwwZVVf2P//gPXb58uaqqzp07V2+//XZVVa2vr9cZM2bo0aNHVVV11apVumzZsjbrmDt3rt5///2qqvrII4/o4MGD9dChQ1pbW6tDhw7V0tLS02qoqanR7OxsLS0t1aNHj2paWpoWFhaets+3v/1tzc3N1ZqaGlVVveKKK/S5555TVdVf//rXetVVV3Xy02uf1v6cgW3aRq72jH93BNCOouOs/fgI916cRWKf9l9ZZozpmFmzZnH33Xdz4403smjRItLS0li7di3Lli0jNtZ3zUpSUhIVFRUcP36cuXPnAnDzzTezePHi5uNcd911AOzevZsPP/yQhQsXAuD1ehk8ePBZa7jySt/dTnNycsjOzm7ef+TIkRQVFZGcnMxjjz3Gyy+/DEBRUREFBQWUlJQwZ86c5ouSkpKSTjtmnz59AHjrrbf405/+BMCSJUu4//77O/GJdZ1eF/Q/W5NPUlwUS2e1/6oyY0zHrVixgssvv5xXX32VWbNm8dprr53XceLifNOSqCrZ2dm89dZb7X5tdLTvRIuwsLDm5abnHo+HDRs2sHbtWt566y1iY2OZN2/eOS9GaqqnJ+lVY/Rb95ezMb+E2+aOJD661/0dZ0y32rt3Lzk5OXzzm9/kggsu4JNPPmHhwoU8++yz1NTUAFBeXk5iYiL9+/dn06ZNAPzmN79p7u79jRkzhpKSkuagb2hoYNeuXZ2qsaKigv79+xMbG8snn3zCli1bAJg+fTobN25k3759zXW2ZubMmaxatQqAF198kdmzZ3eqnq7Sa9JOVfnJa7tJTYhmyfR0t8sxJuQ98sgjrF+/nrCwMLKzs7nsssuIjo5mx44d5OXlERUVxRe+8AV+8IMf8Pzzz3PbbbdRU1PDyJEjefbZZ884XlRUFKtXr+Yb3/gGFRUVeDwe7rzzTrKzz34W0NlceumlPP3004wdO5YxY8Ywffp0AFJTU1m5ciWLFi2isbGRAQMGsGbNmjNe//jjj7Ns2TJ+/OMfk5qa2mrdwUB8Y/jBIy8vT7dt2xbw4765p5Qbf/U23/niOBu2MSHr448/ZuzYsW6XYbpYa3/OIrJdVfNa279XDN2oKj99fTdDEmO4Ydpwt8sxxphu1SuGbjbkl/DugeP84JocoiPC3S7HGBNAX/va13jzzTdPW7d8+XKWLVvmUkXBJ+SDXlX52ev5DEvqw+K8wF6pZ4xx35NPPul2CUEv5IduXv/oCB8crGD5giwiw0P+7RpjzBlCOvkaG33d/MiUOK6eNMTtcowxxhXtCnoRuVREdovIHhFZ0cr24SKyXkTeE5H3ReQLzvpIEXleRD4QkY9F5IFAv4Gz+dsHh9l9pJI7F2YRYd28MaaXOmf6iUg48CRwGTAOuEFExrXY7d+BP6jqZOB64Cln/WIgWlVzgCnAV0UkPUC1n5XH28jDa/MZMzCBK3LOfpm0McaEsva0uVOBPapaqKr1wCrgqhb7KNDXWU4EDvmtjxORCKAPUA8Edt7UNvxlxyEKS6q5a2EWYWF2azVjglF8fPxZt993331kZ2dz3333dVNFoak9Z90MBYr8nhcD01rs8x3gdRH5OhAHXOSsX43vL4XDQCxwl6qecS2xiNwK3AowfHjnz3Nv8Dby6LoCxg/tyyXZAzt9PGN6pP9dAZ99ENhjDsqBy34U2GOexcqVKykvLyc8PPCnRXu93i457vnqyvntAzVwfQPwnKqmAV8AfiMiYfj+NeAFhgAZwD0icsadPlR1parmqWpeampqp4tZvb2YA+U13LNwjN0o2ZhutGLFitNOd/zOd77D9773PRYsWEBubi45OTn85S9/adexrrzySqqqqpgyZQq///3veemllxg/fjwTJ05kzpw5gC+s7733XsaPH8+ECRN4/PHHAVi3bh2TJ08mJyeHL3/5y9TV1QGQnp7ON7/5TXJzc3nppZd4/fXXmTFjBrm5uSxevJiqqqo260lPT+eBBx5g0qRJ5OXl8e6773LJJZcwatQonn76aQCqqqrafK8vvPACEyZMYOLEiSxZsgSApUuXcttttzFt2jTuv//+Nufl77S25i9u+gFmAK/5PX8AeKDFPruAYX7PC4EB+Mb2l/itfwb4p7P9vs7OR1/b4NEZP1irVz+5WRsbGzt1LGN6Grfno3/33Xd1zpw5zc/Hjh2rBw4c0IqKClVVLSkp0VGjRjX/vxkXF3fW4/lvHz9+vBYXF6uq6rFjx1RV9amnntJrr71WGxoaVNU3b/zJkyc1LS1Nd+/eraqqS5Ys0YcfflhVfXPlP/TQQ821zJ49W6uqqlRV9Uc/+pF+97vfbbOWESNG6FNPPaWqqnfeeafm5OToiRMn9OjRozpgwABVVW1oaGj1vX744YeamZmpJSUlzXWqqt588816+eWXq8fjUdW25+VvqaPz0beno98KZIpIhohE4fuy9ZUW+xwAFgCIyFggBihx1s931scB04FPzucvpPZa9U4RhypqrZs3xgWTJ0/m6NGjHDp0iJ07d9K/f38GDRrEv/3bvzFhwgQuuugiDh48yJEjRzp87FmzZrF06VJ++ctf4vV6AVi7di1f/epXm4c8kpKS2L17NxkZGWRlZQG++e03btzYfJym+e23bNnCRx99xKxZs5g0aRLPP/88n3766Vlr8J/fftq0aSQkJJCamkp0dDTHjx9HVVt9r2+88QaLFy8mJSWluc4mixcvJjw8vNV5+f3r7oxzDgipqkdE7gBeA8KBZ1R1l4g8iO9vkFeAe4Bfishd+L6AXaqqKiJPAs+KyC5AgGdV9f2AVN6Kk/Venli/h2kZScwandxVv8YYcxaLFy9m9erVfPbZZ1x33XW8+OKLlJSUsH37diIjI0lPTz/nnO+tefrpp3n77bf529/+xpQpU9i+fft51ec/v/3ChQv53e9+1+7Xnmt++/N5r90xv327xuhV9VVVzVLVUar6fWfdfzohj6p+pKqzVHWiqk5S1ded9VWqulhVs1V1nKr+uOveCvx2y6eUVNZxz8XWzRvjluuuu45Vq1axevVqFi9eTEVFBQMGDCAyMpL169efs2tuy969e5k2bRoPPvggqampFBUVsXDhQn7xi1/g8XgA37zxY8aMYf/+/ezZswdoe3776dOn8+abbzbvV11dTX5+/nm+a5+23uv8+fN56aWXKCsra66zpfbOy38+Qmaum+o6Dz//v73MzkxhakbSuV9gjOkS2dnZVFZWMnToUAYPHsyNN97IF7/4RXJycsjLy+Nzn/vceR33vvvuo6CgAFVlwYIFTJw4kfHjx5Ofn8+ECROIjIzkK1/5CnfccQfPPvssixcvxuPxcMEFF3DbbbedcbzU1FSee+45brjhhuYva7/3ve81D/mcj7bea3Z2Nt/61reYO3cu4eHhTJ48meeee+6M17dnXv7zETLz0R89Ucu3X9nFrXNGMnl4/y6ozJjgZ/PR9w4dnY8+ZDr6AX1j+PlNU9wuwxhjgk7IBL0xpmf64IMPms8rbxIdHc3bb7/tSj3XXHNN871imzz00ENccsklrtQTCBb0xoQYVe1RJyPk5OSwY8cOt8to9vLLL7tdwlmdz3C7TeloTAiJiYmhrKzsvMLABD9VpaysjJiYmA69zjp6Y0JIWloaxcXFlJSUuF2K6SIxMTGkpXXsbnkW9MaEkMjISDIyMtwuwwQZG7oxxpgQZ0FvjDEhzoLeGGNCXNBdGSsiJcD5TYbhkwKUBqic3sA+r46xz6tj7PPqmM58XiNUtdUbegRd0HeWiGxr6zJgcyb7vDrGPq+Osc+rY7rq87KhG2OMCXEW9MYYE+JCMehXul1AD2OfV8fY59Ux9nl1TJd8XiE3Rm+MMeZ0odjRG2OM8WNBb4wxIS5kgl5EnhGRoyLyodu1BDsRGSYi60XkIxHZJSLL3a4pmIlIjIi8IyI7nc/ru27X1BOISLiIvCcif3W7lp5ARPaLyAciskNEOn6bvbMdO1TG6EVkDlAFvKCq492uJ5iJyGBgsKq+KyIJwHbgalX9yOXSgpL4JnePU9UqEYkENgPLVXWLy6UFNRG5G8gD+qrqFW7XE+xEZD+Qp6oBv8AsZDp6Vd0InHlrdXMGVT2squ86y5XAx8BQd6sKXupT5TyNdH5Co0PqIiKSBlwO/MrtWkwIBb05PyKSDkwG3LlvWw/hDEPsAI4Ca1TVPq+zewS4H2h0uY6eRIHXRWS7iNwayANb0PdiIhIP/BG4U1VPuF1PMFNVr6pOAtKAqSJiw4NtEJErgKOqut3tWnqYC1U1F7gM+JozHB0QFvS9lDPW/EfgRVX9k9v19BSqehxYD1zqcinBbBZwpTPmvAqYLyK/dbek4KeqB53Ho8DLwNRAHduCvhdyvlz8NfCxqv7M7XqCnYikikg/Z7kPsBD4xNWigpiqPqCqaaqaDlwPvKGqN7lcVlATkTjnxAhEJA64GAjYGYQhE/Qi8jvgLWCMiBSLyC1u1xTEZgFL8HVaO5yfL7hdVBAbDKwXkfeBrfjG6O2UQRNIA4HNIrITeAf4m6r+PVAHD5nTK40xxrQuZDp6Y4wxrbOgN8aYEGdBb4wxIc6C3hhjQpwFvTHGhDgLetNriIjX73TSHSKyIoDHTreZU02winC7AGO60UlnGgNjehXr6E2v58wD/t/OXODviMhoZ326iLwhIu+LyDoRGe6sHygiLzvz0+8UkZnOocJF5JfOnPWvO1fRIiLfcOb+f19EVrn0Nk0vZkFvepM+LYZurvPbVqGqOcAT+GZeBHgceF5VJwAvAo856x8D/k9VJwK5wC5nfSbwpKpmA8eBa531K4DJznFu65q3Zkzb7MpY02uISJWqxreyfj8wX1ULncnePlPVZBEpxXeDlgZn/WFVTRGREiBNVev8jpGOb2qETOf5N4FIVf2eiPwd301x/gz82W9ue2O6hXX0xvhoG8sdUee37OXUd2CXA0/i6/63ioh9N2a6lQW9MT7X+T2+5Sz/A9/siwA3Apuc5XXA7dB8Q5LEtg4qImHAMFVdD3wTSATO+FeFMV3JOgvTm/Rx7hLV5O+q2nSKZX9ndso64AZn3deBZ0XkPqAEWOasXw6sdGZI9eIL/cNt/M5w4LfOXwYCPObMaW9Mt7ExetPrdeVNmY0JBjZ0Y4wxIc46emOMCXHW0RtjTIizoDfGmBBnQW+MMSHOgt4YY0KcBb0xxoS4/w/N1oh8WpPdLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = plot_history(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to restrict the metrics shown and to customize the plots by using custom labels and titles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxhElEQVR4nO3deXyU5bn/8c+VhQRICFuQJSB7EGQPUHEDV6wWWutabUV76nLqUj0ubU9bPbaeWvVY66m2tVbtYqW2PfLTuuAG4i5BQQWJsgQIi2ySgJCQ5fr9MU/iECYrGZ5J8n2/XvPKPOtcGTTfue/7mec2d0dERKS2pLALEBGRxKSAEBGRmBQQIiISkwJCRERiUkCIiEhMCggREYlJASFtnpk9a2YXtfS+LcnMjjazT8xst5l99VC/vkgsCghJSMEfyupHlZntjVq+oCnncvfT3P2PLb1vC7sV+LW7Z7j73NobzazQzPaZWc9a698zMzezgcHyI8Hy5Kh9hpqZRy0vMLN/i1r+oZmtCd7bIjP7W7B+WdR7XmlmpVHLP2zxd0ASjgJCElLwhzLD3TOAdcBXotY9Wr2fmaWEV2WLOhxY1sA+a4DzqxfMbDTQKcZ+O4CfNeZFg9bSN4GTgvc6D3gJwN1HRf0bvApcGfVv8N+NOb+0bgoIaVXMbFrwKfcmM9sMPGxm3czsX2a21cw+C57nRB1T84nZzGab2Wtmdlew7xozO62Z+w4ys4VmtsvMXjSz+8zsL/XU/h0zW2lmO8zsSTPrG6xfBQwGngo+nafVcYo/A9+KWr4I+FOM/f4IjDGz4xt4OwEmAfPcfRWAu2929wcacZy0AwoIaY16A92JfOq+lMh/xw8HywOAvcCv6zl+ClAA9ATuAP5gZtaMff8KvAP0AG4h8kk8JjM7Afg5cA7QB1gLzAFw9yHs30oqq+M0bwFdzOwIM0sGzgNiBdIe4L+B2+qqp9Y5v2VmN5hZXnBeEUABIa1TFXCzu5e5+1533+7u/3T3Pe6+i8gfxvo+Pa9199+7eyWRT9t9gMOasq+ZDSDy6fsn7r7P3V8DnqznNS8AHnL3d4MA+AFwVPXYQRNUtyJOBj4CNtSx3++AAdEtnljc/S/AVcCpwCvAFjO7qYk1SRulgJDWaKu7l1YvmFknM/udma01sxJgIdC1nk/Dm6ufuPue4GlGE/ftC+yIWgewvp6a+xJpNVSfazewHehXzzGx/Bn4BjCb2N1L1ecvA34aPOrl7o+6+0lAV+By4KdmdmoT65I2SAEhrVHtWxD/B5ALTHH3LsBxwfq6uo1awiagu5lFDxL3r2f/jUS6wAAws85EuqbqagHE5O5riQxWfxn4vwZ2f5jIH/0zG3nucnf/O/A+cGRT6pK2SQEhbUEmkXGHnWbWHbg53i8Y/KHOB24xsw5mdhTwlXoOeQy42MzGBYPQ/w287e6FzXj5bwMnuPvnDdRYQeS9qLPLKBiIP93MMs0sKeiSGgW83Yy6pI1RQEhbcA/QEdhGZND1uUP0uhcARxHpKvoZ8Dcg5gCzu78I/Bj4J5HWxxAig8xN5u6r3D2/kbs/FrxeXUqAHxIZJN9JZCD+imBMRdo504RBIi0j+ILZCnePewtG5FBQC0KkmcxskpkNCbpmZgCzgLkhlyXSYtrKt1BFwtCbyEBxD6CISNfMe+GWJNJy1MUkIiIxqYtJRERiajNdTD179vSBAweGXYaISKuyePHibe6eHWtbmwmIgQMHkp/f2Cv/REQEwMzW1rVNXUwiIhKTAkJERGJSQIiISExtZgxCRA6d8vJyioqKKC0tbXhnSQjp6enk5OSQmpra6GMUECLSZEVFRWRmZjJw4EDqnmtJEoW7s337doqKihg0aFCjj1MXk4g0WWlpKT169FA4tBJmRo8ePZrc4lNAiEizKBxal+b8e7X7gNhVWs6d81ZQuK3eW+uLiLQ77T4gSsureOi1Qu5+4eOwSxGRRtq+fTvjxo1j3Lhx9O7dm379+tUs79u3r95j8/Pzufrqqxt8jalTp7ZIrQsWLOCMM85okXMdau1+kDo7M41LjhnIffNXcdnxgxnVNyvskkSkAT169GDJkiUA3HLLLWRkZHD99dfXbK+oqCAlJfaft7y8PPLy8hp8jTfeeKNFam3N2n0LAuDS44aQ1TGVu+YVhF2KiDTT7Nmzufzyy5kyZQo33ngj77zzDkcddRTjx49n6tSpFBRE/v+O/kR/yy23cMkllzBt2jQGDx7MvffeW3O+jIyMmv2nTZvGWWedxYgRI7jggguovgv2M888w4gRI5g4cSJXX311k1oKjz32GKNHj+bII4/kppsis8JWVlYye/ZsjjzySEaPHs0vf/lLAO69915GjhzJmDFjOO+8Zk1E2CztvgUBkNUxlcuPH8IvnlvBosIdTBrYPeySRFqN/3pqGcs3lrToOUf27cLNXxnV5OOKiop44403SE5OpqSkhFdffZWUlBRefPFFfvjDH/LPf/7zgGNWrFjB/Pnz2bVrF7m5uVxxxRUHfFfgvffeY9myZfTt25ejjz6a119/nby8PC677DIWLlzIoEGDOP/88xtd58aNG7nppptYvHgx3bp145RTTmHu3Ln079+fDRs28OGHHwKwc+dOAG6//XbWrFlDWlpazbpDQS2IwOypA+mVmcYdz61Ac2SItE5nn302ycnJABQXF3P22Wdz5JFHcu2117Js2bKYx5x++umkpaXRs2dPevXqxaeffnrAPpMnTyYnJ4ekpCTGjRtHYWEhK1asYPDgwTXfK2hKQCxatIhp06aRnZ1NSkoKF1xwAQsXLmTw4MGsXr2aq666iueee44uXboAMGbMGC644AL+8pe/1Nl1Fg9qQQQ6dkjmqhOH8eO5H7KgYCvTR/QKuySRVqE5n/TjpXPnzjXPf/zjHzN9+nSeeOIJCgsLmTZtWsxj0tLSap4nJydTUVHRrH1aQrdu3Vi6dCnz5s3jt7/9LY8//jgPPfQQTz/9NAsXLuSpp57itttu44MPPjgkQaEWRJRz8/ozoHsn7phXQFWVWhEirVlxcTH9+vUD4JFHHmnx8+fm5rJ69WoKCwsB+Nvf/tboYydPnswrr7zCtm3bqKys5LHHHuP4449n27ZtVFVV8fWvf52f/exnvPvuu1RVVbF+/XqmT5/OL37xC4qLi9m9e3eL/z6xKCCidEhJ4rqTh/PRphL+9cGmsMsRkYNw44038oMf/IDx48fH5RN/x44duf/++5kxYwYTJ04kMzOTrKzYV0G+9NJL5OTk1DwKCwu5/fbbmT59OmPHjmXixInMmjWLDRs2MG3aNMaNG8eFF17Iz3/+cyorK7nwwgsZPXo048eP5+qrr6Zr164t/vvE0mbmpM7Ly/OWmDCoqsr58r2vUlpeyQvXHU9qsjJUpLaPPvqII444IuwyQrd7924yMjJwd7773e8ybNgwrr322rDLqlOsfzczW+zuMa/71V+/WpKSjBtOzaVw+x7+nl8UdjkiksB+//vfM27cOEaNGkVxcTGXXXZZ2CW1KA1Sx3DCiF5MPLwbv3rpY86c0I/01OSwSxKRBHTttdcmdIvhYKkFEYOZceOpuXxaUsaf3iwMuxwRkVAoIOowZXAPjh+ezf0LVlFSWh52OSIih5wCoh43nJrLzj3lPLhwddiliIgccnENCDObYWYFZrbSzL4fY/vlZvaBmS0xs9fMbGSwfqCZ7Q3WLzGz38azzroc2S+L08f04cHX1rBtd1kYJYiIhCZuAWFmycB9wGnASOD86gCI8ld3H+3u44A7gLujtq1y93HB4/J41dmQ/zh5OGUVVdw3f2VYJYhILdOnT2fevHn7rbvnnnu44oor6jxm2rRpVF8K/+UvfznmPY1uueUW7rrrrnpfe+7cuSxfvrxm+Sc/+QkvvvhiE6qPLRFvCx7PFsRkYKW7r3b3fcAcYFb0Du4efYevzkDCfSljcHYGZ0/M4dG31lH02Z6wyxERIvc9mjNnzn7r5syZ0+j7IT3zzDPN/rJZ7YC49dZbOemkk5p1rkQXz4DoB6yPWi4K1u3HzL5rZquItCCiZ/EYZGbvmdkrZnZsrBcws0vNLN/M8rdu3dqSte/nmpOGgcGvXvwkbq8hIo131lln8fTTT9dMDlRYWMjGjRs59thjueKKK8jLy2PUqFHcfPPNMY8fOHAg27ZtA+C2225j+PDhHHPMMTW3BIfIdxwmTZrE2LFj+frXv86ePXt44403ePLJJ7nhhhsYN24cq1atYvbs2fzjH/8AIt+YHj9+PKNHj+aSSy6hrKys5vVuvvlmJkyYwOjRo1mxYkWjf9cwbwse+vcg3P0+4D4z+wbwI+AiYBMwwN23m9lEYK6ZjarV4sDdHwAegMg3qeNVY5+sjnzrS4fz0OtruOz4wQztlRmvlxJpfZ79Pmz+oGXP2Xs0nHZ7nZu7d+/O5MmTefbZZ5k1axZz5szhnHPOwcy47bbb6N69O5WVlZx44om8//77jBkzJuZ5Fi9ezJw5c1iyZAkVFRVMmDCBiRMnAnDmmWfyne98B4Af/ehH/OEPf+Cqq65i5syZnHHGGZx11ln7nau0tJTZs2fz0ksvMXz4cL71rW/xm9/8hu9973sA9OzZk3fffZf777+fu+66iwcffLDBtyHs24LHswWxAegftZwTrKvLHOCrAO5e5u7bg+eLgVXA8PiU2Tj/Pn0onTqk8D/Pa2pSkUQQ3c0U3b30+OOPM2HCBMaPH8+yZcv26w6q7dVXX+VrX/sanTp1okuXLsycObNm24cffsixxx7L6NGjefTRR+u8XXi1goICBg0axPDhkT9VF110EQsXLqzZfuaZZwIwceLEmhv8NSTs24LHswWxCBhmZoOIBMN5wDeidzCzYe5e3W9zOvBJsD4b2OHulWY2GBgGhHqtaffOHfi3Ywdxz4ufsHT9Tsb27xpmOSKJo55P+vE0a9Ysrr32Wt5991327NnDxIkTWbNmDXfddReLFi2iW7duzJ49m9LS0madf/bs2cydO5exY8fyyCOPsGDBgoOqt/qW4S1xu/BDdVvwuLUg3L0CuBKYB3wEPO7uy8zsVjOrjukrzWyZmS0BriPSvQRwHPB+sP4fwOXuviNetTbWvx07mO6dO3CnpiYVCV1GRgbTp0/nkksuqWk9lJSU0LlzZ7Kysvj000959tln6z3Hcccdx9y5c9m7dy+7du3iqaeeqtm2a9cu+vTpQ3l5OY8++mjN+szMTHbt2nXAuXJzcyksLGTlysgVj3/+8585/vjjD+p3DPu24HEdg3D3Z4Bnaq37SdTza+o47p/AgXMDhiwjLYV/nzaEnz39EW+s3MbUoT3DLkmkXTv//PP52te+VtPVNHbsWMaPH8+IESPo378/Rx99dL3HT5gwgXPPPZexY8fSq1cvJk2aVLPtpz/9KVOmTCE7O5spU6bUhMJ5553Hd77zHe69996awWmA9PR0Hn74Yc4++2wqKiqYNGkSl1/etCv0q28LXu3vf/97zW3B3Z3TTz+dWbNmsXTpUi6++GKqqqoA9rsteHFxMe7eIrcF1+2+m6i0vJIT7lpAdpd05v77VMws7q8pkmh0u+/WSbf7jrP01GSuOWkYS9fv5PnlB85dKyLSViggmuHrE3IYnN2Zu+YVUKmpSUWkjVJANENKchL/cXIun2zZzdz36rtyV6Ttaivd0+1Fc/69FBDNdNqRvTmyXxd++eLH7KuoCrsckUMqPT2d7du3KyRaCXdn+/btpKenN+m40L9J3VolJRk3njqCbz30Do+9s46Lpg4MuySRQyYnJ4eioiLieYsbaVnp6en7XSHVGAqIg3DssJ58aXB3/vfllZydl0OnDno7pX1ITU1l0KBBYZchcaYupoNgZtw4YwTbdpfx8OuFYZcjItKiFBAHacKAbpx0xGH89pVV7NyzL+xyRERajAKiBdxwai67yyr47SuamlRE2g4FRAvI7Z3JV8f145E31rClpHk3BhMRSTQKiBZy7UnDqah07n1ZkwqJSNuggGghA3p04vzJA5jzznrWbv887HJERA6aAqIFXXXCUFKSjV++oEmFRKT1U0C0oF5d0rn46EH8v6UbWbG5pOEDREQSmAKihV1+3BAy01K4S5MKiUgrp4BoYVmdUrns+CG8+NEWFq8NfRI8EZFmU0DEwcVHD6RnRhq/eK5ANzMTkVZLAREHnTqkcPWJQ3lnzQ5e+Vg3MxOR1kkBESfnTRpA/+4duXNeAVWaVEhEWqG4BoSZzTCzAjNbaWbfj7H9cjP7wMyWmNlrZjYyatsPguMKzOzUeNYZDx1Skrj2pOEs21jCMx9uCrscEZEmi1tAmFkycB9wGjASOD86AAJ/dffR7j4OuAO4Ozh2JHAeMAqYAdwfnK9VmTWuH8MPy+Du5z+molKTColI6xLPFsRkYKW7r3b3fcAcYFb0Du4e/WWBzkB1X8wsYI67l7n7GmBlcL5WJTnJuP6UXFZv+5x/LC4KuxwRkSaJZ0D0A9ZHLRcF6/ZjZt81s1VEWhBXN+XY1uDkkYcxfkBXfvXSJ5SWV4ZdjohIo4U+SO3u97n7EOAm4EdNOdbMLjWzfDPLT9SpD80iU5NuKi7lL2+tDbscEZFGi2dAbAD6Ry3nBOvqMgf4alOOdfcH3D3P3fOys7MPrto4OmpID44d1pP75q9kV2l52OWIiDRKPANiETDMzAaZWQcig85PRu9gZsOiFk8Hqu+V/SRwnpmlmdkgYBjwThxrjbsbTx3BZ3vKefDVNWGXIiLSKHELCHevAK4E5gEfAY+7+zIzu9XMZga7XWlmy8xsCXAdcFFw7DLgcWA58BzwXXdv1R34o3Oy+PLo3jz46mq27y4LuxwRkQZZW7kVRF5enufn54ddRr1WbtnNKb98hYuPHsSPz6h9xa+IyKFnZovdPS/WttAHqduTob0yOGtiDn9+ay0bd+4NuxwRkXopIA6xa04aDg6/elFTk4pIYlNAHGL9unbkwi8dzt8Xr2fV1t1hlyMiUicFRAi+O30IHVOTuft5TU0qIolLARGCHhlpfPvYwTz9wSY+3FAcdjkiIjEpIELynWMH0a1TKndoalIRSVAKiJBkpqfy79OGsvDjrby5anvY5YiIHEABEaJvHnU4vbukc8e8FZqaVEQSjgIiROmpyVxz0jDeW7eTFz/aEnY5IiL7UUCE7OyJOQzq2Zm75hVQqalJRSSBKCBClpKcxHUnD6fg0108ubS+m92KiBxaCogEcProPozs04W7X/iYfRWamlREEoMCIgEkJRk3zMhl/Y69/G3RurDLEREBFBAJY9rwbCYP6s69L69kz76KsMsREVFAJAoz46YZuWzdVcYjbxSGXY6IiAIikUw8vDsnjujFbxesoniPpiYVkXApIBLM9afmsqusgt8tXBV2KSLSzikgEswRfbowc2xfHn69kC27SsMuR0TaMQVEArru5OGUV1bx65dXhl2KiLRjCogEdHiPzpw7qT+PvbOO9Tv2hF2OiLRTCogEdfWJw0hOMn75giYVEpFwxDUgzGyGmRWY2Uoz+36M7deZ2XIze9/MXjKzw6O2VZrZkuDxZDzrTESHdUnnoqkDeWLJBgo27wq7HBFph+IWEGaWDNwHnAaMBM43s5G1dnsPyHP3McA/gDuitu1193HBY2a86kxkVxw/hIy0FO56XpMKicihF88WxGRgpbuvdvd9wBxgVvQO7j7f3as72d8CcuJYT6vTtVMHLjtuMC8s/5R3130Wdjki0s7EMyD6AeujlouCdXX5NvBs1HK6meWb2Vtm9tVYB5jZpcE++Vu3bj3oghPRxUcPomdGB+54TpMKicihlRCD1GZ2IZAH3Bm1+nB3zwO+AdxjZkNqH+fuD7h7nrvnZWdnH6JqD63OaSlcOX0ob63ewaufbAu7HBFpR+IZEBuA/lHLOcG6/ZjZScB/AjPdvax6vbtvCH6uBhYA4+NYa0I7f8oA+nXtyJ3zCtSKEJFDJp4BsQgYZmaDzKwDcB6w39VIZjYe+B2RcNgStb6bmaUFz3sCRwPL41hrQktLSebak4fzwYZinv1wc9jliEg7EbeAcPcK4EpgHvAR8Li7LzOzW82s+qqkO4EM4O+1Lmc9Asg3s6XAfOB2d2+3AQHwtfH9GNYrg7ueL6CiUpMKiUj8WVvpssjLy/P8/Pywy4ir5z7czOV/WcwdXx/DOZP6N3yAiEgDzGxxMN57gIQYpJbGOXXUYYzt35V7XvyY0vLKsMsRkTZOAdGKmBk3nprLxuJSHn1bU5OKSHwpIFqZo4f25JihPblv/kp2l2lqUhGJHwVEK3TDqbns+Hwff3h1TdiliEgbpoBohcb278qMUb35/aur2fH5vrDLEZE2SgHRSl1/6nD27KvgNws0qZCIxIcCopUa2iuTMyfk8Mc317KpeG/Y5YhIG6SAaMW+d9IwcLj3pU/CLkVE2iAFRCuW060T35gygMfzi1i9dXfY5YhIG9OogDCzzmaWFDwfbmYzzSw1vqVJY1x5wlDSUpK4W1OTikgLa2wLYiGR+Rn6Ac8D3wQeiVdR0ng9M9L49jGD+Nf7m/hwQ3HY5YhIG9LYgLBg5rczgfvd/WxgVPzKkqb4znGD6dopVVOTikiLanRAmNlRwAXA08G65PiUJE3VJT2VK44fwoKCrbyzZkfY5YhIG9HYgPge8APgieCW3YOJ3IZbEsRFUwdyWJc0TU0qIi2mUQHh7q+4+0x3/0UwWL3N3a+Oc23SBOmpyVx94jDy137Gyyu2NHyAiEgDGnsV01/NrIuZdQY+BJab2Q3xLU2a6py8/gzs0Yk75xVQVaVWhIgcnMZ2MY109xLgq8CzwCAiVzJJAklNTuLak4ezYvMunnp/Y9jliEgr19iASA2+9/BV4El3Lwf0ETUBfWVMX47o04W7X/iYck1NKiIHobEB8TugEOgMLDSzw4GSeBUlzZeUZNxw6nDWbt/D3xatD7scEWnFGjtIfa+793P3L3vEWmB6nGuTZpqe24u8w7tx70ufsHefpiYVkeZp7CB1lpndbWb5weN/iLQmGjpuhpkVmNlKM/t+jO3XmdlyM3vfzF4KWibV2y4ys0+Cx0VN+q3aOTPjxhkj2LKrjD++WRh2OSLSSjW2i+khYBdwTvAoAR6u7wAzSwbuA04DRgLnm9nIWru9B+S5+xjgH8AdwbHdgZuBKcBk4GYz69bIWgWYPKg703Oz+c2CVRTvLQ+7HBFphRobEEPc/WZ3Xx08/gsY3MAxk4GVwf77gDnArOgd3H1+cAsPgLeAnOD5qcAL7r7D3T8DXgBmNLJWCVx/ai7Fe8v5/cLVYZciIq1QYwNir5kdU71gZkcDDc1S0w+IHiUtCtbV5dtELqFt9LFmdml1t9fWrVsbKKf9GdU3i6+M7ctDr69h666ysMsRkVamsQFxOXCfmRWaWSHwa+CylirCzC4E8oA7m3Kcuz/g7nnunpednd1S5bQp1508nLKKKu6br6lJRaRpGnsV01J3HwuMAca4+3jghAYO2wD0j1rOCdbtx8xOAv4TmOnuZU05Vho2qGdnzsnrz6Nvr2X9jj0NHyAiEmjSjHLuXhJ8oxrgugZ2XwQMM7NBZtYBOA94MnoHMxtP5DsWM909+gZC84BTzKxbMDh9SrBOmuGaE4eRZMY9L2pqUhFpvIOZctTq2+juFcCVRP6wfwQ8HtwJ9lYzmxnsdieQAfzdzJaY2ZPBsTuAnxIJmUXArcE6aYbeWelcNHUgT7xXxCef7gq7HBFpJay5t4Y2s3XuPqCF62m2vLw8z8/PD7uMhPXZ5/s47o75TB3ag999My/sckQkQZjZYneP+Ueh3haEme0ys5IYj11A37hUK3HRrXMHvnPcYOYt+5Ql63eGXY6ItAL1BoS7Z7p7lxiPTHdPOVRFSsu45JhB9OjcgTvnrQi7FBFpBQ5mDEJamYy0FL47fSivr9zO6yu3hV2OiCQ4BUQ7c8GXBtCva0dNTSoiDVJAtDNpKclcc9IwlhYVM2/Z5rDLEZEEpoBoh84c348h2Z256/mPqdTUpCJSBwVEO5SSnMT1p+Sycstu/u/dorDLEZEEpYBop2Yc2ZsxOVnc8+InlFVoUiEROZACop0yM244NZcNO/fy17fXhV2OiCQgBUQ7dszQnhw1uAe/fnkln5dVhF2OiCQYBUQ7ZmbcMCOX7Z/v46HX1oRdjogkGAVEOzdhQDdOGXkYDyxczWef7wu7HBFJIAoI4fpTc9m9r4LfvrIq7FJEJIEoIIThh2XytfH9eOSNQjYXl4ZdjogkCAWEAHDtScOpcufelzWpkIhEKCAEgP7dO/GNyQN4fNF6Crd9HnY5IpIAFBBS48oThpGanMTdL3wcdikikgAUEFIjOzONS44ZyJNLN7J8Y0nDB4hIm6aAkP1cetwQsjqmctfzBWGXIiIhU0DIfrI6pnL58UN4ecUW8gt3hF2OiIRIASEHmD11IL0y07jjuQJNKiTSjsU1IMxshpkVmNlKM/t+jO3Hmdm7ZlZhZmfV2lZpZkuCx5PxrFP217FDMledOIx3Cnew4OOtYZcjIiGJW0CYWTJwH3AaMBI438xG1tptHTAb+GuMU+x193HBY2a86pTYzs3rz4DunbjzuQKqNKmQSLsUzxbEZGClu692933AHGBW9A7uXuju7wNVcaxDmqFDShLXnTyc5ZtK+NcHm8IuR0RCEM+A6Aesj1ouCtY1VrqZ5ZvZW2b21Vg7mNmlwT75W7eqK6SlzRzblxG9M7n7+QLKK5XhIu1NIg9SH+7uecA3gHvMbEjtHdz9AXfPc/e87OzsQ19hG5eUZFx/Si6F2/fw93xNTSrS3sQzIDYA/aOWc4J1jeLuG4Kfq4EFwPiWLE4a58QjejFhQFd+9dLHlJZralKR9iSeAbEIGGZmg8ysA3Ae0Kirkcysm5mlBc97AkcDy+NWqdTJzLhxxgg+LSnjT28Whl2OiBxCcQsId68ArgTmAR8Bj7v7MjO71cxmApjZJDMrAs4Gfmdmy4LDjwDyzWwpMB+43d0VECH50uAeHDc8m/sXrKKktDzsckTkELG28kWovLw8z8/PD7uMNuvDDcWc8b+vcfUJQ7nulNywyxGRFmJmi4Px3gMk8iC1JJAj+2Vx+pg+PPjaGrbtLgu7HBE5BBQQ0mj/cfJwyiqquG/+yrBLEZFDQAEhjTY4O4OzJ+bw6FvrWLlld9jliEicKSCkSa45aRjJScZJd7/CV/73Ne5+voD31n1GpW7HIdLmaJBammzNts955oNNzF+xhXfXfUaVQ/fOHZg2PJtpI3px3LCedO3UIewyRaQR6hukVkDIQfns830s/GQr81ds4ZWPt/LZnnKSDCYe3o1pub04YUQvRvTOxMzCLlVEYlBAyCFRWeUsLdrJ/BVbmF+whQ83RKYt7ZOVzrTcXkzPzebooT3pnJYScqUiUk0BIaH4tKSUVwq28vKKLby2chu7yyrokJzElMHdmZ7bi+kjejGoZ+ewyxRp1xQQErp9FVXkr90RtC621lwFNahnZ6blZnPCiF5MHtSdtJTkkCsVaV8UEJJw1m3fw4KPt/Dyii28uWo7ZRVVdOqQzNFDewati2z6ZHUMu0yRNk8BIQlt775K3ly9jfkrIt1RG3buBWBE70ymj4gMdI/v35WUZF2VLdLSFBDSarg7K7fs5uVgoDu/8DMqqpysjqkcNzyb6bnZHD88mx4ZaWGXKtImKCCk1SopLee1T7bx8ootLCjYyrbdZZjB2JyunDCiF9NzezGqbxeSknQZrUhzKCCkTaiqcj7cWMz8FVuZX7CFpUU7cYfszDSmDY8MdB89rCdd0lPDLlWk1VBASJu0bXcZCz+OjFss/HgrJaUVpCQZkwZ2Z/qISGAMyc7Ql/RE6qGAkDavorKKd9ftZH7BFuav2MKKzbsAyOnWkenBN7qPGtKD9FRdRisSTQEh7c7GnXuDsNjK6yu3sbe8krSUJKYO6cH0YOyif/dOYZcpEjoFhLRrpeWVvLNmR03ronD7HgCG9srghBG9mJabzaSB3UnVZbTSDikgRKKs2fZ5cFXUFt5evYN9lVVkpqVwzLCeTA8Co1dmethlihwSCgiROnxeVsHrK7fVdEdtLikFYHS/LKbnZjN9RC/G5HQlWZfRShsVWkCY2QzgV0Ay8KC7315r+3HAPcAY4Dx3/0fUtouAHwWLP3P3P9b3WgoIOVjuzkebdtV0RWmuC2kPQgkIM0sGPgZOBoqARcD57r48ap+BQBfgeuDJ6oAws+5APpAHOLAYmOjun9X1egoIaWk79+xj4SfbmB90R2muC2mL6guIeN6YfzKw0t1XB0XMAWYBNQHh7oXBtqpax54KvODuO4LtLwAzgMfiWK/Ifrp26sDMsX2ZObZvzVwXC1Zs4eWCLdw5r4A75xVorgtp0+L5X3M/YH3UchEw5SCO7Vd7JzO7FLgUYMCAAc2rsrIc/vw16DseBhwFA74Enbo371zSZiUnGRMGdGPCgG5cd0ouW0pKWVAQ+Ub3U0s38tg76zTXhbQ5rfrjjrs/ADwAkS6mZp3k861QuQ/e+g28cW9kXfaISFBUB0bXw0HdCBKlV5d0zpnUn3Mm9T9grotb/7WcW/+1vGaui+m5vRiTk0VWx1R1R0mrEs+A2AD0j1rOCdY19thptY5d0CJV1dalL3z7eSjfCxvfg3Vvwto34cMnYPEjkX0y++4fGIeNgiR9I1ciOqQkMXVIT6YO6cl/ng7rd+xhfkFkrou/vr2Oh18vBCA9NYk+WR3pk5VO76x0+mZ1jPzsmk7vLh3p2zVdISIJJZ6D1ClEBqlPJPIHfxHwDXdfFmPfR4B/1RqkXgxMCHZ5l8gg9Y66Xq/FB6mrKmHLR5HAWPdW5GdJkG9pXaD/5C9Co99ESNXkNnKgvfsqeWvNdlZt2c3m4lI2FZeyqXgvm4tL+XRXGZVV+///11CI9MlKp2snhYi0nDAvc/0ykctYk4GH3P02M7sVyHf3J81sEvAE0A0oBTa7+6jg2EuAHwanus3dH67vtQ7JVUw7130RFuvegi3BeHtSKvQdFwTGVOg/BTr3iG8t0upVVFaxbfc+NgaBsXFn5OemklI27Wx6iPTJSq9ZrxCRxtIX5eJlzw5Y/84XgbHx3ch4BkDP3P27pboN1DiGNFntENlUHAmPpoZIdHgoRCSaAuJQKS/9Yhxj3Zuw7m0oK45sy+xTaxzjSI1jSIuorHK27ipjU/HeoAuraSHSu0s6fboqRNorBURYqqpga9Q4xto3oaQosq1DZjCOEQRGv4nQQXcXlfiIR4hUd28pRFo3BUQi2bk+xjiGQ1IK9BkXCYvDp0L/L2kcQw6p+kJkc/C8sSHSO6sjfRUirYICIpHt/Wz/cYwNi6PGMYZHdUsdpXEMCV2sENlcvJeNxU0IkaxIkChEEoMCojUpL4VNS774Psb6t6A0GMfI6H3gOEZyq/6uo7RB0SGyubg0CI+GQyQtJammC6tHRgeyOqbGfHTpmErXTpHnGWkpCpWDpIBozaqqYOuK/b+PURzchaRDJvSfFDWOkadxDGkVKqucbbvLai7tjQ6RTTv3suPzfRTvLaektOKAIImWnGR0SU/ZLzzqCpb9tndKJaNDCkm6jbsCos3ZuR7Wv/1FaHy6jC/GMcZ+0SU14EvQuWfY1Yo0m7uzu6yC4r3lNY+SqOdfPCpibq8vXJKMAwKloYCp3iczre2EiwKirdu7M8Y4RllkW49hX3RLHX4UdBukcQxpF9ydz/dVRsJiT0MBc+C2igbCJTO9nhZKPY/M9MQKFwVEe1NRBhuXwLo3gm6pt6B0Z2RbxmG1xjFGaxxDpBZ3Z091uNQTIvs99jQuXMwgMy2FrE5ND5jM9NQWn90wrPkgJCwpaTBgSuQBkXGMbQX7fx9j+f+LbOuQATlR4xg5edBBt6mW9s3M6JyWQue0FPp2bdp91poTLpuKS2u2lVfWHy4ZaSkHBMeI3l245qRhB/trH0AB0R4kJUGvIyKPvEsi64qLvmhdrHsLFvycA8cxvhT5PkZGdqjli7QmBxsue8srY7ZK6gqYT0tK6x1rOajfRV1MAkTGMYoWfdHKKMqPGscYuv/Ad/fBGscQaSPUxSQN69gVhp0ceUBkHGPTUlgbjGOs+Be89+fIts69IDsXug6ArBzI6g9d+0d+ZuVEurhEpNVTQEhsKWmRe0X1nxxZrqqCbR9HWhjr34Ydq2HVfNi1CajVCs047Iuw6NofsgYEP4Mw6dj1UP82ItIMCghpnKQk6DUi8si7+Iv1FfsiEykVF0W+wLdzPRSviyxv/gAKnv2iq6paWpfYLY+uAyLPMw6LvJ6IhEoBIQcnpQN0HxR5xFJVFZn3u7goEhw71+8fJtG3EqmWlApZ/YIAUTeWSFgUEBJfSUmQeVjkkTMx9j6lJV+ERk0rJPjZlG6smufqxhJpCQoICV96F0gfCYeNjL29phsraH00qxsrJ6pFom4skcZQQEjia1Y3VlSYNNSNFd19pW4skRoKCGn9mtqNtXPd/uMgqxeoG0skBgWEtA/N7cbauV7dWNJuxTUgzGwG8CsgGXjQ3W+vtT0N+BMwEdgOnOvuhWY2EPgIKAh2fcvdL49nrdLOxbsbq2NXSEmPeqQd+Dy19rbaP2McqwCSOIpbQJhZMnAfcDJQBCwysyfdfXnUbt8GPnP3oWZ2HvAL4Nxg2yp3Hxev+kSa5GC7sbathIrSqEcZlO/lgG6tJteV2ohw6Vh36DR4XD2BlZyqW660cfFsQUwGVrr7agAzmwPMAqIDYhZwS/D8H8CvTfMHSmvVUDdWbe5QVbF/YFSUfbEcHSa1w6X2z5jHlkVCq2IrVMTYXj33eXNZUguFUoxjU9MjdxXukAGpnYLnnSG5g0LpEIpnQPQD1kctFwFT6trH3SvMrBjoEWwbZGbvASXAj9z91dovYGaXApcCDBgwoGWrF4k3s8in8ORUSMs89K9fVRUZV6kojcyFXlf4xAqX6Of1HVu6s+5Aa07rKSkFUjt/ERgdOh0YItWP1Lr2yQjWRe2TkqbgiSFRB6k3AQPcfbuZTQTmmtkody+J3sndHwAegMjdXEOoU6T1SkqCpI6Q2hGadlfqg1fdeqqr1VS+B/btgX27g+ef7/8or34ebNuzDXau3X+fqvLG12NJ9QRNp/1DJdZ+NfvUCqTUjq06eOIZEBuA/lHLOcG6WPsUmVkKkAVs98g9yMsA3H2xma0ChgO6n7dIWxDdeoqXin1BkFQHTIywKQ9CqCZsau2zZwfsWx+1354Dr2arl9XTqulcqzXUhEBK6XhILlCIZ0AsAoaZ2SAiQXAe8I1a+zwJXAS8CZwFvOzubmbZwA53rzSzwcAwYHUcaxWRtialQ+TRsVvLnreyolYLZnetsNlTq5UTI5BKS6Bk0/77VZQ2rY7UqKDpOwHOfrhlf0/iGBDBmMKVwDwil7k+5O7LzOxWIN/dnwT+APzZzFYCO4iECMBxwK1mVg5UAZe7+4541Soi0mjJKZCcBelZLXveqsqoEPm88WGzb0/kezhxoBnlRETasfpmlNO3bEREJCYFhIiIxKSAEBGRmBQQIiISkwJCRERiUkCIiEhMCggREYlJASEiIjG1mS/KmdlWYO1BnKInsK2FymkP9H41jd6vptH71TQH834d7u7ZsTa0mYA4WGaWX9e3CeVAer+aRu9X0+j9app4vV/qYhIRkZgUECIiEpMC4gsPhF1AK6P3q2n0fjWN3q+micv7pTEIERGJSS0IERGJSQEhIiIxtfuAMLOHzGyLmX0Ydi2Jzsz6m9l8M1tuZsvM7Jqwa0pkZpZuZu+Y2dLg/fqvsGtqDcws2czeM7N/hV1La2BmhWb2gZktMbMWnTWt3Y9BmNlxwG7gT+5+ZNj1JDIz6wP0cfd3zSwTWAx81d2Xh1xaQjIzAzq7+24zSwVeA65x97dCLi2hmdl1QB7Qxd3PCLueRGdmhUCeu7f4FwvbfQvC3RcSmQ9bGuDum9z93eD5LuAjoF+4VSUuj9gdLKYGj/b9iawBZpYDnA48GHYtooCQZjKzgcB44O2QS0loQXfJEmAL8IK76/2q3z3AjUBVyHW0Jg48b2aLzezSljyxAkKazMwygH8C33P3krDrSWTuXunu44AcYLKZqRuzDmZ2BrDF3ReHXUsrc4y7TwBOA74bdJu3CAWENEnQl/5P4FF3/7+w62kt3H0nMB+YEXIpiexoYGbQpz4HOMHM/hJuSYnP3TcEP7cATwCTW+rcCghptGDQ9Q/AR+5+d9j1JDozyzazrsHzjsDJwIpQi0pg7v4Dd89x94HAecDL7n5hyGUlNDPrHFwwgpl1Bk4BWuyKzHYfEGb2GPAmkGtmRWb27bBrSmBHA98k8sluSfD4cthFJbA+wHwzex9YRGQMQpduSks6DHjNzJYC7wBPu/tzLXXydn+Zq4iIxNbuWxAiIhKbAkJERGJSQIiISEwKCBERiUkBISIiMSkgRBpgZpVRl/UuMbPvt+C5B+pOwpKoUsIuQKQV2BvcLkOkXVELQqSZgvvw3xHci/8dMxsarB9oZi+b2ftm9pKZDQjWH2ZmTwTzQyw1s6nBqZLN7PfBnBHPB9+6xsyuDubeeN/M5oT0a0o7poAQaVjHWl1M50ZtK3b30cCvidyJFOB/gT+6+xjgUeDeYP29wCvuPhaYACwL1g8D7nP3UcBO4OvB+u8D44PzXB6fX02kbvomtUgDzGy3u2fEWF8InODuq4ObGG529x5mto3IxErlwfpN7t7TzLYCOe5eFnWOgURuwTEsWL4JSHX3n5nZc0Qms5oLzI2aW0LkkFALQuTgeB3Pm6Is6nklX4wNng7cR6S1scjMNGYoh5QCQuTgnBv1883g+RtE7kYKcAHwavD8JeAKqJlIKKuuk5pZEtDf3ecDNwFZwAGtGJF40icSkYZ1DGaFq/acu1df6totuFtrGXB+sO4q4GEzuwHYClwcrL8GeCC4Y3AlkbDYVMdrJgN/CULEgHuDOSVEDhmNQYg0UzwnixdJBOpiEhGRmNSCEBGRmNSCEBGRmBQQIiISkwJCRERiUkCIiEhMCggREYnp/wO6giD37M8nUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAzCElEQVR4nO3deXxU5dn/8c+VfSVAEnYwLGEpYEDCIpvgVhcqFVeqT6W2oNa2gN2sbZ/HLj5dtFaslqe07lq1lupPrVXZRUEgICCbCUuAsCYBspA9c/3+OCchCUmYQCaTZK7368Vr5pw558yVEb/cuc899y2qijHGmMAR5O8CjDHGtCwLfmOMCTAW/MYYE2As+I0xJsBY8BtjTICx4DfGmABjwW/aLBH5j4jc1dzHNicRmSAiGSJSKCJfben3N6Y+FvymRbkBWPXHIyLFNbbvaMq1VPVaVX2huY9tZr8EnlLVGFV9q+6LIpIpImUiklBn/2cioiKS5G4/726PqXHMABHRGtsrReRbNbYfEpF97mebJSKvu/u31/jMK0WkpMb2Q83+CZhWx4LftCg3AGNUNQY4AHylxr5Xqo4TkRD/VdmsLgK2n+OYfcDMqg0RGQ5E1XPcCeDX3ryp+9vNfwFXup91KrAMQFWH1vhvsBr4To3/Bv/rzfVN22bBb1oFEZnitkp/LCJHgedEpJOIvCsi2SJy0n3eq8Y51S1cEZklIh+LyGPusftE5NrzPLaviHwkIgUislREnhaRlxupfbaI7BaREyLytoj0cPfvAfoB77it6fAGLvES8PUa23cBL9Zz3AvAxSJy2Tk+ToDRwAequgdAVY+q6iIvzjMBwILftCbdgM44reQ5OH8/n3O3+wDFwFONnD8W+AJIAH4PPCMich7H/h1YD8QDD+O0nOslIpcDvwFuBboD+4HXAFS1P7V/qylt4DKfAh1EZIiIBAO3A/X9Q1ME/C/wSEP11Lnm10XkhyKS6l7XGMCC37QuHuB/VLVUVYtVNVdVF6tqkaoW4AReY63d/ar6V1WtxGkddwe6NuVYEemD01r+b1UtU9WPgbcbec87gGdVdZMb7D8BLq3qm2+Cqlb/VcBO4FADx/0F6FPzN5T6qOrLwHeBLwOrgOMi8uMm1mTaKQt+05pkq2pJ1YaIRInIX0Rkv4jkAx8BHRtpvR6teqKqRe7TmCYe2wM4UWMfwMFGau6B08qvulYhkAv0bOSc+rwEfA2YRf3dPFXXLwV+5f5plKq+oqpXAh2Be4FficiXm1iXaYcs+E1rUneq2O8Dg4CxqtoBmOzub6j7pjkcATqLSM2bq70bOf4wTlcUACISjdNF1FCLvV6quh/nJu91wL/OcfhzOGE+w8trl6vqG8BWYFhT6jLtkwW/ac1icfr1T4lIZ+B/fP2GbgCnAQ+LSJiIXAp8pZFTXgW+ISIj3Ju3/wusU9XM83j7bwKXq+rpc9RYgfNZNNh1497Avl5EYkUkyO0aGgqsO4+6TDtjwW9asyeASCAH52bl+y30vncAl+J02fwaeB2o98asqi4Ffg4sxvltoT/OzdkmU9U9qprm5eGvuu/XkHzgIZyby6dwbmDf596zMAFObCEWYxrnfvFpl6r6/DcOY1qCtfiNqUNERotIf7eL5BpgOvCWn8syptm0l29HGtOcuuHcYI0HsnC6SD7zb0nGNB/r6jHGmABjXT3GGBNg2kRXT0JCgiYlJfm7DGOMaVM2btyYo6qJdfe3ieBPSkoiLc3bUW7GGGMARGR/ffutq8cYYwKMBb8xxgQYC35jjAkwFvzGGBNgLPiNMSbAWPAbY0yAseA3xpgA0ybG8RtjTJvk8UBlKVSUQIX7WF5Se/usxzr7Um6H+P7NWpYFvzGm/VKFyjLvg/acoVzfY3HDr1eWXeAPINB7rAW/MaYNqqyAskIoO+1dYDb0WB3MTTjnrBU9myg4HEIiIKSBx4iOzmNoROPHNfhY80+d14JDQZp/pVELfmNMbapOaJadhrIC57G0sM5z97Xq54XOn9LCMwFfWlA77M9XUGjjwRkWDVHxtfeHRp4jaBsL4xrnBodBUPu7FWrBb0xbpwrlRTVCt7CJ4Xy6zv5C8FR4995BIRAW4/wJdx/DoiE60Xmstd99rdFQriecg4J9+/kFIAt+Y1qap/LscK5uSTchnGue4213RlULOSwGwmOd5xFx0KGHu+2Gc3gMhMXWeB7tbNd8HhbthLMPuiKMb1nwG9MUHg8Un4CCI1BwDE5nu4Fc0Eiruk43SUWx9+8XGl27JR0eCzFdIKzf2S3ps8K6bnDHOH3GJuBZ8BsDTiv8dDYUHHX+FB51gr3waI19x5w/DXWDSFCdsHUDt2PvOuFct2sk5uyWdHiME/rtsH/Z+J8Fv2nfKsudsK4O8SM1nh9ztgvdlrt6zj4/Kh5iukFsV0gcDLHdnD8xXZ3H6EQI7+AEdUiEdXuYNsGC37RN5SV1WuU1QryqdV5wBIpy6zlZnMCuCvHuKTUCvUawx3SFkLAW/9GM8TULftO6lJ0+d3dLwVEoOXX2uRLstsS7Qsc+0Gt0jdZ5d2d/jNtKD7a/+iZw+fRvv4jMBWYDAvxVVZ8QkRHA/wERQAXwbVVd78s6jJ+pQml+wyFe1TovOObcBK0rOOxMd0v8AEiadCbEa7bUo+KtT9wYL/gs+EVkGE7ojwHKgPdF5F3g98AvVPU/InKduz3FV3UYH1KF4pO1W+dndbe4IV/fSJaQyDPB3XUYDLjy7NZ5bDeI7GR958Y0I1+2+IcA61S1CEBEVgEzcAYcd3CPiQMO+7AGcz48HqdvvFaI1w12t/Ve31wkYbFOcMd2h56jat8Mre5H7+rcFLVAN6bF+TL4twGPiEg8UAxcB6QB84APROQxnGmhx9d3sojMAeYA9OnTx4dlBrjSQji0EQ6uh4Pr4PiOhocsRnQ8E+IXja/RKq/T7RIW3eI/hjHGe6J6gRMYNXZxkW8C3wZOA9uBUpywX6Wqi0XkVmCOql7Z2HVSU1M1LS3NZ3UGDFU4deBMyB9cB8e2nRnGmDjEGeHSoUed7hb3MTTCv/UbY5pERDaqaupZ+30Z/HUK+F8gC/gN0FFVVUQEyFPVDo2da8F/nipK4cjWMyF/cL3TPQPOl4N6pTpTvvYeC71GOX3pxph2o6Hg9/Woni6qelxE+uD0748DvgtcBqwELgcyfFlDQCk8XqM1vx4Of+YsAgHQ8SLoOxl6j3GCvsuXbEijMa1YWYWHwtIKosODCQ9p3onqfP1//mK3j78cuF9VT4nIbGCBiIQAJbj9+KaJPJVwfOeZkD+4Dk7uc14LCoUeI2DM7DNBH9vNr+UaEygqKj2cLq0kv6ScwtIKCksrKCgpp6CkgoISZ7uwxN1X/fzMcc5jBaUVThfsi3ePYfLAxGat0afBr6qT6tn3MTDKl+/bLpXkQVbamZDPSjsz5j060Qn31G84j91HWH+8MU3k8SiFZU4Q1wzrqiBuMKyrAtvdV1xeec73ChKICQ8hNiKU2IgQYsJDSIgJIykhmpjwEDq4+2IiQuib0PyDJex3/dZIFU7srd1tc3wHzkhYga5D4eJb3P75MdCprw2LNAFLVSkqq6wO6Jqt5sKS2sFcfUw9+wpLvVuDINYNZCe4Q4iLDKVXx8jqAI+JcAPdff3MsWdCPiosGPHj/7MW/K1BeTEc3ly726Yox3ktvINzE/ZLNzgh3zMVIhq9F25Mm6CqlFZ4aoV1YUkF+TW7Pdzn+dVdJLVb4VXnebwYoxIVFlwd1jFuMHfrEOF1WMdGhBAdFkJQUNtvZFnw+0P+kdohf2QLeMqd1zr3g+SrzvTNJw62FYhMm+DxKCeLysgpLCO7oJScwtJaj9mFpeQWllFQeibQyyvPndjhIUFnBXCf6ChiIkLoEBFaI8xD3G6S0Fot8tjwUKLDgwkJtuk8qljw+1plhTNWvma3Td4B57XgcOh5CVz6bXdI5RiIad6bOMZcCFXlVFG5E97VQV5/sOeeLqOynqZ3WEgQiTHhJMSG0z0ugkGRsTVa2SFuKzu0zr4z4R0WYoHd3Cz4m1vxSfcmrDt2PmsjlJ92XovpBn3Gwrh7naDvdrFN+2tanKqSX1JRb3jn1An33NOl9bbKQ4OFhJhwEmPD6dohgmE94kiIDasO+KrHhJhwOkSE+LU/25zNgv9CqELu7tpfkMre5bwmwdBtGIy848xN2LjedhPW+ISqUlha0WiLvGaol1WevehMSJAQHxNGohvYg7vFVoe3sy+MLu52XGSohXkbZsHfFGVFcHhT7f754pPOaxEdnXAffrMT9D0ucVZlMuYCnC6tp2XeQLBXjfuuKUggPuZMePfvEkOi2yJPrBXq4XSMDG0XNy7NuVnwNyYvq3bIH/38zORlCQNh8PVnpjyIT7a54I1XissqySks5bgXXS31jQkXgfjosOrQ7psQXd0irwrxqtc6RYURbGFu6rDgr1JZDke31r4Jm3/IeS0k0hlSOWGuexN2NER19m+9plUpKa88K7QbCvXTZfV/wadzdFh1eI/s07FWa7xmsHeOCrMRKuaCBG7wn86FrBohf2jTmcVC4npDn3Fn+ua7DoPgUP/Wa1qN7IJSPtmdw0cZ2Ww+eIrsglIKSur/8k9cZGh1aA/v1bFWq7xml0vn6DBCLcxNCwmM4Pd4IOeL2t02ubud14JCnKmIU7/hhHyvMRDX07/1mlalpLySDZknWJ2Rw+qMHHYeyQegY1QoY5I6Mzk5sTrca7bS42PCmn1yLWOaQ/sO/i2vw+f/gIMboDTP2RcV77TkR97p3oQdCaGR/q3TtCoej7LraAGrM7L5eHcO6/edoLTCQ2iwMOqiTvzwy4OYlJzA0B5x1n9u2qT2Hfy5GZB/GIbdeOYmbOd+NqTSnOVYfgmrM3L42A37nEJnScmBXWO4Y+xFTEpOYGy/zkSFte//ZUxgaN9/i6f+FC7/mb+rMK1QUVkF6/ad4OOMHFZnZJN+rBCAhJgwJgxIYFJyIhMHJNAtzmY5Ne1P+w5+a9kbl8ejbD+cz+rd2axOz2Hj/pOUVXoICwliTFJnbrqkFxOTExjSrYONZTftXvsOfhPQDp8q5uMMZ/TNmj25nDjtdN8M7hbLrAlJTByQwJi+nYkItRuwJrBY8Jt2o7C0gnV7c93RN9nsyXbmSEqMDWfKoEQmJScwYUACXWKt+8YENgt+02ZVepTPD+WxOj2b1btz2LT/JBUeJSI0iLF945k5pg+TkhMZ2DXG5pUxpgYLftOmHDxR5Iy+2Z3NJ7tzySt21jEY1rMD35rUj8nJCVxyUSfrvjGmERb8plXLLyln7Z7c6tE3mblFAHSPi+DqL3Vl0sBEJvSPJz4m3M+VGtN2WPCbVqWi0sOWrFPV35LdfPAUlR4lKiyYcf3iuWt8EpOSE+ifaN03xpwvC37jV6rK/twiVu/OYXV6Nmv35FJQWoEIXNyrI/dd1p9JyQmM7NPJVmIypplY8JsWl1dUzpo9OXzk9tUfPOFMjtezYyTTUrozKTmR8f3j6Rhlq5MZ4wsW/Mbnyis9fHbgFKszslmdkcPWrFN4FGLCQ7i0fzxzJvVjYnIiSfFR1n1jTAuw4DfNTlXZm3Oa1enOvDdr9+RyuqySIIERvTvy3cuTmZScQErvjjYVsTF+YMFvmsWJ02V8sjunevTN4bwSAC6Kj+LGS3oycUAil/aPJy7S1jUwxt8s+M15Ka2oZOP+k27Q57DtcB6q0CEihAkDErj/8gQmDUikT3yUv0s1xtTh0+AXkbnAbECAv6rqE+7+7wL3A5XAv1X1R76sw1w4VSXjeGH1dAjr9p6guLySkCDhkj6dmH/lQCYlJzC8Z5wtC2hMK+ez4BeRYTihPwYoA94XkXeB3sB0IEVVS0Wki69qMBcmp9BdYjDdGX1zLL8UgH6J0dya2otJyYmM6x9PTLj94mhMW+LL/2OHAOtUtQhARFYBM4BU4LeqWgqgqsd9WINpAo9HWbMnt3r0zY4aSwxOGJDA5OQEJiYn0rOjrVhmTFvmy+DfBjwiIvFAMXAdkAYMBCaJyCNACfADVd1Q92QRmQPMAejTp48PyzRVHv3wCxau3GNLDBrTzvks+FV1p4j8DvgQOA1sxunTDwE6A+OA0cA/RKSfqmqd8xcBiwBSU1NrvWaa37H8Ep79eB/TLu7O72++2JYYNKYd8+ldOFV9RlVHqepk4CSQDmQB/1LHesADJPiyDnNuf16xm0qP8qMvD7bQN6ad8/Woni6qelxE+uD074/DCfqpwAoRGQiEATm+rMM07tCpYl5df5BbUnvZ8EtjAoCvm3aL3T7+cuB+VT0lIs8Cz4rINpzRPnfV7eYxLeup5RkAfOfyZD9XYoxpCT4NflWdVM++MuBOX76v8d7+3NO8kZbFHWP72GgdYwKEfdMmwC1YlkFwkHD/1AH+LsUY00Is+APY7uOFvPXZIf5r3EV06WALkBsTKCz4A9iCZRlEhAZz75T+/i7FGNOCLPgD1K6j+byz5TCzxieRYOvVGhNQLPgD1B+XpBMbHsKcyf38XYoxpoVZ8AegbYfy+GD7Mb45qa8tb2hMALLgD0CPL0knLjKUuyf29Xcpxhg/sOAPMBv3n2T5ruPMmdyPDhG2GpYxgciCP8D8cUk68dFhzBqf5O9SjDF+YsEfQD7dm8vHu3O4b0p/om3xFGMClgV/gFBVHv8wnS6x4dw57iJ/l2OM8SML/gDx8e4c1mee4DuXDyAiNNjf5Rhj/MiCPwCoKn/4MJ0ecRHcNrq3v8sxxviZBX8AWL7rOJsPnuK7VyQTHmKtfWMCnQV/O6eqPL4knT6do7h5VC9/l2OMaQUs+Nu5D7YfZfvhfOZekUxosP3nNsZY8LdrlR6ntd8vMZqvjuzp73KMMa2EBX879u7Ww6QfK2T+lQMJDhJ/l2OMaSUs+NupikoPC5ZmMLhbLNcP7+7vcowxrYgFfzv11ubD7M05zbwrBxJkrX1jTA0W/O1QeaWHBcvSGdazA18e2tXf5RhjWhkL/nbojbQsDp4o5vtXDULEWvvGmNos+NuZkvJK/rQ8g5F9OjJlUKK/yzHGtEIW/O3Ma+sPcCSvxFr7xpgGnTP4ReQrImL/QLQBxWWVPL1yD2P7dmbCgHh/l2OMaaW8CfTbgAwR+b2IDPZ1Qeb8vfzpfrILSvn+1dbaN8Y07JzBr6p3AiOBPcDzIrJWROaISKzPqzNeKyytYOGqPUxKTmBM387+LscY04p51YWjqvnAP4HXgO7AjcAmEfluY+eJyFwR2SYi20VkXp3Xvi8iKiIJ51e6qemFNZmcOF3GA1cN9HcpxphWzps+/htE5E1gJRAKjFHVa4EU4PuNnDcMmA2McY+dJiID3Nd6A1cDBy70BzCQV1zOX1bt4YrBXRjZp5O/yzHGtHLetPhvAv6oqsNV9VFVPQ6gqkXANxs5bwiwTlWLVLUCWAXMcF/7I/AjQM+/dFPlmY/3kV9SwXxr7RtjvOBN8D8MrK/aEJFIEUkCUNVljZy3DZgkIvEiEgVcB/QWkenAIVXd0tibuvcR0kQkLTs724syA9PJ02U8+/E+rh3WjWE94/xdjjGmDfAm+N8APDW2K919jVLVncDvgA+B94HNQDjwEPDfXpy/SFVTVTU1MdG+iNSQRav3crrMWvvGGO95E/whqlpWteE+D/Pm4qr6jKqOUtXJwElgO9AX2CIimUAvnJvE3ZpcuSGnsJTnP8nkKxf3YGBXG2RljPGON8GfLSI3VG24XTU53lxcRLq4j31w+vdfUNUuqpqkqklAFnCJqh5tcuWGhSv3UFpRybwrk/1dijGmDQnx4ph7gVdE5ClAgIPA1728/mIRiQfKgftV9dR5VWnOciy/hJc/3c+MS3rRLzHG3+UYY9qQcwa/qu4BxolIjLtd6O3FVXXSOV5P8vZapranV+ym0qPMvcJa+8aYpvGmxY+IXA8MBSKqpgJQ1V/6sC7TiKyTRby6/gC3ju5N785R/i7HGNPGePMFrv/Dma/nuzhdPbcAF/m4LtOIp5bvRhC+M3WAv0sxxrRB3tzcHa+qXwdOquovgEsBGzvoJ5k5p3ljYxZfG9uHHh0j/V2OMaYN8ib4S9zHIhHpgXOj1lbv9pMnl2UQEiR8e0p/f5dijGmjvOnjf0dEOgKPAptwpln4qy+LMvXbfbyQtzYf4luT+tGlQ4S/yzHGtFGNBr+7AMsydxjmYhF5F4hQ1byWKM7U9sTSdCJCg7lncj9/l2KMacMa7epRVQ/wdI3tUgt9/9h5JJ93tx7hGxOSiI8J93c5xpg2zJs+/mUicpPYkk5+9ccl6cRGhDBnkvXtG2MujDfBfw/OpGylIpIvIgUiku/jukwNn2fl8eGOY3xrYj/iokL9XY4xpo3z5pu7NvuXnz2+5As6RoVy98Qkf5dijGkHzhn8IjK5vv2q+lHzl2Pq2rj/JCu+yObH1wwmNsJa+8aYC+fNcM4f1ngegbOU4kbgcp9UZGp5fMkXJMSEcdd4+7K0MaZ5eNPV85Wa2+56uU/4qiBzxto9uXyyO5efXT+EqDCvplUyxphz8ubmbl1ZOOvpGh9SVR5f8gVdO4Rz5zhr7Rtjmo83ffx/4syi6EHACJxv8BofWp2Rw4bMk/xq+lAiQoP9XY4xph3xpv8grcbzCuBVVf3ER/UYnNb+H5ak07NjJLeO7u3vcowx7Yw3wf9PoERVKwFEJFhEolS1yLelBa5lO4+z5eApfnfTcMJDrLVvjGleXn1zF6g5/28ksNQ35RiPR3l8SToXxUcx45Je/i7HGNMOeRP8ETWXW3Sf27JPPvLB9qPsOJLP3CuSCQ0+n3vvxhjTOG+S5bSIXFK1ISKjgGLflRS4Kt3Wfv/EaKaP6Onvcowx7ZQ3ffzzgDdE5DDO0ovdcJZiNM3s3a2HyTheyFNfG0lwkM2JZ4zxDW++wLVBRAYDg9xdX6hquW/LCjwVlR6eWJrB4G6xXDfMFjgzxviON4ut3w9Eq+o2Vd0GxIjIt31fWmB587ND7Ms5zfyrBhJkrX1jjA9508c/212BCwBVPQnM9llFAaiswsOCZRkM7xnH1V/q6u9yjDHtnDfBH1xzERYRCQbCfFdS4Hlj40GyThbzwNUDsfVujDG+5s3N3feB10XkL+72PcB/fFdSYCkpr+Sp5bu5pE9HpgxM9Hc5xpgA4E3w/xiYA9zrbm/FGdljmsGr6w9wJK+EP9ySYq19Y0yLOGdXj7vg+jogE2cu/suBnd5cXETmisg2EdkuIvPcfY+KyC4R2Soib4pIx/Mtvq0rLqvk6RV7GNevM+MHJPi7HGNMgGgw+EVkoIj8j4jsAv4EHABQ1amq+tS5Liwiw3BuAo8BUoBpIjIAWAIMU9WLgXTgJxf+Y7RNL32aSU5hKd+/etC5DzbGmGbSWIt/F07rfpqqTlTVPwGVTbj2EGCdqhapagWwCpihqh+62wCfAgE5IU1haQULV+5h8sBERid19nc5xpgA0ljwzwCOACtE5K8icgXON3e9tQ2YJCLxIhIFXAfUnWP4bhq4USwic0QkTUTSsrOzm/C2bcPzn+zjZFE5D1w10N+lGGMCTIPBr6pvqertwGBgBc7UDV1EZKGIXH2uC6vqTuB3wIc4I4M2U+M3BhH5Kc78/q80cP4iVU1V1dTExPY12iWvuJxFH+3lyiFdGNG7o7/LMcYEGG9u7p5W1b+7a+/2Aj7DGelzTqr6jKqOUtXJwEmcPn1EZBYwDbhDVbWRS7RLz6zeS35JBfOttW+M8YMmzfurqifdlvgV3hwvIl3cxz44XUd/F5FrgB8BNwTiYi4nT5fx7CeZXDe8G0N7xPm7HGNMAPJmHP+FWCwi8UA5cL+qnhKRp4BwYIk7bv1TVb23sYu0J3/5aC+nyyqYd6W19o0x/uHT4FfVSfXsG+DL92zNsgtKeWFNJtNTejCwa6y/yzHGBChb4qkFLVy5h7JKD3OttW+M8SML/hZyNK+El9ftZ8bInvRNiPZ3OcaYAGbB30KeXrEbVeV7VyT7uxRjTICz4G8BWSeLeG3DAW5N7U3vzrZOvTHGvyz4W8Cflu1GRPjO5QF7X9sY04pY8PtYZs5p/rkpi6+N6UP3uEh/l2OMMRb8vrZgWQahwcK3p/b3dynGGANY8PvU7uMFvLX5EHddmkSX2Ah/l2OMMYAFv0/9cWkGUaHB3HOZtfaNMa2HBb+P7Dicz7+3HuHuiX3pHG1r0xtjWg8Lfh/549J0YiNC+NbEfv4uxRhjarHg94GtWadYsuMYsyf1Iy4q1N/lGGNMLRb8PvD4knQ6RYXyjQlJ/i7FGGPOYsHfzDbuP8HKL7K557L+xEZYa98Y0/pY8DezP3yYTkJMGF+/9CJ/l2KMMfWy4G9Ga/bksGZPLt+eMoCoMF+vcWOMMefHgr+ZqCqPf5hOtw4RfG1sH3+XY4wxDbLgbyYfZeSQtv8k918+gIjQYH+XY4wxDbLgbwZOa/8LenaM5LbU3v4uxxhjGmXB3wyW7jzOlqw85l6RTFiIfaTGmNbNUuoCeTzK40vSSYqPYsYlPf1djjHGnJMF/wV6f/tRdh7JZ+6VyYQE28dpjGn9LKkuQKXb2h/QJYYbUqy1b4xpGyz4L8A7Ww6z+3gh868cSHCQ+LscY4zxigX/eaqo9PDE0nQGd4vl2mHd/F2OMcZ4zYL/PP1r0yEyc4v4/tWDCLLWvjGmDbHgPw9lFR4WLMsgpVccVw7p4u9yjDGmSXwa/CIyV0S2ich2EZnn7ussIktEJMN97OTLGnzhH2kHOXSqmPlXDUTEWvvGmLbFZ8EvIsOA2cAYIAWYJiIDgAeBZaqaDCxzt9uMkvJKnlq+m9SLOnHZwER/l2OMMU3myxb/EGCdqhapagWwCpgBTAdecI95AfiqD2todn9fd4Cj+SU8cLW19o0xbZMvg38bMElE4kUkCrgO6A10VdUj7jFHga71nSwic0QkTUTSsrOzfVim94rLKvnzyj1c2i+e8f0T/F2OMcacF58Fv6ruBH4HfAi8D2wGKusco4A2cP4iVU1V1dTExNbRpfLi2kxyCkv5/tUD/V2KMcacN5/e3FXVZ1R1lKpOBk4C6cAxEekO4D4e92UNzaWwtIL/W7WHywYmkprU2d/lGGPMefP1qJ4u7mMfnP79vwNvA3e5h9wF/D9f1tBcnvt4HyeLynngKmvtG2PaNl+vD7hYROKBcuB+VT0lIr8F/iEi3wT2A7f6uIYLlldUzqLVe7nqS11J6d3R3+UYY8wF8Wnwq+qkevblAlf48n2b298+3ktBSYW19o0x7YJ9c/ccTpwu49mP93H98O4M6d7B3+UYY8wFs+A/h798tIfi8krmX5Xs71KMMaZZWPA34nhBCS+syWT6iJ4M6BLr73KMMaZZWPA3YuHKPZRXKnOvsNa+Mab9sOBvwJG8Yl5Zd4CbLulJUkK0v8sxxphmY8HfgKdX7EZV+e7l1to3xrQvFvz1OHiiiNc3HOS20b3p3TnK3+UYY0yz8vUXuNqkPy3PQET4zlRr7ZvWpby8nKysLEpKSvxdimlFIiIi6NWrF6GhoV4db8Ffx76c0yzedIi7Lk2iW1yEv8sxppasrCxiY2NJSkqyacENAKpKbm4uWVlZ9O3b16tzrKunjgVL0wkLDuK+Kf39XYoxZykpKSE+Pt5C31QTEeLj45v0W6AFfw0Zxwr4f1sO8/XxF5EYG+7vcoypl4W+qaupfycs+Gt4YmkG0WEh3DvZWvvGmPbLgt+1/XAe//78CHdPSKJTdJi/yzGmVcrNzWXEiBGMGDGCbt260bNnz+rtsrKyRs9NS0vje9/73jnfY/z48c1VLgDz5s2jZ8+eeDyeZr1uW2Y3d11/XJJBh4gQvjmpn79LMabVio+PZ/PmzQA8/PDDxMTE8IMf/KD69YqKCkJC6o+V1NRUUlNTz/kea9asaZZaATweD2+++Sa9e/dm1apVTJ06tdmuXVNjP3dr1HYq9aEtB0+xdOcxfnD1QOIivRsOZYy//eKd7ew4nN+s1/xSjw78z1eGNumcWbNmERERwWeffcaECRO4/fbbmTt3LiUlJURGRvLcc88xaNAgVq5cyWOPPca7777Lww8/zIEDB9i7dy8HDhxg3rx51b8NxMTEUFhYyMqVK3n44YdJSEhg27ZtjBo1ipdffhkR4b333uOBBx4gOjqaCRMmsHfvXt59992zalu5ciVDhw7ltttu49VXX60O/mPHjnHvvfeyd+9eABYuXMj48eN58cUXeeyxxxARLr74Yl566SVmzZrFtGnTuPnmm8+q7+c//zmdOnVi165dpKen89WvfpWDBw9SUlLC3LlzmTNnDgDvv/8+Dz30EJWVlSQkJLBkyRIGDRrEmjVrSExMxOPxMHDgQNauXUtLLDVrwQ/8YUk6naJCmTXBu6FQxpjasrKyWLNmDcHBweTn57N69WpCQkJYunQpDz30EIsXLz7rnF27drFixQoKCgoYNGgQ991331nj0D/77DO2b99Ojx49mDBhAp988gmpqancc889fPTRR/Tt25eZM2c2WNerr77KzJkzmT59Og899BDl5eWEhobyve99j8suu4w333yTyspKCgsL2b59O7/+9a9Zs2YNCQkJnDhx4pw/96ZNm9i2bVv1MMpnn32Wzp07U1xczOjRo7npppvweDzMnj27ut4TJ04QFBTEnXfeySuvvMK8efNYunQpKSkpLRL6YMFPWuYJPkrP5ifXDiYmPOA/DtOGNLVl7ku33HILwcHBAOTl5XHXXXeRkeF8EbK8vLzec66//nrCw8MJDw+nS5cuHDt2jF69etU6ZsyYMdX7RowYQWZmJjExMfTr1686bGfOnMmiRYvOun5ZWRnvvfcejz/+OLGxsYwdO5YPPviAadOmsXz5cl588UUAgoODiYuL48UXX+SWW24hISEBgM6dz7229pgxY2qNnX/yySd58803ATh48CAZGRlkZ2czefLk6uOqrnv33Xczffp05s2bx7PPPss3vvGNc75fcwn4pPvDh+kkxITz9UuT/F2KMW1WdPSZiQx//vOfM3XqVN58800yMzOZMmVKveeEh58ZMh0cHExFRcV5HdOQDz74gFOnTjF8+HAAioqKiIyMZNq0aV5fAyAkJKT6xrDH46l1E7vmz71y5UqWLl3K2rVriYqKYsqUKY2Ore/duzddu3Zl+fLlrF+/nldeeaVJdV2IgB7Vs2Z3Dmv35nL/1P5EhgX7uxxj2oW8vDx69uwJwPPPP9/s1x80aBB79+4lMzMTgNdff73e41599VX+9re/kZmZSWZmJvv27WPJkiUUFRVxxRVXsHDhQgAqKyvJy8vj8ssv54033iA3NxeguqsnKSmJjRs3AvD22283+BtMXl4enTp1Iioqil27dvHpp58CMG7cOD766CP27dtX67oA3/rWt7jzzjtr/cbUEgI2+FWVPyxJp3tcBDPH9PF3Oca0Gz/60Y/4yU9+wsiRI5vUQvdWZGQkf/7zn7nmmmsYNWoUsbGxxMXF1TqmqKiI999/n+uvv756X3R0NBMnTuSdd95hwYIFrFixguHDhzNq1Ch27NjB0KFD+elPf8pll11GSkoKDzzwAACzZ89m1apVpKSksHbt2lqt/JquueYaKioqGDJkCA8++CDjxo0DIDExkUWLFjFjxgxSUlK47bbbqs+54YYbKCwsbNFuHgBR1RZ9w/ORmpqqaWlpzXrNlV8cZ9ZzG3jkxmHcMfaiZr22Mb6yc+dOhgwZ4u8y/K6wsJCYmBhUlfvvv5/k5GTmz5/v77KaLC0tjfnz57N69eoLvlZ9fzdEZKOqnjWGNiBb/KrK40vS6dUpkltG9fZ3OcaYJvrrX//KiBEjGDp0KHl5edxzzz3+LqnJfvvb33LTTTfxm9/8psXfOyBb/B9uP8qclzby+5sv5tZUC37TdliL3zTEWvyN8Hic1n7fhGhmjOzp73KMMabFBVzw/2fbUXYdLWDelcmEBAfcj2+MMYEV/JUe5Y9L00nuEsO0i3v4uxxjjPGLgAr+t7ccYvfxQuZfNZDgIJvT3BgTmHwa/CIyX0S2i8g2EXlVRCJE5AoR2SQim0XkYxEZ4MsaqpRXeliwNIMvde/ANUO7tcRbGtPuTJ06lQ8++KDWvieeeIL77ruvwXOmTJlC1eCM6667jlOnTp11zMMPP8xjjz3W6Hu/9dZb7Nixo3r7v//7v1m6dGkTqm9cIE3f7LPgF5GewPeAVFUdBgQDtwMLgTtUdQTwd+Bnvqqhpn9tyiIzt4gHrhpIkLX2jTkvM2fO5LXXXqu177XXXmt0orSa3nvvPTp27Hhe7103+H/5y19y5ZVXnte16qo7fbOv+OILbefD13P1hACRIlIORAGHAQU6uK/Huft8qqzCw5PLdpPSuyNXDOni67czpmX850E4+nnzXrPbcLj2tw2+fPPNN/Ozn/2MsrIywsLCyMzM5PDhw0yaNIn77ruPDRs2UFxczM0338wvfvGLs85PSkoiLS2NhIQEHnnkEV544QW6dOlC7969GTVqFOCM0V+0aBFlZWUMGDCAl156ic2bN/P222+zatUqfv3rX7N48WJ+9atfVU+XvGzZMn7wgx9QUVHB6NGjWbhwIeHh4SQlJXHXXXfxzjvvUF5ezhtvvMHgwYPPqivQpm/2WYtfVQ8BjwEHgCNAnqp+CHwLeE9EsoD/Aur9WyYic0QkTUTSsrOzL6iW19MOcuhUMd+/aqCtV2rMBejcuTNjxozhP//5D+C09m+99VZEhEceeYS0tDS2bt3KqlWr2Lp1a4PX2bhxI6+99hqbN2/mvffeY8OGDdWvzZgxgw0bNrBlyxaGDBnCM888w/jx47nhhht49NFH2bx5M/37n1ketaSkhFmzZvH666/z+eefU1FRUT0PD0BCQgKbNm3ivvvua7A7qWr65htvvJF///vf1fPxVE3fvGXLFjZt2sTQoUOrp29evnw5W7ZsYcGCBef83DZt2sSCBQtIT08HnOmbN27cSFpaGk8++SS5ublkZ2cze/ZsFi9ezJYtW3jjjTdqTd8MNNv0zT5r8YtIJ2A60Bc4BbwhIncCM4DrVHWdiPwQeBznH4NaVHURsAicL3Cdbx0l5ZU8tTyD0UmdmJSccL6XMab1aaRl7ktV3T3Tp0/ntdde45lnngHgH//4B4sWLaKiooIjR46wY8cOLr744nqvsXr1am688UaioqIAZ86aKtu2beNnP/sZp06dorCwkC9/+cuN1vPFF1/Qt29fBg4cCMBdd93F008/zbx58wDnHxKAUaNG8a9//eus8wNx+mZfdvVcCexT1WwAEfkXMAFIUdV17jGvA+/7sAZeWXeAY/mlPHHbSGvtG9MMpk+fzvz589m0aRNFRUWMGjWKffv28dhjj7FhwwY6derErFmzGp2SuDGzZs3irbfeIiUlheeff56VK1deUL1VUzs3NK1zIE7f7MtRPQeAcSISJU7iXgHsAOJEZKB7zFXATl8VUFRWwcKVu5kwIJ5L+8f76m2MCSgxMTFMnTqVu+++u/qmbn5+PtHR0cTFxXHs2LHqrqCGTJ48mbfeeovi4mIKCgp45513ql8rKCige/fulJeX1wq52NhYCgoKzrrWoEGDyMzMZPfu3QC89NJLXHbZZV7/PIE4fbMv+/jXAf8ENgGfu++1CJgNLBaRLTh9/D/0VQ0vrt1PTmEZD1w1yFdvYUxAmjlzJlu2bKkO/pSUFEaOHMngwYP52te+xoQJExo9/5JLLuG2224jJSWFa6+9ltGjR1e/9qtf/YqxY8cyYcKEWjdib7/9dh599FFGjhzJnj17qvdHRETw3HPPccsttzB8+HCCgoK49957vfo5AnX65nY9Sds/N2axfl8uv785xQdVGdPybJK2wOTN9M1NmaStXS+9ePOoXtw8qte5DzTGmFbqt7/9LQsXLmzWpRkDasoGY4xpax588EH279/PxIkTm+2aFvzGtDFtoXvWtKym/p2w4DemDYmIiCA3N9fC31RTVXJzc4mIiPD6nHbdx29Me9OrVy+ysrK40G+zm/YlIiKCXr28v59pwW9MGxIaGlrrG6DGnA/r6jHGmABjwW+MMQHGgt8YYwJMm/jmrohkA/vP8/QEIKcZy2nv7PNqGvu8msY+r6a7kM/sIlU9aw7nNhH8F0JE0ur7yrKpn31eTWOfV9PY59V0vvjMrKvHGGMCjAW/McYEmEAI/kX+LqCNsc+raezzahr7vJqu2T+zdt/Hb4wxprZAaPEbY4ypwYLfGGMCTLsNfhF5VkSOi8g2f9fSFohIbxFZISI7RGS7iMz1d02tmYhEiMh6Ednifl6/8HdNbYGIBIvIZyLyrr9rae1EJFNEPheRzSLS9CUIG7t2e+3jF5HJQCHwoqoO83c9rZ2IdAe6q+omEYkFNgJfVdUdfi6tVRIRAaJVtVBEQoGPgbmq+qmfS2vVROQBIBXooKrT/F1PayYimUCqqjb7F97abYtfVT8CTpzzQAOAqh5R1U3u8wJgJ9DTv1W1XuoodDdD3T/tsxXVTESkF3A98Dd/1xLo2m3wm/MnIknASGCdn0tp1dxui83AcWCJqtrn1bgngB8BHj/X0VYo8KGIbBSROc15YQt+U4uIxACLgXmqmu/velozVa1U1RFAL2CMiFiXYgNEZBpwXFU3+ruWNmSiql4CXAvc73ZfNwsLflPN7ateDLyiqv/ydz1thaqeAlYA1/i5lNZsAnCD22/9GnC5iLzs35JaN1U95D4eB94ExjTXtS34DVB9s/IZYKeqPu7velo7EUkUkY7u80jgKmCXX4tqxVT1J6raS1WTgNuB5ap6p5/LarVEJNodZIGIRANXA802QrHdBr+IvAqsBQaJSJaIfNPfNbVyE4D/wmmJbXb/XOfvolqx7sAKEdkKbMDp47chiqa5dAU+FpEtwHrg36r6fnNdvN0O5zTGGFO/dtviN8YYUz8LfmOMCTAW/MYYE2As+I0xJsBY8BtjTICx4DcBTUQqawxf3SwiDzbjtZNsdljTGoX4uwBj/KzYnXbBmIBhLX5j6uHOhf57dz709SIywN2fJCLLRWSriCwTkT7u/q4i8qY7P/8WERnvXipYRP7qztn/ofstX0Tke+7aB1tF5DU//ZgmQFnwm0AXWaer57Yar+Wp6nDgKZyZJQH+BLygqhcDrwBPuvufBFapagpwCbDd3Z8MPK2qQ4FTwE3u/geBke517vXNj2ZM/eybuyagiUihqsbUsz8TuFxV97qT1x1V1XgRycFZsKbc3X9EVRNEJBvopaqlNa6RhDOVQ7K7/WMgVFV/LSLv4ywU9BbwVo25/Y3xOWvxG9MwbeB5U5TWeF7Jmftq1wNP4/x2sEFE7H6baTEW/MY07LYaj2vd52twZpcEuANY7T5fBtwH1Qu0xDV0UREJAnqr6grgx0AccNZvHcb4irUyTKCLdFfRqvK+qlYN6ezkzr5ZCsx0930XeE5EfghkA99w988FFrmzwFbi/CNwpIH3DAZedv9xEOBJd05/Y1qE9fEbUw9fLnRtjL9ZV48xxgQYa/EbY0yAsRa/McYEGAt+Y4wJMBb8xhgTYCz4jTEmwFjwG2NMgPn/24We3RAUqDYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = plot_history(\n",
    "    history,\n",
    "    metrics=['loss', 'acc'],\n",
    "    labels=['Loss', 'Accuracy'],\n",
    "    titles='Training of MNIST',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initilalizing Your Optimizer and Loss Function Yourself\n",
    "\n",
    "Instead of using magic strings for the optimizer and the loss function, it's quite easy to initialize your own and pass them to Poutyne."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m7.01s \u001b[35mloss:\u001b[94m 0.378740\u001b[35m acc:\u001b[94m 87.750000\u001b[35m val_loss:\u001b[94m 0.091613\u001b[35m val_acc:\u001b[94m 97.275000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.52s \u001b[35mloss:\u001b[94m 0.131763\u001b[35m acc:\u001b[94m 95.912500\u001b[35m val_loss:\u001b[94m 0.060723\u001b[35m val_acc:\u001b[94m 98.233333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.38s \u001b[35mloss:\u001b[94m 0.099851\u001b[35m acc:\u001b[94m 96.922917\u001b[35m val_loss:\u001b[94m 0.053079\u001b[35m val_acc:\u001b[94m 98.525000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.53s \u001b[35mloss:\u001b[94m 0.088178\u001b[35m acc:\u001b[94m 97.235417\u001b[35m val_loss:\u001b[94m 0.050858\u001b[35m val_acc:\u001b[94m 98.541667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.58s \u001b[35mloss:\u001b[94m 0.075830\u001b[35m acc:\u001b[94m 97.645833\u001b[35m val_loss:\u001b[94m 0.042662\u001b[35m val_acc:\u001b[94m 98.766667\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.84s \u001b[35mtest_loss:\u001b[94m 0.030614\u001b[35m test_acc:\u001b[94m 98.950000\u001b[0m                                                   \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Instantiating our loss function and optimizer\n",
    "own_optimizer = optim.SGD(network.parameters(), lr=learning_rate)\n",
    "own_loss_function = nn.CrossEntropyLoss()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    own_optimizer,\n",
    "    own_loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs)\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Your Own Metric\n",
    "\n",
    "Poutyne offers [a few metrics out-of-the-box](https://poutyne.org/metrics.html#object-oriented-api) and also supports metrics from [TorchMetrics](https://torchmetrics.readthedocs.io/), a library implementing many known metrics in PyTorch. See the [TorchMetrics documentation](https://torchmetrics.readthedocs.io/en/latest/references/modules.html) for available TorchMetrics metrics.\n",
    "\n",
    "If metrics available in Poutyne and TorchMetrics do not suit your needs, you can define your own metric. Poutyne offers two interfaces for metrics. The first interface is the same as PyTorch loss functions: `metric(y_pred, y_true)`. When using that interface, the metric is assumed to be decomposable and is averaged for the whole epoch.\n",
    "\n",
    "The second interface is defined by the [`Metric` class](https://poutyne.org/metrics.html#interface). As documented in the class, it provides methods for updating and computing the metric. This interface is compatible with [TorchMetrics](https://torchmetrics.readthedocs.io/).\n",
    "\n",
    "In the code below, we demonstrate the usage of the two interfaces by implementing the accuracy in the two ways. The Metric interface is a bit more complex to use but also offers more flexibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating a metric using the PyTorch loss interface.\n",
    "def my_accuracy_func(y_pred, y_true):\n",
    "    y_pred = y_pred.argmax(1)\n",
    "    acc_pred = (y_pred == y_true).float().mean()\n",
    "    return acc_pred * 100\n",
    "\n",
    "\n",
    "# Creating a metric using the Metric interface.\n",
    "class MyAccuracyMetric(Metric):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.reset()\n",
    "\n",
    "    def _compute_stats(self, y_pred, y_true):\n",
    "        \"\"\"\n",
    "        Compute the number of true positives and the total number of elements\n",
    "        of a given batch.\n",
    "        \"\"\"\n",
    "        y_pred = y_pred.argmax(1)\n",
    "        num_true_positives = (y_pred == y_true).long().sum().item()\n",
    "        total_exemples = y_true.numel()\n",
    "        return num_true_positives, total_exemples\n",
    "\n",
    "    def _compute_accuracy_from_stats(self, num_true_positives, total_exemples):\n",
    "        \"\"\"\n",
    "        Compute the accuracy given by the statistics computed in _compute_stats().\n",
    "        \"\"\"\n",
    "        return num_true_positives / total_exemples * 100\n",
    "\n",
    "    def _update(self, y_pred, y_true):\n",
    "        \"\"\"\n",
    "        Increment the running number of true positives and number of elements.\n",
    "        \"\"\"\n",
    "        num_true_positives, total_exemples = self._compute_stats(y_pred, y_true)\n",
    "        self.num_true_positives += num_true_positives\n",
    "        self.total_exemples += total_exemples\n",
    "        return num_true_positives, total_exemples\n",
    "\n",
    "    def forward(self, y_pred, y_true):\n",
    "        \"\"\"\n",
    "        When using the metric as a batch metric, this is called. If the metric is only\n",
    "        used as an epoch metric, there is no need to implement this.\n",
    "        \"\"\"\n",
    "        num_true_positives, total_exemples = self._update(y_pred, y_true)\n",
    "        return self._compute_accuracy_from_stats(num_true_positives, total_exemples)\n",
    "\n",
    "    def update(self, y_pred, y_true):\n",
    "        \"\"\"\n",
    "        When using the metric as an epoch metric, this is called. If the metric is only\n",
    "        used as a batch metric, there is no need to implement this.\n",
    "        \"\"\"\n",
    "        self._update(y_pred, y_true)\n",
    "\n",
    "    def compute(self):\n",
    "        \"\"\"\n",
    "        This is called at the end to get the value for the whole epoch.\n",
    "        \"\"\"\n",
    "        return self._compute_accuracy_from_stats(self.num_true_positives, self.total_exemples)\n",
    "\n",
    "    def reset(self) -> None:\n",
    "        \"\"\"\n",
    "        Reset the statistics for another epoch.\n",
    "        \"\"\"\n",
    "        self.num_true_positives = 0\n",
    "        self.total_exemples = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.74s \u001b[35mloss:\u001b[94m 0.383959\u001b[35m my_accuracy_func:\u001b[94m 87.666667\u001b[35m my_accuracy_metric:\u001b[94m 87.666667\u001b[35m val_loss:\u001b[94m 0.090163\u001b[35m val_my_accuracy_func:\u001b[94m 97.383333\u001b[35m val_my_accuracy_metric:\u001b[94m 97.383333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.66s \u001b[35mloss:\u001b[94m 0.128223\u001b[35m my_accuracy_func:\u001b[94m 96.100000\u001b[35m my_accuracy_metric:\u001b[94m 96.100000\u001b[35m val_loss:\u001b[94m 0.066320\u001b[35m val_my_accuracy_func:\u001b[94m 98.091667\u001b[35m val_my_accuracy_metric:\u001b[94m 98.091667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.70s \u001b[35mloss:\u001b[94m 0.099619\u001b[35m my_accuracy_func:\u001b[94m 97.010417\u001b[35m my_accuracy_metric:\u001b[94m 97.010417\u001b[35m val_loss:\u001b[94m 0.057889\u001b[35m val_my_accuracy_func:\u001b[94m 98.208333\u001b[35m val_my_accuracy_metric:\u001b[94m 98.208333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.63s \u001b[35mloss:\u001b[94m 0.083622\u001b[35m my_accuracy_func:\u001b[94m 97.460417\u001b[35m my_accuracy_metric:\u001b[94m 97.460417\u001b[35m val_loss:\u001b[94m 0.050346\u001b[35m val_my_accuracy_func:\u001b[94m 98.366667\u001b[35m val_my_accuracy_metric:\u001b[94m 98.366667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.92s \u001b[35mloss:\u001b[94m 0.076653\u001b[35m my_accuracy_func:\u001b[94m 97.685417\u001b[35m my_accuracy_metric:\u001b[94m 97.685417\u001b[35m val_loss:\u001b[94m 0.059265\u001b[35m val_my_accuracy_func:\u001b[94m 98.291667\u001b[35m val_my_accuracy_metric:\u001b[94m 98.291667\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.88s \u001b[35mtest_loss:\u001b[94m 0.045160\u001b[35m test_my_accuracy_func:\u001b[94m 98.530000\u001b[35m test_my_accuracy_metric:\u001b[94m 98.530000\u001b[0m                                                    \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=[my_accuracy_func, MyAccuracyMetric()],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs)\n",
    "\n",
    "# Test\n",
    "test_loss, (test_acc, test_f1) = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Furthermore, you could also use the [`SKLearnMetrics`](https://poutyne.org/metrics.html#poutyne.SKLearnMetrics) wrapper to wrap a Scikit-learn metric as an epoch metric. Below, we show how to compute the AUC ROC using the [`SKLearnMetrics`](https://poutyne.org/metrics.html#poutyne.SKLearnMetrics) class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(x, axis=1):\n",
    "    \"\"\"\n",
    "    Compute softmax function.\n",
    "    \"\"\"\n",
    "    e_x = np.exp(x - x.max(axis=axis, keepdims=True))\n",
    "    return e_x / e_x.sum(axis=axis, keepdims=True)\n",
    "\n",
    "\n",
    "def roc_auc(y_true, y_pred, **kwargs):\n",
    "    \"\"\"\n",
    "    Since the `roc_auc_score` from Scikit-learn requires normalized probabilities,\n",
    "    we use the softmax function on the predictions.\n",
    "    \"\"\"\n",
    "    y_pred = softmax(y_pred)\n",
    "    return roc_auc_score(y_true, y_pred, **kwargs)\n",
    "\n",
    "\n",
    "# kwargs are keyword arguments we wish to pass to roc_auc.\n",
    "roc_epoch_metric = SKLearnMetrics(roc_auc, kwargs=dict(multi_class='ovr', average='macro'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.23s \u001b[35mloss:\u001b[94m 0.368582\u001b[35m acc:\u001b[94m 88.110417\u001b[35m fscore_macro:\u001b[94m 0.879912\u001b[35m roc_auc:\u001b[94m 0.991401\u001b[35m val_loss:\u001b[94m 0.106674\u001b[35m val_acc:\u001b[94m 96.800000\u001b[35m val_fscore_macro:\u001b[94m 0.967872\u001b[35m val_roc_auc:\u001b[94m 0.999144\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.68s \u001b[35mloss:\u001b[94m 0.126682\u001b[35m acc:\u001b[94m 96.135417\u001b[35m fscore_macro:\u001b[94m 0.961056\u001b[35m roc_auc:\u001b[94m 0.998664\u001b[35m val_loss:\u001b[94m 0.061894\u001b[35m val_acc:\u001b[94m 98.258333\u001b[35m val_fscore_macro:\u001b[94m 0.982487\u001b[35m val_roc_auc:\u001b[94m 0.999681\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.72s \u001b[35mloss:\u001b[94m 0.098693\u001b[35m acc:\u001b[94m 97.008333\u001b[35m fscore_macro:\u001b[94m 0.969876\u001b[35m roc_auc:\u001b[94m 0.999182\u001b[35m val_loss:\u001b[94m 0.053538\u001b[35m val_acc:\u001b[94m 98.400000\u001b[35m val_fscore_macro:\u001b[94m 0.983909\u001b[35m val_roc_auc:\u001b[94m 0.999758\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.53s \u001b[35mloss:\u001b[94m 0.084671\u001b[35m acc:\u001b[94m 97.391667\u001b[35m fscore_macro:\u001b[94m 0.973760\u001b[35m roc_auc:\u001b[94m 0.999401\u001b[35m val_loss:\u001b[94m 0.050164\u001b[35m val_acc:\u001b[94m 98.541667\u001b[35m val_fscore_macro:\u001b[94m 0.985300\u001b[35m val_roc_auc:\u001b[94m 0.999795\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.20s \u001b[35mloss:\u001b[94m 0.071751\u001b[35m acc:\u001b[94m 97.822917\u001b[35m fscore_macro:\u001b[94m 0.978080\u001b[35m roc_auc:\u001b[94m 0.999565\u001b[35m val_loss:\u001b[94m 0.045306\u001b[35m val_acc:\u001b[94m 98.616667\u001b[35m val_fscore_macro:\u001b[94m 0.986045\u001b[35m val_roc_auc:\u001b[94m 0.999814\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.98s \u001b[35mtest_loss:\u001b[94m 0.033772\u001b[35m test_acc:\u001b[94m 98.890000\u001b[35m test_fscore_macro:\u001b[94m 0.988862\u001b[35m test_roc_auc:\u001b[94m 0.999912\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    epoch_metrics=['f1', roc_epoch_metric],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs)\n",
    "\n",
    "# Test\n",
    "test_loss, (test_acc, test_f1, test_roc) = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bypassing PyTorch DataLoaders\n",
    "\n",
    "Above, we defined DataLoaders for our datasets. However, with Poutyne, it is not strictly necessary since it provides the [`fit_dataset`](https://poutyne.org/model.html#poutyne.Model.fit_dataset) and [`evaluate_dataset`](https://poutyne.org/model.html#poutyne.Model.evaluate_dataset) methods to which you can pass the necessary parameters such as the batch size. Under the hood, Poutyne initializes the DataLoaders for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.40s \u001b[35mloss:\u001b[94m 0.376303\u001b[35m acc:\u001b[94m 87.952083\u001b[35m val_loss:\u001b[94m 0.084193\u001b[35m val_acc:\u001b[94m 97.575000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.70s \u001b[35mloss:\u001b[94m 0.129315\u001b[35m acc:\u001b[94m 96.097917\u001b[35m val_loss:\u001b[94m 0.074739\u001b[35m val_acc:\u001b[94m 97.750000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.64s \u001b[35mloss:\u001b[94m 0.102616\u001b[35m acc:\u001b[94m 96.868750\u001b[35m val_loss:\u001b[94m 0.056323\u001b[35m val_acc:\u001b[94m 98.491667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.48s \u001b[35mloss:\u001b[94m 0.087765\u001b[35m acc:\u001b[94m 97.391667\u001b[35m val_loss:\u001b[94m 0.052291\u001b[35m val_acc:\u001b[94m 98.533333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.55s \u001b[35mloss:\u001b[94m 0.075437\u001b[35m acc:\u001b[94m 97.672917\u001b[35m val_loss:\u001b[94m 0.046242\u001b[35m val_acc:\u001b[94m 98.791667\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.85s \u001b[35mtest_loss:\u001b[94m 0.033271\u001b[35m test_acc:\u001b[94m 98.920000\u001b[0m                                                   \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_dataset(\n",
    "    train_dataset,\n",
    "    valid_dataset,\n",
    "    epochs=num_epochs,\n",
    "    batch_size=batch_size,\n",
    "    num_workers=2,\n",
    ")\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_dataset(\n",
    "    test_dataset,\n",
    "    batch_size=batch_size,\n",
    "    num_workers=2,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Using Callbacks\n",
    "\n",
    "One nice feature of Poutyne is [callbacks](https://poutyne.org/callbacks.html). Callbacks allow doing actions during the training of the neural network. In the following example, we use three callbacks. The first that saves the latest weights in a file to be able to continue the optimization at the end of training if more epochs are needed. The second that saves the best weights according to the performance on the validation dataset. The last that saves the displayed logs into a TSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Saves everything into saves/convnet_mnist\n",
    "save_path = \"saves/convnet_mnist\"\n",
    "os.makedirs(save_path, exist_ok=True)\n",
    "\n",
    "callbacks = [\n",
    "    # Save the latest weights to be able to continue the optimization at the end for more epochs.\n",
    "    ModelCheckpoint(os.path.join(save_path, 'last_epoch.ckpt')),\n",
    "    # Save the weights in a new file when the current model is better than all previous models.\n",
    "    ModelCheckpoint(\n",
    "        os.path.join(save_path, 'best_epoch_{epoch}.ckpt'),\n",
    "        monitor='val_acc',\n",
    "        mode='max',\n",
    "        save_best_only=True,\n",
    "        restore_best=True,\n",
    "        verbose=True,\n",
    "    ),\n",
    "    # Save the losses and accuracies for each epoch in a TSV.\n",
    "    CSVLogger(os.path.join(save_path, 'log.tsv'), separator='\\t'),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.80s \u001b[35mloss:\u001b[94m 0.379219\u001b[35m acc:\u001b[94m 87.589583\u001b[35m val_loss:\u001b[94m 0.095315\u001b[35m val_acc:\u001b[94m 97.183333\u001b[0m\n",
      "Epoch 1: val_acc improved from -inf to 97.18333, saving file to saves/convnet_mnist/best_epoch_1.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.60s \u001b[35mloss:\u001b[94m 0.127790\u001b[35m acc:\u001b[94m 96.127083\u001b[35m val_loss:\u001b[94m 0.069420\u001b[35m val_acc:\u001b[94m 97.966667\u001b[0m\n",
      "Epoch 2: val_acc improved from 97.18333 to 97.96667, saving file to saves/convnet_mnist/best_epoch_2.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.76s \u001b[35mloss:\u001b[94m 0.100507\u001b[35m acc:\u001b[94m 96.997917\u001b[35m val_loss:\u001b[94m 0.052325\u001b[35m val_acc:\u001b[94m 98.375000\u001b[0m\n",
      "Epoch 3: val_acc improved from 97.96667 to 98.37500, saving file to saves/convnet_mnist/best_epoch_3.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.53s \u001b[35mloss:\u001b[94m 0.084314\u001b[35m acc:\u001b[94m 97.416667\u001b[35m val_loss:\u001b[94m 0.049132\u001b[35m val_acc:\u001b[94m 98.633333\u001b[0m\n",
      "Epoch 4: val_acc improved from 98.37500 to 98.63333, saving file to saves/convnet_mnist/best_epoch_4.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.49s \u001b[35mloss:\u001b[94m 0.073190\u001b[35m acc:\u001b[94m 97.775000\u001b[35m val_loss:\u001b[94m 0.044298\u001b[35m val_acc:\u001b[94m 98.750000\u001b[0m\n",
      "Epoch 5: val_acc improved from 98.63333 to 98.75000, saving file to saves/convnet_mnist/best_epoch_5.ckpt\n",
      "Restoring data from saves/convnet_mnist/best_epoch_5.ckpt\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.82s \u001b[35mtest_loss:\u001b[94m 0.030420\u001b[35m test_acc:\u001b[94m 99.000000\u001b[0m                                                   \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs, callbacks=callbacks)\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Making Your Own Callback\n",
    "\n",
    "While Poutyne provides a great number of [predefined callbacks](https://poutyne.org/callbacks.html), it is sometimes useful to make your own callback.\n",
    "\n",
    "In the following example, we want to see the effect of temperature on the optimization of our neural network. To do so, we either increase or decrease the temperature during the optimization. As one can see in the result, temperature either as no effect or has a detrimental effect on the performance of the neural network. This is so because the temperature has for effect to artificially changing the learning rates. Since we have found the right learning rate, increasing or decreasing, it shows no improvement on the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CrossEntropyLossWithTemperature(nn.Module):\n",
    "    \"\"\"\n",
    "    This loss module is the cross-entropy loss function\n",
    "    with temperature. It divides the logits by a temperature\n",
    "    value before computing the cross-entropy loss.\n",
    "\n",
    "    Args:\n",
    "        initial_temperature (float): The initial value of the temperature.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, initial_temperature):\n",
    "        super().__init__()\n",
    "        self.temperature = initial_temperature\n",
    "        self.celoss = nn.CrossEntropyLoss()\n",
    "\n",
    "    def forward(self, y_pred, y_true):\n",
    "        y_pred = y_pred / self.temperature\n",
    "        return self.celoss(y_pred, y_true)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class TemperatureCallback(Callback):\n",
    "    \"\"\"\n",
    "    This callback multiply the loss temperature with a decay before\n",
    "    each batch.\n",
    "\n",
    "    Args:\n",
    "        celoss_with_temp (CrossEntropyLossWithTemperature): the loss module.\n",
    "        decay (float): The value of the temperature decay.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, celoss_with_temp, decay):\n",
    "        super().__init__()\n",
    "        self.celoss_with_temp = celoss_with_temp\n",
    "        self.decay = decay\n",
    "\n",
    "    def on_train_batch_begin(self, batch, logs):\n",
    "        self.celoss_with_temp.temperature *= self.decay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "So our loss function will be the cross-entropy with temperature with an initial temperature of `0.1` and a temperature decay of `1.0008`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "custom_loss_function = CrossEntropyLossWithTemperature(0.1)\n",
    "callbacks = [TemperatureCallback(custom_loss_function, 1.0008)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now let's test our training loop for one epoch using the accuracy as the batch metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.74s \u001b[35mloss:\u001b[94m 0.470274\u001b[35m acc:\u001b[94m 85.037500\u001b[35m val_loss:\u001b[94m 0.089246\u001b[35m val_acc:\u001b[94m 97.325000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.62s \u001b[35mloss:\u001b[94m 0.136065\u001b[35m acc:\u001b[94m 96.068750\u001b[35m val_loss:\u001b[94m 0.058898\u001b[35m val_acc:\u001b[94m 98.283333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.82s \u001b[35mloss:\u001b[94m 0.111425\u001b[35m acc:\u001b[94m 96.741667\u001b[35m val_loss:\u001b[94m 0.057114\u001b[35m val_acc:\u001b[94m 98.291667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.71s \u001b[35mloss:\u001b[94m 0.109230\u001b[35m acc:\u001b[94m 96.927083\u001b[35m val_loss:\u001b[94m 0.063533\u001b[35m val_acc:\u001b[94m 98.183333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m6.82s \u001b[35mloss:\u001b[94m 0.123607\u001b[35m acc:\u001b[94m 96.660417\u001b[35m val_loss:\u001b[94m 0.074446\u001b[35m val_acc:\u001b[94m 98.025000\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.87s \u001b[35mtest_loss:\u001b[94m 0.062896\u001b[35m test_acc:\u001b[94m 98.290000\u001b[0m                                                   \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    custom_loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs, callbacks=callbacks)\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using ModelBundle\n",
    "\n",
    "Most of the time, when using Poutyne (or even PyTorch in general), we will find ourselves in an iterative model hyperparameters finetuning loop. For efficient model search, we will usually wish to save our best performing models, their training and testing statistics and even sometimes wish to retrain an already trained model for further tuning. All of the above can be easily implemented with the flexibility of Poutyne Callbacks, but having to define and initialize each and every Callback object we wish for our model quickly feels cumbersome.\n",
    "\n",
    "This is why Poutyne provides a [ModelBundle class](https://poutyne.org/experiment.html#poutyne.ModelBundle), which aims specifically at enabling quick model iteration search, while not sacrificing the quality of a single experiment - statistics logging, best models saving, etc. ModelBundle is actually a simple wrapper between a PyTorch network and Poutyne's core Callback objects for logging and saving. Given a working directory where to output the various logging files and a PyTorch network, the ModelBundle class reduces the whole training loop to a single line.\n",
    "\n",
    "The following code uses [Poutyne's ModelBundle class](https://poutyne.org/experiment.html#poutyne.ModelBundle) to train a network for 5 epochs. The code is quite simpler than the code in the Poutyne Callbacks section while doing more (only a few lines). Once trained for 5 epochs, it is then possible to resume the optimization at the 5th epoch for 5 more epochs until the 10th epoch using the same function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_model_bundle(network, name, epochs=5):\n",
    "    \"\"\"\n",
    "    This function creates a Poutyne ModelBundle, trains the input module\n",
    "    on the train loader and then tests its performance on the test loader.\n",
    "    All training and testing statistics are saved, as well as best model\n",
    "    checkpoints.\n",
    "\n",
    "    Args:\n",
    "        network (torch.nn.Module): The neural network to train.\n",
    "        working_directory (str): The directory where to output files to save.\n",
    "        epochs (int): The number of epochs. (Default: 5)\n",
    "    \"\"\"\n",
    "    # Everything is going to be saved in ./saves/{name}.\n",
    "    save_path = os.path.join('saves', name)\n",
    "\n",
    "    # Poutyne ModelBundle\n",
    "    model_bundle = ModelBundle.from_network(\n",
    "        save_path,\n",
    "        network,\n",
    "        optimizer=optimizer,\n",
    "        task='classif',\n",
    "        device=device,\n",
    "    )\n",
    "\n",
    "    # Train\n",
    "    model_bundle.train(train_loader, valid_loader, epochs=epochs)\n",
    "\n",
    "    # Test\n",
    "    model_bundle.test(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first remove the output directory if it exists so that we start training from scratch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "shutil.rmtree('saves/convnet_mnist_model_bundle', ignore_errors=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train for 5 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.51s \u001b[35mloss:\u001b[94m 0.366255\u001b[35m acc:\u001b[94m 88.012500\u001b[35m fscore_macro:\u001b[94m 0.879123\u001b[35m val_loss:\u001b[94m 0.077714\u001b[35m val_acc:\u001b[94m 97.741667\u001b[35m val_fscore_macro:\u001b[94m 0.977352\u001b[0m\n",
      "Epoch 1: val_acc improved from -inf to 97.74167, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_1.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.45s \u001b[35mloss:\u001b[94m 0.127324\u001b[35m acc:\u001b[94m 96.200000\u001b[35m fscore_macro:\u001b[94m 0.961720\u001b[35m val_loss:\u001b[94m 0.060472\u001b[35m val_acc:\u001b[94m 98.208333\u001b[35m val_fscore_macro:\u001b[94m 0.982050\u001b[0m\n",
      "Epoch 2: val_acc improved from 97.74167 to 98.20833, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_2.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.12s \u001b[35mloss:\u001b[94m 0.098540\u001b[35m acc:\u001b[94m 97.052083\u001b[35m fscore_macro:\u001b[94m 0.970294\u001b[35m val_loss:\u001b[94m 0.053993\u001b[35m val_acc:\u001b[94m 98.358333\u001b[35m val_fscore_macro:\u001b[94m 0.983539\u001b[0m\n",
      "Epoch 3: val_acc improved from 98.20833 to 98.35833, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_3.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.03s \u001b[35mloss:\u001b[94m 0.081231\u001b[35m acc:\u001b[94m 97.552083\u001b[35m fscore_macro:\u001b[94m 0.975337\u001b[35m val_loss:\u001b[94m 0.044749\u001b[35m val_acc:\u001b[94m 98.608333\u001b[35m val_fscore_macro:\u001b[94m 0.986048\u001b[0m\n",
      "Epoch 4: val_acc improved from 98.35833 to 98.60833, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_4.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.39s \u001b[35mloss:\u001b[94m 0.073298\u001b[35m acc:\u001b[94m 97.839583\u001b[35m fscore_macro:\u001b[94m 0.978247\u001b[35m val_loss:\u001b[94m 0.038756\u001b[35m val_acc:\u001b[94m 98.825000\u001b[35m val_fscore_macro:\u001b[94m 0.988171\u001b[0m\n",
      "Epoch 5: val_acc improved from 98.60833 to 98.82500, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_5.ckpt\n",
      "Restoring data from saves/convnet_mnist_model_bundle/checkpoint_epoch_5.ckpt\n",
      "Found best checkpoint at epoch: 5\n",
      "lr: 0.1, loss: 0.0732985, acc: 97.8396, fscore_macro: 0.978247, val_loss: 0.038756, val_acc: 98.825, val_fscore_macro: 0.988171\n",
      "Loading checkpoint saves/convnet_mnist_model_bundle/checkpoint_epoch_5.ckpt\n",
      "Running test\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m1.11s \u001b[35mtest_loss:\u001b[94m 0.030811\u001b[35m test_acc:\u001b[94m 98.980000\u001b[35m test_fscore_macro:\u001b[94m 0.989685\u001b[0m             \n"
     ]
    }
   ],
   "source": [
    "network = create_network()\n",
    "train_model_bundle(network, 'convnet_mnist_model_bundle', epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how setting `task='classif'` when instantiating `ModelBundle` adds for use our loss function, the batch metric accuracy, the epoch metric F1 and set up callbacks that use them. If you wish, you still can use your own loss function and metrics instead of passing this argument.\n",
    "\n",
    "We have trained for 5 epochs, let's now resume training for another 5 epochs for a total of 10 epochs. Notice that we reinstantiate the network. ModelBundle will load back the weights for us and resume training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading weights from saves/convnet_mnist_model_bundle/checkpoint.ckpt and starting at epoch 6.\n",
      "Loading optimizer state from saves/convnet_mnist_model_bundle/checkpoint.optim and starting at epoch 6.\n",
      "Loading random states from saves/convnet_mnist_model_bundle/checkpoint.randomstate and starting at epoch 6.\n",
      "\u001b[35mEpoch: \u001b[36m 6/10 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.16s \u001b[35mloss:\u001b[94m 0.065763\u001b[35m acc:\u001b[94m 98.022917\u001b[35m fscore_macro:\u001b[94m 0.980102\u001b[35m val_loss:\u001b[94m 0.037387\u001b[35m val_acc:\u001b[94m 98.941667\u001b[35m val_fscore_macro:\u001b[94m 0.989383\u001b[0m\n",
      "Epoch 6: val_acc improved from 98.82500 to 98.94167, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_6.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m 7/10 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.41s \u001b[35mloss:\u001b[94m 0.062799\u001b[35m acc:\u001b[94m 98.081250\u001b[35m fscore_macro:\u001b[94m 0.980670\u001b[35m val_loss:\u001b[94m 0.036032\u001b[35m val_acc:\u001b[94m 98.958333\u001b[35m val_fscore_macro:\u001b[94m 0.989533\u001b[0m\n",
      "Epoch 7: val_acc improved from 98.94167 to 98.95833, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_7.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m 8/10 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.29s \u001b[35mloss:\u001b[94m 0.059569\u001b[35m acc:\u001b[94m 98.181250\u001b[35m fscore_macro:\u001b[94m 0.981675\u001b[35m val_loss:\u001b[94m 0.035780\u001b[35m val_acc:\u001b[94m 99.016667\u001b[35m val_fscore_macro:\u001b[94m 0.990083\u001b[0m\n",
      "Epoch 8: val_acc improved from 98.95833 to 99.01667, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_8.ckpt\n",
      "\u001b[35mEpoch: \u001b[36m 9/10 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.27s \u001b[35mloss:\u001b[94m 0.054568\u001b[35m acc:\u001b[94m 98.297917\u001b[35m fscore_macro:\u001b[94m 0.982850\u001b[35m val_loss:\u001b[94m 0.036158\u001b[35m val_acc:\u001b[94m 98.966667\u001b[35m val_fscore_macro:\u001b[94m 0.989618\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m10/10 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.25s \u001b[35mloss:\u001b[94m 0.049007\u001b[35m acc:\u001b[94m 98.520833\u001b[35m fscore_macro:\u001b[94m 0.985098\u001b[35m val_loss:\u001b[94m 0.035492\u001b[35m val_acc:\u001b[94m 99.041667\u001b[35m val_fscore_macro:\u001b[94m 0.990369\u001b[0m\n",
      "Epoch 10: val_acc improved from 99.01667 to 99.04167, saving file to saves/convnet_mnist_model_bundle/checkpoint_epoch_10.ckpt\n",
      "Restoring data from saves/convnet_mnist_model_bundle/checkpoint_epoch_10.ckpt\n",
      "Found best checkpoint at epoch: 10\n",
      "lr: 0.1, loss: 0.0490065, acc: 98.5208, fscore_macro: 0.985098, val_loss: 0.0354915, val_acc: 99.0417, val_fscore_macro: 0.990369\n",
      "Loading checkpoint saves/convnet_mnist_model_bundle/checkpoint_epoch_10.ckpt\n",
      "Running test\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.96s \u001b[35mtest_loss:\u001b[94m 0.026244\u001b[35m test_acc:\u001b[94m 99.170000\u001b[35m test_fscore_macro:\u001b[94m 0.991575\u001b[0m             \n"
     ]
    }
   ],
   "source": [
    "network = create_network()\n",
    "train_model_bundle(network, 'convnet_mnist_model_bundle', epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Coloring\n",
    "\n",
    "Also, Poutyne use by default a coloring template of the training step when the package `colorama` is installed.\n",
    "One could either remove the coloring (`progress_options=dict(coloring=False)`) or set a different coloring template using the fields:\n",
    "`text_color`, `ratio_color`, `metric_value_color`, `time_color` and `progress_bar_color`.\n",
    "If a field is not specified, the default color will be used.\n",
    "[See available colors in colorama's source code](https://github.com/tartley/colorama/blob/9946cfb/colorama/ansi.py#L49).\n",
    "\n",
    "Here an example where we set the `text_color` to RED and the `progress_bar_color` to LIGHTGREEN_EX."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "progress_options = dict(coloring=dict(text_color=\"RED\", progress_bar_color=\"LIGHTGREEN_EX\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mEpoch: \u001b[36m1/5 \u001b[31mTrain steps: \u001b[36m1500 \u001b[31mVal steps: \u001b[36m375 \u001b[32m6.54s \u001b[31mloss:\u001b[94m 0.370519\u001b[31m acc:\u001b[94m 88.125000\u001b[31m val_loss:\u001b[94m 0.089264\u001b[31m val_acc:\u001b[94m 97.366667\u001b[0m\n",
      "\u001b[31mEpoch: \u001b[36m2/5 \u001b[31mTrain steps: \u001b[36m1500 \u001b[31mVal steps: \u001b[36m375 \u001b[32m6.41s \u001b[31mloss:\u001b[94m 0.129163\u001b[31m acc:\u001b[94m 96.087500\u001b[31m val_loss:\u001b[94m 0.065395\u001b[31m val_acc:\u001b[94m 98.075000\u001b[0m\n",
      "\u001b[31mEpoch: \u001b[36m3/5 \u001b[31mTrain steps: \u001b[36m1500 \u001b[31mVal steps: \u001b[36m375 \u001b[32m6.27s \u001b[31mloss:\u001b[94m 0.098094\u001b[31m acc:\u001b[94m 97.097917\u001b[31m val_loss:\u001b[94m 0.058571\u001b[31m val_acc:\u001b[94m 98.341667\u001b[0m\n",
      "\u001b[31mEpoch: \u001b[36m4/5 \u001b[31mTrain steps: \u001b[36m1500 \u001b[31mVal steps: \u001b[36m375 \u001b[32m6.58s \u001b[31mloss:\u001b[94m 0.085683\u001b[31m acc:\u001b[94m 97.452083\u001b[31m val_loss:\u001b[94m 0.049200\u001b[31m val_acc:\u001b[94m 98.658333\u001b[0m\n",
      "\u001b[31mEpoch: \u001b[36m5/5 \u001b[31mTrain steps: \u001b[36m1500 \u001b[31mVal steps: \u001b[36m375 \u001b[32m6.62s \u001b[31mloss:\u001b[94m 0.074192\u001b[31m acc:\u001b[94m 97.768750\u001b[31m val_loss:\u001b[94m 0.046122\u001b[31m val_acc:\u001b[94m 98.675000\u001b[0m\n",
      "\u001b[31mTest steps: \u001b[36m313 \u001b[32m0.84s \u001b[31mtest_loss:\u001b[94m 0.032857\u001b[31m test_acc:\u001b[94m 98.890000\u001b[0m                                                   \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(\n",
    "    train_loader,\n",
    "    valid_loader,\n",
    "    epochs=num_epochs,\n",
    "    progress_options=progress_options,\n",
    ")\n",
    "\n",
    "# Test\n",
    "test_loss, test_acc = model.evaluate_generator(\n",
    "    test_loader,\n",
    "    progress_options=progress_options,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Metric Names\n",
    "\n",
    "It's also possible to name the metric using a tuple format `(<metric name>, metric)`. That way, it's possible to use multiple times the same metric type (i.e. having micro and macro F1-score)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m7.92s \u001b[35mloss:\u001b[94m 0.355768\u001b[35m My accuracy name:\u001b[94m 88.662500\u001b[35m My f1 name:\u001b[94m 0.885614\u001b[35m val_loss:\u001b[94m 0.090990\u001b[35m val_My accuracy name:\u001b[94m 97.208333\u001b[35m val_My f1 name:\u001b[94m 0.971917\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.21s \u001b[35mloss:\u001b[94m 0.123316\u001b[35m My accuracy name:\u001b[94m 96.358333\u001b[35m My f1 name:\u001b[94m 0.963292\u001b[35m val_loss:\u001b[94m 0.064429\u001b[35m val_My accuracy name:\u001b[94m 98.050000\u001b[35m val_My f1 name:\u001b[94m 0.980377\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.02s \u001b[35mloss:\u001b[94m 0.098334\u001b[35m My accuracy name:\u001b[94m 97.056250\u001b[35m My f1 name:\u001b[94m 0.970343\u001b[35m val_loss:\u001b[94m 0.053564\u001b[35m val_My accuracy name:\u001b[94m 98.408333\u001b[35m val_My f1 name:\u001b[94m 0.983988\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m8.30s \u001b[35mloss:\u001b[94m 0.084703\u001b[35m My accuracy name:\u001b[94m 97.435417\u001b[35m My f1 name:\u001b[94m 0.974177\u001b[35m val_loss:\u001b[94m 0.049505\u001b[35m val_My accuracy name:\u001b[94m 98.591667\u001b[35m val_My f1 name:\u001b[94m 0.985820\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m7.98s \u001b[35mloss:\u001b[94m 0.073237\u001b[35m My accuracy name:\u001b[94m 97.772917\u001b[35m My f1 name:\u001b[94m 0.977587\u001b[35m val_loss:\u001b[94m 0.044072\u001b[35m val_My accuracy name:\u001b[94m 98.725000\u001b[35m val_My f1 name:\u001b[94m 0.987223\u001b[0m\n",
      "\u001b[35mTest steps: \u001b[36m313 \u001b[32m0.96s \u001b[35mtest_loss:\u001b[94m 0.029208\u001b[35m test_My accuracy name:\u001b[94m 99.000000\u001b[35m test_My f1 name:\u001b[94m 0.989950\u001b[0m               \n"
     ]
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=[(\"My accuracy name\", 'accuracy')],\n",
    "    epoch_metrics=[(\"My f1 name\", 'f1')],\n",
    "    device=device,\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs)\n",
    "\n",
    "# Test\n",
    "test_loss, (test_acc, test_f1) = model.evaluate_generator(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-GPUs\n",
    "\n",
    "Finally, it's also possible to use multi-GPUs for your training either by specifying a list of devices or using the arg `\"all\"` to take them all.\n",
    "\n",
    "> Obviously, you need more than one GPUs for that option.\n",
    "\n",
    "In our case here, multi-gpus takes more time because the task is not big enough to profit from multi-gpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35mEpoch: \u001b[36m1/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m17.00s \u001b[35mloss:\u001b[94m 0.386154\u001b[35m acc:\u001b[94m 87.333333\u001b[35m val_loss:\u001b[94m 0.092202\u001b[35m val_acc:\u001b[94m 97.191667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m2/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m14.81s \u001b[35mloss:\u001b[94m 0.125460\u001b[35m acc:\u001b[94m 96.172917\u001b[35m val_loss:\u001b[94m 0.062402\u001b[35m val_acc:\u001b[94m 98.116667\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m3/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m14.63s \u001b[35mloss:\u001b[94m 0.097736\u001b[35m acc:\u001b[94m 96.977083\u001b[35m val_loss:\u001b[94m 0.056723\u001b[35m val_acc:\u001b[94m 98.383333\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m4/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m14.74s \u001b[35mloss:\u001b[94m 0.083548\u001b[35m acc:\u001b[94m 97.508333\u001b[35m val_loss:\u001b[94m 0.045911\u001b[35m val_acc:\u001b[94m 98.725000\u001b[0m\n",
      "\u001b[35mEpoch: \u001b[36m5/5 \u001b[35mTrain steps: \u001b[36m1500 \u001b[35mVal steps: \u001b[36m375 \u001b[32m14.80s \u001b[35mloss:\u001b[94m 0.075282\u001b[35m acc:\u001b[94m 97.731250\u001b[35m val_loss:\u001b[94m 0.045916\u001b[35m val_acc:\u001b[94m 98.650000\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'epoch': 1,\n",
       "  'time': 17.00380860781297,\n",
       "  'loss': 0.3861539592298989,\n",
       "  'acc': 87.33333333333333,\n",
       "  'val_loss': 0.09220197118197879,\n",
       "  'val_acc': 97.19166666666666},\n",
       " {'epoch': 2,\n",
       "  'time': 14.805061359889805,\n",
       "  'loss': 0.12545990319394817,\n",
       "  'acc': 96.17291666666667,\n",
       "  'val_loss': 0.06240178443475937,\n",
       "  'val_acc': 98.11666666666666},\n",
       " {'epoch': 3,\n",
       "  'time': 14.63108725193888,\n",
       "  'loss': 0.09773606853660506,\n",
       "  'acc': 96.97708333333334,\n",
       "  'val_loss': 0.0567228243414623,\n",
       "  'val_acc': 98.38333333333334},\n",
       " {'epoch': 4,\n",
       "  'time': 14.74231273215264,\n",
       "  'loss': 0.08354751471034251,\n",
       "  'acc': 97.50833333333334,\n",
       "  'val_loss': 0.04591141694528051,\n",
       "  'val_acc': 98.725},\n",
       " {'epoch': 5,\n",
       "  'time': 14.79857192421332,\n",
       "  'loss': 0.0752823881568232,\n",
       "  'acc': 97.73125,\n",
       "  'val_loss': 0.04591631960236312,\n",
       "  'val_acc': 98.65}]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Instantiating our network\n",
    "network = create_network()\n",
    "\n",
    "# Poutyne Model on GPU\n",
    "model = Model(\n",
    "    network,\n",
    "    optimizer,\n",
    "    loss_function,\n",
    "    batch_metrics=['accuracy'],\n",
    "    device=\"all\",\n",
    ")\n",
    "\n",
    "# Train\n",
    "model.fit_generator(train_loader, valid_loader, epochs=num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}