Classification_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Zafx6Y6AflAu"
   },
   "source": [
    "# Deep learning programming II: Classification\n",
    "Felix Wiewel, Institute of Signal Processing and System Theory, University of Stuttgart, 24.04.2020\n",
    "\n",
    "## Introduction\n",
    "A common problem that arises in machine learning is the classifiaction of images. The goal of this exercise is to introduce you to the practical implementation of a complete pipeline for solving a classification task with neural networks in Tensorflow. For this we will make extended use of Keras, a specification of a high-level API for implementing powerful classes and functions to create and train neural networks. Tensorflow already comes with an implementation of the Keras specification so there is no need to install Keras if you have already installed Tensorflow. Usign Keras instead of low level tensorflow code to implement your neural networks and their training algorithms comes with some advantages but also some disadvantages. Since Keras makes it very easy to build and train neural networks, even people without strong programming skills can quickly develop solutions for given problems using neural networks. Keras essantially provides an abstract interface to the user and takes care of the low-level implementation details. This, however, comes with a cost. Since many technical details are hidden from the user, it is sometimes quite difficult to modify and change certain parts of your model/training algorithm. To a certain extend Keras provides you with ways to implement custom network architectures, custom layers and custom training algorithms but sometimes having to write code that is compliant with the Keras specification is more difficult than implementing it in low-level Tensorflow code. But for most standard applications, e.g. simple image classification like in this exercise, Keras is flexible enough and can save you a lot of time and frustation.\n",
    "\n",
    "## GPU support\n",
    "In order to speed up calculations with Tensorflow, we need to change the runtime type of this notebook to GPU. For this click on \"Runtime\" in the top left menu and select \"Change runtime type\". Then choose \"GPU\" in the drop down list under \"Hardware accelerator\". This will enable Tensorflow to execute calculations on a GPU provided by Google Colab.\n",
    "\n",
    "## Mathematical Background\n",
    "In classification we are interested in learning a function that maps an input $\\mathbf{x}\\in\\mathbb{R}^{M}$ to one out of possibly many classes. In order to characterize such a mapping, we need a mathematical expression to uniquely identify different classes. This is typically done by assigning each class an integer value. So in the case of classifying images in the three classes Dog, Cat, Bird one could assign these classes with the one-hot vectors $\\left[1,0,0\\right]^{T},\\left[0,1,0\\right]^{T},\\left[0,0,1\\right]^{T}$. Using one-hot vectors to represent labels of indivudual classes is very useful for training neural networks. It closely matches the mathematical model that we will be using for interpreting the networks output where each of its elements describes a probability. In classification using neural networks we are interested in learning the function\n",
    "\n",
    "$\\mathbf{y}=f_{\\boldsymbol{\\theta}}(\\mathbf{x})$,\n",
    "\n",
    "where $f$ is a neural network with parameters $\\boldsymbol{\\theta}$ that maps our possibly high-dimensional input $\\mathbf{x}\\in\\mathbb{R}^{M}$ to an output $\\mathbf{y}\\in\\lbrace0,1\\rbrace^{k}$ that represents one out of $k$ possible classes. Similar to the regression exercise, we will use a probabilistic view on this problem in order to derive a suitable cost function we can use to train our model. This time however, we will not introduce uncertainty through assuming some sort of noise acting on the prediction of the model as we did in the previous exercise. Instead we will treat both $\\mathbf{x}$ and $\\mathbf{y}$ as random variables with probability distribution functions (pdfs) $p(\\mathbf{x})$ and $p(\\mathbf{y})$. We already know that $\\mathbf{y}$ can be one  out of $k$ classes, i.e. $p(\\mathbf{y})$ is a cateorical distribution. Instead of trying to learn a deterministic function that maps an input $\\mathbf{x}$ to an output $\\mathbf{y}$ we can instead learn a function that takes $\\mathbf{x}$ as an input and outputs a pdf over our $k$ different classes, i.e. $p(\\mathbf{y}\\vert\\mathbf{x})$. In order to then classify an input we typically assign it with the class label that is most likely. In this way we can easily interprete the output of the model and also incorporate prior information into our decision. Using this probabilistic perspective on the classification problem we are interested in learning the conditional distribution\n",
    "\n",
    "$p(\\mathbf{y}\\vert\\mathbf{x})=\\prod_{n=1}^{k}\\mu_{n}(\\mathbf{x})^{y_{n}}$,\n",
    "\n",
    "where $y_{n}\\in\\lbrace0,1\\rbrace$,  $\\sum_{n=1}^{k}y_{n}=1$, $0\\leq\\mu_{n}(\\mathbf{x})\\leq1$ and $\\sum_{n=1}^{k}\\mu_{n}(\\mathbf{x})=1$. In other words, we want to learn the parameters of a neural network $\\boldsymbol{\\theta}$ that, given an input $\\mathbf{x}$, outputs $\\mu_{n}(\\mathbf{x})$ for $1\\leq n\\leq k$ satisfying the constraints $0\\leq\\mu_{n}(\\mathbf{x})\\leq1$ and $\\sum_{n=1}^{k}\\mu_{n}(\\mathbf{x})=1$. Practically, we can realize this by designing a neural network that accepts $\\mathbf{x}$ as an input and has an output layer with $k$ neurons and a softmax activation function. In order to derive a cost function for learning the parameters $\\boldsymbol{\\theta}$ of the neural network, we will again use the log likelihood\n",
    "\n",
    "$\\mathcal{L}(\\mathbf{x},\\mathbf{y},\\boldsymbol{\\theta})=\\ln{p(\\mathbf{y}\\vert\\mathbf{x},\\boldsymbol{\\theta})}=\\ln{\\prod_{n=1}^{k}\\mu_{n}(\\mathbf{x})^{y_{n}}}=\\sum_{n=1}^{k}y_{n}\\ln{\\mu_{n}(\\mathbf{x})}$,\n",
    "\n",
    "where $\\mu_{n}(\\mathbf{x})$ depends on the parameters $\\boldsymbol{\\theta}$ of the neural network. The log likelihood measures how likely $\\mathbf{y}$ is given $\\mathbf{x}$ and the parameters $\\boldsymbol{\\theta}$. Maximizing the expected log likelihood over the complete data set yields the optimal parameters\n",
    "\n",
    "$\\boldsymbol{\\theta}^{\\star}=\\arg\\max_{\\boldsymbol{\\theta}}\\mathbb{E}\\left[\\mathcal{L}(\\mathbf{x},\\mathbf{y},\\boldsymbol{\\theta})\\right]=\\arg\\min_{\\boldsymbol{\\theta}}-\\mathbb{E}\\left[\\sum_{n=1}^{k}y_{n}\\ln{\\mu_{n}(\\mathbf{x})}\\right]\\approx\\arg\\min_{\\boldsymbol{\\theta}}-\\dfrac{1}{N}\\sum_{i=1}^{N}\\sum_{n=1}^{k}y_{n,i}\\ln{\\mu_{n}(\\mathbf{x}_{i})}$,\n",
    "\n",
    "where we have reformulated the maximization as a minimization of the negative objective function and approximated the expectation with a sum. As you probably already noticed, the last term is the cross entropy loss that you are familiar with from the lecture. We can now again use SGD or some variant of it in order to minimize the cost function and obtain find some good parameters for our neural network. But keep in mind that while $\\boldsymbol{\\theta}^{\\star}$ are the globally optimal parameters, we are usually only able to find locally optimal parameters due to the non-convex cost function that we are minimizing.\n",
    "\n",
    "## Data set\n",
    "In this exercise we will use common data sets like the MNIST or FashionMNIST data sets. Those and some other data sets are provided through simple functions in the Keras implementation of Tensorflow. For an overview of available data sets and how to use them click [here](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/datasets). The MNIST data set is a simple and very popular data set for handwritten digit classification. It contains $70000$ images with $28\\times 28$ pixels that are split into $60000$ training and $10000$ test samples. Since the task is to classify a handwritten digit, the labels $y$ for each image are from the set $\\lbrace0,\\ldots,9\\rbrace$. The FashionMNIST data set is very similar to MNIST it also has approximately the same number of images with exactly the same dimensions as MNIST. It was provided by Zalando research and contains $10$ classes of different fashion objects.\n",
    "\n",
    "## Implementation\n",
    "We can simply import all required packages and load some data set using Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9Xk5sOe1kLgo"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow.keras as k\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Define constants\n",
    "batch_size = 128\n",
    "epochs = 20\n",
    "learning_rate = 0.001\n",
    "\n",
    "(x_train_mnist, y_train_mnist), (x_test_mnist, y_test_mnist) = tf.keras.datasets.mnist.load_data()\n",
    "x_train_mnist = np.expand_dims(x_train_mnist, axis=-1).astype(np.float32)\n",
    "x_test_mnist = np.expand_dims(x_test_mnist, axis=-1).astype(np.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jrs0EEdOknCB"
   },
   "source": [
    "We can also plot some examples in order to get an impresion of what the data looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 269
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 6508,
     "status": "ok",
     "timestamp": 1562670649545,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "iqN-BjZpkxb4",
    "outputId": "b7254be0-aa98-4a74-ac22-767ca62a73dc"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOx9d1hU19b+e6bAzNCHDoIEEBARuWoAlagYIXI1dmJMYktiSYxBYi+I5cYSQxI1it0YrxpFIkgQsCBIQBFFlCAIBARCkzogZZiyfn/4Yz6NokzxS+53eZ9nPYnMnHX27HPW3muvvda7GSJCD3rQg78fWH91A3rQgx48Hz3G2YMe/E3RY5w96MHfFD3G2YMe/E3RY5w96MHfFJwXfcgwTE8otwc9eMUgIuZ5f++ZOXvQg78peoyzBz34m0IjxskwDBiGgba2NvT09KCvrw8ejwczMzOYm5uDx+OprNfR0RGbN2/GvXv3sGzZMpiZmWmiyc9AS0sLRkZGYJjnehhKQ1tbG2ZmZjAxMQGbzdaIzlcBhmHA4/HA5/PV0sNisbBp0yb8/vvvWLRoEbS1tTXSPjabDS6XCyMjI1haWsLCwkIhRkZGaus2MDDQ2DN/GbS0tBAaGop9+/Z17wIi6lIA0PNEW1ubDAwMyMnJiTw9PWn69Ok0Y8YM2rx5M0VGRtJPP/1Ea9eupaqqKioqKqKAgIDn6nmRMAxDrq6udO7cOero6KD29nZqaWmhOXPmEJvNfuG1XC6XzM3NSUtLq9v3mzBhAp06dYqMjY2VbuufRSAQ0Oeff0719fV0584dsrOzU1unpkVXV5e8vb1p2rRp9NVXX9E333xDNjY2KuszNDSk1NRUkslkdOjQITI0NFSrfVwul+zs7Gj69Om0ZMkSunnzJsnlcpLJZAq5du0aubq6EovFUlo/i8UiX19fio+PJxMTk1fe33w+n+bPn08lJSW0fv36pz7ryv5eGBB6HgwMDPDpp5+ib9++eO2112BsbAxHR0fF7CCTyXDgwAHo6enh1KlTyM3NxY0bN5S6B5vNhoODAzZv3gx/f3+wWCzcu3cPDQ0N+Pjjj1FXV4dz5851eb2pqSmmTZuGn3/+GSUlJd26p6urK5qbm5VqZ1ewtbXF1KlToaenB3XSI/X19fH6669j4MCBsLCwQHZ2Nn777TcYGxsjPT0djY2NSuvk8/nw8vJCYGAg3nzzTRgbG0Mmk4HP56OmpgbffPMNxGKx0nqbm5uRmZkJb29vpa99Hvr27YuQkBCMHDkSPB4PAoHgyUkDADBo0CCsXbsWixYtQl1dnVL6WSwW/Pz8MHz4cFhZWaG2tlYj7e4K1tbWmDdvHuRyOaKiorp1jdLGKZfL4ePjAz8/P7BYLDAMA4lEgtu3b8PExASGhobYtGkT2tvbIZPJ0NHRofTDDgwMxPLly+Hq6qowej6fj/j4eHz44YeYNGnSC43TxsYGfn5+iIuL69b9DAwMMGHCBCQkJEAkEinV1j+Dx+MhICAA/fr1w++//46goCBUV1crrcfIyAjBwcF47733YGBggAcPHmD8+PFgGAZisRirV69GdHS0UjodHBywevVq+Pr6wtraGmKxGHv27EFKSgrOnDkDd3d36OjoqGScXC4Xpqamin+r6yoyDAMbGxvo6OigsLAQ169fB/DYNRw6dCjs7e3BYrFgb28PgUCgtHF2otPwNQULCwvMnj0bP/zwA6qqqgD8z0BgYmKCPXv24N69e91Tpopb6+XlRatWraJ9+/ZRW1sbRUVFkUAgIAsLC/rkk09e6nZ2JQzDkIuLC128eJEkEgnl5ubSpk2b6Nq1a3Ts2DEaOHAgXbp0iWJiYkhbW7tLPaNHj6arV6+Si4tLt+4bHBxMtbW1NHXqVLXdl/fff5/q6uqoubmZ5s6dS/9/O0opMTExoZCQEGpoaKDY2Fjy8fEhQ0NDWr9+PTU1NVFeXh55enoq7cZt3bqVJBIJtbe3U1VVFX355Zekq6tLLi4u1NbWRhERESq7eHp6ehQfH08ymYxSU1Np9OjRKv32J9trYmJCvXr1IqFQSGw2m9hsNmlra9PChQupubmZ2tvbKTQ0VKX3jcPh0ObNm0kul9Nbb72l9nPvlNmzZ1NDQwP5+/sr/qavr0+FhYUUExND1tbWz1yjMbcWANLT03H79m0MGzYMo0aNwr1799Da2orW1laEh4erohLA4xlvxYoV8Pb2RlVVFcLCwhAbG4uEhARkZWXh0aNHqKysxNixY+Hr64v4+PhndHC5XAwZMgQ6OjrduiebzYa5uTkYhkFmZqbKbQcej+pz586Fnp4eEhISEB0drbRba2dnh2XLluGDDz5AVFQUvvzyS+Tn58PDwwMTJkwAj8dDREQEsrKylG5fW1sbEhMTkZubi4yMDERFRaGlpQV2dnZqz3TNzc3YtWsX/Pz84OXlhTFjxiApKQlSqVQlfXK5/Lmu5oABAzB69GjweDy0t7fj6tWrkMlkKt1DIpFALpfD3d0dCQkJKul4Erq6unB2doZYLEZTUxOAx7NmQEAATExMUFpaipaWlu4rVGXm7BRzc3M6cuQI3bt3T+2gB5fLpR07dlB1dTUdP36cpkyZ8lydx44do/b2dvryyy+fq4fP59PZs2cpKSmpW21yc3OjtLQ0KiwsVCuIIRAIKCgoiFpbW6mgoIDefvttpUd0Gxsb+uGHH0gkEtGpU6eod+/exDAMCQQCCg4OptbWVoqJiSEzMzOV2qivr0+9e/cmgUDw1N8XL15MYrFYrZkTAAmFQkXQ5uuvvyYOh6OxGal37960Zs0aunbtGjU2NpJUKqWcnBzi8/kq6WMYhubMmUP19fVdvkvKytChQ6moqIj27t1L+vr6BIB8fHwoIyOD0tLSyMHB4bnXaXTm7ER1dTU2bNiAqKgoHD16FElJSTh8+DDKysogl8uV0jVgwADMnDkTycnJWLVqFf74448uv9u5bdPVZwKBACUlJYrRi2EYRUi+8796enp48803MWPGDAwaNAhr1qxReb3JMAzc3d0xZ84ciMViHD58GJcuXVJqRGcYBkOGDMHEiRNx9epVLFy4EB0dHXB2dsb27dvxxhtv4N69ewgJCcHDhw9VamdTUxMePXoEgUCA+fPnQygUAgB8fX3x6NEjpKSkoL6+XiXdndBUfbCJiQnGjh2LcePGgcfjwcbGBk5OTtDS0kJHRwd++OEH7NmzB21tbSq3s6WlBRKJRG2vgcvlYuDAgdi3bx+amppw7NgxPHr0CM7OzggJCYGjoyPmz5+P33//XSm9ahknADx48ACfffYZtmzZgi+++AL/+Mc/sHnzZty8ebPbLo2uri5WrFgBmUyG6OholJaWdvldFosFIkJra2uX35HL5bC1tcWIESPQ0NAAY2Nj2NjYoG/fvjA2NoahoaEi0PDw4UPI5XKkpaWp/GI5OzvjX//6F1xdXXHmzBns2LFD6ZeGYRjo6OhAIBAgIyMD7777Lt5++230798fBgYG4HK5uHjxIu7fv69SGxmGgbW1NUaNGoXx48fjzTffhFQqVeztpqenIzY2VulB9XlQ10C1tbUxY8YMbNiw4bnLk+bmZly+fBl37txR6z6dsLe3V/lagUCAadOmYdu2bRAKhXj48CFWrlyJ/fv3w9PTEx4eHvj+++9x5swZpXWrbZwAkJqaivnz52P69OmYP38+tmzZgo0bN+LKlSvdur53795wcXFBeXk5UlJSuvyenZ0dLC0tUVBQ0GW0tqOjA6dPn8asWbMQHBwMmUwGHo8HkUiEvLw8XLhwARUVFSgtLUVFRQUWLVqEtrY2VFRUqPTbORwOZs2ahZEjRwIAzp8//8KBoyvI5XLcuXMHGRkZCAkJQXl5OVJTU7Fr1y5MnToVFhYWSEtLU2mmYBgGTk5O2LZtG5ydnZGSkoJVq1bB0tISQUFB0NXVhaOjI2bPno3vvvtO5chn570AwMrKSuUZSSqVIj09HREREYpYRqeuN954A56ennj99dcRHR2tUl93oqGhAa2trejXr59K13O5XEyZMgVr164Fl8vF7t27YWZmhrFjx+L111+Hnp4eWCwW9PT0YGBggIaGBuVuoM6a80lhsVhkYGBAGRkZ1NHRQQcPHuz2Gs7NzY1yc3MpISGhy3WapaUlHTx4kEpLS2nBggUv3HjW0tIiW1tbcnR0VIiVlRXp6uo+9T1zc3NKTk6mK1euqLyO69evH92/f58kEgllZGSolcTAZrPJzs6OBg0aRC4uLmRsbEwuLi5UVlZGmZmZ5OHhobTOzs32lJQUqquro7Vr15KpqSnZ2dlRYmIiNTc305kzZyg1NZVqamooIiKCfH19qV+/fkr/FqFQSFKplGQyGdXV1ZGjo6PKfcFisUgoFJKuri7x+XyFBAQEkFQqpbCwsGfWzqo8u6ysLMrPzycDAwOlrzc2NqaLFy/SyZMnycfHh/T09MjJyYl++uknEovFJBaLqb29nWJjY4nH43Wpp0v7U9c4uVwu2dra0ty5cykyMpJEIhE1NzfT8uXLux1Kd3Nzo3v37tG2bdue+UxHR4f8/f0pLi6OUlJSyNPTU6WMkOeJpaUlpaen05dffvnCznuRpKSkkEQioby8PBo2bJhG2vWkuLi4UF1dHZ09e1bpl5HFYtHIkSMpMzOT7t+/T1OnTiU7Ozvy9/enlJQUysnJoeDgYDIwMCALCwuaOXMmJSQkUGVlJeXn59OUKVOUup9AIKCoqCiSSqUkkUho1apVKv1mHR2d5747+vr6tHHjRo0ZZ+/evSklJYUePHhATk5OSl8vEAjI39+f9PT0FH9zdHSkpKQkKisroxEjRigG2Rfp0XhAiMViwdLSEqNHj8asWbPw+uuvQyAQoL6+HvHx8UhNTe322qMzmWH06NFwc3NDWVkZTE1N0b9/f3h7e2PSpEmor69HSEiI0tlGLwPDMODz+WCxlE8zdnFxga2trWLN1u3NZSVQV1enWAcqux7U09NDSEgIXF1dcfLkSdjY2OC9996Dj48PcnJy8OGHHyIrKwtisRgikQg//vgjfv31V4wYMQK1tbW4dOmSUvfr6OhAcnIy/P39oaWlBVdXV8WWR3fb6+XlhWHDhmH79u0Kl5XFYsHU1BSffvopgoODQUQqb588iZKSEly9ehXvv/++IjimDFpbW3HhwoWn2h8YGIg+ffrg8OHDSE5OVqt9Khmnrq4uBg8ejKCgIHh7e8PU1BStra2IiorCv//9b6SmpioVUex86VxcXLBv3z7k5+fDwcEBTk5OkMlkuHbtGrZu3Yrc3FxVmvtS6OvrK52crqOjgzlz5sDU1BS1tbW4evWqxtL/noSbmxu4XK5K12ppacHJyQlsNhuTJk3CP//5T2RlZWHXrl2Ijo7G3bt3n7mmqKgIRUVFKt1PKpUiLS0N1dXVsLW1haenJ3R1dbttnKNHj8bGjRshFApRUVGBR48eAXj8fEaPHg1fX1+w2WxcvHgRcXFxKmUy/RllZWWora3VyKDfr18/zJ49G3l5edi5c6fa+rptnFwuFwYGBpg6dSree+89uLq6wsDAABKJBBcvXsR3332HGzduoLGxUeloXXl5OX799VdMmzYNXl5eGDx4MMRiMbKzs/HVV18hKSlJ7bS6rkBEsLOzA4/HU8q4dHR04OTkBC6Xi19//RX//ve/Vd5wfxGKi4sV4X5lZ/f6+npMmTJFkZ985swZVFVV4dGjR+jo6NB4W4HH0fsbN27AxsZG6ffA2NgYpqamEAqF+PrrrxXXExE4HA5qa2tx4MABfP3112hsbNTI7KmlpQVjY2O4urrit99+U0vPd999B7FYjJCQEI3k6nbbOCdMmKDYs5HL5aisrER+fj6io6Oxd+9etYynoaEBS5YswY0bN/DPf/4TwOOoZ2RkpNr7bi+CTCZDc3MzOByO0pFFqVSK5uZmyOVySCQSjYziz8ODBw+QkpICKysrODk54e7du912b2UyGW7cuKHxpcCL0NDQgJ9//hlEhPr6eqUGgcjISIjFYixZsgR9+/bFo0ePUFpaioyMDBQXFyMuLg537tzR2F4q8Hi2a2pqUtuYpFIpjhw5AiMjI+Tm5mqkjcyLlDxJU+Lj4wMvLy8AQGNjIwoKClBRUYGioiKN7I39FdDS0oKvry8MDQ0RExOjVFiew+Fg8ODB8PT0RFFREX755ZdX1k5fX1/s27cPqampWL16NSorK1/Zvf5qsFgsDB48GN7e3qivr8dvv/2GvLy8brvGysLLywu9evXC+fPnVU5oUBdd0ZRobCulR16daGtr0/Tp0ykjI+OphOoe+b8hXdlft2fOHvy1YBgGXC4XUqn0P9ZT6cHz0dXM2WOcPejBX4we9r3/JbDZbPj5+WH37t2wsLD4q5vTg/9gqGWcndUhhoaG0NXVVXkz//8KdHV1sW3bNhw8eBCDBg36WxN79eB/wOfzERgYiMTERPj4+PzVzVFA5QwhLS0teHt7Y/z48ZgwYQIKCgpQXV2N77//Hrdu3dJkG5WGQCCAUCiEQCCArq6uIum4c60mlUpRU1ODmpoajdyPxWKhT58+2LJlC8aMGYNbt25h8+bNakdVzc3N4erqioyMDMWG/N8VQqEQNjY2EAqFij3j9PR0SCSSV3I/FouFgQMHAgCysrJU3mM2NDTEggULEBAQgAsXLihd1vVKoWq0VkdHh7Zu3UpisZjkcrlCoqKi/lK2OS6XSwsXLqSYmBhKTU2l3NxcamhooJycHMrKyqKsrCzKyMigU6dO0WeffUYDBgxQiqWvq77YuHEjNTc308OHD2ncuHHPJNkrKwzD0KxZs+jhw4cqsRf+bwnDMOTm5kZ79uyh69ev0/Xr16myspIqKirok08+0ei9OqlwTE1NycXFhW7dukWnT59WOceWz+fTunXrKCkpidzc3NSiVVFHXkni+7Bhw+jOnTsklUqpvr6e5HI5icViWrBgwSt7EYRCITk5OVFISAidP3+etm3bRlwuV/EdPT09ysnJIYlEQhKJhKqqqig9PZ3S09Pp2rVrFBMTQ7dv36bm5mZqamqi48ePk6WlpVptGj16NOXn51NHRwcFBwerbZid0slHtHTp0lfSl7a2tjRz5kxatWoVpaSkUHx8PHl7eyulx9bWli5dukSxsbE0duxYcnFxoREjRtDt27epoKBA7WIAR0dHCg0NpR07dlBERATdunWLIiIiKCwsjOrr62nXrl0qFS2wWCyaNGkSXb16Venf/DLR1dWld999l+Lj4yk1NZUWL178QlaMV2KcDMNQaGgo1dXVUUVFBcnlcuro6FCrVOh5YmpqSu7u7hQSEkJ3796l5uZmKi8vp/v371NsbOxTZF8sFov69etHgwYNIkdHR7KwsCAtLS2FcLlcMjExodjYWJLJZJSVlUUDBw5UuW329vYUERFBN27cIF9fX41Sczg4OFB6ejodO3ZM7UocDodD9vb25OLiQmPHjqWNGzdSamoqNTc3k1gsVlSSFBUVkampabd0dlJ9ZGVlkbe391Mzz4wZM6i5uVktChB9fX1au3YttbS0UFVVFWVnZ9OuXbsoNzeXxGIxiUQi8vHxUUm3o6MjXblyhd555x2NVTkBIGdnZzp58iQ1NjZSeXk5VVRUUHNzM23cuLHLa15ZyZidnR1t2rSJmpqaSC6Xk0QioUmTJmnkh7JYLHJ0dKQTJ05Qfn4+5eTk0JkzZyg8PJwmTpxItra2SrsiQqGQFi1aROXl5SQSiWjJkiWko6OjUvuMjY0pKSmJqqurKSgoSGU+m65ET0+PLl68SBUVFSqVNHWKgYEBzZgxg/Lz8xXMgM3NzbRv3z7atWsX/fzzzyQSiUgqlVJxcXG3jRN4zJEze/bsZ2p3HR0dqbm5mY4dO0ZCoVCldk+ZMoWqq6vp1q1b9MEHH5CNjQ1ZW1srWB9jY2NVGgxZLBZ99NFHFB8f3yWvjypiYmJCBw4coJKSEvrqq68oICCApkyZQoWFhRQXF9elR/VKOISAx7yfFRUVitxUhmFga2urrloAj4l4Q0NDYWFhgdDQUBQUFKCoqAjt7e0qVcC7uLhg1qxZmDt3LgwMDLBjxw7s379fOUa0/w82m40hQ4bAxcUFjx49Qk5OjiK/1srKCkOHDgWHw8GpU6fUzrPk8/l47bXXkJ+fr9L1o0aNwsaNG2FjYwO5XI7k5GQkJCQgPDwcRIT3338fb775Jtra2nD06FGlCgDS0tKQlpb2TGJETU0Namtr0bdvX1hZWamUI11SUoLKykpwOBw8ePAATU1NWLNmDSZMmIBr164hNDRUpUAQl8tV8BR3l3S8OzAzM8PIkSORlpaG9evXo7W1Faampnj48CF4PB60tLSU0qeycWpra+PDDz/EF198ATMzMwXXi1wuR1JSkqpqATyu+FiwYAHc3Nzwyy+/ICsrCwUFBWplxtjb2+PEiRNwcXEBj8eDRCLBzZs3wWKxwOVylY4qstls9O/fH3p6eigvL0dhYSEA4KOPPsKAAQMwadIkCAQCzJgxA7NmzVI7sVpVyg8zMzNMnjwZVlZWaG1txU8//YTNmzejoqICHR0d4HA4cHNzA5/Px927dxEVFaVUHmtXz+TRo0cQiUTgcDjgcFR7zbKzs3H16lV89NFH+OCDD2Bqagp/f39kZ2dj+fLlKu8KDB48GIaGhoiIiADDMDA3N0ffvn1x//59VFVVqTWYMgyDqqoqmJqawsPDA2+99RZaW1uxcuXK/z2aEh0dHVqzZo0iENQpUqmUQkND1XIPgoKCqKamhubNm6ex9UBAQIAiaNPZVrFYTCkpKRQaGkoeHh5KuUh2dnZ04cIFamlpoQMHDpBQKKThw4dTSkoKNTY20o0bNyg6OppEIhGtWrVKpaBFp1vb0NBAY8aMUel3e3h40I0bN6i6uprWr19PVlZWis/09fVpzpw5VFRUREVFRTR//nyV6DqeJwYGBlRcXEwpKSmKGASfz6devXqRkZFRt/WsWrVKcTaKXC6nvLw8mjJlygtJxV8m4eHhtGnTJmKz2TR79my6ePEiJSYm0qlTp7pNRN5VX9fX11NpaSkVFhaSRCKhXbt2PcWU8DzRuFvb0tKCAwcOQC6Xo3///mCxWOjXrx9cXFygr6+vqloAgJOTE3g8HoyNjVUjRnoOrl+/juXLl2PQoEEwMzODk5MTfHx8MGzYMAwbNgzjxo3D1q1bERkZ+VJdWlpaGDlyJFxcXJCcnIzw8HC4urpi9erVcHV1xYkTJ3DixAm89dZbeP3111FbW/tKaj27g8LCQoSEhEAoFOLcuXMKF57NZmPMmDEIDQ2FpaUltm3bhiNHjqhd59lJTdq/f39YWFigsbERo0aNwltvvQU7Ozv06dMH6enp2L59+0v7pJOlovP/S0tLsWrVKly4cEHlEj09PT0MGTJEQWzm7u6O4OBgVFRUYMmSJfjoo4+wfPlylWbPiooKbN++HX369MG7776LxsZG3Lp1S3USMnUDQjo6OmRiYkKmpqY0e/Zsqq6upp9//vmlo8WfpTP4w2azydnZmcLDw+nhw4e0ePFitUbJPwuPx1OQK2/ZsoXKysoUM2l6ejq5u7u/VIe+vj4dPXqUpFIpjRgxgpydnSklJYVaW1vpk08+IWdnZwoLC6OqqiratGmTygGRzpmzpqZGJXKvF4mJiQmVlJSQVCqlQ4cOkbm5udI6GIYhXV1dcnV1pQ8++IDCw8PpyJEjlJCQQLm5uSSTyaitrY3u379P2dnZtHnzZpo+fToNGDCgW4G8qVOnUlFRkWLmTElJUZnrqVMcHR0pNzeXnJ2dSUdHh1xcXBTbHM7OzpSenq6Wt8blcsnV1ZXq6uooMTGRevXq9dJrXllAqKWlBS0tLYrTq4yNjZWuvevduzfCwsKgo6ODpUuXIicnB4cOHcJHH32EXr16gcPhaKyYub29He3t7WhqakJISAjS0tKwd+9eWFpawtLSEtbW1s+l73gSHA4HRkZGyM7ORnFxMYYNG4b+/fsjIiIC6enpmDx5MgICArB9+3bs3LlT7SwZiUSCsrIytXT8GV988QVsbW0hEolw5swZpYmqTUxMsHDhQkybNg1GRkZoa2uDVCpFXV0djI2NYW9vj6qqKqxbtw6//PILRCIRpFIpZDLZS2clLpeLMWPGYN26deDxeDh9+jQcHBygo6OjNgE0h8MBEaGiogItLS3Iy8tTfFZVVaV2yqVEIoFQKASbzUZDQ4Na1DVKGSeLxULfvn1BRM+QWb3++uvw9/cHEeHHH39UqlF+fn4YNmwY4uLiMGzYMAwcOBCTJ09Gbm4uYmNj1eImZRgGlpaW0NfXR2Fh4VOulFQqVXCXAoBIJOoWf61UKkVtbS3c3d1hYWEBU1NTRTpZfHw8SktLERQUhJSUFI2lr2nqgNfOaPrIkSPR3NyM8PBwlQi1Z8yYgYULFyI2Nha//vorcnJyUFNTg46ODixbtgwzZ87EoUOHcPToUaVdemdnZyxZsgREhMWLF6OoqAi7du1CSUmJxsrlnvd7hwwZonbKpba2NmxsbNDY2IiLFy+qN6ko49Y6OjpSdHQ0xcTEkIuLC+np6ZGZmRkNHz5cQYmYlJSktBtnbm5OW7dupYyMDBKJRFRdXU3p6ek0YcIEtfcO7e3t6fTp0zRv3rynMomAx9SY3377LbW2tpJMJqOIiIhupYJpa2vTmjVrqKOjg1JSUig7O5s6OjpIKpUquGHVpW0E/setra2tJS8vL7X1MQxDnp6eFBERQQ0NDbRhwwaysbFRKW1t69atVFBQ8JQ7rKurSzt27KDKykras2ePyu58SEgI1dTU0KxZs0hPT49WrlxJTU1NtGvXrmeeobJiYWFB165deyYryMnJiX788Ue1TppjsVg0YcIEun37Nu3atavbvL8aSUIwNTWlsLAwqqyspJKSEoqPj6fr169TdXU1dXR0UGlpKQ0fPlylh62jo0NmZmZkY2NDNjY2ZGZmpvJRgk++jAcOHCCJREIbN24kS0tL0tPTIyMjI5o1axbdu3ePWlpaSC6XU1lZGQ0dOrTbuh0dHWnXrl1UVlZGNTU1lJaWRps2baJx48apnNTwZ2Gz2XTgwAGqrY2O/ecAACAASURBVK2lsWPHqq1PR0eHNm3aRG1tbZSSkqLWwPf5559TZWUlhYeH03fffUcLFiyg6OhoKiwspPXr16tM0s0wDO3cuZPKyspo4sSJ9M4771BVVRUlJiZqZN3NMAzNmzePLl++TL6+vuTh4UHz5s2jCxcu0IIFC9TqE3t7e7py5QolJycrlRKqsQwhb29vCgwMpOzsbCoqKqLW1laSy+VUUVGhscwgTcrcuXOpoaGB5HI5FRcXU1ZWFuXk5CiMUiQSKVLv/uq2Pk++/PJLEolEtHLlSrVfypkzZ1JOTg5lZmZqJMVy7NixdPbsWYqJiaGYmBhau3at2noZhqG9e/dSRUWFwpOKiooiGxsbjfUpm82muXPnUkxMDMXGxtKxY8dUnlQ6RUdHh7755ht68OABffTRR0pty2k0fY/NZpOnpyf5+vrSwoULaeXKlfTOO+9oxJXTtFhaWtLSpUvp8uXLdPHiRSouLqbs7Gy6ceMGnThxgpYsWfJUxO7vJiNGjKDMzEyaPXu2WnpMTU0pISGBysvLNR4B17SMHz+esrKyqLKyks6cOUMDBgzQaP6rpqUzif7hw4e0YcMGpT2nV5Zb+58gfD6fbG1tycbGhtzc3MjFxYUcHR3JxMTkb/2SAo9D8y4uLmqdHQqAZs6cSZWVlXTw4MHnnq78dxJtbW1ydnYmDw+Pv31bgcfJB+np6XTmzBmysLBQ+vr/auPsEdDAgQMVgby/ui3/l0RfX5/WrVtHa9asUbn0sCv76yH46kEP/mJQD8FXD3rwn4Ue4/wPAMMw0NXVBY/H+6ub0oP/RWjcOPl8PlxdXVUuE+oKVlZW8Pf3h6Oj438dw5+RkRFiY2Px/fffw9jYWG19BgYGsLKygoeHB3x8fGBra/vc49178NeiW2tODocDbW1ttLe3v/RkpxEjRiA0NBSff/65Wqc2PQlnZ2d88cUX+PjjjxEbG4uwsDBcvXpV7SLmVwlHR0dMmjQJycnJah8k5OjoiLS0NPD5fLzxxhvIyspSSQ+Px4O7uzumT5+O1157DX379oWJiQkyMjJw/fp1REVFITs7WyOnd2kCLBYLTk5OCAgIUJyf+ejRI8TFxaGjowONjY2oqqp6qR5bW1u8+eabsLe3V/yttbUVmZmZyM7OVuQE/1W/u6s1Z7eitf7+/hQdHd0tvpbhw4eTSCSi7du3a4S2w9nZmXbu3EkNDQ0klUqppaWFMjMzlaoJfJ4YGxvT0qVL6dKlS2qlmj1P2Gw2rVu3jpqbm+nChQtqUYwAj6lA5HI5FRQUkLOzs0q/NSAggHbt2kX3799XpCs+KU1NTXTv3j3asGGD2u3tFA8PD1q6dKlS+34WFha0ZMkSWrlyJa1bt47S09MVCSNyuZza29spJyeH7t69Sz/99NNLq2lsbGzo2LFjz9Qdi8ViKikpobi4OIqJiaGDBw9SeHi4ypxE6ohaWynTp0+nxsZGSk5OJg8Pjy4zKRiGoSlTplB7ezudOHFC5b05Ho9HLi4uFBoaSuXl5dTa2kpNTU2UkpJCVVVVVFtbq3J6GJfLpdGjR9Ply5fpxx9/pLlz51J6ejqNHDlSIx3NMAx5e3vTrVu3SCKR0PHjx8nExERlfUKhkE6dOkVyuZyOHDmiUp8OHTqUsrKySCwWU3NzM2VmZtKlS5cUkp+frzBSsVhMixYt6rZuFotFDg4ONHDgwKeyYvh8Pp0/f562bNmi1CC9f/9+amtrI7FYTO3t7VRTU0NisfiZwUQmk5FEIqHs7GwaP358l++kj4/PU7/vzyKVShXkZp0EZ/7+/mrvK3cKm80mLpf7wowhtUrGOkvAhgwZgmXLlmHZsmXPrd7Q09PD7NmzweFw0Lt3b0WxrTLQ19fHrFmzMHv2bHh4eACPW4+amhocPHgQ8+bNg4uLi1I6O8FisRAQEIBPPvkEUVFROHz4MIRCISZOnKh0O7uCubk5Pv/8c3h4eKCsrAwbN25UmaJEKBRizZo1mDhxIoDHlCCquPL19fXYu3cv3njjDWRnZ+PEiRMoLS1VfB4QEICoqChwOBywWCylqkiEQiFWrlyJgoIC3Lt3T3Gtj48PHB0dsXHjRqWO1ouOjgafzweXy4VYLMYvv/yCcePGwcbGBr169YKNjY2Ci4fFYsHe3h5Dhw7FlStXnlsJ1adPH/D5fNTX16O4uBhVVVUwNjZG3759oa2trQiydVb99O7dGxEREfjqq68QFhb2VPkjwzAwMzNDdXV1l+1nsVgwMzODm5sbtLW10atXLwiFQjQ1NeHcuXNKlf51yzjT09MRFxeHwMBABAQE4MqVKzh06NAzLwqXy4Wuri4AoLq6WmlSJ4FAgNmzZ2PFihUQCoWor6+Hrq6uguNH3Sp9oVCIoKAgXLx4EYcOHUJ7ezsGDx4MOzs7jRnnmjVrMG7cOEgkEhw9elTBLaQs9PT0MH/+fMyYMQMPHz6EtbW1ym0qKCjA77//juPHj0MsFj/VjwMHDsTbb7+tCLIlJyfjypUr3dZtaGgIGxsbhIeHP/Ui/+Mf/4BMJlO6DrW6uhqfffaZou6zra0NycnJEAgEcHNzw7Zt254anCsqKpCent5ladYvv/yCP/74A1paWigpKcHDhw8hFArh5OQEZ2dnuLu7Q1dXF2PGjFEYva6uLry9vWFiYoI//vjjKX1d1Xvq6enhH//4B/r27Qt/f3/84x//QF5eHvh8PoYMGQKJRAIOh4MdO3Z0vzO6myHk5uZG5eXlJJVKKSMj45lyGC0tLbKzs6OsrCySSqX0ww8/KEWuzOVyaf78+dTQ0EDt7e0UFxdHXl5elJCQQM3NzbRw4ULq168fJScnq+TWcrlcOnnyJB09elSxvuzduzfdvHmT4uPj1XI9n+yD6upqkkqlFB8fT7a2tt2+lmEY6tWrF02cOJE2bdpEd+7coerqajp8+DBNnTqV5HI5HTp0SGMcPwKBgGbPnk0ZGRnU1tZGUqmUqqqqaMaMGUrlGQ8dOpRSUlJowIABT/09JiaGsrKylH5Of3aBjYyM6O2336Zz585RYmIitbS0KFzS8vJymjRpksplZFpaWmRgYEDGxsbk7u5OoaGhVF1dTTKZjAoLC2nw4MHd1nXw4EGqqqqimpoaamxspBMnTpCTkxNNnDiRKioqqLq6mt55553nXqt2+p6+vj5FR0eTVCql9vZ28vLyIi0tLbK0tKQBAwbQzp07KTY2lpqbm6mjo4P27t2rVCL8vHnzSCQSUW1tLR07doxcXV3J3NycVq5cSXPmzCEul0tsNps2btxI5eXlSj/0t956i3JychRVE1wul77++muqq6uj999/X+3EamNjY9q4cSO1tbVRZmYm+fj4KFXlwGazaezYsXT+/HlKS0ujU6dOUUBAAAkEAho8eLBaa84nhcvlkpWVFW3bto0kEoli7ZaVlUUBAQFKJ207ODhQYmIizZgx4ylqmpSUFLp8+TK5uLioTC3C5/Pp1KlTJJVKn1kr/v777zR+/Hi1+uJJYbFY9O6771JFRQV1dHTQ4cOHlRqwd+zYQTExMbRgwQKys7MjLS0tGjlyJGVmZlJzczN9/fXXXdqD2sbJ4XDo448/purqapJIJHTmzBkKCwujqKgoSk1NfarjxGIxHTlypNvFpnp6ehQXF0fl5eU0Z84cRadwuVzS19d/ynDWrFlDFRUVShvnV199RcePHycWi0VsNpveeecdqqiooL1796qdXK2lpUWffPIJlZWVkUgkovnz56tEdswwDOnr65OpqelTRuLu7k6tra104cIFtY6O0NLSIn9/f4qIiKDW1laSSCQkFospKSmJAgICVIqu6+np0Zdffkk5OTm0e/dumjp1Kk2fPp0KCgqosrKSFixYoPLxFPr6+pSenv7cQE5aWhr169dPI4bJMAz169ePkpKSSCaTUVlZGU2ZMkUpD6JXr16kr6+v+Levry8lJSVRc3Mz7dy586nP/iwaSXy3tLSks2fPPtNRnZSYT45wt27d6hZZlkAgoKCgIKqqqqLbt2+/8LudM6eyxmlsbEwxMTH07rvvkqGhIQUHB1NxcTFdv36devfurfYBNr1796YLFy6QVCqlGzduaLT2EABZWVlRUVERxcbGqkTEBTyeGYKDg6mgoIA6Ojro1q1btHTpUho+fLjarOempqY0d+5cunDhAqWkpNCtW7eoubmZgoKC1NqiYrFY5O/vT+vXr6f169c/FXXt6OigDRs2qP3sWCwWTZw4keLi4qi1tZVaW1tp/fr1ShPUPSkuLi6K83h++OGHl5J8acQ4DQwM6Mcff1QYYnt7O1VXV1NiYiKdPHmS1qxZo2B0y87OpkGDBr30h3h6elJBQQFJJBJKTEzs8nsMwyhKc6qqqpQ6MsDAwIAOHDhAmZmZdOfOHSotLaXm5maaMWOG2obD5/MpPDycOjo6SCQSvZKqD1NTU8rLy6ODBw+qtObU09MjPz8/Ki4uJrFYTImJiRqvYWWz2cTj8UggEJCPjw/dv39fqTX3i547h8MhDodDBgYGtG/fPsUkUFtbS66urirp5XA4ZGlpSTNnzqT6+nrFoVenT59W+dQ5FotFgwYNopSUFGpqaqLvvvuuW96jWlspnZBKpaisrERlZSWqq6tx7do1XLx4EXFxcZBIJLC2tsa0adNgbW0NHR0d6OnpvVQni8UCh8OBXC7H4cOHu/yejY0NgoOD4erqilOnTil1hIJIJMK2bdswduxYiMVijBo1Cm1tbd3iqH0ReDweJk+ejMmTJ6Ourg67d+9GUVGRWjo7YWBgAC6XC7lcDi6XiwcPHqC5uRkcDgdmZmawtLREfn7+S7cpOBwOxo8fj2+++QYGBgZITEzEihUrnmKd4/P5MDQ0hLu7uyJiWVNTg7t373abXE0mkykybMzNzUFEGmFMJCLF9oxIJEJMTAzee+89CAQCVFRUqHRuqYmJCXx9ffHZZ59h4MCB0NbWRn5+PtauXYvLly+rtCvAMAwGDhyI7du3Y9iwYfj+++8REhKiFvueUjMnwzDk4OBAfn5+5OHh8cwaxdramu7evauYVWfOnPnSQIu3tzc9ePCApFIpHT169Lkj8jvvvEPnz5+nuro62r9/f7e4QJ8nLBaLAgMD6erVqzRs2DC1XSJHR0fKzs4mmUxGkZGRKhXa/rl/DQ0NacKECXTy5EmKiYmhn3/+mY4ePUq3b9+m7Oxs+uGHHygyMpIiIyNf6o6yWCwaPXo03bhxgyQSCdXW1tLixYtJX1+ftLW1aeLEiTRp0iTaunUrnT9/XsGHVFpaSnPmzFGJ2YLL5dKmTZsoOjpa4wc7CYVC2r17tyIpITw8XKVAnr+/Pz148IBaW1upvb2dIiMjydfXVy3uJ2NjY4qOjqbW1lY6e/asRjiElJo5iQi///57l6f/trW1oaKiAn379gWHw0FgYCB++eWXF+53PnjwAMnJyQgMDMS4ceNw8uRJ3Lt3D7/99hsCAwPB4XAwePBgyGQyrF27FlFRUSrTF+rq6mLdunU4ffo0bt68qXZurrm5uWKDOTk5WaXDejrBYrHg6+uLkJAQuLm5QVdXFyKRCE1NTSgoKMCvv/4K4HEigkgkQl5eHurq6l6ok4jQ3t4OLpcLFosFAwMDLFmyBO+88w5kMhlcXFwUDO3a2toAHntHRITa2lqVKElNTU3h7u6OuLg4pWZOAwMDfPrppxg+fDhqa2uxa9cuPHr0CI8ePQKXy4WnpydmzJiBESNGgMPhQCaTQSwWq0SV+fHHH8PGxgYAkJiYiGPHjuHWrVuQyWTg8Xjo6OhQSq+Ojg727NmDkSNH4uTJk9i0aVO3cn5fCmVmzu6Io6MjJSUlKc587E5QyNLSkvbu3auYQTtTqTo6Oqi+vp5+/PFHtXMeGYah6dOn07Vr11TKT/2zcDgcCgsLo/b2djp//rzap3kLBAL69ttvqaGhgXJzc2np0qXUq1cvYrFYas3wnSmVXaWwicViys7OpitXrtCVK1coJCSEAgICVD5n1MfHh+7cuaM0laepqSmdOnXqmbS6vLw8qqqqeqbNkZGRKqVc6uvr0+3bt5/S19LSQomJiZSQkEDnz5+nBQsWkK+vL/Xq1atbfW9vb095eXmUmJioUsBOIwGh7sqgQYMoMTGRbt682e2DaXV0dGjSpEmUlpZGLS0tVFFRQcePH6fg4GC1tg86xcHBga5fv05z587VCFmUlZUV/fzzz9Ta2kozZ85UWx/DMOTu7k5z5swhT09PtfX9+cWfPn06RUZGUl5eniKafubMGdq/fz8NGjSIeDweaWtrqxUk4nA4FBwcTHfu3FF6ADQyMqJt27ZRQ0NDl3mw+fn5lJycTGvXriVvb2+VnqOFhQXt27ePysvLqa2t7bn3qa+vp6KiIjp27Fi3XHs9PT1aunQpBQQEqNRvXdnfK6Ep4XA4sLOzA4fDQVFRUbcX2BwOB/b29rC0tER7ezuKi4shEok0ElhYtmwZAgMD4evrq9J5nH8Gn89HWFgYJk6ciPHjx+PmzZtq63yVYLPZMDU1hbm5OUxMTFBeXo6GhgZIJBKIRCKNlEvxeDzs2LEDIpEIGzZsUKqf2Ww2LCwsMGfOHKxZswZcLhfA4+MNjhw5goqKCvz666/4448/UFRUpBbzu5mZGfr27QsvLy+YmZlh6NCh8PT0BABERkbi3LlzkEqlsLGxwe7du7uVG6ylpQWpVKpSu6iLkrH/Gg6hkJAQ1NbWIjw8XGM6tbW1weVy0draqrFjAv6TwePxsG7dOmRkZODs2bMq6eByuYqTxTrR3t4OuVzerXNWlAGHwwHDMOByuQpygI6ODsVkomwRgKr4rzfOHvTg74qujPO/i++jBz34D8J/rXFyOBzo6uoqthB60IO/G/4rjdPQ0BBr167FvXv3sHbt2h4D1SC4XC7s7Oxgb2+v9gnn/+3QLEXefwB0dXUxf/58DB8+HGvXrsXdu3c1dvZlDx6zJG7duhVWVlY4c+YMdu/erTHiLGtra7z33nsoLCzE1atXUV9fr9EAkSYxcuRIODk5ISYmRvUzP1/FPqcmhMVikZeXF61Zs4YiIyNp7dq1NG3aNHJyclJ5U77zwJnbt29TYGDg3/pwnCdl8ODBdOHCBcUmuar7acqIKn3M4XBo3rx5VFtbS3K5nK5fv65Wdcefxdvbm5qamqi8vFypksT/TdHR0aH333+fcnNzqbi4mPz8/F56jUaSEBiGIR6PR0KhkIyNjZ8RTZ1LCTzOmImKiqL29nZqa2uj+vp6qqmpobKyMlq6dKlKOg0NDSk+Pp4iIyNfyQE5HA6HDA0Nyd7enuzt7cnKykqtg5JYLBaZmZlRVFSUImtGIpHQgwcPaPHixRp7Ofl8PpmYmJCLiwstX76ckpOTqaioiKZMmaKUHm9vb6qrqyOZTEatra20YcMGjQyAXC6XZsyYQVlZWdTR0UESiYTi4+OVKs1js9mkpaVFurq6ZGhoSEZGRgoxMDDQSDsZhqFJkyZRSUkJSSQSSkhI6FaVklq5tWZmZnjttddgb28PV1fXp6oXnsTdu3exc+dOlJeXd0ftC6GlpYVhw4ZBLpcjIiICWVlZYLPZmDBhAqZOnYqTJ0+iqqpKKZdpxIgRcHV1xaeffqqRNnZCW1sbzs7O8Pb2hq+vL6ZNmwaGYZCbm4vw8HAcP35cpbxbPT097N69G35+fgCgcOF69eqFsWPHIioq6qX5tS+Cubk53N3dMWrUKHh6esLd3R3a2tq4c+cOYmJiusyhfh7s7OywePFiGBoaAgCOHDmCPXv2aGT/V09PD56ennBycgKLxQIR4c0338SyZcuwadMm1NTUvPD6TvJsIyMjWFhYwMTE5CnS89bWVly6dAlXrlxBWVmZyq5yr169MHnyZJiZmUEkEuHLL798qvpHWXTLOD/77DNMmTIFZmZmAICSkhJUVlYqWOXYbDb69++PESNG4I8//sDu3bs1tinf2NiIPXv2ICMjA05OTnjrrbfg4eGBb775Btu3b1cqM2fo0KEoKSlBfn6+RtoGPN6ofvvttxEcHAwOh4Ps7GxcvnwZb7zxBpydnfHJJ58gMzMTqampSuvu06cPXFxcFAGrGzdu4NGjRxg1apTa7RYIBFi9ejWmTp2K+vp6XLt2DampqcjOzkZmZqZShgkA48ePVwwihYWF2LNnzwtZ6rqLiRMnYvLkyXj99dehpaUFhmEUMYIZM2bg/PnziI+Pf2nbZs6c2eXnXC4XEydOxObNm/Htt98+RVTWXTAMAw8PD0yZMgVaWlq4e/cuCgoKlNbzJLplnIaGhhCJRFi/fj0yMzPR3t6uyKRgGAbOzs7YunUrtLS0NB751NHRgZubG2QyGXbu3AkPDw9oaWnBwcEBEomk23o6GdUiIiKUfvFeBC6Xi08//RRyuRxLlixBbm4uBg0aBFNTU7i5uYFhGKUDIiwWC++//z7+9a9/wcLCAi0tLVi7di3OnDmDcePGYdSoUWodScFisfD5559j2LBh2L9/P3766SdUVlZCLBarlCppaGiIQYMGQU9PD1VVVdi4caNGBkA2mw0vLy9Mnz4dwOOa0XPnzqGyshITJkyAhYUFfH19kZSU9EKDOnjwII4fP95lto+lpSVOnjwJb29v6OrqqmScLBYLw4cPh7a2NvLy8hAUFKQYnLhc7lN1qd1Gd9acDMM8N0DA5/PJx8eHbt68STKZjJKSkjRynDnweH1YU1PzVIWCWCymnJwcOnHiBHl7eyulb/bs2VRRUaEx8uhO4fF4dPv2bbp+/TpNmjSJFi5cSHv27KGamhqqqqpS6URqNptNW7ZsUawxCwoKFJ/NmTOHxGIxXbx4UeVKmHHjxtHDhw8pLCyMrK2t1WZE8PX1pcLCQpLJZJSdnd3tYoeupLNueOHChVRSUkIikYguXbpEmzdvJqFQSH5+fnT//n2SyWR09epVtdn/+/XrR3K5nDZv3qwyGZlAIKArV65QaWkpjRkzRsHg4OLiQvv376ewsDByc3N7rh2pteZ8ng+ur6+PDz/8EJ988gkcHBwAPK5UHzJkCPr374/79++joKBAqdntSRgaGio4QhmGQXNzM3766Sfs3btX6fM82Gw2BgwYgLq6uue6wXw+X+XaQKlUivPnz2PBggU4evQoJBKJYt114MABHD9+XGmdnTNGZ7/HxMQAeLyu8/PzU2vWdHR0xPz58yESiSCRSLBgwQIcOXJEZQYHNpsNe3t7mJiYKGpDBwwYAB6Phxs3bqiUmyoQCDBv3jx89tln0NbWRmlpKb766itcuHABbDYbZmZmij42MjJC79690dDQoFL7gcf9Cjz/Pe8uPD094eHhgaSkJNy9exdEBBcXF2zZsgX+/v5gGAZEhNWrV3e/EETVxgiFQrz//vtwdHRU/G3UqFHw9vaGTCbD7du3sWLFCqUPM2IYBq+99ho2bNgAXV1dSCQSZGdnY+fOnfjll1/Q2NiodCcaGBjA1tYWly9ffqpSwtbWFmPHjsWgQYOgq6uLQ4cO4eLFi0rplkql+Pbbb5GcnAwdHR0EBATgo48+wr1793DkyBGVBicWi4U+ffoAAPLy8nDo0CEAwLhx4/DPf/5T5X1ZHx8frF69GlKpFMuWLYNYLMaGDRuQkpKisnHy+Xz069dPcUqZlZUVQkNDIRaLkZ2djYiICERGRiplpEZGRhgwYAD4fD5+//13LF68GLdu3QLwuNj8jz/+QF1dHUxMTFBaWqo2IbibmxvEYjGqqqpARPDy8lJUxbS3t3crruHj4wMOh4Pbt2+jpqYGenp6WLJkCUaNGoWSkhJUV1dDIBAo9exUNs7q6mosXLhQ8VCMjY3h6emJ3Nxc2NraYv78+Th27BjGjRvX7ciogYEBxo0bh6VLl8LNzQ0sFgtxcXEICgpCaWmpykEmDocDPp+P3377DUQELpeLMWPGYMOGDairq8OdO3cwfvx4VFVVKW2cAFBbW4tLly7ByckJoaGhKC8vx6ZNmxQvlLJt3b59O6ysrNDY2Ijly5fj3r17YLPZMDc3h7a2NhobG585UuFlYLFYGDp0KNra2rBmzRrk5+ejf//+isoaVcEwDLS1tRVRVC6XC1NTU2hra8PBwQE2NjYQiUS4cuVKt9azDMNgxYoVGDFiBIgIx48fR1xcnOLZczgcDBw4EA4ODkhNTUVwcDAePHjQrbZyOBzweDwIBALo6+vDzs4OBgYG+Pjjj6GlpYXQ0FB88cUXYLPZaG5uxv379xWsGS+Cjo4O/Pz80NbWpvAWTU1N4e3tjbq6OixatAh+fn4QCATdaqeivUp9+wm0tbUpjraztLTEzJkzkZycjISEBLBYLBQWFmLLli149913ERYW9lJ9+vr6WLp0KRYvXgyBQIDq6mqYm5vj4cOHaGho0Ej0l8/ng8ViYfTo0Vi5ciViY2OxZ88eODg4YMqUKbh06ZLKus3NzREUFAR7e3scPnwYCQkJKrl0rq6uGDZsGIDHs0RbWxuICH369IGXlxcYhkFCQgIuXLigVJ/I5XKEhYVBW1sbra2tCtfZ0NBQLZdQLBajsLAQra2tEAgEqKurw9mzZ2FkZAQ/Pz8MHjwY+/fvx6xZs7p9zIOZmZliq8Pa2hpaWlpob28Hj8dT1PuKRCLs2bOn24EnoVCI9957D/3794eHhwdsbGxgZmYGiUQCbW1tPHz4EMnJySgpKcG1a9dw584d/PHHH91yQdlsNkxMTNDW1oaqqipwuVxMnjwZ+vr62LRpE8rLyzFu3DicP39eKa9PI+l7s2bNgrW1NcLDwxUvZGpqKh48eIBFixZ1yzjnzZuHoKAgtLe34+zZs2hoaMBnn32mieZBIpGgpaUFQqEQDMPAxsYGFy5cwPbt2xWH7WRlZSEjI0Ml/VpaWpg4cSKmTp2KxMREfPfddyqzrhkZGcHExAQA8PDhQzQ1NUEgEGDWrFkYNKwAlgAAIABJREFUOHCggqWwO94Il8uFra2tIjotk8kUvECjR4/GnDlzcO7cOeTk5KjU1ueBx+PB3Nwchw4dgqurKwwMDGBtbY3Zs2d32zg7Z5ja2lpkZmYqBiEHBwfs3LkTdnZ2uHjxolJ7iJaWlggNDUVVVRVyc3Nx+vRpRY3o6tWrkZGRgQULFqg0UMlkMtTX18Pe3h5jx45FZWUlFi5ciPr6eshkMmzcuBGPHj1CdHS0csscTaTvRUZGUkFBAS1YsIBMTEzIzMyMFi1aRAUFBVRaWvrS693c3CgrK4vi4uLIzc2NhEIh7dmzh2QyGe3fv1/t80FYLBZt27aNSkpKaPTo0WRmZkZ2dnYUGBhIWVlZFBUVpTKxMsMw5OPjQ5mZmXT//n21uYRGjBhBpaWlJJPJ6OjRo2RiYkLLly+npqYmRZS2uyx/ZmZmlJWVRcuXLydTU1NFdD0uLo7u379P27ZtU5sAm8vl0pw5c57i+eno6HiKBqS1tZVWrFjR7f6Mjo5WcAXt2LGDAgMD6cyZM1RUVKTgFfL19VUqxVBfX5/mzp1Lw4cPJ2NjY2Kz2Yrj+W7cuEGnT59WOZuLYRjasmULdXR0UFVVFd28eZPEYjG1tbVRdXU1iUQimjt3bpdR8VfKIRQUFER1dXXU3t5OlZWVVFVVRRKJhOrq6igwMPCF1+rq6tLXX39Nra2tFB4eTmw2m8zNzam4uJiamppoxYoVKoe3nxRnZ2e6evUqVVRUKKS4uJgOHjyo1vaPtbU1FRUVUXFxMU2aNEntNLCRI0dSWVkZERGlpaXR0aNHFYfHJiQkkJWVVbd1GRsb0507d6i2tpbS0tIoJyeH8vPz6fTp0+Tg4KA2NWin8Hg8+vzzzyk/P19x/goRKQ67/fnnn7vN/M4w/4+9946K6lrfx985M8zQGToWSgARhQQUosSOooLdqzH6iRiJBYmxEEtkqUnU2DBGkRtrFJWrRk2s1x4UIdhFYS7Se+idcYZhCs/vD9fMV6PCNG+S3/VZ611LmTnv7HPOfvfZZ+/nfV4WDh48qBKOVgq9KUXfampqsGnTJr0WO05OTtYpOImeidRdvHgRNTU1qraWlpbi0aNHWLx4cbvFlnTaSukI+/btI7FYTEFBQeTo6EgMw1BGRgadO3eO/v3vf7d7rKmpKbm4uKjKB86aNYuGDh1KNjY29Ntvv9HZs2e12hT+I7Kzs+mLL74gPz8/cnJyUq3CpaSkUHNzs1Y+LSwsaPHixSrtnOvXr+v8bqwsddjW1kbvv/8+vf/++ypJ0oMHD2rEumlqaqJFixZRQEAAmZmZUXp6OhUWFpJAINCoZmZHkEgktGfPHhIIBDR58mTq378/mZubU1tbGyUnJ9P27dvVni4CoOzsbNVqrHJ1s76+nh4/fkynTp2iEydO6PSe/Efk5OTonN5WUVFBc+fOpZEjR9Jnn31GLi4utHr1akpISKCysjLt+oW+slIYhoGtrS26d+8OT09PuLi4qPXEs7a2xv79+yGVSiEWi1FXVwepVIq6urp2Kxb/2WZsbIyoqChUVlYiOjpaZ0FppVlYWGD16tWq0Vcmk6GiogL+/v7tFsP5q5ipqSm8vb3h7+8Pf39/2NnZaXwP+Xw+1q5di4aGBohEImzatAmhoaE6ZSS1Z1FRUTo/OZXG4XCwefNmXLt2Te0CTm90WqurOTo64uTJk6rp4ZUrVzB27Fi91vLQpxkaGiIsLAxFRUU4e/asRlNNdczIyAhbtmyBQCDA4sWLMWzYsD/9nP/bxuVyVdkjXC73jfaFzp07w8vLS6/TfE0qq/2lg/PvZCwWC1OnTsXNmzexZs0avb77vLX/TXtd/L1V39MQLBaLBg4cSGKxmDIzM/WigfsW/9vAW2nMt3iLvyZeF5z/kwJfb/EWfwe8Dc63eIu/KP5Wweni4kLfffcdTZky5c9uylu8xRuHXt45LSws6Oeff6aamhr64osv9FKbkM1mU9euXcnV1ZUYhqGamhpiGIb2799PjY2NNGzYMJ1/Q59gsVjE5/PJxMREpX5QU1OjNpdSWT9TIpG8kjCvrJv5V4axsTFt2LCB3NzcKD8/ny5evEgCgUCViqUvKOupmJmZvZDbWl1d3WHmC5vNpgEDBtC2bdtIoVBQZGQkpaSk6KV95ubm5O7uTv369aP58+eTtbU1ET27d8HBwVRRUUHNzc0vtfF175x6YQgZGRmRg4MDvf/++7Rz506dg9PMzIyGDh1K69atox49ehDDMJSWlkbr1q0jLpdLycnJOvnncDjUpUsXcnd3p6amJnr06JFO2qpsNpu8vLxo3bp1NGrUKGKz2dTY2EhLliyhuLg4tXxYWVnRjz/+SIWFhVRdXf1CZ1EoFFRQUEBpaWk6V9gi+n/pY126dCGiZ5klBQUF1NjYSGVlZVrJqlhZWRGfz6fCwkKaOHEitbS0UEREBDEMQ5GRkVrni/4Rvr6+FBwcTP7+/jRy5EgyNTVVfbZixQravHlzu8d369aNdu/eTRYWFpSXl0effPIJZWRkaMQ4UmbMPD+I2tvb03fffUdjxowhU1NTVQod0bOB+8aNG1RSUkI7duygf/3rX2oN2noJTg6HQ1wul8rKyrTOxlDC0tKSPv/8c5o5cyaZm5tTSkoKde7cmY4fP04hISFkZ2dHhw8f1so3n8+nAQMG0MCBA6lXr17k5eVFly9fps8//1xrOhvDMDRq1Cj67LPPyMPDg3766SfKz8+nSZMm0ciRI+ngwYNqjcoikYgSExNp2LBhNHr06BfUDZXHp6en07x587QWKWaxWGRjY0Mff/wxzZ8/n5ycnKilpYUYhqHs7GzKysqiNWvWUF5ento++Xw+TZs2jQIDA8nW1pacnJyoS5cuZGlpSYcOHaJvv/2WBg0aREVFRToPKo6OjrR37156//33X/h7fX09paenqxLU20OnTp3I1NSUjhw5QgkJCRQTE0PDhg2jn3/+We12dO3alYieCd0p742lpSXZ29urKqSJRCLKy8sjkUhEfn5+xOfzicvlkq2trdoJ13oJTnt7e3JycqJHjx5RU1OTTr5Gjx5Ny5Yto7y8PJoyZQqxWCyaOHEide/enUJCQignJ0cjLizDMGRjY0OTJk2i0NBQsrGxocLCQqqpqSETExPKzMzUibtrbGxM69ato/T0dFqzZg0JhUIaM2YMlZSU0M6dO9WeLrW0tNDOnTvp6NGjqpFXCQ6HQ+Hh4TRgwIBXSpKqAxaLRb169aLVq1fT4MGDqampiWbPnk0CgYBmzpxJERERqvox6oLNZtP69etp5MiRdPPmTTpw4ADZ2tpSWFgY9erVS5UfmZubq3VgMgyjek349ttvyc3Njc6dO0fFxcWUnp5Od+7cIZlMRs3NzWrLfwCggoICevr0KbFYLJVMibpQzgyfv7f5+fn0zTffkJ+fH1lYWFB1dTVNnTqV3n33XVWZyKNHj1J8fLza7dQLQ2jMmDFQKBS4e/euzilTM2bMgFAoxJkzZ+Dh4QETExN8+OGHEIlEePz4scblzD/66CNkZmYiNzcXa9euhaenJ7hcLrp3747i4mKMHz9ep/ZGRUWhtLRU5adv3744e/Ysrl27plUJ8leZr68vhEIhNmzYoBEtTGksFguenp7IzMxEcXExYmJiVHS1nj174vbt26iqqsInn3yiUbl5Dw8PlJeXY/78+Sp6HYvFgrGxMfr374+kpCSsWLFCa85q586dMWvWLERGRoLoWWYRl8sFwzBgGEYrut2QIUNQUlKC5cuXY8KECSgqKsK+ffv0IiptaGiI/v37Y926daisrIRcLkdLSwsKCwuxdOlS+Pr6vvK4N0rfU+Ze3rp1C87OzjqdoLOzM44fPw6xWIzk5GSsX79e1anCwsJgZGSkkT8/Pz9Mnz4dAwYMAJfLVf190qRJyMvLg7e3t07tjYyMRE1NDb766it4eXkhNDQURUVF2Lx5s14U8Hv27Ilz587h/PnzWg98fD4fJ06cQFVVFebNmwdjY2OwWCx4eXnh4MGDEAqFyMrKgqurq0Z+ra2tcevWLVy+fBkjRoxQBaGnpyfi4uKwd+9erdtsYmKCnTt3oqWlBTExMTpfR6W5urri2rVrePjwIS5dugSBQIAnT55ofO5/NIZhEB4ejocPH0Imk6G1tRW3b9/Gli1bMGzYsHaD/40G5+7du6FQKLBnzx6dE6OVHfLIkSOQyWSqUgzTp0/XS16n0nbu3ImsrCyNg/2PZmVlhdTUVFRWVuLhw4coLy9HXV0dPvroI53b2L17d5w7dw7l5eXw8/PTenT38vJCdnY2bt26hU6dOoGIEBwcjJs3b6K8vBwtLS24ePGiVvfO398fsbGxePz4MdatWwcPDw/s3bsXy5cv12nmMHbsWFRXVyM1NVXnhPDnjcVi4b333sPChQsxY8YMhIeHo6qqChEREToR37lcLq5fv67KQxUKhfjyyy9hbW3dod83FpwmJiZ4+PAhxGIxli9f3m5SqbqmLIij1KvNzc3VS9A/f4Pi4uLwyy+/6GU64+zsjOXLl+PMmTMoLCzEvXv3dH4i29vb48SJE6ioqMCHH34IY2Njra/t5MmTUVFRgaSkJHz++ec4evQosrOzER8fj8jISJSXl2PJkiVat9XQ0BDBwcF48uQJGhoacPDgQdja2up0/jNmzIBUKkVLSwtOnDiBiIgIODk56a0PKJUQTExMcPr0aVy5cgVdu3bVqU/17t0bixYtwo4dO1BZWYna2lps3boVDg4O7QboGwtOPp+P8vJylJWVYdy4cTpfNC6Xi3HjxqGgoAAFBQW4evUqysvL9VpZq0uXLrhz547a0hnqmo2NDerq6nDx4kWdCyV17twZR44cQVZWFvbt24ft27dj+fLlmDp1Kuzs7DQ+3/Pnz0MoFEIul6O0tBQbNmyAra0tNm7ciOzsbJ3b6+rqiosXL0IgEODmzZsYPHiwTk8iFxcXREdH49GjRygqKgIA3LhxQ/Xk16eFhIQgNTUVgwYN0pvPKVOmICEhAc3NzTh16hT4fP5rv/u3Cc4hQ4ZAIBCgvLwcn3zyiapq0/79+/XyVCYijBw5EoWFhfDx8dHbzTAxMcH69etRXV2NadOm6ezPwMAAPXv2xLhx47B69Wr8+OOPSExMRFNTE6KiojTyxeVy4ePjg5kzZ2L+/PkYPXo0jI2NMWDAAOTn5+PixYsaLQT90ezs7LB//36cOXMGffv2RXx8PK5cuaK1LtPz5uPjg3HjxuHixYsQi8XYvXu33u6Z0mxsbHD9+nWEh4frrSwkwzDw9vbGqVOnIBaL2324vC7+/lLFczt16kRLliwhDw8PWrFiBR07doyGDRumUswzNDTUWkH+edjb21NDQ4PaeqfqwMHBgYKCgigtLY1Onz6tsz+ZTEZPnjyhJ0+e0JUrV8jQ0JB8fHzo4MGDau3nPQ+pVEppaWmUlpam+huXy6Xhw4eThYUFHT58WGsSBpvNptGjR9M//vEP+vDDD+nu3bskEonoxx9/pC+++IKWL1+uU1pdWlqaSpjc29ub7O3ttfb1OigUChKJRNStWzcyNDRUKRTqgra2NsrNzaXs7GwKCgoiQ0NDjX3ozK1ls9nEYrFIJBLppLzNYrFo3rx5NGrUKLpw4QLFxcWRXC4nOzs7FcNCV4KDEsbGxiSTyfQS6ETPaFsLFiygnj170pYtW/SiefQ8WltbSSKR0HvvvUdsNpv27Nmjs09LS0v69NNP6dKlS/Trr79qTV8zNzenwMBAqq6upszMTCIi+s9//kOrVq2ikJAQCggI0Nhn165dycnJSSV2/c4779CYMWOIz+erKtu9CXTu3FmjQlwMw5CzszP16dNHpUGkJBp07tyZRowYQUFBQQSAHBwcNG6Pzk/OHj16EJfLpYqKCo2YJX8Ei8WiQYMGkVwup4yMDLKzs6P+/ftTeHg4NTY26qVDEj2jBvr4+FBOTo5Wos+vgqurKw0dOpQEAgHdunVLLz6fh5JAMGfOHMrKytKo7OHr0KdPH7KysqLs7Gx6+vSp1n4MDAzI0tKSrl69+gIFLjs7mwBo/MTw9vamrVu3kqGhISUmJpJIJKKBAwfSmDFjqKqqin766Set26qEo6MjBQUF0e+//063b9/W2o+rqyvt37+fBg4cSGFhYZSWlkaDBg2iDz74gPh8PvXv359MTEzo5MmTdPDgQY396xyc7u7uOsn5Pw+pVEoMw9CkSZNo6NCh5ODgQK2trbR+/Xqd62Eo0blzZ/L396cDBw7oJTiVJQBZLBatW7dOL1OiP8LU1JTmz59PDMNQdHS0TjxgomeJCrNmzaLU1FQ6duyYzk/6trY2ys/PVzFfzM3Nafz48cRmszW+Hl9++SUNHz5cNVgr8Z///IfWrl2rtjB1ewgJCaEvvviCJBIJHT9+nM6fP08mJiYaz8w4HA6ZmZkRAFqxYgVJJBJydnZWPUWrq6vp6NGj9M9//lM7eqiuC0LTp09HXV0dvv/+e50XbIKCgvDzzz+jtLQU169fx65du+Do6KjTYsUfzcfHB+np6ZgwYYJeBJ0mTpyI2tpaREVF6XUflmEYmJubw9zcHHv37oVAIMDYsWN1XrAwNjbGli1b0NzcjP379+t8DbhcLhYuXIiHDx/im2++wapVq3D58mXk5ORg0aJFGveJmTNnori4GAkJCYiLi8PXX38Ne3t78Pl8vS3WuLi4wNnZGcuWLUN2djYqKytRX1+PiIgIjfoah8PBrFmzUFtbC5lMhsbGRuTl5eHBgwf4/PPP0adPH7X20d/Yam337t2xe/duvde9fBPGMAymTp2KJ0+e6FxDkuiZinhSUhLu3r2rF3/Pm42NDaKjo7F3716cO3cOQUFBOg9+RkZGWLlyJRobG5GamoqePXvqpa3W1tZYsWIFjh8/jpMnT+L777/He++996ffb3UsICAABw4cwObNm7XaRzU3N8eUKVNw4MABTJs2TSuB8jfKEPq7GI/Hw8GDB3H+/Hm9yFmamZkhNDQUgYGBL1AD9WEcDgcBAQEYPXq03hgyfD4fBw4cQGVlJcaOHfun34+39sxeF3//UwJfLBaLnJycCIBWeYt/d7DZbOrSpQsZGxtTfn6+3lar30I34K363lu8xV8TrwvOv5WG0Fu8xf8S3gbnW7zFXxRa7XOy2Wzi8/mqzVwLCwvq2bMnFRYW0rZt29QuM98RuFwumZqavsDaAED19fXqZ5O/xX8FbDabeDwemZqaUmtrKzU3N2vNOvpvg8VikYWFBXG5XHr69Kle96q5XC5t2LCBJk+eTNu3b6ft27erfaxWwRkYGEgLFiwgV1dXEgqFVFNTQzwej0aNGkVPnz6ltWvX6qQXw+PxyMXFhcaNG0fTpk0jX19f1WdSqZSWLVtGcXFxOjFb/tdhYmJCjo6OZGVlRba2ttTa2koPHz4kDodDPj4+9Pvvv6s4re2BxWKRg4MDDRs2jAYOHEgffvgh3blzh/bs2UNPnz6luro6ysvL+0vcKwsLCzI2NqaqqqoX+qetrS398ssv1KNHD9qzZw+tWrVKbwPLu+++S5MmTSI7OzuVvpC60Co4jY2Naf/+/VRfX081NTVUWVlJbDabPvvsM5oyZQp99913Wt8MNptN48ePp/DwcOrTpw+ZmJi8cKEMDAxo+fLlJBAIKDExsUN/RkZG5Ofn98LfDAwMqF+/fnTr1i2qqamhnJwcvT2JGYahd999l0aNGkVXrlyh1NRUjX04ODjQBx98QH379iUul0sXLlyghIQE1ecsFovc3NwoICCAbt26pZGynfLYTz75hIKCglT6T2KxmC5fvkw8Ho+GDBlCp0+fptmzZ3fIojI0NKSZM2dSZGQkWVlZEYvFoqCgIOrbty8pFAqqrKyk06dP07Zt23RieZmYmFC/fv1o4MCBxGaz6bfffqOEhAS175u5uTlNnTqVBg8eTEuXLqXy8vIXPgsICCCGYWjAgAHEYrH0EpydOnWiiIgIsre3p9zcXM2pgtrsc76KRWFhYYF9+/bh+vXrWjNlXF1dsW/fPhQXF0Mul6OtrQ1tbW1QKBQQiURobGxUZZlPmjSpXV/GxsYIDAzE2bNnUVhYqLKioiIUFxdDLBajpKQE6enp2LJlC9577z2dGCjGxsbo2rUrunTpgszMTJw9e/YFiQ5fX98O2ScMw2D8+PFITk5GeXk5xGIxWltbkZqa+oIvHx8f3Lx5EzU1NZg3b57abTQwMMC0adPw8OFDNDY24smTJ4iNjcWxY8dUFakVCgXu3LmD4cOHd8geYrFYGDNmDEpKSlTHKk2pCKC8XwsXLtTqujIMA2dnZ+zevRuFhYUQCoVoaWlBUVFRh33geRs+fDgqKyshlUpfSoR3d3dXtfXmzZt6YyIFBgaitLQUUqkUK1aseO39f6MkBB6Ph9mzZ+Px48cICAjQ6kQMDAywadMm1U1VJgWXl5ejtbUV586dQ3x8vNrBGRoaqhJZEgqFKC4uRlFR0QumvHBSqRRFRUVYuXKlVro/RkZGWLJkCdLT0xEQEICCggIEBwe/cJOHDRvWLlHB0NAQixYtwoMHD/D48WNs27YNDg4OcHNze4G2Z21tjatXr0Imk+HChQuwtrZWq40sFgsTJ05EeXk57t27h+nTp8PW1hZmZmbYtm2bKjhLSkoQHBysFq3P0tISlZWVLwSlWCzGlStXEBcXh8LCQtX9LC4u1kgShsViwd3dHTExMaioqEBiYiLWrVuHIUOGYPXq1aivr9coWX78+PGqNv43gtPY2Bjbt2+HRCLB48eP21WGeCPByeFw4OHhgfnz56O0tBRHjhxBQECAVmprkyZNQkFBARQKBVpaWpCcnIyRI0diwoQJiImJQf/+/bF69Wo0NTWpFZzBwcFISkqCTCZDTk4O5s+fj0GDBmHAgAEqGzlyJLZu3Yrs7GwoFAo8fPhQ4wRhAwMDzJw5E7m5udi9ezcWLFiA8+fPv1S3c9CgQe0Gp729PQoKCrB9+3bY2Ni8RNVjs9nw8/NDbGwsJBIJBAIB+vTpo3Y72Ww2jh49iubmZkybNg0Mw4DNZiMwMBCFhYVQKBRoaGhARESE2jTBkJAQSCQSVcdubW3FoUOH4ObmBkNDQwwZMgS//PILCgoKUFdXp9G1dXJywsmTJ5Geno5PPvkEfD5fNcAIBAJcvnxZo2rizwfn8OHD33hwhoSEoLS0FNXV1QgKCmr3u3oPTkNDQ4SGhiIpKQl1dXVoa2tDVVUVHj9+rHGmvo2NDa5duwaFQgGJRIJ169ahZ8+eqg6k1M+ZOnUqSktL1QpOHo8Hb29vHDt2DOXl5YiNjX0lDc7c3BwnT56EQqHAjRs3NNKRYRgGc+fORXZ2No4fPw4fHx8kJibiwIEDLw1QlpaW7T6NHBwcUFZWhp07d74U2D4+PliwYAHu3bsHiUSCyspKzJw5U6NBkGEYLF++HM3NzTh8+DBsbGzQs2dPXLp0CS0tLRCLxdi1a5faHd7GxgYXLlxQPRlTUlIQFhb2gvoim81G586dsXnzZjQ3N2PPnj1qFxseMWIESkpKXqIZbt26FQqFAqtWrdKIMrl48WJVAMbHx4PH46Fz584YPnw4vv/+e70Gp6OjI5KSkiCXy3Hs2DHY2Nj8d4PTyMgIs2bNwtatWzF79mx4eHjAw8MDs2bNQkZGBkaMGKH2yUydOlU1BU1OTn6lOJaFhQX27t0LmUymVnAqjc/nY/bs2Th9+vQrydh9+vRBWloahEIhlixZotGNcXd3R3p6OmpqahAdHY2vvvoK9fX1WLduHczNzWFtbY1hw4ZhwIABHQYSh8PB1KlTkZGRgfv37yMlJQUpKSm4efMm8vLy0NjYCLlcjtbWVmzfvh1mZmYadxpbW1vs2rULzc3NiI+PR0REBCoqKiCXy3Hr1i2NyOrTpk1DdXU1pFIpMjIy8OGHH772ibto0SIIhUKUlpaqLWUTFhaGW7duwd3dHUZGRjA1NcXcuXNRX18PoVCIKVOmqN1WDoejGvyVM7P09HTk5OSgrq4OLS0tqs80eYd/lXXu3Bk//fQTJBIJHjx4AH9//z9Pfe+PxmazcejQIWzevFmt7xsbG2P16tVobm5GYWEhhg8f/soAMTExwbp16yAUCiEUCjF58mSdLiLRs6dmbGwsRCIR4uPjNV7IYrFYCAsLw/3795GXlwexWAyZTIa0tDScOnUKAoEAZWVl2LFjh1pPDA6Hg+DgYGzbtg3Xr1/H9evXsXv3bqxevRpXr16FQqFAUlKS1nq4LBYLzs7O2LFjh6pDymQypKSkaKwWuHjxYgiFQjQ2NiIiIqLd73p6euLOnTtoamrCrFmz1PIfGhqKuro6pKam4ocffsDJkyeRmZkJqVSKS5cuaaSP7O7urnp1ac+kUqlOkqbKrKfKykqIxWJ8/fXXas1uXhd/etcQamtro/v375O7uzux2ewOyeWdOnWioUOHkomJCZ09e5ZSU1NfuUcqEono119/pWnTplGnTp300tbBgwfTjBkzqLW1lX788UeNk44BUHx8PN2+fZs++OAD+uqrryg/P5+OHz+uWoqvra2l3NxcMjc3p8bGxnb3f+VyOV2+fJlu3LhBFhYWRET09OlTamtrI1NTUwoICKBdu3ZpvUkOgEpLSyk5OZlmzJhBXC6XCgsLadWqVWrtaT4PZb0PiURCpaWl7X43KyuLLl68SD169FDb/61bt2jjxo0UHBxM3bt3p9u3b1NxcTF9+umnlJSURL///rvavkpLS2nTpk00e/bsF9pQVlZGubm5FBQURCYmJiQWi8nPz49OnDih1VaKtbU1jR07lmxsbCgxMZHi4+M7rHrWHvQenEq2hVwuV4uIwGKxVPtKhYWFVFdX99rvKuuHtLW16UUNIDIykhiGoe3bt9O9e/e08iOXyykvL4/GjRtHLS0tNH/+fMrOzn7pe05OTmqQGTdPAAAgAElEQVQXsGltbaXq6mrV/8eMGUNTpkyhlJQUSkpK0noPjmEY6tGjB82ZM4eMjIyooaGBKioqqLCwUGNfyjYYGBioSt29DiwWi3g8nkbtzs/Pp5iYGJUsia+vL3377bf022+/0b59+zS6/62trXTkyBG6fv26atAjIhKLxSQUCum7776j//u//yNjY2Py8/PTep/T19eXxowZQyKRiGJjY3WurKYVt9bGxua1Ha1Tp040d+5campq6vAEGYYhX19fcnNz6/A3vb29admyZeTs7EyJiYl08eJFbZpORM+ICYsWLaIPPviABAIBnTx5UusqY0REdnZ2FBAQQFeuXHltRy8pKdFqQGGxWLRgwQIyMjKiY8eO6VRe8d1336UffviBfH19adeuXRQeHk6tra1aFUdS1gs1MTGhwYMHqypvvQqurq4UGhpKbW1tGknDyGQy+v3336m6upq8vLzIysqKNm/erJXIl1QqVRU/UlpeXh5VVVWpdJ/YbLaqvJ+mMDc3p6FDhxKPx6PLly+/QBrRGpq+c/L5fOzduxcjRox4Yd+Kw+HA398fP//8Mx4+fKi2Jqyrqytu3LgBuVyOtWvXvvI7Hh4euHDhguq9YMOGDTq9F4wdOxbFxcVoaGjA3LlzdZIXYRgGM2bMQHJystZ7vO2Zi4sLBAIBjh49qlN5AwcHB1y+fBnV1dX48ssvYW5ujmHDhuH69evw8PDQ2F+fPn1w8+ZNyOVyiEQi7NmzBwEBAS+tF5iammLTpk0QCoW4cuXKa4v5vM5YLBYGDBiA3NxcbNu2TW/axc9bTEyMzqu1S5cuRXNzM3JzczFgwACNjtXbghCfz0dSUhKys7OxbNky+Pr6YtKkSfjqq6+QmpqKgoIChISEqKpOdWTK4FQyU8LCwmBjYwMzMzPY2dlh5syZuHr1KlpbW1ULN7pUMrOyssLevXshkUhw9uxZncs8uLi44MaNG1oTGNozCwsLfP/996itrcX06dO19sPhcBAbGwuhUIhDhw7BwcEBxsbG2LdvH27cuKFV8Sk2mw1/f3+kpaWpGFyPHj3CtGnTYGlpCaL/R04pLi5GY2MjfH19Ne74vr6+uHv3Lm7fvg0/Pz+96D49byYmJkhOTtYpOPl8vspHQUEBAgMDNWqnXldr/fz88PDhQ9TX16O6uhrV1dWorKzE/fv3MXnyZLUDk+jZ5vuBAwcgkUggl8vR3NysUpCvqKiAUCiETCZTCVxps4XwvAUGBqqocaNHj9ZJPIzFYmH16tU4duyYzlWqXmUeHh64e/cuCgoK0L17d639DBkyBEVFRUhLS0Pfvn1hbm6OefPmobKyElFRUVoXc2KxWPD19UVKSgqampoglUrR0NCA/Px8ZGVlqbYqysrK0K9fP40Di8vl4ujRo3j8+DH69eunV6E3pdnY2KCmpka1cn358mWNg9PCwkIVnDk5ORg0aNCfF5xEz5bHV65ciUOHDiEqKgo+Pj5a6+gEBwejqKjopaXttrY2NDQ04Pbt2xg2bJjON8LExATp6elQKBRIT0/Htm3bdCo45ObmhoSEBJ1rfL7OgoKCUFBQgOjoaJ38jBkzBqWlpUhKSsK+ffuQlJQEoVCIn376SSOWzauMYRg4ODhg7NixOHToEDIzMyESidDW1gapVIonT55g4cKFGr86GBkZYeHChaioqNCal6uOmZmZqfZA79+/r3U/W7NmDU6fPo0vv/xS49nY6+LvLyFTYmRkROPHj6cuXbq89Fl1dTXdvn2bCgoKdC5bbmFh8cJq8L///W+aNWtWuyvErwObzaYvvviCGIahmJgYvau8Ez1b/Tt06BBlZGRQdHQ0PX78WCs/3bt3p7Nnz1K3bt1ILpeTQCCg69ev0+HDhzXeQmkPlpaW1LNnT/Ly8iIzMzOSyWSUlpZGDx480Lgkg4+PD+3du5eIiIKDg18QrNY3PD09afTo0ZSRkUE3btzQaftDG+A1MiV6JyH8lc3c3BwtLS2QSCQq9om27zC9e/fGxo0bda7k3Z7Z2toiOjoacXFxOhUFYrFY8Pb2Rp8+feDv7w83NzcYGxv/6fejPevXrx8SExP1Wl3ur2p/6SfnfxPK8gBtbW065XAyDEMMw+itpMProFzaf9O/81cDi8UiAwMDkslkWu/r/l3wuifn/1xwvsVb/NXwuuB8K/D1Fm8Mtra2tHDhQnJycvqzm/K3xF+qPud/EywWi/h8PjEMQ2KxWCeG0Fu8DHNzc7p79y5JpVJ6+PAhlZSU/NlNegk8Ho9MTEyIxWLR06dP/+sLQR1C2wUhU1PT137G4XBgaWmpN7kHfRuHw8GwYcNQXFyM1tZW7Nq1S+f9U6Vff39/jB49Gv7+/jqXaDAzM0O/fv1euRjEYrFgYmLyRhgzupqTkxNOnz6NlpYWrFmzRms/pqamsLCwQPfu3dG9e3e4uLjA1tYWdnZ2Ou15GhkZISgoCNu3b0dDQwPq6+sRFRWll0WyLl26YOTIkRg9ejRGjx6N4cOHw8HBod326nWf08XFBVu3bkVgYOArO40ygVXfK5mdOnXCvHnzdCrrbmRkhOnTpyMjIwMSiURFoNCGJfPHm/Lxxx8jIyMDcrkcWVlZGDNmjNb+LCwssGzZMhQVFWHv3r0vfe7r64utW7dqTBV70+bo6Ihjx45BIpEgOTlZp1Vma2trODg4qJQrevfuDVdXV3h4eCAoKEirAdXb2xvff/89ioqK8Msvv2DlypU4fPgwSktLsXTp0nYfOh0Zj8fD0qVL0dTUBLlcDplMhubmZpw4cQLjxo177UCqt+B0cXHByZMn0dLSgi1btry0FcFisbBw4ULU1dXhwIEDenkiET1jEh09ehSlpaVa59wpB47s7GzI5XLcuHED4eHhePTokdbBqeTqXr9+HWVlZZBKpZDJZJDJZHjy5AlGjhyp1QzC29sbxcXFaG5ufimxmGEYhIWFoaGhAWFhYXq5vs9fI4ZhYGFhARcXF43UFjgcDubPn4/W1lZIpVJ4eXnprV2mpqYIDg7GoEGDYG5uju3bt2tUFYzNZmPixIl49OgRUlNTMWXKFNjZ2am2mRobG1FfX48LFy5g5MiRGm+xMQyDcePGIS0tTdUHpFIp5HI5pFIpEhISXsuN1ktwPp/w3J7agbe3N2pqalBZWak1A4fP58PW1lbVWSIiIiAWi7Ft2zatRzdHR0dkZGSo6IB+fn7g8Xiws7PTiHL4vI0YMQJVVVUqgSzliKn8f25urlask4CAANTV1WH79u0v0et4PB5Wr16NhoYGvZRe5PF46NKlCwYMGIClS5fixIkTKvUFPz8/tf1MmTIFMpkMdXV1mDlzpl5fawIDA5GXl6canPfs2aPRzGzy5MkQiUTYu3evql8pP7OwsMCGDRuwYsUKJCYmIiMjA3PmzNGIKx0UFITi4mJVQCr7gFwuh0QiwTfffANzc/NXHqtzcDIMg5CQEBQXF0MoFGLGjBmvbWjPnj1RXV2tteyDmZkZduzYgW3btsHExARDhgxBQUEBfvrpJ3h6empFHLCxsUFcXBxkMhmKiooQEhKiM4na2dkZJ06cgFwuV0l5Kv+t/L9IJMLKlSs1ekcyNjbGDz/8gKysLAwaNOilzz09PXH79m38+uuvr73hHZmDgwN8fHwwY8YMrFu3DgkJCcjLy0NRUREKCgqQlJSE6Ohotel9fn5+ePDgASorKxEeHq7XQsI8Hg8rV66ESCRCfn4+BgwYgFWrVqFTp05qHW9lZYXCwkKcOXPmlRxlhmFgb28PMzMzsNlsLFiwANnZ2fjqq6/Ufqc/deqUSmpHOUg3NjYiNzcXYrEYFRUVmDNnzivfa3UOTm9vb6SkpEAmk+HAgQPg8/mvbairqysEAgEUCgVOnz6t0Y3gcDiYNGkSKisrsWPHDlhaWuKXX35BRkaG1oFJROjbty/KysogFAqxceNGrTv18/bVV19BJBK1G5xSqRTnz59XewrGMAzmzJkDkUiEo0ePviSpyGKxMHv2bIhEIkRFRWm8IMThcDB37lxcvHgRqampEIlEyMzMxKxZsxASEqJSKHR3d1c7wCwtLXHs2DFIpVIcOHBA7+wjd3d33Lp1CwqFAt988w0sLS3h5uam9oLbypUrUVhYqHZhW6XUaXFxsdqpj4MHD8a9e/dUT85Hjx5h9erVWLNmDWprayGVSnHq1KlXDnY6B+eKFSsgl8tRWFjYoSSjgYEB5s2bh4aGBmRmZqodUMrcvdTUVOTk5MDHxweLFy9GbW0tPv/8c60D08TEBBs2bIBIJEJhYSFGjx4NMzMzmJubw8zMTCspTwsLC/z0008qXdyysjJIJBJIpVI8ePAAH3/8MRYtWgSpVIrGxkaEh4erNXX28fFBZmYmMjMz4evr+9I5W1paqjJVNE1NcnJywqFDh1BTU4Pa2lokJCRgyZIl8Pb21noKymKxMGHCBJSXl6OlpQXR0dGIiop6waZNm6bTQkufPn1QVlYGhUKBkSNHqn5X3eMvX76M5ORkjX7TyMgI58+fR0xMjFrf3717tyqDKisrC3379oWhoSHWrVun0pb64YcfXtnXdA7OrKwsNDc3Y9WqVe2OjAYGBjA3N8eMGTNQV1eH8vJy9O/fX60T9PLywtWrV1FRUYHVq1djyJAhSEpKQnx8PIyMjGBlZaVVepOvry8ePXoEuVyOX3/9FV9++SWSk5ORkZGBR48eYevWrR3KFz5v1tbW2Lp1K5qamtDa2oqDBw8iMDAQR44cwbVr11TvgcOHD0drayuampoQERHR4dTW0tISGzduRF1dHTZs2PDKJ8PChQshkUgQExOj0ROKxWJh2rRpkEqlaGtrQ2xsrNqC1O2Zo6Mj7t27ByVEIhFycnKQk5OD/Px8CIVCANB64YphGMycORPNzc2QSqWv3CHoyK5evYojR45ofNyYMWNw4cKFDr9nZmaGqKgoSKVSNDU1vaBT6+XlhdzcXMjlcuTm5qJv374vHf+6+FObhKDUBeJyucTn81WcTysrK+rVq5dK6qJr167Uo0cPeuedd8jY2JjMzc1p1qxZlJKS0q5/hmFo7ty5NGjQIGpoaKARI0bQlClTqGfPnlRXV0ezZ88ma2tr+te//kV5eXnqNpuIiNzc3MjZ2ZmIiNzd3alXr14kEokoIyODAgICKCIighoaGujbb79Vy9/AgQNp9uzZZGZmRiKRiL7//ntKT0+nzMxM4nK5L1RZY7FYJJVKSSwWt8sR5fF49Mknn9CcOXMoJSWFNm7c+BL319XVlUaOHElCoZDu37+vkdAXAHr48CGdPHmSRo8eTR999BEJBAJV5oc2YLFYNGLECHr//feJiCg5OZkSEhJUEh0GBgY0ffp0+vTTT8nDw0Or3+DxePTxxx+TmZkZFRcXU35+vsY+ampqyMvLS+PjGhsbO2Q38fl8CgsLoyVLlpBYLKZjx45RZmam6vPhw4eTra0ttbW10Z07dygnJ0ft31c7OO/fv08hISH02Wef0ZgxY1Qlyw0NDcnOzk4lvmVoaEgmJiZE9OzmVVdX0+HDhzv039bWRhcvXqTm5mbi8/k0YcIEsrW1pby8PHJwcKDRo0fT7du3tSqGw+VyycDAgFgsFnXp0oXy8/Np3bp1lJKSQkuWLKHPPvuMPvroI1q/fn2HJGtfX1+KiopSFVhis9nk5eVFAoHgJX0fpc4Sn8+nzp07E8Mwr9URMjMzo9DQUKqrq6Nvv/2Wnj59SiwWi2xtbWngwIH0wQcfUFBQEHXv3p1u3LhBV65c0fg65ObmUmRkJJ07d47++c9/kre3t8Y+ngfDMLRq1SoiIvrmm2/o0KFDVF1drRo0LC0t6dNPPyUi0pq87uXlRf7+/iSVSunMmTNUUVFBRM8YSCKRSC1dpl9//ZX69OlDjo6OHSoFPg9ln24PAMjZ2ZmMjY1JKpVSZWXlC+lt2dnZJBQKydjYmJydncnW1lb99Dd1p7XW1taIj49HQUGB6p2loqICBQUFKC0thUAggEAgwM8//4zIyEjExcVBKBRiz549Gr1zstls9OzZEzk5OTh27Bjs7e3B4/HA4/G0ZoVMmzYNQqEQIpEIRUVFmDJlChiGgZWVFfbs2QOFQoFt27Z16MfJyQnXrl2DVCpFS0sLMjIyUFpainv37mHMmDGq9pmYmKBv3764f/8+xGIxzpw5A09Pz3Z929nZIScnB8nJyRgxYgQiIyOxb98+lJSUoLm5GQ0NDark85kzZ6p97jweD9bW1i+87/bv3x+lpaUd6s12ZBwOBy0tLQCAqVOnqv5uYGAAGxsbLF++HBKJBIWFhRrtST5vU6ZMgUKhQH5+Pnr37g2iZ6vZwcHBai/q2djYoKSkBHv27NFoe2ThwoVIS0tr9zuWlpb44Ycf0NLSgmvXrr20GhwUFITS0lLU19cjPj7+lf1A52ltXV0dhYWFkYeHB3l5eRGHw6HGxkYqLi4mS0tLevDgARERKRQKAkBhYWE0ceJEevTokdqjJvBMze2jjz6i1tZW2rx5M1VVVanbxA6Rl5dH+/fvp99++42IiMaPH0//93//R0KhkK5evdrh8QYGBsTn80mhUNCvv/5KmzZtIj6fT8bGxlRTU0MMwxCLxaJevXrRrl27yNPTk65evUqrVq2irKysdn23tbVRQ0MDBQQE0NmzZ4nD4VBdXR0VFBTQoUOHyMjIiCIjI+nf//43HTlyRO1z9vPzo/DwcIqOjqaMjAzicDj04YcfkqGhYYdt6ggAqKamhhwdHemdd94hPz8/Mjc3p379+lGvXr1ozJgxdOPGDYqMjNSaW9u7d28ienbvlKp7HA5HbelVomfawXv37qWIiAgqKiqimJiYDl8JDAwMaPDgwaq+8jrY2dmppuwCgeCF0oJERGPHjiUHBwf6z3/+QytWrNCssLSmDCF1jM1mqxgsmuxzslgsDBo0CGlpaYiMjNRpVH/ehg8fjpKSEtTW1mLmzJlwcXHBJ598opLUiI2NVWsUdnNzw4MHD9DQ0IDZs2e/tMJpbGyMyMhIpKSkoKWlBZGRkWpv/xgYGCAoKAhffvklDhw4gPDwcIwePRqenp5gs9mYOnUqFAoFQkNDNVqpHDJkCAoLC5GYmIjvv/8ecXFxKCkpwfHjx3VmbykV7wGgpKQEAoEARUVFAIC2tjasXr26wxlDR5aQkIDS0lJMnjxZNTPhcDga82ttbW3xww8/oLy8HJ9//nm73zU0NMSMGTOQkZHRoVogj8dTEXPy8/MxZ84c8Hg8GBoaIiwsDHfv3oVMJsOjR4/QpUuXV/rQC0NIEwsODsa1a9c0kkK0sbHB8ePHkZiYqPYGs7o35sCBA2hpaUFDQwNKS0vR2NgImUyGq1evqr3/5ebmhvv370MkEuHMmTMYMGAAzM3NVRYVFYXa2loUFxcjJiZG49VQFosFAwMD1Wa48u/GxsaIjY1FYWEhevbsqZFPc3NzLFy4EImJiSgtLcXZs2fx9ddfq33O6viPjY1FfX096uvr8fjxY2zcuBF9+/bVaovqj9f7yZMnuHPnzms7tiZmamqKWbNmISsrCw8ePEBUVBTc3d1haWkJS0tLODs7Y/LkyXjw4AFycnIQFhbW4faXq6srTp8+rdouqa+vR2FhIQoKCtDQ0ACZTAaJRNKur/96cCrfdTTZP/Pw8EB2djZmzZqld6U1a2trxMbG4smTJygpKUFubi4SExM1Io7b2tpiz549KC0tVdHUSktLUVJSgpKSEojFYiQmJmL69Ok6Z6Q8bzY2Nnjy5AkePHigNQdYm6fNn21LlixBY2MjNm/erFfGUUBAAPbu3Yv79++jpKQEFRUVKktPT0d8fDwGDx6s9j308/PD6tWrVdxqJX1P+e/Kysp2ywDq/M6pKVpbWzXOjzM0NKSGhgZ69OiR3mU56urqaNGiRfT++++TlZUViUQiqq6u1ui9q6amhr799lvKysqi6Oho4vP5ZGFhQQCoqKiITp8+Tfv27aPExES9tl0qlarqrShXyTWFXC5/ocTD3wGdO3em7Oxs2rdvn14F1O7cuUP3798nR0dH6tGjh0q6hujZiramomcPHz6kzMxMEgqFtGTJEurUqRNJJBK6cuUKlZWVUX5+vmpNRhP8pWRK/Pz8KDo6mqKiorSuXfLfgIWFxUvbEE1NTVRZWUnNzc06aRO9CiwWS7Vcn5ubq3WA/t3g4uJCPB7vlbVn/oowNTUld3d3MjExIYVCQUVFRSQUCkkul7f7oMLfQUOIYRjicrkklUp1lsF8i7f4u+BvEZxv8Rb/i3hdcL4V+HqLt/iLQi/ByeFwyNjYWC260/+fwePx6Ouvv6Zz585R9+7d/+zmvMXfHDpHk6mpKX3++ed0/PhxGjhwILHZbH20643B3t6evLy8yMDAQO++u3XrRoGBgeTk5KTiF/8vwcjIiOzs7MjOzo4sLS3/7OaoDTs7OwoMDCR7e/s/uykvQKetFC6XS6GhofT111+TmZkZZWVlaZwt8d8Ei8Wifv360YIFC2ju3LkaZ7d0hNzcXKqqqiJDQ0OdroGHhwf16dNHlf1TV1dHly5d0qpo7OvAMAy5urrSu+++S++99x5VVVXRhQsXqKysTKPFOFNTUxo8eDBxuVzq1q0b+fv7E9Ezylx0dDQVFRXprc1EzzKMpkyZQkZGRrR9+3aqr6/X2efo0aNp586dtGHDBoqNjdUqueKNQFsSgomJCSIjI1FaWqoqahsZGfmXlcNU2qBBg5Cbm6sX7Z0/WkREBCoqKrBz506t1QBcXFxw5coVVFdXo7m5GUKhENXV1YiJiXlJFUEXGzBgAJKTk1FeXg65XI6mpib88ssvaoty8fl8eHt7Y/PmzSgtLUV5eTkaGhrQ2tqKuro6iEQinD17VmvC+6uMxWJh/fr1qnKR+mI5RUZGoq2tDRUVFRg+fPgb6XftxYVeGUIMwyAyMhJNTU1oaWmBXC6HWCzGnDlz9HpCLBYLpqamsLGxQefOndG9e3e4ubnpJDGizKqfN2+e3gYSFosFT09PJCQkICMjo0OliNeZsbExVq1ahdraWhUdTikepuQFaytE9sf7l5WV9VLJRYlEgi1btnSY0G5ra4sdO3agtrYWjY2NuH37NlatWoXw8HCEhoYiJCQEAoEAQqEQS5cu1cs15nK5mDJlCioqKtDY2Ihbt26hc+fOevEdGRmpugYTJ07Uy/VVqhf6+vpi7dq1KCsrQ1RU1CuzYvTKEGKxWDR06FCSSqV05coVGj58ONXU1Ohts5hhGHrvvfeoR48e1K9fP3JxcaFOnTpR7969qbGxkTZv3kybN2/WyndtbS3l5+fT2LFjKS4uTi8q33w+nyIjI6lHjx70zTffaE2gYBiGHjx4QFu3blXlhkokEtq1axfx+Xzq1q0bGRgYqJXD2B58fX2Jz+e/9PfGxkZKSkrqkI3j5uZGffr0IYFAQOfOnaOffvqJqqurVe3y9vZWFXpSJuXrij59+tD69evJ1taWnjx5QgsXLnwpA0RbmJubq3JvdQGHwyF3d3d69913adSoURQcHEwcDocsLCyIzWbTqFGj6NSpU2rHiVZXrq2tjbZv3052dnYkEAiod+/eJJPJdKZYsdls6tKlC40YMYLmzp1L3bp1I3Nzc6qvryeFQkEikYgsLCx0WgkVi8VUW1tLPXr00MvilfI9duTIkXTmzBk6duwYET1buXVwcKDm5ma1k2ufPn1Kly9fpmvXrqk6+tKlS4nH41FjYyPl5OTozA5ycHCgBQsWqBZsRCIR8Xg84nA4pFAoSCgUdpji9+TJE5o3bx6JxWLKzc194ftmZmY0bdo0cnFxoaamJrp//75O7SV6lr7l6+tLNjY2pFAo6PHjx1RYWKizX6JnC4Tjx48nomfXX9t3WDabTaGhoTRnzhxydXUla2tr1e6F8voUFha+lJDfHrQKTgB0/fp1YhiG2traSCwW69zRLSwsaMWKFfSPf/yD7OzsVJnlR44coejoaDI1NaWoqCgaNWqUTjmez03Z9QI+n09Lly4liURCp06dIj6fTxMnTqQxY8ZQr1696P79+/TFF19odFMUCgXZ29vTkCFDKDQ0lLhcLmVnZ78UCJqCw+FQSEgITZgwgVgsFtXU1FBMTAx5eHjQ9OnTiYjUeoI0Nze/spAvi8Wi7t2708iRI4nL5VJCQgLdvXtX6/YqYWtrS4GBgWRqakoVFRV07NgxrQoevwpDhw6lbt26EQDKzMykgoICjY7v1KkTDRo0iD799FPy9/cnMzMzUigUdOnSJUpNTSVvb28aO3Ys1dbW0vnz56m5uVl95/rISklLS9PpXYvD4WDFihUoKSlBamoqLl26hNjYWHz44YdwcHCAubk5oqOjIRQKcf78eZ3SyaytrXHkyBFkZmbqLOHI4XAQHh6O2tpahIeHw8vLC8nJyWhpaUFpaSmysrJQUlKCsWPHauTXysoKZ86cgVwuR3Z2NhobGyGXy3H8+HHY2dlp/R60aNEiCAQC1NbWIjY2FjNnzoSBgQH69OmjepeLiIjQuv6Ku7s7kpKS0NbWhpSUFJ1L2ivN399fJdC2aNEiva0VGBsbY8eOHWhtbUVbWxt+/PFHjdYzDA0NsWbNGjQ2NiIzMxPJycn44YcfsGjRIlhYWGDy5MkQi8VobW1FTEzMazNr3mjKWFpaGu7du/dKwV51O07fvn0xYcIEuLu7w8bGRpXaxDAMZs+ejaqqKsTFxcHLy0unm+Pi4oIbN24gPT1dKyW/583R0RECgQDnzp2Dv78/9u/fj6qqKhw6dAgTJkzA9OnTkZ+fr3FwDho0CEVFRUhOTkZgYCB27twJuVyuWhTSRiLU398fVVVVqpogz587wzCIiYmBQqHAvn37NJLyUJqBgQF27NgBsViM5ORklRSMrgHE5XKxdetWyGQyVFZWaqW+9zobPnw4CgsLVbsNS5Ys0SiljsPhINyhKwcAACAASURBVCQkBCtXrkTfvn3h7OysCkBbW1scOnQITU1NOHPmDDw8PF7r540Fp7OzMwoLC3H+/Hm9V7xiGEbVwRMTE+Hk5KSzSnvPnj3x6NEjHDt2TKecS4ZhMGvWLFRWViI0NBTLli1DbW0tzp8/D3t7e7BYLEycOBFXr16Fq6urRr4tLS1V5eHZbDZcXFxw9uxZKBQKHDx4UOMVW1NTU+zfvx8ymQyHDx+GpaXlS9+ZOHGi1sFpYmKCpUuXoqqqCrm5uRg7dqze+oKHh4cqgObMmaO3vE4ej4c1a9ZAIpFAoVAgIyMDAQEBGvsxMDB4ZT/aunUrqqursW7dug63k95YcAYGBqKurg6nTp0C0bNtha5du8LX11fnaaOLiwtOnz6NnJwcrafMf7S+ffuioqICycnJOt1oR0dHXLp0CZWVlZg3bx5ycnJw8uRJuLq6gsPhwNPTE5cvX9a66NIfO8CSJUsgk8kgFAoREhKi0fERERGor69HSkrKa2c3kyZNgkwmQ0xMjEYzCoZhMHjwYAgEAly5cgVBQUF6m3ZyuVxs2bIFUqlU54plfzRPT0/cunULbW1tEAqFWL58uV7abWBggKlTp6KyshKnTp1Sa6B7XfzpTN9zcHAgDodDfD6f7O3tyc3NjeLj4+nAgQPUs2dPrf06OjrS+vXr6Z133qHw8HC9rPoRPaOYcblcunbtmk4J3Z06daKuXbtSU1MTubu7k729PSUnJ5ORkRH94x//oH379tG7775Lubm5OrdZoVDQkydPqLS0lAwNDWncuHFqH6tc3TYwMKBjx469chnfysqKBg4cSPX19XT9+nWNVt1tbW1p6dKl5OLiQnFxcfTrr7/qLd2vT58+NHnyZFIoFJSamqo3hpShoSHNmTOH+vbtSwAoNTWV9u7dq5d29+7dm6KioojD4VBCQgKJRCLtneny5OTxeNi1axekUikaGhpw5MgRXL58GXK5XFXFqyMfrzIOh4N9+/ahpqYGc+fO1Zu0hoGBATZs2IDm5maMHTtWpyly165dcfnyZdTW1iI+Ph41NTVIT0/H7du3UVJSgnv37iEsLExvciXu7u5ITExEa2urRhv7rq6uSEhIgEQieSWjhs/nY9OmTWhqasKVK1c0Zt24u7urJFo+/vhjvZyr0jZu3AixWIyysjJMmzZNLwQMomfSoI2NjVAoFCgpKXmpxKK25uLighMnTqCqqgqLFy9W+/VA79NaAwMDhIWFqWpYKIv2SCQS3L17F4GBgVotuNja2mLt2rWoqanBpk2b9FoUx8TEBOnp6cjPz9dIeOxVxjAMwsPD0dDQgObmZlW5t6amJsTFxcHd3V3rQSUwMBARERGqaRaPx0NYWBiqq6tRUVEBKysrjTrMlStXIJFI4OPjA0NDQ7DZbHA4HLz33nuIj49HfX09bt26hd69e2s8tTMxMcHWrVtV9U7DwsLg6emJiRMnwtvbG8bGxrC2tsbMmTMRERGBiIgItVbbGYZBfHw8JBIJ1q5dq7d+YG5ujlOnTqkqgS1evFjnhUGiZ7VVIiMjIRaLcfr06XYLff3R9B6cdnZ2qpNsampCZWUlBAIBvvvuO1hYWGh1gmw2G9OmTUNJSQkuXryoN3qW0kxNTZGdnY2EhASdK1kTPStmdPjwYVy7dg3Xrl3DyZMnMWLECJ3fXZYtW4aMjAyEhoYiKCgIy5cvR01NDUpLSzV+32Sz2Vi+fDmEQiEqKytx4cIFLF68GEuXLkV6ejpkMhmSkpK0XmlXFp9KSUlRcWsLCwvR3NyM4uJipKWl4fbt28jPz8fly5dx+vRptRQEfX19kZqaioaGBr0o7xE92zpRioZJpVJcv3693VVUdc3IyAg7d+5EQ0MDHjx4oPHAr1f6HtEzzZyDBw9SWlqaSoI+OzubsrOztWYKOTo60owZM+jBgwe0YsUKvdGzlFAoFJSVlUV1dXV6yZxpamqiGTNm6KFlL6KxsZG6detGsbGx1NLSQra2tlRRUUHbt2+n5ORkjXwpFAq6d+8epaenk5+fHwUHB1NwcDAREVVXV9OBAwdo165dWlMvAZBAIKBVq1ZR7969yc/P7yUiw+PHjykjI4Nu3rxJQqFQLb92dnZkY2Oj+g19gM/nk5OTE7W2ttL58+dp3759emEaTZo0icaPH0+///47bdy4UX+aR/rY59SHmZiYICoqCqmpqRg0aJDOWyavMhaLBXd3d7i4uPyls2fc3NxQXFysKvR75MgRTJkyRWvCv3L1eP369SgoKMC9e/ewadMmhIWFaT3LeZUZGhrCxsbmJdNm2jhixAgUFxejvr5ebzMohmHg4OCAgIAArckcfzQXFxdcvXoVNTU1Wq/Mv1ESgq7GYrHQt29fZGdnv6Ds/b9qymwcc3NzmJqagsfj6W2Z//9r70ujojqzrvetophLmUXGiuAQwUjEAFFiJB0NOCRiNELURN/YtrRx1nZIVuxIa0w0tsPSOCyjOIWgr2iIii0xIpEhTiBBVECmZoZiKKaiKPb3w0W9IU7UkKG/Zq91flh4Tz1173Pufe559tnH0tKS5ubmlEgkBkuw/BpmZGREqVRKqVT6q9yoDWFisZgzZ85kZWUl582bp/O8/UMHp6enJ0+dOsWNGzfq1WS1x3rstzSRSMQxY8bwk08+oYuLi85+Hhd/fwj1PXd3dzz//PO4dOnSH6cKvQc9+I3AHmnM3waCIMDMzAytra092rs96BYeF5z/NXJ5giDAxsYGgwcPxnPPPQcbG5tfRS3Qy8sLycnJ8PHxMbjvHvx3Qa8ydbFYjCFDhuD555+Hs7MzVCoV0tLSkJSUpHe1viHh4OCAYcOGYfLkyQgICICZmRmSk5Px/fffIzY2FvX19Qb5nj59+mDFihWaPp09+GPCyMgIPj4+GDlyJKRSKW7evInExEQ0Njb+3kPrCl0TQiKRiK+//jqvXr3Kqqoq5uXlMScnh5mZmRw/frzeL9uCIDAgIIDHjh3jv/71ry42cuRIrXwFBwczMzOTSqWS7e3tbG9vp1qtZllZmUFFyUJDQzWdrPXtfflr2oABA7hjxw7N+Tx37hzffPNNisViWltbMzAwUKtu5J32e/+u7pixsTFXrlzJrKwsNjc3s6Ojg8XFxYyMjNS7ZaE25+vnnxk8WyuTyXjp0iVWVVXx3XffZe/evWljY8OoqCi+++67ev0AkUjEgQMHagpsOzo6NH0O1Wo1J02apJW/0NBQyuVyDWWrpqaGt27dYkNDA0tKSjh79my9S5wGDRrEY8eOMS8vr9sKdp0Xy97enjKZ7JF7hHZ2dpTJZOzXrx9dXFxoZ2en9yRavHixpn3huXPnuGjRIt67d4/h4eFcvHgxd+zY8djv8PDwYEBAAD/88EPeu3ePNTU1lMvlrK2tZUxMDMPDw2ljY2PQYDU1NaWjo6PmPPTr14/Ozs5aVxWZmZlx5cqVlMvlLCkp4RdffMHExES2t7ezuLiY27Zto0wm02usEonkkedOJBIxIiKCFRUVzM3N7VKeZtDgFASB8+bNo1qtZlxcXJfN8VmzZtHb25ujRo3SaVtEJBIxICCAaWlpXZThMjIyeOzYMTY3N+scnPX19YyPj2d4eDjd3Nz4+eefs7m5mdeuXdMqoH5p1tbWPHjwIOVyudYdua2srBgVFUWFQsHvvvvukVZTU0O1Ws2CggJevHiRy5Yt04q7+XNzd3fnxYsXGRUVxaFDh9LKyooSiYQNDQ0sLy/nvXv36O3t/djjc3NzWV5ezpSUFB4+fJh79+7lN998w7S0NN6+fZvNzc3cu3ev1o2DHxdMAwYMYGRkJK9du6ZRLCDJvLw8rlu37olj/aUFBATw1q1bbG1t5d/+9jeuXbuWZWVlzMjI4O3bt6lSqXjixAmdA9TU1FRzg/sl6d3Dw4MJCQlUqVQPqUQYlL4nCAL69+8PlUqFixcvoqWlRfO38+fPY+PGjfD398ecOXPwww8/dNuvSCTSCFA9//zzAB7Qz6qrq/H5559DEASMGzdO6/Hm5ORgx44dKCoqwoULF1BUVAQA2Lp1K5ydnTF16lRMnjwZWVlZWvsWi8WYNWsW3nzzTRw7dgy7du3S6niVSoWDBw8iNzcXnp6eAB7cMCsrKzU0sD179kAqlQJ4QHGcO3cuLl26hOvXr2s93qCgIAwePBhr1qxBRkYGLCwsMHv2bBgbG8PCwgJ3795FSUnJY49fsWIFVCoVcnNzUVBQAKVSCXt7ezg4OMDX1xfbt2/HCy+8AFtbW710fhwcHDBv3jwEBQXhpZdeQl1dHWpra3Hp0iVYWFhg3LhxWLVqleb/PQ1mZmYYP348+vfvj7NnzyI5ORn/+Mc/kJaWhg0bNsDU1BTLli3Da6+9BoVCgRUrVjy1RM3IyKhL2aGLiwvef/99eHh44Ntvv9WIlguCgBdeeAEvvPACysrKcOTIke5pSun65AwPD6dKpdIQmU1MTDRC0wqFggqFghMmTNDqzmNjY8M9e/ZQoVBQrVZTpVLx9u3bfPXVV+nk5MTo6Gi2t7dr/eQUi8U0MzN7iBEjEokYHh7OmpoapqWl6VTeNWLECMrlcp48eZIymYyCINDJyYkzZ87kokWL6Ovr2y2/JiYmtLKy0tjjqjAiIiJ47949nar2AXDKlCksLy9ndHQ0TU1NOXfuXJaVlVGlUrG0tFSn0iwjIyNOnTqVWVlZrKur49q1a/Umk4wYMYJZWVlUKpXcu3cvn3vuOXp7e9Pe3p4+Pj5MTEykUqnkunXruuXP0dGRycnJbG9vZ3BwMB0cHLh3716+8847FIvFGpZaTk4O5XI5ly1b9tRl88+vga2tLb/66isqFAoePHiwy2rS0tKS586d45kzZ7hw4cKHqooM/s7p4eHBy5cvs6ioiIWFhZrKDLlczsrKSubm5mqduDE3N+ff//53KhQKqlQqnj9/nv7+/jQ2NqaLiwsTEhJ0eud8krm6uvLMmTOsr6/n0KFDtT4+NjaW5eXlfPPNNwmA3t7evHTpEjs6OlhaWkq1Ws0dO3YYRF5DIpEwPT2dd+7c0Wo598tA6rwBXr58mbW1tWxra+OJEyfo7u6uU3LMz8+PN2/epEKhYHNzM2/cuMGFCxfqXE0ilUq5bds2KhQKFhUVMTQ0lFKplBYWFgwLC+OdO3fY1NTEzMzMbiu0v/zyy1Sr1bx3756mEuVR78Uvv/wy6+vrmZ2dzWHDhnXLt6enJ7/++msqFArGxMRw0KBBmr+ZmJho3nOPHj3K4ODgh2h+Bg9Oa2trTpw4kePHj+f27dsZExPD3bt3c+3atXz//feZmJhIf39/rS6Kg4ODRitHLpd3efIGBQWxvLyc7e3tfOONN/Se6J0mEom4ceNGyuVyrYPT09OT9+/f5759+2htbU1vb29+9913VCgUjI6OZmhoKG/evMmkpCSDtA4ICAhgfn4+9+3bp1cCKzAwUPMe2zmh9CnLcnNz4/Lly7ls2TIePnyYWVlZrKmp4YkTJzhixAitA37YsGHMz8/nl19+yXfeeYcDBw5kQEAAhw0bxoqKCqrVat64cYMjR47s9nnoDM7Y2Ngnvg9bWFjwwoULrK6u5oQJE56a2OqU0qmurubOnTsfus4hISEsKCigQqHgpk2bHlm8YNDgNDExYWhoKF1dXQk8qGt0cHCgpaWlpgIiLy9P6y0VT09PJiUlsbq6mu+//36XaoapU6dSrVYzJSVFa8GszjH7+/tz1KhRHDVqFAMDAzUnMjIykrW1tXzttde67c/a2ppffPEFGxoa+PLLLxMAP/roIyoUCu7Zs4f29vYUi8VcvXq1QYLT3NycmzdvZkNDAwMDA3X2Y2ZmpincbmpqYlJSks5P4V+eX5FIRHt7ewYEBPCzzz5jfX09z5w5Q19fX62WyosWLaJcLufy5ctpaWlJU1NTWllZcezYsWxqamJTUxOXLFmi1WvIihUrqFaruWHDhidWyYjFYk17htmzZz/xxmJvb8/Dhw+zvr6eH3/88UO9bOzs7HjgwAG2trYyISHhsfPWoAmhV155BevXr8fcuXNRXFz80CZ+U1MT8vPztZLi7927NzZu3Ag/Pz98++23OHbsWJdEUyfKy8u1UuV2dnbG9OnT4evri6CgIJiammr+1tLSAoVCAWtraxgZGWlFRujXrx9GjhyJK1eu4MqVK5DJZBg3bhzKysqwaNEitLa2aoS28/Pz9RZB7tevH0aMGIGCggLcvn1bJx8mJiZ4++23ERYWhpkzZ+KVV17BG2+8AWNjY73GBkDT1qKqqgpVVVW4efMmqqqqsHTpUmzfvh0rV67sdnIwJSUFTU1N+Nvf/obQ0FAkJiZqEjomJia4evUqzp8/j7a2tm6Pz8PDA8CDBOOTKKsku63QP3/+fLz11lsQiUSYPn06Jk2ahOzsbADAoEGDYGFhgT59+qC5uRmxsbFaC1brFJydreked3KUSiWqqqrQq1cvjSr80zBz5kxMnDgRlZWV2LVrV5cAFAQBLi4uUCqVuHHjRreLuQVBwKpVqxAREaFh7HReGEEQYGFhAVtbWwBAe3s7Ro0ahbS0tCdePOBB68OxY8diyJAhWLlyJdrb2+Hm5gY7OzscP34cra2tkEqlCA4Oxttvv419+/Z1+4I/CiYmJpgyZQqGDBmC9957T6eWAYIg4LXXXsPkyZOxf/9+XLx4EXV1dZgwYQLGjRuH9PR0g3KBlUoltm7diqqqKqxfvx6RkZH405/+1K3vuHnzJubNm4d58+Zh9OjR8Pf3R1VVFSwtLaFUKpGYmPjEjPKjkJaWhj//+c/w8vKChYXFY+eQiYkJvLy8oFQqH/lw+Dlqampw5cqVLp85ODgAACwsLPDMM88gJSUFx44dw+7du7UaL6BjcHbeaR93oi0tLdG/f3+YmJh0i8bW2eylqakJBw8exM2bN7v83crKCtOnT4dSqUR6enq3g3PgwIGYOnUqBEFAZWUlkpKScOvWLVRUVMDFxQVDhgyBl5cXPD09IZFIMHXqVPzwww9ITk5+ol8LCwv4+fkBgKaS/t69e6isrISnpyfCw8MxevRoTJs2DceOHcOJEye6Nd7HobOfR1ZWFlJSUnT2MX/+fMjlcly4cAEqlQpFRUXIzc1F//79u30T1QYqlQqnTp3CyJEjERoaiiFDhiAjI6Nbx8XHxyMrKwvh4eGwsbEBAISFheH+/fvYt2+f1pTLnJwcqNVquLq6wtzc/LErGWdnZ/zP//wP/v3vfyM/P/+J5+TIkSM4ffr0Q5/LZDJ8+OGHIIlPP/0UZ8+efeoN/1HQKTjlcjkcHR3x3nvvoaioCJWVlV0G9tVXX8HNzQ3l5eXd4tj27dsX48ePh1KpxLVr11BbWwtBECAIAuzt7fHBBx/Ay8sLP/30E4qLi7s9zr59+2r2B2/duoWvv/4aZWVlqKmpQWhoKJydneHk5ATyQe+XAQMGwNfX96nB2draqtnDevfddzW/38bGBsOHD8dLL70EMzMz/P3vf8eBAwf0LoOLiIhAv379EBkZqXOfmIULF2LQoEFYunSpZmIaGRmho6MDtbW1Ok2e7qCurg5nz57FuHHjuh2cwIPlZ0FBAbZu3YqhQ4fio48+QmNjIxYtWqSTtEhhYSGys7Ph5eUFNze3R84je3t7/OMf/4BIJMKBAwfw008/PdFnbW3tQysiS0tLrFixAi4uLpgzZw7S0tJ0v+npkhDq27evpifIuXPnGBgYSCcnJwYGBnL//v2sqanhhg0bui0N6OrqytTUVNbW1vLTTz/ltGnTOG3aNC5evJg//vgjW1pamJmZyVGjRmmV+QsMDKRCodDwaVtaWqhQKFhfX69Ry1OpVMzNzWVQUBC9vb27XTQ7ceJEyuVydnR0aEyhUHDHjh0MDQ2lg4ODQShs1tbWzMvL01mRHHiQsIuNjeWOHTs02U2JRMJp06YxNjZWb8ra0yw4OJilpaVctWqV1sfa29vz0KFDLCsr44IFC3RWG7Czs+OFCxeoVqu5atWqLkkhU1NTBgQE8NSpU2xqamJcXJxOTX9NTU25cOFC5uTkaEVhNTh9LygoiImJiWxpaWFeXh7j4uJYWlrK1tZWHjhwQKsMoJmZGTdt2kSVSvVQQ9f29nY2NTVxxYoVWp8sJycnHj16VEN07/TX+W+VSsVr164xJCRE64tuaWnJd955hxs2bNDYggULDKZT22nvvfce6+rquHPnTp20eARB4KRJk1hcXMwZM2bQ1NSUMpmMY8aMYXJyMteuXatXM+Kf2897hfzcwsLCWFJSonVjWrFYzI8//pi1tbXcunWrzpRF4AHh/S9/+QsrKytZW1vL5cuXc/jw4Zw+fTrXrVvHzMxMyuVyHj9+nIMGDdJ6PojFYk6fPp23b9/mpk2btJLyfFz86VxsLRaL0b9/f7z44ouYMWMGOjo60Nraim+++QYXLlxAcXGxVmVjoaGhWLNmDYYNG9bl85KSEqxduxanTp3SKani6Oj42NpKkigpKUF2dvYfqsStE2KxGP/85z8xY8YMLF68GIcOHdLah7GxMdasWYNly5bh6tWrqKiogJubGwRBwPnz57F79269Wip2QiqVYtu2bSgrK+uizi+TyTB9+nS4u7vj2Wef7XbWWiQS4Y033sC2bduQmZmJJUuW6N0C0dnZGdu2bcPrr7+OxsZGlJSUwNHRUZNoOnr0KLZs2YK8vDytfU+cOBHbt29HRkYGli5dqlVmlo8pttaZhNBpIpGIFhYWtLCwoLm5uc7lVxKJhDKZjNOmTWNcXBzz8/OZlJTE6dOn/6HLr35Nc3Bw4HfffcecnByOGDGCffr00clPJ/UxLy+PqampjIiIoLu7u0Gf8oIg0N3dnfHx8czKymJWVhZzc3NZVFTEpKQkhoWFaVWG5uPjw5SUFJ4/f17n5fyjzMLCgtbW1rSxseliVlZWerG4Nm7cyOzsbJ26oBmcIdRjv77Z2dnxX//6F5VKJePj4zl8+PDffUxPM5FIRLFYTLFYzEGDBtHPz09rNpONjQ1jYmJYV1dn0MD8NS0sLIzh4eE65RkMvqztwa8PiUSCoKAguLu7Izs7GykpKX/I5beh0atXL0ydOhWtra04e/asXnvE/wl43LK2Jzh70IPfGY8LTr00hB4FiUQCLy8vNDc3o6CgQCuKVQ960IP/g8Hk5wRBwMCBAzFnzhycPXsWkZGRGmrcfxNkMhmWLFmC0NBQmJmZ/d7D6cF/MgyVEHJ0dNQQEOrq6rh+/Xqt25f/p5uzszNPnz7NtrY2fv3111q16vutzcjIiOHh4YyLi+Pp06c5evTo331M/632q2VrOwkJFy5cYH19PdPT0zl9+nSD9Dw0pD2uZCkkJIRr167Ve7vG0tKSW7duZXNzs0ZRQF+GkCAINDU15eDBgxkVFcXc3FzGxMRw0qRJOjNlBEGgg4ODhnXT0dFBtVrN48eP/ybqc7pcN5lMxuHDhzMkJITe3t7s3bs3zc3NuXPnTn7zzTdd9HieZiKRiB4eHty8eTOLiopYVVXFmJgYvvbaawZvbuXu7s7Y2FhGRkY+MWP9qwRn7969GRwczOLiYpaWlvLTTz81SN9LQ5tUKuWoUaMeYm1IJBIuX76c+/bt04slIxaLOXv2bFZXV/Pu3bsMCQnR++bk5OTE0NBQrl+/ntnZ2czLy2NaWhrLysqYn5+v85PO09OTJ0+eZFtbWxcmVn5+PufOnav3uE1NTenp6cmgoCBOmzaNISEheq0g/P39mZqaqmF3FRYWMjY2lq+//joTEhKYkZHRrX6fwIOa2E6ZVLlczh9//JGXLl1ibW0tz507Rw8PD4POuzFjxrCiooItLS1PLGY3aD0n8IDkPXv2bEyePBm9evXCunXrcOjQIVRVVenq8iHY2dlh2LBh8PPzg52dHe7cuYP79+8jOztbKwL8wIEDNVUdP+/LKQgCJBKJ3srvffv2xZQpU2BtbY1PPvkE58+f16vCw8fHB2vWrMHo0aNRX1+PkydPIj4+HpWVlfDx8cG2bdsQGhqK69evd7vfJQCYmppiypQpGDNmjKbWVK1W49///jfc3Nwwd+5cJCcnP5Xw/UuYm5vDzs4OAwcOxKhRo/DSSy/Bw8MDzs7OqK2txYwZM3Du3DmtfAKAv78/Pv74Yzz77LOIjo7GTz/9BDs7O4SFheHatWtwcnKCSqXqtj8nJycsXboUTk5O+OSTT3Du3Dk0NjZi79698PT0hKurq07soEdBJBLBwcEBJiYmUKlUv11ViiAIGDx4MP7yl7/A1tYWu3btwp49e/RWzBaJRDAzM8OLL76IyZMn48UXX4SDgwOsrKxgYmIChUKB5uZmpKenY8KECd3+wc8++yzefPNNbN26tQt9zMjISO+klUQiwYIFCxAUFITz58/j8OHDepdeLVu2DK+88gqOHj2KQ4cO4c6dO2hqagLwIJgEQYClpaUmwLoLd3d3hISEwNzcHACQlJSEHTt2YOnSpXB3d0ffvn1hb2/fbX+CIGDevHmYOHEiZDIZpFIppFIpmpubkZeXB0dHR9TX12vGrg0kEgkCAwPh5+eHQ4cOYe3atVAoFHBycoKZmRmGDBkCV1dXREdHdzugSktLERMTg88++wxXrlxBS0sLevfuDZlMBrlc/lS1PW1gbm6O4OBgWFpaYufOnTr51ik4+/Xrh71798Ld3R1ffPEFNmzYoHNgdgaInZ0dgoOD8eqrryIwMBCmpqaaJxpJKJVKzcX39vaGm5sbCgsLu/UdxsbGyMrKemiSiMVimJub69U6wcvLC3/9619RX1+P3bt3o6OjAyNGjEDv3r2Rk5OD+/fvax2sIpEIEokExsbGaGlp0Tzte/XqhcWLF0OhUOD06dNalaKJxWKsW7cOgYGBAICCggLs3LkT8fHx+POf/6zV+H4+zpkzZ8LR0RENDQ04c+YMTp8+jatXr2LEiBGIjo7G5cuX36SZSQAAC2tJREFUcePGDa19m5mZQSqVoqOjA+np6WhoaEB7ezsqKytRXFyMt99+G6WlpZg3b163iRnNzc3Yv38/SEIkEmnIDr169cL27du1XjE8CaamppDJZKivr0dKSopWT3gNdHnnXLRoEVUqFYuLizlmzJiHEh9SqZTDhw/n0KFDn5q48Pb2ZkxMDPPy8tja2qqpGqmurua1a9cYFRXFqKgoHj9+XPO3kydParX2f++99xgXF/fQu4+trS1PnTrF/fv36/TOKZFIGBsby5aWFu7cuZODBw/mjh07WFFRwfb2dqampvJPf/qT1n5fffVVFhcXs729nQcPHqSjoyNtbGy4ZMkSlpeX88MPP9RaelImk7GoqEjzjhkWFqZJgMyZM4dtbW0sKSnRmhs6YMAABgYGsm/fvprPjI2NuWnTJsrlci5YsEDnRIufnx9zc3OZkZHBsWPHas5BTU2NptGyLn5NTEw4ZcoU7tq1i6WlpSwsLGRQUJBBE0L+/v6sqanhlStXNFpbjzODJYRMTEx46NAhtre3MykpiQMHDiTwIDXv4eHB119/nfHx8czMzOT//u//PjUZMHbsWNbW1jInJ4e7d+/myZMnGRkZybfeekuTmbOysuLChQvZ3t6utRCXIAiMiIh4ZHDa2dnx1KlTXLJkidb8T4lEwjFjxjA/P58XL16kt7c3V65cSYVCwerqalZUVFClUjEqKkqntgHvvvsuCwoKKJfLuWXLFm7evJl3795ldHQ0ra2ttfJnZGTEZcuWsaWlhS0tLYyKiupyMwoJCWFdXZ1OwfkoCwwMZFZWFjMzM7vIRGprnX1NVCoVU1NTeebMGVZXVzMuLo6jRo3SmbjfefPrLFFUKBRMSEjg2LFjDRKgJiYmnD9/PpVKJTdv3vzUG6nBgtPV1ZUXL15kU1MTIyMjaWpqyj59+nD9+vVMTk5mQkICs7OzqVarWVNT89S6zj59+nDMmDHs27cvTU1NaWFhQWNj4y4nqVevXoyLi6NCoeDOnTu1esr17t2bhw4dYnx8PAcMGEBPT09OmTKF77//Pj/88EOmpqbqNCEdHR0ZGxvL5uZmzpkzh8OGDePNmzdZVlbGadOmcezYsbxz5w5v3rypk1qgqakpFy9ezOrqak2R+IkTJ3QqAra3t+e1a9eoVqt5584djhgx4qHvSkxMNEhwCoLAVatWsbm5mV9++aXe20menp68ceMGVSoVVSoVDx8+TAsLC738ymQyJiUlMSoqiqNHj+b8+fN5//59xsXF6SUR2mm+vr5MT09nampqt4j7BgvO0NBQlpeXs62tjcePH2dAQABPnjzJzMxMzpo1ixKJhKGhoWxra2NSUpLWd/lfmlQq5ZYtW1hZWcmoqCitJ2ffvn2ZlpbGqqoqpqens7CwkMnJyTx79iwvXbrEuro6Tp06VacLfPHiRSoUCvr4+DAyMpLt7e3csWOHppNUREQEMzIydJbFtLOzY2xsLNvb25mZmalzVUpISAhLSkqoVqu5Z8+eh/YzBUHgkiVLDBKcjo6OjImJYVVVFX18fPTyJZFIGBwczJycHE1xvCE62P3SjI2NuXr1aiqVSkZEROjlSyQS8YMPPmBTUxO/+uor9u7d+6nHGGwr5e7du6ivr4e9vT2Cg4MxYMAA5OfnY9myZUhLS4Ofnx9MTEwgl8vR2NioV+bS3t4eERERCAsLw9WrV7F+/XqUlpZq5aOmpgb79u3D2LFjcf36daSnpyMvLw/V1dWQyWQ4ePAgysrKtB6bi4sLXFxc0NbWhoKCAhgZGUEQBCQmJoIkXFxc8MYbb6CiokKnZJmpqSnGjx+P5557DgDQ2Nio1bbJz9G/f39IpVIolUqUl5c/dE2cnJwwY8YMyOVyvXqVCoIAPz8/vPTSS7h8+bJOvWd+jiFDhuCjjz6CVCrVNCRetWoVrl692kW3Sl+oVCrcu3cPjY2NcHFxgSAIOhd129raok+fPqirq8ONGzeequD3JGgdnLm5uUhOToaHhwfMzc3h7e2Nzz//HCkpKQgPD8fy5cthbGyskbjUJ4v71ltvYfny5aioqMDq1atx7949rf20tbXh2LFjOHXqFOrr67tkzTo6OjSmLczMzGBmZobvv/8era2tmos5dOhQ3LhxAzNnzoStrS0iIiK0nkiCIGD48OH4+OOPNVKLhkBtbS1SU1M156BzS2bDhg0YMGAAjhw5ohEu0wVisRgeHh6wt7dHYWGhXqoFAQEB+Oijj+Do6IhVq1ahsbERW7ZsgaOjo9ZbSE8DSbS2tuq9BWZkZITg4GBMmjQJKSkpiI6O1q/wQ5dsrY+PDwsKCjQsE7lczvz8fNbV1VEul/Obb77h1KlTtdJR+bl1Ns5VKBTMzMxkQEDAr9KcdcaMGczOzn7oHaw7FhQUxMLCQl6+fJkymYwLFiygXC6nUqlkYWEhT548qbOS+pgxY3j16lVeuXKFoaGhLCgoYHJysqbHh7a2cOFCNjQ0sKysjG+99RalUim9vLw4ZcoUHjx4kJWVlUxISOi2uNnjzMnJiZmZmaysrKSvr6/OfhwdHfn111+zvLycU6ZMoUQi4cKFC1lXV8cvv/zS4PPA3Nxco2G1evVqnefa8OHD+f333/PcuXOaRGl3zKD0PbFYzODgYO7bt48ZGRmaIL1+/Trnz5+vc1ACD9bs/v7+vHLlCgsKCjhnzpxfjUDfSeXSJTgHDx7MlJQUqlQqJiQk8Nq1a1SpVFQqlTx69KhOiRvgQfImLi6O9+/f56RJkzQqfzExMTr3vOwMzvb2dl6/fp07d+5kYWEhGxoaNO9ys2fP1vt8vvPOO1Sr1Tx48KBeuYYpU6ZQoVBw165d7NWrF3v16sUtW7awpKREr1YUwIMbv42NjWZOCYLACRMmMCsri8XFxTorL4jFYn722WcsKyvTdC7r7rEGpe+p1WrEx8cjNTUVMpkMzs7OEIlEyM/Px927d3XbcMUDVkhISAjWrFmDZ555BosXL8a3336rE8OkO6iqqurSX1Eb5Ofn49ChQ5DJZAgKCkJbWxuSkpJw/PhxjcCZLpgwYQICAgI0y8tZs2ahtbUVW7Zs0bmlQ01NDZRKJSwsLODj49NF8KykpASbN2/GyZMndfL9cwwcOBD19fX44Ycf9Hp37RQkt7W1xcsvv4zRo0cjLCwMt27dekhwXFuYm5vjn//8J+rr63H06FH4+/vjr3/9K9zc3LBp06Zu6+r+El5eXhg5ciTOnDmDS5cuGUaxQt+qFEOZIAgcPnw4k5KSmJGRwdDQUK37RGpr/fr1448//qjTkxN4kOWztbWlg4MD7e3t2atXL52rRTpt8eLFrK+vp1KpZHV1Nauqqnj69Gm9lvWWlpZcv349i4qKWFxcrLFOzWFDtCcUBIHx8fFMTU3VWwd3xYoVbGpqYnNzMysrK1lcXMwDBw5w0KBBer/eSKVSZmVlsbW1lTU1NayurmZGRgZnzZqlM0HfysqKV65c4dmzZ3X67QYnvhsSgiDA19cXn376KRobG7FixQr8+OOPBm8P8Es0NDQgNjZW5ydSW1ub3g2Kfono6Gi4urrC1dUVpaWl+P7775GQkKBXcqWxsREffPABPvjgAwOOtCt69+4NLy8vtLS04JlnntErIbR//36IRCL4+vri/v37iI2NRUZGRrfbcDwJra2tOHLkCHx8fCAIAlJSUnDmzBmdko2daG9vR0pKCo4cOYKCggK9x6jB7/3k7Ez+nD9/nqtXrzZIO7oe++3NxMSEaWlpvHPnDkNDQw1eG/n/sxk0IWRIGzx4ME+cOMHw8PA/XIF2j2ln/fr1o6enZ8911NIeF39PVN/rQQ968PvBYAJfPehBDwyLnuDsQQ/+oOgJzh704A+KnuDsQQ/+oOgJzh704A+KnuDsQQ/+oPh/NjvJdMuHKhsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt_img = np.zeros((280, 280))\n",
    "for i in range(10):\n",
    "  for j in range(10):\n",
    "    plt_img[i*28:(i+1)*28, j*28:(j+1)*28] = np.squeeze(x_train_mnist[i*10+j])\n",
    "plt.imshow(plt_img, cmap=\"gray\")\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "83RW1--_mTH7"
   },
   "source": [
    "Similar to the previous exercises we again derive a class for our model from the Keras Model class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tmk7DlUWoq5X"
   },
   "outputs": [],
   "source": [
    "class MyModel(k.Model):\n",
    "    def __init__(self):\n",
    "        super(MyModel, self).__init__()\n",
    "        self.conv0 = k.layers.Conv2D(8, 3, 2, activation=\"relu\")\n",
    "        self.conv1 = k.layers.Conv2D(16, 3, 2, activation=\"relu\")\n",
    "        self.flatten = k.layers.Flatten()\n",
    "        self.dense0 = k.layers.Dense(128, activation=\"relu\")\n",
    "        self.dense1 = k.layers.Dense(64, activation=\"relu\")\n",
    "        self.dense2 = k.layers.Dense(10, activation=\"softmax\")\n",
    "\n",
    "    def call(self, inputs, training=False):\n",
    "        output = self.conv0(inputs)\n",
    "        output = self.conv1(output)\n",
    "        output = self.flatten(output)\n",
    "        output = self.dense0(output)\n",
    "        output = self.dense1(output)\n",
    "        output = self.dense2(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1yVUfCeso4TS"
   },
   "source": [
    "We can now instanciate an object of this class and compile it using the the cross-entropy loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1-hWI4rVpvdU"
   },
   "outputs": [],
   "source": [
    "mdl = MyModel()\n",
    "opt = tf.keras.optimizers.RMSprop(learning_rate)\n",
    "mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=opt, metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bzMR_g1-qHTx"
   },
   "source": [
    "Now we are ready to train the model and log the metrics for plotting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 770
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 56206,
     "status": "ok",
     "timestamp": 1562670699284,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "V1vU1bnnqJZG",
    "outputId": "35bf9019-41d2-4cee-af2c-af6614d89d6c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.5560 - accuracy: 0.8834 - val_loss: 0.1328 - val_accuracy: 0.9612\n",
      "Epoch 2/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.1062 - accuracy: 0.9674 - val_loss: 0.0837 - val_accuracy: 0.9743\n",
      "Epoch 3/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0671 - accuracy: 0.9789 - val_loss: 0.0800 - val_accuracy: 0.9760\n",
      "Epoch 4/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0462 - accuracy: 0.9854 - val_loss: 0.0754 - val_accuracy: 0.9803\n",
      "Epoch 5/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0364 - accuracy: 0.9887 - val_loss: 0.0839 - val_accuracy: 0.9788\n",
      "Epoch 6/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0276 - accuracy: 0.9913 - val_loss: 0.0745 - val_accuracy: 0.9817\n",
      "Epoch 7/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0214 - accuracy: 0.9933 - val_loss: 0.0898 - val_accuracy: 0.9815\n",
      "Epoch 8/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0185 - accuracy: 0.9940 - val_loss: 0.1065 - val_accuracy: 0.9793\n",
      "Epoch 9/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0159 - accuracy: 0.9953 - val_loss: 0.0880 - val_accuracy: 0.9818\n",
      "Epoch 10/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0138 - accuracy: 0.9957 - val_loss: 0.1057 - val_accuracy: 0.9822\n",
      "Epoch 11/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0136 - accuracy: 0.9956 - val_loss: 0.1042 - val_accuracy: 0.9832\n",
      "Epoch 12/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0107 - accuracy: 0.9967 - val_loss: 0.1228 - val_accuracy: 0.9815\n",
      "Epoch 13/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0113 - accuracy: 0.9966 - val_loss: 0.1637 - val_accuracy: 0.9767\n",
      "Epoch 14/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0106 - accuracy: 0.9970 - val_loss: 0.1178 - val_accuracy: 0.9832\n",
      "Epoch 15/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0096 - accuracy: 0.9971 - val_loss: 0.1158 - val_accuracy: 0.9833\n",
      "Epoch 16/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0083 - accuracy: 0.9977 - val_loss: 0.1251 - val_accuracy: 0.9832\n",
      "Epoch 17/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0084 - accuracy: 0.9976 - val_loss: 0.1597 - val_accuracy: 0.9833\n",
      "Epoch 18/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0090 - accuracy: 0.9976 - val_loss: 0.1518 - val_accuracy: 0.9832\n",
      "Epoch 19/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0091 - accuracy: 0.9978 - val_loss: 0.1664 - val_accuracy: 0.9823\n",
      "Epoch 20/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0094 - accuracy: 0.9975 - val_loss: 0.1777 - val_accuracy: 0.9817\n"
     ]
    }
   ],
   "source": [
    "history_no_dropout = mdl.fit(x_train_mnist, y_train_mnist, batch_size, epochs, validation_split=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4OhKjUnwtr3M"
   },
   "source": [
    "Visualizing the training process in a plot is possible by using the history object that contains a history dictionary. This is returned from the fit function and contains all the metrics logged over the training proces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 56759,
     "status": "ok",
     "timestamp": 1562670699849,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "bPuH5kBQt5yw",
    "outputId": "89b7a021-1e31-41b4-80b2-c4e3be1c2fb9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXiU1fnw8e89M8mEJBPWJOyLsomgIBF3XGvVqlg3VFxwqa2tS2tra2sXa+1Vu9n+3vdntVZUVFQQrVKlahcUfauWgGFXRAQJa9jCEkKSmfv94zxJBgzJTDLJMDP357rmmmc7z3NmMrnnzHnOIqqKMcaY1OdLdgaMMcYkhgV0Y4xJExbQjTEmTVhAN8aYNGEB3Rhj0oQFdGOMSRMW0M0hTUT+LiLXJfpYY9KRWDt0k2gisjtqNRfYB4S99a+r6rSOz5Ux6c8CumlXIrIauElV/9nEvoCq1nV8rtqXiPhVNdzykcYkllW5mA4jIqeJSLmI/EBENgJPiEhXEXlVRCpEZLu33DcqzVsicpO3PFlE3hWR33nHfiYi57by2EEiMldEdonIP0XkIRF5poV8/0hEtojIahGZFLX/SRF5WERmi8ge4HQROcLLzw4RWSoiF0Yd30lEfi8ia0Sk0stnJ2/f8SLyHy/dQhE5LSrdZBFZ5eX5s/o8iMhgEXnbO9cWEZne5j+WSUkW0E1H6wl0AwYAN+M+g0946/2BvcD/NpP+OOBjoAfwG2CKiEgrjn0W+C/QHbgXuCaGfPcA+gDXAY+KyLCo/VcBvwRCwAfA34A3gSLgNmBa1PG/A8YCJ+Lei+8DERHpA7wG3O9t/x7woogUikge8H+Ac1U15KUt8873C+9aXYG+wP9t4bWYNGUB3XS0CPAzVd2nqntVdauqvqiqVaq6CxcUT20m/RpV/YtXpTEV6AUUx3OsiPQHjgV+qqo1qvouMCuGvP/Ey/fbuMB7edS+V1T1/6lqBBgN5AMPeOf/N/AqcKWI+IAbgDtUdZ2qhlX1P6q6D7gamK2qs1U1oqr/AEqB86Leu5Ei0klVN6jqUm97Le4LsbeqVnuvx2QgC+imo1WoanX9iojkisifveqHncBcoIuI+A+SfmP9gqpWeYv5cR7bG9gWtQ1gbQv53q6qe6LW13jnaSp9b2CtF9yjj++DK+XnAJ82cY0BwGVedcsOEdkBnAz08q49EfgGsEFEXhOR4V667wMC/Ner3rmhhddi0pQFdNPRDrwL/11gGHCcqhYA473tB6tGSYQNQDcRyY3a1q+FNF29ao96/YH1UevRr2s90M8rjUcfvw7YAlQDhzdxjbXA06raJeqRp6oPAKjqG6r6JdwvjY+Av3jbN6rq11S1N/B14E8iMriF12PSkAV0k2whXL35DhHpBvysvS+oqmtwVRn3iki2iJwAXBBD0p97x58CnA+8cJDjPgCqgO+LSJZ3Y/MC4Hmv1P448KCI9BYRv4icICJB4BngAhH5src9x7sh21dEikVkgvelsg/YjauCQUQui7qRvB335RLBZBwL6CbZ/gh0wpVc3wde76DrTgJOALbibkJOxwXKg9mIC5brgWnAN1T1o6YOVNUaXAA/F/e6/gRcG3X894DFwDxgG/BrwKeqa4EJwI+AClyJ/S7c/6kPuNO7/jbcfYZbvPMdC3zgtf+fhaufXxXHe2HShLVDNwbwmvp9pKpf+IXglbCfUdW+X0hozCHESugmI4nIsSJyuIj4ROQcXMn45WTny5i2CCQ7A8YkSU/gJVw79HLgFlX9MLlZMqZtrMrFGGPShFW5GGNMmkhalUuPHj104MCBybq8McakpPnz529R1cKm9iUtoA8cOJDS0tJkXd4YY1KSiKw52D6rcjHGmDRhAd0YY9KEBXRjjEkT1g7dGNOhamtrKS8vp7q6uuWDM1hOTg59+/YlKysr5jQW0I0xHaq8vJxQKMTAgQM5+NwkmU1V2bp1K+Xl5QwaNCjmdFblYozpUNXV1XTv3t2CeTNEhO7du8f9K8YCujGmw1kwb1lr3qOUC+jzVm/j169/hA1ZYIwx+0u5gL6ovJKH3/qUyr21yc6KMSZF5ecfbNbC1JZyAb0oFARg867m5iIwxpjMk7IBfdNOa/JkjGkbVeWuu+5i5MiRjBo1iunTpwOwYcMGxo8fz+jRoxk5ciTvvPMO4XCYyZMnNxz7hz/8Icm5/6KUa7ZYVJADwOadVkI3JtX9/G9LWbZ+Z0LPOaJ3AT+74MiYjn3ppZcoKytj4cKFbNmyhWOPPZbx48fz7LPP8uUvf5l77rmHcDhMVVUVZWVlrFu3jiVLlgCwY8eOhOY7EVK2hG5VLsaYtnr33Xe58sor8fv9FBcXc+qppzJv3jyOPfZYnnjiCe69914WL15MKBTisMMOY9WqVdx22228/vrrFBQUJDv7X5ByJfS8YID8YIDNu6zKxZhUF2tJuqONHz+euXPn8tprrzF58mTuvPNOrr32WhYuXMgbb7zBI488wowZM3j88ceTndX9pFwJHVwp3apcjDFtdcoppzB9+nTC4TAVFRXMnTuXcePGsWbNGoqLi/na177GTTfdxIIFC9iyZQuRSIRLLrmE+++/nwULFiQ7+1+QciV0gMJQ0Eroxpg2++pXv8p7773H0UcfjYjwm9/8hp49ezJ16lR++9vfkpWVRX5+Pk899RTr1q3j+uuvJxKJAPCrX/0qybn/oqTNKVpSUqKtneDi9uc+ZGH5Dt6+6/QE58oY096WL1/OEUcckexspISm3isRma+qJU0dn9JVLtZb1BhjGqVmQC8Isrc2zK59dcnOijHGHDJSM6CHrC26McYcKDUDekF9W3S7MWqMMfVSM6BbCd0YY74gNQO6ldCNMeYLUjKgh4IBOmX5rYRujDFRUjKgiwhFBUEbz8UY0+6aGzt99erVjBw5sgNz07yYArqInCMiH4vIShG5u4n9k0WkQkTKvMdNic/q/opCQRtC1xhjorTY9V9E/MBDwJeAcmCeiMxS1WUHHDpdVW9thzw2qSiUw/INiR120xjTwf5+N2xcnNhz9hwF5z5w0N133303/fr141vf+hYA9957L4FAgDlz5rB9+3Zqa2u5//77mTBhQlyXra6u5pZbbqG0tJRAIMCDDz7I6aefztKlS7n++uupqakhEonw4osv0rt3by6//HLKy8sJh8P85Cc/YeLEiW162RDbWC7jgJWqugpARJ4HJgAHBvQOVVQQ5O0VVuVijInPxIkT+fa3v90Q0GfMmMEbb7zB7bffTkFBAVu2bOH444/nwgsvjGui5oceeggRYfHixXz00UecffbZrFixgkceeYQ77riDSZMmUVNTQzgcZvbs2fTu3ZvXXnsNgMrKyoS8tlgCeh9gbdR6OXBcE8ddIiLjgRXAd1R17YEHiMjNwM0A/fv3jz+3UYpCOezeV8eefXXkBVNyjDFjTDMl6fYyZswYNm/ezPr166moqKBr16707NmT73znO8ydOxefz8e6devYtGkTPXv2jPm87777LrfddhsAw4cPZ8CAAaxYsYITTjiBX/7yl5SXl3PxxRczZMgQRo0axXe/+11+8IMfcP7553PKKack5LUl6qbo34CBqnoU8A9galMHqeqjqlqiqiWFhYVtuqBNdGGMaa3LLruMmTNnMn36dCZOnMi0adOoqKhg/vz5lJWVUVxcTHV1Yu7RXXXVVcyaNYtOnTpx3nnn8e9//5uhQ4eyYMECRo0axY9//GPuu+++hFwrloC+DugXtd7X29ZAVbeqan1kfQwYm5DcNaOhLbrdGDXGxGnixIk8//zzzJw5k8suu4zKykqKiorIyspizpw5rFmzJu5znnLKKUybNg2AFStW8PnnnzNs2DBWrVrFYYcdxu23386ECRNYtGgR69evJzc3l6uvvpq77rorYWOrx1JXMQ8YIiKDcIH8CuCq6ANEpJeqbvBWLwSWJyR3zSiun1vUSujGmDgdeeSR7Nq1iz59+tCrVy8mTZrEBRdcwKhRoygpKWH48OFxn/Ob3/wmt9xyC6NGjSIQCPDkk08SDAaZMWMGTz/9NFlZWfTs2ZMf/ehHzJs3j7vuugufz0dWVhYPP/xwQl5XTOOhi8h5wB8BP/C4qv5SRO4DSlV1loj8ChfI64BtwC2q+lFz52zLeOgAO6pqGH3fP/jJ+SO48eRBrT6PMaZj2XjosYt3PPSY7iaq6mxg9gHbfhq1/EPgh3Hntg06d8oiO+CzKhdjjPGkbPMQEaEw33qLGmPa3+LFi7nmmmv22xYMBvnggw+SlKOmpWxABygusLlFjUlFqhpXG+9kGzVqFGVlZR16zdbMyJaSY7nUKwrl2ABdxqSYnJwctm7dalNINkNV2bp1Kzk5OXGlS+kSelFBkP98uiXZ2TDGxKFv376Ul5dTUVGR7Kwc0nJycujbt29caVI7oIeC7Kyuo7o2TE6WP9nZMcbEICsri0GDrGVae0jtKhevLXqF3Rg1xpgUD+he938bRtcYY1I+oFtvUWOMqZfaAd3GczHGmAYpHdC75WYT8ImV0I0xhhQP6D6fUBgKssnaohtjTGoHdHA3Rq23qDHGpENAL8ixZovGGEM6BPSQDdBljDGQFgE9h217aqipiyQ7K8YYk1SpH9C9posVu62UbozJbCkf0IutLboxxgBpENCtt6gxxjhpENCthG6MMZAGAb17fhCfWAndGGNSPqD7fUKP/KDNXGSMyXgpH9DBtXSx3qLGmEyXHgE9lGPjuRhjMl6aBHTrLWqMMekR0Aty2LpnH3Vh6y1qjMlc6RHQQ0FUYcvummRnxRhjkiZtAjpgN0aNMRktpoAuIueIyMcislJE7m7muEtEREWkJHFZbFlRgddb1G6MGmMyWIsBXUT8wEPAucAI4EoRGdHEcSHgDuCDRGeyJQ3judiNUWNMBoulhD4OWKmqq1S1BngemNDEcb8Afg10eL1Hj/wgIrDJuv8bYzJYLAG9D7A2ar3c29ZARI4B+qnqawnMW8yy/D665WZbCd0Yk9HafFNURHzAg8B3Yzj2ZhEpFZHSioqKtl56P4WhIBV2U9QYk8FiCejrgH5R6329bfVCwEjgLRFZDRwPzGrqxqiqPqqqJapaUlhY2PpcN6G4IMdK6MaYjBZLQJ8HDBGRQSKSDVwBzKrfqaqVqtpDVQeq6kDgfeBCVS1tlxwfRFEoaHXoxpiM1mJAV9U64FbgDWA5MENVl4rIfSJyYXtnMFZFBUG27K4hHNFkZ8UYY5IiEMtBqjobmH3Atp8e5NjT2p6t+BWFcghHlG17aij0OhoZY0wmSYueohDdFt2qXYwxmSltAnphyHqLGmMyW9oEdBvPxRiT6dInoNdXuVgJ3RiTodImoAcDfrrkZllbdGNMxkqbgA7WFt0Yk9nSLKBbb1FjTOZKr4BeEKTCAroxJkOlV0AP5bB5VzWq1lvUGJN50iygB6kNK9urapOdFWOM6XDpFdCtt6gxJoOlVUAvtrlFjTEZLK0Cen1vUWu6aIzJRGkW0L0SurV0McZkoLQK6J2y/YSCAWu6aIzJSGkV0MHdGLWbosaYTJR+AT2Uwya7KWqMyUDpF9CthG6MyVDpF9BDQTbv3Ge9RY0xGSftAnpxQQ776iLsrK5LdlaMMaZDpV1Ar58gerO1RTfGZJi0C+jWFt0Yk6nSL6DbeC7GmAyVdgHdxnMxxmSqtAvo+cEAudl+a4tujMk4aRfQwWu6aFUuxpgMk54BvcDmFjXGZJ6YArqInCMiH4vIShG5u4n93xCRxSJSJiLvisiIxGc1dkUhm1vUGJN5WgzoIuIHHgLOBUYAVzYRsJ9V1VGqOhr4DfBgwnMaBzeei1W5GGMySywl9HHASlVdpao1wPPAhOgDVHVn1GoekNR+90UFQapqwuzeZ71FjTGZIxDDMX2AtVHr5cBxBx4kIt8C7gSygTMSkrtWKi5o7C2aX5ifzKwYY0yHSdhNUVV9SFUPB34A/LipY0TkZhEpFZHSioqKRF36C+p7i1rTRWNMJokloK8D+kWt9/W2HczzwEVN7VDVR1W1RFVLCgsLY89lnOrnFrWmi8aYTBJLQJ8HDBGRQSKSDVwBzIo+QESGRK1+BfgkcVmMX30J3Vq6GGMySYt16KpaJyK3Am8AfuBxVV0qIvcBpao6C7hVRM4CaoHtwHXtmemWFHQKEAz4rC26MSajxHJTFFWdDcw+YNtPo5bvSHC+2kREKCoIWtNFY0xGScueouCqXWyALmNMJknjgG7juRhjMkvaBvRiG8/FGJNh0jagF4aC7KquY29NONlZMcaYDpG2Ad3aohtjMk36BvQCm1vUGJNZ0jagN47nYgHdGJMZ0jagN47nYlUuxpjMkLYBvWtuFll+sSoXY0zGSNuALiIU5ltbdGNM5kjbgA7uxqgN0GWMyRTpHdBDNp6LMSZzpHdALwhaHboxJmOkdUAvDuWwo6qWfXXWW9QYk/7SOqAXWVt0Y0wGSe+AHrLeosaYzJHWAb3QG8+lwpouGmMyQFoH9GIbz8UYk0HSOqB3z8vG7xNrumiMyQhpHdB9PqFHfrbdFDXGZIS0DujgzS1qVS7GmAyQ9gG92DoXGWMyRNoH9MJQDputDt0YkwHSPqAXhYJs3VNDbTiS7KwYY0y7Sv+A7vUW3bLbql2MMekt7QN6cX1vUWvpYoxJc2kf0OtL6NYW3RiT7tI/oNt4LsaYDBFTQBeRc0TkYxFZKSJ3N7H/ThFZJiKLRORfIjIg8VltnR752YhYQDfGpL8WA7qI+IGHgHOBEcCVIjLigMM+BEpU9ShgJvCbRGe0tQJ+H93zgjZAlzEm7cVSQh8HrFTVVapaAzwPTIg+QFXnqGqVt/o+0Dex2WwbNxWdldCNMektloDeB1gbtV7ubTuYG4G/N7VDRG4WkVIRKa2oqIg9l23kpqKzEroxJr0l9KaoiFwNlAC/bWq/qj6qqiWqWlJYWJjISzerKBS0ZovGmLQXiOGYdUC/qPW+3rb9iMhZwD3Aqap6SEXP4oIctuzeRzii+H2S7OwYY0y7iKWEPg8YIiKDRCQbuAKYFX2AiIwB/gxcqKqbE5/NtikKBYkobLXeosaYNNZiQFfVOuBW4A1gOTBDVZeKyH0icqF32G+BfOAFESkTkVkHOV1SFFpbdGNMBoilygVVnQ3MPmDbT6OWz0pwvhKqvreouzHaObmZMcaYdpL2PUWhcW5Ra7pojEm6cC3UtU8siqmEnuoK870SugV0Y0xHi4Rh4yL47B34bC58/h585fdw9BUJv1RGBPTsgI+uuVnWFt0Y0/4iEahY7oL3Z+/AmnehutLt6zEUjpoIPYa0y6UzIqCDq3axm6LGmIRThS2fwGovgK9+B6q2un1dB8KICTBwPAw8GQp6tWtWMiagF4aCNhWdMabtVGH7ahe460vhuze6fQV9YMjZMPAUGHQKdOnfoVnLmIBeFMph5ebdyc6GMSYVRSJQPg+WvQIfvQo71rjteUUucA8a74J4t8NAktd5MXMCekGQil37iEQUn/UWNanq0znwj5/Al+6Dw89Idm7SWyQMn7/vgvjyWbBrA/iz3ft+4m0uiPcYmtQAfqCMCejFoSB1EWV7VQ3dvVYvxqSUpS/DizeBhuG5q+Dqma5eNpPU7YMlL0IwBEUjoOsg8CWw9XW4zt3EXPYKLP8b7KmAQA4MPgtGXARDvww5BYm7XoJlTEAvimqLbgHdpJzSx+HVO6HfcTDhIXj+Kph2OVz7MvQbl+zcdYzKcphxHawrbdyWlQuFw6F4BBQdCcXeI69H7Oetq3F14ctfgeWvwt5t7rxDznY3NIecDcH8xL+edpA5AT3U2Ft0BIfuN6wx+1GFd34H/77fBZbLpkJ2Llz7Cjx5HjxziVvuc0yyc9q+Pp0DL97ogu+lT0CXAbB5KWxa5p4/fh0+fKbx+LyiqCA/wpXmC4e79w5cSf/TOa4k/vFrrllhdgiGneOC+OFnNh6bQjIooNt4LibFRCLw5j3w/p9c2+UJD4E/y+0r6AXX/Q2eOBee/ipMfhV6jkpufttDJALvPghzfgk9hsHEpxvbcPcdu/+xuzfDpqXusXmZey6dAnX1rdvE3bTsOgDWzoOaXZDTGYad54L4YadDVk6HvrxEy5yA7o3nUmEB3aSCcC288i1YNB2O+wZ8+VdfrCvu3NcL6ufBUxfB5NegaHhy8tse9u6Al2+Bj2fDyEvhgv9pvuojv8g9Dj+9cVskDNs+2780v3UVHHmRqxMfNB4C2e3/WjpIxgT0nCw/BTkBNllbdHOoq6mCFybDJ2/A6T+G8d87eEuKrgPh2lmu+uWpC+H6v0P3wzsyt+1j4xKYfjVUroVzfg3Hfb11rUl8fugx2D1GTGj5+BSXEYNz1SsqyLHxXMyhbe8OeOZi+ORNN97HqXe1HMh6DHZBPVIHUy9wnV5S2cLn4bGzXFXJ5Nfg+G8cUk0DD2WZFdBDNreoOYTt2gRPfgXKS+HSx+HYm2JPWzTc3Ryt2QNTL3QtQlJN3T7XkuevX4c+Y+Hrc6H/8cnOVUrJqIBeXJBjQ+iaQ9O2z+Dxs93zpBkw8uL4z9FzFFzzEuzd7oL6ro2Jz2d7qSx39wJKp8CJt7svp/yiZOcq5aRmQI9EWpWsKOR6i6pqgjNkTBtsXAKPf9k1nbtuVtt6gPYZC5NmumD+1ATYsyVx+Wwvq96CP4+Hio/h8qfg7F+AP2Nu7yVU6gX0ZbNgylmNw1HGoTAUpCYcoXJvbTtkzJhWWPOeK5mKH65/HfqWtP2c/Y9zpfzta1zrl6ptbT9ne4hE4J3fu2aXeYVw85yMuHHZnlIvoGfnwYaF8PwkqI2vPry+t6i1RTeHhBVvuGCWXwg3vpHYJocDT4Yrn4UtK9xN1lYUgNrV3h0wfRL86z448qtw07/abYzwTJJ6AX3wmXDRw27oyr/e7NqZxqjY6y1qTRdN0i2cDs9dCYXD4IY32meY1cPPcFUYGxfDtMtg3yEy2ujGJfCX011LnnN+DZdMSZmu9Ye61KyoOupy1yvszXvg79+H834XU7OmhhK63Rg1yaIKHzwCr9/thlu94tn2Hexp2DmuxcwL18NzV8BVMzq2S3skAls/gXULYN1899i4CHJ7uCaJ1ooloVIzoAOceCvs3gT/+T+Q39O1121B43guFtBNB6upgsUvwLy/uBLz8PNdybQjupqPmAAXP+pGapw+Ca54rv2uu3N9Y+BeNx/Wl8G+nW5fdj70HgMn3ArHfxNCxe2ThwyWugEd4Kyfu5L6nPtdPeTYyc0enhcMkJftt7bopuNs/RTmTYGyZ1w9dtGRcP4fYcw1HduSY9Slrp33K9+EGdfAMde5knp2vhtZMDuv8ZGVG1tHnr07YP2HXvD2SuD1M/f4styoh6Mucy1v+ox1deQ+f/u+zgyX2gHd54MJ/wtVW+DV77g75cO/0mySYustatpbJAyf/MOVxlf+E3wBOOICGHcz9D8heb0ex0yCsNd555M3mzlQooK8F/TrA312ngvKm5a5qpR63YfAYac2Bu/ikSk/0FUqSu2ADm70ucumunEsZt4A17wMA0446OGF1lvUtJeqbfDh065EvmONqwo87YeuNNzOkwPHrOQGGHqum7ihZg/U7nHPNVVQsxtqq7z1qEett69mj/tFHN7nhqI9+goXvHuPgU5dkv3KDOkQ0MHdIb/qBdfT7rmJrj1v8YgmDy0qyGFR+Y4OzqBJa+sWwLzH3Ew6ddUw4CQ4615XKq8f7vZQUtDr0PmCMQmVHgEdIK87XP0STDnbDfp/45vQpd8XDisOBdm80/UWFRvwp+PVVruf6kUjUrs+tbYalr0M//2Lm0EnKw+OvhLGfc3VHRuTBDEFdBE5B/gfwA88pqoPHLB/PPBH4CjgClWdmeiMxqTrALj6Rdfz7pmLXfve3G77HdKnayf21oZ5acE6LhnbNynZzDiq7oZZ2TRXiq2udDPOlNzgbg7mdW//PEQisHouzH8SVrzp6rEDQQh0cs9Z3nMgp/GRldP0enUlLHwOqrZC98GuLfXoK91kCcYkkbQ0romI+IEVwJeAcmAecKWqLos6ZiBQAHwPmBVLQC8pKdHS0tKWDmud1e/C0xdDr6PcID/ZeQ27du+r42tTS3lv1Va+d/ZQvnX6YCupt5ed690EDWVej8VAJxhxoWt7vOQl1znMH3Q9Bcd9zdXHJvpvsXuz+yKZPxW2fwY5XdzkBtn5ULvXtfyo855r97oqk7pqVwKvqz5gfS9oBMTn6qHH3QSDTkvsJMXGtEBE5qtqk2NExFJCHwesVNVV3smeByYADQFdVVd7+1o3alaiDTwZLnkMXrjOdai4YlpDXWZ+MMDUG8bx/ZkL+d2bK1i3Yy+/mDCSgN/+KROittrN0Vj2LHz6bxcA+58AF/5fN0NMfSeakhtg83J3A3Hh87Doeeh1tBsyduSlbev8EonAqjmuNP7xbDdO+ICT4PQfwREXtq31RbjWvaaATTRuDj2xlNAvBc5R1Zu89WuA41T11iaOfRJ49WAldBG5GbgZoH///mPXrFnTtty3ZN4UeO1OGH21a94YVfpTVX77xsf86a1POX1YIf971THkBdPnlkKHUnVjeJdNcyXvfZXQuZ9rBXH0lS3PoLNvFyya4W4sbl7mqi5GT4KSG93kDbHatdFNFLzgKdfKpFM3GH2Va2VSOLRtr9GYQ0RbS+gJo6qPAo+Cq3Jp9wsee6P7yf32A25s5bN+1rBLRPj+OcPp07UTP3l5CVc8+j5TJpc0TCZtYrBzvStdlz3rbnQGOrleiaOvct3aY62KCIbc36rkBvj8PRfY//sXNznyYae7UvvQc5ruiBMJu18C85+Ej/8OGnbXPvOnrpWJlaRNBokloK8DopuL9PW2pYbT7na91959EPKL3XRWUSYdN4CeBTnc+uyHXPyn//Dk9eMYXGQDBR1UTZWrxih71lVraAT6nwgn3eGCeVvGJRGBASe6x65NrqQ9/wnXXb2gD4y9Ho651nUZ37m+sTReudaNDXLira40ng5zahrTCrFUuQRwN0XPxAXyecBVqrq0iWOfpJkql2jtelP0QJEwzLgWPnrN1a2PuvQLhywq38ENT86jNqw8docWshQAABPOSURBVF0Jxw7s1sSJMlRdjSsFL5kJH812nVE693PVKUdf0b4BNFwHK153pfZVc1yX8t6jXasZjcBhp7khH4Z9Ja1mbzfmYJqrcmkxoHsnOA/XLNEPPK6qvxSR+4BSVZ0lIscCfwW6AtXARlVttjFuhwZ0cDfrnv4qlM9zg/83MSvM51urmPzEfynfsZc/XD6arxx1CHa+CNe6Hon5Re3bhTwSdq1QlrzoJhWp3gGdurpS+MhLYMDJHd+6Y8tKN0XZZ3NhyNlwzDXQ7bCOzYMxSdbmgN4eOjyggxtM6InzYPNS1xEkr7sb/yW3h3vO605VVjemLNjJ/C1+Jpx0NBeddBSSV+jaKXe0mip3k3BDGWxY5IYd3bTMdb3OK2wcN6PPWOhzjAu4bRGJuC+8JS/C0r/Cns2ued/w810QP+w0KwUbk2QW0KPtroCFz7o62qotbs7FPRWuk8ieCgjXNJ0uOx9yvS+AvEII9YSC3u451KvxkdutdSXnvdvdsKobFrkZmTYucm231WsJmtPFtavvdbSrT9642FU7VHwMeH/D7oO94F7innuObPmmoKq71pIXXQuVyrWu88zQL7sgPuTs5HyZGWOaZAE9VqquCd2eCiK7t/DCO2UsWP4JxxdHuGBwFoG929yXwO7NrolcVRMT8PqzvxjkC+qXe0Kotwuym5d5wbvMBdQdnzeeI9S7MXj3PMotd+7X9BdFdaUbc3pdqRtTpLy0cQhTf7abCb5PiZurss9YV0UhAls+gcUzXSDf+okbEfDwM1wb8GHntu+kC8aYVrOA3gZT3v2M+19bxph+XXjsumPplhdV5VC3z02ysXMD7Ip+bHStMHZtdOs1zUz91e3wA4L30ZDXo/UZVvUmGSh1Jfjy+W7M6to9bn9OF1f/vmUFIK4T1shLXN14rt0INuZQZwG9jf6+eAPfnl5G7y6dePL6YxnQPa/lRNH27do/yNfsdoNT9Rzp2mC3t0gYKj7yAnwpVJbDkC+5nps26p4xKcUCegLMX7ONG6eW4hdhyuRjGd3Pxn82xnS85gK6DWASo7EDuvHiLSeSG/Rz+SPv8cOXFrF6y55kZ8sYYxpYQI/D4YX5/PWbJ3FZSV9eXLCOM37/Frc+u4Cl6yuTnTVjjLEql9bavLOaKf/vM6a9/zm799Vx2rBCvnnaYMYNshuLxpj2Y3Xo7ahyby3PvL+Gx9/9jK17aigZ0JVbTjucM4YX2TjrxpiEs4DeAfbWhJlRupZH565i3Y69DO8Z4pbTDucro3rZWOvGmISxgN6BasMR/rZwPQ+/9SmfbN5N/2653Dz+MC4d25ecrBSeQ9MYc0iwgJ4EkYjyz+Wb+NNbn1K2dgc98oPcePIgrj6+P6GcQ3AmeGNMSrCAnkSqynurtvLwW5/yzidbCOUEuGpcf04c3IPR/brQuZMFd2NM7CygHyIWl1fy8NsreX3JRiLe2z64KJ8x/bowpn9XxvTvwtDiEH6f3Uw1xjTNAvohZld1LYvLK1nw+XY+/HwHH67dwbY9bpTHvGw/R/frwpj+XRjTryuj+3ehR75No2aMcQ6ZOUWNE8rJ4sTBPThxsBuES1X5fFsVH36+oyHI//ntVdR5xfj+3XI5pn9jKf6IXgVkWcsZY8wBLKAfAkSEAd3zGNA9j4vG9AFcM8gl6ytZsMYF+P98upWXy9YDkB3wcUSvAkb1KWBk786M7NOZocUhsgMW5I3JZFblkiJUlQ2V1Xz4+Q7K1m5n8bpKlq7bya59dQBk+30M7xXiyN6dGdXHPYb2zCcYsKaSxqQTq0NPU5GIsmZblRfcK1m8rpIl6yrZWe2CfJZfGFocYlQfV4of1aczw3qGrD28MSnMAnoGqa+Pd8F9J0u8QF+5txaAgE8YXJTPYYWuimdg91zvOY+iUBCftbAx5pBmN0UzSHR9/PlH9QZckC/fvrchuC/fsJOPNuzizaWbGm68AuRk+RjQLY+BPXIZ2D0q4PfIo1dBjgV7Yw5xFtAzgIjQr1su/brlcu6oxhmK6sIRNlRWs3rrHlZvrWLNFve8qmIPcz6uoKYu0nBsdsBH/265DOyeS9+uuXTNzaagU4DOnbIoyMmic6733CmLgk4BOmX5bXAyYzqYBfQMFvD7GgL9KUP23xeJKBt3esF+SxVrtu5h9dY9rNlaxfurtrHbuxl7MFl+aQjwoU5ZFOR4wd/7AijoFCCU47aHctxy9HN+dsB+ERgTJwvopkk+n9C7Syd6d+nEiYd/cX9dOMKu6joq99ays7rWPe89cL2WnfXH7K1l3fa9Dftqw83fuxGB/GCAgoZAHx30A+QFA+RlB8jN9pObHSAv6D1n+8kNHvCcHbAmnSYjWEA3rRLw++ial03XvOy406oq++oi7KyuZVd1Hbuq69i5t3658XlndV3UMbVs2lnNys1ueU9NeL8qoZZk+aUh4Odk+/GL4BNBxFVJCeDz4bbhtvkk6hl3rE8Ev08I5QQafmnU//Lo3PDrI4vOnQINy9aqyHQUC+imw4kIOVl+crL8FIVaf566cISq2jBV+8LsqalrfK6pY8++8H7PVTVhqmrC7NlXR1VtGFUlEgFFiaj7klGFiCoKX9zmPYdVqamNsHlXdcOvkr214WbzmR3wecE+QEGnLEI5We76qoQjLh9hVeoiSiTibfP2hdXbpko47K0r+L0vliy/ew74fO65Yf2A7fXrfsHv85HlF7L9PgJ+Icvv22/5wPVsv4+sgDtXlt9HdkDI9vvJDvjIDvgIes/Zfh/BLB/BqH3JGpdIo96/cGT/9zkcURSFqB+J2pCufl2jznXgMe7zoIr391Dv7+k+I5HI/p+ZiDZ+xtzfFgb1yKNn55yEv24L6CZlBfw+Cvw+CpI8HHGN92ujvmqp8oCqpp0HVktV1YAIfoGAz4fPB1k+X0PpP+CThmWfTxqCt99b9vkgEoHaSIRwxH0RhMPecyTiPbv1vbXhxu3hxu214Yj32H85HElsM+aAT/YP+l7gh6h46sVW9b5MVV1AbQikDc+N+5v6oqv/IqyLNKY9VN1/0UiuPn5Aws8bU0AXkXOA/wH8wGOq+sAB+4PAU8BYYCswUVVXJzarxhyasgM+euQH02IQtUhEqY244F4XjlATblyuDUeoqXNfADXhCDV1EfbVhb1n96ipizSs19RFqAmH2Vfrjq9/bqgqE6gvv0tDVRcNVV7iHeOeJGof7stOvC+/A7/0or4Q9/8idF+ifp80tMCKbojlXbFhW/RvC5EvHuPzvlx9XvWd74Bquvr9ErWv/rjDCvMS+4fztBjQRcQPPAR8CSgH5onILFVdFnXYjcB2VR0sIlcAvwYmtkeGjTHtx+cTgj4/QfvtnpJiufU/DlipqqtUtQZ4HphwwDETgKne8kzgTLFGyMYY06FiCeh9gLVR6+XetiaPUdU6oBLofuCJRORmESkVkdKKiorW5dgYY0yTOrRxrqo+qqolqlpSWFjYkZc2xpi0F0tAXwf0i1rv621r8hgRCQCdcTdHjTHGdJBYAvo8YIiIDBKRbOAKYNYBx8wCrvOWLwX+rckaxtEYYzJUi/eyVbVORG4F3sA1W3xcVZeKyH1AqarOAqYAT4vISmAbLugbY4zpQDE1TlLV2cDsA7b9NGq5GrgssVkzxhgTDxuxyBhj0kTSZiwSkQpgTSuT9wC2tOHylt7SJzP9oZAHS5+66QeoatPNBNUbWCaVHri6e0tv6VMy/aGQB0uf2ukP9rAqF2OMSRMW0I0xJk2kakB/1NJb+hROfyjkwdKndvomJe2mqDHGmMRK1RK6McaYA1hAN8aYNJFyAV1EzhGRj0VkpYjcHWfax0Vks4gsaeW1+4nIHBFZJiJLReSOONPniMh/RWShl/7nrcyHX0Q+FJFXW5F2tYgsFpEyESltRfouIjJTRD4SkeUickIcaYd5161/7BSRb8d5/e94790SEXlOROKamFFE7vDSLo3l2k19ZkSkm4j8Q0Q+8Z67xpn+Mu/6EREpacX1f+u9/4tE5K8i0iXO9L/w0paJyJsi0jue9FH7visiKiI94rz+vSKyLupzcF681xeR27z3YKmI/CbO60+PuvZqESmLM/1oEXm//n9IRMbFmf5oEXnP+z/8m4gUHCx93NqjLWR7PXBjyXwKHAZkAwuBEXGkHw8cAyxp5fV7Acd4yyFgRZzXFyDfW84CPgCOb0U+7gSeBV5tRdrVQI82/A2mAjd5y9lAlzb8LTfiOknEmqYP8BnQyVufAUyOI/1IYAmQixv24p/A4Hg/M8BvgLu95buBX8eZ/ghgGPAWUNKK658NBLzlX7fi+gVRy7cDj8ST3tveDze+05rmPk8Huf69wPdi/Js1lf50728X9NaL4s1/1P7fAz+N8/pvAud6y+cBb8WZfh5wqrd8A/CLWD/DLT1SrYQey+xJB6Wqc3GDh7WKqm5Q1QXe8i5gOV+c7KO59Kqqu73VLO8R111pEekLfAV4LJ50iSAinXEf0CkAqlqjqjtaebozgU9VNd7ewgGgk7hhmnOB9XGkPQL4QFWr1E3E8jZwcXMJDvKZiZ6haypwUTzpVXW5qn4cS4YPkv5NL/8A7+OGtI4n/c6o1Tya+Qw28z/zB+D7zaVtIX1MDpL+FuABVd3nHbO5NdcXEQEuB56LM70C9aXqzjTzGTxI+qHAXG/5H8AlB0sfr1QL6LHMntQhRGQgMAZXyo4nnd/7ibcZ+IeqxpUe+CPuHykSZ7p6CrwpIvNF5OY40w4CKoAnvCqfx0SktbPdXkEz/0hNUdV1wO+Az4ENQKWqvhnHKZYAp4hIdxHJxZWu+rWQpinFqrrBW94IFLfiHIlyA/D3eBOJyC9FZC0wCfhpS8cfkHYCsE5VF8Z73Si3etU+jzdXZXUQQ3F/xw9E5G0RObaVeTgF2KSqn8SZ7tvAb73373fAD+NMv5TGguhltO4z2KRUC+iHBBHJB14Evn1AaadFqhpW1dG4UtU4ERkZx3XPBzar6vy4Mry/k1X1GOBc4FsiMj6OtAHcz8eHVXUMsAdX5RAXcePqXwi8EGe6rrh/hEFAbyBPRK6ONb2qLsdVUbwJvA6UAeF48tDEOZU4f2UliojcA9QB0+JNq6r3qGo/L+2tcVwzF/gRcX4JHOBh4HBgNO6L+fdxpg8A3YDjgbuAGV5pO15XEmehwnML8B3v/fsO3i/WONwAfFNE5uOqbmtakYcmpVpAj2X2pHYlIlm4YD5NVV9q7Xm8qoo5wDlxJDsJuFBEVuOqm84QkWfivO4673kz8FdcNVasyoHyqF8VM3EBPl7nAgtUdVOc6c4CPlPVClWtBV4CToznBKo6RVXHqup4YDvuPki8NolILwDv+aA/+duLiEwGzgcmeV8qrTWN+H7yH477Ql3ofQ77AgtEpGesJ1DVTV7BJgL8hfg+g+A+hy95VZj/xf1aPeiN2aZ4VXYXA9PjvDa4yXzq//dfIM78q+pHqnq2qo7FfaF82oo8NCnVAnossye1G68UMAVYrqoPtiJ9YX2LBBHpBHwJ+CjW9Kr6Q1Xtq6oDca/936oacwlVRPJEJFS/jLu5FnOLH1XdCKwVkWHepjOBZbGmj9LaktHnwPEikuv9Lc7E3ceImYgUec/9cf/Qz7YiH9EzdF0HvNKKc7SaiJyDq3a7UFWrWpF+SNTqBOL7DC5W1SJVHeh9DstxDQU2xnH9XlGrXyWOz6DnZdyNUURkKO7mfLwjF54FfKSq5XGmA1dnfqq3fAYQV5VN1GfQB/wYeKQVeWhaou6udtQDV++5Avetdk+caZ/D/cSrxX0Qb4wz/cm4n9eLcD/Xy4Dz4kh/FPChl34Jzdxdj+FcpxFnKxdc66CF3mNpvO+fd47RQKn3Gl4GusaZPg8332znVr7un+MC0BLgabyWDnGkfwf3JbQQOLM1nxmgO/Av3D/yP4Fucab/qre8D9gEvBFn+pW4e0n1n8HmWqk0lf5F7/1bBPwN6NPa/xlaaDV1kOs/DSz2rj8L6BVn+mzgGe81LADOiDf/wJPAN1r59z8ZmO99hj4AxsaZ/g5cDFsBPIDXYz8RD+v6b4wxaSLVqlyMMcYchAV0Y4xJExbQjTEmTVhAN8aYNGEB3Rhj0oQFdGNiJCKnSStGuDSmo1hAN8aYNGEB3aQdEbla3LjzZSLyZ29AtN0i8gdv/Ox/iUihd2z92Nb1Y4t39bYPFpF/ihu7foGIHO6dPl8ax4OfVj+GiIg8IG6c/EUi8rskvXST4Sygm7QiIkcAE4GT1A2CFsaNKJgHlKrqkbhhc3/mJXkK+IGqHoXrvVi/fRrwkKoejRsvpn50xTG40fZG4HreniQi3XG9P4/0znN/+75KY5pmAd2kmzOBscA8b5jiM3GBN0LjQEzPACd747t3UdW3ve1TgfHeeDd9VPWvAKparY1jpvxXVcvVDSxVBgwEKoFqYIqIXAzEPb6KMYlgAd2kGwGmqupo7zFMVe9t4rjWjnmxL2o5jJs5qA434t5M3AiIr7fy3Ma0iQV0k27+BVwaNaJdNxEZgPusX+odcxXwrqpWAttF5BRv+zXA2+pmoyoXkYu8cwS9ccCb5I2P31lVZ+PGxz66PV6YMS0JJDsDxiSSqi4TkR/jZmXy4Ua5+xZuMo5x3r7NuHp2cMPfPuIF7FXA9d72a4A/i8h93jkua+ayIeAVcRNWC27OV2M6nI22aDKCiOxW1fxk58OY9mRVLsYYkyashG6MMWnCSujGGJMmLKAbY0yasIBujDFpwgK6McakCQvoxhiTJv4/ntjaSbvarvEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history_no_dropout.history[\"loss\"])\n",
    "plt.plot(history_no_dropout.history[\"val_loss\"])\n",
    "plt.legend([\"loss\", \"val_loss\"])\n",
    "plt.xticks(range(epochs))\n",
    "plt.xlabel(\"epochs\")\n",
    "plt.title(\"Training process\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bB1KeA-Ww9nE"
   },
   "source": [
    "As you can see, the model is clearly overfitting. The loss on the training data is decreasing further and further with every epoch, while the loss on the validation data rises at the same time. We can avoid this by using dropout, which is a strong regularization that you should be familiar with from the lecture. For this we define a new model that has an additional dropout layer with a drop probability of $0.25$ before the first fully connected layer. Otherwise this model is the same as the model above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NowP4zSsxfhe"
   },
   "outputs": [],
   "source": [
    "class MyDropoutModel(k.Model):\n",
    "    def __init__(self):\n",
    "        super(MyDropoutModel, self).__init__()\n",
    "        self.conv0 = k.layers.Conv2D(8, 3, 2, activation=\"relu\")\n",
    "        self.conv1 = k.layers.Conv2D(16, 3, 2, activation=\"relu\")\n",
    "        self.flatten = k.layers.Flatten()\n",
    "        self.dropout = k.layers.Dropout(0.25)\n",
    "        self.dense0 = k.layers.Dense(128, activation=\"relu\")\n",
    "        self.dense1 = k.layers.Dense(64, activation=\"relu\")\n",
    "        self.dense2 = k.layers.Dense(10, activation=\"softmax\")\n",
    "\n",
    "    def call(self, inputs, training=False):\n",
    "        output = self.conv0(inputs)\n",
    "        output = self.conv1(output)\n",
    "        output = self.flatten(output)\n",
    "        output = self.dropout(output, training)\n",
    "        output = self.dense0(output)\n",
    "        output = self.dense1(output)\n",
    "        output = self.dense2(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_XivWPeUxphB"
   },
   "source": [
    "We can now create an instance of this model, train it and visualize the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 117077,
     "status": "ok",
     "timestamp": 1562670760185,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "VZxNQX2Bxtno",
    "outputId": "b23313fd-e3b6-4457-84c5-9847058cb10b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.5024 - accuracy: 0.8541 - val_loss: 0.1272 - val_accuracy: 0.9628\n",
      "Epoch 2/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.1537 - accuracy: 0.9527 - val_loss: 0.0838 - val_accuracy: 0.9757\n",
      "Epoch 3/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.1077 - accuracy: 0.9662 - val_loss: 0.0741 - val_accuracy: 0.9795\n",
      "Epoch 4/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0853 - accuracy: 0.9733 - val_loss: 0.0716 - val_accuracy: 0.9790\n",
      "Epoch 5/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0746 - accuracy: 0.9768 - val_loss: 0.0650 - val_accuracy: 0.9837\n",
      "Epoch 6/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0642 - accuracy: 0.9804 - val_loss: 0.0618 - val_accuracy: 0.9832\n",
      "Epoch 7/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0575 - accuracy: 0.9815 - val_loss: 0.0643 - val_accuracy: 0.9833\n",
      "Epoch 8/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0524 - accuracy: 0.9840 - val_loss: 0.0544 - val_accuracy: 0.9835\n",
      "Epoch 9/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0494 - accuracy: 0.9845 - val_loss: 0.0505 - val_accuracy: 0.9863\n",
      "Epoch 10/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0477 - accuracy: 0.9847 - val_loss: 0.0507 - val_accuracy: 0.9862\n",
      "Epoch 11/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0432 - accuracy: 0.9861 - val_loss: 0.0567 - val_accuracy: 0.9842\n",
      "Epoch 12/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0391 - accuracy: 0.9873 - val_loss: 0.0568 - val_accuracy: 0.9865\n",
      "Epoch 13/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0391 - accuracy: 0.9876 - val_loss: 0.0542 - val_accuracy: 0.9865\n",
      "Epoch 14/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0372 - accuracy: 0.9885 - val_loss: 0.0615 - val_accuracy: 0.9858\n",
      "Epoch 15/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0367 - accuracy: 0.9883 - val_loss: 0.0563 - val_accuracy: 0.9870\n",
      "Epoch 16/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0340 - accuracy: 0.9891 - val_loss: 0.0549 - val_accuracy: 0.9862\n",
      "Epoch 17/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0346 - accuracy: 0.9890 - val_loss: 0.0520 - val_accuracy: 0.9888\n",
      "Epoch 18/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0319 - accuracy: 0.9896 - val_loss: 0.0542 - val_accuracy: 0.9880\n",
      "Epoch 19/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0318 - accuracy: 0.9900 - val_loss: 0.0469 - val_accuracy: 0.9880\n",
      "Epoch 20/20\n",
      "422/422 [==============================] - 1s 2ms/step - loss: 0.0321 - accuracy: 0.9900 - val_loss: 0.0556 - val_accuracy: 0.9880\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeXxU1f3/8deZPcskJGQjG2FHIBBWlwqi36poVWoVcQctWnD9tha1trbWr/5sa39tv9/+rEtVwA2hbqWVatuvWKBaS8CwCwokkAWy78ss9/z+uJMNAiQwyZDJ59nHPO46d85E+r5nzj33XKW1RgghRP9nCXUBhBBCBIcEuhBChAkJdCGECBMS6EIIESYk0IUQIkxIoAshRJiQQBdnNKXUX5RSC4K9rxDhSEk/dBFsSqn6DouRQAvgDyx/R2v9et+XSojwJ4EuepVSKh9YpLX+exfbbFprX9+Xqncppaxaa//J9xQiuKTJRfQZpdRspVShUuohpdRhYJlSKk4p9WelVJlSqiown97hPR8rpRYF5hcqpTYqpX4Z2PeAUuqyU9x3mFJqvVKqTin1d6XUM0qp105S7keUUuVKqXyl1E0dti9XSj2rlFqrlGoALlRKnRUoT7VSaqdS6qoO+0copf6vUqpAKVUTKGdEYNs5SqlPAu/bqpSa3eF9C5VS+wNlPtBaBqXUSKXUPwLHKldKrTrt/1iiX5JAF30tBYgHhgJ3Yv4bXBZYzgSagP93gvefDewBEoBfAC8ppdQp7PsG8G9gMPAYcEs3yp0ApAELgBeUUmM6bL8ReBJwA58BfwL+CiQB9wKvd9j/l8BU4DzMv8WDgKGUSgPeB54IrP8+8LZSKlEpFQX8D3CZ1todeG9e4Hj/FfisOCAd+O1JvosIUxLooq8ZwE+01i1a6yatdYXW+m2tdaPWug4zFC84wfsLtNa/DzRprACGAMk92VcplQlMB36stfZorTcCa7pR9kcD5f4HZvBe12HbH7XW/9RaG0AOEA38LHD8j4A/AzcopSzA7cD9WusirbVfa/2J1roFuBlYq7Veq7U2tNZ/A3KByzv87SYopSK01iVa652B9V7ME2Kq1ro58H3EACSBLvpamda6uXVBKRWplHo+0PxQC6wHBimlrMd5/+HWGa11Y2A2uof7pgKVHdYBHDpJuau01g0dlgsCx+nq/anAoUC4d9w/DbOW7wL2dfEZQ4F5geaWaqVUNXA+MCTw2fOBxUCJUup9pdTYwPseBBTw70Dzzu0n+S4iTEmgi7529FX4B4AxwNla6xhgVmD98ZpRgqEEiFdKRXZYl3GS98QFmj1aZQLFHZY7fq9iICNQG++4fxFQDjQDI7r4jEPAq1rrQR1eUVrrnwForT/UWl+M+UvjC+D3gfWHtdZ3aK1Tge8Av1NKjTzJ9xFhSAJdhJobs928WikVD/yktz9Qa12A2ZTxmFLKoZQ6F7iyG2/9aWD/mcAVwB+Os99nQCPwoFLKHriweSXwZqDW/jLwK6VUqlLKqpQ6VynlBF4DrlRKXRpY7wpckE1XSiUrpeYGTiotQD1mEwxKqXkdLiRXYZ5cDMSAI4EuQu03QARmzfVfwAd99Lk3AecCFZgXIVdhBuXxHMYMy2LgdWCx1vqLrnbUWnswA/wyzO/1O+DWDvt/H9gObAIqgZ8DFq31IWAu8AhQhlljX4r5/1ML8L3A51diXmdYEjjedOCzQP//NZjt8/t78LcQYUL6oQsBBLr6faG1PuYXQqCG/ZrWOv2YNwpxBpEauhiQlFLTlVIjlFIWpdQczJrxe6EulxCnwxbqAggRIinAO5j90AuBJVrrz0NbJCFOjzS5CCFEmJAmFyGECBMha3JJSEjQWVlZofp4IYTolzZv3lyutU7salvIAj0rK4vc3NxQfbwQQvRLSqmC422TJhchhAgTEuhCCBEmJNCFECJMSD90Ic4wXq+XwsJCmpubT76zCFsul4v09HTsdnu33yOBLsQZprCwELfbTVZWFsd/docIZ1prKioqKCwsZNiwYd1+nzS5CHGGaW5uZvDgwRLmA5hSisGDB/f4V5oEuhBnIAlzcSr/BvpdoG/Kr+TnH3yBDFkghBCd9btA31ZYw7Mf76OmyRvqoggRtqKjj/dUP3Em63eBnuR2AlBad6JnEQghxMDTbwP9SK106RKit2mtWbp0KRMmTCA7O5tVq1YBUFJSwqxZs8jJyWHChAls2LABv9/PwoUL2/b99a9/HeLSDzz9rttiUowLgNJaqaGL8PfTP+1kV3FtUI85LjWGn1w5vlv7vvPOO+Tl5bF161bKy8uZPn06s2bN4o033uDSSy/lhz/8IX6/n8bGRvLy8igqKmLHjh0AVFdXB7Xc4uT6bQ1dmlyE6H0bN27khhtuwGq1kpyczAUXXMCmTZuYPn06y5Yt47HHHmP79u243W6GDx/O/v37uffee/nggw+IiYkJdfEHnH5XQ49y2oh22iitkyYXEf66W5Pua7NmzWL9+vW8//77LFy4kO9973vceuutbN26lQ8//JDnnnuO1atX8/LLL4e6qANKv6uhg1lLlyYXIXrfzJkzWbVqFX6/n7KyMtavX8+MGTMoKCggOTmZO+64g0WLFrFlyxbKy8sxDINrrrmGJ554gi1btoS6+ANOv6uhAyS6nVJDF6IPXH311Xz66adMmjQJpRS/+MUvSElJYcWKFTz99NPY7Xaio6N55ZVXKCoq4rbbbsMwDACeeuqpEJd+4AnZM0WnTZumT/UBF/et/JythdX8Y+mFQS6VEKG3e/duzjrrrFAXQ5wBuvq3oJTarLWe1tX+/brJRe4WFUKIdv0z0GOcNHn91LX4Ql0UIYQ4Y/TPQHdLX3QhhDha/wz0mNa+6HJhVAghWvXPQJcauhBCHKN/BrrU0IUQ4hj9MtDdThsRdqvU0IUQooN+GehKKZJinDKeixBngBONnZ6fn8+ECRP6sDQDW7cCXSk1Rym1Ryn1lVLq4S62L1RKlSml8gKvRcEvamdJbqcMoSuEEB2c9NZ/pZQVeAa4GCgENiml1mitdx216yqt9T29UMZO6j31HKo7RJLbxe6S4A4rKsQZ5y8Pw+HtwT1mSjZc9rPjbn744YfJyMjg7rvvBuCxxx7DZrOxbt06qqqq8Hq9PPHEE8ydO7dHH9vc3MySJUvIzc3FZrPxq1/9igsvvJCdO3dy22234fF4MAyDt99+m9TUVK677joKCwvx+/08+uijzJ8//7S+9kDQnRr6DOArrfV+rbUHeBPo2X/JIFr5xUqu+/N1xLtlCF0hesP8+fNZvXp12/Lq1atZsGAB7777Llu2bGHdunU88MADPb5T+5lnnkEpxfbt21m5ciULFiygubmZ5557jvvvv5+8vDxyc3NJT0/ngw8+IDU1la1bt7Jjxw7mzJkT7K8ZlrozOFcacKjDciFwdhf7XaOUmgXsBb6rtT509A5KqTuBOwEyMzN7XlogNToVAJerlvoWHw0tPqKc/XKMMSFO7gQ16d4yefJkSktLKS4upqysjLi4OFJSUvjud7/L+vXrsVgsFBUVceTIEVJSUrp93I0bN3LvvfcCMHbsWIYOHcrevXs599xzefLJJyksLORb3/oWo0aNIjs7mwceeICHHnqIK664gpkzZ/bW1w0rwboo+icgS2s9EfgbsKKrnbTWL2itp2mtpyUmJp7SB6VFpwFgcVQCUksXojfMmzePt956i1WrVjF//nxef/11ysrK2Lx5M3l5eSQnJ9PcHJxrWDfeeCNr1qwhIiKCyy+/nI8++ojRo0ezZcsWsrOz+dGPfsTjjz8elM8Kd90J9CIgo8NyemBdG611hda6NVlfBKYGp3jHag10v6UCgFK5MCpE0M2fP58333yTt956i3nz5lFTU0NSUhJ2u51169ZRUFDQ42POnDmT119/HYC9e/dy8OBBxowZw/79+xk+fDj33Xcfc+fOZdu2bRQXFxMZGcnNN9/M0qVLZWz1bupOW8UmYJRSahhmkF8P3NhxB6XUEK11SWDxKmB3UEvZQUJEAk6rkyZdBqRKDV2IXjB+/Hjq6upIS0tjyJAh3HTTTVx55ZVkZ2czbdo0xo4d2+Nj3nXXXSxZsoTs7GxsNhvLly/H6XSyevVqXn31Vex2OykpKTzyyCNs2rSJpUuXYrFYsNvtPPvss73wLcNPt8ZDV0pdDvwGsAIva62fVEo9DuRqrdcopZ7CDHIfUAks0Vp/caJjns546Fe+eyVD3SP4898v5tErxvHt84ed0nGEOBPJeOiiVU/HQ+/W1USt9Vpg7VHrftxh/gfAD3pc2lOU5k6jtKkYh80iTS5CCBHQL7uHpEens6N8B4nRcreoEGeC7du3c8stt3Ra53Q6+eyzz0JUooGpXwZ6anQqNS01DI8xZIAuIc4A2dnZ5OXlhboYA16/HMultadLTHSdDNAlhBAB/TrQnZE1Mp6LEEIE9OtAt9grqW320ez1h7hEQggRev0y0Ac5BxFpi8RvMe8WLZMLo0IE1YmGxA215cuXc889vT4OYJeqq6v53e9+F5LP7o5+GehKKVKjUwM3FyHNLkIIfD5fr3+GBHovSY9Op8Z7BJDxXIToLVprli5dyoQJE8jOzmbVqlUAlJSUMGvWLHJycpgwYQIbNmzA7/ezcOHCtn1//etfdzqW3+9n2LBhaK2prq7GarWyfv16AGbNmsWXX3553HIsW7aM0aNHM2PGDP75z3+2rV+4cCGLFy/m7LPP5sEHHyQvL49zzjmHiRMncvXVV1NVVQXA7Nmzuf/++9vK++9//xuAyspKvvnNbzJx4kTOOecctm3bBphDBv/yl79s+5wJEyaQn5/Pww8/zL59+8jJyWHp0qVB+AsHV7/stghm18V/H94EaLm5SIStn//753xRecKbrntsbPxYHprxULf2feedd8jLy2Pr1q2Ul5czffp0Zs2axRtvvMGll17KD3/4Q/x+P42NjeTl5VFUVMSOHTsAszbbkdVqZcyYMezatYsDBw4wZcoUNmzYwNlnn82hQ4cYNWpUl2UoKSnhJz/5CZs3byY2NpYLL7yQyZMnt20vLCzkk08+wWq1MnHiRH77299ywQUX8OMf/5if/vSn/OY3vwFoK+P69eu5/fbb2bFjBz/5yU+YPHky7733Hh999BG33nrrCbtf/uxnP2PHjh1nbBfNfltDT4tOo9HXgM3WJDV0IXrJxo0bueGGG7BarSQnJ3PBBRewadMmpk+fzrJly3jsscfYvn07breb4cOHs3//fu69914++OADYmJijjnezJkzWb9+PevXr+cHP/gBGzdubDve8Xz22WfMnj2bxMREHA7HMQ+6mDdvHlarlZqaGqqrq7ngggsAWLBgQdsvAIAbbrgBMH8N1NbWUl1dzcaNG9tuiLrooouoqKigtrb/Pjin39bQW3u6xA9q4Ij0RRdhqrs16b42a9Ys1q9fz/vvv8/ChQv53ve+x6233srWrVv58MMPee6551i9ejUvv/zyMe979tlnKS4u5vHHH+fpp5/m448/Pq3xzqOiorq1n1LqhMsd2Ww2DMNoWw7WUMG9rf/W0N1moLuja+VuUSF6ycyZM1m1ahV+v5+ysjLWr1/PjBkzKCgoIDk5mTvuuINFixaxZcsWysvLMQyDa665hieeeKLLIW9nzJjBJ598gsViweVykZOTw/PPP8+sWbOOW4azzz6bf/zjH1RUVOD1evnDH/7Q5X6xsbHExcWxYcMGAF599dW22jrQ1v6/ceNGYmNjiY2N7TSk78cff0xCQgIxMTFkZWW1lX/Lli0cOHAAALfbTV1d3Sn8JftGv62htz25KKKGsiqpoQvRG66++mo+/fRTJk2ahFKKX/ziF6SkpLBixQqefvpp7HY70dHRvPLKKxQVFXHbbbe11WyfeuqpY47ndDrJyMjgnHPOAcwTxsqVK8nOzqa4uJhFixaxdm2ncQAZMmQIjz32GOeeey6DBg0iJyfnuOVdsWIFixcvprGxkeHDh7Ns2bK2bS6Xi8mTJ+P1ett+OTz22GPcfvvtTJw4kcjISFasMJ/Nc8011/DKK68wfvx4zj77bEaPHg3A4MGD+drXvsaECRO47LLLePrpp0/jrxt83Ro+tzeczvC5rc5beR6D9TkU7buMLY9eHKSSCRFaMnxu8M2ePZtf/vKXTJvW5aizZ6yeDp/bb5tcwOy66LNUUNngweMzTv4GIYQIY/22yQXMZpe8+r0AlNW3kDYoIsQlEkKciT7++ONQF6FP9Osaelp0GrW+UqQvuhBChEGge40WlLVe+qILIQa8fh/oAMpeJTV0IcSAFxaBbnVUSg1dCDHg9etAb+2LHi1PLhJCiP4d6JH2SOJd8bgiquVuUSFC6ETjp+fn5zNhwoQ+LE3PHD2yYl/Kz8/njTfeCNrx+nWgg9nsYrFXy3guQoig6ovx1YMd6P26HzqYzS77KvKkDV2EpcP/5//Qsju4w+c6zxpLyiOPHHf7ww8/TEZGBnfffTdg1mCjo6NZvHgxc+fOpaqqCq/XyxNPPMHcuXN79NnNzc0sWbKE3NxcbDYbv/rVr7jwwgvZuXMnt912Gx6PB8MwePvtt0lNTeW6666jsLAQv9/Po48+2mmkxdLSUi677DI2b97M1q1bycnJoaCggMzMTEaMGMH27duJjIzsshxPPvkkK1asICkpiYyMDKZOnQqYd5Tm5OS0jTKZk5PD97//fXw+H9OnT+fZZ5/F6XSSlZXFddddx1/+8hciIiJ44403GDlyJPn5+dx+++2Ul5eTmJjIsmXLyMzMZOHChVxxxRVce+21gPmLpr6+nocffpjdu3eTk5PDggUL+O53v9ujv+fRwqKG3qwrqGhowueXu0WFOF3z589n9erVbcurV69m/vz5uFwu3n33XbZs2cK6det44IEH6OnQIc888wxKKbZv387KlStZsGABzc3NPPfcc9x///3k5eWRm5tLeno6H3zwAampqWzdupUdO3YwZ86cTsdKSkqiubmZ2tpaNmzYwLRp09iwYQMFBQUkJSUdN8w3b97Mm2++SV5eHmvXrmXTpk2dtns8HnJzc7n77rtZuHAhq1atYvv27fh8Pp599tm2/WJjY9m+fTv33HMP//mf/wnAvffey4IFC9i2bRs33XQT99133wn/Hj/72c+YOXMmeXl5px3mEAY19LToNAx8YK2jvN5DSqwr1EUSImhOVJPuLZMnT6a0tJTi4mLKysqIi4sjIyMDr9fLI488wvr167FYLBQVFXHkyBFSUlK6feyNGzdy7733AjB27FiGDh3K3r17Offcc3nyyScpLCzkW9/6FqNGjSI7O5sHHniAhx56iCuuuKLLIXbPO+88/vnPf7J+/XoeeeQRPvjgA7TWJxyOd8OGDVx99dVtgX/VVVd12t76K2DPnj0MGzasbWCuBQsW8Mwzz7SFd+v46jfccENbGH/66ae88847ANxyyy08+OCD3f7bBENY1NABLPZKuTAqRJDMmzePt956i1WrVrUF3Ouvv05ZWRmbN28mLy+P5OTkoI0TfuONN7JmzRoiIiK4/PLL+eijjxg9ejRbtmwhOzubH/3oRzz++OPHvG/WrFlttfK5c+eydetWNm7c2Ofjq59obHXoPL66YRh4PJ5TLt+JdCvQlVJzlFJ7lFJfKaUePsF+1yiltFKqz4Y0a+26aN5cJO3oQgTD/PnzefPNN3nrrbeYN28eADU1NSQlJWG321m3bh0FBQU9Pm7H8cf37t3LwYMHGTNmDPv372f48OHcd999zJ07l23btlFcXExkZCQ333wzS5cu7XJ89ZkzZ/Laa68xatQoLBYL8fHxrF27lvPPP/+4ZZg1axbvvfceTU1N1NXV8ac//anL/caMGUN+fj5fffUVcPzx1VetWsW5554LmL8Y3nzzTcA8AbaeWLKysti8eTMAa9aswev1AsEfX/2kTS5KKSvwDHAxUAhsUkqt0VrvOmo/N3A/8FnQStcNrYFukZuLhAia8ePHU1dXR1paGkOGDAHgpptu4sorryQ7O5tp06YxduzYHh/3rrvuYsmSJWRnZ2Oz2Vi+fDlOp5PVq1fz6quvYrfbSUlJ4ZFHHmHTpk0sXboUi8WC3W7v1H7dKisrC6112wMyzj//fAoLC4mLiyM3N5fnnnuOF198sdN7pkyZwvz585k0aRJJSUnHffydy+Vi2bJlzJs3r+2i6OLFi9u2V1VVMXHiRJxOJytXrgTgt7/9LbfddhtPP/1020VRgDvuuIO5c+cyadIk5syZ0/YrYOLEiVitViZNmsTChQtPux39pOOhK6XOBR7TWl8aWP4BgNb6qaP2+w3wN2Ap8H2t9QkHOw/GeOitLlp9EcUlmXxn3CN89+LRQTmmEKEi46Gf+bKyssjNzSUhIaFXP6c3xkNPAw51WC4MrOv4AVOADK31+z0rbnCkRafhcFVLDV0IMaCddi8XpZQF+BWwsBv73gncCZCZmXm6H90mzZ3GdvunlMlFUSFCZvv27dxyyy2d1jmdTj77rE9bYftEfn5+qIvQpe4EehGQ0WE5PbCulRuYAHwcuNKbAqxRSl11dLOL1voF4AUwm1xOo9ydpEal4rdUc6SuMViHFCKktNYn7TlxpsnOziYvLy/UxQgbp/J40O40uWwCRimlhimlHMD1wJoOH1qjtU7QWmdprbOAfwHHhHlvSnengzI43HC4rz5SiF7jcrmoqKg4pf9Di/CgtaaiogKXq2f31Zy0hq619iml7gE+BKzAy1rrnUqpx4FcrfWaEx+h97X2Ra/2lOI3NFZL/6rZCNFReno6hYWFlJWVhbooIoRcLhfp6ek9ek+32tC11muBtUet+/Fx9p3doxIEQWvXRW0zHxid6Hb2dRGECBq73c6wYcNCXQzRD/X7O0UBUqJSUFiwOKrkblEhxIAVFoFut9iJdyZikbtFhRADWFgEOkBqdBpKxnMRQgxgYRPoQ2PTpIYuhBjQwibQM90ZWGx1lNTWh7ooQggREmET6GnuNFCaQ7VFJ99ZCCHCUNgEemqU2XXxSFNJiEsihBChETaBnu42O+BXeeRuUSHEwBQ2gZ4YkYgFK/X+UrllWggxIIVNoFstVty2RLS1kqpGb6iLI4QQfS5sAh0gMWKI3C0qhBiwwirQU6NS5dmiQogBK6wCPSs2A4utnkPV1aEuihBC9LmwCvSR8eZTkPZXF4a4JEII0ffCKtCHDTIfrFRYJ4EuhBh4wirQWx90caSxOMQlEUKIvhdWgZ4QkYDSdipbjoS6KEII0efCKtCVUkRYEqj3S6ALIQaesAp0gFhbMi3IA3aFEANP2AV6QsQQlL2S2mZfqIsihBB9KuwCPTUqFWVt4kBFeaiLIoQQfSrsAn1orNl1cU9FQYhLIoQQfSvsAn104OaifVWHQlwSIYToW2EX6OOTswA4VCs3FwkhBpawC/Q0dwLacHK4UZ5cJIQYWMIu0JVS2Ix4eXKREGLACbtAB4hQidT7S0NdDCGE6FPdCnSl1Byl1B6l1FdKqYe72L5YKbVdKZWnlNqolBoX/KJ2X6w9GY/cXCSEGGBOGuhKKSvwDHAZMA64oYvAfkNrna21zgF+Afwq6CXtgQTXELSlmZqWmlAWQwgh+lR3augzgK+01vu11h7gTWBuxx201rUdFqOAkFaNU6NSAfiySvqiCyEGju4EehrQsVN3YWBdJ0qpu5VS+zBr6PcFp3inJmtQOgBflEmgCyEGjqBdFNVaP6O1HgE8BPyoq32UUncqpXKVUrllZWXB+uhjjIofCsjNRUKIgaU7gV4EZHRYTg+sO543gW92tUFr/YLWeprWelpiYmL3S9lDWXGD0X6XPLlICDGgdCfQNwGjlFLDlFIO4HpgTccdlFKjOix+A/gyeEXsuSS3C8MbLzcXCSEGFNvJdtBa+5RS9wAfAlbgZa31TqXU40Cu1noNcI9S6uuAF6gCFvRmoU8mJsKG8snNRUKIgeWkgQ6gtV4LrD1q3Y87zN8f5HKdFqUUESqBev9etNYopUJdJCGE6HVheacoQIw9BQMPFc0VoS6KEEL0ibAN9ERXCgBF9Se6fiuEEOEjbAM9LdrsKl9UJ4EuhBgYwjbQM2PMm4sKaqTrohBiYAjbQE+LjcXwRbGv+mCoiyKEEH0ibAM9KcaF9sbLk4uEEANG2AZ6cowTwxvHkSa5uUgIMTCEbaAnuV0YnjiqWkrxG/5QF0cIIXpd2AZ6XKQdiz8eAx9lTb03EJgQQpwpwjbQlVLE2JIA6YsuhBgYwjbQARJc5oMuiuuLQ1wSIYTofWEd6KlRQwAorJeeLkKI8BfWgZ4SGw2+GLlbVAgxIIR1oCe7Xfg9cRRKG7oQYgAI60BPCvRFP1QrgS6ECH/hHeiBJxeVN5XiM3yhLo4QQvSqsA70RLcT7YnDwM/hBnl6kRAivIV1oCfHuDC8cYD0RRdChL+wDvTBUQ6UPx6QvuhCiPAX1oFusSjinUmAkr7oQoiwF9aBDpDsjsKh46WGLoQIe+Ef6DFO8MdLG7oQIuyFfaAnul34mgfJ3aJCiLAX9oGe5HbS1BRLaVMpHr8n1MURQoheE/6BHrhbFKSnixAivIV9oCe7XWiPBLoQIvyFfaCbNXSzL7p0XRRChLPwD3S3C+2LwYJNauhCiLDWrUBXSs1RSu1RSn2llHq4i+3fU0rtUkptU0r9r1JqaPCLemoSoh0oZSHKmiBdF4UQYe2kga6UsgLPAJcB44AblFLjjtrtc2Ca1noi8Bbwi2AX9FTZrBYGRzlxIoEuhAhv3amhzwC+0lrv11p7gDeBuR130Fqv01o3Bhb/BaQHt5inJ8ntBJ/cXCSECG/dCfQ04FCH5cLAuuP5NvCXrjYope5USuUqpXLLysq6X8rTlBTjxNcyiMrmShq9jSd/gxBC9ENBvSiqlLoZmAY83dV2rfULWutpWutpiYmJwfzoE0pyO2loiAWgpKGkzz5XCCH6UncCvQjI6LCcHljXiVLq68APgau01i3BKV5wJMe4qK2LBmRcdCFE+OpOoG8CRimlhimlHMD1wJqOOyilJgPPY4Z5afCLeXqS3E78nkBf9Drpiy6ECE8nDXSttQ+4B6Hg/j4AACAASURBVPgQ2A2s1lrvVEo9rpS6KrDb00A08AelVJ5Sas1xDhcSiW4X2h+N3eKQvuhCiLBl685OWuu1wNqj1v24w/zXg1yuoEqKcQKKeEeKNLkIIcJW2N8pCmYbOkCUNVECXQgRWn4v+HrnMuOACPTEaCeA3FwkhOh7hh+KP4d//g+8di38PAt2vtsrH9WtJpf+zmGzEBdpR/niqfXUUuepw+1wh7pYQohwZBhQthsOrIcDG6BgIzTXmNsSRsPE+ZAwqlc+ekAEOpjNLt6W9mF0x8SPCXGJhBBhQWso/xLyAwGevwEaK8xtcVkwbi5kzYKs8yFmSK8WZcAEeqLbSVlDDESZw+hKoAshTonWUJVvBndrLbz+sLktJg1GXQJZM2HYTBiU2adFGzCBnuR28WV5NETJgy6EED1kGFC4CXb9Eb74M1QXmOujkszgHjbLDPH44aBUyIo5cAI9xkl5jY3B6ZFyYVT0X/vWwd8ehYsfhxEXhbo04c3ww8F/mSG+ew3UlYDVYf7dz7vXDPGE0SEN8KMNmEBPdjvxGZASmSqBLvqnne/B24tA+2HljXDzW2a77EDia4Edb4PTDUnjIG4YWILYWc/vMy9i7voj7P4TNJSBzQUjvw7jvgmjLwVXTPA+L8gGTKAnBfqixzmSJdBF/5P7Mvz5e5BxNsx9Bt68EV6/Dm59DzJmhLp0faOmEFYvgKLc9nX2SEgcC8njIGk8JAdeUQndP67PY7aF7/4j7P4zNFWaxx11iXlBc9Ql4IwO/vfpBQMn0N1mX/RoWxK7qz9Ha406g34qCdElrWHDL+GjJ8xgmbcCHJFw6x9h+eXw2jXmfNqUUJe0d+1bB29/2wzfa5fBoKFQuhOO7DKnez6Az19r3z8qqUPIjzNr84ljzb8dmDX9fevMmvie981uhQ43jJljhviI/2jftx8ZQIFu1tCjVTqNvkZWfrGSG8+6McSlEuIEDAP++kP41+/MvstznwGr3dwWMwQW/AmWXQavXg0L/wwp2aEtb28wDNj4K1j3JCSMgfmvtvfhTp/aed/6Ujiy03yV7jKnuS+BrzmwgzIvWsYNhUObwFMHrlgYc7kZ4sMvBLurT79esA2cQI8xa+iptllclLGTp/79FE6rk2tGXxPikgnRBb8X/ng3bFsFZy+GS586tq04Nj0Q6pfDK9+Ehe9D0tjQlLc3NFXDe0tgz1qYcC1c+d8nbvqITjJfIy5sX2f4ofJA59p8xX4Y/02zTXzYLLA5ev+79JEBE+guu5UYl43yOh9PX/E09627j59++lMcVgdXjrgy1MUTop2nEf6wEL78EC78Ecz6/vF7UsRlwa1rzOaXV66C2/4Cg0f0ZWl7x+EdsOpmqDkEc34OZ3/n1HqTWKyQMNJ8jZt78v37uQExlkurpBgXpbUtOKwOfjP7N8xImcGP/vkj/pr/11AXTQhTUzW89i348q/wjf8LFyw9eZAljDRD3fDBiivNm176s61vwotfN5tKFr4P5yw+o7oGnskGVqC7nZTWme1pLpuL/7nof5iUOImH1j/EPw79I8SlEwNe3RFY/g0ozIVrX4bpi7r/3qSx5sVRTwOsuMrsEdLf+FrMnjzvfgfSpsJ31kPmOaEuVb8yoAI9OcbFkdr2YSsj7ZE88x/PMCZ+DN/9+Lt8UvxJCEsnBrTKA/DyJeb0ptUw4Vs9P0ZKNtzyDjRVmaFedzj45ewtNYXmtYDcl+C8+8yTU3RSqEvV7/TPQDeMU3pbkttJWV0LWuu2dW6Hm+cvfp7hscO5/6P72XR4U7BKKUT3HN4BL19qdp1bsOb07gBNmwo3vWWG+StzoaE8eOXsLfs/hudnQdkeuO4VuOS/wDpgLu8FVf8L9F1r4KWvtw9H2QOJbicev0FNk7fT+lhnLC9c8gKp0anc87/3kFeaF6zSCnFiBZ+aNVNlhds+gPRpp3/MzLPNWn5Vgdn7pbHy9I/ZGwwDNvxfs9tlVCLcuW5AXLjsTf0v0B1R6OKt8OZN4G0++f4dtN4tWlp37NNC4l3xvHjJiyREJHDX3+9iV8WuoBRXiOPa+6EZZtGJ8O0Pg9vlMOt8uOENKN9rXmQ9hQpQr2qqhlU3wf8+DuOvhkX/22tjhA8k/S7Q64tsFHw+Df/ejfDunWY/025KDtwteqS26xNBYmQiL17yIm6Hmzv/did7q/YGpcxCHGPrKlh5AySOgds/7J1hVkdcZDZhHN4Or8+Dlvrgf8apOLwDfn+h2ZNnzs/hmpf6za31Z7p+F+jab9C8r4SDmyfi+3wN/OVB8/bobmirodce/3l+Q6KH8OKlL+K0Ornjr3dwoOZAUMotBGD+W/3Xs2ZlZOh55o1BPRl3pKfGzDF7zBTmwsrrzT7ufckwzLbxvJXw/vfhhQvhhQvMckiXxKDrd4HuvuhC0p/5f7SU1HDws7H4NrwM63/Zrfe2jufSVZNLRxnuDF685EUAFv11EYfqDp1eoYXwNMLmFfD8TPjgYRh7hXnxsi9G7hs3F771AuRvNJs5ethU2SO1xeYohX9/zOwT//Oh8MwMeG8xbF0Jjig49x7pkthL+uWl5OhZs8h47lkO3XU3BZ8MZ6j+P9iiE2HqwhO+L8ppI8phbeuLfiLDYofx+0t+z+0f3s6iDxexfM5yhkT37uOjRBiq2AebXoK818x27KTxcMVvYPItfduTI/tas5/3H++C1bfAlAXm4FOOaHNkQUdU+8se2b1ac1O1+fDjos1QtMWctj65x2I3Rz3Mnmf2vEmbaraRW6y9+z0HuH4Z6ABR551HxvPPc2jxYgo2ZpLJA9ijEmHsN074vuTA3aLdMTpuNC9c/AKLPlzEor8uYtmcZSRFSt9YcRKGH778G2z6PXz1d7DY4KwrYcadkHlu6JoYJt8E/sDNO1+e6O5o1SHkA6HfGvSOKDOUj+yCii/b3zJ4FAy/oD28kyf0+4Gu+iOlu9n+HGzTpk3Tubm5J9/xJBo3b+bQHXdidbQw9MIK7IvfgaHnHnf/+c9/iqE1f1h8Xrc/Y2vZVu78652kRKXw8qUvMzhi8GmXW4Shxkr4/FWzRl5dANEpMO02szbcyw8H7pHaEvPBDZ4G8DaYU08jeOrB2xhY7vDyBrZ5Atv8LeZQtKmTzfBOnQwRg0L9rQYMpdRmrXWX/Vv7faADNOXlcfDbi7BaGsm8pAHHfX8xx0Duwr0rP2dbYTX/WHphl9uPJ/dwLkv+voShMUN56dKXiHXGBqPoIhwUbYFNL5pP0vE1w9Cvmbftn3Vl+3C3QgTJiQK9310U7UpETg6Zy5fjJ4qCDyLw/O4aqO76Qmay20lpbee7RbtjWso0/vui/+ZAzQHu+Osd7CjfEYyiDzzeZrMbXQ+6m56RvM3mIFK//w+zC97O92DSDbDkE7htrXnrvoS56GPdakNXSs0B/huwAi9qrX921PZZwG+AicD1Wuu3gl3Qk4nInsDQFSs4uHAhBWtqyLR9E+cDf4PI+E77pcVF0OT1886WIq6Zmt6jzzgv9Tx+feGveWTjI9zw/g3MSp/FXTl3MX7w+GB+lfCjtXnBLO91sxbbXGM+cWba7ebFwag+aMIyDMhfD5uXw96/mu3YNifYIsypPTC1udpfdlfXy801Zo+NxgoYPNLsS51zg/mwBCFC6KRNLkopK7AXuBgoBDYBN2itd3XYJwuIAb4PrOlOoAezyaWj5j17OXjrzdBSw9DrU3B+b615ISegvsXHHSty+XR/Bd+/ZDR3Xziyx4+ia/A28MbuN1i+czm1nlpmZ8zmrkl3cdbgs4L9dfq32mLzAQ15gTsWbREw7iqzu9qOdyB/A1id5p2CM+4w22ODfcGwvtQ8kWxeAVUHwDXIfLiBIxq8TWbPD19g6m0ym0x8zWYN3Nd81HITaAOUBUZfBjMWwbDZwX1IsRAncVpt6Eqpc4HHtNaXBpZ/AKC1fqqLfZcDfw5loAO0fPUVBTffCE3VZC4Yi+v+tzv9/PX4DB58ayvv5RVzw4wM/mvuBGzWnv+fst5Tz2u7X+OVXa9Q56njooyLuCvnLsbEjwnm1+lfvM3mMxrz3oB9H5kBmHku5NxoPiGmY7/r0t3mBcStb5qPAxsyyWx7nnDt6T3P0TBg/zqzNr5nrTlO+NCvmd1az7rq9Hpf+L3md7I5T/0YQpyG0w30a4E5WutFgeVbgLO11vd0se9yThDoSqk7gTsBMjMzpxYUFPTke/RIy4EDHLzxOnRDDZnfOQfXXSs61f601jz94R5+9/E+LhyTyP+7cQpRzlPrxVnrqeX1Xa/z6q5XqfPW8fXMr7MkZwmj40YH6+uc2bQ270TMe92sebfUQGwGTLrebFc+2RN0Wupg22rzwmLpLrPpIucmmPZt8+EN3VV32HxQ8JZXzF4mEfHmiWTKAkgcIP8tRNg7YwK9o96sobfyHDxIwfXfwqirI/M/LyXi2/9zzD6vf1bAo+/tYHxqLC8tnNb2MOlTUeup5dVdr/Larteo99ZzydBLWDJpCSPjehBK/UltsVm7znvD7JNsizDvSsy5EbJm9rwpQms4+KkZ7LvWgOE1H9w7fRGMntP1jTiG3/wlsHk57PkLaL/52VMXmr1MpCYtwsyAa3LpyFNYyMHrrsJf10DGQ9cTefNPj9nnf3cf4Z43PmdwtIPlt81gZNLpDRRU01LDip0reH336zT5mpiTNYfFkxYzfNDw0zruGcHTaDZj5L1hNmtoAzLPCzSpzA3erex1R8ya9uZlUFsEMWkw9TaYciu4k82TSWttvOYQRCaYN85MWRAez9QU4jhON9BtmBdF/wMowrwoeqPWemcX+y7nDAt0AG9RIQXzrsBf20TGj+8g8rrvH7PPtsJqbl++Ca9f8+KCaUzPiu/iSD1T3VzNil1msDf7mrls2GUsnrSYYbHDTvvYfcrnMWvBO96CL9aaN6PEZpjNKZOu790A9ftg7wdmrX3/OvOW8tQcs9eMNmD4bLM2PuYbYfX0diGO57RvLFJKXY7ZLdEKvKy1flIp9TiQq7Veo5SaDrwLxAHNwGGt9Qn78vVloAN4iw5ycN438NZ4yfiv+4n61pJj9jlY0cjCZf+msLqJX1+XwzcmBufuvsrmSpbvXM6bX7xJi7+FS4Zewqz0WUxNnkpqdGrPDub3mnckRif17i3kht/shbLjbbP5o7kaIuLMWviEa2Do+X3fu6P8K/MRZQfWw6hLYMotEB8Gv3qE6IGwv1O0u3yF+zl43ZW0VBrY3X4iUmxEZLiJGJaIa3gGKiaRRns8L22pZXO5lblfm8Q3vzYRFZVo9lM+TRVNFSzfuZx3vnyHWk8tAClRKUxNntr2GhYzrL0bpafRvEhYkgcl2+DwNnMMDX+L+YSX1nEz0qZC2hQzcE+HYUDhJjPEd74LDaVm976xV5ghPny21IKFCDEJ9A78Rfuofu4JmvYcpOlAOb46D2A+AcwV7yUivoWIBA8Rgz3YIzs8u9QRDZGDzSCNSgR3CsSkmlP3kPZXZPxJa86GNviy6ks2H9nMltItbD6ymfIm89mP8VYXU4hkamMdUysKGd3SghXM/tNDJppd+2LSzLstizabY00T+G84eGQg3KeZ05QJJ78oqLV5otjxttlDpeaQefPM6EvNEB91SVBOZkKI4JBAPw6tNb7Dh2naupWmvK00bd1K886daI8Z8h53NMUx0URmxjJlagJRCQYWb5V5s0rdYWjs4gG8VsexIR/TOp8C7lQzZEt3mbXukjz04W0crC9ms8tpviKjKLKaJ4Voq5PJCdlMSf0a01KmMX7weOwdbylvroHiPCjKNccUKcxtH8LU6jCfBJ82zXxWZdpUs4lCKSj/Era/ZQZ5xZfmiIAjLjL7gI+5rG/G6RZC9JgEeg8YHg8tX3zRFvCln23GVd4akDZcY8fgGncW1kGDsES4sDg0VosXi6UJCw1YdR0WfxUWbznWllJU42FzpLrjiR/RXvNOCUyjEjjccJjNRza3vfbX7AfAZXWRnZjNlKQpTEmawsTEiUQ7OvTK0drsAVKUa9bgCzebY1Z7G8ztrkFm+3v5XkCZz56ccI3ZNh55+heChRC9SwL9NH24YSevLlvL1IYiLrdWoPL346+rA6/3pO9VLheW6CiskS4sLhtWpxVrpB378FE4xk/FMWIMjqwsbHEnbv+uaKrg89LP2wJ+T9UeDG1gURbGxI1hSvIUJidNZnLS5GPHbDf8UPZFIOBzoaYQRl1s3rl5Jg3rKoQ4KQn0INhcUMm3V+RiVYqXFk5nUnos2uPBqKvDqK/HX1ePUV+Hv74e44Tz9fgqK/EWFYG/fcRBa2wsjqws8zUsC8fQoeb80KFYIo+9Db7B28DWsq18Xvo5nx/5nG3l22jyNQGQHp3eFvBTkqYwLLbDhVYhRL8mgR4k+8rqWbjs3xypaeGaqWl8Z9YIshKiTv7GLmivF09hIZ78fDz5BXgKAtP8fHyHD3fa15ac3B72Q4fiyMzAGh+PddAgrLGxWGNj8Vk0eyr3sOXIFj4v/ZwtpVuobK4EYJBzEDlJOWYzTfIUxsWP69wO35Nyaw0+H8ouQ8MKEQoS6EFUXt/Cr/+2lz9sLsTnN7g8ewhLZo9gfGrwhk41GhvxHDzYHvb5+W0vf3V1l++xuN1mwLe+YmNoiLRy2NZAARXs8ReTTzl1EQqfy8ZQWwpDrYmkW+JJIYZEI4pBPifOJh9GvflLwl9fh1HfgFFXh78h8AujoQG0xjlqFBGTJxORk0NEziQcWVnyK6Cf0H4/yirP9uyvJNB7QWltMy/98wCv/+sg9S0+Zo9J5K7ZI5kxrHcvLPqqqvAWFuGvrjZfNTXt8x1fgfVGXV2Pju+xgcdlw4hyoaIiscfE4oqJJyouCYc7Fos7GrSmefsOmrZuxag3L/haBw0iYtKk9pDPnoAl6tR+vYjg8NfX49m3j5av9tGybx8t+77C89U+vMXFWGNjsaUOwT4kFfuQIdiHpGAfMgTbkCHYU1OxJSRI6J+hJNB7UU2Tl9f+VcDLGw9Q0eBh2tA4lswewUVjk86IGqv2+dpDvzXkGxqwREVhiYpGR0VQYWnkkK4g31dKflMh+TX55Nfmc6TxSKdjDYkawtCYoQyNGcqQqCGkRCQxpNwg7qtS7Dv307J1G579Zm8cLBacY8YQOTknUIvPwZ6RcUb8TU6V0dyMt6gIb8lhtM9r9ijSADowr80mKa3B0F2sx1xWCktUJFa32/xlFR2NJSYGS1QU6hTuvvVXV9Oyfz8tX33VKcA7Nt0pux3H8OE4R4zAnpGBv6Yab0kJvuISvIcPH3vit9mwJyebId8x+FOHYEtOwRIViXI4sDidKKcT5XCcUtkHKm0Yp/z3kkDvA00eP6tzD/HC+v0UVTcxNsXNktkj+Eb2kFMaa/1M0OhtpKC2gILaAg7UHqCgtoD8mnwO1h6kzts5ABSKhIgEslQCE444GVHoJXlfDe6vSrA0tQBgHTzYrMVPzMaWnIJtcDzWwYOxDR6MLT4e5QjtXahaa/xVVXgPHcJz8BDeQnPqOXQQ76FCfEeOnPwgp0Mp80TbGvJud1voW9zRWKMD804HnoOH2mrd/rL2+yGUy4Vz+HAcI0fgHDES58gRZoinp6Nsxx8e2l9XZwZ8SQnekhK8xYFpSbEZ+keOdLqI32Xx7XYz3J1OlNOBxeFsW7Y4HJ22WaPdWGNjsMTEYo2JMefd5tQaE4Mlxpyeyq8ErTW6qQmjocF8NTZ2nja3YB+SgmPECGyJib1aydBa4yspoXnXLpp27qR51y6ad+0i+cEHib3yylM6pgR6H/L6Df60tZhnP97Hl6X1ZMZHcues4Vw7NR2XPXx+wjZ4GzjccLj91Xi40/KRxiM0+ZpQhiajHEYXacYUwVklFpLKfV0e0x8dgY6LwRIXhy0hAWdCIhGJKTgSk7DGx2NLSMAWH481Ls68Oapj7RfaljGM9hpxh1py68vwePAWF5vBfegQ3oOB6aFD5jWCDmxJSdgzM3CkZ5jTjAzsqanmRWGlAGVOFWYwKBUY4+aoda0vFKAxGhrw19a294KqqzevVdTVmdP6OnNbp3X14DP/dpbISBwjR+IcMQLnyBE4RozAOXKkWbZeqClrvx9fWRne4hJ8pUcwmprRLS1oTwtGSwu6xdPlsuHpsK2lBcPjQTc1mb2+amrQJ+n6a4mONgM+NhD8MW4s0W6zh1nHkD4quOlmrlliYjr/DQMnQVtKSo+DXmuN99AhM7R37qI5EOBt172sVpwjRuAaN45B115D5LQuM/mkJNBDwDA0f999hN99vI+8Q9UkRDv59vnDuPmcTNyu8O8horWm1lNLSUPJMcFfVVWCv6ISVVWDtboeV20LsY0Q26Dbpw0Q2wDu5t4tp3I4sKenm0GdkYEjMzDNyMCeno7FdRpPNwoyrTW6uRmjuRnroEH9uvkK2r+Pv7YWf02NedKqqcVfW4NRWxuYr8WorcFfW9c+X1dvNvdERWGJisQSGRmYj8IaFYWKjMQaWO64rXVZOZ14i4oCTVPmdYWWffvwV1a2le2YE+bw4eYJMy0NZbGgDQNPfkEgvNtr3m1NV3Y7zlEjcY0bR8T48bjGjcM5ZkxQ/j1JoIeQ1ppP91fw7Mf72PBlOW6XjRtnZHLeyARyMgYRGxH+4X4yXsNLnaeOmpYaaj211LbUmlNPLbUNlTRXlOErL8eorISqGlRtAz6/F6/hw6u9eLUPn/abTdTKfNE6DxCYtq73WaA2zok/NQFHUgqJ0UkkRiSSFJlEYmQiSRGBaWQSUXa5sDtQ+CorzWsQbdchzLD3lZW17aNcLhwZ6XiLis1fApiVAueYMbjGjcM1fhyuceNxjh6FpZeaECXQzxDbC2t49h9f8cGOw+Y1M2BkUjSTMwYxOTOOyZmDGJ3sxmrp3zWvUPAZPlr8LTT5mmj2NZsvf3P7sr99XYOngfKmckqbSilvKqessaytiehokbZIEiMTSYxI7BT2CREJDI4YTILLnMY6Y7Go/nmtRJyYv6aGln372wLec/Ag9tTUtgB3jhjRp/dlSKCfYeqavWwvrGHLwSo+P1jN54eqqWwwBwSLcliZlDGIyZmDmJwRR07mIBKi5TFqfaHB20BpYylljWWUNpnTsqYyc7mxtG2+2X9sO5BN2Yh3xZsh3xr2EQkMdrUvt65z2939vrlEhI4E+hlOa83BykY+P1jdFvK7S2rxBarxmfGRTMlsr8WfNSQGez/tOdPfaa2p89ZR0VRBeVM5FU0VVDS3z5c3lZvzzRVUNlXi08deALYpG1ZL5wvkis4Bf6LAVyhcNhcRtohOr+Oti7RFHrPOqqxYlRWlFFZlxaIs5tRi6bR8zHZlwWaxtR1TTkx9TwK9H2ry+NlRXMOWgqq2oC+tM7v/OWwWzhoSQ3ZaDBNSY5mQFsvoZDcOm4T8mcTQBrUttWbIN7cHflVzFQYdxto/6v+C+qgVR/9/1MBoa1Zq8jWd8OU1Tj6A3KmyKRvRjmjcDjfR9mhiHDHHXXbb3eZ6RzTRdnN0UL/2Y2ij89Qwp12ta13WWhNhj8Btd7cf3+HGaR0Yv2Ql0MOA1pqSmmY+P1hN3qEqthfVsLOolroWswbosFoYO8TN+NRYstPM1+iUaJy28OkqKXrOZ/i6DP7WgOwYqIY28Bv+Y9cdNe8zfDR4G6j31lPnqWt7tS7Xemqp99TT6Gvs0+9qt9jbTiYdTyStJ5HWE0y0PRqrxdrpV5FSqm1ZoTotmz1QO2xXCq01Bmb32I7zhjbQ6E7zhjY6749mesp0RseNPqXvKYEepgxDU1DZGAj3GrYX1bCjqIbaZjPk7VbF6GQ32WlmLT47LZYxKe6w6g8vzlytwd8a8HWeOuq8ddR76lFKtTXjtL4syoLVYu20vqt1SikavY2dTiit80d/Tr23PmQnmBN59JxHuW7Mdaf0Xgn0AaS1Pd4M91p2BIK+psn86W2zKEYmRTM8MYqhg6PIGhwZmEaR5HZikR42Ikz5DT/1XjPkDcNs8tKt/wvkYOtya6tX67bWZrDWZYuyYFGWttr80fMWLG21fIvqPG9RFiJsETisp9atUQJ9gNNaU1jV1Bbuu0tqKaho5GBlY9uFVwCX3cLQ+CiyEiLJGtwh8BOiGBLjkrAX4gxwokA//uAOImwopciIjyQjPpLLstufUOTzG5TUNJNf0UB+RSMF5eZ0f1kD6/aU4fG1X7hz2CxkxkeSNTiS9LhI4iIdxETYiI2wE+OyExsZmEbYiYmwEWG3Sg8IIfqYBPoAZrNa2oJ+5qjO2wxDc7g2EPbljRRUNJBf0UBBRSP/2l9JfUvX47G0sltVW8C7I+zEuALhHzgBxETYcLvM9W6XOd9xGu2wyS8CIXpIAl10yWJRpA6KIHVQBOeNOHa7z29Q1+yjpslLbbPXnDYdveyltnWfJi9FVU1t27z+Ezf1KQXRThsxbUHfMfRtRDltRDlsRDqsRDpsRDkDU4eVSOdRU4dNunSKAUECXZwSm9VCXJSDuKieX9jRWtPiM6ht9lLX7KOu2UdtU+t8+7S22ddhHy9Hapv5qtScb/D4OzUJnYzdqtoC3+WwYlUKi1KBQRDNDmkWC+Y6CFzc6jDF3NeiFFaLwu2ytf3SaP3lEdv268NObIStbV56FYm+IoEu+pxSCpfdistuJcl96sfx+Q0avX4aW/w0eHztU4+PhhZ/p2mjx0+jx09Di49Gr3lzimEQ6CccGPlPg6HN/gzHrAtM/Vrj8RqU1jW3/Spp8p54nHCHzRIIexsxEXbcLnugn7LGb5jl8GuNz9AYRmBdYJtfB9Zpjd8fWNZgDZxY7FZzarNYzGnb8lHrW5etCqvFgt2qcFgt2KwKu9XSaf7oZYfVgt1mHstuteCwKRxWKw6bBYfNgjMwdVgtOO0WnB22hWpcIt3h7+c3Ov+d/UbnnizQPqs79G7hmHXtx24djdkf2BRI6wAAC5xJREFU+O+oA/9djMC/q47/Zgzd/m/M/G8LwxKiSIkN/kieEuii37JZLcRYLcSEeDhiT+DXRmvTUs1RTU21RzdLNXpAKawKbBYLFgvYLZa22r/NotrmLRbVFt7WwLzFAoYBXsPAb5gnAr8/MDWMwNRcbvL629f729d7/Ubg1XnebwS315vNojqHfiD4oUOetj3MSbc91MnsHhjY3DZt397Via71ROgz2t97pnrimxO4+ZyhQT9utwJdKTUH+G/ACryotf7ZUdudwCvAVKACmK+1zg9uUYU4MzlsFhKinWExiJphaLyGGe4+v4HH3z7v9Rt4fOYJwOM38PgMWnz+wNR8eXxG27LHZ+Dx+2nxmvu3TtuaysxngJizbU1dtDV5qcA+5kR12IZ5slOBk9/RJ70OJ8TOJ0LzJGq1qLYeWB07YrXfCdpWvPZt6th9LIGTqyXQfGc5qpmudbvqsK11v+GJvTMs80kDXSllBZ4BLgYKgU1KqTVa610ddvs2UKW1HqmUuv7/t3fuwVpVZRj/PYiYgHITDYHUvI2XURRiLBFNzFFzRE3SUkezxsm01K6ajnmpGa/ZP42XxCJFU/HumOIVayZRQEAQVFRUSC5NhpnjBXj6Y60zfNF3Dt/anHPgfL6/mT3f2vvbz1pr7732u9dea+13AVcAx3dEhoMg6Di6dRObdduEzeLdvUvSSNf/SGCB7ddtfwz8CRi71j5jgQk5PAkYoxiEHARB0Kk0YtAHA2/XrC/K2+ruY3slsAIYsHZEkk6XNE3StOU1s4AEQRAE60+nDs61faPtEbZHDBw4sDOTDoIgaHoaMeiLgaE160Pytrr7SOoO9CF1jgZBEASdRCMG/XlgZ0k7SOoBnAA8sNY+DwCn5PBxwJPeUF6/giAIPqWssy/b9kpJZwGPkoYt3mx7rqRLgWm2HwDGA7dIWgD8k2T0gyAIgk6kocFJth8GHl5r20U14Q+Bce2btSAIgqCE8FgUBEHQJGywCS4kLQferCjfCvjHeiQf+tBvSP3GkIfQd139drbrDxNsmeS0Ky2ktvvQh75L6jeGPIS+a+tbW6LJJQiCoEkIgx4EQdAkdFWDfmPoQ9+F9RtDHkLftfV12WCdokEQBEH70lVr6EEQBMFahEEPgiBoErqcQZd0mKSXJS2QdF6h9mZJyyTNqZj2UElPSXpJ0lxJZxfqPyPpOUmzsv6SivnYRNILkh6qoF0o6UVJMyVNq6DvK2mSpPmS5kn6YoF215xuy/KepHMK0z83n7s5km6XVDQxo6Szs3ZuI2nXKzOS+kt6TNKr+bdfoX5cTn+1pBEV0r8qn//Zku6V1LdQf1nWzpQ0WdK2Jfqa/34kyZK2Kkz/YkmLa8rBEaXpS/p+PgdzJV1ZmP4dNWkvlDSzUD9M0rMt95CkkYX6vSX9Ld+HD0rasjV9MR0xFrKjFpIvmdeAzwM9gFnA7gX60cC+wJyK6Q8C9s3hLYBXCtMX0DuHNwWmAvtVyMcPgduAhypoFwJbrcc1mAB8J4d7AH3X41ouIX0k0ahmMPAGsHlevxM4tUC/JzAH6Elye/E4sFNpmQGuBM7L4fOAKwr1uwG7Ak8DIyqkfyjQPYevqJD+ljXhHwDXl+jz9qEk/05vtlWeWkn/YuDHDV6zevov52u3WV7fujT/Nf9fA1xUmP5k4PAcPgJ4ulD/PHBgDp8GXNZoGV7X0tVq6I3MntQqtp8hOQ+rhO13bM/I4X8D8/j/yT7a0tv2+3l107wU9UpLGgJ8FbipRNceSOpDKqDjAWx/bPtfFaMbA7xmu/Rr4e7A5kpumnsCfy/Q7gZMtf2B00QsU4Bj2xK0UmZqZ+iaABxdorc9z/bLjWS4Ff3knH+AZ0kurUv079Ws9qKNMtjGPXMt8NO2tOvQN0Qr+jOAy21/lPdZViV9SQK+DtxeqDfQUqvuQxtlsBX9LsAzOfwY8LXW9KV0NYPeyOxJnYKk7YF9SLXsEt0m+RVvGfCY7SI98BvSjbS6UNeCgcmSpks6vVC7A7Ac+H1u8rlJUtXZbk+gjRupHrYXA1cDbwHvACtsTy6IYg5wgKQBknqSaldD16Gpxza238nhJcA2FeJoL04D/lwqkvQrSW8DJwIXrWv/tbRjgcW2Z5WmW8NZudnn5raarFphF9J1nCppiqQvVMzDAcBS268W6s4Brsrn72rg/EL9XNZURMdRrQzWpasZ9I0CSb2Bu4Fz1qrtrBPbq2wPI9WqRkrasyDdI4FltqcXZfh/GWV7X+Bw4ExJowu03Umvj9fZ3gf4D6nJoQglv/pHAXcV6vqRboQdgG2BXpJOalRvex6piWIy8AgwE1hVkoc6cZrCt6z2QtIFwEpgYqnW9gW2h2btWQVp9gR+TuFDYC2uA3YEhpEezNcU6rsD/YH9gJ8Ad+badinfoLBSkTkDODefv3PJb6wFnAZ8T9J0UtPtxxXyUJeuZtAbmT2pQ5G0KcmYT7R9T9V4clPFU8BhBbL9gaMkLSQ1Nx0s6dbCdBfn32XAvaRmrEZZBCyqeauYRDLwpRwOzLC9tFB3CPCG7eW2PwHuAb5UEoHt8baH2x4NvEvqByllqaRBAPm31Vf+jkLSqcCRwIn5oVKViZS98u9IeqDOyuVwCDBD0mcbjcD20lyxWQ38jrIyCKkc3pObMJ8jva222jFbj9xkdyxwR2HakCbzabn376Iw/7bn2z7U9nDSA+W1CnmoS1cz6I3MntRh5FrAeGCe7V9X0A9sGZEgaXPgK8D8RvW2z7c9xPb2pGN/0nbDNVRJvSRt0RImda41POLH9hLgbUm75k1jgJca1ddQtWb0FrCfpJ75Wowh9WM0jKSt8+/nSDf0bRXyUTtD1ynA/RXiqIykw0jNbkfZ/qCCfuea1bGUlcEXbW9te/tcDheRBgosKUh/UM3qMRSUwcx9pI5RJO1C6pwv9Vx4CDDf9qJCHaQ28wNz+GCgqMmmpgx2Ay4Erq+Qh/q0V+9qZy2kds9XSE+1Cwq1t5Ne8T4hFcRvF+pHkV6vZ5Ne12cCRxTo9wJeyPo5tNG73kBcB1E4yoU0OmhWXuaWnr8cxzBgWj6G+4B+hfpepPlm+1Q87ktIBmgOcAt5pEOB/i+kh9AsYEyVMgMMAJ4g3ciPA/0L9cfk8EfAUuDRQv0CUl9SSxlsa5RKPf3d+fzNBh4EBle9Z1jHqKlW0r8FeDGn/wAwqFDfA7g1H8MM4ODS/AN/AL5b8fqPAqbnMjQVGF6oP5tkw14BLid/sd8eS3z6HwRB0CR0tSaXIAiCoBXCoAdBEDQJYdCDIAiahDDoQRAETUIY9CAIgiYhDHoQNIikg1TBw2UQdBZh0IMgCJqEMOhB0yHpJCW/8zMl3ZAdor0v6drsP/sJSQPzvi2+rVt8i/fL23eS9LiS7/oZknbM0ffWGn/wE1t8iEi6XMlP/mxJV2+gQw8+5YRBD5oKSbsBxwP7OzlBW0XyKNgLmGZ7D5Lb3F9kyR+Bn9nei/T1Ysv2icBvbe9N8hfT4l1xH5K3vd1JX97uL2kA6evPPXI8v+zYowyC+oRBD5qNMcBw4PnspngMyfCuZo0jpluBUdm/e1/bU/L2CcDo7O9msO17AWx/6DU+U56zvcjJsdRMYHtgBfAhMF7SsUCxf5UgaA/CoAfNhoAJtoflZVfbF9fZr6rPi49qwqtIMwetJHncm0TygPhIxbiDYL0Igx40G08Ax9V4tOsvaTtSWT8u7/NN4K+2VwDvSjogbz8ZmOI0G9UiSUfnODbLfsDrkv3j97H9MMk/9t4dcWBBsC66b+gMBEF7YvslSReSZmXqRvJydyZpMo6R+b9lpHZ2SO5vr88G+3XgW3n7ycANki7NcYxrI9ktgPuVJqwWac7XIOh0wtti8KlA0vu2e2/ofARBRxJNLkEQBE1C1NCDIAiahKihB0EQNAlh0IMgCJqEMOhBEARNQhj0IAiCJiEMehAEQZPwX68TGrtUvjjJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "dropout_mdl = MyDropoutModel()\n",
    "dropout_opt = tf.keras.optimizers.RMSprop(learning_rate)\n",
    "dropout_mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=dropout_opt, metrics=[\"accuracy\"])\n",
    "\n",
    "history_dropout = dropout_mdl.fit(x_train_mnist, y_train_mnist, batch_size, epochs, validation_split=0.1)\n",
    "\n",
    "plt.plot(history_no_dropout.history[\"loss\"])\n",
    "plt.plot(history_no_dropout.history[\"val_loss\"])\n",
    "plt.plot(history_dropout.history[\"loss\"])\n",
    "plt.plot(history_dropout.history[\"val_loss\"])\n",
    "plt.legend([\"loss\", \"val_loss\", \"loss w. dropout\", \"val_loss w. dropout\"])\n",
    "plt.xticks(range(epochs))\n",
    "plt.xlabel(\"epochs\")\n",
    "plt.title(\"Training process\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bKRoJPWNNBxS"
   },
   "source": [
    "While the model without dropout can reach a much lower loss on the training data, it generalizes very poorly to the unseen validation data. The model with dropout however generalizes quite good to the validation data and achieves a similar loss on both the data used during training and unseen data. This demonstrates that using dropout can be a good way to regularize your networks and prevent overfitting. Although dropout helps to prevent overfitting very well, it introduces another hyperparameter, the drop probability, that needs to be optimized. Common values for this hyperparameter are on the interval $\\left[0.2,0.5\\right]$.\n",
    "\n",
    "## Transfer Learning\n",
    "In many applications there is only a limited amount of annotated data available. In order to still train a neural network that generalizes well on such a data set, one can use transfer learning. In transfer learning the goal is to transfer knowledge from a source domain or task to a target domain or task. The hope is that this transfer will be positive, i.e. the performance on the target domain or task increases compared to only training on the target data set. A requirement for succesfull transfer learning is that the source and target have something in common, e.g. similar features. There is a broad literature on transfer learning methods but in this exercise we will restrict ourselves to the most basic approach, fine tuning. Fine tuning can be implemented by using a part of a neural network, which was trained on the source domain or task, as a feature extractor. A common approach is to use neral network pretrained on the ImageNet data set as a feature extractor. In this exercise, we will also use a neural network pretrained on ImageNet in order to fine tune it for the [Caltech 101](http://www.vision.caltech.edu/Image_Datasets/Caltech101/) data set. For a guide on transfer learning using fine tuning with Keras click [here](https://www.tensorflow.org/beta/tutorials/images/transfer_learning).\n",
    "\n",
    "In order to do this we first need to download an extract the data set. Note that there is a wide collection of data sets available through the [tensorflow_datasets](https://www.tensorflow.org/datasets) package including Caltech101. But for demonstration purposses we manually download and load the images using Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 219
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 141439,
     "status": "ok",
     "timestamp": 1562670784561,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "WZLz-aD6tgvR",
    "outputId": "ab6dea3e-8c64-4701-d4a8-8024a0a81023"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-04-24 15:27:12--  http://www.vision.caltech.edu/Image_Datasets/Caltech101/101_ObjectCategories.tar.gz\n",
      "Resolving www.vision.caltech.edu (www.vision.caltech.edu)... 34.208.54.77\n",
      "Connecting to www.vision.caltech.edu (www.vision.caltech.edu)|34.208.54.77|:80... connected.\n",
      "HTTP request sent, awaiting response... 304 Not Modified\n",
      "File ‘101_ObjectCategories.tar.gz’ not modified on server. Omitting download.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -N http://www.vision.caltech.edu/Image_Datasets/Caltech101/101_ObjectCategories.tar.gz\n",
    "!tar -xzf 101_ObjectCategories.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4YVR0yd-3A3w"
   },
   "source": [
    "In order to feed our model with the data, we will use the ImageDataGenerator class provided by Keras. This class can be used for reading files from a structured directory, create the labels based on the structure of the directory and apply data augmentation techniques. The ImageDataGenerator already supports a lot of techniques for data augmentation. In our example, we use a random rotation, width shift, height shift, shearing, zomming and horizontal flipping. All of those operations are applied randomly to individual images. More information on the available transformations for data augmentation and how to use them see the [documentation](https://www.tensorflow.org/versions/r2.2/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator#__init__)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WueTCO3F3iKl"
   },
   "outputs": [],
   "source": [
    "N_samples_Caltech101 = 9247\n",
    "val_split = 0.1\n",
    "datagen = k.preprocessing.image.ImageDataGenerator(validation_split=val_split,\n",
    "                                                   preprocessing_function=k.applications.mobilenet_v2.preprocess_input,\n",
    "                                                   rotation_range=20,\n",
    "                                                   width_shift_range=0.1,\n",
    "                                                   height_shift_range=0.1,\n",
    "                                                   shear_range=0.1,\n",
    "                                                   zoom_range=0.1,\n",
    "                                                   horizontal_flip=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PyqbHZfO0ein"
   },
   "source": [
    "Now that we have the data set we can load the pretrained model. Keras provides a range of pretrained models called \"applications\", click [here](https://www.tensorflow.org/api_docs/python/tf/keras/applications) for a link to its documentation. We will use the MobileNetV2 architecture without it's output layer, since we want to modify it in order to apply the model to the Caltech 101 data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 146354,
     "status": "ok",
     "timestamp": 1562670789494,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "ZkGHDLKZ0uUy",
    "outputId": "64658f88-3f28-4169-ef24-f51c60ea3c06"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"mobilenetv2_1.00_224\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            \n",
      "__________________________________________________________________________________________________\n",
      "Conv1_pad (ZeroPadding2D)       (None, 225, 225, 3)  0           input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "Conv1 (Conv2D)                  (None, 112, 112, 32) 864         Conv1_pad[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "bn_Conv1 (BatchNormalization)   (None, 112, 112, 32) 128         Conv1[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "Conv1_relu (ReLU)               (None, 112, 112, 32) 0           bn_Conv1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "expanded_conv_depthwise (Depthw (None, 112, 112, 32) 288         Conv1_relu[0][0]                 \n",
      "__________________________________________________________________________________________________\n",
      "expanded_conv_depthwise_BN (Bat (None, 112, 112, 32) 128         expanded_conv_depthwise[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "expanded_conv_depthwise_relu (R (None, 112, 112, 32) 0           expanded_conv_depthwise_BN[0][0] \n",
      "__________________________________________________________________________________________________\n",
      "expanded_conv_project (Conv2D)  (None, 112, 112, 16) 512         expanded_conv_depthwise_relu[0][0\n",
      "__________________________________________________________________________________________________\n",
      "expanded_conv_project_BN (Batch (None, 112, 112, 16) 64          expanded_conv_project[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_1_expand (Conv2D)         (None, 112, 112, 96) 1536        expanded_conv_project_BN[0][0]   \n",
      "__________________________________________________________________________________________________\n",
      "block_1_expand_BN (BatchNormali (None, 112, 112, 96) 384         block_1_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_1_expand_relu (ReLU)      (None, 112, 112, 96) 0           block_1_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_1_pad (ZeroPadding2D)     (None, 113, 113, 96) 0           block_1_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_1_depthwise (DepthwiseCon (None, 56, 56, 96)   864         block_1_pad[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_1_depthwise_BN (BatchNorm (None, 56, 56, 96)   384         block_1_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_1_depthwise_relu (ReLU)   (None, 56, 56, 96)   0           block_1_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_1_project (Conv2D)        (None, 56, 56, 24)   2304        block_1_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_1_project_BN (BatchNormal (None, 56, 56, 24)   96          block_1_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_2_expand (Conv2D)         (None, 56, 56, 144)  3456        block_1_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_2_expand_BN (BatchNormali (None, 56, 56, 144)  576         block_2_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_2_expand_relu (ReLU)      (None, 56, 56, 144)  0           block_2_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_2_depthwise (DepthwiseCon (None, 56, 56, 144)  1296        block_2_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_2_depthwise_BN (BatchNorm (None, 56, 56, 144)  576         block_2_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_2_depthwise_relu (ReLU)   (None, 56, 56, 144)  0           block_2_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_2_project (Conv2D)        (None, 56, 56, 24)   3456        block_2_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_2_project_BN (BatchNormal (None, 56, 56, 24)   96          block_2_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_2_add (Add)               (None, 56, 56, 24)   0           block_1_project_BN[0][0]         \n",
      "                                                                 block_2_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_3_expand (Conv2D)         (None, 56, 56, 144)  3456        block_2_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_3_expand_BN (BatchNormali (None, 56, 56, 144)  576         block_3_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_3_expand_relu (ReLU)      (None, 56, 56, 144)  0           block_3_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_3_pad (ZeroPadding2D)     (None, 57, 57, 144)  0           block_3_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_3_depthwise (DepthwiseCon (None, 28, 28, 144)  1296        block_3_pad[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_3_depthwise_BN (BatchNorm (None, 28, 28, 144)  576         block_3_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_3_depthwise_relu (ReLU)   (None, 28, 28, 144)  0           block_3_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_3_project (Conv2D)        (None, 28, 28, 32)   4608        block_3_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_3_project_BN (BatchNormal (None, 28, 28, 32)   128         block_3_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_4_expand (Conv2D)         (None, 28, 28, 192)  6144        block_3_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_4_expand_BN (BatchNormali (None, 28, 28, 192)  768         block_4_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_4_expand_relu (ReLU)      (None, 28, 28, 192)  0           block_4_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_4_depthwise (DepthwiseCon (None, 28, 28, 192)  1728        block_4_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_4_depthwise_BN (BatchNorm (None, 28, 28, 192)  768         block_4_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_4_depthwise_relu (ReLU)   (None, 28, 28, 192)  0           block_4_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_4_project (Conv2D)        (None, 28, 28, 32)   6144        block_4_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_4_project_BN (BatchNormal (None, 28, 28, 32)   128         block_4_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_4_add (Add)               (None, 28, 28, 32)   0           block_3_project_BN[0][0]         \n",
      "                                                                 block_4_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_5_expand (Conv2D)         (None, 28, 28, 192)  6144        block_4_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_5_expand_BN (BatchNormali (None, 28, 28, 192)  768         block_5_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_5_expand_relu (ReLU)      (None, 28, 28, 192)  0           block_5_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_5_depthwise (DepthwiseCon (None, 28, 28, 192)  1728        block_5_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_5_depthwise_BN (BatchNorm (None, 28, 28, 192)  768         block_5_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_5_depthwise_relu (ReLU)   (None, 28, 28, 192)  0           block_5_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_5_project (Conv2D)        (None, 28, 28, 32)   6144        block_5_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_5_project_BN (BatchNormal (None, 28, 28, 32)   128         block_5_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_5_add (Add)               (None, 28, 28, 32)   0           block_4_add[0][0]                \n",
      "                                                                 block_5_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_6_expand (Conv2D)         (None, 28, 28, 192)  6144        block_5_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_6_expand_BN (BatchNormali (None, 28, 28, 192)  768         block_6_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_6_expand_relu (ReLU)      (None, 28, 28, 192)  0           block_6_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_6_pad (ZeroPadding2D)     (None, 29, 29, 192)  0           block_6_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_6_depthwise (DepthwiseCon (None, 14, 14, 192)  1728        block_6_pad[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_6_depthwise_BN (BatchNorm (None, 14, 14, 192)  768         block_6_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_6_depthwise_relu (ReLU)   (None, 14, 14, 192)  0           block_6_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_6_project (Conv2D)        (None, 14, 14, 64)   12288       block_6_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_6_project_BN (BatchNormal (None, 14, 14, 64)   256         block_6_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_7_expand (Conv2D)         (None, 14, 14, 384)  24576       block_6_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_7_expand_BN (BatchNormali (None, 14, 14, 384)  1536        block_7_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_7_expand_relu (ReLU)      (None, 14, 14, 384)  0           block_7_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_7_depthwise (DepthwiseCon (None, 14, 14, 384)  3456        block_7_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_7_depthwise_BN (BatchNorm (None, 14, 14, 384)  1536        block_7_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_7_depthwise_relu (ReLU)   (None, 14, 14, 384)  0           block_7_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_7_project (Conv2D)        (None, 14, 14, 64)   24576       block_7_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_7_project_BN (BatchNormal (None, 14, 14, 64)   256         block_7_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_7_add (Add)               (None, 14, 14, 64)   0           block_6_project_BN[0][0]         \n",
      "                                                                 block_7_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_8_expand (Conv2D)         (None, 14, 14, 384)  24576       block_7_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_8_expand_BN (BatchNormali (None, 14, 14, 384)  1536        block_8_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_8_expand_relu (ReLU)      (None, 14, 14, 384)  0           block_8_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_8_depthwise (DepthwiseCon (None, 14, 14, 384)  3456        block_8_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_8_depthwise_BN (BatchNorm (None, 14, 14, 384)  1536        block_8_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_8_depthwise_relu (ReLU)   (None, 14, 14, 384)  0           block_8_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_8_project (Conv2D)        (None, 14, 14, 64)   24576       block_8_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_8_project_BN (BatchNormal (None, 14, 14, 64)   256         block_8_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_8_add (Add)               (None, 14, 14, 64)   0           block_7_add[0][0]                \n",
      "                                                                 block_8_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_9_expand (Conv2D)         (None, 14, 14, 384)  24576       block_8_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_9_expand_BN (BatchNormali (None, 14, 14, 384)  1536        block_9_expand[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "block_9_expand_relu (ReLU)      (None, 14, 14, 384)  0           block_9_expand_BN[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_9_depthwise (DepthwiseCon (None, 14, 14, 384)  3456        block_9_expand_relu[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_9_depthwise_BN (BatchNorm (None, 14, 14, 384)  1536        block_9_depthwise[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "block_9_depthwise_relu (ReLU)   (None, 14, 14, 384)  0           block_9_depthwise_BN[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_9_project (Conv2D)        (None, 14, 14, 64)   24576       block_9_depthwise_relu[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "block_9_project_BN (BatchNormal (None, 14, 14, 64)   256         block_9_project[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_9_add (Add)               (None, 14, 14, 64)   0           block_8_add[0][0]                \n",
      "                                                                 block_9_project_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_10_expand (Conv2D)        (None, 14, 14, 384)  24576       block_9_add[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "block_10_expand_BN (BatchNormal (None, 14, 14, 384)  1536        block_10_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_10_expand_relu (ReLU)     (None, 14, 14, 384)  0           block_10_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_10_depthwise (DepthwiseCo (None, 14, 14, 384)  3456        block_10_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_10_depthwise_BN (BatchNor (None, 14, 14, 384)  1536        block_10_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_10_depthwise_relu (ReLU)  (None, 14, 14, 384)  0           block_10_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_10_project (Conv2D)       (None, 14, 14, 96)   36864       block_10_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_10_project_BN (BatchNorma (None, 14, 14, 96)   384         block_10_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_11_expand (Conv2D)        (None, 14, 14, 576)  55296       block_10_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_11_expand_BN (BatchNormal (None, 14, 14, 576)  2304        block_11_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_11_expand_relu (ReLU)     (None, 14, 14, 576)  0           block_11_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_11_depthwise (DepthwiseCo (None, 14, 14, 576)  5184        block_11_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_11_depthwise_BN (BatchNor (None, 14, 14, 576)  2304        block_11_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_11_depthwise_relu (ReLU)  (None, 14, 14, 576)  0           block_11_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_11_project (Conv2D)       (None, 14, 14, 96)   55296       block_11_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_11_project_BN (BatchNorma (None, 14, 14, 96)   384         block_11_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_11_add (Add)              (None, 14, 14, 96)   0           block_10_project_BN[0][0]        \n",
      "                                                                 block_11_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_12_expand (Conv2D)        (None, 14, 14, 576)  55296       block_11_add[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "block_12_expand_BN (BatchNormal (None, 14, 14, 576)  2304        block_12_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_12_expand_relu (ReLU)     (None, 14, 14, 576)  0           block_12_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_12_depthwise (DepthwiseCo (None, 14, 14, 576)  5184        block_12_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_12_depthwise_BN (BatchNor (None, 14, 14, 576)  2304        block_12_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_12_depthwise_relu (ReLU)  (None, 14, 14, 576)  0           block_12_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_12_project (Conv2D)       (None, 14, 14, 96)   55296       block_12_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_12_project_BN (BatchNorma (None, 14, 14, 96)   384         block_12_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_12_add (Add)              (None, 14, 14, 96)   0           block_11_add[0][0]               \n",
      "                                                                 block_12_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_13_expand (Conv2D)        (None, 14, 14, 576)  55296       block_12_add[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "block_13_expand_BN (BatchNormal (None, 14, 14, 576)  2304        block_13_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_13_expand_relu (ReLU)     (None, 14, 14, 576)  0           block_13_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_13_pad (ZeroPadding2D)    (None, 15, 15, 576)  0           block_13_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_13_depthwise (DepthwiseCo (None, 7, 7, 576)    5184        block_13_pad[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "block_13_depthwise_BN (BatchNor (None, 7, 7, 576)    2304        block_13_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_13_depthwise_relu (ReLU)  (None, 7, 7, 576)    0           block_13_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_13_project (Conv2D)       (None, 7, 7, 160)    92160       block_13_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_13_project_BN (BatchNorma (None, 7, 7, 160)    640         block_13_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_14_expand (Conv2D)        (None, 7, 7, 960)    153600      block_13_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_14_expand_BN (BatchNormal (None, 7, 7, 960)    3840        block_14_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_14_expand_relu (ReLU)     (None, 7, 7, 960)    0           block_14_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_14_depthwise (DepthwiseCo (None, 7, 7, 960)    8640        block_14_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_14_depthwise_BN (BatchNor (None, 7, 7, 960)    3840        block_14_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_14_depthwise_relu (ReLU)  (None, 7, 7, 960)    0           block_14_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_14_project (Conv2D)       (None, 7, 7, 160)    153600      block_14_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_14_project_BN (BatchNorma (None, 7, 7, 160)    640         block_14_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_14_add (Add)              (None, 7, 7, 160)    0           block_13_project_BN[0][0]        \n",
      "                                                                 block_14_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_15_expand (Conv2D)        (None, 7, 7, 960)    153600      block_14_add[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "block_15_expand_BN (BatchNormal (None, 7, 7, 960)    3840        block_15_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_15_expand_relu (ReLU)     (None, 7, 7, 960)    0           block_15_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_15_depthwise (DepthwiseCo (None, 7, 7, 960)    8640        block_15_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_15_depthwise_BN (BatchNor (None, 7, 7, 960)    3840        block_15_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_15_depthwise_relu (ReLU)  (None, 7, 7, 960)    0           block_15_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_15_project (Conv2D)       (None, 7, 7, 160)    153600      block_15_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_15_project_BN (BatchNorma (None, 7, 7, 160)    640         block_15_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "block_15_add (Add)              (None, 7, 7, 160)    0           block_14_add[0][0]               \n",
      "                                                                 block_15_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "block_16_expand (Conv2D)        (None, 7, 7, 960)    153600      block_15_add[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "block_16_expand_BN (BatchNormal (None, 7, 7, 960)    3840        block_16_expand[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "block_16_expand_relu (ReLU)     (None, 7, 7, 960)    0           block_16_expand_BN[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_16_depthwise (DepthwiseCo (None, 7, 7, 960)    8640        block_16_expand_relu[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "block_16_depthwise_BN (BatchNor (None, 7, 7, 960)    3840        block_16_depthwise[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "block_16_depthwise_relu (ReLU)  (None, 7, 7, 960)    0           block_16_depthwise_BN[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "block_16_project (Conv2D)       (None, 7, 7, 320)    307200      block_16_depthwise_relu[0][0]    \n",
      "__________________________________________________________________________________________________\n",
      "block_16_project_BN (BatchNorma (None, 7, 7, 320)    1280        block_16_project[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "Conv_1 (Conv2D)                 (None, 7, 7, 1280)   409600      block_16_project_BN[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "Conv_1_bn (BatchNormalization)  (None, 7, 7, 1280)   5120        Conv_1[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "out_relu (ReLU)                 (None, 7, 7, 1280)   0           Conv_1_bn[0][0]                  \n",
      "==================================================================================================\n",
      "Total params: 2,257,984\n",
      "Trainable params: 2,223,872\n",
      "Non-trainable params: 34,112\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "base_model = k.applications.MobileNetV2(input_shape=(224, 224, 3), include_top=False, weights='imagenet')\n",
    "base_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ngVtdWor2z62"
   },
   "source": [
    "With the pretrained model we can build our actual model that adds an output layer to the MobileNetV2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HQNsTLZI4GOW"
   },
   "outputs": [],
   "source": [
    "class MyTransferModel(k.Model):\n",
    "    def __init__(self, pretrained_model):\n",
    "        super(MyTransferModel, self).__init__()\n",
    "        self.pretrained_model = pretrained_model\n",
    "        self.glbl_avg_pool = k.layers.GlobalAveragePooling2D()\n",
    "        self.dropout = k.layers.Dropout(0.25)\n",
    "        self.dense = k.layers.Dense(102, activation=\"softmax\")\n",
    "\n",
    "    def call(self, inputs, training=False):\n",
    "        output = self.pretrained_model(inputs)\n",
    "        output = self.glbl_avg_pool(output)\n",
    "        output = self.dropout(output)\n",
    "        output = self.dense(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "w5w6aV4S4nHu"
   },
   "source": [
    "Now we just need to instantiate our model for transfer learning and fine tune it. But instead of directly training all layers we will just train the last layer first. If you do not do this and directly train the complete model, the random initialization of the last layer can cause gradients with very big magnitude that will be propageted into the MobileNetV2 layers and cause them to \"forget\" what they have learned on ImageNet. This is undesireable since we want to transfer that knowledge over into our model in order to achieve better performance on the Caltech 101 data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 715
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1428690,
     "status": "ok",
     "timestamp": 1562672071845,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "ayjQnTqz47CO",
    "outputId": "327ec09e-3af8-4e03-d80d-3e58d8086e78"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"my_transfer_model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "mobilenetv2_1.00_224 (Model) (None, 7, 7, 1280)        2257984   \n",
      "_________________________________________________________________\n",
      "global_average_pooling2d (Gl multiple                  0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          multiple                  0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              multiple                  130662    \n",
      "=================================================================\n",
      "Total params: 2,388,646\n",
      "Trainable params: 130,662\n",
      "Non-trainable params: 2,257,984\n",
      "_________________________________________________________________\n",
      "Found 8278 images belonging to 102 classes.\n",
      "Found 866 images belonging to 102 classes.\n",
      "Epoch 1/10\n",
      "260/260 [==============================] - 63s 243ms/step - loss: 1.3355 - accuracy: 0.6999 - val_loss: 0.4454 - val_accuracy: 0.8707\n",
      "Epoch 2/10\n",
      "260/260 [==============================] - 63s 244ms/step - loss: 0.3653 - accuracy: 0.8934 - val_loss: 0.4732 - val_accuracy: 0.8926\n",
      "Epoch 3/10\n",
      "260/260 [==============================] - 63s 243ms/step - loss: 0.2790 - accuracy: 0.9146 - val_loss: 0.3368 - val_accuracy: 0.8995\n",
      "Epoch 4/10\n",
      "260/260 [==============================] - 64s 245ms/step - loss: 0.2226 - accuracy: 0.9325 - val_loss: 0.3785 - val_accuracy: 0.8949\n",
      "Epoch 5/10\n",
      "260/260 [==============================] - 64s 245ms/step - loss: 0.2067 - accuracy: 0.9406 - val_loss: 0.3367 - val_accuracy: 0.9145\n",
      "Epoch 6/10\n",
      "260/260 [==============================] - 64s 244ms/step - loss: 0.1856 - accuracy: 0.9422 - val_loss: 0.3387 - val_accuracy: 0.9053\n",
      "Epoch 7/10\n",
      "260/260 [==============================] - 64s 244ms/step - loss: 0.1687 - accuracy: 0.9508 - val_loss: 0.3004 - val_accuracy: 0.9169\n",
      "Epoch 8/10\n",
      "260/260 [==============================] - 63s 241ms/step - loss: 0.1599 - accuracy: 0.9542 - val_loss: 0.3013 - val_accuracy: 0.9192\n",
      "Epoch 9/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 0.1434 - accuracy: 0.9555 - val_loss: 0.3728 - val_accuracy: 0.9122\n",
      "Epoch 10/10\n",
      "260/260 [==============================] - 62s 239ms/step - loss: 0.1357 - accuracy: 0.9586 - val_loss: 0.3274 - val_accuracy: 0.9134\n"
     ]
    }
   ],
   "source": [
    "tf_batch_size = 32\n",
    "tf_epochs = 10\n",
    "tf_learning_rate = 0.001\n",
    "tf_mdl = MyTransferModel(base_model)\n",
    "tf_opt = k.optimizers.RMSprop(tf_learning_rate)\n",
    "base_model.trainable = False\n",
    "tf_mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=tf_opt, metrics=[\"accuracy\"])\n",
    "tf_mdl.build((tf_batch_size, 224, 224, 3))\n",
    "tf_mdl.summary()\n",
    "\n",
    "train_gen = datagen.flow_from_directory(\"101_ObjectCategories\", target_size=(224, 224), class_mode=\"sparse\", batch_size=tf_batch_size, subset=\"training\")\n",
    "val_gen = datagen.flow_from_directory(\"101_ObjectCategories\", target_size=(224, 224), class_mode=\"sparse\", batch_size=tf_batch_size, subset=\"validation\")\n",
    "tf_history_0 = tf_mdl.fit(train_gen, validation_data=val_gen, steps_per_epoch=int((1.0-val_split)*N_samples_Caltech101/tf_batch_size), validation_steps=int(val_split*N_samples_Caltech101/tf_batch_size), epochs=tf_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HbKiWk_T4zH1"
   },
   "source": [
    "Now that we have trained the outputlayer of our model on the Caltech 101 data set, we can make the last layers of the MobileNetV2 model trainable and continue to fine tune  it with a low learning rate. For this we need to recompile our model in order for the change of the MobileNetV2 parameters to trainable to have an effect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 678
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 3501498,
     "status": "ok",
     "timestamp": 1562674144665,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "yNL1HtEp5R2W",
    "outputId": "5d7f0927-96e4-4e03-a1c9-639a3e8f5283"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"my_transfer_model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "mobilenetv2_1.00_224 (Model) (None, 7, 7, 1280)        2257984   \n",
      "_________________________________________________________________\n",
      "global_average_pooling2d (Gl multiple                  0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          multiple                  0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              multiple                  130662    \n",
      "=================================================================\n",
      "Total params: 2,388,646\n",
      "Trainable params: 2,354,534\n",
      "Non-trainable params: 34,112\n",
      "_________________________________________________________________\n",
      "Epoch 1/10\n",
      "260/260 [==============================] - 64s 246ms/step - loss: 0.4107 - accuracy: 0.8862 - val_loss: 0.3158 - val_accuracy: 0.9203\n",
      "Epoch 2/10\n",
      "260/260 [==============================] - 63s 241ms/step - loss: 0.2256 - accuracy: 0.9342 - val_loss: 0.3229 - val_accuracy: 0.9145\n",
      "Epoch 3/10\n",
      "260/260 [==============================] - 63s 241ms/step - loss: 0.1694 - accuracy: 0.9483 - val_loss: 0.3285 - val_accuracy: 0.9180\n",
      "Epoch 4/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 0.1426 - accuracy: 0.9542 - val_loss: 0.2743 - val_accuracy: 0.9169\n",
      "Epoch 5/10\n",
      "260/260 [==============================] - 63s 243ms/step - loss: 0.1198 - accuracy: 0.9591 - val_loss: 0.2728 - val_accuracy: 0.9192\n",
      "Epoch 6/10\n",
      "260/260 [==============================] - 64s 244ms/step - loss: 0.1091 - accuracy: 0.9638 - val_loss: 0.2651 - val_accuracy: 0.9342\n",
      "Epoch 7/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 0.0996 - accuracy: 0.9670 - val_loss: 0.2382 - val_accuracy: 0.9249\n",
      "Epoch 8/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 0.0959 - accuracy: 0.9681 - val_loss: 0.2475 - val_accuracy: 0.9307\n",
      "Epoch 9/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 0.0838 - accuracy: 0.9721 - val_loss: 0.2783 - val_accuracy: 0.9238\n",
      "Epoch 10/10\n",
      "260/260 [==============================] - 63s 241ms/step - loss: 0.0815 - accuracy: 0.9739 - val_loss: 0.2763 - val_accuracy: 0.9284\n"
     ]
    }
   ],
   "source": [
    "tf_learning_rate = 0.00001\n",
    "tf_opt = k.optimizers.RMSprop(tf_learning_rate)\n",
    "base_model.trainable = True\n",
    "tf_mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=tf_opt, metrics=[\"accuracy\"])\n",
    "tf_mdl.build((tf_batch_size, 224, 224, 3))\n",
    "tf_mdl.summary()\n",
    "\n",
    "tf_history_1 = tf_mdl.fit(train_gen, validation_data=val_gen, steps_per_epoch=int((1.0-val_split)*N_samples_Caltech101/tf_batch_size), validation_steps=int(val_split*N_samples_Caltech101/tf_batch_size), epochs=tf_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GNo2frodRH_i"
   },
   "source": [
    "As you can see from the increasing gap between training and validation loss and accuracy, overfitting is becoming a bigger problem now. This is due to the much higher number of trainable parameters if we not only train the output layer on the rather small Caltech 101 data set. But still we can improve the performance of our model. We can visualize this if we plot the accuracy over the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 3501833,
     "status": "ok",
     "timestamp": 1562674145014,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "g2QHrVsvRv0C",
    "outputId": "b4eac623-0229-464e-87c6-e61a3f9f10b2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhU5dn48e+dnewhYQ2EBGQXCRBxQwoqFrXVurSAS6Uu1LrUpfpq39pWa/3pW9u3dq+4FFst4IuKiLgAIlJBZd8SkC1IFrZA9kzW5/fHcxIGmISZkEmGzP25rrly5sx5znkmmZx7nl2MMSillFInCunoDCillApMGiCUUkp5pAFCKaWURxoglFJKeaQBQimllEdhHZ2BtpKSkmLS09M7OhsqCG0v2g7A4OTBHZwTpXy3du3aw8aYbp5e6zQBIj09nTVr1nR0NlQQmjBrAgCfTP+kQ/OhVGuIyN7mXtMqJqWUUh5pgFBKKeWRBgillFIeaYBQSinlkQYIpZRSHmmAUEop5ZEGCKWUUh51mnEQSinVmRljKK2q40hlDUcqqikqr+FIRQ1FFTUkRodz03n92vyaGiCUUqqDGGPIO1pF3tEqjlQ4N/6KGmf72KOoooajFTXUNXhev2d0WqIGCKWUOlOVumr5an8ZOfvL2FZYyrb9ZWzfX0Z5dd1Jx8ZHhZEcG0nXmAj6do0ms28iSTERJMdE0NV5JMdEkhQTTnJMJF0iQv2SZw0QSinVhurqG8gtqiCn0AaAbftLySksI7+4qumY+KgwhvSK5/rRqQzuGU96cjRdY+2NPyk6gvDQwGge1gChlFJeqK6rp7K6nvLqOiprGn/WUVFdR97RKrY5weCrA+XU1DUAEBoiDOgWw5h+Sdx0fhpDe8YzuGccvRKiEJEOfkenpgFCKRV0SqpqyT1cwR7ncbi8+ribfnl1PZXV9uZfUVNPZU0dtfWe6/8bdYuLZEjPOKZfmM6QnnEM6RnPgO4xRIb5p/qnPWiAUEp1SpU1deQeriS36Fgg2HO4gtzDFRRV1DQdJwJJ0RHERIYSExFGTGQYCV3C6Z0QRUxkGDERofZnZBjRjdsRYfZ4Z7tHfCTJsZEd+G79QwOEUuqMZYzh6yOVfHWgnD2Hy9lzuLKpZLC/1HXcsT3iI0lPjuHy4T1IT44hI8U++naNJir8zP2W708aIJRSZ4yi8mo25hWzYV8JG/cVszGvmOLK2qbXu8ZEkJ4czUVnpZCREk1GSizpKdGkJ8cQE6m3O1/pb0wpFZCqaurZUmADwQbnkXfU9gQKERjUI47Jw3sysm8iQ3rG0T8lloTo8A7OdeeiAUIp1eHq6hvYcbC8qVSwYV8JXx0oo94ZGJaa2IXMvol8/4J+jOyTyNmpCVoiaAf6G1aqFVy19eQdrST3cCX7S1wk6jdXnxwur2bd3qOs+7qYdV8fZXNeCVW19YAdIzCybyKXDR3AyD6JjOybSLe4ztcAfCbQAKFUM6pq6tl7pILcw5XsLaogt8j+3FtUSUFJFcbp9bg/ooKkqoiOzWwAq61vYFthGeu+Ptr02HfEVhWFhQjDe8cz5dy+ZPa1wSA9OfqMGCMQDDRAqKDkqq2n1FVLmauO0qpaCopd5BZVHBcIDpRWH5ema0wE/ZKjGZvRlX7JtuGzX3I018yN4nB5NVU19X6b8uBMcmLpYFNeMa5aO3Cse1wko9OSuOX8foxOS+Ls1ATtQRTANECoM5YxhqKKGvYcrqCovIYyVy2lrjrK3G78Za46yqpPeO6qo6a+weM5u8VFkp4czcUDu5GeHE2/5BjSk2NIS44moYvnaqTk2AgOlrlYseMQlw/v6c+3HHAaGgzb9pexZu+RpqDw9ZFK4FjpYOq5aYzul8TotERSE7to6eAM4tcAISKTgT8AocBLxphnT3i9H/AK0A04AtxsjMlzXqsHNjuHfm2MudqfeVWBy1Vbz96iSnYfKmf34Qp2HSpn96EKdh8qp9R18kRnANERocRHhRMXFUZcVBhJ0RH0S45peh4fFU58VBhxzjE9E6LolxxDbCsaPuOjwgkNEZbkHAiKALHvSCWf7TzMf3YeZuWuIo44g84aSwc3n5+mpYNOwm8BQkRCgb8Ak4A8YLWILDDGZLsd9lvgn8aYV0XkEuAZ4BbntSpjTKa/8qcCizGG/aWuphv/rkMV7D5st/OLj9X3A/SMj6J/txiuzuxNRkos/bvF0D0usikgxEaGEdaOk501jsRdmnOQ+gZDaEjn+oZcXFnDql1FrNh5mM92HmZvkS0hdI+LZMKgblx0Vgrn9e+qpYNOyJ8liLHATmPMbgARmQNcA7gHiGHAQ872MmC+H/OjAkypq5Z3Nxbw9rp8sgtLqaypb3otOiKU/t1iGJ2WxPWj+9C/WwwDusWSkRKYA56SoiMoOlLDhn1HGdOva0dn57S4autZu/co/3ECwub8EoyB2Mgwzu/flekXpjPurBTO6h6rAaGT8+d/Wiqwz+15HnDeCcdsBK7DVkNdC8SJSLIxpgiIEpE1QB3wrDHmpOAhIjOAGQBpaWlt/w5UmzPG8MWeI7yxeh+LthTiqm1gcI84ppzbl/7dYhmQEkP/brH0iI88o24+idHhVIUIH2UfOOMCRH2DIbugtCkgrM49QnVdA+Ghwqi0JB68bBAXnZXCOX0SAmYaatU+Ovqr2MPAn0VkOvApkA80fo3sZ4zJF5H+wMcistkYs8s9sTFmJjATICsrq+WpFlWH2l/i4s11ebyxZh97iyqJiwzj+tF9+F5WX87pk3BGBQNPQkOE8/snsyT7AD+9YmhHZ8drn351iMfnb2lqWB7SM45bzu/HRQNTGJveNSBLa6r9+POvnw/0dXvex9nXxBhTgC1BICKxwPXGmGLntXzn524R+QQYBRwXIFRgq6lrYGnOAd5Ys4/lXx2iwcD5/bvywGUDmTy8V6frEjppWA9+uWAruw+V079bbEdnp0VF5dX8+r0c3l6fT/+UGH4/ZSTjzuqmA9LUcfwZIFYDA0UkAxsYpgI3uh8gIinAEWNMA/BTbI8mRCQJqDTGVDvHXAT8xo95VW3oqwNlzF29j7fX53Okooae8VHcPeEsvpvVh37JMR2dPb+5dGh3frlgK0tyDjAjQAOEMYa31uXz6/eyKa+u48eXnMXdE8/S3kbKI78FCGNMnYjcC3yI7eb6ijFmq4j8ClhjjFkATACeERGDrWK6x0k+FHhBRBqAEGwbRPZJF1EBo9RVy8KNhcxds4+N+4oJDxUmDevBd7P6Mn5gt07Xs8eTPknRDOsVz+LsA8wYP6Cjs3OS3MMV/Gz+Zj7bWcSYfkk8c90IBvWI6+hsqQDm1wpGY8wiYNEJ+37htj0PmOch3UpghD/zprxjjKG8uo7iylqOVtZwpKKG4spa52cNRytrOVDq4tMdh5oanH/+rWF8J7N3p1xA5VQuG9aDP3+8g6Ly6oB5/7X1Dby4Yjd/WLKDiNAQnvrO2dw0No2QIAja6vRoC1SQO1JRw4db97PvSCVHK2spdg8ClTYINLfUoggkdgknKTqC60b3YUonaXA+HZcP68Efl+7g420H+W5W31Mn8LMN+4p57M1NbNtfxuThPXni6uH0TIjq6GypM4QGiCDkqq1nac5B3l6fxyfbD1HnDO5KirY3ezvqOJpRaYkkRkfQNSbc/oyOIMltO75LeFBUHflieO94esZHsSTnQIcGiPLqOn730XZmrcylR1wUL9wyhm8GwShv1bY0QASJhgbDl7lHeHtdPos2F1JWXUeP+EhuH5fBd0alMqRnXFB/828rIsJlw7rz5tp8XLX1HdL4uzTnAD+fv4XCUhe3nN+PR745mLgonY5c+U4DRCe382A5b6/PY/76AvKLq4iOCGXy2T25blQfLhiQrCUAP5g0rCevff41K3cd5pIhPdrtugdLXTz5bjbvbS5kUI9Y5t14IWP6JbXb9VXnowGiEzpcXm2nsFifz6a8EkIExg3sxiPfHMzlw3sQHaF/dn86v39XYiPDWJx9oF0CREODYe6affy/RTlU1zXw8OWDmDF+ABFhOupZnR69U3QSrtp6Fmcf4O31+Sz/6hD1DYZhveJ5/KqhXD2yN93jtWGyvUSGhfKNQd1YknOQpxuM33sL3f36Oj7Yup/z+3fl/107IuAH6bWp0gL46HGoq4Yx02HApRCigbGtaIA4w1VU1/Hs+9uYvz6fsuo6esZHccfFGVw3qg+De2of945y2bDuvLe5kI15xYxK8181z5rcI3ywdT/3XXIWD00aFDztSMbAprnw/n9BXQ1ExsK2hZCUDlm3w6ibIfrMmhMrEGmAOIPtLargzn+uYefBcr4zKpUbRvfhvP7arhAIJg7u3rRGhD8DxN+X7yYpOpy7J5wVPMGh/CC8+wBsfw/6ngff+Rsk9IWcBbD6JVj8c1j2NJx9PZx7B6SO9l9eaqtg35cQGgF9x0JI5xqRrgHiDLVixyHu/fd6ROCft53HuIEpHZ0l5SYxOoJz05NYnH2AR745xC/X2HGgjCU5B3jgsoGdbl6rZm19GxY+BDUVMOkpuOCeYzflETfYx/4tNlBsegM2vA6pY2ygGH4dhJ9mVWt9HRSsg93LYc9yGxzqnaVpo5Nh0BUw5CoYMBHCu5zetQKABogzjDGGF1fs5tn3tzGoRxwzb8kiLTm6o7OlPJg0rCdPLcxmb1GFX+agmvnpbrqEh3LrBeltfu6AU1EEix6GrW9B79G21NC9mcDb82z49vMw6UnYOMcGi/k/gg9/Zquezr3dVkV5o6EBDm6FPZ/aoLB3JdSUOdcZAWPvhIzxtiSx7T3IeRc2vAbh0TDgEhssBk1u++qu+lo4mGODVf5aiIyHbz7dttdAA8QZxVVbz2NvbmL+hgKuHNGT524YqdMxB7BJQ3vw1MJsFmcf4I6L+7fpufeXuJi/IZ+bzutHUkxEm5474GxbBO/eD1VH4ZLH4aIHIdSLz31UApz3Qxg7w97gV78Eq/4CK/8EAyfBuXfCWZcd36htDBzZbUsHez61j8oi+1rXAXDOdyHjG5B+McQkH3+94d+x7SF7P7PBYtt7tl1EQqHfhTZYDL4Skvr59v4bGmyeCtZBvhMQ9m+COpfzPhNh8BW+ndNLenc5Q+QXV/HDf61ha0EpD18+iHsmBlGd8xkqLTmawT3iWJLT9gHilc/20GDg9nEZbXregFJVDB88BhtnQ48RcMtb9lu7r0Sg/zfsoyQf1r0Ka2fBv78Lif1siSK257GgUOKscxbXGwZebksIGeMhoc+prxUWYauXBkyEK5+DgvXHgsUHj9lHzxEw5Fs2YPQ42+avkTG2Z1ZjMChYB/nrobrEvh4eDb1G2ob41NH2kZRx/DnakAaIM8AXu4u4+/V11NQ18NL3s7h0aPsNvlKn57Jh3fn78t0UV9aQGN023/RLqmr59xdf861zetG3ayetXtyxBBbcB+UHYPx/wfhH7M33dCWkwsT/hosftt/uV78Ei535Q7t0hYyLYdwDkDEBkgec3o1X5NhN/NKfQ9Eu2L7IBotPnoVPnoHENBh8lW2/aAwK5ftt+pAw6DEczr7OnqP3aOg2xLvSUxvRABHAjDG89vlennw3m7TkaGbeksVZ3YOoj3sncNnQHvxl2S6WbT/ItaO8+Abqhde/2Et5dR0zxrdtqSQgVJfZtoJ1r9qb4dTX/dMLKSzC3njPvg4OfWWra3qc7d8xFMkD4ML77KP8EHz1vg0Wa16xDd3JA6H/hGPBoOeI029UP00aIAJUdV09v3xnK3NW7+OSId15fmom8TqfzhlnZJ9EusVFsiS7bQKEq7aef3yWy/hB3RjeO6ENchhA9nwK8++xVTwX3Q8T/rt9bpDdBvn/GieK7Qajv28fNZXQUAdR8e2fj1PQABGADpa6uOu1taz7uph7J57Fg5MG6diGljQ0BOzo2ZAQ4bKh3VmwoYDqunoiw5wumcbAoW22muOrj+xAr8a67l6Zzfann78+n0Nl1fxhSicqPdRUwJIn4csXbEPwbR9C2nkdnav2ExG41YQaIALM+q+PctdraymtquOvN43myhG9OjpLHccY23OlrBBKC+3PskLbiFe2H8qcn+UHoWt/2+g35FvQ59yAChiThvVg9pf7+HznIb4RnWuDwrb3bM8UsNUJZfthyRP2eVSC7SWTMd72mOk2GESobzDM/HQ3I1ITuGBAcnOX8019LRzdC/XOaOSIWIiMg9A2KK0aA9Wl9r2VFrj9/QqP3y4/AKYezrsLLv1lQN8wg40GiADyf2v28bO3t9AjIZK37r6Qob0Cr8jZ5oyBXR/DwezjbxxlhfbG0tiVz110MsT1so9eIyGmGxRsgM//Biv/CDHdbbe/Id+yN9mOrMetdTGuYR3PRbzEmHn3QN1RCAm3PWouvM8OrIp3vgSUH3S6Vi63fe63LbT7Y3tAxniyI0ZSUxTHT6Z90/cebK4SOLzDeXzlPHbYINVQe/LxYVHHgkVkLETEuW037nceEbFQW+n55l9bcfK5oxJsD6G4njBgiP058HJIO9+396T8TozxvFrYmSYrK8usWbOmo7PRKrX1DTz9Xg6zVuZy0VnJ/Hna6M7ft90Y2LUUPv617QoIENbF3iwbb/7HbTs3lLheENbMUp6uEtix2H4737HYDmgKj4GBl9lgMXASdGn7aS8mzJoAwCfTP7E7qo7aaqNtC2HnUqitoCokhhWMYtK1tyEDJ3lX33w0t2mAltnzKVJxEACT2A/p/w1busgYD7Hd7fENDVCad+zm7/6z/MCx84aE2RJXyiBIGWgbRyOibQNxdTnUlNtv/k3b5fa1mjK37XIbFNyFRh77G8X3OhYE3P92cb20hBBgRGStMSbL42saIDpWqauWu/61lpW7irh9XAY/vWIIYaHtWD1SXWarNpLSIfOm9pngbO9KWPoUfL0SEtJgwqO2eigqse36c9dVw54V9ia9fZG9QYaEQfo4GywGX+Fdv3YvTJg1Aeqq+WTED+z19n5mGx1jezrVXlcx70gGD7+Vw7v3jmNEH98bl7/YdZjHX3qTpzOPMLZhM+T+51jf+G5DbdfHwzuhrupYoqgEJwg4gaBxOym9baqQ6utsoKgpt/3zuyT5rT++8h8NEAHqaEUNt/7jS7ILSnn2+nO4YUzb3LC8VrYfXnfmrsHYaoWzb4Cxd0DvUW1/vfx1tsSwa6mtNhn/iO3F0VyJoK00NNjRp43B4vBXdn+vTGfA0pX2m23jN+PmvjFXlx1/jPNNe8KhdVDn4hNiIGXwsbaQ3qOa2kKKyqs59+kl3HvJQB6a5HuvmdtmrWbjvmI+e+wSu0pdfR3s32ironL/AxJyciCISdEbtjolDRAB6GCZi1te+pI9RRX87abR7T/47dB2eO0GO43A9161N8g1L8PGubbeOHWMnYpg+LWnX4d/MMcGhm0L7bfMcQ/ac3dUVcOhr+xMoNveg7zV3qcLizpW5x4Za+e/iYhlQsEKiIjhk5vetzfoZnz37yspr67n/fsv9im72/eX8c3nP+UnkwZx36XNn1+p1mgpQGgjdQcoKK7ippe+YH+Ji39MP5eLzmrnmVj3roLZU+0UxT9471hp4Vu/h8uesEFi9Ysw/y748L9h9C2QdZv3E5w1OrLbjhjd9Ia9qU74KZx/d8f39+42yD7GPWhLUTs+sn3RT2yAbQoEznZz1TJOG0RLwQHsoLln3t9G3tFK+iR5Hxxf+HQX0RGh3HKBj3P4KHWaNEC0s9zDFdz00heUVtXyr9vHkpXezouabJ0Pb82wQ/xvnnfyTT8qAc6bYWepzF0BX74IK/8Mn/3RbYKzS1ue974kHz79Dax/zfbYuejHcNEDgbmAS1xPW83VDiYNswFiSfYBpl/k3RxK+cVVLNhQwPcvSG+zqTqU8pYGiHa040AZN730BbX1DcyecT5np7bzSNhVf7Ulgr5jYdqclm/YIscGbpUW2MnNTpzgLPPm42e0LD8E//lfWP0ymAZb6rj4J/YmrOjfLZb+3WJYknPQ6wDxyn/2AHD7xZ14Uj4VsDRAtJMt+SV8/5UvCQ0R5sy4oH2XA21osKtsrfqzbTy9/iXfFjOJ720nOBv/iJ3vvnGCs4+ftnPZjLoZdi2z4xDqqmDkjfCN//J9WuMgMGlYD15esYeSqloSurTck6ikspbZX37N1SN7k5p45i8+o848GiDawdq9R5n+jy+Jiwzj9TvPJyOl7RePaVaty7YlbH0bxv4QJj/T+mURQ8OPTXB2INtp1J5jp2MGu2LXxP8+ZV18MJs0tAcvLN/N8q8OcfXI3i0e+6/Pc6msqWfGNzrRtBrqjOLXDvciMllEtovIThF5zMPr/URkqYhsEpFPRKSP22u3isgO53GrP/PpTyt3HeaWl78gOSaCN+66oH2DQ9VReO06GxwmPQVX/E/brZnbYxhc9Tt4KAeuexHu+gy++w8NDqcwKi2J5JgIFmcfaPE4V209s1bmMnFwN4b0DIIR9Sog+a0EISKhwF+ASUAesFpEFhhjst0O+y3wT2PMqyJyCfAMcIuIdAV+CWQBBljrpD3qr/z6w7JtB7nrtbX0S47mtdvPo3t8O075ULwPXrseju6B61+2a/X6Q1Q8nPM9/5y7EwoNES4Z0p0Ptu6npq6BiDDP39Hmrc3jcHkNP/zGgHbOoVLH+LMEMRbYaYzZbYypAeYA15xwzDDgY2d7mdvr3wQWG2OOOEFhMTDZj3ltc4s2FzLjX2sY2COWOTMuaN/gULgJXrrMduG8+S3/BQfVKpOG9aDMVceXe454fL2+wa47PrJvIudlBGDPLxU0/NkGkQrsc3ueB5w4h+9G4DrgD8C1QJyIJDeTNtV/WW1bb67N45F5GxmVlsQ/fnCu53Uc3Ef37vrYTkCXOubYYiHxrZzFddfHMPf79pv9bR/YqiAVUMYNTCEyLIQlOQcYN/DkMTAfbt3P3qJKfnrFEF1WVnWojm6kfhj4s4hMBz4F8oF6bxOLyAxgBkBaWpo/8uezf32+l5/P38KFA5J58ftZxES6/Yqbmx8o7QKoPAz/+b2d9hjsyObUMXYQW+po+/NUE81tnAPv3GOne7h5nu19pAJOdEQYFw9MYXH2AX757WHHBQFjDH9fvouMlBgmDdPuwapj+TNA5AN93Z73cfY1McYUYEsQiEgscL0xplhE8oEJJ6T95MQLGGNmAjPBTrXRhnlvlRc/3c3Ti3K4ZEh3/nrTaDtnTtMMowvtOrs1ZXZU7lnuM4wm2hPUVML+zccvWN445TPYxVQaSxipY6DXOba7qjGw4nfw8VN23MKU1+yANxWwLhvagyU5B8kpLGNY72ON0Kt2F7Epr4Rnrhuhi0SpDufPALEaGCgiGdjAMBW40f0AEUkBjhhjGoCfAq84L30I/D8RafzKfLnzekAyxvCHpTt4fskOrhrRi99f0Z2IDf+wc/3sWWHn24/pDiOutwuUN7dGQUS0XUnLfTWtqmI7HXZj0Mj9DDb/n31NQqH7MDvgbc9yGPE9uOYvbbO4u/KrS4Z2RwQWZx84LkD8ffluUmIjuXbUGVOjqjoxvwUIY0ydiNyLvdmHAq8YY7aKyK+ANcaYBdhSwjMiYrBVTPc4aY+IyFPYIAPwK2OM5xa9APA/729j8YoV/DVtB1dUrEX+uNa+0HUAXHC3LSmkZrVulbMuiTBgon00Ktt/rISRv9ZOvDf+EZj4M5298wzRPS6KzL6JLMk5wP2X2a7B2QWlfPrVIR755mBb+lSqg/m1DcIYswhYdMK+X7htzwPmNZP2FY6VKALWkbyvuPrzKTwWuRcOYqt+Lv2FDQopg/xzw47raaeoHnJl259btZvLhvbguQ+3U1hSRa+ELsz8dBcxEaHcfL6OQFeBoaMbqc9sR3YTO/saeksJOaN+wdCJN7a+95EKOpcPswFiSc5BJgzqxrubCrntovRTTsGhVHvRANFaRbvg1W9DbQU31fyM/8m6GeK1YVh576zusfRLjmZJ9gF2HSwnROC2cTopnwoc7bi2ZSdyeCfMugpqq3hv1Ey2mnR662RqykciwqShPVi1q4i5q/dxTWYqvRL0c6QChwYIXx36ygaH+lqYvpCt9WlEhYeQFK3VAsp3lw3rQU19A1W19fxwvE7KpwKLVjH54tB2mPUtwMD0hdB9KAUla+md0EVHvKpWyeqXREpsBJl9kxjYox2ngFfKCxogvHUwx7Y5SAjcuhC6DQagoNhFr8R2nGdJdSphoSHMv+ci4rVhWgUgrWLyxoGttuQgoTD9vabgAFBYUkVvrTdWp6FPUrTn+bqU6mAaIE5l/2ZbcggNt8HBbb2DmroGDpZV00sbqJVSnZAGiJYUbrLBISzKCQ5nHffygVIXxkDvBK1iUkp1PhogmlOwwQaH8BjbIJ188sIthSUuAO3iqpTqlDRAeJK/Dv55NUTGww/eg66eux8WFFcB0FsbqZVSnZAGiBPlrYV/fgeiEm3JISm92UMLSmyA0MFNSqnOSLu5utu3Gl67zk6ffetCSOzb4uEFxVUkdAk/flEgpZTqJLQE0ejrL+Bf10JMCkxfdMrgAFBY7KKXNlArpTopDRAAe1fZkkNcD9tbKcG7xVoKSlykagO1UqqT0gBxeAe8dr1dA3r6ez6t41xQXKWjqJVSnZZWnncdAOMegNG32hKElypr6iipqtUGaqVUp6UBIiQEvvFfPicrKLZjILSKSSnVWWkVUys1joHQRmqlVGelAaKVCksaB8lpCUIp1TmdMkCIyLdFRAPJCfKLXYhATy1BKKU6KW9u/FOAHSLyGxEZ4u8MnSkKi6voFhtJeKjGTqVU53TKu5sx5mZgFLALmCUiq0RkhogE9fJXhSUurV5SSnVqXn39NcaUAvOAOUAv4FpgnYjc58e8BbSC4iqdpE8p1al50wZxtYi8DXwChANjjTFXACOBn/g3e4HJGENBSZWOgVBKdWrejIO4Hvi9MeZT953GmEoRud0/2QpsxZW1uGobtIpJKdWpeRMgngAKG5+ISBeghzEm1xiz1F8ZC2T5jetAaA8mpVQn5k0bxP8BDW7P6519pyQik0Vku4jsFJHHPLyeJiLLRGS9iGwSkSud/ekiUiUiG9dkqfgAAB9jSURBVJzH3725XnvRleSUUsHAmxJEmDGmpvGJMaZGRCJOlUhEQoG/AJOAPGC1iCwwxmS7HfY48IYx5m8iMgxYBKQ7r+0yxmR6+T7aVdMoam2kVkp1Yt6UIA6JyNWNT0TkGuCwF+nGAjuNMbudADMHuOaEYwwQ72wnAAVenLfDFZRUER4qpMREdnRWlFLKb7wpQdwFvC4ifwYE2Ad834t0qc6xjfKA80445gngI6e7bAxwmdtrGSKyHigFHjfGrPDimu3CLhTUhZAQ6eisKKWU35wyQBhjdgHni0is87y8Da8/DZhljPmdiFwA/EtEzsY2iqcZY4pEZAwwX0SGO+MxmojIDGAGQFpaWhtmq2UFxVU6SZ9SqtPzarpvEbkKGA5EidhvzcaYX50iWT7gvm5nH2efu9uByc75VolIFJBijDkIVDv714rILmAQsMY9sTFmJjATICsry3jzXtpCYYmL8zK6ttfllFKqQ3gzUO7v2PmY7sNWMX0X6OfFuVcDA0Ukw2nUngosOOGYr4FLnesMBaKwbR7dnEZuRKQ/MBDY7dU78rP6BsP+Upc2UCulOj1vGqkvNMZ8HzhqjHkSuAD7bb5Fxpg64F7gQyAH21tpq4j8yq3R+yfAnSKyEZgNTDfGGGA8sElENmCn+LjLGHPE1zfnDwfLXNQ3GB1FrZTq9LypYnI5PytFpDdQhJ2P6ZSMMYuwXVfd9/3CbTsbuMhDujeBN725RnvTleSUUsHCmwDxrogkAs8B67BdU1/0a64CmI6BUEoFixYDhLNQ0FJjTDHwpogsBKKMMSXtkrsA1LiSnFYxKaU6uxbbIIwxDdjR0I3Pq4M5OICtYoqNDCM+yqsOYEopdcbyppF6qYhcL439W4Nc4xgI/XUopTo7bwLED7GT81WLSKmIlIlI6akSdVa6kpxSKlh4M5I6qJcWPVFBcRVnp8af+kCllDrDnTJAiMh4T/tPXEAoGLhq6ymqqNEGaqVUUPCmpfURt+0o7Cyta4FL/JKjALZf14FQSgURb6qYvu3+XET6As/7LUcBrEBXklNKBRFvGqlPlAcMbeuMnAkKnBJELy1BKKWCgDdtEH/Cjp4GG1AysSOqg05h4yhqLUEopYKAN20Q7lNs1wGzjTGf+Sk/Aa2gpIrkmAiiwkM7OitKKeV33gSIeYDLGFMPdq1pEYk2xlT6N2uBp6BYx0AopYKHVyOpAfe7YhdgiX+yE9h0JTmlVDDxJkBEuS8z6mxH+y9LgUtHUSulgok3AaJCREY3PnHWiK7yX5YCU6mrlvLqOnrrNN9KqSDhTRvEA8D/iUgBdsnRntglSINK0zoQOopaKRUkvBkot1pEhgCDnV3bjTG1/s1W4CksbhxFrSUIpVRwOGUVk4jcA8QYY7YYY7YAsSJyt/+zFlgKnIWCtA1CKRUsvGmDuNNZUQ4AY8xR4E7/ZSkwFRRXERoidI/TEoRSKjh4EyBC3RcLEpFQIMJ/WQpMhcUuesZHERqiCwUppYKDN43UHwBzReQF5/kPgff9l6XAlK9jIJRSQcabAPEoMAO4y3m+CduTKagUlrgY2Texo7OhlFLt5pRVTMaYBuALIBe7FsQlQI5/sxVYGhoM+0tc2oNJKRVUmi1BiMggYJrzOAzMBTDGTGyfrAWOwxXV1NQ30FvHQCilgkhLVUzbgBXAt4wxOwFE5MF2yVWAaRwDoW0QSqlg0lIV03VAIbBMRF4UkUuxI6mDTqGOgVBKBaFmA4QxZr4xZiowBFiGnXKju4j8TUQub68MBoL8Yl2LWikVfLxppK4wxvzbWZu6D7Ae27PplERksohsF5GdIvKYh9fTRGSZiKwXkU0icqXbaz910m0XkW/68J7aXGFxFVHhISRFh3dkNpRSql35tCa1MeaoMWamMebSUx3rDKj7C3AFMAyYJiLDTjjsceANY8woYCrwVyftMOf5cGAy8FfnfB2ioKSK3gldcBsvqJRSnZ5PAcJHY4GdxpjdxpgaYA5wzQnHGCDe2U4ACpzta4A5xphqY8weYKdzvg5RUOyil3ZxVUoFGX8GiFRgn9vzPGefuyeAm0UkD1gE3OdDWkRkhoisEZE1hw4daqt8n6TQKUEopVQw8WeA8MY0YJYxpg9wJfAvEfE6T051V5YxJqtbt25+yWBNXQMHy6rppQ3USqkg481UG62VD/R1e97H2efudmwbA8aYVSISBaR4mbZdHCh1YQz01jEQSqkg488SxGpgoIhkiEgEttF5wQnHfA1cCiAiQ4Eo4JBz3FQRiRSRDGAg8KUf89qsxpXktIurUirY+K0EYYypE5F7gQ+BUOAVY8xWEfkVsMYYswD4CfCiM0LbANONMQbYKiJvANlAHXCPMabeX3ltSWGJriSnlApO/qxiwhizCNv47L7vF27b2cBFzaR9Gnjan/nzRuNKcroWtVIq2HR0I3XAKyiuIqFLODGRfo2lSikVcDRAnEJhsUsn6VNKBSUNEKdQUOIiVRuolVJBSAPEKRQUV+koaqVUUNIA0YKK6jpKqmq1i6tSKihpgGhB0zoQ2oNJKRWENEC0oEBXklNKBTENEC3QleSUUsFMA0QL8otdiEBPLUEopYKQBogWFBZX0S02kvBQ/TUppYKP3vlaUFji0uolpVTQ0gDRgoLiKp2kTykVtDRANMMY07QWtVJKBSMNEM04WlmLq7ZBV5JTSgUtDRDNaFooSHswKaWClAaIZhxbKEhLEEqp4KQBohmNJQidqE8pFaw0QDSjoKSK8FAhJSayo7OilFIdQgNEM+xCQV0ICZGOzopSSnUIDRDNKCiu0kn6lFJBTQNEMwp1JTmlVJDTAOFBfYNhf6lLG6iVUkFNA4QHB8tc1DcYeukoaqVUEAvr6AwEosaFgrSKSQWz2tpa8vLycLlcHZ0V1QaioqLo06cP4eHhXqfRAOGBjoFQCvLy8oiLiyM9PR0R7c13JjPGUFRURF5eHhkZGV6n0yomDxpXktMqJhXMXC4XycnJGhw6AREhOTnZ59KgBggPCopdxEaGER+lBSwV3DQ4dB6t+Vv6NUCIyGQR2S4iO0XkMQ+v/15ENjiPr0Sk2O21erfXFvgznydqHAOh/xxKqWDmtwAhIqHAX4ArgGHANBEZ5n6MMeZBY0ymMSYT+BPwltvLVY2vGWOu9lc+PdGV5JQKHPPnz0dE2LZtW0dnJej4swQxFthpjNltjKkB5gDXtHD8NGC2H/PjNV1JTqnAMXv2bMaNG8fs2QFxewgq/qxkTwX2uT3PA87zdKCI9AMygI/ddkeJyBqgDnjWGDPfQ7oZwAyAtLS0Nsm0q7aeoooabaBWys2T724lu6C0Tc85rHc8v/z28BaPKS8v5z//+Q/Lli3j29/+Nk8++ST19fU8+uijfPDBB4SEhHDnnXdy3333sXr1au6//34qKiqIjIxk6dKlxMXFtWmeg02gtMJOBeYZY+rd9vUzxuSLSH/gYxHZbIzZ5Z7IGDMTmAmQlZVl2iIj+3UdCKUCxjvvvMPkyZMZNGgQycnJrF27li+//JLc3Fw2bNhAWFgYR44coaamhilTpjB37lzOPfdcSktL6dJF/4dPlz8DRD7Q1+15H2efJ1OBe9x3GGPynZ+7ReQTYBSw6+SkbUtXklPqZKf6pu8vs2fP5v777wdg6tSpzJ49mz179nDXXXcRFmZvX127dmXz5s306tWLc889F4D4+PgOyW9n488AsRoYKCIZ2MAwFbjxxINEZAiQBKxy25cEVBpjqkUkBbgI+I0f89qkwClB6FrUSnWsI0eO8PHHH7N582ZEhPr6ekSkKQgo//NbI7Uxpg64F/gQyAHeMMZsFZFfiYh7r6SpwBxjjHsV0VBgjYhsBJZh2yCy/ZVXd02jqLUEoVSHmjdvHrfccgt79+4lNzeXffv2kZGRwciRI3nhhReoq6sDbCAZPHgwhYWFrF69GoCysrKm11Xr+bUNwhizCFh0wr5fnPD8CQ/pVgIj/Jm35hSWVJEcE0FUeGhHXF4p5Zg9ezaPPvrocfuuv/56cnJySEtL45xzziE8PJw777yTe++9l7lz53LfffdRVVVFly5dWLJkCbGxsR2U+84hUBqpA0ZBsY6BUCoQLFu27KR9P/7xj5u2//d///e4184991w+//xzv+crmOhUGyfQleSUUsrSAHECHUWtlFKWBgg3pa5ayqvrdBS1UkqhAeI4x3owaQlCKaU0QLgpLNZR1Eop1UgDhJv8xlHUWsWklFIaINwVllQRGiJ0j9MAoVRHmjhxIh9++OFx+55//nl+9KMfNZtmwoQJrFmzxqvzZ2ZmMnXq1NPKYzDQAOGmsNhFz/goQkN0oSClOtK0adOYM2fOcfvmzJnDtGnTTvvcOTk51NfXs2LFCioqKk77fJ2ZDpRzk69jIJTy7P3HYP/mtj1nzxFwxbMeX7rhhht4/PHHqampISIigtzcXAoKCrj44ov50Y9+xOrVq6mqquKGG27gySef9Omys2fP5pZbbiEnJ4d33nmHG2+0U8R5mi48Ojra49TiwUIDhJvCEhcj+yZ2dDaUCnpdu3Zl7NixvP/++1xzzTXMmTOH733ve4gITz/9NF27dqW+vp5LL72UTZs2cc4553h97rlz57J48WK2bdvGn/70J2688cZmpwufOXPmSVOLBxMNEI6GBsP+EhdXjNAShFInaeabvj81VjM1BoiXX34ZgDfeeIOZM2dSV1dHYWEh2dnZXgeINWvWkJKSQlpaGqmpqdx2220cOXKE/Px8j9OFL1my5KSpxYOJtkE4DldUU1PfQG8dA6FUQLjmmmtYunQp69ato7KykjFjxrBnzx5++9vfsnTpUjZt2sRVV12Fy+Xy+pyzZ89m27ZtpKenM2DAAEpLS3nzzTf9+C7ObBogHDoGQqnAEhsby8SJE7ntttuaGqdLS0uJiYkhISGBAwcO8P7773t9voaGBt544w02b95Mbm4uubm5vPPOO8yePbvZ6cInTZp00tTiwUQDhEPXgVAq8EybNo2NGzc2BYiRI0cyatQohgwZwo033shFF13kMd0dd9xxUpfXFStWkJqaSu/evZv2jR8/nuzsbIqKipqmCx85ciSTJk3C5XJxxx13NE0tPnLkSP7973/7780GIDl+nZ4zV1ZWlvG2D7QnL/9nD08tzGbdzyfRNSaiDXOmOrsJsyYA8Mn0Tzo0H20tJyeHoUOHdnQ2VBvy9DcVkbXGmCxPx2sJwlFYXEVUeAhJ0eEdnRWllAoIGiAcBSVV9E7ogogOklNKKdAA0aSg2EUvnYNJKaWaaIBwFDolCKWUUpYGCKCmroGDZdX00i6uSinVRAMEcKDUhTGQqlVMSinVRAMEupKcUoHo6aefZvjw4ZxzzjlkZmbyxRdfAHba78rKSp/PN2vWLAoKCjy+tm3bNjIzMxk1ahS7du3iwgsvPK28N5o/fz7Z2dmndY4rr7yS4uLiNsmPrzRAYCfpA10oSKlAsWrVKhYuXMi6devYtGkTS5YsoW/fvkDrAkR9fX2LAWL+/PnccMMNrF+/ngEDBrBy5crTfg+N5z3dALFo0SISEztmElGdrA/bxRW0BKFUcx744AE27N/QpufM7JnJ85Of9/haYWEhKSkpREZGApCSkgLAH//4RwoKCpg4cSIpKSksW7as2em/09PTmTJlCosXL+ahhx5izZo13HTTTXTp0oVVq1bRpYv9f1+0aBHPP/88oaGhLF26lGXLlhEbG0t5eTmffPIJTzzxBCkpKWzZsoUxY8bw2muvISKsXbuWhx56iPLyclJSUpg1axa9evVqeg8rV65kwYIFLF++nF//+te8+eab3H777fz2t78lKyuLw4cPk5WVRW5uLrNmzWLBggVUVlaya9curr32Wn7zm980vY81a9ZQXl7OFVdcwbhx41i5ciWpqam88847dOnShdWrV3P77bcTEhLCpEmTeP/999myZctp/420BIGtYkroEk5MpMZLpQLB5Zdfzr59+xg0aBB33303y5cvB+DHP/4xvXv3ZtmyZSxbtgywVVFr1qxh06ZNLF++nE2bNjWdJzk5mXXr1nHzzTeTlZXF66+/zoYNG5qCA9gqnLvuuosHH3yw6Zzu1q9fz/PPP092dja7d+/ms88+o7a2lvvuu4958+axdu1abrvtNn72s58dl+7CCy/k6quv5rnnnmPDhg0MGDCgxfe8YcMG5s6dy+bNm5k7dy779u076ZgdO3Zwzz33sHXrVhITE5smGvzBD37ACy+8wIYNGwgNDfXyt3xqekfETtSnczAp1bzmvun7S2xsLGvXrmXFihUsW7aMKVOm8OyzzzJ9+vSTjm1p+u8pU6acdl7Gjh1Lnz59ALtUaW5uLomJiWzZsoVJkyYBtgrLvfTQGpdeeikJCQkADBs2jL179zZVqzXKyMggMzMTgDFjxpCbm0txcTFlZWVccMEFANx4440sXLjwtPLSyK8BQkQmA38AQoGXjDHPnvD674GJztNooLsxJtF57Vbgcee1XxtjXvVXPgtKXKRqF1elAkpoaCgTJkxgwoQJjBgxgldfffWkANE4/ffq1atJSkpi+vTpx03/HRMTc9r5aKzmasxTXV0dxhiGDx/OqlWrfDpXWFgYDQ0NACdNU+7pOqfKS1VVlU/X95XfqphEJBT4C3AFMAyYJiLD3I8xxjxojMk0xmQCfwLectJ2BX4JnAeMBX4pIkn+ymtBcZWOolYqgGzfvp0dO3Y0Pd+wYQP9+vUDIC4ujrKyMsC36b/d052uwYMHc+jQoaYAUVtby9atW095zfT0dNauXQvAvHnz2iQviYmJxMXFNfXyOnEt79PhzzaIscBOY8xuY0wNMAe4poXjpwGzne1vAouNMUeMMUeBxcBkf2SyorqOkqpaXQdCqQBSXl7OrbfeyrBhwzjnnHPIzs7miSeeAGDGjBlMnjyZiRMnej39N8D06dO56667yMzMPO1v3hEREcybN49HH32UkSNHkpmZ6bHn09SpU3nuueeaus8+/PDD/O1vf2PUqFEcPnz4tPLg7uWXX+bOO+8kMzOTioqKpqqq0+W36b5F5AZgsjHmDuf5LcB5xph7PRzbD/gc6GOMqReRh4EoY8yvndd/DlQZY357QroZwAyAtLS0MXv37vU5n0cqanhiwVa+m9WHiwd28zm9Ujrdt+po5eXlxMbGAvDss89SWFjIH/7wh5OO83W670BppJ4KzDPG1PuSyBgzE5gJdj2I1ly4a0wEf5w2qjVJlVIqILz33ns888wz1NXV0a9fP2bNmtUm5/VngMgH3Jvg+zj7PJkK3HNC2gknpP2kDfOmlFKdxpQpU9qkx9aJ/NkGsRoYKCIZIhKBDQILTjxIRIYASYB7d4APgctFJMlpnL7c2aeUakedZcVJ1bq/pd8ChDGmDrgXe2PPAd4wxmwVkV+JyNVuh04F5hi33BtjjgBPYYPMauBXzj6lVDuJioqiqKhIg0QnYIyhqKiIqCjfemv6tQ3CGLMIWHTCvl+c8PyJZtK+Arzit8wppVrUp08f8vLyOHToUEdnRbWBqKiopgF/3gqURmqlVIAJDw8nIyOjo7OhOpDOxaSUUsojDRBKKaU80gChlFLKI7+NpG5vInII8H0o9TEpwOmMfdf0ml7Ta/ozMX0/Y4znaSSMMfqwQXKNptf0ml7TB2P65h5axaSUUsojDRBKKaU80gBxzExNr+k1vaYP0vQedZpGaqWUUm1LSxBKKaU80gChlFLKo6APECIyWUS2i8hOEXmsFelfEZGDIrKlFWn7isgyEckWka0icr+P6aNE5EsR2eikf9LXPDjnCRWR9SKysBVpc0Vks4hsEJE1rUifKCLzRGSbiOSIyAU+pB3sXLfxUSoiD/h4/Qed390WEZktIj5Ndyki9ztpt3p7bU+fGRHpKiKLRWSH87PZNdibSf9dJw8NIuJxdbBTpH/O+RtsEpG3RSTRx/RPOWk3iMhHItLbl/Rur/1ERIyIpPh4/SdEJN/ts3Clr9cXkfuc38FWEfmNj9ef63btXBHZ4GP6TBH5vPH/SETG+ph+pIiscv4X3xWR+ObS+8QffWfPlAcQCuwC+gMRwEZgmI/nGA+MBra04vq9gNHOdhzwlS/XBwSIdbbDgS+A81uRj4eAfwMLW5E2F0g5jb/Bq8AdznYEkHgaf8v92EE/3qZJBfYAXZznbwDTfUh/NrAFiMZOfLkEOKs1nxngN8BjzvZjwP/4mH4oMBi7sFZWK65/ORDmbP9PK64f77b9Y+DvvqR39vfFLg+wt6XPVDPXfwJ42Mu/m6f0E52/X6TzvLuv+Xd7/XfAL3y8/kfAFc72lcAnPqZfDXzD2b4NeMrbz3FLj2AvQYwFdhpjdhtjaoA5wDW+nMAY8ynQqrUqjDGFxph1znYZdt2MVB/SG2NMufM03Hn41OtARPoAVwEv+ZKuLYhIAvbD/jKAMabGGFPcytNdCuwyxvg6mj4M6CIiYdgbfYEPaYcCXxhjKo1d/2Q5cN2pEjXzmbkGGyxxfn7Hl/TGmBxjzHZvMt1M+o+c9wDO+vA+pi91expDC5/DFv5nfg/8V0tpT5HeK82k/xHwrDGm2jnmYGuuLyICfA+Y7WN6AzR+60+ghc9hM+kHAZ8624uB65tL74tgDxCpwD6353n4cINuSyKSDozClgJ8SRfqFGcPAouNMT6lB57H/lM2+JiukQE+EpG1IjLDx7QZwCHgH04V10siEtPKfEylhX9KT4wx+cBvga+BQqDEGPORD6fYAlwsIskiEo395tf3FGma08MYU+hs7wd6tPI8beE24H1fE4nI0yKyD7gJ+MWpjj8h7TVAvjFmo6/XdXOvU831SktVdM0YhP1bfiEiy0Xk3Fbm4WLggDFmh4/pHgCec35/vwV+6mP6rRz7cvtdWv85PE6wB4iAICKxwJvAAyd8EzslY0y9MSYT+41vrIic7cN1vwUcNMas9SnDxxtnjBkNXAHcIyLjfUgbhi0q/80YMwqowFav+ETskrZXA//nY7ok7D9VBtAbiBGRm71Nb4zJwVbHfAR8AGwA6n3JQzPnNfhYEmwrIvIzoA543de0xpifGWP6Omnv9eGa0cB/42NQOcHfgAFAJjbY/87H9GFAV+B84BHgDac04Ktp+PhFxfEj4EHn9/cgTqnaB7cBd4vIWmx1dU0r8nCSYA8Q+Rwfafs4+9qNiIRjg8Prxpi3Wnsep2pmGTDZh2QXAVeLSC62eu0SEXnNx+vmOz8PAm9jq+28lQfkuZV65mEDhq+uANYZYw74mO4yYI8x5pAxphZ4C7jQlxMYY142xowxxowHjmLbkVrjgIj0AnB+NlvF4S8iMh34FnCTE6Ra63V8q+IYgA3SG53PYh9gnYj09PYExpgDzpelBuBFfPscgv0svuVU236JLVE321DuiVNNeR0w18drA9yK/fyB/aLjU/6NMduMMZcbY8ZgA9SuVuThJMEeIFYDA0Ukw/kWOhVY0F4Xd76hvAzkGGP+txXpuzX2NhGRLsAkYJu36Y0xPzXG9DHGpGPf+8fGGK+/QYtIjIjENW5jGzq97s1ljNkP7BORwc6uS4Fsb9O7ae23tq+B80Uk2vlbXIptB/KaiHR3fqZhbw7/bkU+wH7ubnW2bwXeaeV5WkVEJmOrGq82xlS2Iv1At6fX4NvncLMxprsxJt35LOZhO2/s9+H6vdyeXosPn0PHfGxDNSIyCNthwtfZUS8Dthlj8nxMB7bN4RvO9iWAT1VUbp/DEOBx4O+tyMPJ2qKl+0x+YOuNv8JG3J+1Iv1sbJG2FvvBvt2HtOOwVQmbsNUTG4ArfUh/DrDeSb+FFnpOeHGuCfjYiwnb+2uj89jayt9fJrDGeQ/zgSQf08cARUBCK9/3k9ib2RbgXzi9WHxIvwIb1DYCl7b2MwMkA0uxN4YlQFcf01/rbFcDB4APfUy/E9se1/g5bKkXkqf0bzq/w03Au0Bqa/9nOEXPuGau/y9gs3P9BUAvH9NHAK8572EdcImv+QdmAXe18u8/DljrfI6+AMb4mP5+7H3sK+BZnFkyTvehU20opZTyKNirmJRSSjVDA4RSSimPNEAopZTySAOEUkopjzRAKKWU8kgDhFKnICL1cvyssT6P9m7h3OmeZjVVKhCEdXQGlDoDVBk7nYlSQUVLEEq1kjPv/2+cOfi/FJGznP3pIvKxM3HcUmeUNSLSQ+xaCxudR+O0HqEi8qKzDsFHzqh4ROTHYtcK2SQiczrobaogpgFCqVPrckIV0xS310qMMSOAP2NnxgX4E/CqMeYc7LxEf3T2/xFYbowZiZ1zaquzfyDwF2PMcKCYY/MYPQaMcs5zl7/enFLN0ZHUSp2CiJQbY2I97M/FTsmw25l0cb8xJllEDmOneqh19hcaY1JE5BDQxzhrDjjnSMdO0z7Qef4oEG6M+bWIfACUY6cgmW+Orf2hVLvQEoRSp8c0s+2Larfteo61DV4F/AVb2ljtzBaqVLvRAKHU6Zni9nOVs70SOzsu2MVzVjjbS7Hz/jcu9JTQ3EmdWTn7GmOWAY9iVxk7qRSjlD/pNxKlTq2LHL8I/QfGmMaurkkisglbCpjm7LsPu0reI9gV837g7L8fmCkit2NLCj/CzsrpSSjwmhNEBPijaf1yrEq1irZBKNVKThtEljHG13UDlDojaBWTUkopj7QEoZRSyiMtQSillPJIA4RSSimPNEAopZTySAOEUkopjzRAKKWU8uj/A49qCH/mxvXdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(tf_history_0.history[\"accuracy\"]+tf_history_1.history[\"accuracy\"])\n",
    "plt.plot(tf_history_0.history[\"val_accuracy\"]+tf_history_1.history[\"val_accuracy\"])\n",
    "plt.xticks(range(len(tf_history_0.history[\"accuracy\"]+tf_history_1.history[\"accuracy\"])))\n",
    "plt.axvline(9, color=\"green\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.legend([\"Acc\", \"Val. Acc\", \"Start fine tuning\"])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z2Jhnwh5Y4IB"
   },
   "source": [
    "Overall the resulting performance of the model is quite close to published results on the Caltach 101, e.g. this [paper](https://arxiv.org/abs/1406.4729v1), while the definintion and training of the model only required very little code.\n",
    "\n",
    "In order to evaluate if transfer learning actually makes a difference, we can simply train the MobileNetV2 on the Caltech 101 data set. As we already discussed, training such a high capacity model on such a small data set leads to heavy overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 678
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5624284,
     "status": "ok",
     "timestamp": 1562676267477,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "Ugaum5NNdxb1",
    "outputId": "9ab79ad0-9636-4ab4-fdd8-d1149660e270"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"my_transfer_model_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "mobilenetv2_1.00_224 (Model) (None, 7, 7, 1280)        2257984   \n",
      "_________________________________________________________________\n",
      "global_average_pooling2d_1 ( multiple                  0         \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          multiple                  0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              multiple                  130662    \n",
      "=================================================================\n",
      "Total params: 2,388,646\n",
      "Trainable params: 2,354,534\n",
      "Non-trainable params: 34,112\n",
      "_________________________________________________________________\n",
      "Epoch 1/10\n",
      "260/260 [==============================] - 64s 245ms/step - loss: 3.7655 - accuracy: 0.2298 - val_loss: 4.4664 - val_accuracy: 0.0508\n",
      "Epoch 2/10\n",
      "260/260 [==============================] - 62s 240ms/step - loss: 3.0558 - accuracy: 0.3278 - val_loss: 5.0592 - val_accuracy: 0.0462\n",
      "Epoch 3/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 2.7501 - accuracy: 0.3697 - val_loss: 5.4238 - val_accuracy: 0.0520\n",
      "Epoch 4/10\n",
      "260/260 [==============================] - 63s 242ms/step - loss: 2.4769 - accuracy: 0.4159 - val_loss: 5.9812 - val_accuracy: 0.0554\n",
      "Epoch 5/10\n",
      " 28/260 [==>...........................] - ETA: 50s - loss: 2.3722 - accuracy: 0.4453"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-16-ad4fd510dca2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mtf_mdl\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msummary\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m \u001b[0mtf_history_2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_mdl\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrain_gen\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidation_data\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mval_gen\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1.0\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mval_split\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mN_samples_Caltech101\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mtf_batch_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidation_steps\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mval_split\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mN_samples_Caltech101\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mtf_batch_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf_epochs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_method_wrapper\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     64\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_method_wrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     65\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_in_multi_worker_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     68\u001b[0m     \u001b[0;31m# Running inside `run_distribute_coordinator` already.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)\u001b[0m\n\u001b[1;32m    849\u001b[0m                 batch_size=batch_size):\n\u001b[1;32m    850\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 851\u001b[0;31m               \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    852\u001b[0m               \u001b[0;31m# Catch OutOfRangeError for Datasets of unknown size.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    853\u001b[0m               \u001b[0;31m# This blocks until the batch has finished executing.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    578\u001b[0m         \u001b[0mxla_context\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mExit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    579\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 580\u001b[0;31m       \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    581\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    582\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mtracing_count\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    609\u001b[0m       \u001b[0;31m# In this case we have created variables on the first call, so we run the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    610\u001b[0m       \u001b[0;31m# defunned version which is guaranteed to never create variables.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 611\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=not-callable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    612\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    613\u001b[0m       \u001b[0;31m# Release the lock early so that multiple threads can perform the call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   2418\u001b[0m     \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2419\u001b[0m       \u001b[0mgraph_function\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maybe_define_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2420\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_filtered_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2421\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2422\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_filtered_call\u001b[0;34m(self, args, kwargs)\u001b[0m\n\u001b[1;32m   1659\u001b[0m       \u001b[0;31m`\u001b[0m\u001b[0margs\u001b[0m\u001b[0;31m`\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1660\u001b[0m     \"\"\"\n\u001b[0;32m-> 1661\u001b[0;31m     return self._call_flat(\n\u001b[0m\u001b[1;32m   1662\u001b[0m         (t for t in nest.flatten((args, kwargs), expand_composites=True)\n\u001b[1;32m   1663\u001b[0m          if isinstance(t, (ops.Tensor,\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[0;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[1;32m   1743\u001b[0m         and executing_eagerly):\n\u001b[1;32m   1744\u001b[0m       \u001b[0;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1745\u001b[0;31m       return self._build_call_outputs(self._inference_function.call(\n\u001b[0m\u001b[1;32m   1746\u001b[0m           ctx, args, cancellation_manager=cancellation_manager))\n\u001b[1;32m   1747\u001b[0m     forward_backward = self._select_forward_and_backward_functions(\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[1;32m    591\u001b[0m       \u001b[0;32mwith\u001b[0m \u001b[0m_InterpolateFunctionError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    592\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcancellation_manager\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 593\u001b[0;31m           outputs = execute.execute(\n\u001b[0m\u001b[1;32m    594\u001b[0m               \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msignature\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    595\u001b[0m               \u001b[0mnum_outputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_num_outputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[1;32m     57\u001b[0m   \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 59\u001b[0;31m     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0m\u001b[1;32m     60\u001b[0m                                         inputs, attrs, num_outputs)\n\u001b[1;32m     61\u001b[0m   \u001b[0;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "tf_batch_size = 32\n",
    "tf_epochs = 5\n",
    "tf_learning_rate = 0.001\n",
    "base_model = k.applications.MobileNetV2(input_shape=(224, 224, 3), include_top=False, weights=None)\n",
    "tf_mdl = MyTransferModel(base_model)\n",
    "tf_opt = k.optimizers.RMSprop(tf_learning_rate)\n",
    "tf_mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=tf_opt, metrics=[\"accuracy\"])\n",
    "tf_mdl.build((tf_batch_size, 224, 224, 3))\n",
    "tf_mdl.summary()\n",
    "\n",
    "tf_history_2 = tf_mdl.fit(train_gen, validation_data=val_gen, steps_per_epoch=int((1.0-val_split)*N_samples_Caltech101/tf_batch_size), validation_steps=int(val_split*N_samples_Caltech101/tf_batch_size), epochs=tf_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "l7gvjWFvtlxr"
   },
   "source": [
    "As you can see, training MobileNetV2 from scratch does not only suffer from severe overfitting but is also rather slow when compared with fine tuning. So overall it is recommended to use pretrained models and transfer learning if you want to train a high capacity neural network on a small data set, as it is often the case in practical applications. By doing this you can benefit from not only the architecture search of many experts but also save time and computational resources since a part of your model has already be pretrained by someone else. But you should always keep in mind that transfer learning is only a suitable method if there is a realistic chance of a positive transfer, i.e. the data set and task the model was pretrained on has to have something in common with your data set and task. Otherwise transfer learning can not only slow down the convergence speed but also hurt the final performance of your model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3zzJJ51h2GP8"
   },
   "source": [
    "## Catastrophic Forgetting\n",
    "Despite many advances in better architectures and training algorithms for training, neural networks still suffer from a long known phenomenon called \"catastrophic forgetting\". In order to understand what this phenomenon is we can compare human learning with the way neural networks learn. Humans can quickly learn from few examples and most importantly, they can learn to solve tasks in a sequential way. This means one can learn a language, e.g. english, and after a certain period of time learn a second language, e.g. chinese, without having to repeatedly refresh everthing that was learned on the first language. Neural networks are currently not capable of learning on a sequence of different tasks. In order to demonstrate this, we will train a neural network first on MNIST and then on the FashionMNIST data set. We want to be able to uniquely identify any class from both data sets. Since both data sets contain $10$ classes and use the labels $\\lbrace0,\\ldots,9\\rbrace$ for them, we need to choose different labels for the FashionMNIST data set. We do this by loading the data set and shifting the labels by $10$ so that we get labels $\\lbrace10,\\ldots,19\\rbrace$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 164
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5625876,
     "status": "ok",
     "timestamp": 1562676269083,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "x0e8W8Wc3CS_",
    "outputId": "7935fece-bd6c-40a7-bcac-820941d85827"
   },
   "outputs": [],
   "source": [
    "(x_train_fmnist, y_train_fmnist), (x_test_fmnist, y_test_fmnist) = tf.keras.datasets.fashion_mnist.load_data()\n",
    "\n",
    "x_train_fmnist = np.expand_dims(x_train_fmnist, axis=-1).astype(np.float32)\n",
    "y_train_fmnist = y_train_fmnist + 10\n",
    "x_test_fmnist = np.expand_dims(x_test_fmnist, axis=-1).astype(np.float32)\n",
    "y_test_fmnist = y_test_fmnist + 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fz6e_B1vFj96"
   },
   "source": [
    "We also plot some examples of this data set and varify if the labels were shifted correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 324
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5626086,
     "status": "ok",
     "timestamp": 1562676269304,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "bZkh_WV-FqD_",
    "outputId": "7a7c1b98-114f-4593-b55a-fce386328d7e"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9eZScVZk//ql936uX6j29b+nsIQnZIRBg2GQRCQZxAEUUiMqAjI5w1FGZQZQZEIYRD4IyiIBsARnWhJiQkBBCJ53uTu9rdVV17fvy/v7oeZ681emlKuKM3/PLc06fpLur73vfe++zf57nSgRBwBk6Q2fob4+k/9cTOENn6AzNTGeY8wydob9ROsOcZ+gM/Y3SGeY8Q2fob5TOMOcZOkN/oySf65cSieR/LZSrVqtRWFiIsrIy3HDDDfB4PDh27BiSySSKi4uxZs0afPjhh9i1axcmJiYwOjqKRCKR09hmsxlXXXUVBgYG0N/fj0wmAwDQarVoaWlBV1cXDh8+jHQ6ndN4FosFFRUVWLRoEc4//3xMTk7inXfeAQBcdNFFaG5uxsGDB/H000+jv78fTqcz53UoLCyE1WpFNBrF8PAwz8lqtaKxsRGBQADHjh3jd5iPJBIJTCYTCgsLUVlZiYsvvhjNzc0YHh5GKpVCRUUFxsfH8fOf/xyTk5NwuVyIRCL4v47iGwwGlJaWYuXKlRgbG4PX6wUAVFVVQavVYteuXRgaGsp5z/6WSRAEyUw/l8y1Cf8bzCmRSNDS0oKtW7di06ZNWLBgAQBAoVDAaDRCKpUinU7D5XJBpVJBpVKhs7MTr776Kl599VX09vbOO35DQwOuvfZabNq0CVqtFgqFAoIgIJ1O49NPP8Vbb72FF154AeFweN756nQ63HzzzVixYgXPzWg0orq6GvF4HOFwGC6XC8lkEgqFAu3t7XjiiSfQ39+f03rcfvvtuPHGG+FyufDKK6+gu7sbZWVlWL58Oc4991yMj49j48aNiMViOa/ttm3bsHTpUqhUKiSTSWi1WpjNZmQyGYTDYUQiEcjlcng8HuzatQs7d+5EZ2dnTvP9LEkimTqj27Ztw8qVK3HBBReguLgYMpkM0WgUCoUCSqUSyWQSR48exTPPPIO9e/di//79f5X5WK1W+P1+qNXqnM5GLrR06VKMjo6ywBYE4a/PnFKpFOXl5Sx5JRJJTtK3oaEBP/rRj1BTU8MMk0qleEx6Ado4iUQCuVwOiUSC3/zmN3j44Yfn1aDV1dVobW3FwoULce2117Jm2r9/P1566SWEw2G89dZbiEQi8873iiuuwPr16yEIAnQ6HQRBQDwe53nSO6fTaUSjUajVavT19eE3v/kNAoHAvON/9atfxTe/+U2oVCqYzWbcfffd2L59O2praxEIBDA6OoqNGzfmpDH0ej3uv/9+1NXVIRaLIZ1OI5lMIhqNIplMIplMQq1Ww2g0Qi6XQyaTQSaTobOzE9/+9reRTCbnfcZnTV/5yldw33338R5nMhlIpVJeWzpX9O+JEydw55134oMPPsj7WXSmpp9TrVaLxYsXY9GiRSgpKUFPTw9+85vf5GytTCepVAq5XI5LLrkEd911F15//XXce++9AIBMJjMrc85p1ub6YIlEgk2bNmHdunU4cOAAhoeHcfz4cZbuYiYT/yuXy7F+/XrU1dUhkUhAKpVCr9dDq9Xy4tOXXC5HOp1GMBhEPB6HQqHA5s2b8eKLL86rldLpNJRKJfbv34+jR4/C4XAgHA5jfHwcBoMBMpks5/ft6OjAli1bEI1G4fP5IJVKkUwmIZVKoVQqEQ6HeRPT6TTkcjm6u7sRDAZzGn98fBzxeByZTAZGoxGjo6Po6upCU1NT1lrmQiUlJWhqamIzVavVQiaTQRAEpFIpFjAymQypVAqJRAKxWAwNDQ1YvXo1du3alfOzPgtqa2vDLbfcgkwmg1QqBaVSyQwqFtC0BplMBlVVVbj55ptx4sQJjI+P5/U8QRAgk8mg1+uxdu1aSCQSXHvttUin07DZbOjp6cGzzz6LI0eO5OVGSCQSnmMqlYJMJsOXv/xl3H333QiHw3jhhRdyGu8vZk5gyl+8/PLLsXTpUpSWlsLpdGJ4eBi7d+/G+Pg4YrEYSkpKkEgk2NcBAJVKhc2bN0OlUvFLvPXWW3jttdcQjUbZlInH45DL5dBoNLjllltQW1sLqVSKtrY2LFiwYF7mpIXQ6XTIZDLo7e2FVCrlg5mPRDx27BjeffddbNu2DSdOnIBMJmOBQVKeGFWpVKKrqwvvvPNOzj7c8PAwC6pQKIStW7fC4XAgEolAJpMhFArlPNfy8nIUFhYiHA5DrVYjlUohGo0iGAwimUwilUqxlWOz2VBUVMTfL1269H+NOSUSCYxGI2677TaUlZVBLpcjGAxCLp86nmLNOV3AA8Cll16K999/H7/+9a+zPjMf6fV6FBcXw2QyYd26dSgoKMArr7yCvXv3IhgM4pxzzoFCoUBhYWFOVg/NSy6XQ61WQ6fTwWQyoaSkBPfeey/i8Tgee+wxHD16NKex/mLmzGQyqK2txZIlS1BQUIC2tjb09PTAZrPBZrNhZGQEwNRC6PV6PP/88+jq6gIwxZyrV69GNBqFTCaDUqnE6OgompqasGbNGuzevRutra1QKBRQq9X4/e9/j8HBQbS0tACY0kwtLS3YtWvXnGaeTCaDQqFAOp1mSSmWwgqFgr+fjyQSCd544w00NzfDbDbD7/cjGo3OaFqrVCq88cYbeZmHbrcb4XAYWq0W8XgcF110EQRBgN/vh0KhyMnXJBobG8Nzzz2HRYsWob29HZ2dnfB6vfD7/VmCJJPJQKFQ4KKLLsJFF10EqVQKq9Wa83P+UhIEAU1NTdiyZQsSiQQMBgMCgQDvKVln4s8LgsAaNZ1OY8WKFXjqqafmdXHIIpPL5SgtLUVZWRnq6+thNBrxxhtv4LXXXkM6nYZCoUAwGMSyZcvQ1taGxx9/fE63h9Zy+fLlWL16NbRaLSorK9lKczqdEAThlCDWXOfuM9GcO3bsQHl5OWQyGVQqFTvSLS0tOHHiBIaHh1FUVISlS5fi0Ucf5b+rqKhgqahUKqFQKAAAe/fuRW9vLxKJBPbv3w+ZTAatVotIJMLmIQUzli9fDplMNidzyuVyKJVKJBKJLKYkEyQf5iRGeemll7B9+3ZIJBIkEgk2bZPJJAwGAyQSCdxuN0ZHR/NaS5/PB4PBAEEQEI1GEY/HeVwyR3Ol1atXY3x8HB0dHRgfH8dXvvIVAEAgEODDL5PJoNFoYLfb8dFHH+GOO+6A1+vFxMREXvMGwIwujjfI5XJkMhm2TsRaUEwqlQoajYb3kf5GrDEFQTjF/6T12LJlC+/xXCTWvOl0GkuWLEF3dzfefvtt9PX1sSC1WCzQarXo7u6G1WqFzWZDLBab1cqiuQ4NDbFlQufqoYcegtVqxYcffoienh4AJ83fubT8Z8KcWq0WqVSKD5JcLofVaoXJZILD4QAwFRo/duxYllm2bds2pNNpSKVSjohVV1fD5XLBarVCKpUiFoshHo9DqVRCo9Ggra0NqVQKOp0OiUQCVVVVOflhZCrS5hKRBsmVOYloTjKZjMdMJpNIJBJ8eMhqyId8Ph8A8LjiAIhMJoPf78/ZbIvH41i2bBnq6uowMDCAffv2oaCgAHa7HUajkX3tTCaDI0eOwGQy4XOf+xycTifkcjnuvffenNNVYlKr1TCbzaisrMStt96K9957D08++SRSqRQymQysViu8Xi+/h0wmQ319PQeByMUhxiMtKf4/RcrlcjkUCgULxFwpnU6jp6cHDz74IM9BbOGYTCYAwMDAAHp6euBwOOByuea0XGjPR0dHeX4A8P7776OtrQ0GgwEGg4HnX1RUBJvNNut4fzFzymQymM1mlupkNlVWVrK5JJVKEYlEYDKZYLPZ2HF//vnnMTo6igULFmD58uUwGAxYtWoVtm7dmrVxdEAzmQy8Xi9cLhf+8Ic/IBQK4fjx43MeIIlEApVKlcWY06N0Go2GtXYuJJFIEAwGs/yidDoNlUoFQRA4NTE5OQngpEbJlRKJBDQaDX8vPpy5BpYAYMOGDWhoaEAwGIRKpcJVV12FSCTCh4PGFgQBCxYsQDgchslk4nRQPoEyYOpAt7S0YN26dVi8eDEaGhqQSqXQ1NSEnTt3YmxsDEVFRfjyl7+MBx54gPdNIpHAbrezYKdDTRbRdGuH/p/JZPgzOp0ONpstr/Uh7TmTxtfpdGhpaUEikeAYRb7j0lyfffZZfPWrX4XJZEJVVRUGBwdx0UUXYcuWLfj4449nHee0mZPU8sKFC1FVVcUbLpfLodPpIJfLkUwmIZfLIZVKceLECXi9XhQXFzNz7tu3Dx9++CG0Wi3Wrl2LBx98kIMrYu0glogKhQK7d+/GD37wg7zmSodbPBb9PJ1O5510j0ajiEQiMBqNUCqVCAaDLIR0Oh0Lg9Mh8RzFAiWTyUCpVOY8js/nQyqVwujoKFwuFzQaDRKJBGtlelYmk0EymUR/fz+WLl2K6upqPqxzzU1sum7evBnXX389ysrKONAWDAYRCoUQj8eRTCZRWVmJb3zjGxgYGIDD4cDAwAC/Y2FhIc9F7EvS+08XFAqFAolEgtcjnU7j85//PH7605/mvD5E9EyKslK02GKxYN26deju7ubP5UuCIKC3txdutxtGoxE7duzATTfdhJqaGng8Hnz/+9+f9Sz/RcxJkou0EzAlPePxOCKRCFQqFdRqNUKhEIqKivDWW2+ho6PjlHGi0Sh6enoQDoezwtC0+eIcp1QqxcjIyKw5qplors/Q5ueDNCFNFggEoFKp2BehwFYsFkMqlTrlPXIlEnw0P7FUr6ioyNl8q6+vh8Vigd/v5/QACT6xwFKr1RAEAS6XC6FQCFarFSqVakZrgv5OoVBAr9fj3HPPxVVXXcUpG/odCWu9Xg+dToedO3didHQUL7zwAnbt2pVlEYnHTiQSWVpnujAV/5+YNhaLQalUcrrpdImsCLvdDrlcjl/84hdYsmQJGhsb0d3dnXfel94tmUziu9/9Ln72s5/hlVdeweTkJHbs2IFPPvlkzvRPTswpNgnpEBPU7vvf/z4jOORyOSKRCGtQiUTCgYzS0lIYDIZTAhp0QEZGRhAMBjndIXaYxbkucTohF6AD+WrTF0y8+TNJ5vnGlEql7BcVFBTAZDJxCiWdTiOTyaC0tDTrmbmQUqlkU5nWgJ4nCALKy8tzZk4yESmVIk7oTxduFAmORCJYsGABzGYzjEYj/H4/z4t8JKVSicWLF+O6666Dw+GA3+9HMBiEyWTK8g3T6TTS6TQ0Gg28Xi8ee+wxNDc347nnnsP4+DjuvPNOHD9+HGq1GldffTVSqRQkEglrw7mYIZ1OQ61W83uk02ksW7Ys53WeiYjBr776avT19cHj8SAajaK9vR3hcHjGfZxLSZBgPeuss/DP//zP+O53v4tnnnkGxcXF+NrXvoa9e/ey6zMT5cScmUwGarUaBQUFKCgoQGtrKzZs2IDGxkYUFBQw6oQirsSUwWAQlZWVUKlUyGQy2LRp04yRVXHEk7Qxvdh0wSB2tHMhQhSJF09sbtLiEkPkQoIgQKVSYWJiAt3d3YjFYpBIJDCbzRxuV6lUKCsrg0ajyQl5RFReXg4AHM4XR5VJEDgcDgwNDc07Fv2dz+djQUHzp9+TWU/5ZAIsUIoJmAr4XXLJJdiwYQPKysqgUqmg1+shCALGx8d5fUOhEI9FUXBya6qrq/Hwww8jnU7D6/XCZrOhpaUFx48fh9FoZNSWTCaDx+OBwWDICt6R8KTvKShIP5se0MmV6HzJ5XJs2bIF5557LjKZDI4dO4bJyUnY7XbYbLY5mYjWcjqDEmPecsst+PrXv453330XABisMjQ0NKepPOeJ1Gq1uOKKK3DOOefAZrNBo9FAJpNxNC6VSiEYDGJ4eBitra1wu92w2WxwOBwIBAK84RTgqK2tnVHq00tRhEysMcTfk/bM11yc/vdizQGchFflOpZUKkVdXR1KSko4h+rxeFBUVIREIsE+l0KhwPr16/HGG2/kNDYAtLS0ZGk3sW8IAOFwGFdeeSVHGXOZL/nD9O6UQiFKpVIs+AiKOB3l0tvbC4fDAa1Wi/LycqjVamZeqVQKrVYLuVzOaSux1k4kEkilUgx8IKQX+XJLlizJEsLEZOKDS/tDPyP4IaXhBEGAXq/PeZ2n08qVK7FixQrI5XL8+Mc/xvDwMABg0aJFfI7FRPPV6XTYtGkT2tvbceLECf69QqFAZWUlvvzlL+Pxxx/PghdKJBKEQqF5CyLmPJFLlizBP/zDP0Cr1QI4mY5Ip9OcNikqKsIf//hHeL1eLFmyBG63G11dXchkMnA4HOyPkgMvl8tPeVHxpMWBFDGD0oLkGxQRM6Z4vOnmrdFoxNjY2LzjEeStsbGRNaLFYoHL5YJer8fo6CgKCgoATDFSfX09Dh48CJfLldN8m5ubodFoTglgkbmXTCaxYcOGnJiTLAzS9MTwYilPDJtMJhkEMl1bpVIpHDt2DJ9++imvVVVVFaqrq1FcXAy1Ws174/P5EAwG4ff74ff7OSBEiX+pVIpAIMCmMI1PQRnyU8Vzo0IFmhedwWg0CgCsdeciArmQgCAiEM2VV14Jv9+PH/zgB1l+oEQigV6vZ4EmkUiQTqdx/fXX4x//8R8RDAZhNpsRCoVw9tlnM5JIIpHg/PPPx8cff4z9+/dDoVCw0DGbzRgZGUF/f//pgxBWrVoFmUyGQCDA8DmqDBkcHIQgCHC73bjuuuvw/PPPw+PxoLS0FFVVVbDZbCzZxP5OVVXVrPClRCLBfoSYechsIV/OaDSeov3mIjoUM5kQdCjySaVQSsjlcrG2EEt2QRC4+sXpdPJn5yOpVAqHw5FldhMWlgSXx+OBzWaDyWRif3AmkkgknJPLZDJQqVR8sMQMT/BFAlGIBRR9LpPJIBKJsOUUi8XQ0dGBrq4uyGQyXl+yFsgSUavVjMiigKFCoYDJZILRaERfXx8SiQTnblOpFD9DnGYhjUoCls4CMbw4uDUbVVVVobS0FH19fVkugVqtxte//nUIgoDf/va3zJj0DKVSCbvdDgBsPQ4ODqKhoQGlpaUYHx9HKBSCWq3Gl770JTz00EMApiCFmzdvxne+851TUn12ux1jY2Nwu92nz5zDw8Pw+/1IpVLQarUs6SgAJE78XnvttXyISHISUkIqlcJgMECtVqO5uXlW5hQn3YHsjRH7GpQDzMWsJZ+TPj99MTKZDGv0XKm8vJz9TKVSiUwmw+YcSXqfz4eSkhJ4PB6YzeZ5UUzAFNMXFxdn5SGnC5ZwOAyLxYKqqip88skns45Ff59IJBAKhU5ZA1o7MUhfJpMxoknsPtDnxGVTFBegapbpATbS0CRQxXsJTJmlZEFdcMEFbI2ZzWY888wzuPrqqzmlIdZ0NA4Jv7GxMVgsFl4zjUYzoxb1+XyIxWIM8iC6+uqr0dbWhu9+97s4fvx41vvS+peVlQEAli1bhjvuuANdXV0466yzkEwmEYvFYDabodVqcdNNN+HQoUMoLy/H3XffjZdffjnL1CWy2Ww5xSHmPJF79uzBeeedh/r6erjdbsTjceh0Omi1WphMJpZYkUgEk5OTvJhiCofDCAQCMJlMKCgomBe4TRstPqDAyUNFjJArUWCDJO/0qCwdoNLSUhw+fDinMcnMIWuCKjzo5yqVCmNjY2hoaIDJZEJRURGUSuW8pldFRQVKSkoQj8fZp0smk4hEIrDZbAiHwygtLYVMJkNlZeWczAlMrZm4ekasecQmIq21SqWCx+M5xQWYiSgS+1mUlRUXF3PqKZPJ4Gc/+xl0Oh2WLl0Ko9GIyclJFny0h1qtFp988gn27NmDO++8k4EIK1aswJ49e3gssmTkcjkeeughHD16FA888AACgQDWrVuHHTt24OWXX8bevXtPWTtgStMTiufPf/4zfvSjH2HFihUQBAFerxcmkwkajQZyuRzV1dV46aWXWAg8+eSTWRqdBJfVamXz97The8PDw/jOd77DNYwlJSVZJhCZFPF4nDeSzBoACIVCUCgUMJvNCAaDOHjwIN58881Zn0cbNH3C0xPf+aQ9ZDIZCxKxVJ9Oc5mI04ki1MCUvzkyMsIak8w5QRAwOTkJrVYLq9Wak0AhGJrdbkcymYRKpcIjjzyCJ598EkePHmWfSSKRwGKx5DTXSCTC1o/H4+GqEzJvSROpVCrI5fKsNFW+CKHTpa6uLmg0Gtakw8PD2LZtG/9erVazzywIAgcZAWDjxo3YsWMHn8f6+npMTk6iurqaA1Jkdur1eqxfvx7vvPMOUqkU/v7v/x7t7e3413/911nN4lQqxZHacDiMgwcPoq6ujrU8BabobJGLtHv37lO0JrkQOp2OXaI5MeHzLdzExAR++ctf4vXXX8f69etRVFSEwsJCPoyhUAjDw8OIx+NcnSE2wyjN0t/fj8HBwVknIwgCw8fIFKJxKMJIY+ZzaJLJJNxuN2pra/lnNCYtpFarhdvtznlMilRrNBrG+FIVg1Kp5MQ7pQbIT58PWtbd3Y1PP/0UFRUViMfjkMlkXIpGdaIymQxutxsHDx6ccywKmpBZ2N/fj6KiIp7TdEwxHXoy18Uoor82Uf7z4osvRnd39ylVG5RmEwexiHw+H0ZGRmC1WnH06FG88847iMfjmJiYgMFggNFoRFFREQwGA7eSOeecc1BZWYnu7m48++yzCIVCs+bMyYoBphj1wQcfhMvlwvnnn4/m5mZeIypUVygUcLlc7HvORBUVFTh69Oi8MZOc7cP+/n709/ezZqDNFfsOfymRBFcoFCzJxRFDOjzFxcU5j5lMJnH48GFG11BujFIHsVgMBw8enLfdiZiKi4sxOjrKB5iCZj6fD4IgoLOzE5lMBlqtFj6fL+dia7/fj1/84hdYtWoVTCYTCw7ypUhz/PCHPzwFaTWdxGkOpVKJRx55hGs6iaYfDgrwEfA8n6j4X0KBQABPP/00Lr/8cgSDQc6REkPOBJqgd/N4POju7sbmzZvx2GOP8T5S0QG9P33J5XIu5vd4PByMms28HB4ezqrQOXbsGO677z786le/QkFBAZqamrBo0SJUVVXBYrEgEongsccem7HNCwWx3n33XRw7dmxe2GjebUokkimQsk6nw8TExIyOrdVqhVarxcTERM5VDRKJBLfffjvkcjkjRchflMvliMViCAQCkMlkOH78OCd0cyUaz2azwW63s4Rzu91ZUjkXUqlUWUgpmtd0f1qMXslnfLVajZUrV2Lp0qXYt28fjh07hm984xsYHh7GK6+8klXRMRfdeuutyGQyeOWVVzhvlwvdfPPN0Ol0eO655+b9O4lkqoGYWq3Oqr9MpVLsN7vd7nljDUVFRVi+fDmcTicOHToEAFnW00wklUphMpnwzW9+E2effTbuuece7Nu3b87nkHKJRqNZGloqnaqMmqksbK45iNN/xPik6SmyPRPoRrx/wmfRQ8hut2PdunW45JJLUFlZiQ8++AAff/wxvF4vkskkbDYbjEYj1q9fj+LiYrz++uv4/e9/D4/HM+sz5qLy8nJ8/etfZ3Ps4MGDmJiYgEajQUdHB9rb2+eUemfor09GoxE33XQTm3harRYajQYulwsmkwkmkwkPPPBAXkAMYCro1tjYiEWLFkGpVMLlcjGYRa1W4+DBgzl1ISQBSYX91IxtfHycu0JQMKmmpgadnZ0Ih8OMEMqVDAYDHA4HVq5ciVAohPHxcZhMJvT09KCvr29O33I25szZrJXL5di0aRO2b9/OpuGll16Kyy+/HIODgxgdHUVzczPjayUSCa644gpYrVb86Ec/yvklgZOSpaKiAldccQUsFgukUimuvPJKTpYfOnQId955J7fPPEP/N6RWq1FaWgqr1coIIrlcjqqqKng8HsjlchQUFORcNmc2m7Fjxw44HA7U19dzoXM4HM6KKPf19WFgYADPPfcc9uzZM+eYgiDg6quvxqZNmxCNRqFSqbB+/XoEg0E888wzuPHGGzmV09jYCIlEgmPHjuG//uu/OHo9FxkMBtTX10OhUKC2thYnTpzgvDBFemdKqcxHOXv8arUaK1asgNvthtfrRTgcxuDgICYmJqBWq1FRUYFYLIaJiQkIgoBgMAiXy4XGxkY0NDTkNSmKIlqtVlgsFs65xWIxaDQarr/Mt6HTGfrsSaVSoby8HBaLhQEkZMJSH522tras+tTZSC6X484778Tll1+OxYsXQ6vVIhQKoaurC2NjYxgcHMShQ4cwNjYGnU6H5uZm3H333fjCF74wZ3ClpKQEW7Zs4dgI4VojkQiKi4shl8vh8/ng9Xo5iNbY2IjS0tJ5wS4UOU+n04hEIjCbzbjrrrtwww03QC6XY3x8nIsIcgXNEOXMnCQBKYxNpmQgEEAwGEQkEuEyMTFWU6lUYvny5XlNipL7q1ev5meTr0CRs46ODjgcjqwE+BnKnaRSKWpqarB48eIsUHy+JJPJ2Icni4q6Vuh0Om6ilUsqaePGjTj77LMRDoc5h0opH8olUwfBZDLJOfQVK1YwUGA6CYKA1atXM7qNOidQznzt2rXcaJz6TNE7ENZ2NpJIJCgsLERxcTFqampQWVkJQRBQVVWF2tpaNDQ0YOvWrdiyZQsuueQSVFRU5JWjz/mTZrMZGo0GwWCQG1CRwy/GroojrIRjnG3hZiMqB1q4cCE8Hg/S6TSsVitkMhmbzASaPl1/U6/Xc+vJ/z+SWq3GLbfcgtbWVoRCITz88MNZQbaSkhJO6M+VZqKcbk9PD2KxGGw2G/fzpRYzxcXFOcEj6+rquOaUzEwgO/UFnATqE+ilpqYGFRUVs1bq1NTUcK+kTCbD4wPgogBSOJlMhi2zioqKOatd5HI5iouLIZVK4XK5IJVK8d5772Hjxo2oqqriYnKCKdbW1iIWi+V8A0DOzFlWVsYMODY2xtUphNygiCQli0mbUlcElUqVU8pFDGZobW2FxWLhHB+ZHb29vfjd734Ht9uddzt+iUSC6upq3HHHHejv78dTTz2VFSonk5oq4PNhXjHIYTahMdfv/jeInk81mV8itDgAACAASURBVCqVCgUFBbjvvvtw/PhxOJ1OOBwOPPLII6itrcV9992H5557btY56/V6hEIhhEIhBAIBbgfZ39+PyclJ6HS6LGDKbFRVVYUNGzZAq9WiqKgILpcLgUDgFKQPfVEk1GKxsOn84YcfnoJQIz+VUEUSiYRrjqfPSYyUUigU3G1+NqLPEYaZ8smdnZ1QqVT49NNPYbfboVaruVqpqKgo53ObM3MWFBQgEolg6dKlOHz4MIaHh1FaWsqmBy0eJY0lEgna2towMDAAo9GI4uJiliRzER0C2qihoSFYrVaudtfr9Th8+DA/N1ciptu0aRO+/e1vAwAWLlwIk8mEH//4x6yp161bh1QqhYmJiZzmC0wdgMWLF0Oj0SAWi6GpqQnDw8Po6emB3+/nDg8OhwOXXXYZXnvttbzyqsBJ0356C5d8if62srISFRUVmJycRDgcRllZGd555x189NFH2Lx5MwKBQE6RcLPZzMAGQRDw5z//GR6PB5dddhlcLhcXes83DgEr6uvrUVpaing8DrfbjUQikaU5CZJIV18sX74cPp8P5eXlMwoAsb9H4BFyhabjttPpNBKJBCoqKjiYRbWXM5HNZuO0USAQgFKphE6nw29/+1vEYjHo9Xr+PZWXAScBIvNRzszpdDrh8XjgdrsZJEBlYIQ0IZPWbrfj6NGjXJR75MiRvDXclVdeCQDsw5LfIZFI8OKLL+Lzn/88/vSnP+V1yEljWK1WDAwMQKfTYe3atbjjjjvYp+3v78euXbvQ3d0974GiHN/ixYtRVFSEgYEBtLW1oby8HIsXL0YkEsHAwAAGBgaQTCYZsF5cXJzzvDUaDZYtW8aVHK+88sopOUMqVhY37J6PbrzxRkYhEZpJq9Xiggsu4MoUikLOdZhIg1F1SjKZ5DiERqPhwoD54gKhUAj/8i//go6ODpx33nlYs2YNjEYjp+koj0hQUWLExx9/HB0dHdi3b9+MOXWHw8EuERX9E4n3l3xaCj5SCWFNTQ0++uijGees0+mgVCpZYBK01W63w+PxcJUV5T8pl5or8ipn5ty7dy/6+/uhVCqxZMkSXHvttQgGg/xC5OiGQiGYzWYMDQ3h3//932E2m/HJJ5/kLO0pSrty5UokEgnuF2owGDAxMQG5XI7zzjsP+/fvx0UXXYR/+7d/y2lcev6RI0cwODiIZcuWcYVCTU0NnnrqKRw9epSbYtFcZpofjVdaWopt27bB7XajsLAQjY2NjKqJx+MIBAIoLCxEVVUV9u3bhxMnTuCCCy5AUVHRvFUq1AZ00aJFWLlyJcMQe3p6si7uaWxsxI4dO6BUKnHPPffkVJNqNBpx4YUXIhKJcMyADj4B7Sk4snLlyjmZk0q9AHBs4MiRI5BKp3o9LViwIK9c9KuvvoqDBw/iqaeegsViYbNSXAwhBjk8+uijc45tsVi4awNd5CTGFNO49BxSNhqNBhLJVAO7mZhTLpfDaDRy3GVsbIwriAYHB/mWAhqXmJIUVi4uXs7MKQhTLSmIEcU9fej3tInUFa2/vz9vfCZF1wwGA/clIsnr9/ths9nwhS98gSXim2++OSNUSqFQoKioCFdddRUSiQQee+wx7kT3u9/9DtXV1TAajdzEub+/n7G9BoOBO9RTl3Qi2jibzYYLL7wQR48eRSAQwNtvvw2v18vQs5aWFqxcuRKVlZUYGBhAMBhEQ0MDA9uVSiVisdgp5XFSqRSlpaW47bbbIAgCAoEAOjo6oFQqUVZWhosvvhiLFi1CIBCA3W5HfX09CgsLuQJmLuak/kbXXnstZDIZm5xUwEDWiZgBVq1axYd6tv0i8Df1LyZoYzgcht/vz7unLB1iOmt0zqYXoOdi4ttsNlgsFm7yTO9H+GFxMJOYn7oqajQaVFZWzjpHags7WykdxVqooNvv98PtdqOgoCAnYE5e3fdocahXDL2M2GlWKpUIhULsJOcjNckv+OIXvwi3243S0lJotVoEAgFkMhn2OzOZDDZv3gy3241rrrkG//Ef/4GxsbGsYMv69etxzz33cAVGV1cX3nrrLQiCgD/+8Y/weDy49dZbUVRUhLq6Ovzwhz/E7t27YTabGYpG/VuJOY1GI5YtW4bi4mKsW7cOL774IlKpFLZt24a7774bwWCQ1+i9997D7t27sW3bNu63VFRUlNVeg+pcNRoNl5bZ7XZUVFRwRE8qleLiiy/G6OgoDhw4AGAKOE3Rz0gkwm1H58PCSiQSbN26FTfddBOXtokbOFOFkUqlYuGhVCqxbt06vP7667OOSxhr6nIwODjIgo6ab+dDFLyhQNBMIHxBEHK6N4YQS9QrSdzRQMykdJbp/FosFhgMBhQVFc04rticJSVCY1Hkl4JZ5DI5nU5kMpmc26nkxZwksYxGI0sgcYtDkrjUKEqsYXMhQRCwZs0anHXWWdzdXCKZanMSCoWyqlSAqYqEwcFB6HS6U8DSR48eZR/Z6/Vi+fLl2LNnDwwGAyorK1FQUABBELB//354vV4MDQ2xdqbFpe+JKioqUFZWBr/fjwMHDsBqtSKVSuHw4cOnFPECU+byLbfcglgsxmbNxMQEJicnUV9fj3POOQfj4+Nc0UBrEAqF2EeamJjA22+/jWQyCY/HA61WC6fTya0pFQoFwuEw3842034JgoDq6mps374dl156KUcPiTEpMCI+sHTw4vE4ampqZt0zAsiTGexyuTA2NsYpLzq0+QSxSPNQRQ2dMZofVX/kwpxKpZK7MBBUT9xdYvrcFAoFY6XJtJ1p/nq9njv+U8YCONkaRqfT8ZkgzC6VilGvrPnotPrWtrS0ZLX2oFwQMY9MJuMQdz63Ymm1WmzYsIEd+FQqBafTiXg8zn4DPQcAnn76aTzzzDMcGVu9ejXXOUYiEcTjcRQUFMDlcuGaa65BKBTiQ5lIJPDcc89Br9fD5/PxvMmcUqvVrAnFQqmyspIv9vX7/XxNHZledrsdFosFNpsNNTU1ePbZZ9lkppK6wcFBBlSsXr2aq0Xi8TiH3qmGkcD/1LwamDq84sZZgiDAZDLhww8/xIcffghgijFtNhvOPvtsrF69Gps3b4bBYGDNDSDrmj1CxogbfdF7z+WaUCG7QqGATCbD4OAgR1OJqfK534WeR1oslUqxX0waGgAzxFyMr1AouBCaNJe45Ym4AJ2eS2lBnU7HfiNdrCQmQj9RIE5sdtP4lIc1mUzweDxsKVHUdj7KizlpERobGwFMRVKplpEkBnUqKyws5MZHuVJ1dTUuvfRS9PT08Ms7HA74fD4oFAosWrSIN4jKm8TmczQa5a7yAPDOO+/A7XZDp9NhZGQETqeTI2iZTIYbQ1E5EfVYTSQSWcx9/fXX8wEjRjGbzSgqKoJUKoXT6cQXvvAFTmBTkITKkqgJlrjY9qOPPsIDDzzAAo7Wlg40rSnlCMXMM1ugiiqGbrvtNpx33nnceYJqTV0uFzMj9eoR1+DSXpIPRT7TXLWj1A2CWr309vZynpvMt1zrb8WMRnlxuuCXGFEcIKILn2YjKl6nOQYCAV4n4KTQEac7yOck4AWlR2a6ApCsglAoxO1RqKE2pWFCoRAKCgowMjKCdDrN4J2ysrJ5K37yNmspLEymAi0CbTC9rM1mg9lsxuDg4LxjEl199dUoKSnhsp3JyUl4PB7Y7XZkMhmMjIzA4XBw0Enc9oOYrrS0lEu6vF4v11SSdqUNp/wVXcRLBc3kP1CR+ODgIN5++20UFhYypEyv13P+TKVSQavVoqmpidEntDkVFRWsEeVyOaLRKFdDENRRfJ2A+HCKC6XF60SHi1IXYqamKwMvu+wyxGIxuFwuBocQGIRiADSuGAFDELxwOIxwOIxkMomhoaF526EQtlQqlfKBc7vdKC8vZwBJPmYttc4UFzKToCItnUql5oXC0ZpRBwm6kEh82RRZPGR60ro4nU5UV1dzTe10onXz+XyYmJhgpBCtrVwuRyKRgFar5SYDpGToxvP5KG/mpCJgChuTtCUpQhJfqVRmSbzpmyNuRyIIAtauXYvPf/7zcDqdsFgsDJ+iaoR4PI5du3Zhy5YtMBgMnIQmSiaTCIVCqKysZJ8lEolAqVTCZrPxtRAmk4krJ8T+BwA0NTXxNXLBYBAHDhxANBrFkSNHsjaF3pekuLiej+oEyUQW+0qJRIL9ceq9lE/D6flIJpNhyZIlUKvV3OaE3o0OOjXupgNE2kWhUHBRulwu53UnV2AuIixtKpViP8vpdEKn07F7kAsRM5EPTjWRdFZIIIk/M994Eokk653J2iPUkPi5RqMR8XicLxkGTr1UiogUhPjdxDERmrNUKoXb7YbD4YDX62V3aL5+UsBp+JwlJSUoKipCKpWCXq/nnJhKpUI0Gs1qM0Jdu4FTMbCEmKCmvPfccw+bkoIgcNMnYvodO3YgEolgy5YtnOydbjIfOnQInZ2dWLBgAYqKilBUVIRMZqrPLXUyHxkZQSAQQCKRgMvlyoqw6nQ6GAwGAGB/dzpRxO9vkdLpNB577DE4HA40NTVxQI7WdHr7D4pivvjii3j00UcRCARw++23o6GhgX0xCrbNR1qtFuFwmN0Mj8fDJmm+hQnitiDEQNMFoSAI81a6SKVTPXApLadWqxkfrlAoYLFYUFBQgM7OTvh8Pu7uPjY2xsFGcUBpOpG1Qv62GClH6SBxRDgSiSAajeZ8q13ezEmBHpJC9HClUolAIMCmYyaTQUFBwYzJdqlUissuuwwPPvggzGZzVmd4Mvvi8Tjnzb71rW/hD3/4A84991zemEgkkoW9BKakXCKRyMpL5kMzRVz/X6Njx47hmmuuwerVq3HNNdegqqqKO9PTISOz/d1338VDDz2UlSeWSE42UY5Go+jt7Z2VuchSIu0izrFS9BtA3mYt+arT0xwkbOgMEFZ2trHpxjvKcRYUFDAcEDjZ4jUSibCGLykpYaFC59xoNJ4yNj0zEonwGSfflVwUSleRtUSuihjEPxflbdaSE07mayaT4dvEqLaTJkTBh+mFtg6HA1/84hchlUoRDAbZXFEqlRwAGhgYQF1dHXbu3ImXX34ZwMkqFwr1E2b1DGVTJpPBnj17sGfPHmi1WlRXV6OsrAx2u5398ePHj8/Y0uNXv/oVnnnmGTbtJiYm5ixmJ7dgemqDkFYkCPJhTnHtIwl/+r/YpZrP56RYwujoKAwGA5vqgiCwaUuKBEDW7XCxWIzjHzPNnQKDAwMDaGxshMvlYpdAIpHwdSSUXSBGJX7Jqb415xX7HyJbm5pME4yKGIdA7+QYzwRCCAQC6OnpwcKFC2E2m3kDyFelPq/d3d34+c9/Do/Hw9KJAhypVIpzaWdodopEImhvb0d7e3tOn58vgDcTES6X/GfKD4sDYbkwJ50bAozTPosRS+LKn/namRKzkGak5t7i5nTk9hDuWnwDejweh9frnbFkzuPxoKOjgwsESMtTQGlycpJBHhRsMhqNaG9vn7dbP1HezGk0Grlej/q5kHNeUFCAwsJCNitng1cFg0HcddddePjhh7Ft2zZs3LgRpaWl7J8kk0mMj4/j1ltv5UihIAiYmJhAR0cHqqqqsGfPHkxMTORkHpyhvx7RpU1U0kdBGAp4zFSaNRuJI9UejwcWi4UB45TrFFti1EhutqqRYDCIrq4uLFu2DPfff3/WJU2tra3cNM5msyEajWJgYAB79uxhk/b888+H1+udsdOh1+uFz+fjKxpIEVEQlPxmgkeS1qQIeC6UV4MviUSCuro6NDU1wWw2w+/3Y3BwECaTCUqlEh6PB1arFQ0NDfB6vThw4AC6urqyFn42slqtfPekx+PByMjIjN2y6XIkn893WpFOq9WKkpISyOVyjIyM5HzB0P8mSSQSNDQ0wOFwMPrG5/Oho6PjtJulEU0vk6Jke75AAaINGzbgoosugkajwUsvvYRDhw5Bq9Wirq4Ot912G/r6+rBv3z68/PLLHCyaj2QyGerq6rBhwwYsWLAAANi/JO2zd+9eDAwMzHvQCwoKsGTJErz//vt5tXCVy+U499xz4fV6GdgxE+n1eu5xlUwmuY0Okdi6i8fjGB8fP2Wthc+i+x4wFTbfvn07Vq1aBafTic7OTjYf1Go1jEYjli5dirGxMTz++OOz3osyE1HQgvKJFPGiXKLP58sL1DCd2tracPnll3OU0ufz4aWXXpoROP9/Sd///vdx3nnnsVUCTAm3gYEBfOMb38hZ8opJKpVi+/btaG1thVQqxcDAAMrKylBUVISxsTE8++yz3JJyPpLJZHA4HFi4cCGam5tRVlaGpUuXIh6PZwXyDh8+jImJCfT39+PgwYPo7++fd2yz2Ywbb7wRixYtQn19Pd8yTRo4k8lgfHwcx48fx7Fjx/DII4/k3K1fIpGgsrISq1evRkVFBeejY7EYYrEY9u7di/b29izzOR+qqKjA9773PajVar6F74knnsAHH3ww53ifGXNedtll+Kd/+ieEw2GkUilu9d/V1YXy8nJUVFRApVLxVeP33ntvTjkdYCpQ9MADD3DEjBaPIogvvPACfvKTn+Q01rT3QFtbG772ta+huLgY4+Pj3Lx4bGwMTzzxRM69YP/aVFxcjFdeeQU6nS4rGa/VaqHX6/H888/je9/7Xs5aiOiGG27Aj3/8Y8RiMY6OkyAkgMf1118/Z5c4CsJ87Wtfw6WXXoqJiQnO2Wk0GjgcDgBT5uTQ0BAUCgX7elarFXv27MHDDz88Y8tJqXTqOsn7778fS5cu5ci7RqPhwnqj0cj1omazGUqlEh999BHuuuuuOX1limlcccUVuP322+F0OvlKBbqE1+VywefzYXh4GDt27Mg5XUamtkajwd13340vfvGL7CsrFAoMDQ3hq1/9Ko4fP54F7xPTbMyZl8+p1+t54ag9RXl5OdddEhCcLkq98MIL8etf/zrr9qa5iDqqURCgvLwcwWAQbrebsZanQ4IgoLW1FQ6HA5lMBsFgEHa7HfF4HEuXLsXBgwfx3nvvzSndKLw/vRkxkRhkfjqkUCiwYsUKfO5zn4NOp4PZbOYqHb1eD5fLhVQqhXPPPRfhcJgrcXJ93tatW7MileQXUqCtvLwcV1xxBX7605/OOkYmk4HdbkddXR1effVVTE5OZuUkye+SSCQM9BfD2+rr61FXVzejmSgIAi677DKsXr0awWCQUx7ULsTpdEKr1XJAkiKqy5Ytw8UXX4xf/vKXs8YfMpkMmpubsWzZMhw+fBjj4+NcvkdC4eOPP4bL5UJZWRmWLFmC48ePz2ulUVFGVVUVbr31VqxYsQLhcJhL7Eig/Od//iceeeQR7N69GyMjIzm3cs2LOZubm7FhwwYui6ENoYgrSTtyguVyObZv34577rknt8nI5dDr9Qzbs1gsHPWjm4LFl5DmQ6lUCm63G52dnVwQq9FoYLfbuaPb9EQ9MGXCtbS0YM2aNfD5fHC5XHxQCBwuLhmitAKZYORfkFmmVqsxPj6OwcHBrMPU0tKCn/70p1Cr1fD5fKzRCIZI6Sqv14utW7dCo9Hg/vvvz9kHdTgcWcgVYmp6hkwmY7jaXOtbUlKCsrIyzv1RyoRydzSuGDpHbTNTqRRKSkpmHNdut2PNmjW8hmRuut3urNvbxBhX6qbe1tY25x2oer0el19+OQeO9Ho92tvbcf7550OlUuHEiRNwOp2QSCQ4cuQINm/eDJPJhHfffXfOgKNarcZ1112HjRs3ctcG8jlJoFD39xtuuAGbN2/Gm2++ieeffz6nmxDyYs7Ozk7s3LkTV111FYejxQxJCVmZbOrC1PHxcc5R5kKE4KDDQ/1xxbk0vV6fM8iANrq8vBwlJSUYHh7mxLRer0drayucTieam5uxe/fuGWFmdXV1uOCCC9DT04Ply5fDbrfjtddeyxIUEomE607tdjvnbincb7VaUVNTg4GBAfh8PoyNjcHpdGaZ+1dffTUMBgOnp2KxGOfiCO7n9XpRWloKqVTK7VByYU4yi8WXQVFKir7kcjlKSkpgs9nm7AdcXl4Op9OJnp4eqFQqmEymrDwmjS+uHurv7+fKH2oKN/1wlpaWYvHixRytr66uZvOYCuLpwim9Xs8WTEFBAVatWsVNwWaihoYGJBIJTE5Owmw2o7GxEQMDA5iYmIDZbEZvb2+WMJBKpaioqIDFYplzfb/yla9g2bJlaG9vx/LlyxEKhZBKpTiwSRo+Eolwm5pLL70U4+PjOV0nkhdz+v1+PPDAA2y/kzZIp9OM1qGo2jvvvIP7778/r2CLxWJh/CkARg1R4yxqKp0Pc2q1Wpx//vkoLS3lQ07P2L9/PxKJBOx2O771rW/hvvvumxFmqNfrUVlZCZvNhkwmg7a2Nhw6dAiXXXYZwuEwjEYjjh07Bq/Xi9bWVpSXl0OtVqOnpwcejwfV1dV81XhLS0tWa0Yih8MBtVoNv9+f1UqDUgekdSlXF41GYTabc1oHm82W5V/SmGQpUGmW1WqFyWSakznr6+tRW1uL2tpadHV18ZUIYtQLgUokEgncbjcaGhpw7rnnIhqN4tNPP8XOnTtPYU4CrygUCiiVSgwNDaGkpARVVVV8ixiV0gUCAU5fUFH2XAXMhYWFbMEkk0n09vaip6eHhSlFmHU6XVaaRq/Xz8qcKpWKe96GQiGOEdD58vv9mJiYYDAOKRiXy4Xbbrvts2dOYEri/uQnP8GHH36IJ554ApOTk6fgHiORCG6++ea8Mah6vZ4PD2lgitRRE6l8/E4q4VqwYAEymQxfxErtDEmQUBWMw+HIutkZmMrrVlVVMVC9rKwMZrMZY2NjPKbf7+eKe7oa0Gw2w2w2c3Gt0+lEQ0MDqqur0dXVdUruz2q1QqVSwWq18thEVGBss9kgCAIqKysxODiIBQsW4IMPPpg3DUK3q9EBoYQ7md1qtRq7du1CS0vLvMgVjUaDnp4eDvacddZZDFmjsUlgE+aaLowaHh7G+Pj4jFhV6gZBUc5EIoHh4WGUlJRgwYIFiEQi8Hq93H5Go9FAq9UimUxyHnE2Im0tjvqaTCYuMVSpVOx/Ui0nCYPZqKmpCQUFBVAoFFi7di0AcJUR1XJSIJMEYzqdRllZGeRyOcxm87xw0dO+gJE6HRCEig7QbL5bLkSmIEnVYDDIWkbc3CkXIp+HKtbptiuDwcD+D91KTJ3tWlpaTkEcUZ3o8PAwrFYrEokEPB4PTCYTvF4v31pGAiUYDHKbk4GBAS4IaGpqQlFREXp7e09hPmDKlyN4JHUmp6oScdUPrbFUKkV1dXVOHcQLCwv5nUkz0BetO4H856v0sFqtKC4uhl6vx+DgIHw+H0ZHR+F2u+HxeLjpeCqV4hK8EydOQKvVoqKiAs3NzTNiVQEwhFNcAC6OBo+OjsLlcsHj8bApSgUUc60D3cZeWFjIwpgiz3q9ngvCqfKFakDnIofDwVU3LpeLu1cAU0qB6oi1Wi0MBgPS6TRDAk0mE9+hMhflzZykUcRoC3EQRCqV5pw6mU6keUgaUsUIISvyRQNJJBKcffbZGB0dRV9fH8LhMHw+H/vIdOGvwWBAKpVCaWnpKSBtg8GAyclJZmoAWdULRUVFeP/995FMJrFlyxYMDQ1heHiY/crVq1fDZDJxhNvj8cDn82UJAb1ej4KCAgaSazQa7nxA/jF9iWsyW1pacqpuoJu1n3322azSMdJ4ZB6S+TUbEdNkMhkcPXoUL730Es477zysXr0ajY2NKC4u5ooLalZtMpnQ2dmJF154AR6PB8ePHz8l4ERCKZFIsG9qMBhQVVXFKDOv18sX+4rbl9DP5oJx6vV6TpnQNQ60jnR+xV0HqQftXGOWlJTAaDTy2P39/dztgPaP4H80d61Wy8XWubgkp605Y7EY+34Er6LQOfmf+RCF34k5JRIJJicnuZ/L9Hq+XGnBggWwWCyorq5mWJZUKuXbuYGTfXto4cREPYqcTie6u7uh0Wjw8ssvQ6vVYvPmzXj66afh9/uxePFimEwmlJeX45NPPkEymcR1112Hrq4uHDx4kP3S0tJSbvNPVFJSwoXEBIUjEDYl9umLmEkikXDVz3xEB/H9999nrSv2P2WyqduyxZbQTFRWVobly5ejrKwMwWAQ7e3teOONN7Bz507s3bsXXV1d8Hq9iEajCIfDGBsbQygUQjQaxccff4yVK1fic5/7HJqbm7MOPlkBWq2W/XrSvK2trSgtLYXNZoPJZGJgClXaJBIJyGQymEymGZlJoVBwzx6y6MRpL3HxOgAupp/PVKaetZTqqays5NJEyiWTQpmcnGThR6mbXJjztHoISSQSNDU1MdCXtA1JNKPRiLa2Nhw+fDivccUpCKlUylUEALK0cy4kCFN9SgsLC1kikt9ByWbyw5RKJecVS0tL0dPTw+MolUrG/R4+fBhjY2NcSE6+nsPhQF9fH1dBVFVVccU7her/+7//G8lkEn/3d3/HVfNEVquV5ywOrEw/bNRCZLqpOx/RZ6hnENW2ktYgUDkwd78gs9mMAwcOoKmpiYNzZC1Q0QOZh6R5aE26urowMjKCgYGBrKZw9EybzcYFFel0GgaDAZlMBk6nE3a7HWazGTabDRMTE3wZLxVMC4IAm802Y56aUlAA2P+jckc6D3Q2SBuTRpzL/yZwiFQq5a56NG+q3VQoFAwxpVyv1WplsP18dFrMKZVOtWskv1LcVSCVSvElu/kyp9hnVSqV8Pl8WdJrPsk+nbRaLQRhqp+NwWDA8PAwjEYjmxfifqa0mAUFBVnMmUql4PV6EQwGs/K5drsdoVAICxYsQE1NDcLhMA4fPowvfelL+OCDD7hAoLm5mRmacpZifw8A3wdD1gLVBoo/Q2gpYOoqdJVKlfe1cqSZxRYD/Z/2bi5Qw6JFi1BRUYG6ujruoPDuu+9yaxIxQJ0sAeqtQ+kRmUyG888/H2+++SbHFqgemProkl9oNBohl8vR19fHkEtBEBjsQnfxkFk+01qQUNCGzAAAIABJREFUBiTBQcXnFPghYUTta8g6o1Y8swFT6J2tViuGhobYbBWnpuhsEdiDTN5MJvPX1ZxVVVXstJMEp0NHzEWfzRXFIja31Go1Tpw4weU84mvNcyW69ySTyWDhwoVYu3YtLBYLurq6OKJK0DWp9GQP2el0+PBh2O12rFq1Ch6PBx999BGjeACgvb2dfZC+vj6Mj4+jsLCQL31VqVS48MIL4XQ6sWfPHu4CR0QHknoMUdJdvCZkkqlUKnR1daGlpYWL3ecjWgOqmJiuNQFwnna2Cg9gqtvh7t272XRvbm7G2rVr2SSmThbiyhFKRcXjcTz66KOYnJzEwYMHs/xOimrGYjFuTi38T0E99TVSqVQYHR1lNBMFj8hfnK3jAiHLCK8NTGlQq9WKwsJCyGQyVP3PRb8AOFpMHRBmO7+rVq3iyLzJZGJLjQJNZElSaoW6HlKPptmCYmI6beak24rJZidzjH4mrsnLlcThfaVSicHBQe6ZSr5CPpqTWmyQfyK+Ak6pVLIJRbA+iuaKSavVIpFIIBaLob29HUqlEmeddRbi8ThCoRDsdjvcbjcOHTqEaDQKl8uFhQsXIhKJYGhoCEajESUlJTh+/DgnzkOhUJZWJBym2+3GyMgIFi5cyMW6ROKcpNPpxKJFiyCRTLURmeuKPmDKpOrs7ORGVSTZaUzKE/b09MwZzEsmkzhx4gR6enogCFNX/L399ttwOp1ZZqG4lQiBBxKJxLzQQPqbvr4+lJeXQxAEFBYWcp+qiooKuFwubvBFYAaC+c1E1ItIDFygAutYLAa73Y7e3l7Ol5IrRX7obJaJUqmE2+1GTU1NVjpHjLgCwO4HAC7iIEtqPjot5qSN0el0fMhIwgPg0h6VSpUXQFtc5a7X6zExMcEmAYB5o4nTyWq1YmRkBOXl5dz3loQISfd0Os0+SVVVFbflEEeltVotWltbMTk5icLCQrS3t0Oj0aC2thZ79+6FSqXC4sWLEQ6HoVAocOzYMVRUVKClpQVvvfUWTCYTampqMDk5iUWLFmF0dDRLaGUyGY48hkIhjkhOb0BFc52YmEAoFILVakVBQcG8t6Ht2bMHfX193FqUTDcKLGk0Gnz88cd466238iq2bm1txeLFi+H3+7ldCc2X1lkmk2FsbAzvvffenIxP1pZMJsPOnTtx++23I5lMcqScruJwOBzw+/0YHh5GW1sb56xn87/pDFqtVoZFkvAQ7zPlusm0JXTWbEQtU2QyGVfh0B5SegcAM6q4EwJZU/NR3sxJEmV6WkNc0iMIAt81kQ9zkmYkDUpt/cnXpMOUC6lUKlgsFrhcLs5HTTfl6HsqdRsZGeGLaeig0S3LqVQKhYWFGB8fx4EDB9Da2orOzk74/X5OrVBDsd7eXrhcLr71i0LpjY2NbF6JmZPMLeo1QykOWktqCUMHh+7coAbW8xHlIJcsWcLRTVpPCl5Q0CYXS4fMYofDgT//+c/c6YAOPhG1nKyrq+ObzGYiQnKJc8p0GVYsFkNxcTEikQgmJyfZXyMEVkFBAUPjZvM5A4EAKisrORNAQSqy+kjr0vmz2+2IRqNzpmjEWtVoNLJQJXM4FotlYQGGhoZQX1/P7/hXYU46LDQ5McQMOFnMK1bnuZL4hqdEIoFQKMQVCmTK5JrrpPIiulVbHDYXl+5Qq0vyo4qKiqDT6Zg5bTYbVq1axdUcjY2NnINtaGjAoUOH+LIbasVZUlLCQSeVSoW6uroscEZlZWWWVUFMSHBF8Y3OpN1p3hQlJSvAbrfntB40vhgOSO9PY+fjghQXF6O5uZlrcEkj01iEiCHLJ5eeOel0Gh0dHTwOoa6Gh4e5gVgmk8HY2Bi0Wi06OjoQDAZRXl7ObtZsRLnT8fFxzsMCJzUgxTSCwSD737msaSaTgcfjYXQbuQzUy5fcGKPRiImJCdjtdk4VzkenZdZS/mb6A0gTEcon36JVAnur1Wq43W5OPoubQOfKnHTDsvjgACeRTHQ4Jycn8dJLL6GtrW1G7ZxOp3HgwAFUVFSgtrYWhw8fhtvtRkVFBWuFEydOoLa2FgBw8OBBCILA2oKAB5OTk6ioqMChQ4fQ39+f9R7ilhb0TOCkQKFNl0gk7C+TCZZLc2IiKhWbaT1yqZIQU01NDV8BOROJ3R2j0YhFixbNWSROZu3AwADGxsYYzTQxMcF7ZbVaEQgE2EcfGBjgmMJc45IAIfOShJ/YAqSIMvUSziXQlslkEAgEMDIygqqqqixfniqVqFeyyWRCT08Pqqurs/Kqc9FpMSeZWNMZT6ydKBmbjzS22WyQyWSwWq2chnG73ezQExQvV6JO3JQCoQNN2og2ec2aNRxcoDwVUSAQ4GvCqQInHo/z/RgymYy7BcZiMQSDQd5gSsIToJ2YjDQfEVVEGI1G3lhaz+nVI4FAgCV7vsxJaBoxBJC+5orSzkR07yW5IcDJTnwk3JLJJCN85tPwtOZDQ0PweDysccj6Il+OPmcymdDV1QWLxcLpspnOGq2nVCqFx+PBggULYDQakU6n2WUxGAwoKytDKpXC8PAwRkZGuPXIbET5U5/Px9VCpJ0lEgl3zBc3DYvFYjhw4EDOgc3TjtZS0l5sJomB1SSJ8tGc1K5RqVRi9+7dAKYQOkeOHIHFYoHb7Z6zYkJMZEpZLBZEIhF0dXWhsLCQI5Z06Ckw0NPTA61Wi4GBgSyt1tfXxxJfDIgQf09WhDh/SOtE/4oPLZnIYiIfhSwHAFlBDnGAhYqvqVIlFxIEAR6Ph01l+hnNtb+/P6+9ItB5JBJhxhZbU/R/qiSpqqrKWgsxZTIZbsE5NjaGvr4+bNiwYVaBQQUNLpcLDQ0NAIBPPvlkxlQbWXmJRAKdnZ245JJLIAgCxsbG0N3dzakaujtGr9dj586d2Lp165zvH41GkUqlEAwGcfToUTQ3N7PWJJAKpVBI2I6OjnJZ3AcffDDvGp92tNbj8aCoqIjNBMo30SEUI4dyHfP555/HH/7wB/4ZFUV/61vfysrN5UJ+v5+bgf3iF7/ARx99hJKSEi4fotA3aT2pVIonn3wy63IkmtfpFHfnQ9T0aXBwEDt37sTIyAgAZPlwJDCUSiWOHj2KZcuWYc2aNTk1OaM1c7lc6OnpgVwuh9frRUlJCbxeL3p7e3MOBomDauPj4wyrE8cf6JkU1KKufLNRJpNBX18fRkdHcejQIUQiEe5NNdc8JBIJ9u3bh82bN6O3t3dG5iQgeygUwp/+9Ce8+OKL877jWWedlRUomon8fj/8fj88Hg/ee+89jIyMZOWnac/I73a5XNBqtdi4cSOSyWRObkTePYT+5+c455xzOHFMUT8ai3KU4jtGciGJZKq7n1arxaeffsraori4GEuWLMEbb7yRl5m8bNkySCQSvja8qqoKxcXF7IdYLBZ0dnbixIkTkMlkWLFiBcbGxuZNTUyn2tpabN++HfF4HGazGc8+++yMV5XPRlR65fP58Omnn/IFutPHOOusszAyMoLh4WGUlZWhpqYGHR0dmJiYyPk5ixcv5gDbVVddheeeew5+v39eZphOFOARM93/x96bh0dZnvvjn1mSyexbZjKZrBPIQtiXABJANkFEQOWgl9ux2lrcWrW2te2pPZe1x6OHeqq2p7We6tVSrYIiVcEqsggiIiD7ErIRkpBkMpnJZJYkM5PJ/P5I75tnss4EbP3+rnNfl5cKyTvvvO9zP8+9fD6fe7A0R6PRoLS0FGPHjoVUKsXx48dx5MiROJobVX6zs7O5yCOR9IHhCwsLceTIEd6c8/LyEIlEuB1F071Pnz49qNCXVqvlXvmJEycglUqxfPlyzmkBcJibmpqK7du3IxKJoLi4GC0tLbhw4cKgz4WGLre3t6Oqqoqr4EuWLMHChQsBAFVVVdiyZQs8Hg/3pIuKitg/BHrilRH4+qptzpw5uOOOO9DZ2ckkW5PJxALTyZi4i6tUKrz00kuMpTWbzfD7/Th27Bh+85vfwOVyxf18oteXSCRYuHAhXnzxRW7Sv/LKK/jwww+TuhZVjCn//MUvfoG2tjaYzWZEo32j47q6uvCzn/2M80Xqdyb7TFJTU7Fs2TKmtZ0+fTopx0z0c26//XaYTCZEIhG4XC6MGzcOK1aswAMPPDCo0h+lSEajEVOnTkV2djauuuoqRng1NDSgvb0dFy9eZKHs+vr6pBTly8vL8corr3DrhXCu1Ab66U9/ipdeeikpVBvQl4Pm5eXh97//PcrKyniq2A9+8AO8++67rCk0RF58+QJf/wgj2hABzkkyP9m2DADeWQ0GA+655x6e89Le3o7Tp0/DYDBg/vz5cLlcHJokorJAMqAajQZpaWmorKxkGZOWlhacOHGCKWCBQIC1boa7XmFhIcLhMJxOJ5PVlyxZgrvvvhvBYBBvvfUWPvjgAx5Kq9frkZ2djfr6+qT1e8vKyji/nTp1alLypckYgddpYLDL5cLs2bMxYcIEHDt2LC63l8lkzHopKCjgZv3evXsRjUaxf/9+LuJptVqUlJRg0qRJ8Pl82LdvHzOBRjKr1cpYZppcTbIwWq2WdXITMam0T4Ru3rx5+Na3voWpU6cyl5V6yb/5zW/w6KOP4p133sHmzZtx4cKFr0bg6x9lPp8vjvNIFbBkrbi4GPfccw/r/RA5OhqNMrb2/PnzuOqqq7B48WIcPHgQmzZtigO+D2bi5G3SC3r55Zfx2muvIRwOs/IBndDBYJAB7/1NIpEgOzs7jilC8v1bt27F1q1bAVzaoeml9/b2IhAIwG63o6GhISnB5IkTJ+Jvf/sbi4HTKIUreXrGYjH87ne/g9lsxqRJk3DVVVdh2rRpaGhoQH5+/gAgi91ux8KFC3kCNeW1BKQn8ADBBFtaWjgiKisrg9frHVbWk4yUCFQqFYNNRFxsMs5pMpnw4osvYvLkyZDJZPB4PJDL5Sy52dvbi66uLuTm5uKhhx7CsmXL8JOf/ARffPFFQs/6a+OclK90d3czQ4DaHwRISNaWL1+OoqIi1NTUsPgSsRymTp2Kc+fOobGxESqVCiUlJTAajVi4cCFaW1uHnClJyoJUBKOF8+mnnzIDgyrA1IQmbZzBHIjkNuhlkpqCaNTnI50aypdIfkOv1yeUd1LISCp+BKQg9I5YZb4SjpqXl4clS5Zg8uTJMBgM+PLLL3H27FkO8chkMhk/fzFyoRZEf8K9OHKvs7MTOp0OeXl5qK2tHbEPbrVaOQwmpo8InKCIJ5Hvf8MNN2DmzJkIBAL8fcjZqSpO997T04Pi4mLcf//9OHnyZEKtq6+Nc9LDoIVHNDTaPZPtw+Xn52PRokXcaojFLgn9ajQaRgMFAgEEg0GG+l199dXYsWPHkM5JO7mI8CECMP05ARnI2YaDgRFjnpQAHA4HrrnmGpw5c4ZPUYfDwQUyGiVx6NAhyGQyuFyuhNA3wKUZITTpOxQKwe12c5+SNoEr4ZhSqRRr165FKBTCpk2buChCsLv+IS21WghsQgUjsfVD4WIkEuG+J313h8OBvXv3juic1MaivL2jowMWi4VnZyYzlmLevHl8HbGtKNYu6B/6DmVlZVAoFAmt51ErIXzVJjaApVJp0nmVw+FgYnJpaSkMBgOCwSBcLhcsFgvy8vIwa9Ys7pN9/PHHcDqdXHwZysjJSTQZGCgoLfb6CAAxlLW1tTETIjMzE4FAAK+88gr279+PzMxMVFRUIBbrU21LTU3FW2+9hU2bNrFSHsHHknkuYhgcDAZRUlIS9/2uhPX29mLfvn0IhUK45ppr8O1vfxvXX389j2oXjSqZJG2Sn5/PIbfIbKJ+OuX8VquV4XCJDukV+56xWAwPPvggO4rYs03EMjMzmV4mgumBS4AccXOWy+VDKjYMZl+bk5NMo9FAr9ejs7MTKpWKd8hEAN6iWa1WZjKEQiEW4jKbzfD5fHA6ndi5cye3arq6umAymeImXQ9mBEInnV7g0otITU0dELpqNBpuDQxmPT09cLlcMBgMzBrp6upCIBDAtm3b4PP58Je//AWBQAC1tbWIxfqmflP/sKWlJWFygcFgQH5+Pk6dOsWhW1tbG6666irs3r17VKJsw9kXX3yBAwcOQKFQYMmSJSwLOljYGIv18TeLi4sRDAbR0NDA4afonJR7Ushos9lYlS8RSFxHRwefvhKJhFUdKUJLhqhhs9n4vkRNYCpeUiFTxDEnysEFLgMhJDJQrqQRrUYulyM7OxvNzc3o7u5mHZhkrkPTuXbu3In8/Hxcf/31qKqqgs/ng91uh8vlwrFjx1BeXo7Fixdj+/btKCkpGfaz6LuLTBkKvcLhcJw2D41IBPrCtaFYGdFolItQs2fPhlwuR2trK9xuN7M1bDYbN8bVajVOnjwJt9s9ZBgmggJogWdlZUEmk6Gqqoo3i/r6emRnZ3OBRtxEEsUxD2VGoxF33nknZsyYgdOnT+O1117jQo5oBAHs7u7GmTNn0NjYCL1eH5ceiFxhQuFUV1fHUbuMRuOI4nJNTU2s7yOCJDIzMxmcnuiaFpkwg+HM6b7E640kHBb3XBL6qb+bwWBgIaaSkhI0NjZyYcHtdg+7WEYyOvZJ1l6j0eD06dPQ6XQcRhIkaqRFQ6eY3W6HyWRCbm4uCgoK8Oabb2L+/Pks55ifn8+5F81lHEmXh3ZAsUhBC0ikIpGmDjlwf/5gf4tEInA6nawIGAqFWESKwjAALNtIBa6hrKioCLm5udw3bWhowC9/+Us0Nzcz95SYFx0dHSgsLIRE0keSp889cOBAwgLe/U0ul2PdunVoamrCQw89xENmBzN6blKplLVt7XY7S7dQpEIneyQSYWnSSCSCsrIyhEKhYYWlyUTAALW5XC4XqyJQGpHI9yP+qfhe+0M2ycRNkjaBkWzY81VcpHK5HCtXrsQ3v/lNpKSkoKOjA1lZWVi0aBFuuukmrF69GnPnzmUd0ESuKRoBwvP/ju43Go3ciojF+jiX2dnZCT04ygvr6urg8/kwf/58+Hw+lJaWwmKxcJWTJCZSU1NRWFjIYk0jce3ECixwaYqVSI4mKho1uBOxUCjErAuRKiZqJwUCAWi12hGfg1qtRmZmJp+0ZrOZcyqVSgWLxYKCggIUFxeju7sbU6ZMQV5eHrKysmA0GmG32xMuNIlG71ehUOD06dP405/+hI6ODkgkEuTk5MBqtQ74HaIDpqWlobe3Fw0NDTy/hnJReqaU66empqKpqQk+n4//LJFwkSr34XCY8d8dHR08MoGGNY9kdrud35GIKRfx1SJxgbDnvb29cTn+cDbsyTlmzBjWhTGbzVi+fDmeeOIJnD9/Hr29fbPtMzIysHTpUu5NLVq0CPv37x9SOmO4RaVWq2Gz2eDxeHj31uv1XEBIT09PqJdFISX17lwuFyKRCC5cuMBy/p2dnTh79iznpSTpSA472CknPmBRYpGqsnSKUtGBANUUpo9kJAcJgPt5YrWSTmEKn4cz8QSmTaShoQENDQ1MEYtEIggEApBK+6RCicwciUSSyo36G+VyXV1dmDZtGuRyOcxmM0pLS7Fz5060tbXFRT/0jHp7+/ScxNF89Ozo3+KJ1dnZySp8gUAgoY07EAigqakJdrs9ji8bjUbhcrnQ3Nyc0HccM2YMO6VYfwAw6Ikpl8s5t83JyUnoM4ZdMbfffjvy8/NRWVnJY7mnTZuG2bNnA+hbQDQpuqSkBCkpKbDb7ZgyZQpOnjwJoI/aQyFJb28vvvjiC5w6dWrQz8vMzOQhudRsFycFJxK2AGB1b5rafPHiRXg8HlYPIKjVqVOn0NraCrlczvoyWq0WGRkZcbIrZOKpT8glEjjr3/KRSPqGGIkVvJGMTnwypVIZV2CiBR0OhwctPolG1Ds6cbq6uuB2u5GXl4cLFy5w+0Ymk3EeptPpeDARYaZHazRa4o477uAWCIXI/Z2I3itpIVNqQC0HkRtKGyH1OUkcm6RtErETJ04gOzubawckUVJVVZVwVyAnJ4fvwev1stoevW+x/UPiZ0qlkqvyidiwzvmrX/0Kq1atQmFhIffXrrnmGtZ0ARDXbO/p6eHTacqUKXys08smOYzBTCKRICMjI+7PSE2BTh2j0ZjQCECSHSkoKOCFQDqkmZmZsFgsqK+vR15eHs8WJSFg2gwGcyj6roQOIigdgLhTjnJQmu+RlpaWsAo+vUwRmE0mlUr5VDEYDEP2YoG+FgmFahJJH8WvsbER8+bNw6lTp1j3h2RBqXHu9/vjcrtETHSaWbNmQalU8lgFUgFoamrCjh07WBxMNK1Wy+ANUr9TKpWMvur/8/TMZTIZ92pp4FUiFgwGOTKh70nAkkSLYIQ0C4fDOHjwIObMmcPzYYB4GqFSqWRA/5gxYxLuPAzrnD6fD6+99hokEglMJhPPfugfolFRhWJ4quKKchWJ7MIZGRm8IChcoNwuGo3CYDBArVaPOAAmGu2betbU1ITS0lJMmDCBCdB2ux2VlZUIBoMYO3YsfD4fcnNzIZVKUVNTgwsXLgzpSJTXEHmaxIlFOhftwrSAnE7ngLBnKKNcmMS5+xPLqVorViiHKjKRLisBL2g2jNVqxbhx41gTmE5+el/0nugehjM6IaiIVF5ejpUrV/KmFAwGYbFY0NHRgddff51ZIf2NoHTApbYU/Ru4ROKmaEYMc8PhMMvDJGqUhxMuuaWlJW4aQCJGg5C6u7tx4cIFlJeXc57ZH3QikfSRrysrK5Gbm5uQLCaQYLWW+JvJNLtHYyTyLMoKUnO5t7eXe6AjOScVEJxOJwoKCuD1etHQ0ICuri7ukbW1tUGhUDCrwWq1wmAwoL29fUixZuJ2El5WFOESCwCirEhXVxcr/41kVGYnpxDFpenaJLZNoyOGCm1piBDlaHQ6yOVy2Gw2HpYkfk/aROkdDNc3pFB+0aJFePDBB/Hyyy+jqakJXq+XNySz2Yy2tjZs2rSJtYEGew7Uu6RTXpwJIwpxiYue8n/aDEUF/pGMvv/58+fR1taG48ePQyaTwWq1JjR7BgBvfJSCDfd7tHnX1NRg0aJFrMw3kn1tQAgSiSTOOal6R4soHA5Dp9MN0JUdzAjUnpWVhba2NjidTgQCAXR0dMDn83Gl2e12s5zEhQsXUFtbi0WLFsXRx0Qj59RoNNzaoFCWCkL0c1RlDYVCMBqNCfWESSWdWjD0XGiBkqPSGIHhpEftdjtycnKYESOXy1FQUIDU1FTYbDZkZmbGzfSg+yYABMl8DvaeMjIysHz5cpSVlaGlpQWbN2/GqlWrAAAejwdms5kFu3t7+0Yrfutb30J3dzcuXryIffv2xUUSNA+HlAdtNhuH1iL8kp4pPfuMjAxWrqB7T8RIE9fj8XBhsLe3F3a7PeETmDaupqYm1iEWI0qxpUL5Pb2/RDeAr41zEv2Kci4A3IqgXhJN3RrJFAoFMjMz4XA4UFtby/kbFYrC4TCamppQUlKCjo4O1oPJz8+HVqtlMer+JoZX4nQrESEkCoSJUL9ETs7+Vdn+n01Ah0RkQhUKBcaPHw+Xy4VYLMbSjTRinUYaUHWZwmSS5wwGgzh69GjcNalnTEU1GodoNBrR0NCA9PR06HQ6pKWl8XPMysrC3LlzOUffvn07Dhw4EHdd8bvGYjGuBdCCF0NNWh9E8QLiZ8EkYmJdQAzlk6EliuG3UqkcIAsqflYkEkF6ejqPjPh/jjJGfSNRFU6cXEaFGIvFMujiFS0UCuHo0aMIhULIyMhg9flIJAKHw8EzMAmzm52djfb2dqSnp6OyshLNzc2DOhM5I+WW/aFl1OKg3lZ/tslwRpsHsV3E70hVQQIk0M8PZ8FgEE1NTWhvb+ef9fv9aG5uZhQMkQuoYEXjDSid6L/D2+123HvvvVAoFFAqlfD5fJgwYQI0Gg33IAlpI/aBCcealpaGpqamQfGrdIKTPjBVpAfrX1Konp6ezs8/0dMIAI+W93q93Fum55Oog7e1tTFx32QysXNSD5zeO61dypMBJFwR/to4JzWnSa2MSukieFgmk3FjfbiHSACDcDiMM2fOYOzYsTxmnLRwZ86cic8//xyNjY3o7Ozkh0jiUYOdSiINSFTFAxD3MmhnJxmXRFopxNig+xAXpNjSIQelKVlDmYjWIjB/amoq/H4/QwkpKlEoFJznEZ61paVlQI2hpaUFRqMRWq0WBoMBWq2WHZOciCq/JGwlthRIUW+wFpVEIuHTMi8vj6vGFM6Lz5v+jEAp/YtwIxnJg7jd7rhCpsvlSlgitLq6Gr29vXC5XOx4dI/URxZz5lAohJaWFkgkkoTpj18b5zSbzZxL0cumwaSkjJaWlgabzcaTm4Yyv9+PmpoaVjj48Y9/jGnTpsHv93Pv8+jRo8jOzmYmRDQaxfPPP4/8/Hy4XK5BnZ9yYavVisbGRh6ySuEXTekS1eGUSiVPNh7OqOcmOhQ5NRVMxD8jh6PNor+1tLQwb5VkI6nfSCcj5WpiO4gW1okTJwZA99ra2lBZWYmysrK4eZMiIJ1OX7H/Sz1fArf3N6VSGafKf/LkSX4OItBDfA8iGosQQomGpQSdo88gB3e73QmHnGL+S4OpKOog7q64JkilnyKvROxr45wHDhxAd3c3cnNzYbVaEQgE4HQ64fV6uULrdrvx+eefjxgWNDc3Y8OGDQD6XuT69esxa9YsWK1WWCwWqFQqFvl1Op344osvUF1djU8++WTY61Kb6MyZM6wbSyc6Oa7Y96W+WX19/YhKBbRAgEu5D1VXqX9HC5R4hxQCD2VvvvkmmpqaMGbMGNhstrgmeVdXF/edKSIgVsauXbsG5IVkv/jFL1hqpKioCDk5OdDr9fx9Sf6Ehj+FQiF0d3ejpaUFLpcLn3322YBrejwenD17FufPn0dnZydef/31YZ8VcOm0VSqVTGbweDwj/h7QB0L48ssvGW3W1dWFPXv2oKWlJWFFiZqaGuzduxeHDx/G3/72NzidTsyaNQs5OTlMEySALEt3AAAgAElEQVTlvYsXL+L3v/89wuEw3n//fWzbti2hz7gsgS+9Xo/7778fWVlZkEql2LdvHzZv3jxgoE2iRj1EcVcPhUJcUKGZjKNhwhAhmtoVtOuR8PFoGRi0SETeIZ2e4s8kcs8iVpd+z263Y968eXj//ff5GmKIR43woU5muVzOfchJkybhzjvvREtLCyorK1FeXo6Ghga8++67qK+vZ4C91+sd8X5JF5hyU7GNRPdH/y/2uwd7zrQxUOgrk8mQlZWF/Px8HD9+HF1dXbyBGAwG3HDDDdi3bx8qKiq4ek2AhMGuTxuoTqfDhAkTMGvWLMybNw9vv/02ampqYLfbcfvtt+PkyZN47bXXUFdXxxvxcN/faDTy8CmVSgWHw4G1a9eira0NKSkpmDRpErZv346DBw+irq6Oo8H+uW3sSqjv0a6rVCphs9lQXl6O++67D3V1ddDr9cjIyMAzzzyDqqoq5hpS7P1/lrwpFAr88pe/xF133RXXSxOfZygUwuOPP47f/e53A36fnHv+/Pmsaq9UKjF+/HiYTCY4nU6cPHkScrkcHR0d2LdvHz777LNBaV3/SLv33nvxyCOP8EleVVWFyspKzJgxg6ueAPDUU0/htddeG/ZaMpkMkydPxtq1azFhwgTIZDK0tbVBr9ejoKCAhxDV1dXB4/EgPT0dfr8f77//Pj744IMRe/sSiQRWqxXr16/H1KlTkZ6ezo6XmprKSKx///d/x5tvvjmoUuAVcU6DwYCZM2di+vTpmDt3LrKzsxGJRHg6V25uLiuZnT9/Hl988QW2bNmC2traYb/g/9ngNnfuXKxfvx5ZWVnQ6XQDCh7UM3vllVfwne98Z8Dvq1QqPPbYY3EkcvodlUrFhQmKJJRKJVpbW/Gb3/xmWGjgV2kSiQR/+ctfMGXKFKaBUauCmCSUbx8+fBi33XbbsNcbP3487rvvPpaDodDe5/PxiU25PFXwCXp46NAhvPTSSyMWiR566CE89thjXAchcgS18KRSKdrb23Hffffh1KlTA07koZwzYdpBSkoKVq5ciUceeQQLFy7kIa4pKSk8io6S6o6ODphMJqxduxaPPPJIoh/xf9bPMjMzuacohosU9kqlUi5QDdZKKCoqgtVq5cIOVWap70nTtohF4vP5kJmZienTp/9Dv2d/MxqNfPoQYN/r9fLipyJYcXHxsJVwCi0JzSOSyTMzM5GRkcGQRhGXS4rsZWVlKCgoGPF+1Wo11x8ohSHHJ8QXyXkmkz4lXBDSaDS4+eabuecG9IVXNTU1cZA64i8SOmbWrFmwWCxwuVwJ39Q/wgwGA2655RZkZWVhw4YNcLvdcfqy1A74ZxoN0xH5omI+S4ThMWPGoLCwEGfOnIn7/cLCQsjlclgsFh7GJLanSPmdqoqk7jdhwoQRi2NflUkkl/SEaNJ4fweMxfqkLEcqAJnNZkyYMIEBE+Q8aWlp2LVrF44ePYpotG+M/TXXXMPyOABYl7i8vBwVFRVDfobIm+0/QJiADsSwSlbNI2HnJOyhz+eL83764mIPkP6cQAUFBQWX7ZwkyiWTyXD+/PmERxAMZiqVCuvXr8eqVauQlpaGxx57DPv27cOJEyfg8XgglUpRXV2NjRs3XtY9X46lpqYiJycHJpMpTpBKdFYqnhQVFWHp0qWorKyMKzQQsEOtVqOlpYVHqwOXsKnijl9UVIS6urqE+YZfhWm1Wmi12jhh5v7AC6KzpaWlQaVSDSnKZTAYkJuby0wj6p1u3rwZ0WgUt912G3Q6Hc6dO4d3330XS5Ys4Wlo1L+dNGnSsPdLukznzp1DbW0t1q5dy2uIqu4bN27EnXfeyZttoqdnws5Juy3F5KRsR5VKCrGox0cPTSKRJL1j9LexY8fiqaeegs1m44T+3LlzqKmpwdmzZ/H5558n9IUp5CgvL0d5eTnLVUSjUcyaNQtz5szhBv/rr7+etHMSJnSojUiE9Y1ker0ehYWFLHBGNhiNLBaL4Y477sBf//pX1NXV8d9ptVoWvqLvRSdvLBbjKKenp4f7lsReIWHrRGy4arT4d3Svw31/UsCgKWpiKEonkd/vZ5AHqbcPZmq1mgH0Go0GoVAIBw8exKlTp/Dd734XOTk56OzsxJw5cyCTybB//37ceuutCIfDnA4Q4GIosoXRaITVakVNTQ1Onz6NtWvXxr1nuVyOXbt24bbbbmO+8hUHIVCFLBQKwWQycQ+LnBIAl8sNBgMviFgsljAKfzDLzs7GD3/4Q0yfPp1V4CdOnIiJEyeiq6sLHo8Ha9asSaiqRg33efPmMegBuCQpIX6PZLRgASA3Nxe33XYblEol3n77bSaUkzOJaBdqMwxnGRkZPC6O7rs/FYk2x56eHhQVFaGsrCxu8A6RBkjyg6IbcVMlh6SCC1WFs7OzE1Kd6P986d8LFizAkSNHBhSWJk6cCI/Hg8bGxkGvk56ezm0uUkLoPyaeCmMKhQI2m23Qa8lkMkyYMIEhoDTHkxBKDQ0NOHToEK9Rkicl9Q2DwYDW1lYoFArk5+fzvNj+plKpoNFocODAAVy8eDGOoUSRjsfjQTAYRH5+PoxG45V3TovFAuCSeBZxAalQQY14SoTpRBqtc0qlUkyaNAmPPvooZs+eDb/fzyEaLe6enh6W+9+9e3dC1509ezbvbv0pSEBfyES0qkRMpVJh5syZWLZsGfcUv/Od72DHjh349NNPmZCdlpaGkpISTJ8+HS6XC2+++eaQL0kqleK6666D3W7n04vuVewViqEtOcTmzZvZOZVKJVJSUnjCtwg4IPU5YukXFhbCYrHwyWuz2RJ2Tvo8uVyOjIwM5OTkYM2aNbj//vvx8ssv4+zZs+jq6kJWVhZyc3MBYMCoRbLJkyfzcGECYdAaI4obOS8NfxpsolssFkNjYyPOnj2LgoICHDhwACkpKcjIyIBarca8efPg8/nYwV9//XWGMJLebktLC9rb24dFHimVSsjlcuzZswdz5szh90friRBT77//PhYsWJCULyTsnCQDGIlEcOrUKYwbN46TZ1Hnhv6/oaGBS979FQ5EE0MA2nXz8/OxdOlSrF69GhkZGZwr0GlGC5QW5erVq4d1TvGU+P73v4+MjIw4Uqx4GlEzPCcnh3G+Q90zAKxatQqrVq3Chg0bsHr1akSjUZw9exbl5eWYPHkyPB4PZDIZDAYDwuEwjykfSgIlFovBYDDgoYce4s+mTa7/s6LTmHIzcYwfbWSkcE9Acipc0DujzbSxsREOh4NpeyaTadjnKQLOMzIyoFKpMGnSJCxZsgRGoxE+nw9+vx8PP/wwOjo6WDjt3LlzKC8vx/bt2wd9Z9XV1XjnnXdQV1eHpUuXwmQyxQmuRSIR1NbW4syZM3A4HEO2fHp7e7Fz507s3LkTZrMZcrkcjz32GIqLi3H11Vdj165dcDgcUKvVOHjwILq7uzFr1iyEQiE4nU7cfffdCc3RTEtLg8lkglwux7XXXhsHRIlEItDpdDCbzWhtbeXUIVFLqlpL4dUHH3yAOXPmMJBX/IdQPidOnEBRUREADHBOWlCEOySbPHkybrrpJuTm5iIzM5MhZgS3o5OZRKjoBJg/fz6ysrJ46Gx/o5P2+eefx5w5cxCJRJgx0r93SD+rUqkwYcKEQUfV0c/l5uZi7ty5OHfuHLZv347z58+jpKSEh7lS71Cn06GzsxMHDhzAqVOn4HK5Bpwa4nP42c9+BrVazYLH5AzkjPSzhJzq6OiAx+PB9u3b4/6Oohuz2Rx3ypKwM21uWq2WZTlNJhMPxe3/zujaEokEDocDLpeLmSpFRUXo6enheSUpKSlobm5m0TS73Y6ioiLMmDGDye6DOSdB4nJycrBw4UKG09EJ2tPTA41Gg40bN6KtrS2hEJEI9D09Pejq6sL8+fNx/PhxnDhxAmlpaTAajViwYEGcygTVS0ZCd0mlUgSDQeTm5mLevHno7OyMq64DwE033cTV3+GmCfS3hJ1Tp9PB5/OxHgqx7KnaB1yiNslkMtTW1sLhcDA8rv8XUigUDNzWaDSYMGECvvWtb0GlUnHZmwpA4udQlZFyLaCvFzZ37lxs3LiRH4wY95tMJqxfvx7XXnstN5vFlon4D+U2Op0Od9xxB44cOcJ5mlwuh8lkwty5czF79mwUFBRALpfD7XYjGo2ioqICVVVVaGpqwp133on29nakpqaioqIC7777Lrxe77Ai0HK5HN///vexfPly1qMRoxLKn+gZe71eeL1erpaL16YQzel0oq2tjUnsFBnQwiEGTFpaGuvFSiQSTmPI0tLSkJeXh3HjxmHy5MlYunQpDh48iPz8fASDQRw+fBi9vb0oLCzkPiXJs6Smpsap1UUiEZSWlg651oLBILKzs2G32wHEa/z09PTAYrHAarWiqqoqoeIaPRuXy4X09HSEQiFMnz49jsNKeGVRrWGka0ulUthsNmi1WjzwwAMDiAQSSZ88yaJFiyCXy6FWqxMSCyBL2DlJK5WcjWJ+8UWLpX4alkM9KdEcDgdWr16N/Px8GAwG6PV61mRJTU3lih01cWlRisUV4v4RBpJK4JSPkRxjcXEx7rrrLtx44418YpDzA5cGJ4mNffrzzMxMJoBLpVJMnToVt956Kzo7O9Ha2oq2tjbMnDkzLjKIxWI4f/483nrrLZSVlbFYtN1uh91uZ80aEZ1Dz2316tW45557YDAY+DMpwhCdKRwO47nnnoNMJsO6devg9XoH9GTp51NTU9Ha2srVWnE8AG12vb29PKmM3rP4d2lpabjllluwYsUKxj3X1dVh4sSJzDQxm80s5KbVarna29XVBYfDwRs0XXeoghs5RP+IRnw/xGJJBmIoMmXIaYgqKFaEKcxPxORyObKysiCXyzFr1ixGHYmbPEURVDkfSRM57vqJ/iDleR0dHcw4EB+kyHME+hJlwhESmJ3aAt3d3Th06BDq6+uRm5uLrKwsZGZmwmw2Iy0tDUqlEhqNBgqFgnd78TSMRqMIBoMIh8Os9ep0OjF37lwsXLiQq3P5+fkoKCiAzWZjREx/pQLg0qkl5nVSqRQ6nQ4LFy5ERkYGOjo6kJGRAafTyadZd3c3PvnkE0yfPh3jx49HTk4OzGYz1Go1V/woBKRnE4lEcPToUWzduhUKhQLXXXcdCgsLYbPZUFBQgK6uLtbNpXuhFohcLkdzczMef/xxvPfee7j//vtZdlSczAWAcxuC5NH3FGlS9DOdnZ0siuZwOFhxQqlUIhgMcjFLrVbzxkFpjlKpRF5eHgoKChgpQw5AqUdzczMzaILBILq7u7F///5B1xm9Z5FoLxbA6N2QqoD49yOZWC0XHVCMRgAkLsAll8NqtbIzitVw8TPpedN6TdQSdk6a30iKZWIzVWQgUGPbYDCgubkZRUVFXG4m55RIJGhtbcWZM2cQiUS4+kvVOeJ0ktNQJVj8PDo1acFHIhE8/fTTmDdvHp841OahXZuU6ADwyTxUi6KzsxMejwfNzc244447oNPp4PV60dzcjMbGRiYtd3d34/jx41i3bh38fj9jjO12O3Q6HUcaoVCINVhbWlr4pHzyyScRiUTi+nWiZhCZUqnE+fPnsW7dOuzbtw9AH29V1HkVjfIni8WCc+fOcfhKOSc9DwrD6L9F56J+tt/vx9atW7Fnzx50d3fDbrfDarVCo9Hw86Tnde7cObjdbr6WSDqmZ9vb28vaSv05o7Q+LBYLo5co4gEubaSFhYWsHZyISSQSPq3FXittyiLBvX8veSij6eudnZ0DnJ0Ogf5V9mSACAk7J31YdXU1/H4/dDoda9cS2ZiS4K6uLhiNRtTW1mL+/PnMhqDrNDU1QafTwWg0cg+OWiPkNJRnAvHVUdrtyZl7e/smPNM07FAoFAdRIzrRYA4bCAT4d0noi3iJkUgEzc3NaGhowN13341Vq1Zh9erVmD17NhdiAoEAb1Qi9cxgMKCpqQkXL17klgwtShLQAoCrrrqKZ34QeEM8JeieY7E+8aqf//zn7JgSiQT19fX80sXQnJ5TOByGSqViGQ7a3ekkpiowyZU4nU74/X4YjUZ0dnZyCyEajeLw4cNYt24dxowZA6fTiaamJtTV1fEGZzKZEIlEeDPu7e2F2WxmZ+vvZHq9HuvXrx90FgtBDul70jMQN6xx48ZBqVQmBJQgB6EITHR0chzx9BRzxuFCZ3pmtKHRdyUwPaUrJAAeDAbR1taWcDiesHOSsBSFk6KeDXDp9KQvrtVqcfToUdx2222s+SJ+qfb2dta3ofK+GBKIsT+FLnR9Cs8I9BCJRKDVanHkyBGEQiHk5uZCr9dDp9Px4qEiUDAYhNfrxYULF1BTU4OWlhaUlZVh8uTJnB/rdDomDGu1Wrjdbrz99tt4++23AfSpt1EYrlKp0NHRgUgkgoaGBpw/fz7u4VutVj5lgL7KIclxiiGWCDYQe3uRSARKpRK7d+/Gxx9/HPdOCEpJm4P4PuhEILUCijzo1KICBXFapVIpj28QW09k3d3deOGFF6DX6+FwOHjGDPETSeaDtHdpjVDuTIUzCm3b29sHra5TjmYymXix0/unjVUqlSallCdeu79j9C/+DFYjGcr8fj+eeOIJ7N69G88++yw/dzHNo83697//PTZv3oxTp059Nc5J4Sft0pRf0kMUd2+VSoXKykro9Xp4PJ4hk2xaLImGJ0OZ1+vFG2+8AYvFwk33lJQUGI1Gzo+CwSCCwSA6Ojrgcrl4horT6URFRQXn0uT49fX1g2J4aeZIItba2jroNch5AMRNWqZTQsxjYrEYtmzZwgOB6BShKqxUKuXREGR0IlDoTUUwOhVEYWYArMNDIbKYAojW0dExJFrmShhtVunp6XGhsNhWoiJVouEnGX23/s4hVsUHa68NZ263mzdbIvLTfYlFR41Gg4MHDyZ1vwk7J4V6nZ2dca0I4FJY29NzacgOISdIWyaZLzxao1F3ouPQxkH3OdiutWPHDuzcuRNAfD4yWnWERI2qhWLUQQ4qsn9qamrw5ptvDrh/OrFE8gEZCYvR96cqLIVaYnWXwk5xLs1I8MKv0uRyOQwGA98bRVGis1JEkIyJGxNZfyCK+P4TteLi4rjqM20iwCVVRkJGJWMJfzuajdne3o5QKMTIDAq/RD0bmnkYi8VQUVHBUvf/DBMlMoZ66JTv0olJJ8hXadFoFDt37kRFRQWTe8Xqt+hYzz33XJyiBL14krugeSdiJZCGFbW2tuLw4cOwWq3IyMjgf9tsNpjNZpjNZhiNRqSnpyM1NRVHjx7lVsM/i+ZHMEjajCQSCUcG5JDJTkGjDah/4UxMmcSTORmzWq1xLTGxbw5cwpxT/SNRS0rgi4aVRqNRNDU1sRizKG5F2NSGhgZ0dnaisbGRy8j/Z5csFovho48+Qk1NDaZMmYLi4mLMmDEDM2fO5Jynra0Nf/jDH7B58+YBvwuASQaRSCRuGhfQt0H6/X6cO3cOZ86cwYoVK+Dz+bjYRo15cYR7d3c3Tp48ibKyMm5R/TOMTkoCq9DmKUZAGo0GGRkZCacXBGekwg3VOehgodbMaNapSqXiIiT5g3gSExzU4XDwBPNELGGZEo1Gw9SZQCDAk5NJbYx2OK/Xi9raWpw8eRIejwcWiwUymYxP3OFM3An773DiTpRs2PFVGuXZYuOZ8vFk7pPEiU0mExc6wuEwD1Ya6ruXlpayfEdLS0tcOEo7Oo0+pNOjf72A/t/n8/HUNWL8JPMcqOhEYeJoyerZ2dn47W9/y+ru1POmPBroy/tfeOEFfPHFFwldU6PR4N5770V+fj4AxJ3KlItTvu/3+/GjH/1oxGotmc1mw9133w2FQoGKigrk5ORgxowZqK+vR1NTEwoKCvDll19i06ZNce+SLHYlNIREM5vNyMnJYcQIAJ6t2dzcDKfTOeKXEk2v12PdunWcW5DWK53ONCSHRsRXVlYmdf0raQTbslqtWLJkCebNm4fTp0/jxIkTyMrKwpQpU7Br1y6cPHlyyKLScJaSkgKHw8HaQRQueTweuN1utLe3o62tLeFdXiKRID09HQ6HA4sWLYLZbEZlZSU6OzsxceJEWK1WHDp0CDt37kR9fX3CPE4APAV7zZo1WLBgAaqrq6FWq+FwOPDzn/8cFRUVaGpqSvpEEtsZCoUCU6dOxcSJE/HHP/4xTskvmQ1QKpXC4XDge9/7HioqKvDpp58y0m3s2LFYtmwZXn31VRw6dCjpnFsm6xt/eMcdd0AqleL48eMoKytDVVUV3njjjTgqX3+7os4pk8nwP//zP1ixYgXLLhIY3eVy4dNPP8W///u/xxF/R7J169bhnnvugclk4jHglENR5dVsNsPtduPs2bO46667Er72lbZly5bh0Ucfxbx585i+1NPTN5t0zJgx3DoAgC1btuCZZ55JqsJZUlKCf/mXf+EeMJX3CcRRU1ODHTt2JCScZjAYUFZWhuuuuw4zZszgeZlUqDCZTPB6vTAYDHC73diyZQv++te/4sKFCyNe+9prr8UNN9yAhQsXco5IFU9qb505cwZbt27Fn//851GpV8jlcmRnZ+PGG2+E3+/Hrl27cP78eQCji6BKSkrw0ksvcXuro6OD20ESiQSPPPIIFweTsZSUFPzlL39BdnY2X6unpwc6nQ4bN27Er3/966Sdc1Si0vn5+UyPoSohhXUymQxz587FqlWr8OKLLyZ0veLiYtx4440Ih8Pw+/1obW2NyzGo8tjd3Q2z2QyHw4Hy8vJBBYq/alMoFHjmmWcwbtw4lg8B+jYsh8MB4BLUUSqVYsWKFTAajXjggQcSzjdo0G1vby9KS0vR2NiItrY2tLS0cFk+kWJISkoKli9fzqwIijxoNqder0dDQwNXi2UyGdasWYPi4mI88MADw167qKgIjz/+OE/sopGI1LOl6r7dbse6detQVFSEdevWJfT9RYtGo0hPT2e00fLly/HSSy/x3yVrFRUV8Pl8KC4uRiQSgV6vR09PD1eHkzlQRIvFYmhra2OkF4XgMpmMQSDJWnK16L/bpEmTWJOTWicEoaNpykajMWF5fLlcjosXL6KiooJZB3RqEGJIp9OhpaUF77//Pk6fPp2wUsGVtrKyMowdO5bzKTH8EtkjIkpp9uzZKCwsTOj6MpmMNz+bzYYDBw7gk08+QVVVFQNB5HJ5QuPu1Go1Vq1aFTc2TyLpm3/pcrnQ29vL6BURIjl9+nRce+21Q16XlBgzMjL4lAQuER/ElgcxPhYsWIAVK1YMqCUMZ/RdZ82axeCNa665Jg4OOBojMgMROOhEo4HIo7FYrG++jMFgYNANdSmOHTs2qlN+VM45depUBAIBpKWlwe/384Agr9fL+WJWVlbCagKnT5/GN7/5Tdx777341a9+xax9Cg8J1vfxxx/jRz/6Ee6//37s2LFjNLd+2TZ16tQhNx1yTq/Xi3379nF7RqFQ8PTs4UylUmHcuHEIBoP44IMP0NDQAIlEgjFjxsButyMjIwMKhQJqtRpms3lE9oRKpWKiMZ2OIlWOhjqJz5qqjfPnzx/0mhKJBNdeey0efPDBIed3is1+KrooFAo88sgjyMvLG/ae+1tKSgoKCwvR2NiIY8eOobCwEIWFhUn3OEUj9hMJhNHUOSIAjMai0Sj27t0bVxFOTU1Fc3PzqIdOjyqsffHFF3Hw4EHccsstKC8v55NTo9Hgk08+wd69e3Hy5Em0tbUlfe2WlhYOv8RTqbu7e1TXu9JGoyeGM6K9AZdUC8aNGxd3gvU3UtHLycmJI2OLiKCOjg6eIUPh/XCcxqKiInR3dyMtLY030JSUlDg1OjpJKSWhzXDMmDGDCltRNEADpmiyMy1qOtFEeCIhy0REUiImwkGPHj3K9zt58mRUV1cnXE3tbx0dHRyZ0T1TBHE54AuPx8PrlpBOlZWVI05iH8pGtf243W5s3boVe/fu5dia6EPNzc344x//iEOHDiVV9RONYnV6kaSxSvjUyzWpVAqz2Yy5c+cmFWYB4GIKmbg4aGGmpaWhuLg4TsUgPz9/2IUpk12aPUowsHA4jM7OTvj9fng8HrS2tjJnUCbrmycyXFN7/vz5fD2qC4g8WQrpRGQS9f4MBsOggsq9vb3Yu3cv9uzZw8AS6kOKzkQ/Sw6pUqnw+uuvJyUaFovFmAtKeZvT6URxcXHSrSrRKK+kIibVCOrq6hKejj2Y0SAnMb1paWkZ9TVHdXKKSCDaQcPhMDwez2WDDQwGAxQKRdwELwrL8vPzR71bAn3KbsXFxZgzZw4aGxtxxx13YOPGjXjnnXeYKSKXy5Gbmwu5XI7z588P6NXRGHTg0qnY38hJRcchzd2hLDMzk6d1kfIbhfOpqakIBALIyspCUVERPvjgA+j1eoZFDpUnjRs3jh1dr9czN1OpVPK4RQq/6FqkikCAhcHM4/Fgz549uOGGG6BUKhEIBAb0qEWnT0tLQ0VFxYhzTciys7MZ+nnzzTdzrh2LxVBdXY2ioiJWT6eBwMlYdXU1gw/oO5Ok6WhzTuDSUF8xvxfho0lfbzS/RDlEdnY29Ho9lEolI4HKysrimtzJXlelUsXxOWlBk1iSSqVKetKYUqnE5MmTsXLlSowZM4ZJ3J2dnVi7di0A4I033uDPIKii2WxGS0tL3LUIvSPKWSRiNptt2HCY5s4Q26O1tZVbNJQbUQSh1WoZipaWljYkuOOJJ57A7bffjuXLl8fhhSn37H+SR6NRaLVaNDc34z/+4z+GbfC/++67+Pa3vw2Hw8GzN2lToutTWCuVSvHLX/4yoSKORCLBT37yE6Snp8PpdKK0tBTHjh3DNddcwwD+q6++Ghs2bMCePXtQUVGBV155ZcTriibO86TnGIvFWDZnNCaRSJCfnw+tVsvRTSQSwZgxY6DVapMCdJCN+uQ0mUyYOXMmurq6uHTc09OD7OxsmEwm7mkluoDJMXNzc5k2RFzD2N+Ju1qtFunp6W7FdKYAACAASURBVKivrx/xWvTZarUaN998M6ZNm8Y51NixY2GxWJhJcNNNNyEnJwe1tbWIxWI4dOgQioqKYDAY8MYbb8Rdm1jyI6GV+i/+9PT0YV+80+mEVquF2WyGXq/HxYsXmd9J1yJit91uh16vh8/ni+PV9rfq6mr8+te/Rk9PD+bNm8fjJogJREgkciKZTIZPP/0UL7zwwqByk6L5/X6sX78ezz33XBzNbzDEzfHjxxOWLgX6csKsrCw+MS0WC1auXAmz2YympiZmMVExMlm76qqruFUnMnEmTZrEzpWsyWQyLF26NG6jjEQiGDt2LHQ63T/OOYG+RUqKa2RU9SsrK8PWrVuTup5MJoNKpWIhKOASY4DMYrHAaDSO2CAnp05LS8PKlStRXl4OnU6H1NRUTJs2DTKZDFVVVWhsbGRy9owZM7Bs2TLEYjHcddddrCD4/vvvc8hL8h0iAF10zv5hrggPozxvKCOkFanm0efQ6HYRjK9Wq3lU3Uhqbh6PB8899xyysrKQkZHBIAFRW5g2wrq6Ojz33HMJAyY+/PBD3HbbbSgtLY1D65CjUx775z//OeGKZSwWw7lz5zBlyhRoNBqkpqYiNzcXDQ0NCIVCsNvtHO4T5zMZk8lkmDFjBrq7u+PC9lAohNLSUpjN5iFVHIczqVSKCRMmIBQK8XWj0SiMRmNSukGijdo5RdaAuEBlMhmWLFmStHNSxYwwoCLlhv6t1WqH1XfR6/XIzMxk+BhNGqZilUqlwpkzZ9Dd3Q2VSoXi4mJIpX0yjT6fD21tbbDZbMjNzYVCoUB9fX0coZdmQ4pOKPb0xHsWCxZiYWQoo7D14sWLPFSIIhKR25mWlsajCiisHcloIYp8WxqRIUrL2O12bsYnYtFolAfFkuAaGf0ZpQ/JpAAejwexWAx5eXnYtWsXcnNzkZ2djdraWowdOxZbt25FWVkZlErlgLRjJJsyZQoMBkMc5Uwk+69YsQInT55Muq7Rv+pNlpaW9o/POUtLSxkHK7JSotHosILEQ1l/BnzcTf69cEG56GCm0WiwbNky7kNSXkwqa16vF0qlkk9oKjikpqbCYrEgHA4znlXkN4q7a6Kao/3Z+/T/Yrjf3wi40NrayiE+NcuJ/U9yGMTjBDDirkyfLW4y5DhdXV1xA6doElYiRTdacCLLQxQPo1w4EWHm/lZZWQmFQsF98/T0dPh8PsRifSruVEWmzSoZy8/P5yhFlISh9TJu3LikrieaWCOh6OFyWjOjyn4lEglycnLiwk/gUpFktDsFgLjwj+hNANiZhnrRY8eOxeLFi2G322Gz2aDX61mqUaVSMf6XigBpaWkwGAywWq1IS0tDQUEBxo8fj9TUVEQiEXi9XharIqNijdgiEZ+JaHR6ilHASM4di/UJiwUCAWg0Gq6sUnRC359QOTKZbMTFSbhcQiyJOSFFKyKPsbCwkCu8iRi9DzHvpFOJGB7JDuIlnqqoz6tSqXhqdEdHR9wYimTMYrFwfUSUuiFNXJpskKwRNUyMUOjQGi2aadQnZ0FBQVxVVvxCBIZPlg8YjUYRCASgVqv55YoyiySSNJi1tLSgtrYWGo0Ger0edrsdeXl5rGkzGPm1pqYGTU1NaGtrg9PpRDgcRlNTE1wuF/x+/wBCr0ajiRMFHumh9184wzmnXC5nxTziWgLg3JBEvKl1QYszkZ3ZYrEgGAxCqVTywhEVF+iE7+3txZw5c/CHP/whYXqfVqvlTZpaQXSaE9Ah2cUp0u/Iic6cOYOzZ89i+vTpfM80TzQZEyMhWltiUWi0FovFcPHiRZ7ITdcm0e/R2KidMzMzk29KPB0kEgnDy5JxTmqKezweLlzQS6DiQigUGhLY0NLSgs2bN2PChAkAwAuPwmE6helkcrvdHC4Cl3Kv1NRUjB07FikpKQgEAmhra+NKWzQa5c8n+ZXB+mJiwUh0sv4q6qLROD4qBrW3t/PCBMDIHVFsmuh0Iz1Xi8XCP0fPROR20kYYiURQXFyckHAWOcXTTz+NiRMnwmAwwGAw8LyVI0eOwOv1orW1FefOnRvxev3vmcJM6huGw2EWHifRaq/Xm7RzUvoEXNJxohCUQBijMQK+i4oWEokELpdr1PpYoy4I6fX6OMIq/Vsmk0GpVMJgMIzY8hCNciqfzxcXfomLqLu7mxkqg4Uz1dXVqKmpQWZmJmuqdnd3M5SMQmSqWIr/TS0GAKwK4PV64z7HYDDw3JPDhw9j9uzZg34Xuk5qairOnTvHinXDgd+pYU0avy6XKy7kpKJKOByGTqeDyWRCZ2dnQqe3zWbjU1PU6SV5Ufp/eoc6nS5helddXR3Onz/Pm9t9992HaDSK3/3ud/y9kjXRYWjjyM/Ph1qthslkQlNTEwNFkg1rPR4PF+uItEFhPEVoo7FYLAa32833TE5OiiCjsVGfnDabjcMBMTyi4sNoZAsjkQj37Gghif+dSHEhFouhqalpNF9rRCNF+s7OThw/fhzz5s0bcnFQftvQ0AC/3w+Hw4H8v7PwBzPafMTZptSXJOX8cDjM0QjBxIZDHZFj5ObmxjFlSGWQFiUBBYC+51deXp4wzE7c5GQyGfR6PaqqqvjvR8XG+PviJqnO9PR0HvcXi8WQkZERN1kgGSMUGzkhpRBi1X001tvbi+PHj3N+T0U3GhA1Ghs1tJ/YC2IFlcLHlJSUpKtolPOIOEQRhEDg93+mFhE9dL/fjxMnTvCpTDkshZ3iP06nE7W1tYhGo8jKyhry2pQbp6amoq2tbcB4BfpsUQEfwLDtGVJsoE2BficcDnM/tT/NKxKJYPny5aN6PlLppSG1l2MUdovCzAqFAk1NTVwUkslkDOpPxkR2jojfpu9/OZSx2tpa7sGSuVyuf2zOCYBxmlSSBxC3cJKN3WkHExH8Yh4rlV4SPf5nGQmcnT17lnVU+5fj+1t6ejoOHDiAaDQ6bO+Xwkmfz4e7774bW7ZsiXM82t0p56Qq9HAj8Hp7+wYUFRQUsPo6nXDUThBzNplMBrVaPaomPL0jGrtxOUYbMaVN1Njv7u5GZ2cnj4YcTE1vJKMNlTZDgkRSJNF/LGUyRrUMIkf4fD643e5/bLUWAO+QlBOSGBNhFJMNa4G+IoM4wVqn00GpVDKpGxj9znYlzOVyoaamBp999hl27tzJCgM045HCfApJOzs7eVhsXV0djh8/PuS16fk5nU5IJBLmy4oVZuCSMFVHRweys7OHbcLHYn0Tz9555x3cf//9yMnJ4VaS+DN0IsdiMezduxe//e1vk342lNIQzPByrLe3l9XwrFYrFwMnTZoEp9OJzMxM1lZKBjQBXNrkxC4DIZFkMtllbf51dXX47//+b+Tm5uLixYswGAwJC5ANZqNyzt7eXmzbti0On0hyD3K5HD6fLyENGtFoQG5nZyecTiecTic2btwIpVKJ5cuXw+FwMKD4n2V79+7FggULOCz68MMP4053YGDOQnC0P/3pT8MWGzweD49HuOWWW4YsfFG+bzQaYTKZRsRshkIhZt4sX74cc+fORWZmJp9MKSkpcLvd2LFjBw4cOIBAIJB0UYRkKgHg2LFjTIqQyWRwu90JDbgVraurCzt27IBEIkFjYyP3C1tbW2Gz2ZCamgqdTof29vakF39dXR0OHDjACv+UwxKd7nKcSavVYufOndxipE1ATBuSsVGr710pS0lJwZgxY1BeXo7CwkIeq3by5EmW4ZRK+yZytbW1oaqqCocOHeKBPv9/sgkTJuDf/u3f8OSTT6KiomLQn0lJScGbb76JDRs24N133x3xmiqVCkVFRcjLy0NpaSnmzZvHk6zD4TDsdjs8Hg/ee+89NDc348KFC6iqqhqyRSFuGBKJBFdffTWuu+46LF68GCdPnoRareYZLhaLBZWVldi9ezd27dqV0KlkMBgwb948NDc3o6amZsCgIxpvr9PpcPz48aRVHskmT56Mp59+GidOnEBeXh7eeustbNmyZcB3TNTS0tLwzDPPwOFwMLopFArB4/Hg2WefHSnCuXICX1fKpFIpZs2ahaVLl+LcuXNMO2tsbEQkEsHPf/5zyGQyvPDCCzh37hxLLC5evBhTp07Fq6+++k8TPv4qTKPRYMqUKbjxxhvx7LPPDsAtA8DcuXMxc+ZMfPjhhyNeT6VS4eGHH8bUqVNhMBg4byOZDhIok8vluP3229HV1YX29nZs27YNmzZtGvSadD9jx47FmjVrsHbtWuTk5CAYDMJiscDpdOLChQuYNWsWzGYzpkyZguXLl2P37t3YvHkz9uzZM+w9L168GN3d3cjLyxtUEK2zsxMNDQ2YP38+MjMz4fP5kiYz2+12XH/99bDb7WhpaUEsFsNNN92E5uZmHDx4MK49mKiZTCYUFxfD6/VydJeamori4mKMGTMmaQwwcBnV2riL/L3QQBO6SktLkZ+fj7Fjx+K73/0ufvazn/HPiWY0GrFmzRq4XC7Y7XZUVlZyO8VsNiMQCGDbtm2QSCRoamqCz+eD0WjEmTNnMGXKFNx8882Xher4uhlVwK+//nrO2fsXxaiPmEiYNGnSJCxZsgRKpZIBHL29vayVo1AoOOSkKWU6nQ7XX389rFbrkNctKSnBs88+ixtvvJGrqG63GxaLBXl5eZg9ezZPaqPK89y5c/G9730PM2bMGPae9+7di88//xyBQGDI3FWtVvPs1OEq4ED8wN3MzEzceeed+PGPfwy73Y6TJ0+iqamJ07L7778fTz75JBwOR1JyKkCfuDeleWq1GiqVitspJSUlo1qnV+TkzMzMRE5ODubPn4/FixcjNzeXoWrhcBjbtm2DXq8fgLFcunQpx+OlpaXcI+ro6MCRI0fQ3t7OquVpaWmw2+3QarWMPMnIyEB6evo/babHlTYqJOXk5ODBBx/E5s2b0dzcjHA4jPT0dMydOxclJSUAEpu+/K//+q9cByCdWrHAJDo+jdGgqvG1116LDRs2DLimTCZDWVkZKxFQdZ0WM0E3qTcbDAYhlUrR2dkJtVqN2bNn49SpU0M6Xm5uLs6cOYOOjg5YrdYBtYvU1FTk5eXB7XbD7/ePGNbSd5w9ezZuvfVWdHV1MVjAYDBgzZo12L9/Py5cuID29nZotVo8/PDDOHLkCLZs2ZIwtzMnJ4cr6TTUl4jy6enpCV2jv12WcxJ646GHHsLy5ctRVVUFq9Ua1+vy+XwYP348pk2bhr1798b9/r59+2Cz2WAwGDi3dDqdMBgMyMnJYdQQyUcEAgFMmjQJBQUFyMzMRGVl5WX31L5ORiyOaDSKxx57DMuXL+e2TV5eHqZPn85MoJFQJ0qlEmPGjEEoFGJ2C0H0KFymjZFOOeKORqNRVrjrf0Kr1Wp885vfZIA7cEnCkhA2VFGn0JBaFTqdDtdccw22b98+pGJ/TU0NNBoN7rrrLly8eBGvvvoqg0oUCgXuvfderFixAs8//3xCyBuqRi9fvpzhmwSrPHv2LBwOB44fP84jH4C+KKK8vBxffvklTp06NeJnAJd0lPx+P0M1T506xVHlaPLYywprqaxvtVoRDAZhtVrhcrlw/PhxHDt2DO+99x5aW1uxf/9+HDt2bMCLbmhowMcff8xTpel09Pv9aGlpwZdffomWlhbMmDEDFy9e5D5qOBxGbW0ttm3bdlmKfP/Myu9g5na70dbWxifouHHjsGzZMqxevRrz589ndE8kEhmxF+lwOHgu6WBT1ui0pDSC/pv6tQaDAWazecB1p0+fDqvVyqcCPUNafLQJ0GfRpkpsjfT0dIwbN27IMM/r9UKtVkOj0WD8+PF45plnIJfLodPp8NOf/hQ33XQTFAoFMjMzRySwA33OScguyk3FcYvV1dXcXiHAPYFfCgsL44YSDWdOp5OfR3V1NZqbm2EymRj8PiqkVNK/MYjdd999WLhwIZ5++mm8/fbbaGpqwjvvvIOamhrI5XI89dRTg44XB8ALw+l0MkwrOzsbNpuNpQXffPNNLtW73W6MHz8eSqUyaXl/MYxbsGABfv3rXzPp+Otg7e3taG1tZSCH0+nEvn378Nlnn6GxsZFz0vb29hFnxWRkZMSV8kXAhAi1JIci/DHB/LRabRwDh2zhwoXcsCfwPS1gylv792a7urpgMBh4/s2UKVOGhR2SjIxGo0Fubi7+9re/Ydu2bSgvL2c87MSJExPm106cOJEFnnt6eqDVaiGVSqFUKhlETwOXu7u7+bSdOnUqq0SMZI2NjVAqlTAajTh27Bh27drFpO7q6upROecVyTmpL/bRRx9BKpXCZDIhLS0NN954Ix599FFOlAcT/qLcoaSkBFVVVbDZbAxd6+3txeTJk5Gdnc0iyk6nE16vl0fAJ2sSSZ9WzPe+9z2Ew2Hce++92L17N06cODFq0POVMp/PxwufxveRziyNK7darQiFQiPm2SQjKirCKRQK+P3+OHa+CPZOSUmBSqXiSeSDLf6nnnoKJ06cwG233YbZs2dDqVSiq6uLKX3i2HoAHKYTU+XVV1/F3r17hwWTmEwmFsWinFWsyhM4IRHHSUlJQV5eHoLBIJ9kEomEq8s08JkcMhqNMiQwLy8v4cJQd3c3Y5dpk9JqtQiHw/94VspQptfreUclwjLQ90Xvuusu/OlPf4qbR9HV1QWr1QqHw4HOzk5oNBrehWbOnImOjo64WZ85OTmYNm3aoKLCQ8X19Oepqan4xje+gby8PPzv//4v2tvbsWbNGjz88MM4e/YsXC4XDhw4gNbWVh7+O5ilpqZi5syZsNvtMBqNrB7ucrngdDpx+PDhURWpIpEI2tvbufkulUpZU6m1tRUXL15kpYihIhEyOjV7e3uZIyvOUKU8iNIEmtnZ2dnJCoeDhf2RSARvv/02Dh8+jKuvvho//vGPmSTe/3kRGdpms+HJJ5/Evn37EppFQtVfune73Q4APD4vGAzG8UiHM61WC5PJxJVqml1CJ7BIuqbrBYNBJuQbjcaEQBQUMstkMvj9fgSDQSZrjHZWyhVzTtqRMjIysHbtWrS3t+Ojjz7C+PHj8cQTT8Bms0Gj0eD999+Pe0HEg6uvr0dZWRk+//xzyGQyjBs3Dl9++SW6urrg9/tZgdxkMsHpdMLv9w9wzMGck3CkarUaTz31FAKBAJ5//nk0NjYCAE6dOoWSkhKsXLkSCxcuxHXXXQeZTIaXX34ZH3zwwYDPWLp0KW6++WZoNBq89957+OCDD2CxWPD4448jPz8fx44dw8qVK7Fp0ybs2bMn6XCGTh+v14vc3FxWf3c4HHC5XAMmWI/0PqLRKLxeL4eolLcTw0eUr+zt7UVLSwtyc3M5/xrKLly4gE2bNuG//uu/+HQMh8NxGyEAVnTYvXt3wpjd7u5upKenw+12w2g0coh97NgxSKVSBAIBnvs6kplMJmg0GgSDQb4/kqcRR1MAfc+eUFft7e083jCRAb3t7e38bP1+P9ra2lhvKlmEFFnCziny/YZbcCqVCu+88w70ej0efPBB/OAHP2Bd0w0bNuDEiRMDfodgeZ9//jncbjfy8/N5TsjEiRPR2dmJmpoart6KM0HJRNQK/bfFYkF5eTnWrl2L7u5uHDp0iCdUkXm9Xhw4cAA9PT2YMGECNm7cCKPRiAkTJmDnzp1xLzI/Px/z58/HQw89xAsb6CsG3HrrrZg4cSK+8Y1voLe3F4899hgikQj2798/QE1QVDrobxSCmkwmBninpKSgqakJ2dnZHH4l+s7opKRQmXCqALjyC1yifHV0dCQ0NoGqsJWVlUhPT48j3dOiJ/ZIKBRKCr6WmprKs14p966vr4fJZEJ9fT1UKhVT60Yyk8mEaDQKnU4HnU7HJ5yodi/+W6FQwGAw8KY1ceLEhCB9BOigAV9ut5urw6PFgyd1cophzmAY0lgshszMTIbhNTQ0oKGhAdXV1RyKDvaSKMzo7e1FMBjE6dOnodfrUVtbi1AohAULFuDYsWPIzs7m+ZKDGd1Teno6Fi5ciKlTp3Jf7eWXX8bZs2cHwM/ov2fNmgWfz4cDBw7gwoULyM3NRU9PDw8DIiX27du3c+hOL5B24+bmZjz11FPQ6XRYvHgxDAYDSyNSCEnSIs3NzYO+NKVSyURop9PJoWFPTw98Ph/3fZMxKgpRH06MMshBKSelaquo1DeUUWumP66Y1oJcLh+V1g9V/UVa14YNGzBjxgzMnDkTtbW1vImMZPT9LBYLX48ck+RZxO+pVCpZnKyrqwulpaUJ3bPInunp6YHH4+F3/ZWzUmgRZmVlob29fdCjetasWfjhD3+I9PR0fP7559iwYQOi0SimTp0Kn8+H48ePD3qjVNUiJoLD4WBQQ0VFBdxuN8xmM4d8brd72BNcLpcjLy8PU6ZMYaZLW1vbgFke4ucbjca4Hh2pOBCaJiUlBR0dHYNyAMXFTFKQmzdvhkwmg0ajYbYGgDieYn9TKpXQ6/VMPQIuVTxpJINE0jdawWKxjJjX9t+IotEoF3lisRiTuSnfJPofgRZGMnJyEh0DLqHFaGMLBAJxSneJ2Lhx43jjo81x0aJFcDgcqK+vR21tLcvkjHR/pN6v1WoRDAahUCjgdruh1+s5lBfJ/ACYDBCLxZJSkgwEAjAajVz4TElJSUosrb8N65xiGFZUVISHHnqIp4otXbqU0SESiQTl5eVYv349pFIpzpw5g1gshmnTpuHkyZOw2Wy4ePHikIlxWloaAoEA2tvbYbPZoFarsWvXLixatAg1NTWYNm0asxx6enrQ1dUV17eieyB0ikKhwKeffoq8vDxkZmZi0qRJuOGGG3DgwAEAYC6mWq1mJj2NX8j/uxwGiWi1t7ePWmYCwJCskcGcMzc3FxkZGTyinE484BKlrL6+HpMnT8bcuXMZqD2c0TskZ6HTk9onFLLTz9E7GikUpUiAAAyi2BhtBKSnI/ZDE7nfoqIibtXk/50ofvXVVyMlJQUNDQ346KOPMGPGDOTk5Aybx1J/NBQKQa1Ww+v1xkHsaKYs/SxJ2ZCavs/n40HFicjB9K/KJpIeDGfDPjGaDiWVSvGjH/0IN954I5RKJVdeaUdYsGAB/vM///P/a+/Ko9uqr/SnXbIky5JsWbbkfU9ix85GtiZt9gRC0hZoSg9Qlq4MDEPb4XToclq6TIcUSqfLTNPpHCZ0Cg0UCgmdEpqFJs6+J7YT76tsedG+WNYyf3juzZOxZUmBwpmTew6HQOynp/d+9/e797vf/S6qq6sRi8Xw5ptvwmKxwOfzYWRkBFKpFB0dHTMu8tzcXAwPD6OjowNyuRxutxvz5s3D4cOH4XK50N/fj7GxMZw7dw52ux1lZWXv6heVyWQoLS1FZWUlSkpKIJPJ8Pzzz+PIkSPYv38/w/mkfK7X6yGTyfgkPnXqFF577TUUFhZi8eLFWL16NTZv3pzUDp3IKMSb+s90VlJSgvz8fOTl5bFKgVD1IBQKwWq1QqFQoK6ubtbPpt8TkhCmqisA11UoaKAshbmzmdVqZRRYSEYQntgUVgr/fjZTKpU4dOgQ7HY7r6Hh4WGeSP2xj30MMpkMH/nIRxJeh8pIkUgEOp0OoVCIHY7QWqEUDgBW0ddqtTykK9laOL1boUZRuiEtMMvJ+dBDD7F0Pem3ZGVl4eTJkzyz44tf/CI++clPQqlU4u233+YTyWq1wmQyYcuWLUxYpzxNeMPkIKRKfunSJYjFYixcuBDLly9Ha2sr1/mEujparTZuRkgoFEJTU1Ocpmw0GsW5c+d4R6T6G4VZ9DMU7p0/f55ZJxSepTPjIh0TiUSsqepwOJCfn4+BgQEOM4HJYUiEvubn50OpVCZsbKYFAuBduY8w3KWwjhxXGOolstra2rh6qfDZU/2QaqrFxcVJTQMTiUQoLy/H008/jdtuuw3l5eWQSqU8BDgcDmPLli2QSqXTjigUmkKhYCV96pUNh8MwGo1wu91Qq9WM5isUCq7XApNAUmdnJ1QqFQwGw6ylK+D6TFJaW8JNKx1L+JuNjY3Q6/Wora3F/PnzGSaur6/Hs88+iwMHDkAqlWL37t146qmn8PzzzyMrKwvf+ta3YLVakZmZiZ///Oc4c+YMTp8+/a4hoqQ5QzF+Xl4eszmCwSD+/Oc/QyqVwmq1Mge0t7cXAwMDPBty6sOhlijSGyKBLEI+vV4v/H4/gsEgc1mpnkYMnaGhIdhsNvT399/QvMZUTC6XM5A2Pj7O6GxWVhYikQiKioq4vhmJRGAymRKqAJDzkdYO5dNCEohwNiXV/EiseqqEyXTXr6mpiVN5p1OT/iH50PHx8ZSU1MPhMHJycrB8+XIA4CgiHA7DZDLBaDQyeJjIlEolT62mHJPU8glJpu4c+s5URhHmn4kkTYVGa4mck0L69yXnHBwcRE9PDy5duoRXXnkFFosFHR0dGBgYgMFgwEc/+lFcvnwZH/vYx7Bx40ZUVFTA5XLh8OHDWLp0Kc6ePYt9+/ZhaGiIezSFuzfJUchkMqxatQpmsxn79+9HMBhES0sLrFYrMjIy0N3dDYPBgNraWixevJjH3f9/MqVSiXnz5jFrpaurC729vbBYLDCZTHA6nfzci4uLIRaLkZWVlbBP0OfzcYhFoylIpJryo2g0Cr1eD7fbDYlEwmURIsgnsry8PKYB0oIUop/k+FKplK87m9GYx9WrV7P6ulKp5GsRiZ4GTSXKBwmQyc3Nhd/vh8Vi4VCXBMpjsRi0Wi0KCgqYPlpUVISWlhaeE5tsakPPgJ45sY2SKflMZwmd0+Fw4MiRI3j44YcxNDQEr9fL49OOHz+O0dFRuFwulJeXo6CgAH/4wx/w/PPPw+12Y9++fYhGoygtLUVpaSk3Uk+1SCQCv9+PpqYmOBwOHpFAaCJxSX0+H1paWrjVh8K+/y8mlUpZ8tFoNEIul6O/vx+9vb2sy1NUVMRj7zUaTUJuKS2UUCgEr9eLQCDAwBAZ7eiEAhPhQ6hHnMgIUSUHpTKNcCK5SCRizmkyVlxcfzy1CAAAIABJREFUDI1GwydtNBpl8eze3l6enzI0NASTyYTMzMwZx7oTMSIjIwOHDh1Cb28v6uvrodVqEY1G0dbWxlq4RO2j73T27FmYTCYEg0HmdScyOimFJTZKl96XkzMUCuGNN95AS0sLbrnlFmzZsgXZ2dkwGAyoqanB+Pg4KioqcPr0aTz++OO4du0avF4vysrKYLfbMXfuXHz2s59FJBLBG2+8EZfXkOXm5rJUhtlsZqTy9OnTKCwsxNq1a3HlyhUWi1ar1dy98v/JxGIxw/7U8VFSUoLe3l54vV4UFxcjOzubd2VaBDMZ5XxEeaNaKyGzNNdDiKzK5XL4fD4Gh2biGgsBH2FdlMJD4eAlKugn65wSiQQjIyNci+zs7MTBgwdhMplQXV3N5PShoSHucpnJOckxrVYr/v3f/x2HDh1CdnZ2HHtpy5YteOWVV7g+SU7ldrvxne98h+elJmP0PoTEjnRaxchmxXnHx8dx+fJlNDU1Yffu3XFFbOD6ABfhi+zo6MDatWv5v2kHn+4mR0ZGYLfbuZ/TYDCwzL7JZMLu3bsRi8Wg0WjQ2dkJvV4Po9F4Q+WND6MJcxRh61ZlZSX/WTjwlnLEmYz0UoPBIIaHh9HW1obR0VFGLylEFJIT6Gfp+rOp6NG9khSqsHeRxLdJwrKsrCyp52A0GnnaWmZmJsrLy5Gdnc2yqIODg7h8+TKWLVuG+fPnw2azzSiALZFIYLFYUFBQgNOnT/NQJSEYduHChRk3oUAggKKioqS6X+jUpahB6BfpThq7IYEvo9EIi8XCQINUKoVWq0VWVhba2tqSVg2nEodarUZpaSmWLl0Ku93OspNyuRwDAwPo6+tDf38/U/FS/dJlZWUoKCjg8K25uTntXW2qFRcXs15pW1tbynxKhUKBrVu3ory8HJFIBJWVlXGiz3RS6nQ6nDhxgtUJZ0KT9Xo91q5dC5fLhaNHj8ZtZnQKUTsTRSvAZCSzZs0ajI6OorGxMeH3oPmnWq0WOp0OKpWKi+6U03k8Hvj9fnR2duLkyZOzPgeDwcBzQleuXMljD4FJ0Kyvrw8+nw8WiwXnzp3D0aNHZ3QuqVSKefPmoaamBi+//DL/XHV1NcLhMPx+P8bHxznPj0aj6O3t5Z+jkZJ79+6dtelaLBbjjjvugMViwauvvorh4WF88YtfxMTEBHbv3p0Q7Y3NIPCVlnPm5OTgk5/8JNauXQuLxRI3qUqj0WB8fBwulwtHjhzBK6+8kvSkZLFYjAceeAAPPvggdwaQ0zc1NaGtrQ3PP/982g3WTz75JLZv3w6v14vm5mb84z/+Y9qkZLL8/Hxs2LAB9913HyOCf/nLX/Dss88yuT5Vk8vl2LNnDyOmdHLEYjEUFBTg0UcfTWmMOxmFWN/97ndx9913QyaT4a233sLXv/51Hlh7I5vVj3/8Yy6JGQwGfPnLX05Z7by+vh7bt29HTk4OlzdoRk5hYSE6OjpQV1cHu92OwcFB7Nu3D2fOnAGQ3CgFkUiEf/u3f0NhYSG8Xi/a2tqwYsUK9Pf3o62tDc8999wNNfCnYzM5Z8r0hRUrVuDJJ5/E3LlzWaOWHDIjI4NZPkajEWVlZdi8eTO+8pWvzKq6BkxKYOTl5SEUCkGlUsFoNDJVz2q1cjtTsiYWTw4GKi8vx8aNGzEyMgKPx4OcnBycO3cO//Iv/4IzZ87g0KFDGBwcTEnJTyqVYs2aNdi5cyf0ej2cTieysrIQDAZxxx134Pbbb8c777yDhx56KOUFX15eDovFgoGBAY4efD4fK+Zt2rQpbedUqVQoLy9nmlltbS3MZjO8Xm9afYfUVlVVVYW6ujrccsstCIVC2L9/Px5++GHs378f7e3ts5akRCIRbr/9dnz605/mfJV0dQHgypUrKCwsxB//+EcolUpYLBZUVVWhoKAAL774Ig4cOABgdgetq6tDQ0MDHA4HrFYrFi1axPk51TTfa+cUkuvlcjmKi4tZByvR/SbtnDQeb+vWraioqIDP5+MBtQMDA5BIJOjs7ERWVhY8Hg8sFgv3aj766KNobGyctZlZrVZj3rx53BQsFotx7do1mM1mKJVK5OXlJT29TCaTYeXKlVi/fj1GR0eh0WgQiURw4cIFqFQqqNVqHD9+HPPnz4fBYIBKpcLLL7+cdKgbiURw7do1LiNQexAwuSkkWxubzurq6jjvJAU3YvMEg8GUWUsSiQTV1dWoq6vD3Xffjfr6eu6WqKysxOuvv463334b+/btw+HDh2cEWKZaVVUVnnjiCSxbtgzRaBQ7d+6E2WyG3+/HwYMH8aMf/QhPPPEEDhw4gJ///Oc4evTojNcqLCzEhg0buJODABqRSITs7GysXr0aBoMBX//611msjOqot956K4fns5nVasXw8DC8Xi9CoRD8fj9GRka4ubykpGRWlYnpbGojgVARQqVSobS0FMXFxSw78x//8R/4zne+k/CaKZ2clZWV/CJofuWf/vQnLFu2DJcuXYJGo8Hg4CCuXr2Khx56CMAkYqbRaFBRUYGmpqaE11cqlTCZTPB6vZxrkd6My+XiLvtkrKGhARs3boTD4eB+xlgshrGxMZ4lSd37Ho8HKpUK9913H3bt2pV0rixE5TQaDbNNCKhpbGxMK0zMzc3lcHBiYgJ6vR7j4+NxAE4qtmzZMjz99NOwWq1McqAOFKp9bt26FZs2bcI3vvEN/Pa3v02Yz9M7WLlyJVasWMHXe+yxx1jZ7vOf/zxL0KxcuRLhcHha5yQAadmyZSgsLOTBs8ISDaHXY2NjcfNJCYAymUxYtGgRent745xiOjMajaxOoFKp2CmDwSB8Ph9KSkrSQlhjsRhUKhVqa2u5E8loNLKKx4oVK1BUVMQc3IaGBuTl5SVkTSXlnAS3+3w+DA0NoaKigoWJx8fH0d3dzbucw+HAqlWreOdTq9XQarUseZjISGafQmUATM6mkCwZIrFKpUJNTQ3kcjk0Gg0X2IPBIDweD/cIkl4MOSnRBru7u2c95QmRpHybHIccNhKJpKxxREajLcLhMHfMjI6OssZPqp31S5cuRWlpKQKBQBy1DLiOEhMSvG3bNrz22msJP4MW7quvvoq/+7u/455RmUyGvLw8HghEPF3K86azaHRyQvWdd97JxXphpDBTqxttEBT2btu2Dfv27Zs1fCZpE6ogZGZmMvLtcrkSKmAkMrFYjNzcXHzjG99AcXExy8sQok3PmDadhoYGrFu3Drt37575msl+uEgkQnt7O+655x58/vOf54W3efNmLFy4EHfffTdWr16Nbdu2cWgTjUbx2muv4e6778b+/ftn/YyioqK4mZESiYQFmKilJ5kHZzAYIJVKceXKFXg8HmY1jY2Nwel0Moro9Xpht9v5NO3u7kZxcfGsQsXA5AJtbW3F6OgoN38LT1K/34/m5uZZrzOd0QKSSqUoLi7G5cuXma2iVCpTVg+n/BLAu9q3qKxCz3Xu3LnsGImMoplTp05Bp9PF1Qf7+voY/ZTL5ejp6UF7e/uM1128eDHUajVisRjUajUTD4QDsoSUOEolCISMRqPQaDRJNQPQjFMiuFy6dIlP07a2tqRkVKYzSm1yc3P5AKFSEt2zMO0RiydHjCQiOCTlnMJuivHxcezduxePPfYYMjMzodPpoFarMTg4yDKIGo0GWq0WP/zhD/Hwww+ju7t71rKHSCRCWVkZotEoN7rKZDIWeBJSoxIZhcJFRUUwGAyYM2cOFixYgNLSUpSXl6O4uJgbwktKSlBeXo66ujqsX78eVqsV5eXlHNok80wOHjwYF2IBk2DR6OjoDc0SjcUmJR1jsRj+9V//FRqNhruEUkWBqYGbmp6FxH66Z/pzsgoD9DPd3d3sRKTRo1Kp+L9FosnR67SxTGe33nornyjEdyVwinojhSUlej70PUiY684775z1vkmmkgDMAwcOMEFCq9VOKwc6m9GpTsJkNKdWoVAwW2gq9zgSiWDdunWYN2/ezNdN50YA8BENTO5GRNAmlkksFmOqVbJ5osViYVUEmUyGixcv4uTJk3HNzcmMl/N6vRgbG8PcuXN5XuTY2BhGRkYwNjYGj8eD0dFR2O129Pb28osPh8M4cOBASsNx7HY7LyoSNhOLxThz5gzcbnda1C2/38+LfGxsDL29vdzfSCdRsiaclUo6QrSYhLQ7OqlINnI2o2tMPZFJxWGqFMtMglyVlZUoLy/nsJj6a4W/P5X0IuyioWtGo1Hk5eWhsrIy4X339vby7/r9fnR0dGB4eJhZQOl0kZDjVVRUMHdZeM/0HYQ/T5+VaFBx2p2g0WiUNV7oYZFMIsXZRDROhiwgEk0KGRMKl5mZiV//+tfYuHEj7/rUWjbbfZ06dQqXL1/G/PnzodVqUVJSAgA8M5PEnoh1EgwG8fTTT2P//v2M4s0WPtPD7unpQSQSQWdnJwoLC2G32yEWi9HZ2TmtIl0y5nQ6eTp4U1MTs0wo5CdFw2SMtFnpz8KmaGFEQpzYZDeTSCSC/Px8tLW1YXh4mEEcn8+H8fFxHqsXCoXQ3NyMiooKnDp1Ku4aIpEIW7ZsgU6ng8vl4pms5HR0alIjNDk/PQtSkSd+tlQqxfr16xOirUNDQ3F5t8/nQ2trK3Q6Hc6cOZNWbZpSsJUrV/L3EmoDA9fb8IR/Pz4+zp0301lazWYUgtIpQUNnxsfH41q0li9fnlJdUqPRMDxut9tx+PBh+P3+OApUMtqy1AJ29OhRtLS0YP78+SgpKUFtbS2qqqpQVFSEyspKVFdXY968eSgtLcX+/fsxMjLCEoqzGYUq69evh9frZXL5yMgI3G43cnNz0xogDIARVLFYjCNHjsDv98Pj8YAkQZItdQDXpSGF4SCZcLEQAk+1uNmMWEzBYBBut5t/V6fTofj/1Avoug6HA1VVVe8Ca4qKilBWVgaXy8VhL3WlANfBnqmnEKHioVAIer2eN3TSCko0m2RwcJC7RWhtdXR08LDitMW4pFIsWrQIPp8PgUAgTl1CGKlQ4zt9j0Q01JSdk3bZOXPm8G6gVqvj4mm6GbPZnJK4E+2UYrGYdzB6cUQRTObhTe2ouHTpEq5evYquri50dnZiZGQEnZ2duHbtGlpbW2Gz2dLiP1LbnEajwbx587hOW1FRgc2bN6c9wIb4qhKJBG1tbazWIJVKuQ0sWVMoFPx+aNOh00eI2ApzxGTvWy6Xo6ysDKdOnWKmFaUMpKFz/vx5Bvam5uCU+wkBwI6ODvT393M3h3DtCAWz6CRVq9X8PEguZbpeX+FntrW1IRAI8CZgt9t5/EIqRBThmi8rK2PhOSr/CJUo6LuQVhZ931deeWXG66d1corFYjQ0NHCySwgbiV+RcxYVFSUt40hG8XpHRweAyQdH+a3L5Uq6+ZleKgEL1D4Ui8Xi5Arp79PJDTdv3gyDwRCnIEAhk16vT9s56RQj8jnNRpHJZO/S653NqNuEBLyA6991quYP8XSTHeXucDgQCATQ3d2NjIwMRmdVKhWfTo2NjWhpaWFdV6F1d3fjmWeewdmzZ5n0Pz4+Dp1Ox044NRwUIstSqZTVGFUqFZqamvCrX/1q1tDU5/Ph2LFjeOutt5gaSKBTKs31dB+xWAw7duyAw+Hgzhxh7zLhGVqtFoODg/jDH/6AiYkJOJ1OHDt2bMbrJ13nFKJOAOJqnUNDQ5wrkFQD1c1SCWup/41UDIBJ5yReqdvtTppiRjuVwWBAeXk5o3qjo6Nc6hkbG0N2dnbcLpesicVirFq1ivmfU5vIxWIxNm7ciDNnzqScd7rdbj49KaQfGxtLSzaFkENgsgOIwBCVSsWLn9rHxsbGkJubO6tzknOrVCoMDAwwg4v40MIQOj8/H5cvX0ZRURHnYcJnNTAwgH/+53/G0qVL8dWvfhXHjh3D8uXLWX9nqmMKIzO5XI6uri6MjIzg+eefx6lTp5JaHxSCEl5AwKXD4Ug6rBXe1+LFi7Flyxb+XbfbzcLcNApQLBZj586d+OUvf4lAIID77rsPX/jCFxKmKEmXUoSnglKpRHFxMTurRqPhBydUXFOpVLjlllv4S8z2Gf39/ax2RjUxkhJRKBR8AiR7z8Ckw3d0dKCtrQ2tra1oamqCzWZDa2srWlpacPXqVXR0dKQ8JyUvL495wFNFouhzt23bltaJTDRIiiLEYjGDOqkSEIQlqLGxMV7kwsladHJS0/V0A4ymmkgkYqGs+vp6jpycTifcbjekUilCoRAKCgq4bkt59FQLhUJ45513cPvtt+PAgQNQqVRM6hA+P3r3lOKo1Wo0NzfjC1/4Ao4cOZL0xu31epGVlQWFQgG3241Lly5BJBKllDLQxvGRj3wETz75JG/+CoUCJpOJZ9243W54vV5861vfwg9/+EM4nU4WGhsYGEgIcCZ1ci5ZsgSf+cxnOL8k6hchimNjY3GNtqQbo1QqsWHDBrz00kuznh6xWAyHDh3Ctm3boFAoWC+UrieTyZCZmTltw3YioxIBcP0UodkgKpWKTwnhJOlkTrry8nJUVVXF5UtCm5iYmFWAaibzer3weDxQq9UMzWdlZUEkEnH3SCoWi8Xg8Xhgs9mQl5fHuRsplE8tks+m1UrIrt1uR0ZGBhYsWMBOZDQauc4Zi8WwYsUKHDlyBCMjIwziTHc9+uzs7Gx4vd64mux0pRT6N21iqXTUFBQUQKfTYWhoiFu5RCIRCgoKZmWgiUQilJaW8kiOtWvXQq1Wx/XaUiXDbDbD5XLh+9//Pl5//XW+BuX4g4ODN+6cK1aswI4dO+LU1IhWF41OjuymE46cVavVIhAIYMGCBcl8BIDJHIReDOW0VDOlXS7VhanRaGCxWDiH0+v1KCgoYE0e0pdJNTc2mUzIzs7mfFi4w9PpQCdFsqc9mcPh4Bopncw0wCjZeSNCm5iYgM/nY7AjEolAo9HwIFla6DSrM5nOf2LEeDwedHV1Qa1WIz8/HyMjIzAajZBIJOjr64Pb7UZTUxNqampmFPUWvlNC++k0p78X6vII73d0dJTJ5smeekQ6EKZcer0+qalisdjkiIa77rqLKZ9U2snIyIibDj4xMYFnn30W+/bti6ttEtFmtjA6Kec0GAy8wKjUIGSXkBK7QqGA3++HyWRiAIAAo2RQMLfbjba2NhQXFyMUCmHOnDks7kz81VRtYmICXV1dXHMTqvANDQ1xZ0KqzklN4MKZlEIlAHJYs9mccu0sHA7HhfekHA9g1jajqZaRkcHhpxBIIaAIuM5fHhgYQHl5OX/WTEbUQrlcDqfTicbGRjidTtxzzz2Qy+W8iZAoHBHNiYaXyFQqFd8PUfeEJyudkMLrpJo6DAwMYHBwMA69pmgv0f0ZDAZUVlZi/vz5CIfDTAyh1kQ6wakc+Otf/xovvvhi3PsSEndsNlvCdCop59y/fz/uuusuZGdnxzkIfSkAXCOSSqWYmJjgqcpyuRxz585Nqgve4XDg2LFjXENdsWIFXn75ZR4NmM7cCVL3IyU2ItcTmkh0q1RqkkR6n3piCoEzOiXKyspSds5YLAa73Y6ioiJu4KbPI0ZTsma32/HSSy8hEolgyZIlXAAXnkqUL7a3t/M49tnuTywWIxAIYHBwEAcPHoRSqcSuXbv4Wfp8PgwODuLixYtoaWmBxWJhZYdEJqT4kVMKR0YIy3XAdXQ/Fbt8+TI+9alPMcGDHMpmsyU8AAwGA5588klen9QHKuR9k0/88Y9/xM9+9rMZ0yQhUj6TJeWc77zzDh5++GG8+uqrDAzR6Xnu3DlG4YgT63Q6MTExwZPCaO7HbDYxMYG+vj7Oaerq6vDrX/+aT91UHJOcRC6Xs3RKMBhEOBzmcFSYP6dSO5TJZDCZTHHooTA/ikajXFJKRzE+Go2iv78fZWVlyMnJgVQq5YWdCnUPmJy4/Mwzz0AkEuHtt9/mE4xyQrJgMIj29nb87Gc/S0qfKRgMoqenBz6fj5UuaNyF0GhxDg0Ncf0zkdHJS4AVAYNCKh+lKMLnnIpdunQJjz/+eFwJjBokEpVS2tvb8dWvfhVLlizBxz/+cSxYsICF5qLRKNxuN1QqFV577TX86Ec/mvY5TiWB3LBzikQiHDx4EI888giefvpp5ng+88wz2Llz57RhB90wgKQBHJFIhObmZvT398NisSA/P59ncRAsnQpaS4uZBvOSWDApvPX390MikSAQCLCwVTKnkkqlwpw5c+KACirH0H8TWplOrTMWi+Hq1atYtWoV528ymYxD8lSNTp6pQA+F43SSEmF/tmdgtVrx+OOPIxaL4Yknnkj4fula69atw5e+9CXs3LkTFy5cmPHnqWRE708o/jyVkEAHRKrzSPr7+1kAjWx8fBxDQ0OzfperV6+itbUVe/bsgVwuR05ODutfRaNRdHd3o7Ozc1bkuLW1FXa7/cYBIQotXn/9dYyPj6O8vBwA8Ktf/SrhTiOU50/GRCIRurq6cOHCBZjNZgwPD8NoNCIjIwMTExMMYCR7z8CkEmBmZiYWLlzI9wRM7lpyuRwOhwONjY0plSiIDPD73/8eBoOBxyhIJBLo9XqEw2H09fVxV02qFo1G0dLSgmvXruHq1asYHx9Hf39/QrnKREYL3ePxsPqecPETsJFsS57T6cSpU6fi1B9ms9bWVkaMZzKRaLIfOBqdVHMfHh5GZmYmS6iQvKdIJILf70coFILL5UqZ3RWNRlkoDJh8Pg6HI2mhcmrAIOpie3t7Sp8PICnVhrTV93JycrBp0ybY7XY4nU7Ow7RaLQtKp2srVqzA448/juzsbPzP//wPBgYGMDAwwKdqqoitRCJBTk4O5zM0bcvn83F/Z6pGhH6tVvsu56TwXKFQpByGkmk0GlitVnR0dCAcDqO4uBh6vR5XrlxJqjNnqolEIixZsgTZ2dnvAvSosfvSpUtJ9YpKJBKe2zI0NASNRoOHHnoIGo2Gw9FgMAiFQoFTp07h4MGDyMnJ4XEas+V1NTU18Hg8rKEkbLSm+xW2vHV1dSW93uhaq1atglgshs1mg0ajwYIFCzA0NMTEeGAydPd6vRgcHLxhIbhEFnsv1fdEIhEee+wxfPOb3+R5JAAYWGhoaEip7YrMaDTitttuw4oVK1BTU4PBwUEeP37ixAn8+c9/5oFGN23SqA6aLpr9Xlh9fT127dqF3NxcLk8MDQ1BKpXijTfewNe+9rWUN9T3y6iu++ijj+Jzn/sc5HJ5XJuXcG15PB4Eg0EcPnwYjz/+eNr6s7PZTM6ZVsuYVCrFl770JXg8HmZy0KzDgoIC3HPPPdi5c2dK18zIyMBjjz0GpVIJt9uNO++8kx18x44dqKiowPbt2wFMJvQflpf9fhudGMKcXiwWw2KxoLKyElu2bEFpaSk6Ozvx/e9/P2nw7b20ZcuWwWq1wuPxcBRCJ82cOXNQVlaWtC7T+23kfFSfplq08FSm5gtKfZIBst4PS8s5jUYj8vLy4HA4GOIn0MLj8WDdunUpO+fy5cuhVqvhdrtZwp8WJLX3iEQiVFdXcx52I0YynBaLBTabDRkZGTz12GazpRXqvh9mNBpRXV2Njo4O2Gw2iMVifOpTn8Jdd92FpUuXQiqV8gxUalT+W5pKpYLVaoXP52MeMJVnqI1sJgWED8LIOR0OBzukUqlkUIlI+9T7Gw6H0dvb+4Hca1rOuX79enYeIWomFk9KFhYXF6O4uDglPRan08n0QBqEOzAwwBS78fHxuOlY6ZhEIsHtt9+OBQsWMHmAGCn79+9HRkYG1q9fj6NHj+LQoUMf6OksFk8q+j344INYs2YN7HY7nnvuOaxfvx4PPvhgXIlBrVZjbGwsrVz0Rs1gMOCWW27hpnWag0nrg3LQD4vRvVDDOqlDCrtdiJlFHTLpioPfqKXlnIsXL45TEgMQF68rFApYrdaUnPPs2bPo6OjAtm3bmIVx4cIF1NXVsabM8ePH0dTUlFZDrBDlPX36NA4ePMjyKlu3boXL5WKE+MMQNhORevHixSgrK0NxcTH+8z//k7WEiB4WCoXSItcLzWq1QiwWx4FXwpptItPpdJgzZw4LMQsbIOi9fRCbRjJGm7OQwkhlO/r/FA2ma/n5+fjJT36CgwcP4vXXX+fQP5kwOe2wlmQlhW09Qkums0Fo0WgULpcLfX19KC4uRkZGBnbs2MENrCRenc4uLOxffPXVV/n/09zIiooKqNVqbNiwAS+++OLfXI5/OiPpDb/fD7VaDalUCpfLxUqBFEIKQ7Pp2t6EhHEyiUSCzMxM+P1+5ObmYsuWLXA6neyc1IubmZmJlpaWhEBTVlYW5HI5QqEQt88RtZHapdIp/7xfRo5IpzoRMoRrmNYYKRWUlZVNS7CYySiazMvLw7e//W3k5eVhx44d+NznPodjx47h1KlTsNlscLlcCWu+aTmnkBlDRi+D/n8qfZxkkUgEDoeDcxVS/zYajXC5XNx3l6pNZWVEo1EYjUbcc889AIDm5mbMnTsXfX192Lt3LwBw/paojkvfVbgL1tfXc9ngL3/5S1y/3v3334/BwUH86U9/Svq+tVpt3Lg+0gEipxN22E/dJGeqMefn58NoNKK+vh6VlZUsXL1o0SKcPXsWMpkMK1asgFwuh9fr5cb36ay8vDyOqC4SibhURX25N4oPCE0ul8NgMGBoaOhd321qz/FsRicn/TyVlCKRCCtRyGQyLF++HC+99FLSERuFyCSNY7fbodPp4HA4UFdXh4ULF0IqlcJms+GRRx6Z8TppOafX64VKpeIJwz09PayZQ7tkOipmAPiF0nWoIO3xeNLWdyEjZ1q+fDnuvPNO7N27F6dPn0YgEMCuXbvwxBNPYGJiAuXl5Vi6dCm0Wi327Nkz7UlKzkGI3qpVq3D//fczC0er1UIul+OPf/wjIpEIPvGJT+CBBx5AV1cXrly5klT9MxKZHIkeiUQQCARYMY5CRuHPTSfOJcyhSFZTqVQiNzcX69evR2VlJfr6+tDV1QUf1fAMAAAgAElEQVSj0YgHH3wQ8+fPh91uh9VqxdDQELZt24af/vSn04ZhIpEIJSUlHD2Nj4+zXOrY2BgkEgmPcb8Ro01p1apVqK+vh0wmw29+8xvYbLa00w8hOju1JY3IDX19fTCZTKirq0NdXR3Onj2b1LVjsUltqR/84AdMVCBlDKFuUGlpKYuCTWdpOeebb76Je++9lztOmpubEYlEUF5ezrtGugiXkIgtLJLfSA1P2Gu4detWqNVqPPXUU+x0WVlZcLvdyMvLw6pVq5CXlwez2Yyuri5+ecJCeCQSgdlsRnFxMdavX4+lS5ey5g2dEoFAAI899hj+4R/+ATabDTk5OXA4HDCZTKitrZ3VOYXKePQsRKJJ2Umhajktzqlq+HRqajQaBtkqKipQVVUFrVaLY8eOoa+vD4sXL0Z9fT0AMCOHxvqp1WpoNBrI5fJpIwiRSITc3FyIRCJWVzh37hzKysqYdkiqDumaRqNBdXU1br31VhQVFXE9d926ddi7dy/PHM3Pz8eKFSvQ3Nw8qxNRozg9T2F7GjDZtXLo0CH09PRgx44dKC0tRW1tLc6dO5fUZjBv3jx873vf4/xSrVbDbrdjdHQUWq2WedKBQCBhuJyWcx48eJBDWJFIhBMnTiAQCKC0tBRSqRROpxNXrlxJ+brUPEs1U1Iqi0QiyMrKSrr1bLrrZmZmYuvWrejr68OBAwfiugJMJhO6urqwfft21NTU8HyV4eFhuFwuBjh0Oh3MZjOqqqqwfv161NTUcEcCbR704GlBBwIBaDQaVujT6XQoKCiI6wyZzsgxnU4nt09RXQ4Ah14U5vt8vrjIghrW8/PzWddn4cKF2LdvHwtqS6VS/PWvf4XZbMbixYsRCoVgsVggEolw/vx5aLVaZGdnQ6/XzxjeC1FZs9mMH/zgB/j7v/97Hp6cqqwM/Tk3NxelpaWoq6vDnDlzuN64ePFidHd344477sDSpUvR3t4OpVKJBQsWoKqqCn/4wx8SOhFt+PQ9VSpVXK+lWDw5hGrFihVYunQpCgsLAQAlJSU8VVsYoQg/R6VSYdOmTXj00UeZXkgN7XRi9vf3IxqNMrnG5XLN+EzSck7qYDCbzQAmSQF0A6Sglg7CRShvIBBg1IwABY1Gg4KCArS0tKR83Ugkgrlz5+LMmTO4ePFi3N9VVVXhs5/9LCwWC1paWvDTn/4UarUaW7ZsYcW7+++/H/X19cjNzUVeXh6DXSS+BSCuXY5U8hQKBTQaDavWh0IhOBwOFBcXQ6VSJaSE0Ut/4YUXWH6TOkqCwSCjuYTWGgwGzjt1Oh2USiUyMjIQCARQUFCA7OxsVqagjWJ4eBgqlQp+vx+9vb08h0Wj0fBA2dlOCmKIGQwGFvNyu93sBMkitfQ5GRkZWL58ORYvXgyTycRC1zqdDuvWrYNIJILdbkckEkFZWRnq6+s5siKZG71eP6PWEnWyUHhMPZhUpvN4PPz8NBoNO49er4+bWj3VJBIJPvOZz+BLX/oS3G43XC4Xb5aESygUCk4Hg8Egs5NmsrScMxaL4eTJk/jkJz8JYFILlOqUEokkJWRLaEqlEgaDgXdb0sWldrR0OjxEokmpx2AwiKtXr8adWFqtFsuWLcMtt9yCgwcPYufOnTzp+BOf+ATsdju2b9+O+++/n0+OcDjM5Gyqy9JUtGAwiIyMDIyPjzOQEwwGodVqYbPZeEgTTYGeja+pUCiwZs0aKBQKJklQCUso1yKRSFBWVgadTofVq1cjNzcXIyMjiMViGBwcRGdnJzIyMnDp0iUUFRWhtrYWVqsVMpkMfr+fT3giDsRiMWRlZaGjowN6vT5x58T/5bo6nQ6/+MUv+DvTop+ti4acwGq1oqqqCtXV1cxZJl0jasdrbW2FyWRCTU0N2tvbWZyLekslEgkyMjJgMBhmdM5YLIaMjAwsWbIEwPWSFcmvGo1G+P1+DA8P84ZFqRuNtFSpVLxRajQarFy5Eg8//DDrNBEhg9IJAsWE71upVKKgoCBhY3vazincHamPE5hcKOkCAASuUBGbUEnhrMpUulyogTo3Nxc2my0uxJLJZCgrK8PKlStx+PBhPPvss3FoczQahc1mY/YLqdUJ5TLC4TCPDzSZTDx/QywWw+l08g4MADU1NRgYGEA4HIbNZourD1ONjXLI7OxsNDQ0YMWKFdi0aRM7NKnRC3dbcgKtVguNRsPKguTIDQ0NGBwcRF9fH0pLSzE0NASTyYS//vWv0Ol0yMvLg9FoRG5uLhMfQqEQWlpaeOEk6tgxGo1874TqChdhIseWy+XYvHkz6urqOKSkjYc+m76XXq/nsfZKpRJFRUXM8hECOvTziey2226DyWTidjNh0/TIyAjsdjvr3xLxniKhXbt2IS8vDx6PB2azGVKplGmsvb29GB4eZp1gGjFB90fvkEpemZmZCZv803JOkUgEo9HITApq56K/S1b3dKrRC6KeS2ETM+1wyTonLTiZTIb58+dj7969/LsikQhVVVW49dZbEQqF8Itf/OJdpxhtCu+88w5CoRADP2q1mhex0+mE1+vFxMQEPB4PSktLeSxFfn4+56xut5ulULKystDf3/+u3DkjIwNVVVVYvHgx1qxZg8rKSp5ARSExjb+YStQmQIPaqsxmM487VCgUWL16NU8fDwQCMJvNWLJkCQKBAIeI1DdpsVggl8vR19cHuVyO1tZWznmnGgF/KpUKw8PD/DM0LGi2k1MkEqGnpwfBYBAGgwEymQwajQYKhYJLakIdJpL/CIfDCAQC8Hg88Pv9LJMTjUYRCARmBNtEIhFycnKwY8cOjI+Pw2w2c8RDh8rIyAiys7PhdruRk5PDgtPBYBB6vR4ikYhJ/S0tLXG8XJVKxYgsvTdac1SHJg0u2rQS9aKm7Zy1tbUAwOPevF4v3G431Go1iv9Pjj9dIweimpPw5ExWMOuuu+5Cf38/+vv74yZ+iUQi5OXloaamBgsXLsTjjz/OISB9Np1MNLD24MGDOH36NCu9FRYWIj8/H1VVVayIEA6H8fLLL+Py5ctwu91QKBQoKirC4sWLodPp4HQ60dnZiba2Npw8eRLBYBBSqRRWqxUPPPAANm7cCKPRyF35NAtVpVIhMzOT74VEpCj8pF2ZNoo///nPWLlyJYqKinhGaVZWFqcLExMTCAaDsNlsPC3cZrPh2rVr8Pv9uHDhAoaGhnDq1CmmOFZUVMxYLKd5M7RxAvGnZSJAaHx8HOfOneNOEYqWhFHEdE389G8KqUkrlzSrptto6Vl95jOfQX5+fhzwCIBz8IqKCjz33HPQarX48pe/zJhBKBTCxz72MbS1teHatWswGo3YsGEDRCIR4wHRaJQ3FyHSHY1GubTk8/kwOjqKnp4enDp16sabracaARAi0eTMThLJ8nq9yMjISPvkBK4THOhh0ucBiJszMdXoNKPFIJVK8d3vfhePPfYY9Ho98vPzeRFpNBrm0E5HMaTFISwBeDweBrnod6ajzQlP9cbGRrz44ovT/r1EIsHXvvY13HbbbYhGJ2e7dHd3M7AjLIhTEZsmXBNBgp5RLBZjsKqlpQUvv/wyAynUYaHT6ZCfn49oNMoDp8jZaaPQarW8EZK8ysTEBFpbW2d8X21tbRxWTw3RKCdMZNT7+X6YUPg5KysLGzZswObNm+PSApqtOjw8jLKyMrzzzjt45plneLw9RXJ+vx/79u2D2+1GcXExxsbG8MILL6C6uhpZWVnMZKPNob+/H2NjY+jv74fD4eCUhCIAkWhSX2rdunUz3n/aU8boOO7q6mLpRbfbDbPZnDbRmXI6CgeERAYCQGYKaTMyMrB161aYTCb4/X4MDg7ihRdewKJFiyCTyWAwGHD27FkUFRVh48aNuHbtGp577rl3XUfIcJotfE4mvJ7pZyQSCVavXh0nlk0OSKMEiRkknG1Jqny0idE9A9fFsWhjEeY3BK7QtenkJXSWQCFy+OHhYXi9Xvj9/hnpjLFYDFeuXGFAJjs7G11dXfzeAoFAQuWD99s0Gg1uueUWWCwWVFdXo6SkBMeOHUM0GsXWrVu5TELUwxMnTuDRRx9ljWR6dxRCh8NhvP7661AqlSgvL2ft28zMTBY2o2ijr6+P2/caGhqQm5sbVyL0+Xzo6+tLuPGlDQjRkNxjx44hGAxicHAQra2tqK6uTnnyMhmVTcgxKGSjxZmomB0MBnHixAnk5uYyUVylUsFiscBgMEChULDMf2NjI37/+9+/6xQWAgvp0A9TMbFYDLPZzCgsLWjqNqFwjTYL+vPExAR36AjV3KVSKSsy0KISEjdSmQGSijmdToyNjcFgMHDEJOyZTHV8xHtlcrkc//RP/4Ta2lp4vV7s3bsXr732GmMExN82Go3weDxwOp145plnMDY2BrVazZO2KW+eCnJdvnyZyQw6nY6dk6K74eFhxhV6enp4Mjk5L8nDJDrI0j45HQ4HtFotcxzdbjfTqdJprKUck0omdNP0bwq/ZrJwOIy2tja0tbXh+PHjrMom1JKlz6DBNVONXkZPTw/uu+++lL9DKnb77bdDoVDA6/XC5/NxTk0RCeVhsViMSe/CGSTkhHSCKhSKtNTgb9RcLhcuXryINWvWsBg1gVU+n++GqXvp2tKlS7F161YG0ObMmcMbrlKp5DKS2+2GRCLBnj17cPz4cQZvSKuZTrnpyl6xWCwu3ZnJ0t2g0nJOCp+IAQGA1cJjsRiKi4uhVCpTziXIcXQ6Hfx+P0PRVE+kGZWzLcBEuWkyRjng+2kdHR24cOECMjMzkZ2dzdq0U/Nrog9SVEHACaHXxJqy2Wwptei9V+b1etHS0oL169fzkGJCWal2+EGY0WhkoTi9Xs/gjJAKKZFIYLPZ8Morr+AnP/kJgOs8Xuq8otTig/geaYe1//3f/41gMBi3IA4cOIBwOIyhoSHIZLKUnZOKuAMDA0yhI+fMz89PaWjsh91Onz6N+++/n4vaUqkUWVlZTCeLxWKMrgLgqAK4XnIigWzaID8I5wwGg2hubmatYgCw2WxwOBxpqdK9V/bqq6/i4MGDsFqtyM3NZUKAXC7n+TmBQABXr16N4+KKRJMKgM899xzn/UQ8+Vtb2up7tJsIf5/yJKLdpWN06lKJhsLRFStW4OTJkwm5iIlMLBajtLQU+fn56OjogEgkwrp169DU1IQTJ06kdc33y4h5IhKJ4uRAiblD+ruphLDCEtS2bdtQV1fHpIvnn38+bZwgIyMDJSUlqKur43sLBAKskHDgwAEcOnQIdrs95WuLxWLs2bOHaZFUH8zIyMCPf/xjHDlyJKXrVVdXY9u2bTxzhyRBqb6bmZmJQCCAJ5988m+aHsTeS4EvYPrx71PLD+kYcVRpdDcAzqlSPYmJ5UHtOuXl5bBarVi7di3C4TAqKipw7do1ZtUIh+p+EEZO+Mgjj+C2227jeqZer2cluImJCXR3d+N73/teypKQBoMBZrMZNTU12LBhA0wmE86ePYvm5macO3eOa9XJpgRarRZPPfUUrFYr9Ho99Ho9g4XRaBQXLlzAnDlzMD4+jjfffDMprq7QNmzYgJUrV3IIT/mfRqPB22+/jcbGxpQqAw0NDbjvvvviRhESSENECoVCgW9961sf6DogS9s53y+zWCxoaGhAR0cHs0skEglqa2vR2NiYUuOu1WqFxWKB3++Hx+PBlStXEAgEUFZWBgB45pln0NPTA6VSCbVazc3FH5Sshkg0OZ1qyZIlPG6AejgJ/IlEIqioqMDKlSsTNkHT9ajnsrKyEmVlZazHdP78eeh0OrS3t6OhoQFz587F4OAgbDYbTp48mRSIUVZWBqfTiaamJlgsFtxxxx3Q6/WQyWTo6enB2bNnYTQaYTAYmCaXii1btgxyuRx+v58jBcIlFi5cCI1Gk7QYOLWVeTwe2O12KJVK7pMV1tVpPGCqKobCFr3s7GzI5XKMjIxwfTMde0+ds6qqCosWLcKLL76YFiAjkUhQWlqKuXPn4q677sIrr7wCmUyGJUuWwGg0Yvny5axUMJupVCrk5eXxIhOLJ6dCe71eXLlyhQnTQqEsuVzO3NIPQpQqFovx8CLKO4Xq7GTU1pTM9QoKCrBp0yZubaNF19HRAa/Xy8AdzWOpq6tDYWEhfvWrX816fcIWsrKyMHfuXLzwwgvcfP3iiy/CarVCo9HgypUrKZ+awORGTaM/qIxE07wKCwv53SVjVOsWNmhoNBpmX9Gm7Pf7kZeXl5JzUsRTWFiIW2+9FU1NTYjFYsjMzITZbEZPT8+7uqGSsfTkCqa5MWBSo/Tee+/Frl27eGQD/cx0bJqpplKpYDab0dTUBKVSiVtvvRVr165FZmYm2tvbU1JX0Gg0mDdvHvx+P+dbGRkZPFKQposBiCNbC2US07ENGzYwJxZ4N4topunOdB8FBQXM36UaJy0mqVTKJxBNUU5k1K1Ck6KFs2toGBWFoOQ8fr8fWVlZqKurm/W7BgIBtLW1weVy4be//S2OHz+OV199Fbt27cLExAQuXbqE3/3ud7Db7SlrSgHx64ba4Yj8Pp0sSyIjBhRFIHQSU+8tbQDBYDCtocdisRhr1qxBe3s7jh49isbGRpw/fx42mw1f+cpXUFRUFMfDTeqaKd/FFBM2yFIb1aJFi/DNb34TxcXFcfW62czr9eKtt97CyMgI2tvb4fF4uOA+PDyc0mgDosDV1tbGiZBRrUu40In0TQ8vXefMzc3FN7/5TTz11FOYP3/+u4axrly5Evfeey8zU2a6hkKhgFKphEqlinPOWCwGlUrF3SdEGZvJiLtrMpn4/dA/UqmUydjC8o3BYOCBw4mMTszGxka8/PLLOHr0KOx2Ozo7O5lRQzlyMBhMq91PWKYDrs9EJbJ6KtEN4Q+0Fqg0RfVlctJwODwr/XTqYROLTQ53tlgsPNOGSmGBQAAXL17kkZgzXWM6e0/CWgJsdDod/vrXv0Imk2HTpk340Y9+BJvNhu7ubng8Hpw/fx5nzpxJ6Ki0o1FrELXVSKVSdHZ2Jn0/Go0Gg4ODyMzM5DBFqLJGnEqxWMxdCTRXcjoVu2Ts05/+NHQ6HZYsWYKKigqMjo5ibGwM165dQyAQwI4dOxAIBHDhwoUZyx4GgwGZmZk4ePAgKy1EIhGe3kxaQHl5edDpdDwdbTojvnFWVhaKi4tx+fJlqNVqZhdR/yadpNnZ2aivr8f58+dndXxabDk5OQgEAnwd6lOltEY4v5Vqn8kavZdIJILu7m44nU4sWbKEi/+p5LBCUIkoooFAAFlZWfz9hcJeM9lMDiWRSJi/DVw/tIqKiuKULITXme3Aes9yTr1ej4qKCkSjUZSVlUGv16OhoQELFizgYu5LL72Ec+fOJcxHhacX1fFUKhXUajUqKipw6dKlWRNsQjmdTic79/bt2/k0NhgM8Hq93M1w4sQJ3pUpdKIm32RMJBLBYrEwiV0ulyM3N5clQlavXg2lUokjR47A7/fPGOLJZDKmLl6+fBn19fU85ZmK4729vWhubsa8efNm3eGJI0z6QefPn+fanXBgMJ3Ownat2VTaaeI2qTIAkxtrUVERy6YS24by2cLCwpRqn0QEUSgUuHr1Ko4ePYr6+nqEQqGUUGXgOrfYaDRya10oFIJKpWJyC0UosxFQsrOzUVRUxM3h69evx1tvvYWTJ0+ioqKCS1QymQxVVVW4du0aampqsHLlSuYre71eXLp0KSHw9p44p0ajQVlZGeukkBIfhQ6EPF69ejVhKEKd7LRwDAYD8vLyoFAooNVqk65xymQyHuEQi00KKY2NjfG4uNHRUSiVSoyMjCAYDLITFxQUYGxsLCWgAZjcDD796U8jNzeX81fhiUyk9p6eHhQXF0+7qMRiMdatW4ePfvSjPMVZyGSheZ/A5Di9j3zkIygrK0u42ZHTjIyMYPXq1fj973/PHF1yUgBMvKcRFV6vN2F7Hp1CMpmMyd80DoIiAvoOgUCAR8mnKpE5PDzMoWdTUxMaGxu5pc9ut6d0ChMYRvpNwOSmKtTXJcQ2kXNWVVXhl7/8JYDJZmmJRAKPx4N58+ZBqVTinnvuYR4tnY7Hjx+H1WpFQUEBHA4HvxeZTPauriWhvSfO+dGPfhT19fXweDzIzs7GyZMnMTAwgMLCwrjm09lGu0ulUuTm5iIrKwsKhQImk4kdPTs7G06nM26RzmT0eaFQiBdJXl4eQqEQhoeHOf958803OdQhnSL6+WSMdtu5c+di+/btTJqgz6VrUyd9XV0dS65MNYL6SaFt/fr1fB/EBgoGg7yJSKVSbNy4Efv27ZtxMVGIRjRIIl1TOEvOT85NKgvER6UNbrrvTZsvteONjo4y6EbEiXA4jJycHHbMVBFwGiBEOeLw8DA7kNPpTCmsFXblEAI+NDSEjo4OHndBzyxR1DAxMYGLFy+ioqICGRkZcLvdjOzOnTsXLpeLN1ZaBw0NDYjFYhgZGeHnT5GK0Wic8bNuyDnFYjEaGhpQVVUFuVweJ9ExMjKCs2fPIhqNsrARyUfOFC5SUk11IqvVCrPZzF+EegZn06Wh/JH+TYvT4/EwPA9MIqtyuZxDRQpJhT2lQOL8QKVS4fHHH+deSwrJSd6TTk63243CwkL4/f5pBw4Fg0G89NJLOHz4MHQ6HR544AGUlpbC5XJxV0ooFILJZMIvfvEL7NmzBzabLeGzoGdA6CRtjiT+LBxuTCEosZB8Pt+M4lOUd2m12jgFuUgkgpKSErS2tvI9j4yMwGq18vS4VGxsbIzzbeB6Zw1tdqk4+8TEBEZGRnjTUCgUGBwcxIEDB7Bt27Z3IdgzGf2+0WiEUqmE3W5HIBCAQqFgpUYK+UmkzufzMYYiBCfPnTv33ktjUjy9fPly1NbWstIYTU7WaDTw+/3IyMhgkIV0WRIhVFKpFEajkZHI4uJiVmCjzozMzMxZObb0OTRZOhwOc1hRWFgIg8EAn8+H/v5+OJ1ONDY24vLlyyySPZWEIJFIuHGZmpWNRiMaGhqwZcsWVhOgzYkI7NS0TQR2cpaZapSBQADt7e2QSqUoLCxkRI96MGmhqlQqvP7667O+J0oRaMMEENd+Rj2y9DlutxuBQAA5OTlwu90JndPv96OwsBA+nw+BQIDD2p6eHpYTodCZupdSrX2PjY3xcxM6okgkgsfjSel64XCYKYoUlvt8Ply6dInTBgqTE9ECzWYzFi5cyNEF9dcSy4iuD1yvDAild2gNiEQiLF68GHv27Jnxs1J2TvqAOXPmoKqqimtktHvSixJOm6ITczoJfaHRCWkymaDRaJCfn8/d+ZSPpjLmLisriydEnTp1Cl6vFxcvXuR6Fi1OatkiUS4Kf2hBbN26FR//+McxOjqKQCCAhQsXwmw2M6VQiPIFg0FGL4UiXqRnKxKJWHB56uIS0hWzsrJ45yWHEovF8Hg8qKqqSur7q9VqDtcHBwc5h6QwkfIsun+Hw8HynfSeZ3pPVCYoKCjA0NAQb7qkSieckRIMBlkeNBWjTZ+QVGrNSgbpnM6InkmbEzWSU78wAWMz5cb5+flYvnw5pxfj4+Nwu90cBgvVC2nd0ztVKBTcf0vXn605JKFz0u5P7TYkTpyVlYWioiJ4vV4+HSh0pWNfo9HwnEO6Ib1ez0SA6Yy+iNlshkqlgk6nY2BFCELMZnRPNCrCbDbj7Nmz3NI2VWdGLJ4UnR4dHUV1dTUvTNoN161bB5PJBJPJxE5rs9kYTaYTinIxv9/PITKFuJFIhE8A0iKaacSB8DsIW8To7/V6/azPAABT1DIyMnDs2DEOW4VEeuEzcLvdGBkZ4VB6JmeaSg4gUoTf74fJZGJiOREfqO5Hv5esYwnLD0LV/2R1pKZaIBDgtUSMKQKAKKRNJFq+fPlyPPTQQ/D7/cjMzMTQ0BA/I3r/9M5ozQvTG3r2BDiSbOZMltA5xWIxSkpKkJ+fzxejnIpU5ig8ooI5LUr+AIEIMoWFiT4vOzsbeXl5vBE4nc44acpkCAJ0ygCTcDw9DAqzqd5JyB9NxnI6nXGbDABUVFRAJpOhu7sbeXl5yM3N5QGrwnFxOp2OXw45Ky0AOvlp96fcdKoJ/x/p+9BzEbJLkiVJ0PvS6XQ4evRonFPTAqXohu6Nyh7CEsl0RguQwB4iJdDmSc3xRBag8Fmotzub0alJ/GL6PcqhUzWaw0PDoOh7kDwL1U9nMho/mZWVxbkpdQ/RQUanI60Lul8h44siC9oQZrKEb1mn02HevHkA4kEEAllIMFnI3KAFSS+Fcj9qz0n0YpRKJUpKSqDT6XhXIRkMlUoFu90+K1IrEomgVqsBXO+BpAVD4QvVMIXfRVi+kMvl0Gq18Hq9GBoagkKh4DzswoULTATIzs7mUX1UhyT2CoUuFD7SIpuYmIjT/BWaEIAigjq9aCE5O1lgRSqVYnx8nJXLc3NzWdKETjbhCU5Dd+RyOdf+ZjJahDQLBJisdVNumZmZCbF4crwGhc20BpI1UvYjiRr6/sLxF6kYqRbQ5kprkZT96d3MdLI3NTVh/fr1+P73v4/S0lJGuoVT4MjZCCehqJI2QOA6XZTWxUyW0DkLCwtRXl7OkvvRaJTVD4irSPE2hU/UnU96tvQy6IinFzid0QIHwJ0Ig4OD6O/vh06nQ0tLS1JjHqhkQOUC4fQzOnWI0UIADnDdkYSixENDQ3jkkUewbds21NbWoqCgAGazGePj4xy+0vMhJxfS7ajBl3bVSCQSp3gwnVHYTeQD4DrIQMyWZIzyH41Gg6GhIdTV1cHlckGhUExby6Vmd4pSZrpH2jCodELPjKQ8qa5MCHM4HGZl/FSck0462kiE95lOS5fL5YLD4WAZHHIMmqpGof1s1+jt7UVZWRnnpxTS0slILCQ6FCi0FSoqTExM8OS4mSyhc3Z1daGxsRELFixgAjV9ITpJKfElriYtVkp2ycmIKZKbmzujc9IXcTqdLIpst9sxNDQEj8eDoaGhpF4KPQnfX9QAAARVSURBVIRQKMSfS7sbPTDqhg+FQgw60c5MiBqZ2+3G7t27WSGdQm+TycSh8tQiu9CZyDHdbjecTicuXLiQEAigmpjVao3LOylySXaYk3ABUkpAmICQT0zfXSKRYHBwEDk5OWhubk54f9S+JfzuEokEFRUV6O7u5hKPsPxB6yVZo8U7MTHBaDPloekAQoFAgDWD6JkAkySNgoICiESiWZvCA4EArl27hmXLlgEAR0rkfBTO0kkszEEpUrt06RLOnz+P1atXJ/yshM7pdDpx/PhxnD17FpmZmTAYDJg3bx58Ph/y8/OhVqvR29uLrKwsdHZ24urVq/D7/cjOzsa9996LkZERrv+5XC54vV5cu3Zt5pv5v1PNbrcjGAzC6/XC4XBw4p1MG49YPNlUTDs1yUDSgqbQOhqdHKBLOzwREZRKJXJyctDZ2clIL5nL5WJUmk5HITgylTs51chZkikBBAKBd4Ey5JzJqhZQONnW1gan0xl3n/SdyUEpirh48SLWrl2bMKcTOic9SwLs2tvbOWKif9PEa2GNLxmj6CsQCDCHeHh4GFqtNq2BvD6fL462SM5JABgwOf5vJsenMDYjIwOZmZnweDzIz8+PA8eEz5OwBvpdwiO6u7uZwnlD6nvRaJTzS7vdntSUL5fLhW9/+9uz/txUm5iYQE9PD9dGSVCZmEGDg4Oz5hq0cymVSmaUCBcR/UwoFOLZjsS1JCdOJnSeLZm/USNHJHSYFr9EMjnzMhnUkyZZdXR0xJ1gZEIpRwpRBwcH+bvN9KwJRwDAiCfloBSl0HsQopmz1bmnM5r7QhtzV1cXKisr056WTfcLXH8WQlQ4UVhLP7Nnzx6cOXOGyfyEOwhNWKoC8K4ZO5mZmfjd736H/v7+GT/vQ6WEQIViCpNpBoZarWYwYTakkvJWGr/n9XqhVqs5jFQqlZxzUS2SyjxEYE8mvHm/zefzce2NCOk6nY7rZdOd1lONsAEiiRNQI0xJRCIRl1xGR0eZGUQA1nRGdW1SSNRqtXyq5OTkYHh4mPNviWRyJKTb7Y4TLEvWiIRP81eampqQm5ubltgb1dqpl5fQcJrVkqxaw9jY2N9Ej/dD5ZwEZcdiMVgsFjgcDmg0GixZsgSvvfZaUvIh0WgUbW1tGBwchMPhgNvtxr59+ziUJDQZuF5LpHzM5/PhyJEjkMvlH6hyXDgcxu7duxEIBDAxMQGbzYZQKMQN7EQAn+3kdDqdcSnCb37zGxQXF6O8vJwRRJfLhdHRUTQ3N6Ojo4M5zBRSTme0oY2Pj6O7u5tD/DVr1uD48eOYmJjAmjVr8NZbb8UBZunYf/3Xf6GgoIDD2pdeegn9/f1paSPHYjH86U9/glKphFarZSbQG2+8gaKiIrjdbly+fDmt+3w/LKH63k27aTftg7MbVkK4aTftpr0/dtM5b9pN+5DaTee8aTftQ2o3nfOm3bQPqd10zpt20z6kdtM5b9pN+5Da/wJGbBMwDoeNQQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labels\n",
      "MNIST: [0 1 2 3 4 5 6 7 8 9]\n",
      "FashionMNIST: [10 11 12 13 14 15 16 17 18 19]\n"
     ]
    }
   ],
   "source": [
    "plt_img = np.zeros((280, 280))\n",
    "for i in range(10):\n",
    "  for j in range(10):\n",
    "    plt_img[i*28:(i+1)*28, j*28:(j+1)*28] = np.squeeze(x_train_fmnist[i*10+j])\n",
    "plt.imshow(plt_img, cmap=\"gray\")\n",
    "plt.axis(\"off\")\n",
    "plt.show()\n",
    "\n",
    "print(\"Labels\")\n",
    "print(\"MNIST: \"+str(np.unique(y_test_mnist)))\n",
    "print(\"FashionMNIST: \"+str(np.unique(y_test_fmnist)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "W6SBIqCcOdWe"
   },
   "source": [
    "Now the data is prepared we need to define a new model that can classify images into $20$ different classes. This is necessary since the models we used up to this point only have $10$ neurons in their output layers and therefore are only capable of classifying into $10$ different categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8WC9fu-hOzVH"
   },
   "outputs": [],
   "source": [
    "class MyExtendedModel(k.Model):\n",
    "    def __init__(self):\n",
    "        super(MyExtendedModel, self).__init__()\n",
    "        self.conv0 = k.layers.Conv2D(8, 3, 2, activation=\"relu\")\n",
    "        self.conv1 = k.layers.Conv2D(16, 3, 2, activation=\"relu\")\n",
    "        self.flatten = k.layers.Flatten()\n",
    "        self.dropout = k.layers.Dropout(0.25)\n",
    "        self.dense0 = k.layers.Dense(128, activation=\"relu\")\n",
    "        self.dense1 = k.layers.Dense(64, activation=\"relu\")\n",
    "        self.dense2 = k.layers.Dense(20, activation=\"softmax\")\n",
    "\n",
    "    def call(self, inputs, training=False):\n",
    "        output = self.conv0(inputs)\n",
    "        output = self.conv1(output)\n",
    "        output = self.flatten(output)\n",
    "        output = self.dropout(output, training)\n",
    "        output = self.dense0(output)\n",
    "        output = self.dense1(output)\n",
    "        output = self.dense2(output)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "a9xyVgZrQNjg"
   },
   "source": [
    "With this model we are almost redy to start training on a sequence of tasks, i.e. we will first train on the MNIST data set in order to learn the classes $0$ up to $9$ and after that train on the FshionMNIST data set in order to learn the remaining classes from $10$ to $19$. During this process we want to evaluate the model separately on MNIST and FashionMNIST validation data. Since this is not a standard procedure, we need to implement a Callback class. Callbacks in Keras are used to implement actions that are executed at different points during the training process, e.g. at the beginning of the training, after a batch is processed or at the end of each epoch. See the [documentation](https://www.tensorflow.org/versions/r2.2/api_docs/python/tf/keras/callbacks) for an overview of all provided callbacks and the [guide](https://www.tensorflow.org/beta/guide/keras/custom_callback#an_overview_of_callback_methods) on writing custom callbacks. While there are predefined callbacks, e.g. for logging metrics into a tensorboard log file, there is unfortunately no callback that we could use to evaluate our model simultaneously on two different validation data sets. The only option we have is to implement our own callback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7bOiLxIRSCz5"
   },
   "outputs": [],
   "source": [
    "class MyCallback(tf.keras.callbacks.Callback):\n",
    "  # Get the two different data sets and create lists for storing results\n",
    "  def __init__(self, x_0, y_0, x_1, y_1, batch_size):\n",
    "    super(MyCallback, self).__init__()\n",
    "    self.x_0 = x_0\n",
    "    self.y_0 = y_0\n",
    "    self.x_1 = x_1\n",
    "    self.y_1 = y_1\n",
    "    self.loss_0 = []\n",
    "    self.acc_0 = []\n",
    "    self.loss_1 = []\n",
    "    self.acc_1 = []\n",
    "    self.batch_size = batch_size\n",
    "    \n",
    "  def on_epoch_end(self, epoch, logs=None):\n",
    "    # Evaluate the model on both data sets and store results\n",
    "    print(\"\\nStarting callback...\")\n",
    "    print(\"+----------------------+\")\n",
    "    print(\"| Data set 0           |\")\n",
    "    print(\"+----------------------+\")\n",
    "    metrics_0 = self.model.evaluate(self.x_0, self.y_0, self.batch_size)\n",
    "    self.loss_0.append(metrics_0[0])\n",
    "    self.acc_0.append(metrics_0[1])\n",
    "    print(\"+----------------------+\")\n",
    "    print(\"| Data set 1           |\")\n",
    "    print(\"+----------------------+\")\n",
    "    metrics_1 = self.model.evaluate(self.x_1, self.y_1, self.batch_size)\n",
    "    self.loss_1.append(metrics_1[0])\n",
    "    self.acc_1.append(metrics_1[1])\n",
    "    print(\"Callback completed...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "h3ZUw3WqWmSL"
   },
   "source": [
    "This callback will accept two data sets on its initialization that can be used during training to evaluate on. For this we override the on_epoch_end function to evaluate on both data sets, print the results and store them for later use. We now train our neural network first on MNIST in order to learn the classes $\\lbrace0,\\ldots,9\\rbrace$ and then on FashionMNIST in order to learn the classes $\\lbrace10,\\ldots,19\\rbrace$  while simultaneously evaluating the model on both, the MNIST and FashionMNIST test sets. If our neural network would be capable of learning continually, we would expect to see the loss on MNIST decrease as we train on it and remain low as we continue with the training on FashionMNIST. Similarly we would expect the accuracy to rise on MNIST as we train on it and remain stable even if we continue to train on FashionMNIST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5775197,
     "status": "ok",
     "timestamp": 1562676418446,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "n3tZfi6jVSec",
    "outputId": "a37bfecc-f939-4100-af5b-cddb0be8f13d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "465/469 [============================>.] - ETA: 0s - loss: 0.5914 - accuracy: 0.8643\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.1235 - accuracy: 0.9595\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 14.8833 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.5879 - accuracy: 0.8650\n",
      "Epoch 2/20\n",
      "461/469 [============================>.] - ETA: 0s - loss: 0.1416 - accuracy: 0.9561\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0833 - accuracy: 0.9737\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 14.4861 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.1416 - accuracy: 0.9561\n",
      "Epoch 3/20\n",
      "462/469 [============================>.] - ETA: 0s - loss: 0.1004 - accuracy: 0.9681\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0720 - accuracy: 0.9757\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 15.2568 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.1009 - accuracy: 0.9681\n",
      "Epoch 4/20\n",
      "456/469 [============================>.] - ETA: 0s - loss: 0.0876 - accuracy: 0.9728\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0674 - accuracy: 0.9777\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 14.6159 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0873 - accuracy: 0.9728\n",
      "Epoch 5/20\n",
      "453/469 [===========================>..] - ETA: 0s - loss: 0.0763 - accuracy: 0.9758\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0646 - accuracy: 0.9786\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 20.7221 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0762 - accuracy: 0.9760\n",
      "Epoch 6/20\n",
      "466/469 [============================>.] - ETA: 0s - loss: 0.0658 - accuracy: 0.9789\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0662 - accuracy: 0.9809\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 17.9167 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0658 - accuracy: 0.9789\n",
      "Epoch 7/20\n",
      "460/469 [============================>.] - ETA: 0s - loss: 0.0587 - accuracy: 0.9812\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0582 - accuracy: 0.9822\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 21.5782 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0584 - accuracy: 0.9812\n",
      "Epoch 8/20\n",
      "469/469 [==============================] - ETA: 0s - loss: 0.0568 - accuracy: 0.9819\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0523 - accuracy: 0.9842\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 18.8078 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0568 - accuracy: 0.9819\n",
      "Epoch 9/20\n",
      "468/469 [============================>.] - ETA: 0s - loss: 0.0518 - accuracy: 0.9830\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0476 - accuracy: 0.9856\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 18.5827 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0518 - accuracy: 0.9830\n",
      "Epoch 10/20\n",
      "461/469 [============================>.] - ETA: 0s - loss: 0.0503 - accuracy: 0.9845\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0556 - accuracy: 0.9843\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 24.6296 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0503 - accuracy: 0.9845\n",
      "Epoch 11/20\n",
      "448/469 [===========================>..] - ETA: 0s - loss: 0.0477 - accuracy: 0.9851\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0520 - accuracy: 0.9856\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 19.0023 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0481 - accuracy: 0.9849\n",
      "Epoch 12/20\n",
      "459/469 [============================>.] - ETA: 0s - loss: 0.0462 - accuracy: 0.9860\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0578 - accuracy: 0.9847\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 23.5102 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0459 - accuracy: 0.9862\n",
      "Epoch 13/20\n",
      "462/469 [============================>.] - ETA: 0s - loss: 0.0438 - accuracy: 0.9861\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0571 - accuracy: 0.9850\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 26.4608 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0436 - accuracy: 0.9862\n",
      "Epoch 14/20\n",
      "447/469 [===========================>..] - ETA: 0s - loss: 0.0404 - accuracy: 0.9873\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0511 - accuracy: 0.9859\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 26.3339 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0406 - accuracy: 0.9873\n",
      "Epoch 15/20\n",
      "451/469 [===========================>..] - ETA: 0s - loss: 0.0397 - accuracy: 0.9869\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0521 - accuracy: 0.9851\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 24.0854 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0398 - accuracy: 0.9869\n",
      "Epoch 16/20\n",
      "469/469 [==============================] - ETA: 0s - loss: 0.0382 - accuracy: 0.9885\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0561 - accuracy: 0.9848\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 20.9028 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0382 - accuracy: 0.9885\n",
      "Epoch 17/20\n",
      "456/469 [============================>.] - ETA: 0s - loss: 0.0367 - accuracy: 0.9884\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0568 - accuracy: 0.9847\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 21.9710 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0373 - accuracy: 0.9883\n",
      "Epoch 18/20\n",
      "469/469 [==============================] - ETA: 0s - loss: 0.0372 - accuracy: 0.9884\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0493 - accuracy: 0.9859\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 24.4688 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0372 - accuracy: 0.9884\n",
      "Epoch 19/20\n",
      "462/469 [============================>.] - ETA: 0s - loss: 0.0360 - accuracy: 0.9891\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0542 - accuracy: 0.9858\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 27.9090 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.0363 - accuracy: 0.9890\n",
      "Epoch 20/20\n",
      "468/469 [============================>.] - ETA: 0s - loss: 0.0351 - accuracy: 0.9890\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.0499 - accuracy: 0.9865\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 23.9937 - accuracy: 0.0000e+00\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.0352 - accuracy: 0.9890\n",
      "Epoch 1/20\n",
      "466/469 [============================>.] - ETA: 0s - loss: 0.8005 - accuracy: 0.7543\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 28.9105 - accuracy: 0.0011\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.4612 - accuracy: 0.8338\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.7984 - accuracy: 0.7546\n",
      "Epoch 2/20\n",
      "446/469 [===========================>..] - ETA: 0s - loss: 0.4429 - accuracy: 0.8406\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 32.9707 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3895 - accuracy: 0.8615\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.4423 - accuracy: 0.8409\n",
      "Epoch 3/20\n",
      "461/469 [============================>.] - ETA: 0s - loss: 0.3900 - accuracy: 0.8575\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 44.7675 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3715 - accuracy: 0.8666\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.3896 - accuracy: 0.8576\n",
      "Epoch 4/20\n",
      "461/469 [============================>.] - ETA: 0s - loss: 0.3594 - accuracy: 0.8680\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 42.1162 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3536 - accuracy: 0.8714\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3590 - accuracy: 0.8680\n",
      "Epoch 5/20\n",
      "467/469 [============================>.] - ETA: 0s - loss: 0.3380 - accuracy: 0.8748\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 40.2410 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3418 - accuracy: 0.8742\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3380 - accuracy: 0.8748\n",
      "Epoch 6/20\n",
      "460/469 [============================>.] - ETA: 0s - loss: 0.3212 - accuracy: 0.8814\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 48.1656 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3375 - accuracy: 0.8784\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3209 - accuracy: 0.8816\n",
      "Epoch 7/20\n",
      "464/469 [============================>.] - ETA: 0s - loss: 0.3071 - accuracy: 0.8873\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 45.3209 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3367 - accuracy: 0.8753\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3071 - accuracy: 0.8872\n",
      "Epoch 8/20\n",
      "446/469 [===========================>..] - ETA: 0s - loss: 0.2978 - accuracy: 0.8896\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 51.7852 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3357 - accuracy: 0.8799\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2973 - accuracy: 0.8899\n",
      "Epoch 9/20\n",
      "468/469 [============================>.] - ETA: 0s - loss: 0.2900 - accuracy: 0.8927\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 49.1386 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3143 - accuracy: 0.8867\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2902 - accuracy: 0.8926\n",
      "Epoch 10/20\n",
      "465/469 [============================>.] - ETA: 0s - loss: 0.2807 - accuracy: 0.8944\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "79/79 [==============================] - 0s 1ms/step - loss: 47.8905 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3205 - accuracy: 0.8827\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2810 - accuracy: 0.8942\n",
      "Epoch 11/20\n",
      "468/469 [============================>.] - ETA: 0s - loss: 0.2748 - accuracy: 0.8972\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 56.5178 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3098 - accuracy: 0.8911\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2745 - accuracy: 0.8973\n",
      "Epoch 12/20\n",
      "469/469 [==============================] - ETA: 0s - loss: 0.2648 - accuracy: 0.9007\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 53.5114 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3129 - accuracy: 0.8839\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2648 - accuracy: 0.9007\n",
      "Epoch 13/20\n",
      "445/469 [===========================>..] - ETA: 0s - loss: 0.2620 - accuracy: 0.9025\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 55.1841 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3232 - accuracy: 0.8854\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2611 - accuracy: 0.9028\n",
      "Epoch 14/20\n",
      "464/469 [============================>.] - ETA: 0s - loss: 0.2545 - accuracy: 0.9052\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 53.5365 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3124 - accuracy: 0.8886\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2547 - accuracy: 0.9051\n",
      "Epoch 15/20\n",
      "468/469 [============================>.] - ETA: 0s - loss: 0.2498 - accuracy: 0.9068\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 55.4699 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3143 - accuracy: 0.8927\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2498 - accuracy: 0.9068\n",
      "Epoch 16/20\n",
      "469/469 [==============================] - ETA: 0s - loss: 0.2460 - accuracy: 0.9079\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 60.4543 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3206 - accuracy: 0.8852\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2460 - accuracy: 0.9079\n",
      "Epoch 17/20\n",
      "466/469 [============================>.] - ETA: 0s - loss: 0.2423 - accuracy: 0.9101\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 55.9491 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3057 - accuracy: 0.8920\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2424 - accuracy: 0.9100\n",
      "Epoch 18/20\n",
      "447/469 [===========================>..] - ETA: 0s - loss: 0.2369 - accuracy: 0.9109\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 56.3408 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3191 - accuracy: 0.8882\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2374 - accuracy: 0.9107\n",
      "Epoch 19/20\n",
      "467/469 [============================>.] - ETA: 0s - loss: 0.2327 - accuracy: 0.9139\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 53.7505 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3123 - accuracy: 0.8925\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2325 - accuracy: 0.9140\n",
      "Epoch 20/20\n",
      "444/469 [===========================>..] - ETA: 0s - loss: 0.2286 - accuracy: 0.9147\n",
      "Starting callback...\n",
      "+----------------------+\n",
      "| Data set 0           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 56.2578 - accuracy: 0.0000e+00\n",
      "+----------------------+\n",
      "| Data set 1           |\n",
      "+----------------------+\n",
      "79/79 [==============================] - 0s 1ms/step - loss: 0.3087 - accuracy: 0.8913\n",
      "Callback completed...\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2282 - accuracy: 0.9148\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f7100042910>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extended_mdl = MyExtendedModel()\n",
    "extended_opt = tf.keras.optimizers.RMSprop(learning_rate)\n",
    "extended_mdl.compile(loss=\"sparse_categorical_crossentropy\", optimizer=extended_opt, metrics=[\"accuracy\"])\n",
    "\n",
    "my_cb = MyCallback(x_test_mnist, y_test_mnist, x_test_fmnist, y_test_fmnist, batch_size)\n",
    "\n",
    "extended_mdl.fit(x_train_mnist, y_train_mnist, batch_size, epochs, callbacks=[my_cb])\n",
    "extended_mdl.fit(x_train_fmnist, y_train_fmnist, batch_size, epochs, callbacks=[my_cb])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xaUL-QYT8tfy"
   },
   "source": [
    "If we now plot the loss on the test sets during the sequential training process on MNIST and FashionMNIST, we can clearly see what catastrophic forgetting means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5775958,
     "status": "ok",
     "timestamp": 1562676419217,
     "user": {
      "displayName": "Felix Wiewel",
      "photoUrl": "https://lh3.googleusercontent.com/-XBgpqQQzeMU/AAAAAAAAAAI/AAAAAAAAAZY/12qzl-EFXfM/s64/photo.jpg",
      "userId": "11509421729312238102"
     },
     "user_tz": -120
    },
    "id": "vziuxbrI9NLF",
    "outputId": "1da1a345-058f-4604-c2c8-427953aa4238"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeXgUVbr48e/bnaUTQhJCAiSEEFR2glFWcWPEUUTEZRxQlAFx9M6iOCMqjj/3KzOuVx115uq4wIwOIK6Iy3UBFJdBQoyALIImQEhYsrAEsvf5/VHVSSfprCTdHXw/z9NP115vVZJ6c6pOnSPGGJRSSqlg4wh0AEoppZQvmqCUUkoFJU1QSimlgpImKKWUUkFJE5RSSqmgpAlKKaVUUNIEpZRSKihpglKqHYlIjoicG+g4lDoeaIJSSikVlDRBKdXBRCRcRJ4QkTz784SIhNvz4kVkuYgcEJEiEVktIg573jwR2S0ih0Vkq4hMCOyRKOVfIYEOQKmfgP8HjAXSAQO8DdwJ3AXMBXKBBHvZsYARkYHADcAoY0yeiKQCTv+GrVRgaQlKqY53FXC/MWafMWY/cB8ww55XCSQCfY0xlcaY1cZqILMaCAeGiEioMSbHGPNDQKJXKkA0QSnV8ZKAHV7jO+xpAI8A24EPReRHEbkdwBizHfgDcC+wT0QWi0gSSv2EaIJSquPlAX29xlPsaRhjDhtj5hpjTgCmADd7njUZY/5tjDnDXtcAD/k3bKUCSxOUUu0vVERcng+wCLhTRBJEJB64G3gZQEQmi8hJIiLAQaxbe24RGSgi59iVKcqAUsAdmMNRKjA0QSnV/t7DSiiejwvIANYDG4BM4AF72f7Ax0AJ8BXwN2PMSqznTw8CBcAeoAfwJ/8dglKBJ9phoVJKqWCkJSillFJBSROUUkqpoKQJSimlVFDSBKWUUiooBaypo/j4eJOamhqo3SullAoS69atKzDGJNSfHrAElZqaSkZGRqB2r5RSKkiIyA5f01t9i09EYkXkNRHZIiKbReQ0EYkTkY9EZJv93e3YQ1ZKKfVT1pZnUE8CHxhjBgEnA5uB24FPjDH9gU/scaWUUu0ka9cBdh8oDXQYftWqW3wiEgOcBcwCMMZUABUicjEw3l5sIbAKmNdeQSql1E9VeVU1D3+wlRc+zyYyzMm8iYOYMbYvDocEOrQO19pnUP2A/cBLInIysA64CehpjMm3l9kD9PS1sohcD1wPkJKS0mB+ZWUlubm5lJWVtTIs5Q8ul4vk5GRCQ0MDHYpSPwk/7i9hzuJv2Lj7EFeNSWFXcSn3LPuOd9fn89Dlw+kX3yXQIXaoVjV1JCIjgf8Apxtj1ojIk8Ah4EZjTKzXcsXGmCafQ40cOdLUrySRnZ1N165d6d69O1bbmSpYGGMoLCzk8OHD9OvXL9DhKHXce31dLne9vZGwEAcP/2I45w3thTGG19blcv/yTVRWu7nlvIFcc3o/nJ28NCUi64wxI+tPb+0zqFwg1xizxh5/DTgV2CsiifaOEoF9bQmyrKxMk1OQEhG6d++upVulOlhJeRV/XJLF3KXfMqx3DO/fdCbnDe0FWH+HvxzZh49vPpszTorngXc3c/n/fsn2fYcDHHXHaFWCMsbsAXbZ3VEDTAA2AcuAmfa0mVhdWreJJqfgpT8bdbwoPlJBtTv4Gspen3uAC/+6mrezdnPzzwew6LqxJMZENFiuZ7SLf/xqJE9ekU52wREm/fVz/rZqO1XV/u2RpaMbG2/Le1A3Aq+ISBjwI3ANVqJ7VUSuxeotdGr7haiUUu1n3Y4irnxuDYmxLmaNS+WXI/sQFR6wV0IBcLsNz3/+Iw9/sJUeXcNZ8l+nMSo1rsl1RISL03sz7sR47n57Iw9/sJX3N+xhyslJHC6voqSsisNllRwuq6KkvHb4cHkV/eK7cPmIZC5MS6RLK4+9vKqalVv28XrmbuKjwvjLZcOP5dCbFLDuNnw9g9q8eTODBw8OSDweIsJVV13Fyy+/DEBVVRWJiYmMGTOG5cuXs2DBAmbPnk1WVhbDh1s/mGHDhrF8+XJSU1NrXkCOj49n/vz5/Pvf/8bpdOJwOHj22Wd58MEHyc7OpqSkhP3799c8z/nb3/7GuHHjAnbcLRUMPyOl2mrfoTIufOpzXKEOenR1sW5HMV3DQ5g2qg8zx6XSJy6yxdvKO1BK5s5ienR1MTw5Bleos1WxGGPILjjCp9/vZ/n6fNbtKOb8oT156BfDiY0Ma+2h8d6GfO5+eyMFJRUARIWH0NUVUvvtCqWrK4QuYU7W5hSTXXCEyDAnk9IS+eWIZEb3i2v0LokxhsydB3gjM5fl6/M5WFpJQtdwrhydws0/H9DqWOtr7BlUYP9tCEJdunRh48aNlJaWEhERwUcffUTv3r3rLJOcnMz8+fNZsmRJo9v56quvWL58OZmZmYSHh1NQUEBFRQVvvvkmAKtWreLRRx9l+fLlHXo8SilLRZWb376SSUlZFf+6dhyDekWTtesAL36ezYIvc3jxi2zOH9qLa8/ox4i+3epcrI0x7Cw6yprsItb8WMSa7EJyi2vfSQoLcZCeHMvofnGM7hfHqX27+SyVHSmv4qsfCvn0+/2s+n4fu4qsbZyQ0IU/X5rGlaP7tPlW+qS0RM4d3JOyqmq6hIU0WXHCGMO6HcUszchl+fo8XluXS9/ukVx+ajK/GJFMUqx1W3FX0VHe/GY3b2TmklN4FFeog/OH9uKyU5M5/cTuhDg7tjlXTVA+TJo0iXfffZfLL7+cRYsWceWVV7J69eqa+ZMnT+azzz5j69atDBw40Oc28vPziY+PJzw8HID4+Hi/xK6U8u2/l29i3Y5i/nrlKQzqFQ1Aep9Y/nrlKfxp0iD++dUO/r1mJ+9v3MPw5BhmnpZKeZWbNdmFrPmxiD2HrApCcV3CGJ0aV5PI9h4q5+vsQr7OKebvn/7A0yu343QIQ5OiGZ0ax4i+3dhVfJRPv9/P2uxiKqrdRIY5GXdid64/60TO7p9ASveWl9yaEhbiICyk+aQhIoxMjWNkahz3TBnC+xv28Nq6XB776Hv+5+PvOeOkeMqr3HydXQTA2BPi+N3PTuKCYb3o6vLfayZBm6Due+c7NuUdatdtDkmK5p6Lhja73BVXXMH999/P5MmTWb9+PbNnz66ToBwOB7fddht//vOfWbhwoc9tnHfeedx///0MGDCAc889l2nTpnH22We327EopVpuacYu/vWfHVx3Zj+mnJzUYH5iTATzJg7ixnNO4o3M3bz4RTZzl34LQELXcMb0i2PMCd0Z2y+Ok3pENSjl/HyI9ernkfIqMncW83V2EV9nF/HP/+zg+c+zARjYsyuzTk9l/IAERqR2IzykdbcEO0pkWAi/GGGVnHYVHeW1dbm8lbWbEIdw6/kDuTg9ieRu7ZNAWytoE1QgDR8+nJycHBYtWsSkSZN8LjN9+nTmz59Pdna2z/lRUVGsW7eO1atXs3LlSqZNm8aDDz7IrFmzOjBypVR9G3IP8v/e2si4E7szb+KgJpeNDAvh6rF9mT46hW92FRPXJZzU7pEtvu3WJTyEM/sncGZ/q2Hu8qpqvss7RK9oV81ts2DWJy6SP/58AH9sh+dK7SFoE1RLSjodacqUKdxyyy2sWrWKwsLCBvNDQkKYO3cuDz30UKPbcDqdjB8/nvHjx5OWlsbChQs1QSnlwzc7i7n99Q2k94ll6qg+nJoS2y6vNRSWlPObl9eREBXOU1ee0uJnJg6HMKJv07XoWiI8xMmpKdp2dlsFbYIKtNmzZxMbG0taWhqrVq3yucysWbN4+OGHOXy44UtyW7duxeFw0L9/fwCysrLo27dvR4asVKf05Q8F/HphBl3CQ3hnfR5LMnbRv0cU00b14dJTetM9KrxN262qdnPjom/YX1LOa785rc3bUYGjCaoRycnJzJkzp8llwsLCmDNnDjfddFODeSUlJdx4440cOHCAkJAQTjrpJJ577rmOClepTmnFlr385uVMUrtH8vK1Y4gMD2H5t1aSeuDdzTz0wRbOHdyTqaP6cFb/hFY16fPw/23lyx8Kefjy4QxPjm1+BRV09D0o1Sr6M1Lt5Z1v8/jjkiwGJ0azcPZo4rrUfffn+72HeXXtLt74ZjdFRypIjHHxi1OTOe3E7gzq1bXJEtE73+Zx46JvuHpsCg9cktbRh6KOkb4HpZTqEIfLKltd9XjJ2p3c/sYGRvWN44VZI32uP6BnV+6cPITbJg7i4817WbJ2F8+s2s7TK7cD0KNrOIMTo+1PVwYnRnNCfBe27y/httfWM6JvN+6eHNhn2erYaIJSSrXZG5m5zF36Lel9YvnVaX25YFhisy0qvPB5Nv+9fBNnDUjg2atHEBHW9PJhIQ4mpSUyKS2RoiMVbMo7xOZ8+7PnMF/+8COV1aZm2XCngyhXCH+76tQWvROkgpcmKKVUm3z1QyHzXl/PsKQYDh6t5I9LvuW/l29m2qg+TB+d0qDZIGMMf/1kO49//D0XDOvFE1ekt/pdoLguYZzRP54z+te++F5R5eaH/SVszj/Elj2H2V1cyvVnnUDPaFe7HKcKHE1QSqlW277vMP/1rwz6du/Cy78eQ7QrhC+2F/Kv/+Tw7Kc/8L+f/sA5A3sw47S+nNU/ARH4y/tbeO6zH7ns1N48/Ivh7dZMTliIo+ZWnzq+aIJSSrVKQUk51yxYS1iIg5dmjSImwnp+5CnZ5B0oZdHXO1n09S4+eWktfbtHclJCFJ9s2cevTuvLvRcN/Ul0V66Ond6gVeo4s/dQGfe/s4nV2/bjbuc+j8oqq/n1wgz2Hy7n+ZmjfLb+nRQbwdzzBvLl7efw1ytPoUfXcD7Zso/fjj+R+6ZoclItpwmqHqfTSXp6es0nJyenVeunpqZSUFDQYPqyZct48MEH2xRTTk4OIsKdd95ZM62goIDQ0FBuuOEGAO69914iIyPZt6+2M+OoqKgGw263mzlz5jBs2DDS0tIYNWoU2dnZjBkzhvT0dFJSUkhISGjz8avAu++d73jxi2xmvPA1Zz68kic+/p7dB0qbX7EZbrfhj0uy+Db3AE9MO4X0Pk2/WxQW4mDKyUks/c04vr37POZNHKSdXqpW0Vt89URERJCVldXu250yZQpTpkxp8/r9+vXj3Xff5YEHHgBg6dKlDB1atwptfHw8jz32WJPNLy1ZsoS8vDzWr1+Pw+EgNzeXLl26sGbNGgAWLFhARkYGTz/9dJtjVYHz5Q8FvLdhDzeecxL9e3bl1bW7eOLjbTz5yTbO7J/AFaP6cO7gnm2q3fbQB1t4f+Me7rxwMBOH9WrVujGR/msBWx0/tATVjJKSEiZMmMCpp55KWloab79t9WZ/5MgRLrzwQk4++WSGDRtWp2+op556qmb5LVu2ANaF31PaycnJ4ZxzzmH48OFMmDCBnTt3AlbTSXPmzGHcuHGccMIJvPbaazXbjIyMZPDgwXhebl6yZAlTp9btuHj27NksWbKEoqKiRo8nPz+fxMREHA7rR5+cnEy3btpW2PGgqtrN/e9sIrlbBL//2UlMOTmJl389htW3/Ywbz+nPtr2H+d0rmYz9yyf89/JNbN1zuMVddr+yZgfPfvYjM8b25doz+nXwkShlaVMJSkScQAaw2xgzWUT6AYuB7sA6YIYxpuKYInv/dtiz4Zg20UCvNLig6dtspaWlpKenA1apZenSpbz55ptER0dTUFDA2LFjmTJlCh988AFJSUm8++67ABw8eLBmG/Hx8WRmZvK3v/2NRx99lOeff77OPm688UZmzpzJzJkzefHFF5kzZw5vvfUWYCWQzz//nC1btjBlyhQuv/zymvWuuOIKFi9eTM+ePXE6nSQlJZGXl1czPyoqitmzZ/Pkk09y3333+Ty+qVOncsYZZ7B69WomTJjA1VdfzSmnnNKKk6iC1eK1u9iy5zB/v+rUOu8i9YmL5OafD+CmCf1ZvW0/r2bs4p9f5fDC59n0iYvgzP4JnNU/ntNOjK+p8OBt1dZ93P32d/xsYAL3XDREb9Mpv2nrLb6bgM2Ap17nQ8DjxpjFIvK/wLXA39shPr+rf4uvsrKSO+64g88++wyHw8Hu3bvZu3cvaWlpzJ07l3nz5jF58mTOPPPMmnUuu+wyAEaMGMEbb7zRYB9fffVVzfQZM2Zw22231cy75JJLcDgcDBkyhL1799ZZb+LEidx111307NmTadOm+Yx/zpw5pKenc8stt/icn5yczNatW1mxYgUrVqxgwoQJLF26lAkTJrTwDKlgdPBoJY99uJWxJ8Q1evvN6RDGD+zB+IE9KCwp572Ne1j9/X6WZeXx7zU7cYjVgd+Z/RM4a0A8JyfH8v3eEn7/SiYDe3blqemndngPqkp5a3WCEpFk4EJgPnCzWP9OnQNMtxdZCNzLsSaoZko6/vLKK6+wf/9+1q1bR2hoKKmpqZSVlTFgwAAyMzN57733uPPOO5kwYQJ33303QE0vuk6nk6qqqlbtz7Mu0OD2S1hYGCNGjOCxxx5j06ZNLFu2rMH6sbGxTJ8+nWeeeabJfVxwwQVccMEF9OzZk7feeksTVCf3+Mffc7C0knsuGtqiEk73qHBmjO3LjLF9qax2k7XrAKu/389n2wr46wrrmVVXl9VteFdXKC/OGuWzC3OlOlJbfuOeAG4Dutrj3YEDxhjPlTgX6O1rRRG5HrgeICUlpQ279r+DBw/So0cPQkNDWblyJTt27AAgLy+PuLg4rr76amJjYxvcxmvKuHHjWLx4MTNmzOCVV16pU/pqzty5czn77LOJi2u8r5qbb76ZUaNG+UyOmZmZ9OrVi6SkJNxuN+vXr2f48OEt3r8KPt/vPcy//rOD6WNS2vSyaqjTwajUOEalxnHzeQM5cLSCL7YXsnrbfrbsOcyfL02jV4y2yqD8r1UJSkQmA/uMMetEZHxrd2aMeQ54DqzWzFu7fiBcddVVXHTRRaSlpTFy5EgGDbJ65NywYQO33norDoeD0NBQ/v73lhcYn3rqKa655hoeeeQREhISeOmll1q87tChQxvU3qsvPj6eSy+9lMcff7zBvH379nHddddRXl4OwOjRo2sqb6jOxxjDfy/fRJcwJzf/fGC7bDM2MowLhydy4fDEdtmeUm3Vqu42ROQvwAygCnBhPYN6Ezgf6GWMqRKR04B7jTHnN7Ut7W6jc9KfUXD5aNNervtnBvdcNIRrTtfadapzaqy7jVY98TTG/MkYk2yMSQWuAFYYY64CVgKe6mYzgbePMV6lVDPKq6p54N1N9O8RxdVjtbdmdfxpryo587AqTGzHeib1QjttVynViBc/z2FH4VHuvmgIoVq7Th2H2lwtxxizClhlD/8IjG6PgIwx+p5FkApU78uqoX2Hynh6xTZ+PqQnZ/ZPCHQ4SnWIoPq3y+VyUVhYqBfCIGSMobCwEJdLa3MFg4c+2EplteH/TdLnger4FVQvNiQnJ5Obm8v+/fsDHYryweVykZycHOgwfvKydh3g9cxcfnP2iaTGdwl0OEp1mKBKUKGhofTrpzWRlGqM2224d9l3JHQN54ZzTgp0OEp1qKC6xaeUatzeQ2XctCSLrF0HmDdxkLbsoI57+huuVJCrqHKz4Mtsnvx4G5Vuw00T+nPZKT4ba1HquKIJSqkgtnrbfu5d9h0/7D/CuYN7cNfkIfTtrs+d1E+DJiilglBu8VHmv7uZ9zfuoW/3SF6cNZJzBvUMdFhK+ZUmKKWCSFllNf/47EeeWbUdgFvPH8i1Z/Sr07+TUj8VmqCUChI7Co8w44Wv2Vl0lElpvfh/Fw6hd2xEoMNSKmA0QSl1rA7vhaIfIWUsHEMrKK9m7GL3gVL+de1obR1CKTRBKdV2Jfvg8ycg4wWoKoPkUTDxIUge0abNrc0uZljvGE1OStk0QSn/MAZyPoe9G8ERUvtxhtb9doRCdBL0HHpMpZEOVbIfvngC1r4A1eUwfBr0HgGfPQLPnwPDr4Bz77GOo4XKq6rJyj3Ar7RVcqVqaIJSHaviKGx4FdY8C/s2tXy9bqkw5GLrk3RqcCSrIwXwxZOw9nmrxJT2SzjrNoi3W3Q4+QpY/T/w1TOweRmc8UcYdyOENv8caUPuQSqq3Izq13hPyUr91GiC+qnzNMzb3gngYC58/Q/IXAilxdAzDS5+BgZMtPbprgJ3JVRXgrvaa7jKSmTfvWVd6L94EmJSYMgUGHKJVVJx+LkBlCOF8OVfreOpPGolprNvg/j+dZcL72qVnEbMhA/vgpXzIfOf8PP7YOhlTZ7jr3OKABjZt1tHHolSnUqretRtT7561FV+dngvLJ4O4oCpC1t1S8onY2Dnf2DN32HzcsDAoMkw5jfQd1zrk+DRItj6Pmx6G35YYSWx6N4weIqVJNr4rKdFjIHctfDtYli/BCqOwLBfwNnzIGFAy7aR8zl8cDvs2QB9xsIFD0LSKT4XveYlq/beJ3PHt98xKNVJNNajriaon6qCbfDyZdZtK3FAWBRc8e+2X/Q3vwOfPgx71oMr1ipFjPo1xKa0T7ylB+D7/4NNb8H2T6xnPynj4Iw/QP/z2q8EWJwD61+FbxdZNfNCIqzS2xk3Q49Brd+euxqyXoFP7ofyw/CHDRDVo+4ibsPJ93/I5OGJ/OWy4e1zHEp1Io0lKL3F91O08z+w6AoQJ8xaDiEua/ylC+Dip2H41JZvq2QfvDvXeuYSPwAmP25VGghr5+Z4ImLh5GnWp+yQddH/8mn491ToMQROv8kq4ThDW7/t0gNW4vt2Cez80pqWeiacOdcqrbmi2x63wwmn/gp6DYfnzobtH0P69DqLbN17mMNlVYzsq8+flPKmCeqnZvM78Pqvrdt5V78OcSdY069bBa/+Ct64DvZ+BxPuti6ujTEGNrwG798GFSXW8uNuAqcffqVc0TD2t1YJbePrVlXvN/8LVjwAp90Ap85oPEEaYyXVgu+tT/Zn1m3E6nIrwU64G9KmQmyf9o058WSI6mWVAuslqLX286fRWkFCqTpadTURkT7AP4GegAGeM8Y8KSJxwBIgFcgBphpjits31J8At9u64K5+1CrVjJhlPWsJj2qf7a95zkoovUfA9CXQJb52XpfuMONNa/4XT8D+LXDZP3yXHg7vgeV/hK3vWe/+XPwMJAxsnxhbwxlq1ZxLmwrbPrTi/mAefPoQjL4eBl0IB3bYyWhb7af8YO02Irtb5/nkaR1bW1AE+p8Lm96xKoN4lfTW5hTTK9pFcjdtNUIpb616BiUiiUCiMSZTRLoC64BLgFlAkTHmQRG5HehmjJnX1LaOu2dQRwrgrd+CKwZGXtu6VgWMsZ6rfHKv9UC95zBr+t6NENbVuuU28hrolda22Nxu+PgeqybawEnwixcgLLLx5b/+B7w/z6qlduWi2lKWMdazmQ9uh6pyOOdOGPu7pkta/rbzP1aJ6vv3607vmmQdT/wA+2MPRyf5rwr7pmXw6gyY9S6kngGAMYbT/rKCkandeHr6qf6JQ6kg0y7PoIwx+UC+PXxYRDYDvYGLgfH2YguBVUCTCeq4UpRtVTg4lAfOcNiw1HouMnK29TymqWcYuRnw8b2Qsxpi+8Jlz1vPUkSseRkvWs9bMl6wSisjZ8PQS1v0bg1gJZK3fgcbX7MS56RHmk8oo6+zLt5LZ8I/zoFfLoTuJ8I7f4DtH0HKaTDl6dr3f4JJyliYvhj2bbGSffcToHv/Y3uO1F5OGG+9jLztw5oElVtcyp5DZXp7Tykf2lyLT0RSgc+AYcBOY0ysPV2AYs94vXWuB64HSElJGbFjx462RR1M8r6BV35pvb8z/VWrBYQNr1kJJf9bq3Zc2i9h1LV1S0D7v4cV91vPhCLjrfdqRlwDIWEN93G0yKrunPEiFG6zSmknT4fU061aZqEu65ZgSLg1HhJujZtq63lTzmqYcI/14mhrSgtFP8KiK63bYqGR1vbOvRdGXef/d5GOFwsmw9FC+N1XALyRmcvNr37L+zedyeDEIEiiSgVAu1YzF5Eo4FNgvjHmDRE54J2QRKTYGNPkG4fHxS2+7R/Dkl9ZzzFmvFH3xU1jYHemlag2vm631Tbaqn6962v45mWrFDTuRjjt99ZLns3xNBe07iXrdpG7svl1HKHWM6KTp7XtGMsOwTtzoLzEKn3F9WvbdpTly6fgwzvhDxshtg9/emM9y9fnk3X3eTgdQdBahlIB0G7VzEUkFHgdeMUY84Y9ea+IJBpj8u3nVPuOLdxOIGsRLLsBEgbD1a9B115154tY7xQlj4DzHrCe3ax9Ad7+vZU0Rl8HZ94CUa1oGFQE+p1pfY4WwcFd1i28ylLru6qs9lNZBlWl0PeMY3uh1RUNv1zQ9vVVXf3PsxLUtg9h1LWszSlmZN9umpyU8qG1tfgEeAHYbIz5H69Zy4CZwIP299vtFmGwMQY+fxw+uQ/6nQ3TXm7++UZknFVKGvNb2J0BXROPvRpzZJz1UZ1L/ADrWeO2jygaMoPt+0q47NTegY5KqaDU2hLU6cAMYIOIZNnT7sBKTK+KyLXADqAVb3p2Iu5qq3bb2n9Yz5Uu/pvvZ0aNcTigz+iOi08FPxGrFJX1Cut+yAdgVKr+o6GUL62txfc50Ni9iAnHHk4ruN1WhYHwaKsEExrZsgoAZYfg0G7rc9D+PpRnrR/VA6J62h97uEu89c5KZZn1EuvmZdZzo3Pv14oCqm36nwdr/0HBxk8IC0lkeHJMoCNSKih13pYkKg7DM16lEXFaFQ1c0RAeY39HWy+5lhbXJqPyQ/U2JFYyqiqDsoM0JFYlCEcIlOyF8/8Cp/2uI49MHe/6nQkhLqJ2rSQ9+XeEhwTRe2RKBZHOm6Cc4dYLp2UHrUY4yw9ZpSPv70O51jxXrPUeT7+zIKa31SJ2dG9ruGti7Vv9lWVwZJ/VFE7JXvtjDx8psF6YHXxRYI9bdX6hEVT3PZPh29cwcvgdgY5GqaDVeRNUqAvSLm//bcamtF8L3Eo1Ykfc6ZwgH3FWfP0SvVLKQx+iKBUAq0w6ACeXfh3gSJQKXpqglAqAT/ZEsNPRh4icjwMdilJBSxOUUn5WWe3mm50H2BV/BuR8YbXSoZRqQBOUUn62Ke8QRyuqkQHnW81VZX8a6JCUCkcmhdYAACAASURBVEqaoJTyM08HhSeOmGB1p/L9/wU4IqWCkyYopfxsbU4RKXGR9OwWDSf+DLZ9ZDWhpZSqQxOUUn5kjCEjp5iRqXZj//3Pg8N5sPe7wAamVBDSBKWUH/2w/wiFRyoY7Wl/r//Pre9teptPqfo0QSnlRxn286dRnh50u/aCxJOt23xKqTo0QSnlR1/nFNG9SxgnxHepndj/PNi1xurjSylVQxOUUn60NqeIkandEO+W9/ufB8YNP6wIXGBKBSFNUEr5yZ6DZewqKm3Y/1PvERARp7f5lKpHE5RSfuJ5/6lBgnI44aRzYftHVqeYSilAE5RSfpORU0RkmJOhSdENZw44H44WQt43/g9MqSDVbglKRCaKyFYR2S4it7fXdpU6XnydU8ypKd0Icfr4szvxHBAHbPvQ/4EpFaTaJUGJiBN4BrgAGAJcKSJD2mPbSh0PDpZWsmXPodoXdOuLjIPkUbD1fThSaNXoKztoNSRbWQpV5VBdpS1OqJ+U9uqwcDSw3RjzI4CILAYuBja10/YbKK2o5vY31iOAQwTE+vaMi2B/rGlWXLXre6Z6pgkN1alp1UF8xVR/eu38xtetneY7ZuN1YWuPa1x7XiZ9n/tm9t+CANoSo/duG4uhpefPe/29h8oxhtoXdH0ZcD58cj88ckJLtu61g2McrhNsnV/IhtMaDaclfyst/HtqEKd3jL620dQPxPt4fBxjk+zt1vmB+5rmvbvGzmtzu/K1vRYcV1P7q9lmMzHXbLLBFabp5QFSxsIvX2p+uTZqrwTVG9jlNZ4LjKm/kIhcD1wPkJJybL3WVrrdZO06gDFgMLjd1nS3MRhjf1N7Yfb+2dT82DzzfGy//s/StOCqZGj5r36D/fqIr6n9tyhmTKNJr7Wp19extUcC93VeW5pYWnRJbEWMdZJ4nekN/3ab26qvYzghvgunpDRSggIYdR1EdrdKS+5qq+q5qfYatj/uaupedFo7TL3hNlyI62jJfwst/anWi9k7xprj8PmfWTP7rH88Lfxr9Zm4G0t0vvaB718g3ztrYv/eu2ns52Z8HFYT/4DU3Wgj+2hGwsCWLddGfu3y3RjzHPAcwMiRI4/pn/BoVyif3vqzdolLqaDgioYRswIdhVJBo70qSewG+niNJ9vTlFJKqTZprwS1FugvIv1EJAy4AljWTttWSin1EyQtebbSog2JTAKeAJzAi8aY+c0svx/Y0Q67jgcK2mE7/qQx+4fG7D+dMW6N2T9aEnNfY0xC/YntlqACRUQyjDEjAx1Ha2jM/qEx+09njFtj9o9jiVlbklBKKRWUNEEppZQKSsdDgnou0AG0gcbsHxqz/3TGuDVm/2hzzJ3+GZRSSqnj0/FQglJKKXUc0gSllFIqKHXaBNUZu/cQkRwR2SAiWSKSEeh4GiMiL4rIPhHZ6DUtTkQ+EpFt9ncTjcr5XyMx3ysiu+3znWW/qxc0RKSPiKwUkU0i8p2I3GRPD9pz3UTMQXuuRcQlIl+LyLd2zPfZ0/uJyBr7GrLEbmQgKDQR8wIRyfY6z+mBjrU+EXGKyDcistweb/N57pQJqpN37/EzY0x6kL/LsACYWG/a7cAnxpj+wCf2eDBZQMOYAR63z3e6MeY9P8fUnCpgrjFmCDAW+L39exzM57qxmCF4z3U5cI4x5mQgHZgoImOBh7BiPgkoBq4NYIz1NRYzwK1e5zkrcCE26iZgs9d4m89zp0xQeHXvYYypADzde6h2YIz5DCiqN/liYKE9vBC4xK9BNaORmIOaMSbfGJNpDx/G+qPuTRCf6yZiDlrGUmKPhtofA5wDvGZPD7bz3FjMQU1EkoELgeftceEYznNnTVC+uvcI6j8SmwE+FJF1dtcjnUlPY0y+PbwH6BnIYFrhBhFZb98CDJpbZfWJSCpwCrCGTnKu68UMQXyu7dtOWcA+4CPgB+CAMabKXiToriH1YzbGeM7zfPs8Py4i4QEM0ZcngNsAuwMkunMM57mzJqjO6gxjzKlYtyZ/LyJnBTqgtjDGu4OhoPZ34ESsWyT5wGOBDcc3EYkCXgf+YIw55D0vWM+1j5iD+lwbY6qNMelYPS2MBgYFOKRm1Y9ZRIYBf8KKfRQQB8wLYIh1iMhkYJ8xZl17bbOzJqhO2b2HMWa3/b0PeBPrD6Wz2CsiiQD2974Ax9MsY8xe+4/cDfyDIDzfIhKKdaF/xRjzhj05qM+1r5g7w7kGMMYcAFYCpwGxIuLpEy9oryFeMU+0b7EaY0w58BLBdZ5PB6aISA7WY5dzgCc5hvPcWRNUp+veQ0S6iEhXzzBwHrCx6bWCyjJgpj08E3g7gLG0iOcib7uUIDvf9v35F4DNxpj/8ZoVtOe6sZiD+VyLSIKIxNrDEcDPsZ6drQQutxcLtvPsK+YtXv+4CNaznKA5z8aYPxljko0xqVjX5BXGmKs4hvPcaVuSkFZ27xFoInICVqkJrJ6M/x2sMYvIImA8VjP5e4F7gLeAV4EUrG5SphpjgqZSQiMxj8e65WSAHOC/vJ7tBJyInAGsBjZQe8/+DqxnOkF5rpuI+UqC9FyLyHCsh/NOrH/KXzXG3G//TS7GulX2DXC1XTIJuCZiXgEkYPXdngX8xqsyRdAQkfHALcaYycdynjttglJKKXV866y3+JRSSh3nNEEppZQKSpqglFJKBSVNUEoppYKSJiillFJBSROUUkqpoKQJSimlVFDSBKWUUiooaYJSSikVlDRBKaWUCkqaoJRSSgUlTVBKKaWCkiYopZRSQUkTlFJNEJFVIlIchF1rK3Xc0wSlVCNEJBU4E6uPoyl+3G9I80spdfzTBKVU434F/AdYQG0Pt4hIHxF5Q0T2i0ihiDztNe86EdksIodFZJOInGpPNyJyktdyC0TkAXt4vIjkisg8EdkDvCQi3URkub2PYns42Wv9OBF5SUTy7Plv2dM3ishFXsuFikiBiJzSYWdJqQ6iCUqpxv0KeMX+nC8iPUXECSzH6uk2FeiN1VsoIvJL4F57vWisUldhC/fVC6vH0b7A9Vh/my/Z4ylAKfC01/L/AiKBoUAP4HF7+j+Bq72WmwTkG2O+aWEcSgUN7VFXKR/srs1XAonGmAIR2QI8i1WiWmZPr6q3zv8B7xljnvSxPQP0N8Zst8cXALnGmDvt7rE/BKKNMWWNxJMOrDTGdBORRGA30N0YU1xvuSRgK9DbGHNIRF4DvjbGPNzmk6FUgGgJSinfZgIfGmMK7PF/29P6ADvqJydbH+CHNu5vv3dyEpFIEXlWRHaIyCHgMyDWLsH1AYrqJycAY0we8AXwCxGJBS7AKgEq1enow1il6hGRCGAq4LSfCQGEA7HAXiBFREJ8JKldwImNbPYo1i05j15Artd4/VsZc4GBwBhjzB67BPUNIPZ+4kQk1hhzwMe+FgK/xvr7/soYs7vxo1UqeGkJSqmGLgGqgSFAuv0ZDKy25+UDD4pIFxFxicjp9nrPA7eIyAixnCQife15WcB0EXGKyETg7GZi6Ir13OmAiMQB93hmGGPygfeBv9mVKUJF5Cyvdd8CTgVuwnompVSnpAlKqYZmAi8ZY3YaY/Z4PliVFK4ELgJOAnZilYKmARhjlgLzsW4HHsZKFHH2Nm+y1zsAXGXPa8oTQARQgPXc64N682cAlcAWYB/wB88MY0wp8DrQD3ijlceuVNDQShJKHYdE5G5ggDHm6mYXVipI6TMopY4z9i3Ba7FKWUp1WnqLT6njiIhch1WJ4n1jzGeBjkepY6G3+JRSSgUlLUEppZQKSs0+gxKRF4HJwD5jzDAf8wV4EqtJlaPALGNMZnPbjY+PN6mpqa0OWCml1PFl3bp1BcaYhPrTW1JJYgFW9drG3qe4AOhvf8YAf7e/m5SamkpGRkYLdq+UUup4JiI7fE1v9haf/aC1qIlFLgb+aSz/wWqOJbFtYSqllFKW9qhm3hur1pBHrj0tv/6CInI9VkvNpKSktMOuO57bbah0u6l2G6rdBrcBY6xvtzG43V7DxlqmstpNZbWhqtpat6raM80arnLXLuv5rh2GamOornZT5baWrfIM29urtrdR5XZT7bZirPbaTrUdV7XbqgDjdAgOh+AUsYZFcDqoGTb2cVZ7HUv97RjAKYLDASLWeg4Bhwji+aa2vR5jwHjGjPd0Q7W9D2MMbrfn3HnOq7WWZ/tSbz+efQtY4/Z2xZ5mjYBnTGom1syqw1D7c6w23ueh9mfjqxpR/e0A1jHYx2hqjpGaY+oV7eLxaek4Hb7WVkrV59f3oIwxzwHPAYwcObJdqw8aYzhYWklucSm7io5a38VH2V1cSmllde3F3W3s5GEPu2uThnci8MwLpkqODoEQp4NQh5VoQpyOmmQT4nDgcHiSSG0yAuomGzspeE/zJBhPwmpsO8YrEdcO170QW4mj9gLsGRSvpGHtx5OEahOcw2F9191XbeLyDHtf9D3LesaModGfWWM1Vh01SduKq+Y8eE2rsx2f26YmoXonbIedRYuOVPB1dhG3nDeQlO6RPraglKqvPRLUbqzWlT2S7Wkd6lBZJU98tI1dxVYyyi06yuHyum13dg0PoXe3CLq6QnA6hKjQEEIcgtPhINRpXeBDHGJ9PMPOeuMOByFOOyE4pM5/9J4LkuciJtjr20kkxGmtG1azbWu/TjvB1E8Cjppp1Oy3JgZ7vuqcPt9WwNUvrCHvYKkmKKVaqD0S1DLgBhFZjFU54qDdmGWHCnM6WLJ2J727RZDcLZLRqd3oExdJsj3ep1skMZGhHR2GUi2SGOsCIP9gaYAjUarzaEk180XAeCBeRHKxWlUOBTDG/C/wHlYV8+1Y1cyv6ahgvblCnWy87/w6t5OUClZJMREA5B3w2R+hUsqHZhOUMebKZuYb4PftFlEraHJSnUVEmJPYyFAtQSnVCtqShFJ+khgTQb6WoJRqMW3NXCk/SYpxkXdQE1RQqKoARwg4AvQ/enUVVJVCZRlUHoWqMggJB1cshEe3Pi5jrG2UHQIMhLggNAKcYQ3ftWhs/erK2lgqj4Lbba3rcII47W9H3WFnGIR1XKUfTVBK+UlirIt1O4sDHUb7c7vbfqF3V1sXw4qjUHnEusCWH4Kyg9Zw2UHr45lWfggcodbF13MRDnFBqAtCI63hkHAoL4GjhVBaBEeL7O9COFpsDVcetS6w4dEQEWslBleMPRxTOy4OK0ZTDe4qa9hdBcZdO15dAVXlUF1uJb6qstppnuHKUvvCX2p93JWNnxNx1MbgiS2imzXsrqp7XrzPTXWFr43Vnp+QCPvbZSWkyqO18VQetY6xtfqdDTOXtX69FtIEpZSfJMZEcOBoJaUV1USEOTtuR9WVUFFiX/SPQsWRuknA811Zal9YK62Lm+dTM6289iLr/Z9+zYXN8592pf3fdDiEhNnf4dZ/1yEua5oj1OsC7Ymp1NpHS4R1tS7a4VHWRbqyzC6B2J/GXqeOiIWIOIiMg+je0DPNGnbFWvsuPWBf6O3vfVtqx6vql3bFLnV5ShR2CazOcdvHG+KCsC4Q2d0ej/BKEhF1E6xnuKrMjudA3e/SYji4yxp2hIAr2joXkXHQLdVOZjHW9PBo62fhOddV5bUltSp7vLLUWiY00t5/pJ3gI2qnhURY+zLVtQnauO1hr+/opGP7XW2GJiil/CTJrmqed7CUExOiWrdydRXsWQ85n1vf5Yeti7wnEdUMH2n6v/PGOELthBJmfXt/PBeuyLjai5f3hc0ZbiUNTwmiznd5beKLjLPXibRuC9UZjoDQLrUX2foXXUcTCd2Y2lKKJ+mFdbWSU1PrNaeq3H4D25OQ9JG9v2mCUspPEu2q5vkHyppPUNVVkP8t5KyGHV/Ajq+g4rA1L6aPdcsnLAoi4yE20hoO62Jf8KNqE0DNtEgrAdSZbv/n7gjt3BdfEavEFhJuJaX2EhLefttSbaIJSik/qXkXqrGq5uWHYe0LVlLa+R+rRAQQPwDSLofUM6Dv6RCtbTGrnwZNUEr5Sc8Y6z/yRquaf7sYPr4H4gfC8Gm1CalrTz9GqVTw0ASllJ+EhziJjwpv/GXdwh+s23C/X9OyqsFKHec68Y1npTqfpNgm3oUqzrFqZWlyUgrQBKWUXyXGuMg/0EgJypOglFKAJiil/CoxJoJ8XyUoYzRBKVWPJiil/Cgp1kVJeRWHyuq9q1Sy13qRUhOUUjU0QSnlR97vQtVRnGN9x/Xzb0BKBTFNUEr5kXdrEnV4EpSWoJSqoQlKKT9qtARVlA0IxKb4PyilgpQmKKX8qEfXcBzio+v34hyrMVNtXkepGpqglPKjEKeDntGuhl2/aw0+pRrQBKWUnyXGuHyXoDRBKVWHJiil/Cwxtt67UBVHoWSPJiil6tEEpZSfJcW4yDtQijF2J3sHdljfWsVcqTo0QSnlZ4kxEZRXuSk+ar+sq1XMlfJJE5RSflbzLpSnTT5NUEr5pAlKKT+reRfK8xyqKNvqojyyewCjUir4aIJSys8S7RJUTU0+7WZDKZ80QSnlZ/Fdwgl1Su27UMU50K1vQGNSKhhpglLKzxwOoZfnXSi3W9+BUqoRmqCUCoDEmAirPb6SPVBdrlXMlfJBE5RSAZAU47JaNNcafEo1ShOUUgGQGBvB3kNluIuyrQndtASlVH2aoJQKgKQYF5XVhtI920EcENMn0CEpFXRalKBEZKKIbBWR7SJyu4/5s0Rkv4hk2Z9ft3+oSh0/PO9ClRf8CNHJEBIW4IiUCj4hzS0gIk7gGeDnQC6wVkSWGWM21Vt0iTHmhg6IUanjjuddKNEq5ko1qiUlqNHAdmPMj8aYCmAxcHHHhqXU8S3JLkGFl+zSChJKNaIlCao3sMtrPNeeVt8vRGS9iLwmIj5vqIvI9SKSISIZ+/fvb0O4Sh0fYiNDiQutILKiUKuYK9WI9qok8Q6QaowZDnwELPS1kDHmOWPMSGPMyISEhHbatVKdj4hwStRBa0RLUEr51JIEtRvwLhEl29NqGGMKjTHl9ujzwIj2CU+p49fQyCJrQBOUUj61JEGtBfqLSD8RCQOuAJZ5LyAiiV6jU4DN7ReiUsenk0IKrAF9B0opn5qtxWeMqRKRG4D/A5zAi8aY70TkfiDDGLMMmCMiU4AqoAiY1YExK3VcSJF9HDKRRIbFNP+HqNRPUIv+Lowx7wHv1Zt2t9fwn4A/tW9oSh3felTns9P0oFtJBb1jIwIdjlJBR1uSUCpAYsvy2Gl6kO/pWVcpVUdQ3VmorKwkNzeXsrKyQIeifHC5XCQnJxMaGhroUDo/dzWuI7nsNEOpOqi/70r5ElQJKjc3l65du5Kamopo76JBxRhDYWEhubm59OunD/WP2eF8HO4KdpoeiJaglPIpqG7xlZWV0b17d01OQUhE6N69u5Zu24vdzUZBSCL5WoJSyqegSlCAJqcgpj+bdmR3s1EWnUKelqCU8inoEpRSPwnFOSBOQmL7aAlKqUZogqpHRLj66qtrxquqqkhISGDy5MkALFiwAIfDwfr162uWGTZsGDk5OQCkpqZSUGC9gDl//nyGDh3K8OHDSU9PZ82aNVx66aWkp6dz0kknERMTQ3p6Ounp6Xz55Zf+O0gVeMU5EJNMz25R5B/UEpRSvgRVJYlg0KVLFzZu3EhpaSkRERF89NFH9O5dt23c5ORk5s+fz5IlSxrdzldffcXy5cvJzMwkPDycgoICKioqePPNNwFYtWoVjz76KMuXL+/Q41FBqjgHuqWSGBNBQUkF5VXVhIc4Ax2VUkElaBPUfe98x6a8Q+26zSFJ0dxz0dBml5s0aRLvvvsul19+OYsWLeLKK69k9erVNfMnT57MZ599xtatWxk4cKDPbeTn5xMfH094eDgA8fHx7XMQ6vhQnA2DLyIxxuoXas/BMvp27xLgoJQKLnqLz4crrriCxYsXU1ZWxvr16xkzZkyd+Q6Hg9tuu40///nPjW7jvPPOY9euXQwYMIDf/e53fPrppx0dtuosyg7B0ULolkqS3YJE3gF9DqVUfUFbgmpJSaejDB8+nJycHBYtWsSkSZN8LjN9+nTmz59Pdna2z/lRUVGsW7eO1atXs3LlSqZNm8aDDz7IrFmzOjBy1Skc2GF9d0utKUHpcyilGgraBBVoU6ZM4ZZbbmHVqlUUFhY2mB8SEsLcuXN56KGHGt2G0+lk/PjxjB8/nrS0NBYuXKgJStW8A0W3fiTaPetqTT6lGtIE1YjZs2cTGxtLWloaq1at8rnMrFmzePjhhzl8+HCDeVu3bsXhcNC/f38AsrKy6Nu3b0eGrDoL+x0ouqUSEeakW2SovgullA/6DKoRycnJzJkzp8llwsLCmDNnDvv27Wswr6SkhJkzZzJkyBCGDx/Opk2buPfeezsoWtWpFOeAKxYiYgFIjInQEpRSPogxJiA7HjlypMnIyKgzbfPmzQwePDgg8aiW0Z9RO/jXZVBaBNevAuDXC9eSW1zKB384K6BhKRUoIrLOGDOy/nQtQSnlb8XZdbp51xKUUr5pglLKn9zVcGBn3QQV6+JgaSVHK6oCF5dSQUgTlFL+dGg3uKvqJKikGH0XSilfNEEp5U9eVcw99F0opXzTBKWUP3lVMffwtCaRryUoperQBKWUPxXngCMEomsbIO4Z7UIE8rQEpVQdmqDqcTqdNV1gpKen13Sj0VLe3W14W7ZsGQ8++GCbYsrJyUFEuPPOO2umFRQUEBoayg033ADAvffeS2RkZJ13sqKiohoMu91u5syZw7Bhw0hLS2PUqFFkZ2czZswY0tPTSUlJISEhoc3Hr5pRnAOxKeCsfUc+LMRBfFS4lqCUqkdbkqgnIiKCrKysdt/ulClTmDJlSpvX79evH++++y4PPPAAAEuXLmXo0LrtFcbHx/PYY4812fzSkiVLyMvLY/369TgcDnJzc+nSpQtr1qwBrP6uMjIyePrpp9scq2qC3c1GfUkxLi1BKVVP8Cao92+HPRvad5u90uCC1pViSkpKuPjiiykuLqayspIHHniAiy++mCNHjjB16lRyc3Oprq7mrrvuYtq0aQA89dRTvPPOO1RWVrJ06VIGDRpU58Kfk5PD7NmzKSgoICEhgZdeeomUlBRmzZpFdHQ0GRkZ7Nmzh4cffpjLL78cgMjISAYPHkxGRgYjR45kyZIlTJ06lby8vJpYZ8+ezYIFC5g3bx5xcXE+jyc/P5/ExEQcDqvwnJyc3JYzqdqqOBuSLm0wOTEmgu37SwIQkFLBS2/x1VNaWlpze+vSSy/F5XLx5ptvkpmZycqVK5k7dy7GGD744AOSkpL49ttv2bhxIxMnTqzZRnx8PJmZmfz2t7/l0UcfbbCPG2+8kZkzZ7J+/XquuuqqOk0q5efn8/nnn7N8+XJuv/32Out5ugHZtWsXTqeTpKSkOvOjoqKYPXs2Tz75ZKPHN3XqVN555x3S09OZO3cu33zzTVtPlWqt0gNQWuyzBJUY6yL/QCmBatlFqWAUvCWoVpZ02kv9W3yVlZXccccdfPbZZzgcDnbv3s3evXtJS0tj7ty5zJs3j8mTJ3PmmWfWrHPZZZcBMGLECN54440G+/jqq69qps+YMYPbbrutZt4ll1yCw+FgyJAh7N27t856EydO5K677qJnz541pbX65syZQ3p6OrfccovP+cnJyWzdupUVK1awYsUKJkyYwNKlS5kwYUILz5Bqs5puNvo1mJUUE8GRimoOlVURExHq58CUCk5agmrGK6+8wv79+1m3bh1ZWVn07NmTsrIyBgwYQGZmJmlpadx5553cf//9Net4etF1Op1UVbWudQDPukCD/6bDwsIYMWIEjz32WM2tv/piY2OZPn06zzzzTJP7uOCCC3jkkUe44447eOutt1oVo2ojH1XMPRJj9V0operTBNWMgwcP0qNHD0JDQ1m5ciU7dlj/Befl5REZGcnVV1/NrbfeSmZmZou3OW7cOBYvXgxYCdC79NUcTx9UjT1jArj55pt59tlnfSbHzMzMmudWbreb9evXazcg/lLzkm5qg1k1/UJpTT6lagTvLb4gcdVVV3HRRReRlpbGyJEjGTRoEAAbNmzg1ltvxeFwEBoayt///vcWb/Opp57immuu4ZFHHqmpJNFSQ4cObVB7r774+HguvfRSHn/88Qbz9u3bx3XXXUd5eTkAo0ePrqmqrjpYcQ5EdgdXdINZSXYJSmvyKVVLu9tQraI/o2Pwz0ug/BBct6LBrGq3YcCd7/Pbs0/klvMHBiA4pQJHu9tQKtDqdbPhzekQenYN1xKUUl40QSnlD9VVcGBXowkKIDE2Qp9BKeWlRQlKRCaKyFYR2S4it/uYHy4iS+z5a0Qkta0B6XsgwUt/NsfgUC6Yap9VzD0SY1xai08pL80mKBFxAs8AFwBDgCtFZEi9xa4Fio0xJwGPA423tdMEl8tFYWGhXgiDkDGGwsJCXC5XoEPpnJqoweeRFBtB3sEyKqrcVLuN/h2on7yW1OIbDWw3xvwIICKLgYuBTV7LXAzcaw+/BjwtImJa+ReWnJxMbm4u+/fvb35h44ZDec0vp9qHMbhKdpC8/kmoOBjoaDqf6krru4kE1Ts2gooqNwPufL9mmgg4RHCK4HBYww4RpIPDVaolxpzQnednNqjb0G5akqB6A7u8xnOBMY0tY4ypEpGDQHegTrPeInI9cD1ASkpKgx2FhobSr1/jt0DqqCyFT15u2bKq/Qy9KNARdF7RSRDTeNuHl6T3pqLKTXlVNW5DTSmq2hjcBtxuQ7XbGlYqGPSLj+zQ7fv1PShjzHPAc2BVMz+mjYVGwMS/tEdYSgWFmMhQrjvrhECHoVTQaEklid1AH6/xZHuaz2VEJASIAQrbI0CllFI/TS1JUGuB/iLST0TCgCuAZfWWWQbMtIcvB1a09vmTUkop5a1FLUmIyCTgCcAJvGiMmS8i9wMZxphlIuIC/gWcAhQBV3gqVTSxzf3AjmM9ACCees+6OlZ+dAAABklJREFUOgGN2T80Zv/pjHFrzP7Rkpj7GmMS6k8MWFNH7UVEMnw1kRHMNGb/0Jj9pzPGrTH7x7HErC1JKKWUCkqaoJRSSgWl4yFBPRfoANpAY/YPjdl/OmPcGrN/tDnmTv8MSiml1PHpeChBKaWUOg5pglJKKRWUOm2Caq4LkGAkIjkiskFEskQko/k1AkNEXhSRfSKy0WtanIh8JCLb7O9ugYyxvkZivldEdtvnO8t+ny9oiEgfEVkpIptE5DsRucmeHrTnuomYg/Zci4hLRL4WkW/tmO+zp/ezuwfabncXFBboWD2aiHmBiGR7nef0QMdan4g4ReQbEVluj7f5PHfKBNXCLkCC1c+MMelB/i7DAmBivWm3A58YY/oDn9jjwWQBDWMGeNw+3+nGmPf8HFNzqoC5xpghwFjg9/bvcTCf68ZihuA91+XAOcaYk4F0YKKIjMXqFuhxu5ugYqxug4JFYzED3Op1nrMCF2KjbgI2e423+Tx3ygSFVxcgxpgKwNMFiGoHxpjPsFoE8XYxsNAeXghc4tegmtFIzEHNGJNvjMm0hw9j/VH3JojPdRMxBy1jKbFHQ+2PAc7B6h4Igu88NxZzUBORZOBC4Hl7XDiG89xZE5SvLkCC+o/EZoAPRWSd3fVIZ9LTGJNvD+8BegYymFa4QUTW27cAg+ZWWX12L9SnAGvoJOe6XswQxOfavu2UBewDPgJ+AA4YY6rsRYLuGlI/ZmOM5zzPt8/z4yISHsAQfXkCuA1w2+PdOYbz3FkTVGd1hjH/v727CbG6CuM4/v1lLoYM7UVCMBl6gaCSGirohRiCIkiCCBIxiGiREr0sCrNNEK2CgqwIkpAgS4TSdlLkEEFBEZkVtQo3YjO6sBAiwn4tzpn8Z3fGaV64506/z+Zezp3589wH7v/cc87leTxC2Zp8RNKt/Q5oNmoh4Oa/zQGvA5dStkiOAC/2N5zeJC0D3gOesP1r97VWc90j5qZzbfuk7Wso3RhuAK7oc0hndHrMkq4CtlJivx44H9jSxxD/QdI6YML2V/N1zUGdoGbSAqQ5tg/XxwlgD+WDMijGJa0CqI8TfY7njGyP1w/5n8B2Gsy3pKWUG/1O2+/X4aZz3SvmQcg1gO3jwBhwI7CitgeChu8hnZjvrFustv07sIO28nwzcLekQ5Rjl9uAl5lDngd1gppJC5CmSDpH0rmTz4E7gO+m/6+mdFuqPAB80MdYZmTyJl/dQ2P5rvvzbwI/2H6p81KzuZ4q5pZzLWmlpBX1+RBwO+XsbIzSHgjay3OvmH/sfHER5SynmTzb3mp7te1hyj15v+2NzCHPA1tJQj1agPQ5pGlJuoSyaoLSyfidVmOW9C4wSimTPw48C+wFdgNrKG1S7rPdzI8Spoh5lLLlZOAQ8HDnbKfvJN0CfAp8y6k9+2coZzpN5nqamDfQaK4lraUczi+hfCnfbfu5+pncRdkq+xq4v65M+m6amPcDKwEBB4BNnR9TNEPSKPCk7XVzyfPATlAREbG4DeoWX0RELHKZoCIiokmZoCIiokmZoCIiokmZoCIiokmZoCJmSdLJTlXpA5rHqvqShtWpzB7xf3T2mf8kIqbwWy1FExELICuoiHmm0vfrBZXeX19IuqyOD0vaXwt9fixpTR2/SNKe2vvnG0k31UstkbS99gP6sFYUQNJjKv2YDkra1ae3GbHgMkFFzN7QaVt86zuv/WL7auBVSsUTgFeAt2yvBXYC2+r4NuCT2vtnBPi+jl8OvGb7SuA4cG8dfxq4tl5n00K9uYh+SyWJiFmSdML2sh7jhyjN5n6qhVV/tn2BpGPAKtt/1PEjti+UdBRY3S3/UltZfFSbFiJpC7DU9vOS9gEnKOWn9rZY6iZiPmQFFbEwPMXz/6Jbr+wkp86M76J0lB4BvuxUio5YVDJBRSyM9Z3Hz+vzzyhVngE2UoquQmnrvhn+blK3fKqLSjoLuNj2GKUX0HLgX6u4iMUg37wiZm+odjydtM/25E/Nz5N0kLIK2lDHHgV2SHoKOAo8WMcfB96Q9BBlpbSZ0vSvlyXA23USE7Ct9guKWHRyBhUxz+oZ1HW2j/U7lohBli2+iIhoUlZQERHRpKygIiKiSZmgIiKiSZmgIiKiSZmgIiKiSZmgIiKiSX8BgbpM2gDVZDAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.subplot(2, 1, 1)\n",
    "plt.plot(my_cb.loss_0)\n",
    "plt.plot(my_cb.loss_1)\n",
    "plt.legend([\"MNIST\", \"FashionMNIST\"])\n",
    "plt.title(\"Loss\")\n",
    "plt.subplot(2, 1, 2)\n",
    "plt.plot(my_cb.acc_0)\n",
    "plt.plot(my_cb.acc_1)\n",
    "plt.legend([\"MNIST\", \"FashionMNIST\"])\n",
    "plt.title(\"Accuracy\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iSfJAOLbBsv3"
   },
   "source": [
    "We can clearly see the transition from training on MNIST to training on FashionMNITS on epoch $20$. At this epoch the loss on MNIST rapidly increases again while the loss on FashionMNIST rapidly decreases. For the accuracy we see a similar but opposite behaivior. It first is high while training on MNIST and then rapidly decreases towards $0$ as we transition to training on FashionMNIST. Avioding this phenomenon is an active area of research on deep learning and there already exist some methods in order to mitigate catastrophic forgetting. But so far solving continual learning with neural networks remains an interesting and challenging problem. If it can be solved, neural networks could learn more human like without a constant need to refresh previously learned examples and enable interesting new applications where knowledge is accumulated over time.\n",
    "\n",
    "## Conclusion\n",
    "In this exercise we have learned how to use Keras in order to quickly develop neural networks and train them on different data sets. We also refreshed the mathematical background on classification and explored how dropout can act as an effective regularization in order to mitigate overfitting. While the Keras specification offers a high-level of abstraction in order to simplify build and training neural networks, it also enables us to customize some parts of the complete workflow. In order to evaluate a model on two different data sets in order to visualize the phenomenon of catastrophic forgetting, we implemented a custom callback that executed our required action at the end of every epoch. Besides that Keras offers many other ways to implement non standard layers/methods. If you want to learn more on Keras, you can visit the documentation on the Keras implementation in Tensorflow [here](https://www.tensorflow.org/versions/r2.2/api_docs/python/tf/keras) or directly use the Keras specification available [here](https://keras.io). Additionally we explored a very basic transfer learning technique, i.e. fine tuning of pretrained models. For this we utilized the MobileNetV2 architecture that was trained on the ImageNet data set and fine tuned it on the Caltech 101 data set. The resulting model was simple to implement and fast to train, while it still achieved competetive results."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Classification_solution.ipynb",
   "provenance": [
    {
     "file_id": "1AcDxk3US9zCHQhEEmXQBHUYRGfpTbe6B",
     "timestamp": 1562663692376
    }
   ],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}