Homeworks/2_Multi_class_classification_with_MNIST.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "wDlWLbfkJtvu",
        "cellView": "form"
      },
      "source": [
        "#@title Copyright 2020 Google LLC. Double-click here for license information.\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License.\n",
        "\n",
        "# Yann LeCun and Corinna Cortes hold the copyright of MNIST dataset,\n",
        "# which is a derivative work from original NIST datasets. \n",
        "# MNIST dataset is made available under the terms of the \n",
        "# Creative Commons Attribution-Share Alike 3.0 license."
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T4r2z30vJSbA"
      },
      "source": [
        "# Colabs\n",
        "\n",
        "Machine Learning Crash Course uses Colaboratories (Colabs) for all programming exercises. Colab is Google's implementation of [Jupyter Notebook](https://jupyter.org/). For more information about Colabs and how to use them, go to [Welcome to Colaboratory](https://research.google.com/colaboratory)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TL5y5fY9Jy_x"
      },
      "source": [
        "# Multi-Class Classification\n",
        "\n",
        "This Colab explores multi-class classification problems through the classic MNIST dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XuKlphuImFSN"
      },
      "source": [
        "## Learning Objectives:\n",
        "\n",
        "After doing this Colab, you'll know how to do the following:\n",
        "\n",
        "  * Understand the classic MNIST problem.\n",
        "  * Create a deep neural network that performs multi-class classification.\n",
        "  * Tune the deep neural network.\n",
        "\n",
        "This exercise introduces image classification with machine learning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cxj8yVh4mFl5"
      },
      "source": [
        "## The Dataset\n",
        "  \n",
        "This MNIST dataset contains a lot of examples:\n",
        "\n",
        "* The MNIST training set contains 60,000 examples.\n",
        "* The MNIST test set contains 10,000 examples.\n",
        "\n",
        "Each example contains a pixel map showing how a person wrote a digit. For example, the following images shows how a person wrote the digit `1` and how that digit might be represented in a 14x14 pixel map (after the input data is normalized). \n",
        "\n",
        "![Two images. The first image shows a somewhat fuzzy digit one. The second image shows a 14x14 floating-point array in which most of the cells contain 0 but a few cells contain values between 0.0 and 1.0. The pattern of nonzero values corresponds to the image of the fuzzy digit in the first image.](https://www.tensorflow.org/images/MNIST-Matrix.png)\n",
        "\n",
        "Each example in the MNIST dataset consists of:\n",
        "\n",
        "* A label specified by a [rater](https://developers.google.com/machine-learning/glossary/#rater).  Each label must be an integer from 0 to 9.  For example, in the preceding image, the rater would almost certainly assign the label `1` to the example.\n",
        "* A 28x28 pixel map, where each pixel is an integer between 0 and 255. The pixel values are on a gray scale in which 0 represents white, 255 represents black, and values between 0 and 255 represent various shades of gray.  \n",
        "\n",
        "This is a multi-class classification problem with 10 output classes, one for each digit."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xchnxAsaKKqO"
      },
      "source": [
        "## Import relevant modules\n",
        "\n",
        "The following hidden code cell imports the necessary code to run the code in the rest of this Colaboratory."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9n9_cTveKmse"
      },
      "source": [
        "#@title Import relevant modules\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras import layers\n",
        "from matplotlib import pyplot as plt\n",
        "\n",
        "# The following lines adjust the granularity of reporting. \n",
        "pd.options.display.max_rows = 10\n",
        "pd.options.display.float_format = \"{:.1f}\".format\n",
        "\n",
        "# The following line improves formatting when ouputting NumPy arrays.\n",
        "np.set_printoptions(linewidth = 200)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_TaJhU4KcuY"
      },
      "source": [
        "## Load the dataset\n",
        "\n",
        "`tf.keras` provides a set of convenience functions for loading well-known datasets. Each of these convenience functions does the following:\n",
        "\n",
        "* Loads both the training set and the test set.\n",
        "* Separates each set into features and labels.\n",
        "\n",
        "The relevant convenience function for MNIST is called `mnist.load_data()`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JZlvdpyYKx7V",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "79cb3754-fdbf-4c76-83d6-5ca974cf0999"
      },
      "source": [
        "(x_train, y_train),(x_test, y_test) = tf.keras.datasets.mnist.load_data()\n"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SfQkr3hxJGXU"
      },
      "source": [
        "Notice that `mnist.load_data()` returned four separate values:\n",
        "\n",
        "* `x_train` contains the training set's features.\n",
        "* `y_train` contains the training set's labels.\n",
        "* `x_test` contains the test set's features.\n",
        "* `y_test` contains the test set's labels.\n",
        "\n",
        "**Note:** The MNIST .csv training set is already shuffled."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "71vsSUM7pdmu"
      },
      "source": [
        "## View the dataset\n",
        "\n",
        "The .csv file for the California Housing Dataset contains column names (for example, `latitude`, `longitude`, `population`). By contrast, the .csv file for MNIST does not contain column names. Instead of column names, you use ordinal numbers to access different subsets of the MNIST dataset. In fact, it is probably best to think of `x_train` and `x_test` as three-dimensional NumPy arrays:  \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IoOhpjkeCL8Q",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8f44f464-c38a-41e7-eb13-ee748cfea41f"
      },
      "source": [
        "# Output example #2917 of the training set.\n",
        "x_train[2917]"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0, 127, 100, 156, 239, 224, 177, 213, 159,  70,  13,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0, 110, 250, 254, 254, 254, 254, 254, 254, 254, 254, 184,  10,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0, 216, 254, 254, 254, 254, 254, 254, 254, 254, 254, 251,  54,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   3, 131, 197,  68, 137, 101,  83,  41,  70, 221, 254, 108,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  58, 254, 216,  11,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 110, 254, 254,  24,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   4,  83, 247, 254, 192,  10,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  37, 124, 254, 254, 250,  47,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 113, 178, 250, 254, 254, 254, 252, 178, 128,  50,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  69, 252, 254, 254, 254, 254, 254, 254, 254, 254, 153,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 141, 252, 254, 241, 241, 254, 247, 252, 254, 254, 153,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  68,  82,   5,   5,  82,  37,  65, 167, 254, 190,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 162, 254, 153,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1, 172, 253,  75,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 102, 254, 152,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  38, 230, 216,  20,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1,  71, 222, 254, 142,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  66, 254, 254, 157,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   4,  34,  34,  78, 161, 226, 249, 254, 154,   6,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,  96, 184, 254, 254, 254, 254, 254, 249,  93,   5,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
              "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LNJrJKUwvZMR"
      },
      "source": [
        "Alternatively, you can call `matplotlib.pyplot.imshow` to interpret the preceding numeric array as an image. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "siRC8a1hJvmq",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "6f9866f5-adfb-4823-e758-ade12d18e332"
      },
      "source": [
        "# Use false colors to visualize the array.\n",
        "plt.imshow(x_train[2917])"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7fde7c7b36d0>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAN+ElEQVR4nO3df6zddX3H8der7NKuLVUKrjbQSOlaA84N8NoCawgLkQAhKZjI7DatsXrJlEQWNkdYMtmWJWxTmDEDV2xnnQrBUEKzkQk2GOacDbe1QgEHHRZs6Q+gjLY4Sn+898f94m7hns+5Ped7fvS+n4/k5pzzfZ/v9/vOgVe/33M+33M+jggBmPgm9boBAN1B2IEkCDuQBGEHkiDsQBK/0s2dnejJMUXTurlLIJXX9ZreiAMeq9ZW2G1fJunLkk6Q9LWIuKX0/CmapkW+pJ1dAihYH+sa1lo+jbd9gqR/kHS5pLMlLbV9dqvbA9BZ7bxnXyhpS0Q8GxFvSLpb0pJ62gJQt3bCfpqkn496vK1adhTbQ7aHbQ8f1IE2dgegHR3/ND4iVkTEYEQMDmhyp3cHoIF2wr5d0pxRj0+vlgHoQ+2E/VFJ823PtX2ipI9KWltPWwDq1vLQW0Qcsn2dpO9qZOhtVUQ8UVtnAGrV1jh7RDwg6YGaegHQQVwuCyRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiirSmbbW+VtE/SYUmHImKwjqYA1K+tsFd+JyJeqmE7ADqI03ggiXbDHpIetL3B9tBYT7A9ZHvY9vBBHWhzdwBa1e5p/OKI2G771yQ9ZPunEfHI6CdExApJKyRphmdGm/sD0KK2juwRsb263S3pPkkL62gKQP1aDrvtabZPevO+pEslba6rMQD1auc0fpak+2y/uZ1vR8S/1dLVBLPltvOL9akvlP/NPfjBfcX62kVfbVj76RvvKq77lecvKdavmr2pWL/9qYuK9Xfcc1LD2ju/93Rx3cMv7ynWcWxaDntEPCvpt2rsBUAHMfQGJEHYgSQIO5AEYQeSIOxAEo7o3kVtMzwzFrk81HM8ev7PLyzWNwz9fbE+ddKJdbZz3Pjwlg8V669fWb68+vDevXW2MyGsj3XaG3s8Vo0jO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kUccPTqb3x0vXFOtZx9GbuWveA8X65ef/YbE+8OBwne1MeBzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlr8J33n16s//VtS4r1Ky/YWKxv+JvzivUXLmn8mwTTny3/J568p/x7Bqc8tr9Y3zd3WrF+/V/d1bB2zfRXi+vuXFi+PmHOg8Uy3oIjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwe/Go6MOXPHBhrXvf+3O4rorX313sX7vBe8t1g//T3kcfyJq63fjba+yvdv25lHLZtp+yPYz1e3JdTYMoH7jOY3/uqTL3rLsRknrImK+pHXVYwB9rGnYI+IRSXvesniJpNXV/dWSrqq5LwA1a/Xa+FkRsaO6v1PSrEZPtD0kaUiSpmhqi7sD0K62P42PkU/4Gn7KFxErImIwIgYHNLnd3QFoUath32V7tiRVt7vrawlAJ7Qa9rWSllX3l0m6v552AHRK0/fstu+SdLGkU21vk/QFSbdIusf2cknPSbqmk03i+LVrcKDldZe/Y2exvubkReUNJBxnL2ka9ohY2qDE1THAcYTLZYEkCDuQBGEHkiDsQBKEHUiCn5JGkQfKP+f80rIPFOs//PQXC9Xy5dPLn19crMfLrxTrOBpHdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH25Hzu+4r1Lb83o1z//Tua7KHxWPovjrxRXPP5P5lfrE/a++Mm+8ZoHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2SeA5/7iwoa19yx+vrjumvf+U7E+dVL5++ztGPAJxfqkm18s1rc8fn6xPvf+xuP4Jzy8sbjuRMSRHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYJz9OLD/mvJ48uOf+krDWrOxbKlz4+jNNOvtu2f9S3kDZ5XLC2Yta1ib+3B53Ymo6ZHd9irbu21vHrXsZtvbbW+q/q7obJsA2jWe0/ivS7psjOW3RcQ51d8D9bYFoG5Nwx4Rj0ja04VeAHRQOx/QXWf7seo0/+RGT7I9ZHvY9vBBHWhjdwDa0WrY75A0T9I5knZI+lKjJ0bEiogYjIjBAU1ucXcA2tVS2CNiV0Qcjogjku6UtLDetgDUraWw25496uHVkjY3ei6A/tB0nN32XZIulnSq7W2SviDpYtvnSApJWyVd28Ee03vh0kMd2/bPDu5va/25A9OL9VcO/6Jhbfqk8tu65tcI4Fg0DXtELB1j8coO9AKgg7hcFkiCsANJEHYgCcIOJEHYgST4iutxYMGnhov1iz/ymYa1AzPK/56f+s32flL5pT84r7z9jXsb1l6bWx62e/2TrxTrj553T7F+/hk/a1jbVVxzYuLIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM4+AUz/zvrGtSbrRpv7PmXlf7a8/ak/Lm/71Y83+a3oJn60dW7D2lz9pK1tH484sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzo3fsYnnGlPJ0YYfjSLE+819/9Zhbmsg4sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzo2e2f/6CYn3zb95erM///ieL9TO/+aNj7mkia3pktz3H9sO2n7T9hO3PVctn2n7I9jPV7cmdbxdAq8ZzGn9I0g0Rcbak8yV91vbZkm6UtC4i5ktaVz0G0Keahj0idkTExur+PklPSTpN0hJJq6unrZZ0VaeaBNC+Y3rPbvsMSedKWi9pVkTsqEo7Jc1qsM6QpCFJmqKprfYJoE3j/jTe9nRJ90q6PiKOmq0vIkINflswIlZExGBEDA5oclvNAmjduMJue0AjQf9WRKypFu+yPbuqz5a0uzMtAqhD09N425a0UtJTEXHrqNJaScsk3VLd3t+RDnFc8wfe17B2+7XlobWVr767WF/w+ReL9UPFaj7jec/+25I+Julx25uqZTdpJOT32F4u6TlJ13SmRQB1aBr2iPiBpEa/MnBJve0A6BQulwWSIOxAEoQdSIKwA0kQdiAJvuKKsiY/97zvdxcV6zf85bcb1i6aUt71soevLNYXbBsubwBH4cgOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzo6i1z68sFj/4a1fbXnbZ/3Hx4r1BcsZR68TR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9i7wwInl+llntrX9l89tPIHu/171anHdv3v/vcX64inNpj0ufyl9wSMfb1ib94mni+seabJnHBuO7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQxHjmZ58j6RuSZkkKSSsi4su2b5b0aUlvTpJ9U0Q80KlG+9nOP7qwWJ939TPF+ppfv7vOdmq1v8lg9zm3fKZYP/MfNzSsHTlwoJWW0KLxXFRzSNINEbHR9kmSNth+qKrdFhFf7Fx7AOoynvnZd0jaUd3fZ/spSad1ujEA9Tqm9+y2z5B0rqT11aLrbD9me5XtMa/ZtD1ke9j28EFx2gb0yrjDbnu6pHslXR8ReyXdIWmepHM0cuT/0ljrRcSKiBiMiMEBTa6hZQCtGFfYbQ9oJOjfiog1khQRuyLicEQckXSnpPIvEwLoqaZht21JKyU9FRG3jlo+e9TTrpa0uf72ANTFEVF+gr1Y0r9Lelz//63DmyQt1cgpfEjaKuna6sO8hmZ4ZizyJW22DKCR9bFOe2PPmPNsj+fT+B9IGmvllGPqwPGKK+iAJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJNP0+e607s1+U9NyoRadKeqlrDRybfu2tX/uS6K1Vdfb2noh411iFrob9bTu3hyNisGcNFPRrb/3al0RvrepWb5zGA0kQdiCJXod9RY/3X9KvvfVrXxK9taorvfX0PTuA7un1kR1AlxB2IImehN32Zbb/y/YW2zf2oodGbG+1/bjtTbaHe9zLKtu7bW8etWym7YdsP1PdjjnHXo96u9n29uq122T7ih71Nsf2w7aftP2E7c9Vy3v62hX66srr1vX37LZPkPS0pA9J2ibpUUlLI+LJrjbSgO2tkgYjoucXYNi+SNJ+Sd+IiN+olv2tpD0RcUv1D+XJEfGnfdLbzZL293oa72q2otmjpxmXdJWkT6iHr12hr2vUhdetF0f2hZK2RMSzEfGGpLslLelBH30vIh6RtOcti5dIWl3dX62R/1m6rkFvfSEidkTExur+PklvTjPe09eu0FdX9CLsp0n6+ajH29Rf872HpAdtb7A91OtmxjBr1DRbOyXN6mUzY2g6jXc3vWWa8b557VqZ/rxdfED3dosj4jxJl0v6bHW62pdi5D1YP42djmsa724ZY5rxX+rla9fq9Oft6kXYt0uaM+rx6dWyvhAR26vb3ZLuU/9NRb3rzRl0q9vdPe7nl/ppGu+xphlXH7x2vZz+vBdhf1TSfNtzbZ8o6aOS1vagj7exPa364ES2p0m6VP03FfVaScuq+8sk3d/DXo7SL9N4N5pmXD1+7Xo+/XlEdP1P0hUa+UT+vyX9WS96aNDXmZJ+Uv090eveJN2lkdO6gxr5bGO5pFMkrZP0jKTvSZrZR739s0am9n5MI8Ga3aPeFmvkFP0xSZuqvyt6/doV+urK68blskASfEAHJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0n8H6gnHrP+4lGwAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V-he9IcihDxb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "569ab070-a19d-4c12-b579-ec5ff69baa5d"
      },
      "source": [
        "# Output row #10 of example #2917.\n",
        "x_train[2917][10]"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  58, 254, 216,  11,   0,   0,   0,   0,   0,   0,   0,   0], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DUEWipalhQ8J",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e99d98e7-89a6-43ca-ac8f-d04df25a7cc8"
      },
      "source": [
        "# Output pixel #16 of row #10 of example #2900.\n",
        "x_train[2900][10][16]"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8ldP-5z1B2vL"
      },
      "source": [
        "## Task 1: Normalize feature values\n",
        "\n",
        "Complete the following code cell to map each feature value from its current representation (an integer between 0 and 255) to a floating-point value between 0 and 1.0. Store the floating-point values in `x_train_normalized` and `x_test_normalized`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4YQljE-wizDw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ebd731f9-a9c1-4cfd-c7a0-620958d12bfc"
      },
      "source": [
        "x_train_normalized = x_train / 255.0\n",
        "x_test_normalized = x_test / 255.0\n",
        "print(x_train_normalized[2900][10]) # Output a normalized row"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0.         0.         0.         0.         0.         0.         0.         0.55294118 1.         0.66666667 0.11372549 0.         0.         0.         0.         0.         0.         0.\n",
            " 0.         0.         0.         0.         0.         0.         0.         0.         0.         0.        ]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZBWRF6CStuNA"
      },
      "source": [
        "## Define a plotting function\n",
        "\n",
        "The following function plots an accuracy curve:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QF0BFRXTOeR3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "74f2f103-9620-4e9b-b535-fcfd8719b96e"
      },
      "source": [
        "#@title Define the plotting function\n",
        "def plot_curve(epochs, hist, list_of_metrics):\n",
        "  \"\"\"Plot a curve of one or more classification metrics vs. epoch.\"\"\"  \n",
        "  # list_of_metrics should be one of the names shown in:\n",
        "  # https://www.tensorflow.org/tutorials/structured_data/imbalanced_data#define_the_model_and_metrics  \n",
        "\n",
        "  plt.figure()\n",
        "  plt.xlabel(\"Epoch\")\n",
        "  plt.ylabel(\"Value\")\n",
        "\n",
        "  for m in list_of_metrics:\n",
        "    x = hist[m]\n",
        "    plt.plot(epochs[1:], x[1:], label=m)\n",
        "\n",
        "  plt.legend()\n",
        "\n",
        "print(\"Loaded the plot_curve function.\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Loaded the plot_curve function.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3014ezH3C7jT"
      },
      "source": [
        "## Create a deep neural net model\n",
        "\n",
        "The `create_model` function defines the topography of the deep neural net, specifying the following:\n",
        "\n",
        "* The number of [layers](https://developers.google.com/machine-learning/glossary/#layer) in the deep neural net.\n",
        "* The number of [nodes](https://developers.google.com/machine-learning/glossary/#node) in each layer.\n",
        "* Any [regularization](https://developers.google.com/machine-learning/glossary/#regularization) layers.\n",
        "\n",
        "The `create_model` function also defines the [activation function](https://developers.google.com/machine-learning/glossary/#activation_function) of each layer.  The activation function of the output layer is [softmax](https://developers.google.com/machine-learning/glossary/#softmax), which will yield 10 different outputs for each example. Each of the 10 outputs provides the probability that the input example is a certain digit.\n",
        "\n",
        "**Note:** Unlike several of the recent Colabs, this exercise does not define feature columns or a feature layer.  Instead, the model will train on the NumPy array."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pedD5GhlDC-y",
        "cellView": "both"
      },
      "source": [
        "def create_model(my_learning_rate):\n",
        "  \"\"\"Create and compile a deep neural net.\"\"\"\n",
        "  \n",
        "  # All models in this course are sequential.\n",
        "  model = tf.keras.models.Sequential()\n",
        "\n",
        "  # The features are stored in a two-dimensional 28X28 array. \n",
        "  # Flatten that two-dimensional array into a one-dimensional \n",
        "  # 784-element array.\n",
        "  model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
        "\n",
        "  # Define the first hidden layer.   \n",
        "  model.add(tf.keras.layers.Dense(units=32, activation='relu'))\n",
        "  \n",
        "  # Define a dropout regularization layer. \n",
        "  model.add(tf.keras.layers.Dropout(rate=0.2))\n",
        "\n",
        "  # Define the output layer. The units parameter is set to 10 because\n",
        "  # the model must choose among 10 possible output values (representing\n",
        "  # the digits from 0 to 9, inclusive).\n",
        "  #\n",
        "  # Don't change this layer.\n",
        "  model.add(tf.keras.layers.Dense(units=10, activation='softmax'))     \n",
        "                           \n",
        "  # Construct the layers into a model that TensorFlow can execute.  \n",
        "  # Notice that the loss function for multi-class classification\n",
        "  # is different than the loss function for binary classification.  \n",
        "  model.compile(optimizer=tf.keras.optimizers.Adam(lr=my_learning_rate),\n",
        "                loss=\"sparse_categorical_crossentropy\",\n",
        "                metrics=['accuracy'])\n",
        "  \n",
        "  return model    \n",
        "\n",
        "\n",
        "def train_model(model, train_features, train_label, epochs,\n",
        "                batch_size=None, validation_split=0.1):\n",
        "  \"\"\"Train the model by feeding it data.\"\"\"\n",
        "\n",
        "  history = model.fit(x=train_features, y=train_label, batch_size=batch_size,\n",
        "                      epochs=epochs, shuffle=True, \n",
        "                      validation_split=validation_split)\n",
        " \n",
        "  # To track the progression of training, gather a snapshot\n",
        "  # of the model's metrics at each epoch. \n",
        "  epochs = history.epoch\n",
        "  hist = pd.DataFrame(history.history)\n",
        "\n",
        "  return epochs, history, hist    "
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D-IXYVfvM4gD"
      },
      "source": [
        "## Invoke the previous functions\n",
        "\n",
        "Run the following code cell to invoke the preceding functions and actually train the model on the training set. \n",
        "\n",
        "**Note:** Due to several factors (for example, more examples and a more complex neural network) training MNIST might take longer than training the California Housing Dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nj3v5EKQFY8s",
        "cellView": "both",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "89b42ee9-7d48-4357-e029-6f67d1260cf3"
      },
      "source": [
        "# The following variables are the hyperparameters.\n",
        "learning_rate = 0.003\n",
        "epochs = 50\n",
        "batch_size = 4000\n",
        "validation_split = 0.2\n",
        "\n",
        "# Establish the model's topography.\n",
        "my_model = create_model(learning_rate)\n",
        "\n",
        "# Train the model on the normalized training set.\n",
        "epochs, history, hist = train_model(my_model, x_train_normalized, y_train, \n",
        "                           epochs, batch_size, validation_split)\n",
        "\n",
        "# Plot a graph of the metric vs. epochs.\n",
        "list_of_metrics_to_plot = ['accuracy']\n",
        "plot_curve(epochs, hist, list_of_metrics_to_plot)\n",
        "\n",
        "# Evaluate against the test set.\n",
        "print(\"\\n Evaluate the new model against the test set:\")\n",
        "my_model.evaluate(x=x_test_normalized, y=y_test, batch_size=batch_size)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/keras/optimizers/optimizer_v2/adam.py:110: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.\n",
            "  super(Adam, self).__init__(name, **kwargs)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n",
            "12/12 [==============================] - 3s 109ms/step - loss: 1.6381 - accuracy: 0.4863 - val_loss: 0.8837 - val_accuracy: 0.8193\n",
            "Epoch 2/50\n",
            "12/12 [==============================] - 1s 65ms/step - loss: 0.8348 - accuracy: 0.7553 - val_loss: 0.5024 - val_accuracy: 0.8712\n",
            "Epoch 3/50\n",
            "12/12 [==============================] - 1s 64ms/step - loss: 0.6018 - accuracy: 0.8186 - val_loss: 0.3969 - val_accuracy: 0.8897\n",
            "Epoch 4/50\n",
            "12/12 [==============================] - 1s 63ms/step - loss: 0.5056 - accuracy: 0.8497 - val_loss: 0.3455 - val_accuracy: 0.9045\n",
            "Epoch 5/50\n",
            "12/12 [==============================] - 1s 67ms/step - loss: 0.4498 - accuracy: 0.8684 - val_loss: 0.3167 - val_accuracy: 0.9109\n",
            "Epoch 6/50\n",
            "12/12 [==============================] - 1s 64ms/step - loss: 0.4160 - accuracy: 0.8772 - val_loss: 0.2939 - val_accuracy: 0.9167\n",
            "Epoch 7/50\n",
            "12/12 [==============================] - 1s 57ms/step - loss: 0.3885 - accuracy: 0.8871 - val_loss: 0.2752 - val_accuracy: 0.9232\n",
            "Epoch 8/50\n",
            "12/12 [==============================] - 1s 64ms/step - loss: 0.3634 - accuracy: 0.8935 - val_loss: 0.2585 - val_accuracy: 0.9277\n",
            "Epoch 9/50\n",
            "12/12 [==============================] - 1s 52ms/step - loss: 0.3452 - accuracy: 0.8996 - val_loss: 0.2467 - val_accuracy: 0.9306\n",
            "Epoch 10/50\n",
            "12/12 [==============================] - 1s 66ms/step - loss: 0.3320 - accuracy: 0.9030 - val_loss: 0.2370 - val_accuracy: 0.9339\n",
            "Epoch 11/50\n",
            "12/12 [==============================] - 1s 60ms/step - loss: 0.3171 - accuracy: 0.9058 - val_loss: 0.2280 - val_accuracy: 0.9371\n",
            "Epoch 12/50\n",
            "12/12 [==============================] - 1s 60ms/step - loss: 0.3035 - accuracy: 0.9126 - val_loss: 0.2203 - val_accuracy: 0.9389\n",
            "Epoch 13/50\n",
            "12/12 [==============================] - 1s 71ms/step - loss: 0.2943 - accuracy: 0.9135 - val_loss: 0.2129 - val_accuracy: 0.9411\n",
            "Epoch 14/50\n",
            "12/12 [==============================] - 1s 57ms/step - loss: 0.2871 - accuracy: 0.9159 - val_loss: 0.2068 - val_accuracy: 0.9429\n",
            "Epoch 15/50\n",
            "12/12 [==============================] - 1s 60ms/step - loss: 0.2784 - accuracy: 0.9177 - val_loss: 0.2003 - val_accuracy: 0.9439\n",
            "Epoch 16/50\n",
            "12/12 [==============================] - 1s 75ms/step - loss: 0.2706 - accuracy: 0.9202 - val_loss: 0.1959 - val_accuracy: 0.9462\n",
            "Epoch 17/50\n",
            "12/12 [==============================] - 1s 62ms/step - loss: 0.2640 - accuracy: 0.9227 - val_loss: 0.1909 - val_accuracy: 0.9475\n",
            "Epoch 18/50\n",
            "12/12 [==============================] - 1s 60ms/step - loss: 0.2556 - accuracy: 0.9244 - val_loss: 0.1866 - val_accuracy: 0.9483\n",
            "Epoch 19/50\n",
            "12/12 [==============================] - 1s 59ms/step - loss: 0.2527 - accuracy: 0.9241 - val_loss: 0.1833 - val_accuracy: 0.9492\n",
            "Epoch 20/50\n",
            "12/12 [==============================] - 1s 60ms/step - loss: 0.2459 - accuracy: 0.9261 - val_loss: 0.1790 - val_accuracy: 0.9503\n",
            "Epoch 21/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.2422 - accuracy: 0.9284 - val_loss: 0.1749 - val_accuracy: 0.9520\n",
            "Epoch 22/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2401 - accuracy: 0.9275 - val_loss: 0.1722 - val_accuracy: 0.9518\n",
            "Epoch 23/50\n",
            "12/12 [==============================] - 0s 33ms/step - loss: 0.2334 - accuracy: 0.9300 - val_loss: 0.1694 - val_accuracy: 0.9522\n",
            "Epoch 24/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2249 - accuracy: 0.9331 - val_loss: 0.1670 - val_accuracy: 0.9533\n",
            "Epoch 25/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2253 - accuracy: 0.9317 - val_loss: 0.1644 - val_accuracy: 0.9539\n",
            "Epoch 26/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2210 - accuracy: 0.9341 - val_loss: 0.1615 - val_accuracy: 0.9553\n",
            "Epoch 27/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2166 - accuracy: 0.9352 - val_loss: 0.1594 - val_accuracy: 0.9553\n",
            "Epoch 28/50\n",
            "12/12 [==============================] - 0s 33ms/step - loss: 0.2145 - accuracy: 0.9354 - val_loss: 0.1581 - val_accuracy: 0.9556\n",
            "Epoch 29/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.2110 - accuracy: 0.9367 - val_loss: 0.1561 - val_accuracy: 0.9556\n",
            "Epoch 30/50\n",
            "12/12 [==============================] - 0s 36ms/step - loss: 0.2076 - accuracy: 0.9378 - val_loss: 0.1537 - val_accuracy: 0.9569\n",
            "Epoch 31/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.2035 - accuracy: 0.9386 - val_loss: 0.1511 - val_accuracy: 0.9563\n",
            "Epoch 32/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.2024 - accuracy: 0.9387 - val_loss: 0.1506 - val_accuracy: 0.9568\n",
            "Epoch 33/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1983 - accuracy: 0.9400 - val_loss: 0.1489 - val_accuracy: 0.9567\n",
            "Epoch 34/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.1994 - accuracy: 0.9395 - val_loss: 0.1485 - val_accuracy: 0.9572\n",
            "Epoch 35/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1952 - accuracy: 0.9409 - val_loss: 0.1463 - val_accuracy: 0.9572\n",
            "Epoch 36/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1905 - accuracy: 0.9413 - val_loss: 0.1447 - val_accuracy: 0.9577\n",
            "Epoch 37/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.1897 - accuracy: 0.9414 - val_loss: 0.1438 - val_accuracy: 0.9579\n",
            "Epoch 38/50\n",
            "12/12 [==============================] - 0s 33ms/step - loss: 0.1850 - accuracy: 0.9437 - val_loss: 0.1427 - val_accuracy: 0.9580\n",
            "Epoch 39/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.1864 - accuracy: 0.9433 - val_loss: 0.1428 - val_accuracy: 0.9588\n",
            "Epoch 40/50\n",
            "12/12 [==============================] - 0s 33ms/step - loss: 0.1848 - accuracy: 0.9438 - val_loss: 0.1406 - val_accuracy: 0.9582\n",
            "Epoch 41/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1827 - accuracy: 0.9451 - val_loss: 0.1402 - val_accuracy: 0.9592\n",
            "Epoch 42/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1789 - accuracy: 0.9459 - val_loss: 0.1390 - val_accuracy: 0.9588\n",
            "Epoch 43/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1779 - accuracy: 0.9457 - val_loss: 0.1385 - val_accuracy: 0.9600\n",
            "Epoch 44/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.1779 - accuracy: 0.9448 - val_loss: 0.1376 - val_accuracy: 0.9593\n",
            "Epoch 45/50\n",
            "12/12 [==============================] - 0s 32ms/step - loss: 0.1734 - accuracy: 0.9469 - val_loss: 0.1382 - val_accuracy: 0.9597\n",
            "Epoch 46/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1751 - accuracy: 0.9468 - val_loss: 0.1366 - val_accuracy: 0.9588\n",
            "Epoch 47/50\n",
            "12/12 [==============================] - 0s 33ms/step - loss: 0.1719 - accuracy: 0.9476 - val_loss: 0.1344 - val_accuracy: 0.9601\n",
            "Epoch 48/50\n",
            "12/12 [==============================] - 0s 34ms/step - loss: 0.1712 - accuracy: 0.9471 - val_loss: 0.1343 - val_accuracy: 0.9604\n",
            "Epoch 49/50\n",
            "12/12 [==============================] - 0s 35ms/step - loss: 0.1698 - accuracy: 0.9478 - val_loss: 0.1342 - val_accuracy: 0.9593\n",
            "Epoch 50/50\n",
            "12/12 [==============================] - 0s 32ms/step - loss: 0.1686 - accuracy: 0.9478 - val_loss: 0.1340 - val_accuracy: 0.9603\n",
            "\n",
            " Evaluate the new model against the test set:\n",
            "3/3 [==============================] - 0s 10ms/step - loss: 0.1348 - accuracy: 0.9602\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.13479799032211304, 0.9602000117301941]"
            ]
          },
          "metadata": {},
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXzV1Z3/8dcnG9khgbAGBCu7ItZobZ0qoliqtlitrY5at5HpVP3Z3WVcOo6dOkunrR27YEdRa6WOFmUs7svgTEEJsi8iIktCIIGQFbJ/fn98v8FrCJCEXG6S+34+Hvdxv9/z/X7PPQfj/dxzzvd7jrk7IiIiHZUQ6wKIiEjvosAhIiKdosAhIiKdosAhIiKdosAhIiKdkhTrAhwLgwYN8tGjR8e6GCIivcqyZct2u3te2/S4CByjR4+msLAw1sUQEelVzGxre+nqqhIRkU5R4BARkU5R4BARkU6J6hiHmc0EfgEkAr9z9wfaHD8OeATIA8qBq9y9KDzWDKwOT93m7l8O08cA84CBwDLgandv6GzZGhsbKSoqoq6urkt1i3epqank5+eTnJwc66KIyDEWtcBhZonAQ8AMoAhYamYL3H1dxGn/Bjzu7o+Z2XTgJ8DV4bH97j61naz/GfiZu88zs98ANwC/7mz5ioqKyMrKYvTo0ZhZZy+Pa+7Onj17KCoqYsyYMbEujogcY9Hsqjod2OTum8MWwTxgVptzJgFvhNtvtnP8Eyz4hp8OPBMmPQZc3JXC1dXVMXDgQAWNLjAzBg4cqNaaSJyKZuAYAWyP2C8K0yKtBC4Jt78CZJnZwHA/1cwKzWyJmbUGh4FAhbs3HSZPAMxsdnh9YVlZWbsFVNDoOv3bicSvWD/H8X3gP8zsWmARUAw0h8eOc/diMzseeMPMVgOVHc3Y3ecAcwAKCgo0d7yI9Gj1Tc3sqKhjT009/dOSyc1IYUB6CokJn/yR5u6U1zZQXLGf4r37Ka7YT9X+xuCgGRa8YRhmcNUZx5GbkdKtZY1m4CgGRkbs54dpB7j7DsIWh5llApe6e0V4rDh832xmbwGnAM8CA8wsKWx1HJSniEhPsKm0hlfX7aKmvpFEMxISjAQzEsP3mvpGivbuD1/72FVVf1AeZjAgDCI56SlU7G+keO9+9jc2t/OJ7bvgpGG9KnAsBcaGd0EVA5cDfx15gpkNAsrdvQW4g+AOK8wsB9jn7vXhOWcC/+LubmZvAl8lGDO5Bng+inXo9ZqamkhKinXDUqTnamlxdtfUUxT+gt9ZWYfjpCQmkJyUQEpiAilJCfRLSiA3ox9jBmUwKDOl3e7a7eX7eGFVCf+9cgfrSqoASDBoaafPIynBGD4gjfycNM4el0d+Tjr5OWkMzOxH5f5G9tY2sKe2gfLaesprG9hb28gJeZlMG5fHiJw0RgxIY0ROGvkD0slOSzpQHnfHHTzcbtti6Q5R+0Zx9yYzuxl4meB23Efcfa2Z3QcUuvsCYBrwEzNzgq6qm8LLJwK/NbMWgnGYByLuxroNmGdm9wPLgf+MVh2i7eKLL2b79u3U1dVx6623Mnv2bF566SXuvPNOmpubGTRoEK+//jo1NTXccsstFBYWYmbce++9XHrppWRmZlJTUwPAM888wwsvvMDcuXO59tprSU1NZfny5Zx55plcfvnl3HrrrdTV1ZGWlsajjz7K+PHjaW5u5rbbbuOll14iISGBG2+8kcmTJ/Pggw/y3HPPAfDqq6/yq1/9ivnz58fyn0rkqLk7RXv3s2J7BauKKlhfUk3R3n3sqKijobmlU3llpSZxfF4mnxqUwZhBGaQkJfDS2p0s31YBwCmjBnDPRZO4cMowhmSnAkGAanGnOfxiT05MiMqXulnQRRXudXv+EOUxDndfCCxsk3ZPxPYzfHyHVOQ5fwFOOkSemwnu2Oo2//Dfa1m3o6o7s2TS8Gzu/dLkw57zyCOPkJuby/79+znttNOYNWsWN954I4sWLWLMmDGUl5cD8I//+I/079+f1auDx1r27t17xM8vKiriL3/5C4mJiVRVVfH222+TlJTEa6+9xp133smzzz7LnDlz2LJlCytWrCApKYny8nJycnL41re+RVlZGXl5eTz66KNcf/31R/8PInKUauub2LCzivSUJLJSk8hOSyYzJYmEiC/fhqYW9u5roLw2eO2pbeDD0hpWFlWwcnsFe/cFYwEpSQlMHJrF5BH9+cLkoZ/4BT+sfxpJCUZDUwsNzS00NLVQ3xS8l9XUs7mshs1ltWzeXcPizXv40/Kgt3zSsGxumzmBi6YMY2Ru+kHlT0gwErCYDyx3h75Qh17rwQcfPPBLfvv27cyZM4ezzjrrwLMRubm5ALz22mvMmzfvwHU5OTlHzPuyyy4jMTERgMrKSq655ho++OADzIzGxsYD+X7zm9880JXV+nlXX301v//977nuuutYvHgxjz/+eDfVWKRzWlqcxZv38OyyIl5cs/Ogvn0zyExJIjM1iZr6Jqrrmg7KI8Fg3JAsZkwawskjB3By/gDGD80iOfHwN5Vm9Gs//exxn5wsdl9D8LmtLYt4oMABR2wZRMNbb73Fa6+9xuLFi0lPT2fatGlMnTqVDRs2dDiPyD7Wts9UZGRkHNi+++67Oeecc5g/fz5btmxh2rRph833uuuu40tf+hKpqalcdtllGiORY25zWQ3PvlfE/PeK2VFZR1a/JC4+ZTjnThhCU0sLVfubqKprpKquieq6Rqrrmsjsl0RuRsonXgMzUhg+II2MftH7G05PSSI9Jb7+H4mv2vYglZWV5OTkkJ6ezoYNG1iyZAl1dXUsWrSIjz766EBXVW5uLjNmzOChhx7i5z//ORB0VeXk5DBkyBDWr1/P+PHjmT9/PllZWYf8rBEjgsdd5s6deyB9xowZ/Pa3v+Wcc8450FWVm5vL8OHDGT58OPfffz+vvfZa1P8tpG+qrW9i+bYK3t9VTU1dE7UNTdTUN1F74NVMQ3MLjWF3UGNzC43NTn1TM7uq6kkw+PzYPG6/YCLnTxpCanJirKskIQWOGJk5cya/+c1vmDhxIuPHj+eMM84gLy+POXPmcMkll9DS0sLgwYN59dVXueuuu7jppps48cQTSUxM5N577+WSSy7hgQce4KKLLiIvL4+CgoIDA+Vt/fCHP+Saa67h/vvv58ILLzyQ/jd/8zds3LiRKVOmkJyczI033sjNN98MwJVXXklZWRkTJ048Jv8e0rPV1jexuriS9SVV9EtKDH7NZ4a/7NNT6J+WTFlNPYVb9rJ0SzmFW8tZX1JNc8TtRP2SEsjsl0RG6yslkbTkRLJTk0iOuIMpKcE4YXAmF58yIq66f3oTc+/7z8YVFBR424Wc1q9fry/Fw7j55ps55ZRTuOGGGw55jv4Nez93p6nFaWp2GppbaGpuoSm8PXVVUSUrtlWwsqiCjbuq272ltFXkLaepyQmcMjKHgtE5FIzO5cTh2fRPSybpCGMK0vOY2TJ3L2ibrhaHHOTUU08lIyODn/70p7EuihylhqYWtu6pZeuefRTt3cf2vfvZXh68F+3d1+5gcqQB6clMHTmAL0weytSRA5g8IpvmFmdPTXDX0t59DQe2B6Qnc9roXCYNzz7iwLP0bgoccpBly5bFugjSSS0tzrbyfby/q5qNO6uD913VbC6rpSmiqZCanMDInHRG5qZz+ugcBqSnkJxoJIVdRMmJCSQlGtmpyUzJ78+o3PR2H3Qb1j/tWFZPepi4Dhzursn6uigeujh7qtYH2VYVVbKquIJV2ytZU1xJdf3HrYeRuWmMH5LFeROHMG5IFqMGpjMyJ/2QTzyLdEbcBo7U1FT27NmjqdW7oHU9jtRUDVx2p8bmFjaX1bJhZxUfltVSXddIXWMz+xua2dfQzP5we/PuWsprg7XLkhONicOymXXKcE4c3p8Jw7IZOzgzqrefisTtX1d+fj5FRUUcasp1ObzWFQCl69buqOSt98t4f2c17++sZvPuGhqbg5ZcgkFGShJpKYnBKzmR1OTg/dwJg5kycgAn5/dn/NAs+iXpNlU5tuI2cCQnJ2v1Ouk2VXWNvL5+FyNz0jlxRP9DPnOwt7aB51cU83Rh0YFJ8EYMSGP80CymTxzMhKFZjB+axfGDMklJ0gCz9ExxGzhEukN1XSNz/28LD7+9marwDqXkROPEEf05dVQOpx6Xw9RRA3h/ZzX/VVjEq+t20dDcwokjsrlv1mQumjK826e8Fok2BQ6RLqipb+KxvwQBo2JfI+dNHMLfnn08e2sbWLZtL+9t3cvjS7byu//96MA1OenJXHnGKC47dSSThmfHsPQiR0eBQ6QTauubeGLJVuYs2kx5bQPTJwzm2+eNZUr+gAPnnD95KBA8Q7F2RyXLt1UwrH8q0ycO1niE9AkKHCIdsG5HFU++s5XnV+ygpr6Js8fl8e3zxnLKqEPPVJySlMApo3IOe45Ib6TAIXGppcV5d0s5izaWMTCzH8cPymD0oAzyc9IOPPVc19jMC6tKePKdrSzfVkG/pAQunDKMq884TsFA4poCh8SVzWU1zF9ezJ/eK6a4Yj9mEPksY2KCMTInjZG56azcXkFVXROfysvg7osmcemnRzAgXQPZIlENHGY2E/gFwdKxv3P3B9ocP45gnfE8oBy4yt2LzGwq8GsgG2gGfuzufwyvmQucDVSG2Vzr7iuiWQ/p3fbWNvDn1SX86b0i3ttWQYLBmScM4gdfGM/5k4ewv6GZLXtq+Wj3Pj7aXcOW3fvYWl7L2eMHc+VnRvGZMbl6SFQkQtQCh5klAg8BM4AiYKmZLYhYOxzg34DH3f0xM5sO/AS4GtgHfMPdPzCz4cAyM3vZ3SvC634QLjsr0q6y6npeWbeTF1fvZPHmPTS3OOOGZHLHFycwa+oIhvb/+Kn39JQkBmb249TjcmNYYpHeI5otjtOBTeEa4ZjZPGAWEBk4JgHfDbffBJ4DcPeNrSe4+w4zKyVolVQgcau5xVlZVMGKbRUkJRrpKcGaDunh2g6pyYm8t20vC1eX8O5H5bQ4jBmUwd+edTwXnDSMycOz1XIQ6QbRDBwjgO0R+0XAZ9qcsxK4hKA76ytAlpkNdPc9rSeY2elACvBhxHU/NrN7gNeB2929vu2Hm9lsYDbAqFGjjr42EhMllftZtLGMRRt387+bdlO5v/GI14wdnMnN08dywUlDGT8kS8FCpJvFenD8+8B/mNm1wCKgmGBMAwAzGwY8AVzj7i1h8h3AToJgMge4DbivbcbuPic8TkFBgaZy7UV2VOxn3tLtvLi6hA9Kg1UNh2T34/xJQzhrXB6fOT6XBDP21TdT29DEvoYmauqb2VffxNghmZwwuP0ldEWke0QzcBQDIyP288O0A9x9B0GLAzPLBC5tHccws2zgz8Dfu/uSiGtKws16M3uUIPhIL9fc4ry5oZSn3t3Gm++X4sBnjx/I1wpGcta4PMYNyTy45ZAZk6KKxL1oBo6lwFgzG0MQMC4H/jryBDMbBJSHrYk7CO6wwsxSgPkEA+fPtLlmmLuXWPAtcjGwJop1kCgrqdzPH5du549Lt1NSWUdeVj++Ne0Evn7aSEbmpse6eCLSjqgFDndvMrObgZcJbsd9xN3Xmtl9QKG7LwCmAT8xMyfoqropvPxrwFnAwLAbCz6+7fZJM8sDDFgBfDNadZDoaG5xFn1Qxh/e2cbr63fhwOfH5nHvlyZx7sQhWnZUpIezeFjJraCgwAsLC2NdjLhXWl3HfxUW8dS72yjau59BmSlcVjCSK04bxaiBal2I9DRmtszdC9qmx3pwXPq46rpGFm3czZ9X7+CVtbtoanE+96mB3P7FCZw/aajWnBDphRQ4pNsV7d3H6+tLeW39LpZs3kNjs5ObkcK1nxvNFZ8ZxafyNKot0pspcEi3KKncz7PLinhhVQkbdlYDcHxeBtefOYbzJg3h06NySEzQ8xQifYECh3RZfVMzr68v5Y9Lt/P2B2W0OJw2Ooe/v2Ai504czPFqWYj0SQoc0inVdY18UFrDCytLmL+8iL37Ghmancq3pp3AZQX5HDcwI9ZFFJEoU+CQdjW3OG9sKGXjrmq27K4NZ4+tZXdNAxCsqz1j0hAuKxjJWWPz1A0lEkcUOOQg7s5tz67imWVFAORl9WPMoAzOnTCE0YMyGDMondNG5zIws1+MSyoisaDAIQd54KUNPLOsiFumn8Dfnv0pMvvpz0REPqZvBPmEOYs+5Lf/s5mrzziO784Yp5llReQgevpKDvivwu3808INXDhlGD/68mQFDRFplwKHAPDaul3c/qfV/NUJg/j3r52swW4ROSQFDmHplnJu+sN7nDg8m99cfSr9khJjXSQR6cEUOOLc8m17uX7uUkbkpPHItadpIFxEjkjfEnHI3Xn7g908/PZm3v5gN0OzU3n8+tN1e62IdIgCRxxpaGphwcod/O7tzWzYWc3grH78cOZ4rjz9OPqnJ8e6eCLSSyhwxIH9Dc08vngLj/zfR+yqqmf8kCz+7bKT+dLJwzSeISKdFtXAYWYzgV8QrAD4O3d/oM3x4wiWi80DyoGr3L0oPHYNcFd46v3u/liYfiowF0gDFgK3ejysRtUFzS3On94r4t9f3UhJZR1nnjCQf/nqyZw1dpButRWRLota4DCzROAhYAZQBCw1swXuvi7itH8jWFf8MTObDvwEuNrMcoF7gQLAgWXhtXuBXwM3Au8QBI6ZwIvRqkdvtWhjGT95cQPrS6o4Ob8/P/v6VM44fmCsiyUifUA0WxynA5vcfTOAmc0DZgGRgWMS8N1w+03guXD7C8Cr7l4eXvsqMNPM3gKy3X1JmP44cDEKHAes21HFT15cz9sf7GZkbhoPXnEKF500jAQ9lyEi3SSagWMEsD1ivwj4TJtzVgKXEHRnfQXIMrOBh7h2RPgqaif9IGY2G5gNMGrUqC5Xojd5YslW7nl+Ddmpydx14USu/uxxGsMQkW4X6+c4vg+cbWbLgbOBYqC5OzJ29znuXuDuBXl5ed2RZY/2xOIt3P3cGqaPH8yiH5zD33z+eAUNEYmKaLY4ioGREfv5YdoB7r6DoMWBmWUCl7p7hZkVA9PaXPtWeH3+4fKMR08s2crdz6/lvImDeejKTytgiEhURbPFsRQYa2ZjzCwFuBxYEHmCmQ0ys9Yy3EFwhxXAy8D5ZpZjZjnA+cDL7l4CVJnZGRbcFvQN4Pko1qHHe2LJVu5+bo2ChogcM1ELHO7eBNxMEATWA0+7+1ozu8/MvhyeNg1438w2AkOAH4fXlgP/SBB8lgL3tQ6UA98CfgdsAj4kjgfGfx8GjXMnKGiIyLFj8fAIREFBgRcWFsa6GN3q90u2clcYNH51lYKGiHQ/M1vm7gVt02M9OC5d8OQ7ChoiEjsKHL3M00u38/fz1zBdQUNEYkSBoxd5dlkRt/1pFWeNy+NXGtMQkRhR4Oglnl9RzA+eWcnnPjWQOVefSmqygoaIxIYCRy+wcHUJ3316JaeNzuV33zhNQUNEYkqBo4d7Ze1O/t9Tyzll5AAeufY00lIUNEQkthQ4erA3NuwK1gIf0Z9HrzuNDC3rKiI9gAJHD7V1Ty1/9/v3mDgsm8euP52sVK3QJyI9gwJHD+Tu3PXcGpITE5hzdQH90xQ0RKTnUODogf57VQlvf7Cb758/jqH9U2NdHBGRT1Dg6GEq9zdy33+vY0p+f67+7OhYF0dE5CAabe1h/vXlDZTX1jP3utNI1Kp9ItIDqcXRgyzftpcn39nGNZ8bzYkj+se6OCIi7VLg6CGamlu4c/4ahmSl8r3zx8e6OCIih6Suqh7i0f/bwvqSKn5z1afJ1PMaItKDqcXRAxRX7OffX93IuRMG84XJQ2NdHBGRw1Lg6AHufX4tAP8wazLBirgiIj1XVAOHmc00s/fNbJOZ3d7O8VFm9qaZLTezVWZ2QZh+pZmtiHi1mNnU8NhbYZ6txwZHsw7RtnB1Ca+t38W3zxtLfk56rIsjInJEUetMN7NE4CFgBlAELDWzBe6+LuK0uwjWIv+1mU0CFgKj3f1J4Mkwn5OA59x9RcR1V7p7r18LtrSqjjvnr2ZKfn+u/6sxsS6OiEiHRLPFcTqwyd03u3sDMA+Y1eYcB7LD7f7AjnbyuSK8tk9xd37wzCrqGpv52denkpyoXkMR6R2i+W01AtgesV8UpkX6EXCVmRURtDZuaSefrwNPtUl7NOymutsOMShgZrPNrNDMCsvKyrpUgWh6YslW/mdjGX9/wUQ+lZcZ6+KIiHRYrH/mXgHMdfd84ALgCTM7UCYz+wywz93XRFxzpbufBHw+fF3dXsbuPsfdC9y9IC8vL3o16IJNpTX808L1nD0uj6vOOC7WxRER6ZRoBo5iYGTEfn6YFukG4GkAd18MpAKDIo5fTpvWhrsXh+/VwB8IusR6jcbmFr779ApSkxP5169O0V1UItLrRDNwLAXGmtkYM0shCAIL2pyzDTgXwMwmEgSOsnA/AfgaEeMbZpZkZoPC7WTgImANvcgvX/+AVUWV/OQrJzE4WzPfikjvE7W7qty9ycxuBl4GEoFH3H2tmd0HFLr7AuB7wMNm9h2CgfJr3d3DLM4Ctrv75ohs+wEvh0EjEXgNeDhadehuy7bu5T/e3MSln87niycNi3VxRES6xD7+nu67CgoKvLAwtnfv1tY3ccGDb9PU7Lz07c9rRT8R6fHMbJm7F7RN16RIx8hPX9nItvJ9/HH2ZxU0RKRXi/VdVXGhsbmFPy0v4qIpwzl9TG6siyMiclQUOI6BJZv3ULGvkYumaFxDRHo/BY5jYOHqEjJSEjl7XM96nkREpCsUOKKsqbmFl9fu4tyJQ0hNTox1cUREjpoCR5Qt2VxOeW0DF+j2WxHpIxQ4ouzPq0tIT0lk2nh1U4lI39DhwGFmWiyik5qaW3hl7U6mTxisbioR6TOOGDjM7HNmtg7YEO6fbGa/inrJ+oB3PypnT20DF6qbSkT6kI60OH4GfAHYA+DuKwmmA5Ej+PPqEtKSE5k2vlcvUigi8gkd6qpy9+1tkpqjUJY+pbnFeTnspkpLUTeViPQdHZlyZLuZfQ7wcHLBW4H10S1W7/fOR3vYXaO7qUSk7+lIi+ObwE0Eq/cVA1PDfTmMF1fvJDU5gXMm6G4qEelbjtjicPfdwJXHoCx9RnOL8+KaoJsqPUXzSIpI33LEbzUze5RgrYxPcPfro1KiPmDplnJ219TzxRPVTSUifU9Hfg6/ELGdCnwF2BGd4vQNC1eX0C8pgekTdDeViPQ9HemqejZy38yeAv43aiXq5Vq7qc4ZP5iMfuqmEpG+pytTjowFOvRT2sxmmtn7ZrbJzG5v5/goM3vTzJab2SozuyBMH21m+81sRfj6TcQ1p5rZ6jDPB83MulCHqFm2dS9l1fVcoCnURaSP6sgYRzXBGIeF7zuB2zpwXSLwEDADKAKWmtkCd18XcdpdwNPu/mszmwQsBEaHxz5096ntZP1r4EbgnfD8mcCLRyrPsbJwdQkp6qYSkT6sI11VWV3M+3Rgk7tvBjCzecAsIDJwOJAdbvfnCGMnZjYMyHb3JeH+48DF9JDA0dLivLimhGnj8shUN5WI9FGH/HYzs08f7kJ3f+8IeY8AIp84LwI+0+acHwGvmNktQAZwXsSxMWa2HKgC7nL3t8M8i9rkOeIQ5Z8NzAYYNWrUEYraPbaV72NXVb2mGBGRPu1wP4t/ephjDkzvhs+/Apjr7j81s88CT5jZiUAJMMrd95jZqcBzZja5Mxm7+xxgDkBBQcFBtxNHw+riSgCm5Pc/Fh8nIhIThwwc7n7OUeZdDIyM2M8P0yLdQDBGgbsvNrNUYJC7lwL1YfoyM/sQGBden3+EPGNmTXElKYkJjBvS1d49EZGer0Md8WErYBLBcxwAuPvjR7hsKTDWzMYQfLlfDvx1m3O2AecCc81sYph/mZnlAeXu3mxmxxPcybXZ3cvNrMrMziAYHP8G8MuO1OFYWF1cyfihWaQkaX0sEem7OnJX1b3ANILAsRD4IsFzHIcNHO7eZGY3Ay8DicAj7r7WzO4DCt19AfA94GEz+w5B99e17u5mdhZwn5k1Ai3AN929PMz6W8BcII1gULxHDIy7O2uKK7lwyvBYF0VEJKo60uL4KnAysNzdrzOzIcDvO5K5uy8kCDaRafdEbK8DzmznumeBZ9umh8cKgRM78vnH0vby/VTVNXHSCI1viEjf1pE+lTp3bwGazCwbKOWTYxfCxwPjJ47IPsKZIiK92+Fux30IeAp418wGAA8Dy4AaYPGxKV7vsbq4kuREY/xQDYyLSN92uK6qjcC/AsOBWoIgMoPgAbxVx6Bsvcqa4krGDcmiX5JW+xORvu2QXVXu/gt3/yzB+uJ7gEeAl4CvmNnYY1S+XsHdWbOjUuMbIhIXjjjG4e5b3f2f3f0Uggf2LgY2RL1kvUjR3v1U7GvkRAUOEYkDRwwcZpZkZl8ysycJbn19H7gk6iXrRdYcGBhX4BCRvu9wg+MzCFoYFwDvAvOA2e5ee4zK1musLq4kKcGYoIFxEYkDhxscvwP4A/A9d997jMrTK60urmTskCxSkzUwLiJ93+HmquqOSQz7PHdn7Y4qzpuoGXFFJD5oUqWjtKOyjvLaBt1RJSJxQ4HjKK0u0sC4iMQXBY6jtKa4ksQEY+IwTTUiIvFBgeMordlRydjBmRoYF5G4ocBxFFqnUlc3lYjEEwWOo7Czqo7dNRoYF5H4osBxFDQwLiLxKKqBw8xmmtn7ZrbJzG5v5/goM3vTzJab2SozuyBMn2Fmy8xsdfg+PeKat8I8V4SvmD1Asaa4kgSDSRoYF5E40qE1x7vCzBKBhwimYi8ClprZgnDVv1Z3AU+7+6/NrHVp2tHAbuBL7r4jXO/8ZWBExHVXhisBxtSaHVWcMDiTtBQNjItI/Ihmi+N0YJO7b3b3BoK5rma1OceB1p/r/YEdAO6+3N13hOlrgTQz6xfFsnbJag2Mi0gcimbgGAFsj9gv4pOtBoAfAVeZWRFBa+OWdvK5FHjP3esj0h4Nu6nuNjNr78PNbLaZFZpZYVlZWZcrcSi7quooq67XwLiIxJ1YD45fAcx193yCWXifMLMDZTKzycA/A38bcc2V7mJYGfcAAA2oSURBVH4S8PnwdXV7Gbv7HHcvcPeCvLy8bi9468C4AoeIxJtoBo5iYGTEfn6YFukG4GkAd18MpAKDAMwsH5gPfMPdP2y9wN2Lw/dqgtl7T49S+Q9rdXElZuiJcRGJO9EMHEuBsWY2xsxSgMuBBW3O2QacC2BmEwkCR5mZDQD+DNzu7v/XenK4qFRrYEkGLgLWRLEOh7R2RyWfyssko1/U7i8QEemRohY43L0JuJngjqj1BHdPrTWz+8zsy+Fp3wNuNLOVwFPAte7u4XUnAPe0ue22H/Cyma0CVhC0YB6OVh0OZ3Wx1hgXkfgU1Z/L7r6QYNA7Mu2eiO11wJntXHc/cP8hsj21O8vYFaXVdeyqqtcdVSISl2I9ON4rrS2uAjQwLiLxSYGjC7buCZZdPz4vI8YlERE59hQ4uqC0up6kBCM3PSXWRREROeYUOLqgtLqevKx+JCS0++yhiEifpsDRBaXV9QzO6nEzoIiIHBMKHF1QWlVHXlZqrIshIhITChxdUFZdz+BstThEJD4pcHRSY3MLe2ob1FUlInFLgaOTdtcEk/QOVleViMQpBY5OKq0KAkeeWhwiEqcUODqptLq1xaHAISLxSYGjk0qr6wA0OC4icUuBo5NKq+oxg0GZChwiEp8UODqptLqe3PQUkhP1Tyci8Unffp1UVl2ngXERiWsKHJ1UWl3P4Gzdiisi8UuBo5NKqzRPlYjEt6gGDjObaWbvm9kmM7u9neOjzOxNM1tuZqvM7IKIY3eE171vZl/oaJ7R1NLi7K5R4BCR+Ba1wGFmicBDwBeBScAVZjapzWl3EaxFfgpwOfCr8NpJ4f5kYCbwKzNL7GCeUVO+r4GmFlfgEJG4Fs0Wx+nAJnff7O4NwDxgVptzHMgOt/sDO8LtWcA8d69394+ATWF+HckzalqfGtcYh4jEs2gGjhHA9oj9ojAt0o+Aq8ysCFgI3HKEazuSJwBmNtvMCs2ssKysrKt1+IQDD/+pxSEicSzWg+NXAHPdPR+4AHjCzLqlTO4+x90L3L0gLy+vO7KMmG5ELQ4RiV9JUcy7GBgZsZ8fpkW6gWAMA3dfbGapwKAjXHukPKOmrDVwaLoREYlj0WxxLAXGmtkYM0shGOxe0OacbcC5AGY2EUgFysLzLjezfmY2BhgLvNvBPKOmtKqOrNQkUpMTj9VHioj0OFFrcbh7k5ndDLwMJAKPuPtaM7sPKHT3BcD3gIfN7DsEA+XXursDa83saWAd0ATc5O7NAO3lGa06tKW1xkVEottVhbsvJBj0jky7J2J7HXDmIa79MfDjjuR5rASBQ+MbIhLfYj043quUVtdpfENE4p4CRwe5u6YbERFBgaPDquqaqG9qUVeViMQ9BY4OKtPKfyIigAJHh7VON6K1OEQk3ilwdJCeGhcRCShwdFCpuqpERAAFjg4rraonNTmBrH5RffRFRKTHU+DooNaH/8ws1kUREYkpBY4OKq2u0zMcIiIocHRYaXW9xjdERFDg6LCyKs1TJSICChwdsr+hmer6Jj3DISKCAkeHaMlYEZGPKXB0wIGH/7LVVSUiosDRAa3TjajFISKiwNEh6qoSEflYVAOHmc00s/fNbJOZ3d7O8Z+Z2YrwtdHMKsL0cyLSV5hZnZldHB6ba2YfRRybGs06QNBVlZRg5KSnRPujRER6vKjNn2FmicBDwAygCFhqZgvC5WIBcPfvRJx/C3BKmP4mMDVMzwU2Aa9EZP8Dd38mWmVvq7SqnrysfiQk6KlxEZFotjhOBza5+2Z3bwDmAbMOc/4VwFPtpH8VeNHd90WhjB2ip8ZFRD4WzcAxAtgesV8Uph3EzI4DxgBvtHP4cg4OKD82s1VhV1e73+hmNtvMCs2ssKysrPOlj1BWXU+eHv4TEQF6zuD45cAz7t4cmWhmw4CTgJcjku8AJgCnAbnAbe1l6O5z3L3A3Qvy8vKOqnCabkRE5GPRDBzFwMiI/fwwrT3ttSoAvgbMd/fG1gR3L/FAPfAoQZdY1DQ0tVBe26CuKhGRUDQDx1JgrJmNMbMUguCwoO1JZjYByAEWt5PHQeMeYSsEC+Y3vxhY083l/oTdNVr5T0QkUtTuqnL3JjO7maCbKRF4xN3Xmtl9QKG7twaRy4F57u6R15vZaIIWy/+0yfpJM8sDDFgBfDNadYDIJWPV4hARgSgGDgB3XwgsbJN2T5v9Hx3i2i20M5ju7tO7r4RHVlqlJWNFRCL1lMHxHuvjFoe6qkREQIHjiEqr6zGDQZl6alxEBBQ4jqisuo6BGSkkJeqfSkQEFDiOKJhuRN1UIiKtFDiOoLS6XndUiYhEUOA4As1TJSLySQoch9Hc4uyuadCtuCIiERQ4DqO8toHmFtetuCIiERQ4DkMr/4mIHEyB4zAOPPynrioRkQMUOA6jrEpPjYuItKXAcRitXVV56qoSETlAgeMwSqvryU5NIjU5MdZFERHpMRQ4DqO0qp7B2eqmEhGJFNVp1Xu7k/L7MyYvI9bFEBHpURQ4DuOmc06IdRFERHqcqHZVmdlMM3vfzDaZ2e3tHP+Zma0IXxvNrCLiWHPEsQUR6WPM7J0wzz+Gy9KKiMgxErXAYWaJwEPAF4FJwBVmNinyHHf/jrtPdfepwC+BP0Uc3t96zN2/HJH+z8DP3P0EYC9wQ7TqICIiB4tmi+N0YJO7b3b3BmAeMOsw518BPHW4DM3MgOnAM2HSY8DF3VBWERHpoGgGjhHA9oj9ItpZQxzAzI4DxgBvRCSnmlmhmS0xs9bgMBCocPemDuQ5O7y+sKys7GjqISIiEXrK4PjlwDPu3hyRdpy7F5vZ8cAbZrYaqOxohu4+B5gDUFBQ4N1aWhGROBbNFkcxMDJiPz9Ma8/ltOmmcvfi8H0z8BZwCrAHGGBmrQHvcHmKiEgURDNwLAXGhndBpRAEhwVtTzKzCUAOsDgiLcfM+oXbg4AzgXXu7sCbwFfDU68Bno9iHUREpI2oBY5wHOJm4GVgPfC0u681s/vMLPIuqcuBeWFQaDURKDSzlQSB4gF3Xxceuw34rpltIhjz+M9o1UFERA5mn/y+7pvMrAzYeoTTBgG7j0FxeqJ4rjvEd/1V9/jVkfof5+55bRPjInB0hJkVuntBrMsRC/Fcd4jv+qvu8Vl3OLr6a5JDERHpFAUOERHpFAWOj82JdQFiKJ7rDvFdf9U9fnW5/hrjEBGRTlGLQ0REOkWBQ0REOiXuA8eR1gzpa8zsETMrNbM1EWm5ZvaqmX0QvufEsozRYmYjzexNM1tnZmvN7NYwvc/X38xSzexdM1sZ1v0fwvS4Wt/GzBLNbLmZvRDux0X9zWyLma0O1zcqDNO6/Hcf14GjI2uG9EFzgZlt0m4HXnf3scDr4X5f1AR8z90nAWcAN4X/veOh/vXAdHc/GZgKzDSzM4i/9W1uJZjJolU81f+ccH2j1mc3uvx3H9eBg86vGdLrufsioLxN8iyCtU2gD69x4u4l7v5euF1N8AUygjiovwdqwt3k8OXE0fo2ZpYPXAj8LtyP9/V9uvx3H++Bo8NrhvRxQ9y9JNzeCQyJZWGOBTMbTTDj8jvESf3DbpoVQCnwKvAhHVzfpo/4OfBDoCXc7/D6Pn2AA6+Y2TIzmx2mdfnvvqesxyE9hLu7mfXpe7TNLBN4Fvi2u1cFPzwDfbn+4Xo3U81sADAfmBDjIh0zZnYRUOruy8xsWqzLEwN/Fa5vNBh41cw2RB7s7N99vLc4OrNmSF+2y8yGAYTvpTEuT9SYWTJB0HjS3VvXuI+b+gO4ewXBrNOfJX7WtzkT+LKZbSHokp4O/II4qX/E+kalBD8aTuco/u7jPXB0aM2QOLCAYG0T6MNrnIR92v8JrHf3f4841Ofrb2Z5YUsDM0sDZhCM8cTF+jbufoe757v7aIL/z99w9yuJg/qbWYaZZbVuA+cDaziKv/u4f3LczC4g6PtMBB5x9x/HuEhRZWZPAdMIplTeBdwLPAc8DYwimH7+a+7edgC91zOzvwLeBlbzcT/3nQTjHH26/mY2hWAANJHgB+PT7n5fuDTzPCAXWA5c5e71sStp9IVdVd9394viof5hHeeHu0nAH9z9x2Y2kC7+3cd94BARkc6J964qERHpJAUOERHpFAUOERHpFAUOERHpFAUOERHpFAUOkW5gZs3hzKOtr26bKNHMRkfOZiwSa5pyRKR77Hf3qbEuhMixoBaHSBSF6yD8S7gWwrtmdkKYPtrM3jCzVWb2upmNCtOHmNn8cN2MlWb2uTCrRDN7OFxL45Xw6W+RmFDgEOkeaW26qr4ecazS3U8C/oNglgKAXwKPufsU4EngwTD9QeB/wnUzPg2sDdPHAg+5+2SgArg0yvUROSQ9OS7SDcysxt0z20nfQrCA0uZwgsWd7j7QzHYDw9y9MUwvcfdBZlYG5EdOexFOAf9quOAOZnYbkOzu90e/ZiIHU4tDJPr8ENudETl/UjMan5QYUuAQib6vR7wvDrf/QjBLK8CVBJMvQrCE59/BgYWX+h+rQop0lH61iHSPtHB1vVYvuXvrLbk5ZraKoNVwRZh2C/Comf0AKAOuC9NvBeaY2Q0ELYu/A0oQ6UE0xiESReEYR4G77451WUS6i7qqRESkU9TiEBGRTlGLQ0REOkWBQ0REOkWBQ0REOkWBQ0REOkWBQ0REOuX/A9V2H4tIQ6mZAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#-----------------------------------------------------------\n",
        "# Retrieve a list of list results on training and test data\n",
        "# sets for each training epoch\n",
        "#-----------------------------------------------------------\n",
        "acc= history.history['accuracy']\n",
        "val_acc=history.history['val_accuracy']\n",
        "loss=history.history['loss']\n",
        "val_loss=history.history['val_loss']\n",
        "\n",
        "epochs=range(len(acc)) # Get number of epochs\n",
        "\n",
        "#------------------------------------------------\n",
        "# Plot training and validation accuracy per epoch\n",
        "#------------------------------------------------\n",
        "plt.plot(epochs, acc, 'r', \"Training Accuracy\")\n",
        "plt.plot(epochs, val_acc, 'b', \"Validation Accuracy\")\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.show()\n",
        "print(\"\")\n",
        "\n",
        "#------------------------------------------------\n",
        "# Plot training and validation loss per epoch\n",
        "#------------------------------------------------\n",
        "plt.plot(epochs, loss, 'r', \"Training Loss\")\n",
        "plt.plot(epochs, val_loss, 'b', \"Validation Loss\")\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "LLGuKNLluJ6F",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 546
        },
        "outputId": "a4376d3c-f8f1-4558-eb5a-8088c1c61b95"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcYAAAEICAYAAADFgFTtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xdVX338c93cr+Qe7gJOAgoghaEAIKKCPgIiAqVtqJi0dq0Ykust1ZrbUTpQ/so8HivtIh3EblIUXkpEFBQgSQkRAQUuYRLQkIIgdwmycyvf6y1OWsm58xMkpmcZM73/Xrt177vs/aZM/t71tp7n62IwMzMzJK2ZhfAzMxsR+JgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjN+iDpp5L+cqCXbSZJD0s6cRC2G5L2z8NflfQv/Vl2K17nHZJ+trXlNOuNfB+jDUWSVhejY4EOoDOP/01EfGf7l2rHIelh4L0RccMAbzeAAyLigYFaVlI78BAwIiI2DUQ5zXozvNkFMBsMETG+Gu4tBCQN98HWdhT+PO4Y3JRqLUXScZIek/SPkpYCX5c0WdJ1kpZLWpmH9yrWuVnSe/Pw2ZJulfTZvOxDkk7eymX3lfQLSc9JukHSlyR9u0G5+1PGT0u6LW/vZ5KmFfPPkvSIpBWS/rmX9+coSUslDSumnS7p7jx8pKRfS3pG0hJJX5Q0ssG2LpP0mWL8I3mdJyS9p8eyb5R0l6RnJT0qaXYx+xe5/4yk1ZKOrt7bYv1jJN0paVXuH9Pf92YL3+cpkr6e92GlpGuKeW+RtCDvwx8lnZSnd2u2ljS7+jtLas9Nyn8laTFwU55+Rf47rMqfkYOL9cdI+lz+e67Kn7Exkn4s6e977M/dkk6vt6/WmIPRWtHuwBTghcBM0v/B1/P4PsA64Iu9rH8UcD8wDfgP4L8laSuW/S5wBzAVmA2c1ctr9qeMbwfeDewKjAQ+DCDpIOAreft75tfbizoi4nZgDXB8j+1+Nw93Av+Q9+do4ATgnF7KTS7DSbk8rwcOAHqe31wDvAuYBLwReJ+k0/K8Y3N/UkSMj4hf99j2FODHwOfzvl0I/FjS1B77sNl7U0df7/O3SE3zB+dtXZTLcCTwTeAjeR+OBR5u9H7U8VrgpcAb8vhPSe/TrsB8oGz6/yxwOHAM6XP8UaAL+AbwzmohSYcALyC9N7YlIsKduyHdkQ5QJ+bh44ANwOhelj8UWFmM30xqigU4G3igmDcWCGD3LVmWdNDdBIwt5n8b+HY/96leGT9RjJ8DXJ+HPwl8v5g3Lr8HJzbY9meAS/PwLqTQemGDZT8AXF2MB7B/Hr4M+EwevhS4oFjuxeWydbZ7MXBRHm7Pyw4v5p8N3JqHzwLu6LH+r4Gz+3pvtuR9BvYgBdDkOsv9Z1Xe3j5/eXx29Xcu9u1FvZRhUl5mIim41wGH1FluNLCSdN4WUoB+eXv/vw2FzjVGa0XLI2J9NSJprKT/zE1Tz5Ka7iaVzYk9LK0GImJtHhy/hcvuCTxdTAN4tFGB+1nGpcXw2qJMe5bbjog1wIpGr0WqHf6ppFHAnwLzI+KRXI4X5+bFpbkc/0aqPfalWxmAR3rs31GS5uQmzFXA3/Zzu9W2H+kx7RFSbanS6L3ppo/3eW/S32xlnVX3Bv7Yz/LW8/x7I2mYpAtyc+yz1Gqe03I3ut5r5c/05cA7JbUBZ5JquLaFHIzWinpeiv0h4CXAURExgVrTXaPm0YGwBJgiaWwxbe9elt+WMi4pt51fc2qjhSPid6RgOZnuzaiQmmTvI9VKJgAf35oykGrMpe8C1wJ7R8RE4KvFdvu6dP4JUtNnaR/g8X6Uq6fe3udHSX+zSXXWexTYr8E215BaCyq711mm3Me3A28hNTdPJNUqqzI8Bazv5bW+AbyD1MS9Nno0O1v/OBjNUnPhOtLFHVOAfx3sF8w1sLnAbEkjJR0NvGmQyvhD4FRJr84XypxH3//73wVmkYLhih7leBZYLelA4H39LMMPgLMlHZSDuWf5dyHVxtbn83VvL+YtJzVhvqjBtn8CvFjS2yUNl/QXwEHAdf0sW89y1H2fI2IJ6dzfl/NFOiMkVcH538C7JZ0gqU3SC/L7A7AAeFtefgZwRj/K0EGq1Y8l1cqrMnSRmqUvlLRnrl0enWv35CDsAj6Ha4tbzcFols5njSF9G/8NcP12et13kC5gWUE6r3c56YBYz1aXMSLuAd5PCrslpPNQj/Wx2vdIF4TcFBFPFdM/TAqt54BLcpn7U4af5n24CXgg90vnAOdJeo50TvQHxbprgfOB25Suhn1lj22vAE4l1fZWkC5GObVHufurr/f5LGAjqda8jHSOlYi4g3Rxz0XAKuAWarXYfyHV8FYCn6J7Dbyeb5Jq7I8Dv8vlKH0YWATcCTwN/Dvdj+XfBF5OOmdtW8E3+JvtICRdDtwXEYNeY7WhS9K7gJkR8epml2Vn5RqjWZNIOkLSfrnp7STSeaVr+lrPrJHcTH0O8LVml2Vn5mA0a57dSbcSrCbdg/e+iLirqSWynZakN5DOxz5J38211gs3pZqZmRVcYzQzMyv4R8SHgGnTpkV7e3uzi2FmtlOZN2/eUxExved0B+MQ0N7ezty5c5tdDDOznYqknr+YBLgp1czMrBsHo5mZWcHBaGZmVnAwmpmZFRyMZmZmhV6DMT8f7Q09pn1A0ld6Wefm/AvySPpJvUe0SJotqdETtKtlTstPHq/Gz5PU86nfW03SxZIez88tMzMzA/quMX4PeFuPaW/L0/sUEadExDNbUzDgNNKjY6ptfTIibtjKbXWTw/B00jPUXjsQ22zwOr4dxsxsJ9NXMP4QeGN+hhuS2klPy/6lpK9ImivpHkmfqreypIclTcvD/yzp95JuJT0ItFrmryXdKWmhpCvzE7SPAd4M/D9JC/IPLV8m6Yy8zgmS7pK0SNKl1bPI8ut9StL8PO/AOsUCOA64h/TQ1TOLsuwm6epcloW5HEh6l6S787Rv5WnPlyePr8794yT9UtK1pEfGIOkaSfPyezWzWOekXNaFkm7MPyb9B0nT8/w2SQ9U42ZmNvh6DcaIeBq4g/Qkb0i1xR9E+oHVf46IGcCfAK+V9CeNtiPp8LzuocApwBHF7Ksi4oiIOAS4F/iriPgV6WneH4mIQyPij8W2RgOXAX8RES8n/UhB+bDUpyLiMFLoNWquPZNU672aFPwj8vTPA7fkshwG3CPpYOATwPF5+qxG+1k4DJgVES/O4++JiMOBGcC5kqbmsLsEeGve7p/lh5B+m/ScPkhP8F4YEct7voCkmfmLydzlyzebbWZmW6k/59fK5tSyGfXPJc0H7gIOpmj2rOM1wNURsTYiniWFXuVluYa1iBQIB/dRnpcAD0XE7/P4N0hPGa9clfvzgPaeK+fa7ynANbkstwPVedTjSYFKRHRGxKo87Yrqoaf5y0Jf7oiIh4rxcyUtJD1wdG/gAOCVwC+q5YrtXgq8Kw+/B/h6vReIiK9FxIyImDF9uiuUZmYDpT/nwH4EXCTpMGBsRMyTtC+pNnZERKyUdBkweivLcBlwWkQslHQ2qZlzW1RPQO+k/v69AZgELJIEMBZYB1y3ha+zifzFIp+zHFnMW1MNSDqOVPM7OiLWSrqZXt6riHhU0pOSjgeOpFZ7NDOz7aDPGmNErAbmkGoyVW1xAungv0rSbtSaWhv5BXCapDGSdgHeVMzbBViSmzPLEHguz+vpfqBd0v55/Czglr72o3Am8N6IaI+IdmBf4PX5AZ83kptlJQ2TNBG4CfgzSVPz9Cl5Ow8Dh+fhNwMjqG8isDKH4oGkmiKk2uOx+UtGuV2A/yI1qV4REZ1bsG9mZraN+nurwveAQ3KfiFhIakK9j/RAzNt6Wzki5gOXAwuBnwJ3FrP/hdSceVveXuX7wEfyRTb7FdtaD7wbuCI3v3YBX+3PTuTwOwn4cbG9NcCtpLCeBbwub3cecFBE3AOcD9ySm0MvzKteQjq3uhA4mqKW2MP1wHBJ9wIXkAKRfN5wJnBV3sblxTrXAuNp0IxqZmaDxw8q3gHl+0AviojX9Gf5GTNmhJ+uYWa2ZSTNyxeRduP77HYwkv6J1Jzrc4tmZk3gYNzBRMQFpCZXM9sBRMCmTbWuUSNbV1ea39mZunK4q6t+F1HrqteqXm/DhtR1dNSGAYYP37yLSMt1dMD69bXhDRugrQ2GDUtdNdzWlpZbtw7Wrk1dNSzByJEwYkTqquG2tto+lfsGMHp0927MmLTO2rWwZk33bu3azd+banjEiLRutY2qq8pbr/v+99N7MJAcjGYtojrgSulAI6WunF8dsKsDVWdnOmBWB82yv2EDbNy4ebd+/eYHxLVr04G6UbnqHSS7uhov39Gx+QF3zZrawbU6mFf9YcNS2TZtqpWzCrpK+X5ItQBoVI6hpK0Nxo5NIRSx+d+0pypoq1Bev77xF4bKiBEwblx6nepv0jOwN25Mn60qtNetq71+W1v30KxCeONGB6PZkLJpUy04qvAov+2X42VNoBquvjVXB5OqW70ann0WVq1K/arrrHONcxUIAx0AUjoIVgfDUaO6B3Gp3kGyCu96Ro2CPfaobXvcuNRVB9ey27ChFpgjRqSDaDU8bFh6jXq1tno1s6pcjfa3Wqbq99yvsqu+oFTrlqFchXrVjRqVpknda69lLXb06LRc2R8xovZlo+eXjipgqqBq9F5XX6i6utJ+bfZ3iXh+mfUdev6zuKEjGDtyE+OGdzB2+AZGxIZa1betrf4b3NlZWyZ3nes20LWxkxGNrvsfdSgD/TwMB6PZFohIAbZqVS101q6t38SzahU8/TSsWJH61fCqVbUaTtU8ti16NmONGgXjx8OECbDffqk/YQJMnJjmVwf+qimv6tcLp2HDas1ZVY2i6pfNbWU3alQKqTFjGh9sLevqgmeeqX1Iqn5nZ/dqWTXcs1236jo6atX0srq+bl33ttiyq9K4bEJoa0t/xOLDpNGjGVFVzZ5+GlauTF01vG4dIt2vNoL699hti2G5a2jdulTWAeRgtJbT0VELqHpd9f9ehVk1Xs2vV+tqZPRomDoVpkxJ3UtekgKqquGUNZ6qVlV1VU2hZy2g7PdWCxuyqra+sqpcVpnLqnY1vHFj9xN65beDqrpetvtWJ9vq1Wo2bEhV8jVrUr/q1q/fvFpW9Xt+G6naj1et6rsNckuNHl37cI0ZU/swVR+oqooIm5etqyu9V6tWwbJltaaJdevSvk+ZApMnwwEHpP7kyelbWFXtrv4+lfJ1q+ERI+oH/MaNKfx7Lj9yZJreSMOq5NZzMNqQ0dkJTz4Jjz9e6554InVLltSGV6zofTvDh6f/9yrM9tgDXvpSmDQphdrEibUa2IQJ6fhTXnRQhdbEiWl8yIhI3xKWLIGlS9PBcNy4dGCsDsTjx6cD3IoVqXvqqdrwM8/UThyVJ5E6OrpfoVIdpKsTnGUtqAqtLfl20l9tbbVvKGPHpmn1Dt4jR6b9rLpx42DvvdMfvmdbcNkm3PPkbltb7YM2dWr3b1BVs2LZbdrUuAmyPIHXqK3X+s3BaDu8iPTl9cEH0/F46dIUgNXw0qUpBJcs2fx4OWwY7L477LknvOhF8KpXpeHp02shV3aTJtW+AO/0Ojq6N8+tWJECqayWVikupWWWL09hVvarbxVLlza+gqa/qssOy27UqNrJq57dpEnpD1YGVtWVly5Ww/Wq3FUtpWw6LMOpqrKPHDlE/vC2rRyMtsPYuBEWLYK774Y//AEeeKDWf+657stKKdx22y11J5wAL3gB7LVX6lfd9Om9t8I03aZN3WtW5ZUyVbd6de0g3vMqjtWrU01s5cru/aefTjWsrdXWlmow06alKvNrXpP6e+6Z+rvvnspUNT+WTYvDh9dqQNU2pk5N3zwG+vJBs0HgT6k1RVcX3H8/3HlnrVuwoFYhGTYM2tvTqYxXvzr199uvdkyePn0HO8ZGpGBYtSoFU3Wysqp5lbWwcvyZPp7jXTXvVZeN9jx3NX58qlVNnpz6++2X+lWTXNU8V/XHjOl+qWt1aWtXV5o/fXoKssmT3SRnLWtHOrTYEPXcc6kWuHBhrVu0KJ0qgnTcP/xw+Lu/gyOOgFe8Avbdd1DOqffuySdTFbW8F6IKj3XraufK6p076+uqnJEja6EzbVra4Wq4mj51au0S0qobO9bNe2bbmYPRBtSzz8L8+TB3bq374x9r8ydNgkMOgfe+NwXgEUfAgQc2oblz3bpU0Ntvr3WPPNL3ehMmdG8ifPGLa1fllFfnTJxYW2batCF04tJs6HMw2ja7/Xb46lfhN79JzaPV1dovfGGqGL373SkMDzkknQMc1Hzo6kpX6jz2WOqqK3Wqbtmy1H/oodrPnuyzDxx1FPz938PLXpaqsPXuk5g8OdX8zGxIczDaVomAn/8cLrgA5sxJFaRjj4W3vx1mzEiBuOuug1iAFStSjW/+/HRycvHiFIRPPNH9d74qU6bUrtQ57DA444wUhkcdlU5ampllDkbbIp2dcPXVKRDnzUsXKX72szBzJuwy0D95UVm+PL1Y1c2f373Zs7093Ytx3HG1S1Ory1P32COdw9vuJyzNbGflYLR+iYCrroKPfxx+/3vYf3+45BI466zU0jhgnnsOfvWrdHJy3rzUf/TR2vwDDoCjj4b3vz/V/A47LDVxmpkNEAej9emee+Dcc+Gmm9IpuMsvh7e+dQAvmFm2DK69NlVFb7ih9gOi1b0ahx+eule8IrXZmpkNIgejNfTMMzB7Nnzxi+lizC9+Ef7mbwbo/sEHH4Qf/SiF4W23pYtm2ttTTfCNb0wnKh2CZtYEDkbbTFcXXHYZfOxj6fTezJnwmc+kuw622tq1cMst8NOfwvXXp/sFAV7+cvjEJ+D009Nlq76lwcyazMFom/nbv03nD485JuXYYYdt5YY2bIBLL021wltuSTfKjxkDr3tdujXilFPSL7WYme1AHIzWza23plCcNQsuumgbKnA33JDC77770h3855wDJ52UfnNzSD1ywsyGGgejPW/TppRf++wD55+/laG4eDF88INw5ZWpNvg//wOnnjrgZTUzGywORnveF76QfsP0qqvSj79skfXr4XOfS4kKqf/BDw74k7XNzAabg9GA9DzDT34STj4ZTjttC1e+915405vSj6KecUYKyH32GZRympkNNgejAfChD6XnIX7hC1vYhPq736WLaaT0G3EnnjhoZTQz2x4cjMaNN6ab9mfP3sKLRH/7Wzj++HSn/5w56SIbM7OdnJ9E2uI6OtI99fvtB//4j1uw4qJFqaY4fDjcfLND0cyGDNcYW9yFF6ZHRf3kJ1twnczChXDCCWmFOXPST7eZmQ0RrjG2sEcegU9/Ov3ozMkn93OlBQtS8+mYMamm6FA0syHGwdjCZs1K18xcfHE/V7j33hSK48alUNx//8EsnplZU7gptUVt2pQeVzh79hbcWfHRj6b+zTen5x+amQ1BDsYWNXw4fOlLW7DCvHlw3XWp7dWhaGZDmJtSrX/OOw8mTUq/f2pmNoQ5GK1vd92VHiT8D//gZySa2ZDnYLS+nXdeCsRzz212SczMBp2D0Xq3YAFcc02qLU6a1OzSmJkNOgej9a6qLc6a1eySmJltFw5Ga+zuu+Hqq1MourZoZi3CwWiNnXceTJgAH/hAs0tiZrbdOBitvkWL4Mor0wU3kyc3uzRmZtuNg9Hq+/SnYZdd0kU3ZmYtxMFom/vtb+GKK1JtccqUZpfGzGy7cjDa5r785fRD4a4tmlkLcjDa5n7/e3j5y2Hq1GaXxMxsu3Mw2uYWL96CR26YmQ0tDkbrLiIF4957N7skZmZN4WC07pYvh44O1xjNrGU5GK27Rx9NfQejmbUoB6N1t3hx6jsYzaxFORitOwejmbU4B6N1t3gxjBnjWzXMrGU5GK276lYNqdklMTNrCgejded7GM2sxTkYrTsHo5m1OAej1XR0wNKlDkYza2kORqt5/PHUdzCaWQtzMFpNdauGfw7OzFqYg9FqfA+jmZmD0QpVMO61V3PLYWbWRA5Gq1m8GHbdNd3gb2bWohyMVuNbNczMHIxWcDCamTkYLaseUOxgNLMW52C0ZOVKWLPGwWhmLc/BaIkfUGxmBjgYreJ7GM3MAAejVfyrN2ZmgIPRKosXw8iR6T5GM7MW5mC0ZPHiVFts80fCzFqbj4KW+FYNMzPAwWgVB6OZGeBgNICNG+GJJxyMZmY4GA1SKHZ1ORjNzHAwGvjmfjOzgoPRfHO/mVnBwWi+ud/MrOBgtBSMU6fCuHHNLomZWdM5GK12c7+ZmTkYDd/DaGZWcDCag9HMrOBgbHWrVqXOwWhmBjgYzfcwmpl142BsdQ5GM7NuHIytzjf3m5l142BsdYsXw/DhsPvuzS6JmdkOwcHY6hYvhr32gmHDml0SM7MdgoOx1flWDTOzbhyMrc6/emNm1o2DsZV1dsJjj7nGaGZWcDC2sqVLYdMmB6OZWcHB2Mp8q4aZ2WYcjK3MN/ebmW3GwdjKXGM0M9uMg7GVLV4MEyfChAnNLomZ2Q7DwdjKfA+jmdlmHIytzMFoZraZ4c0ugDXRmWfCbrs1uxRmZjsUB2Mr+8hHml0CM7MdjptSzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKzgYDQzMys4GM3MzAoORjMzs4KD0czMrOBgNDMzKzgYzczMCg5GMzOzgoPRzMys4GA0MzMrOBjNzMwKDkYzM7OCg9HMzKwwIMEoaaqkBblbKunxYnxkH+vOkPT5frzGrwairMX2Ls7l9JcDMzN73vCB2EhErAAOBZA0G1gdEZ+t5ksaHhGbGqw7F5jbj9c4ZiDKmsvTBpwOPAq8FpgzUNvu8ToN99vMzHZMg1ZbknSZpK9Kuh34D0lHSvq1pLsk/UrSS/Jyx0m6Lg/PlnSppJslPSjp3GJ7q4vlb5b0Q0n3SfqOJOV5p+Rp8yR9vtpuHccB9wBfAc4sXmM3SVdLWpi7Y/L0d0m6O0/7VrF/ZzQo3y8lXQv8Lk+7JpfpHkkzi3VOkjQ/b/dGSW2S/iBpep7fJumBatzMzAbfgNQYe7EXcExEdEqaALwmIjZJOhH4N+CtddY5EHgdsAtwv6SvRMTGHsu8AjgYeAK4DXiVpLnAfwLHRsRDkr7XS7nOBL4H/Aj4N0kj8mt8HrglIk6XNAwYL+lg4BN5P56SNKUf+30Y8LKIeCiPvycinpY0BrhT0pWkLyWXFOWdEhFdkr4NvAO4GDgRWBgRy3u+QA7YmQD77LNPP4pkZmb9Mdjn166IiM48PBG4QtJvgYtIwVbPjyOiIyKeApYBu9VZ5o6IeCwiuoAFQDspUB8swqhuMOZznqcA10TEs8DtwBvy7ONJtUgiojMiVuVpV+TyEBFP92O/7yjKAXCupIXAb4C9gQOAVwK/qJYrtnsp8K48/B7g6/VeICK+FhEzImLG9OmuUJqZDZTBrjGuKYY/DczJtbF24OYG63QUw53UL2N/lmnkDcAkYFFugR0LrAMaNbs2son8xSKfsywvMnp+vyUdR6r5HR0RayXdDIxutNGIeFTSk5KOB44k1R7NzGw72Z5XZE4EHs/DZw/C9u8HXpRDF+AvGix3JvDeiGiPiHZgX+D1ksYCNwLvA5A0TNJE4CbgzyRNzdOrptSHgcPz8JuBEQ1ebyKwMofigaSaIqTa47GS9u2xXYD/Ar5N9xq3mZltB9szGP8D+L+S7mIQaqoRsQ44B7he0jzgOWBVuUwOv5OAHxfrrQFuBd4EzAJeJ2kRMA84KCLuAc4HbsnNoRfmVS8BXpunHU332nHpemC4pHuBC0iBSD5vOBO4Km/j8mKda4HxNGhGNTOzwaOIaHYZBoyk8RGxOl+l+iXgDxFxUbPLtaUkzQAuiojX9Gf5GTNmxNy5fd7xYmZmBUnzImJGz+lD7eb2v5a0gHQrxkTSVao7FUn/BFwJfKzZZTEza0VDqsbYqlxjNDPbcq1SYzQzM9smDkYzM7OCm1KHAEnLgUe2cvVpwFMDWJydhfe7tXi/W0t/9/uFEbHZL6Q4GFucpLn12tiHOu93a/F+t5Zt3W83pZqZmRUcjGZmZgUHo32t2QVoEu93a/F+t5Zt2m+fYzQzMyu4xmhmZlZwMJqZmRUcjC1K0kmS7pf0QP591iFL0qWSluWHZFfTpkj6uaQ/5P7kZpZxMEjaW9IcSb+TdI+kWXn6kN53SaMl3SFpYd7vT+Xp+0q6PX/mL88PLR9y8iPz7pJ0XR4f8vst6WFJiyQtkDQ3T9vqz7mDsQVJGkZ6+sjJwEHAmZIOam6pBtVlpMeNlf4JuDEiDiA9h3MofjnYBHwoIg4iPQf0/fnvPNT3vQM4PiIOAQ4FTpL0SuDfSU+t2R9YCfxVE8s4mGYB9xbjrbLfr4uIQ4v7F7f6c+5gbE1HAg9ExIMRsQH4PvCWJpdp0ETEL4Cne0x+C/CNPPwN4LTtWqjtICKWRMT8PPwc6WD5Aob4vkeyOo+OyF0AxwM/zNOH3H4DSNoLeFg2zx4AAAITSURBVCPpYefkR/AN+f1uYKs/5w7G1vQC4NFi/LE8rZXsFhFL8vBSYLdmFmawSWoHXgHcTgvse25OXAAsA34O/BF4JiI25UWG6mf+YuCjQFcen0pr7HcAP5M0T9LMPG2rP+fDB7p0ZjubiAhJQ/a+JUnjSc/4/EBEPJsqEclQ3feI6AQOlTQJuBo4sMlFGnSSTgWWRcQ8Scc1uzzb2asj4nFJuwI/l3RfOXNLP+euMbamx4G9i/G98rRW8qSkPQByf1mTyzMoJI0gheJ3IuKqPLkl9h0gIp4B5gBHA5MkVZWBofiZfxXwZkkPk06PHA/8f4b+fhMRj+f+MtIXoSPZhs+5g7E13QkckK9WGwm8Dbi2yWXa3q4F/jIP/yXwoyaWZVDk80v/DdwbERcWs4b0vkuanmuKSBoDvJ50fnUOcEZebMjtd0R8LCL2ioh20v/0TRHxDob4fksaJ2mXahj4P8Bv2YbPuX/5pkVJOoV0PmIYcGlEnN/kIg0aSd8DjiM9iuZJ4F+Ba4AfAPuQHtn15xHR8wKdnZqkVwO/BBZRO+f0cdJ5xiG775L+hHSxxTDSl/8fRMR5kl5EqklNAe4C3hkRHc0r6eDJTakfjohTh/p+5/27Oo8OB74bEedLmspWfs4djGZmZgU3pZqZmRUcjGZmZgUHo5mZWcHBaGZmVnAwmpmZFRyMZmZmBQejmZlZ4X8B+lFhSlK7ZyoAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAD4CAYAAAC0VQLEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfEUlEQVR4nO3de3RdZZ3/8fc3bdP0RpLSCxQKIUGKIFBKwSIjUkBFnHFwxNvyfmM5Px3U8Qa/8edlBp3BK4OLBaKizIgoCqijjoNoEQUHmgq0pdzsRRCBpqVpmzakTfv9/fHdu2cnza1tTvbZ53xea+21T8712TScT77P8+xnm7sjIiJSFHV5N0BERGRfKLhERKRQFFwiIlIoCi4RESkUBZeIiBTK+LwbUAtmzJjhLS0teTdDRKRQli1btsHdZ/a/X8E1BlpaWmhvb8+7GSIihWJmfxrofnUViohIoSi4RESkUBRcIiJSKAouEREpFAWXiIgUioJLREQKRcElIiKFouCqZN/5DlxzTd6tEBGpKAquSnbTTfC1r+XdChGRiqLgqmRNTdDZmXcrREQqioKrkjU3w6ZNebdCRKSiKLgqWVMTbNkCu3fn3RIRkYqh4Kpkzc3gDps3590SEZGKoeCqZE1Nsdc4l4jIHgquSqbgEhHZi4KrkjU3x14TNERE9lBwVTJVXCIie1FwVTJVXCIie1FwVTJVXCIie1FwVbJp06CuThWXiEiGgquSmWnZJxGRfhRclU7BJSLSh4Kr0mm9QhGRPhRclU4Vl4hIHwquSqeKS0SkDwVXpVPFJSLSh4Kr0im4RET6UHBVuuZm6O6Gnp68WyIiUhEUXJVOq2eIiPSh4Kp0Wq9QRKQPBVelU8UlItKHgqvSKbhERPpQcFU6dRWKiPSh4Kp0qrhERPpQcFW6NLhUcYmIAAquytfQEJsqLhERQMFVDFo9Q0RkDwVXEWihXRGRPRRcRaCKS0RkDwVXEajiEhHZQ8FVBKq4RET2UHAVgSouEZE9FFxFkFZc7nm3REQkdwquImhqgt27oasr75aIiOROwVUEWq9QRGQPBVcRaL1CEZE9FFxFoIpLRGQPBVcRqOISEdlDwVUECi4RkT0UXEWgrkIRkT0UXEVw0EGxV8UlIqLgKoRx46CxURWXiAgKruLQeoUiIoCCqzgUXCIigIKrOLTQrogIoOAqDlVcIiKAgqs4VHGJiAAKruJQxSUiAii4iqO5OS5r0tubd0tERHKl4CoKLfskIgIouIpDwSUiAii4ikPrFYqIAAqu4lDFJSICKLiKQxWXiAig4CoOVVwiIoCCqzgUXCIigIKrOCZPhgkT1FUoIjVPwVUUZlo9Q0QEBVexaL1CEREFV6Go4hIRUXAVioJLRETBVSjqKhQRUXAViiouEREFV6GkFZd73i0REcmNgqtImppg507o7s67JSIiuVFwFYlWzxARUXAVihbaFRFRcBWKKi4REQVXoajiEhFRcBWKKi4REQVXoaQVl4JLRGqYgqtIGhtjr65CEalhQwaXmS0xs5f3u++DZnb1EK+5w8wWJrd/bmZNAzzn02b2kWE++wIzOy7z8z+b2blDvWYkzOwsM/vpgb5PLiZMgClTVHGJSE0bruK6EXhDv/vekNw/LHc/393391v2AmBPcLn7J9399v18r+qh9QpFpMYNF1w/BF5pZvUAZtYCzAF+a2ZXm1m7mT1oZp8Z6MVmts7MZiS3/8nMHjWz3wHzMs95j5ktNbMHzOxmM5tsZi8CXgV8wczuN7M2M/u2mV2YvOYcM7vPzFaY2XVmNjHzeZ8xsz8kjx070v8QZvbG5DUrzezy5L5xyeeuTB77UHL/xWa2ysyWm9n3RvoZo0LrFYpIjRsyuNz9WeBe4BXJXW8AbnJ3B/7J3RcCJwIvMbMTB3sfMzslee184Hzg1MzDt7j7qe5+EvAQ8C53vxv4CfBRd5/v7qsz79UAfBt4vbufAIwH/j7zfhvcfQFwNTBkd2TmPecAlwNnJ2081cwuSG4f5u4vSD7rW8lLLgFOdvcTgfcO8p4XJcHe3tHRMZJmjIwqLhGpcSOZnJHtLsx2E77OzP4A3AccT6ZbbwAvBm519+3uvoUIpdQLzOy3ZrYCeFPyXkOZB6x190eTn68Hzsw8fkuyXwa0DPNeqVOBO9y9w917gRuS91wDtJrZV83sPGBL8vzlwA1m9magd6A3dPdr3X2huy+cOXPmCJsxAqq4RKTGjSS4fgycY2YLgMnuvszMjiKqmXOSquNnQMN+tuHbwPuTiuYzB/A+qZ5kv4uoxvabu28CTgLuICqrbyQPvRK4ClgALDWzA/qcfaLgEpEaN2xwuXsXsAS4jlK1dRCwDdhsZrMpdSUO5k7gAjObZGbTgL/JPDYNeMrMJhAVV2pr8lh/jwAtZnZ08vNbgN8MdxzDuJfo7pxhZuOANwK/Scbn6tz9ZuATwAIzqwPmuvsS4ONAIzD1AD9/5NRVKCI1bqSVwo3ArSRdhu7+gJndBzwMPAHcNdSL3f0PZvZ94AFgPbA08/D/A+4BOpJ9GlbfA75uZhcDF2be6zkzewfwg6TSWQpcM8LjSJ1jZn/O/PxaYtxqCWDAz9z9x2Z2EvCtJKwALgXGAd8xs8bkuVcewMzJfdfUBFu2wO7dUKfT8ESk9pjrooRlt3DhQm9vbx+dN7viCvjQh+DZZ0sraYiIVCEzW5ZMAuxDf7IXjdYrFJEap+AqGgWXiNQ4BVfR6NImIlLjFFxFo4pLRGqcgqtoVHGJSI1TcBWNKi4RqXEKrqKZNi3O31JwiUiNUnBVsJtuguuu63enWVRd6ioUkRql4Kpg3/0ufOlLAzyg9QpFpIYpuCpYWxusWQN7LW6i9QpFpIYpuCpYays89xw89VS/B1RxiUgNU3BVsLa22K9Z0++B5mYFl4jULAVXBUuDa/Xqfg9ocoaI1DAFVwU78siY+b5XxaWuQhGpYQquClZfD3PnDlBxHXYYdHcPMPglIlL9FFwVrrV1gIrrhS+M/T33jHl7RETypuCqcG1tA1RcJ58MEybA73+fS5tERPKk4Kpwra2wfj1s3Zq5s6EBFixQcIlITVJwVbh0ZuHatf0eWLQI2tth584xb5OISJ4UXBWutTX2e3UXnn56TNBYvnzM2yQikicFV4Ub9CTkRYti/7//O6btERHJm4KrwjU3x7ZXxXXEEXDooRrnEpGao+AqgAGnxJtFd6EqLhGpMQquAhhwSjxEd+Hq1THtUESkRii4CqC1Fdatg127+j1w+umx14nIIlJDFFwF0NYGvb3wxBP9HjjlFBg/XuNcIlJTFFwFkE6J32uca9IkmD9fwSUiNUXBVQCDXt4EYpxr6dIoyUREaoCCqwAOPzyWJtyr4oIY59q2DVauHPN2iYjkQcFVAOPGQUvLIBVXOkFD0+JFpEYouApi0CnxLS0wa5bGuUSkZii4CmLAk5BBJyKLSM1RcBVEWxt0dsKzzw7w4KJF8OijsHHjmLdLRGSsKbgKYtAp8aATkUWkpii4CmLIKfELF8YMDo1ziUgNUHAVxFFHxX7AimvKFDjxRAWXiNQEBVdBTJ0Ks2cPUnFBjHPde+8ACxqKiFQXBVeBDDqzEGKca+tWWLVqTNskIjLWFFwFMui5XKATkUWkZii4CqStLVaI37FjkAdnzNA4l4hUPQVXgbS2gntcm2svZjHOpYpLRKqcgqtA0inxg45zLVoEDz0EmzaNWZtERMaagqtA0pOQBx3nWrw49v/5n2PSHhGRPCi4CuSQQ+LakUNO0Fi8GC67LGYYiohUIQVXgZgNMyXeDP7t36CjA770pTFtm4jIWFFwFcyQU+IBTjsNXvOaCK7168esXSIiY0XBVTBpxeU+xJM++1no7o69iEiVUXAVTFsbbN8OzzwzxJPmzYN3vhOuvhrWrh2ztomIjAUFV8EMOyU+9alPxYrxn/xk2dskIjKWFFwFM+yU+NRhh8EHPgA33ADLl5e9XSIiY0XBVTAtLTF5cNiKC+DjH4fGRrj00nI3S0RkzCi4CmbiRDj88BFUXADNzXDJJfDzn8Odd5a9bSIiY0HBVUBtbSOsuAD+4R9gzpyovoaciigiUgwKrgJqbR1hxQUweTJ8+tOx+K6WghKRKqDgKqATT4Snn4b77hvhC97xDnjxi+Hd74bbbitr20REyk3BVUBvf3vMubjsshG+YPx4+MlP4PnPh1e/Wpc+EZFCU3AVUGMjXHwx3HILrFw5whc1NcH//A8ceiicfz48+GBZ2ygiUi4KroL64Adh6tR9XNXpkEPgl7+EhgZ42csGuSKliEhlU3AV1PTp8L73wfe/D488sg8vPOqoqLy2b4eXvnSYtaNERCqPgqvA/vEfo3j63Of28YUnnBDndv3lL3DeebB5c1naJyJSDgquAps1C9773ljVacTndaVOPx1uvjkGyU4/XRM2RKQwFFwF99GPxqTBf/3X/XjxeefBf/83dHXBGWfARz4SXYgiIhVMwVVwhx4ap2ddfz08/vh+vMG550bVddFFcfHJk06C3/521NspIjJaFFxV4GMfi/3nP7+fb3DQQXHtrl/9CnbtgjPPjKWiurpGrY0iIqNFwVUFjjgC3vY2+MY34KmnDuCNzj4bVqyIy6FcdRUceyx885vQ2ztqbRUROVAKripx6aWRL1/4wgG+0ZQpcMUV8LvfxTL07353rDF1661apFdEKoKCq0q0tsKb3gTXXANr147CG77oRfD738fyHO7wd38X9/3mN6Pw5iIi+0/BVUU++ck4r2vx4lFaFMMs1jZcsSL6IZ94As46C17xCk2fF5HcKLiqSFsb3H47bNkCL3nJKFVeEPPt3/UueOyxmAGydGmc+3XuuarARGTMKbiqzIIFEV5bt0ZxNGrhBTBpUpw4tm4dfPGLMY3+rLNiFuJtt2kMTETGhIKrCi1YEDPbu7oiV/Z5VY3hTJ0KH/5wpOJXvxr7l78cXvhC+PrXYdOmUf5AEZESBVeVOvnkqLy6umLMa9TDC6ICe//74Y9/hGuvjT7Kiy6KVehf/Wr4wQ+gu7sMHywitUzBVcVOPrlv5XX//WX6oIkT4T3vgYcegvb2WLb+nnvgda+D2bPjypc33wwbN5apASJSSxRcVW7+/Aivnp7oQnznO2NR+LIwg1NOgS9/OWYg3n47XHhhnAN24YUwc2ak6Yc/DD/9aVRoIiL7yFwD6mW3cOFCb29vz7UNnZ1x0ckrr4xJgh/7WKypO2XKGHz4zp0xE/HXv4YlS+CuuyJJx42LcbHzz49t/vwIPxERwMyWufvCve5XcJVfJQRXas0auOSSGH6aMwcuuwze+tbIkDHz3HNxcvOvfw2/+EV0L0KsGJyG2LnnxhqKIlKzFFw5qqTgSt19d1yI8p57YN68WJ7wrW8dowqsv2eeiQD7+c/j6sybN5eqsXPOiRBbtAjq63NonIjkRcGVo0oMLojTrn74wzinuL0dmpvjwpTvex8cdlhOjdq5M6qx226LMbKlS2H3bpg8Oc6qXrw4xslOOCEmfohI1VJw5ahSgyvlHsNOX/kK/OhHUFcHr399XNnktNNyHnbq7IQ77ogZJrffDg8/XHps1qxYAPjEEyPIjjkGjj46JoForEyk8BRcOar04MpauzYmcHzzm7H6xrx58Ja3wJvfDEcemXfrgI6OWDtx+fLS9uCDMW6WmjYt1r86+ujYjj0Wjj8env/8nPpCRWR/KLhyVKTgSm3ZEhM4/uM/4M47474zz4wQe+1robEx3/b1sWsXrF4daymuXh0nRKfb2rWl64mZQUtLhNjxx8Nxx0UyH3NM9JOKSEVRcOWoiMGVtW4d3HBDhNijj8YciZe+FF7zGnjVq+Dgg/Nu4RB6eyPMVq6MyizdHnmk7wUyZ84shdgxx8BRR8XW0gIzZqjrUSQHCq4cFT24Uu4xV+LGG+MyXY8/HpP/Fi+OELvggljtqRB27IhzAx59NELskUdKt9ev7/vcqVMjwFpbo7vxuONiO/bYeExEykLBlaNqCa4s95iJeMstsZrTY4/F/ccdB2ecUdra2gpYrGzZEmXm2rWxpbdXr45w27mz9Nwjj4wwmzUrzjtrbOy7nzMnAm/OnDE+WU6k+BRcOarG4Mpyj564//ov+N3v4hyxzZvjsdmz48LJp54aC2PMnx9VWeHCLLVzZwTYqlWl7eGH4dlnI/A2b47p+/3V10fItbbGdvjhMH169LNOn17amptjAolCTkTBladqD67+du+O7/O77ort7rvjuz41a1YpxE46KWayz5tXJecXu8P27RFinZ3w5JPRJblmTVRta9bEf4zhLv1SXx8BNmVKnMM2bRrMnVsae8uOwWmmpFQpBVeOai24BtLZGTPX77+/tK1cWep1Gz8+wuuEE+AFL4hJf/PmRXEycWK+bS+L556LKi3dNm6M/aZNEXzbtvXdb94Mf/pTdFv2v1TM3LnRR5vOlEz32SWz3GPbvTtO1KvT+tpS+RRcOVJwDWzHjpgLsXJlnJq1YkXcXreu9Jy6uigs0sl+xxwDz3tebHPn1mCPmntMHknH39asicvJrFoV++z5bOPHR1CloZWqq4uZkjNnRvmb7vvfTrfGxgL37UqRKbhypODaN1u2xLDRo4/uvW3bVnpefX1M/kiDrKUlhpGOOCL2Nfd9u2tXpP6qVTHlf+vWCCmzUpVVVxfh1tERAZjdd3YO/L7jx8fsycmTY0u7LydPhqamGJdLx+fSfXZraop/jJr7K0MOlIIrRwqu0eEe1xJ77LG9t9Wr+xYbEMNCRx4ZW0tLbOmwUEtLfMfWVLANZ8cO2LAhgqx/qG3btnf35bZt0YWZdnfu2DH4e5tF1+X06VHtpRVfum9ujj7h+vrYJkwo3U5fM3MmNDSM3X8PyZ2CK0cKrvJLe9AefzyGgtJ9uq1bt3dB0dAwcO/Y7Nkx8/HQQ0v7piaF3JDcY+xt06bSWF1nZ+zTrbMzxvI2bIitoyP227eP/HOmTCkF3vjxEZb9t4kT4znZrtD09uzZpW3mzHgPqViDBZf+1aQqmJW+j049deDndHZGgKWnZT31VKm4WL8+xteeeWbgwmHixAixbNBl99Onl3rN0m3SpBoKO7NS9+G+Xlpg+/YItjR4du4s3e7piSBMQy7db9gQ43dpVZat1NKu0L/8BR54IG739Az82QcfHP+AEydGiGW3CRPiH/aQQ0pb+ldN+jk9PaUtLfkbGkrbpEml22k366RJ6jY9QAouqRlNTaVp+INxj96vp5+OYEv36e2Ojri9fPnQ34cQ36Pp9+JgW7bXbPr0Gi0A0sArF/cY71u/Pv4y6b9t2BBh2dvbd9u2Lcr1Z56JgdfRNHFi6XSH5ub4RUm39Py+SZMiINMtDebsjNBsj9nEiaWqctaseG6VqsX/TUQGZRYB19QUKzoNJf0+7OjYu0csvZ0WCR0dsdJIR0fp5OyBPjtdcCO7TZs29O2pU/tu6fdhTYbgQNLxtYMOiqsF7I/u7giwp5+Orbc3gqKhIfbpBqXqK916eqKqTLfsGGFXV/yibNwYJX96WsSuXQd+3M3NEWJNTVG9dndHe9L9jh3xC5P+wqcTaZqaShXouHGlLa1C6+v7HnO69a9+06CdN2/Uxyb1qy2yn7Lfh/uip6c01NO/B2zjxgjDLVti/+yz0bWZ/rx168g/p6Ehwm2grbGxtGVXqEonDGYnDk6eXPpemjChhro/syZNKs3qKTf3+Ad/7rmoBNOu0/R2/3kJ6T9Id3epqszuOztLIZvtuqyvj+Ds7Cxta9fGfseOCOddu2Lr7R14RZiRWLUqlkUbRQoukTE2cWIsXThnzr6/dvfu+K7ZsqW0pX+4d3X1vd3VVQq7dOvoiBmYmzfHa/ufyzwS2T+oJ03qG3DZGfNpBTht2t630wBNb6erXI0bFz1h2X3NDQelpXdFXTuICMze3r7jej09pbHIgSbK7NgRy5uNMgWXSIHU1e1flTeYHTv6huBAi3Zs3z7w91LaI5btBdu+ParGNEC3bo39gRg/vm+xkO7T4Z7+W3a1rOw2eXKpZ6//Ph1GSnvD0tvZmfn9h5myhU96e/z4+JyqDFuz0n+cnK+KoOASqWH19aUJIuWye3dUdmnVl60G0/22bdEjtXt33336B352aKa7O7b0ef237u7odk1PPUu30Rg2GqmGhr6hmU4kHChos6esZbdsBZqtQrOTHrP7urq+PXvpbfe+Ex2zW1od99/S8VGzUk9k9naq/2Pplp73PtBrRoOCS0TKqq6u9IWY1/Xa3GN4KDtvIr3d01MaPurtLd1Of87O0E/3aYWV/eKGeLx/YG7bFmE6UMju2hWPp8NK2Wo2G8zZMN+1q9S2kTDbe1hsLD300PATnfaVgktEqp5ZqZKZNi3v1owO91J1tXNnBFv/iYBpl2Zvb9+Jjmnlmu0W7l+dZpe47L/cZXpfKrskZnY9Z/fyVPMKLhGRAjIrdRsONds8OzRVLaGtaxuIiEihKLhERKRQFFwiIlIoCi4RESkUBZeIiBSKgktERApFwSUiIoWi4BIRkUJRcImISKEouEREpFAUXCIiUiijElxmdrCZ3Z9sT5vZk5mf64d57UIzu3IEn3H3KLX1LDP76Wi8l4iIjL1RWWTX3TcC8wHM7NNAl7t/MX3czMa7+4CL8Lt7O9A+gs940Wi0VUREiq1sXYVm9m0zu8bM7gE+b2anmdnvzew+M7vbzOYlz9tTAZnZp83sOjO7w8zWmNnFmffryjz/DjP7oZk9bGY3mMXVcMzs/OS+ZWZ25b5UVmb2RjNbYWYrzezy5L5xyXGsTB77UHL/xWa2ysyWm9n3Ru0/moiIDKvclzU5HHiRu+8ys4OAF7t7r5mdC3wOeM0ArzkWWAxMAx4xs6vdfWe/55wMHA/8BbgLOMPM2oGvAWe6+1ozu3GkjTSzOcDlwCnAJuA2M7sAeAI4zN1fkDyvKXnJJcBR7t6Tua//e14EXARwxBFHjLQpIiIyjHJPzviBu6cXzG4EfmBmK4GvEMEzkJ+5e4+7bwDWA7MHeM697v5nd98N3A+0EIG3xt3XJs8ZcXABpwJ3uHtH0qV5A3AmsAZoNbOvmtl5wJbk+cuBG8zszcBgXaDXuvtCd184c+bMfWiKiIgMpdzBtS1z+1+AJUn18jfAYJc+68nc3sXAVeFInnPA3H0TcBJwB/Be4BvJQ68ErgIWAEvNTBfkFBEZI2M5Hb4ReDK5/fYyvP8jRHXUkvz8+n147b3AS8xshpmNA94I/MbMZgB17n4z8AlggZnVAXPdfQnwceK4po7SMYiIyDDGslL4PHC9mX0C+Nlov7m7d5vZ/wF+YWbbgKVDPP0cM/tz5ufXEuNWSwAjuit/bGYnAd9KwgrgUmAc8B0za0yee6W7d4728YiIyMDM3fNuw6gxs6nu3pXMMrwKeMzdv5J3uxYuXOjt7cPO+BcRkQwzW+buC/vfX20rZ7zHzO4HHiS68L6Wc3tERGSUVdWkgqS6yr3CEhGR8qm2iktERKqcgktERAqlqiZnVCoz6wD+tJ8vnwFsGMXmFIWOu7bU6nFD7R77SI77SHffawUHBVeFM7P2gWbVVDsdd22p1eOG2j32AzludRWKiEihKLhERKRQFFyV79q8G5ATHXdtqdXjhto99v0+bo1xiYhIoajiEhGRQlFwiYhIoSi4KpiZnWdmj5jZH83skrzbUy5mdp2ZrU8uMpreN93MfmlmjyX75jzbWA5mNtfMlpjZKjN70Mw+kNxf1cduZg1mdq+ZPZAc92eS+48ys3uS3/fvm1l93m0tBzMbZ2b3mdlPk5+r/rjNbJ2ZrTCz+5Or1R/Q77mCq0Il1wW7CngFcBzwRjM7Lt9Wlc23gfP63XcJ8Ct3fx7wq+TnatMLfNjdjwMWAe9L/o2r/dh7gLPd/SRgPnCemS0CLge+4u5HA5uAd+XYxnL6APBQ5udaOe7F7j4/c+7Wfv+eK7gq12nAH919jbvvAL4H/G3ObSoLd78TeLbf3X8LXJ/cvh64YEwbNQbc/Sl3/0NyeyvxZXYYVX7sHrqSHyckmwNnAz9M7q+64wYws8OJK6h/I/nZqIHjHsR+/54ruCrXYcATmZ//nNxXK2a7+1PJ7aeB2Xk2ptySK3efDNxDDRx70l12P7Ae+CWwGuh0997kKdX6+34F8DFgd/LzwdTGcTtwm5ktM7OLkvv2+/e8qi5rItXJ3d3Mqva8DTObCtwMfNDdt8Qf4aFaj93ddwHzzawJuBU4NucmlZ2Z/TWw3t2XmdlZebdnjP2Vuz9pZrOAX5rZw9kH9/X3XBVX5XoSmJv5+fDkvlrxjJkdCpDs1+fcnrIwswlEaN3g7rckd9fEsQO4eyewBDgdaDKz9I/pavx9PwN4lZmtI7r+zwb+neo/btz9yWS/nvhD5TQO4PdcwVW5lgLPS2Yc1QNvAH6Sc5vG0k+AtyW33wb8OMe2lEUyvvFN4CF3/3Lmoao+djObmVRamNkk4KXE+N4S4MLkaVV33O5+qbsf7u4txP/Pv3b3N1Hlx21mU8xsWnobeBmwkgP4PdfKGRXMzM4n+sTHAde5+2dzblJZmNmNwFnEZQ6eAT4F/Ai4CTiCuCTM69y9/wSOQjOzvwJ+C6ygNObxf4lxrqo9djM7kRiMH0f88XyTu/+zmbUSlch04D7gze7ek19LyyfpKvyIu/91tR93cny3Jj+OB77r7p81s4PZz99zBZeIiBSKugpFRKRQFFwiIlIoCi4RESkUBZeIiBSKgktERApFwSUiIoWi4BIRkUL5/xgokUP/I/JbAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y5IKmk7D49_n"
      },
      "source": [
        "## Task 2: Optimize the model\n",
        "\n",
        "Experiment with the following:\n",
        "\n",
        "* number of hidden layers \n",
        "* number of nodes in each layer\n",
        "* dropout regularization rate\n",
        "\n",
        "What trends did you discover?  Can you reach at least 98% accuracy against the test set? \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def create_model2(my_learning_rate):\n",
        "  \"\"\"Create and compile a deep neural net.\"\"\"\n",
        "  \n",
        "  # All models in this course are sequential.\n",
        "  model = tf.keras.models.Sequential([\n",
        "      tf.keras.layers.Flatten(input_shape = (28,28)),\n",
        "      tf.keras.layers.Dense(256, activation = \"relu\"),\n",
        "      tf.keras.layers.Dropout(rate = 0.2),\n",
        "      tf.keras.layers.Dense(128, activation = \"relu\"),\n",
        "      tf.keras.layers.Dense(units=10, activation='softmax')\n",
        "  ]) \n",
        "                           \n",
        "  model.compile(optimizer=tf.keras.optimizers.Adam(lr=my_learning_rate),\n",
        "                loss=\"sparse_categorical_crossentropy\",\n",
        "                metrics=['accuracy'])\n",
        "  \n",
        "  return model    \n",
        "\n",
        "\n",
        "def train_model2(model, train_features, train_label, epochs,\n",
        "                batch_size=None, validation_split=0.1):\n",
        "  \"\"\"Train the model by feeding it data.\"\"\"\n",
        "\n",
        "  history = model.fit(x=train_features, y=train_label, batch_size=batch_size,\n",
        "                      epochs=epochs, shuffle=True, \n",
        "                      validation_split=validation_split)\n",
        " \n",
        "  epochs = history.epoch\n",
        "  hist = pd.DataFrame(history.history)\n",
        "\n",
        "  return epochs, history, hist    "
      ],
      "metadata": {
        "id": "uNJOhtbMuwVs"
      },
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# The following variables are the hyperparameters.\n",
        "learning_rate = 0.003\n",
        "epochs = 50\n",
        "batch_size = 4000\n",
        "validation_split = 0.2\n",
        "\n",
        "# Establish the model's topography.\n",
        "my_model = create_model2(learning_rate)\n",
        "\n",
        "# Train the model on the normalized training set.\n",
        "epochs2, history2, hist2 = train_model2(my_model, x_train_normalized, y_train, \n",
        "                           epochs, batch_size, validation_split)\n",
        "\n",
        "# Plot a graph of the metric vs. epochs.\n",
        "list_of_metrics_to_plot = ['accuracy']\n",
        "plot_curve(epochs2, hist2, list_of_metrics_to_plot)\n",
        "\n",
        "# Evaluate against the test set.\n",
        "print(\"\\n Evaluate the new model against the test set:\")\n",
        "my_model.evaluate(x=x_test_normalized, y=y_test, batch_size=batch_size)"
      ],
      "metadata": {
        "id": "EwhzIPwnu0a0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "9a7d0911-30b0-4dca-fce6-ab91e11565e9"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/keras/optimizers/optimizer_v2/adam.py:110: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.\n",
            "  super(Adam, self).__init__(name, **kwargs)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "12/12 [==============================] - 2s 131ms/step - loss: 0.9945 - accuracy: 0.7100 - val_loss: 0.3525 - val_accuracy: 0.8946\n",
            "Epoch 2/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.3698 - accuracy: 0.8934 - val_loss: 0.2480 - val_accuracy: 0.9290\n",
            "Epoch 3/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.2559 - accuracy: 0.9247 - val_loss: 0.1916 - val_accuracy: 0.9448\n",
            "Epoch 4/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.2009 - accuracy: 0.9416 - val_loss: 0.1599 - val_accuracy: 0.9551\n",
            "Epoch 5/50\n",
            "12/12 [==============================] - 1s 117ms/step - loss: 0.1665 - accuracy: 0.9503 - val_loss: 0.1402 - val_accuracy: 0.9597\n",
            "Epoch 6/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.1409 - accuracy: 0.9584 - val_loss: 0.1276 - val_accuracy: 0.9617\n",
            "Epoch 7/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.1230 - accuracy: 0.9637 - val_loss: 0.1178 - val_accuracy: 0.9652\n",
            "Epoch 8/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.1060 - accuracy: 0.9689 - val_loss: 0.1077 - val_accuracy: 0.9678\n",
            "Epoch 9/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0933 - accuracy: 0.9719 - val_loss: 0.1012 - val_accuracy: 0.9701\n",
            "Epoch 10/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0831 - accuracy: 0.9752 - val_loss: 0.0971 - val_accuracy: 0.9706\n",
            "Epoch 11/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0756 - accuracy: 0.9774 - val_loss: 0.0923 - val_accuracy: 0.9722\n",
            "Epoch 12/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0687 - accuracy: 0.9794 - val_loss: 0.0902 - val_accuracy: 0.9725\n",
            "Epoch 13/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0627 - accuracy: 0.9810 - val_loss: 0.0883 - val_accuracy: 0.9727\n",
            "Epoch 14/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0571 - accuracy: 0.9828 - val_loss: 0.0863 - val_accuracy: 0.9746\n",
            "Epoch 15/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0524 - accuracy: 0.9847 - val_loss: 0.0834 - val_accuracy: 0.9747\n",
            "Epoch 16/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0470 - accuracy: 0.9863 - val_loss: 0.0826 - val_accuracy: 0.9762\n",
            "Epoch 17/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0423 - accuracy: 0.9876 - val_loss: 0.0817 - val_accuracy: 0.9763\n",
            "Epoch 18/50\n",
            "12/12 [==============================] - 2s 168ms/step - loss: 0.0396 - accuracy: 0.9884 - val_loss: 0.0790 - val_accuracy: 0.9772\n",
            "Epoch 19/50\n",
            "12/12 [==============================] - 2s 164ms/step - loss: 0.0388 - accuracy: 0.9877 - val_loss: 0.0787 - val_accuracy: 0.9761\n",
            "Epoch 20/50\n",
            "12/12 [==============================] - 1s 118ms/step - loss: 0.0360 - accuracy: 0.9891 - val_loss: 0.0798 - val_accuracy: 0.9778\n",
            "Epoch 21/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0330 - accuracy: 0.9900 - val_loss: 0.0774 - val_accuracy: 0.9774\n",
            "Epoch 22/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0307 - accuracy: 0.9912 - val_loss: 0.0788 - val_accuracy: 0.9786\n",
            "Epoch 23/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0274 - accuracy: 0.9917 - val_loss: 0.0768 - val_accuracy: 0.9787\n",
            "Epoch 24/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0256 - accuracy: 0.9925 - val_loss: 0.0764 - val_accuracy: 0.9797\n",
            "Epoch 25/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0239 - accuracy: 0.9929 - val_loss: 0.0775 - val_accuracy: 0.9787\n",
            "Epoch 26/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0227 - accuracy: 0.9933 - val_loss: 0.0760 - val_accuracy: 0.9787\n",
            "Epoch 27/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0219 - accuracy: 0.9933 - val_loss: 0.0791 - val_accuracy: 0.9788\n",
            "Epoch 28/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0208 - accuracy: 0.9942 - val_loss: 0.0782 - val_accuracy: 0.9788\n",
            "Epoch 29/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0195 - accuracy: 0.9942 - val_loss: 0.0775 - val_accuracy: 0.9791\n",
            "Epoch 30/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0175 - accuracy: 0.9948 - val_loss: 0.0783 - val_accuracy: 0.9787\n",
            "Epoch 31/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0170 - accuracy: 0.9952 - val_loss: 0.0757 - val_accuracy: 0.9797\n",
            "Epoch 32/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0149 - accuracy: 0.9959 - val_loss: 0.0776 - val_accuracy: 0.9794\n",
            "Epoch 33/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0142 - accuracy: 0.9959 - val_loss: 0.0811 - val_accuracy: 0.9787\n",
            "Epoch 34/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0150 - accuracy: 0.9954 - val_loss: 0.0770 - val_accuracy: 0.9803\n",
            "Epoch 35/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0144 - accuracy: 0.9957 - val_loss: 0.0778 - val_accuracy: 0.9795\n",
            "Epoch 36/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0136 - accuracy: 0.9961 - val_loss: 0.0815 - val_accuracy: 0.9788\n",
            "Epoch 37/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0122 - accuracy: 0.9964 - val_loss: 0.0780 - val_accuracy: 0.9800\n",
            "Epoch 38/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0125 - accuracy: 0.9965 - val_loss: 0.0829 - val_accuracy: 0.9792\n",
            "Epoch 39/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0113 - accuracy: 0.9968 - val_loss: 0.0808 - val_accuracy: 0.9801\n",
            "Epoch 40/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0102 - accuracy: 0.9970 - val_loss: 0.0805 - val_accuracy: 0.9804\n",
            "Epoch 41/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0115 - accuracy: 0.9966 - val_loss: 0.0830 - val_accuracy: 0.9814\n",
            "Epoch 42/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0098 - accuracy: 0.9972 - val_loss: 0.0816 - val_accuracy: 0.9799\n",
            "Epoch 43/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0099 - accuracy: 0.9970 - val_loss: 0.0818 - val_accuracy: 0.9803\n",
            "Epoch 44/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0089 - accuracy: 0.9973 - val_loss: 0.0829 - val_accuracy: 0.9803\n",
            "Epoch 45/50\n",
            "12/12 [==============================] - 1s 116ms/step - loss: 0.0096 - accuracy: 0.9969 - val_loss: 0.0837 - val_accuracy: 0.9803\n",
            "Epoch 46/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0079 - accuracy: 0.9978 - val_loss: 0.0819 - val_accuracy: 0.9804\n",
            "Epoch 47/50\n",
            "12/12 [==============================] - 1s 115ms/step - loss: 0.0079 - accuracy: 0.9980 - val_loss: 0.0850 - val_accuracy: 0.9795\n",
            "Epoch 48/50\n",
            "12/12 [==============================] - 1s 114ms/step - loss: 0.0073 - accuracy: 0.9980 - val_loss: 0.0850 - val_accuracy: 0.9799\n",
            "Epoch 49/50\n",
            "12/12 [==============================] - 2s 157ms/step - loss: 0.0076 - accuracy: 0.9977 - val_loss: 0.0867 - val_accuracy: 0.9799\n",
            "Epoch 50/50\n",
            "12/12 [==============================] - 1s 113ms/step - loss: 0.0077 - accuracy: 0.9979 - val_loss: 0.0851 - val_accuracy: 0.9797\n",
            "\n",
            " Evaluate the new model against the test set:\n",
            "3/3 [==============================] - 0s 32ms/step - loss: 0.0733 - accuracy: 0.9814\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.0733022391796112, 0.9814000129699707]"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxddZ3/8dcne9LsabqmG3SjQCkQyqa0oDAwLIUiCoNYQOkwWn/MT/kpMCoO4A919DfIiKN1LIjMWBEpMrLZIhUcQJoC3ehKaUnSJk2bfblZP78/7mm4pGmbltzeJPf9fDzy6LnnnHvzOeFy3/f7/Z7zPebuiIiI9JQQ6wJERGRgUkCIiEivFBAiItIrBYSIiPRKASEiIr1KinUB/WX48OE+ceLEWJchIjKorF69eq+7F/a2bcgExMSJEykpKYl1GSIig4qZ7TzYNnUxiYhIr6IWEGa2xMz2mNn6g2w3M3vQzLaZ2VozOy1i2wIz2xr8LIhWjSIicnDRbEE8Alx8iO2XAFOCn4XAvwOYWT5wN3AmMBu428zyoliniIj0ImpjEO7+splNPMQu84BHPTzXx+tmlmtmo4G5wHJ3rwYws+WEg+bXR1pDe3s7ZWVlhEKhI32qBNLS0igqKiI5OTnWpYjIMRbLQeqxQGnE47Jg3cHWH8DMFhJufTB+/PgDtpeVlZGVlcXEiRMxs34qO364O/v27aOsrIxJkybFuhwROcYG9SC1uy9292J3Ly4sPPAsrVAoREFBgcLhKJkZBQUFaoGJxKlYBkQ5MC7icVGw7mDrj4rC4aPR308kfsWyi+lpYJGZLSU8IF3n7rvN7AXg/0YMTF8E3BmrIkVEjoa7s6suxL7GVhpDHdSHOmhs7aAx1E5DqIP2zi7MDDNIMMOAhAQjKcHIH5bC8KxUhg9LpSAzhYLMFFKTEnF3Gls7qGlqp6a5jZrmNmqb28lISeSiE0f1+zFELSDM7NeEB5yHm1kZ4TOTkgHc/afAs8DfAtuAZuCmYFu1md0LrApe6p79A9YiIv3F3Q/bQnZ3Glo7qGpopaqhlaQEoygvgxFZqSQkHPjcPfUhXn13H/+zbS+vvruP8tqWfqs3MzWJUHsnHV0H3sNnZlHO4AoId7/uMNsd+NJBti0BlkSjrqGqo6ODpKQhc2G8SLeGUDuV9SEq6lrZ29hKY2sHTft/2jppau2gpb2TvIwUxuSmMSonnTE5aYzOTWdEViqh9k627mlka2UDWyob2VLZwNbKRiobQqQnJzIsNYlhKcG/qUmkJydSH2rvDoXWjq4DakpONMbkplOUl05RbgbJScbr26vZtqcRgJz0ZM4+roCF5x3H2Nx0MtOSyExNIjstuXs5JSkBd8cdHOgKlts6u6hubGNvUyt7G1rZ19TG3oZWqpvbSEtOJC8jmdyMFPIyUsgfFl4uGJYSlb+9PlGOgSuvvJLS0lJCoRC33XYbCxcu5Pnnn+euu+6is7OT4cOH8+KLL9LY2MiXv/xlSkpKMDPuvvturr76ajIzM2lsDL/xnnjiCf7whz/wyCOPcOONN5KWlsZbb73Fueeey7XXXsttt91GKBQiPT2dhx9+mGnTptHZ2cnXv/51nn/+eRISErjllls48cQTefDBB3nqqacAWL58OT/5yU9YtmxZLP9UMkR1djm1zW3sbWxjX2Mre5vaqG5spamtk9b2TlraOwm1dwX/dlLd1EZFfYjKuhBNbZ0Hfd1hKYlkpCaRlpxAdWPbAfsmGER+4U5LTmDyiEzOOb6AMbnphNo7u0OmqTXcBVTT3EZOejKTCoZRmJXK8MzU7n87urooq2kJfpopr23hT5v30NzawekT87nm9CLOOX44M8Zkk9hLC6On/V1MAImEF1KSEshMTWJ8QcaR/6H7WdwExD//9wbe2VXfr685Y0w2d19+4mH3W7JkCfn5+bS0tHDGGWcwb948brnlFl5++WUmTZpEdXW4B+3ee+8lJyeHdevWAVBTU3PY1y4rK+PVV18lMTGR+vp6XnnlFZKSklixYgV33XUXv/vd71i8eDE7duzg7bffJikpierqavLy8vjiF79IVVUVhYWFPPzww9x8880f7Q8ig0JTawcbd9ezvryOmuZ2utzp7Ir4cSc1KZHjC4cxeUQmk0dkkpXW+3UwnV3O3sZWdtW2UFkfYk9DK3vqWz9YbmilqiFEdVMbvfSMAGAGaUmJpKckkpaUQFpyIrkZyUwflcWcqYWMyk5jVE4aI7PTKMxKJSs1iYzUJDKSEz/UzbO/O6iiLsSu2pbwv3UhUpMSmDoyi6kjMynKy+jTB7eExU1AxNKDDz7Y/c28tLSUxYsXc95553VfW5Cfnw/AihUrWLp0affz8vIOfwH5NddcQ2JiIgB1dXUsWLCArVu3Yma0t7d3v+6tt97a3QW1//fdcMMNPPbYY9x000289tprPProo/10xDIQdHR2saehlferm9mwKxwI68rreLeqkchb0ScYJCYYCWYkJhiJZoQ6Omnv/GCnUdlpTB6RyYSCDOpa2qmoC7G7LkRlfeiAPvHEBGN4Zgojs9MYm5vGrHE5DM8MfwMvyEyhYFgqhVkp5A9LJSMlkdSkhH45W87MyE5LJjstmakjsz7y60kcBURfvulHw8qVK1mxYgWvvfYaGRkZzJ07l1mzZrFp06Y+v0bk/zw9r0kYNmxY9/I3v/lNzj//fJYtW8aOHTuYO3fuIV/3pptu4vLLLyctLY1rrrlGYxiDUGtHJ9v2NLJpdwPvVjWyq7aFXbUhymtbqKgP0Rnx4T0iK5WTx+Zw6cmjOXlsDicX5TAiK7XXD+eOzi5Ka1rYWtnAtqpGtu1p5N09jTyzbje56cmMzknnzEn5jI7o8x+ZncaI7FQKhqXqW/oQoU+EKKurqyMvL4+MjAw2bdrE66+/TigU4uWXX+a9997r7mLKz8/nwgsv5KGHHuKBBx4Awl1MeXl5jBw5ko0bNzJt2jSWLVtGVlbv347q6uoYOzZ80fkjjzzSvf7CCy/kZz/7Geeff353F1N+fj5jxoxhzJgx3HfffaxYsSLqfwvpm5qmNtaW19Ha3hl0yzhdHh7E7HIoq2lm0+4GNlXU825VU3cIJCUYo3LSGJObzuxJ+YzNTWdMbjpj89I5YVQWI7LT+lxDUmICk4YPY9LwYVwUncOUQUABEWUXX3wxP/3pTznhhBOYNm0aZ511FoWFhSxevJj58+fT1dXFiBEjWL58Od/4xjf40pe+xEknnURiYiJ333038+fP57vf/S6XXXYZhYWFFBcXdw9Y9/S1r32NBQsWcN9993HppZd2r//CF77Ali1bmDlzJsnJydxyyy0sWrQIgOuvv56qqipOOOGEY/L3kAPVh9p5Y3s1r23fx2vv7mNjRf2HuoB6MzY3nemjsrhwxkhOGJ3N9FHZTCzIIClxUE+OIAOM+eHeiYNEcXGx97xh0MaNG/XBdxiLFi3i1FNP5fOf//xB99HfsW/cndrmdqoaW7tPkdwbLDe3ddLR1UV7p9PR2UVHl9PR6eyqa2F9eR1dDqlJCZw+IY+zjyugeGI+WWlJH1xEFXEx1YjsNHLSNXmi9A8zW+3uxb1tUwsijp1++ukMGzaMH/7wh7EuZdBo7+zivb1N7NzXzM59Tbxf3czOfc28X91MWU3zhwZ299t/2mJSgpGcmEBigpGUaCQnJJA3LJlFF0zh7OMKOHV8LmnJiTE4KpHeKSDi2OrVq2NdwoBX19zOm+/XULKzmpIdNawpqyXU/sGFU1nB+eonjM7iohNHMjIrfCpm5E9WapLmtJJBacgHRF8up5eDGypdkIeyv2toZ3W4VVBa3cyOfc2sKa1la3BlbGKCceKYbK6bPZ5TinKZUJDBhIJh5GUk6/0lQ9aQDoi0tDT27dunKb+P0v77QaSl9f3sl8GirKaZp94qZ/k7lWzf20RDqOND20dkpTJjTDbzZo3htAl5zBqXS0bKkP7fReQAQ/odX1RURFlZGVVVVbEuZdDaf0e5oaAh1M5z6yt48s0yXt8evnr99Al5XDlrLBMKMhifH24VjM/PID1FYwEiQzogkpOTdSe0ONfY2sFftu7l2XW7+eM7FYTau5hYkMFXLpzKVaeOZVx+7Oe7ERmohnRASPxxd7ZUNrJy8x5Wbq6iZGc17Z1OTnoynzq9iPmnFXHquFx1OYr0gQJCBj13Z315PY+XlPLixkp21YWnI5k+KoubPzaJuVNHcPqEPFKSdBGZyJFQQMiAU93URsmOahpCHZw4NpvJhZm9XiFc19zO79eUs/SNUt7ZXU9acgJzphbyvz4xhTnTChmdkx6D6kWGDgWExJS7U1bTwhvvVVOys5o33qvm3aqmD+2TlpzACaOzOXlsDieNzaEwM5Wn1+zi2XW7ae3o4sQx2dx75UlcccoYXWEs0o8UEBITu2pb+G1JGU+8WUppdfi2jFlpSRRPyOPq04s4Y2I+uenJbNhVz7pgmuon3yzn0dd2hvdNTeKa4iKuPWM8J43NieWhiAxZCgg5Zto7u3hx4x5+s+p9/rylii6Hj08ZzsKPH8cZk/KZOiLrgPv8ThmZxZWnhmeo7epyduxroqymhTMm5utUVJEoU0BI1JXXtvDoazv43eoy9ja2MSo7jUXnT+aa4nFHdJppQoJxXGEmxxVmRq9YEemmgJCo2bangX9fuZ3fv12OA5+YPoJrZ4/jvCmFmpZaZBBQQEi/W1Nay09WbuOP71SSmpTAZ8+awC3nHcfYXJ1VJDKYKCCk36zaUc0DK7bwP9v2kZ2WxKLzJ3PjORMpyEyNdWkichQUEPKRldU0c/9zm3hm7W4Ks1K585Lp/N2Z48lK0ymnIoOZAkKOWnNbBz/983Z+9ud3MYPbPjGFW+ccr7OLRIYIBYQcMXfn6TW7+O5zm9hdF+LyU8ZwxyXTNcYgMsQoIOSIlOyo5v7nNrF6Zw0njsnmR9eeyuxJ+bEuS0SiQAEhfbJtTwPfe34zy9+ppDArle/OP5lriseRmKBZUUWGKgWEHFJFXYgHVmzh8ZJSMlKSuP2iqdz8sUm6u5pIHIjq/+VmdjHwIyAR+A93/26P7ROAJUAhUA181t3Lgm3fBy4FEoDlwG0eDzdIHiBqmtpY/Mp2lvzlPbrcWXDORBadP1mnrIrEkagFhJklAg8BFwJlwCoze9rd34nY7QfAo+7+SzO7ALgfuMHMzgHOBWYG+/0FmAOsjFa9ElZe28J/vLKdpW+U0tLeyZWzxvDVi6bpzmsicSiaLYjZwDZ33w5gZkuBeUBkQMwAvhIsvwQ8FSw7kAakAAYkA5VRrDXuba5o4Gd/fpen1+wC4IpZY/j7845n2qisGFcmIrESzYAYC5RGPC4DzuyxzxpgPuFuqKuALDMrcPfXzOwlYDfhgPixu2/s+QvMbCGwEGD8+PH9fwRxYG1ZLT9asZUXN+0hPTmRG86ewBc+rmkxRCT2g9S3Az82sxuBl4FyoNPMJgMnAEXBfsvN7OPu/krkk919MbAYoLi4WOMTR6CmqY3vv7CZpaveJzc9mf/9yal87uwJ5A1LiXVpIjJARDMgyoFxEY+LgnXd3H0X4RYEZpYJXO3utWZ2C/C6uzcG254DzgY+FBBy5Lq6nN+UlPK95zfREOrg8+dO4rZPTtG0GCJygGgGxCpgiplNIhwM1wJ/F7mDmQ0Hqt29C7iT8BlNAO8Dt5jZ/YS7mOYAD0Sx1riwtqyWb/5+A2tKa5k9KZ97552kMQYROaioBYS7d5jZIuAFwqe5LnH3DWZ2D1Di7k8Dc4H7zcwJdzF9KXj6E8AFwDrCA9bPu/t/R6vWoa4+1M73n9/Ef/71fQqGpfLAZ2Yxb9YYzHSRm4gcnA2VSwuKi4u9pKQk1mUMOMvfqeQbT62jqqGVz509ka9cNJVsdSeJSMDMVrt7cW/bYj1ILVFS1dDKt/97A8+s3c30UVksvqGYU8blxrosERlEFBBDjLvz5Jvl3PvMOzS3dvLVC6fy93OOJyVJt/gUkSOjgBhCqhpa+epv1/DylipOn5DH964+mckjNAgtIkdHATFEVDW0ct3PX6esppl/vuJEbjhrAgmaaVVEPgIFxBCwPxzKa1p45KbZnHVcQaxLEpEhQB3Tg1xkODx80xkKBxHpNwqIQUzhICLRpIAYpBQOIhJtCohBSOEgIseCBqkHmff3NbPg4TeoqAspHEQkqhQQg8j68jpufHgV7Z1d/OrzsymemB/rkkRkCFNADBKvbK3i1l+tJjcjhaULz9QFcCISdQqIQWDZW2X8n9+uZfKITH5582xGZqfFuiQRiQMKiAHM3Vn88nbuf24TZx9XwM8+d7pmYhWRY0YBMUC5O/c9s5Ff/OU9Lps5mh9++hRSkxJjXZaIxBEFxAD1h7W7+cVf3mPB2RO4+/ITNa+SiBxzug5iAKppauPbT29gZlEO37xshsJBRGJCLYgB6N5n3qGupZ3HvnAmSYnKcBGJDX36DDB/3lLFk2+Wc+uc4zlhdHasyxGROKaAGECaWju468l1HFc4jEUXTI51OSIS59TFNID88I9bKK9t4be3nk1ass5YEpHYUgtigHjr/RoefvU9bjhrAmdoCg0RGQAUEANAW0cXd/xuHaOy0/jaxdNiXY6ICKAupgHh31e+y+bKBpbcWEyWrpQWkQFCLYgY21rZwI9f2soVp4zhgukjY12OiEg3BUQMdXU5dz65jszUJL51+YxYlyMi8iEKiBj6rzfep2RnDd+4dAbDM1NjXY6IyIcoIGKksj7E957bxLmTC5h/2thYlyMicgAFRIx8++kNtHV28Z0rT8ZMcy2JyMAT1YAws4vNbLOZbTOzO3rZPsHMXjSztWa20syKIraNN7M/mtlGM3vHzCZGs9Zj6Y8bKnhufQX/+MmpTBw+LNbliIj0KmoBYWaJwEPAJcAM4Doz6zkS+wPgUXefCdwD3B+x7VHgX9z9BGA2sCdatR5LDaF2vvX7DUwflcUXPj4p1uWIiBxUNFsQs4Ft7r7d3duApcC8HvvMAP4ULL+0f3sQJEnuvhzA3RvdvTmKtR4zP3hhM5UNIb579UySNVOriAxg0fyEGguURjwuC9ZFWgPMD5avArLMrACYCtSa2ZNm9paZ/UvQIvkQM1toZiVmVlJVVRWFQ+hfq3fW8OjrO1lw9kRmjcuNdTkiIocU66+wtwNzzOwtYA5QDnQSvsL748H2M4DjgBt7PtndF7t7sbsXFxYWHrOij0ZbRxd3PbmO0dlp3P43mk5DRAa+aAZEOTAu4nFRsK6bu+9y9/nufirwT8G6WsKtjbeD7qkO4CngtCjWGnU/f2U7mysbuGfeSWSmaoYTERn4ohkQq4ApZjbJzFKAa4GnI3cws+Fmtr+GO4ElEc/NNbP9zYILgHeiWGtUhdo7+cVf3uMT00fwyRmaTkNEBoeoBUTwzX8R8AKwEXjc3TeY2T1mdkWw21xgs5ltAUYC3wme20m4e+lFM1sHGPDzaNUabc+t3011Uxs3f0xnLYnI4BHVvg53fxZ4tse6b0UsPwE8cZDnLgdmRrO+Y+VXr+3kuMJhnHN8QaxLERHps1gPUg95G3bV8eb7tVx/5gRdMS0ig4oCIsoee/190pIT+NRpRYffWURkAFFARFF9qJ2n3ipn3iljycnQjYBEZHBRQETRsjfLaWnv5LNnTYh1KSIiR0wBESXuzq9e38kp43I5uSgn1uWIiBwxBUSUvL69mm17GrlBrQcRGaQUEFHy2F93kpOezGUzR8e6FBGRo6KAiII99SFeWF/Bp4uLSEs+YI5BEZFBQQERBUtXldLR5Vx/prqXRGTwUkD0s47OLn79xvt8fMpw3S1ORAa1PgeEmWVEs5Ch4sVNe9hdF9LgtIgMeocNCDM7x8zeATYFj08xs59EvbJB6rHXdzImJ40Lpo+IdSkiIh9JX1oQ/wr8DbAPwN3XAOdFs6jBaue+Jl7ZupfrZo8nSbcTFZFBrk+fYu5e2mNVZxRqGfQeLyklweCa4nGH31lEZIDry3TfpWZ2DuBmlgzcRvj+DhKho7OLJ1aXMXfaCEblpMW6HBGRj6wvLYhbgS8BYwnfMnRW8FgivLJ1L5X1rXy6WLO2isjQcNgWhLvvBa4/BrUMar9ZVUrBsBQumK5biorI0HDYgDCzhwHvud7db45KRYPQ3sZWVmys5KZzJ5KSpMFpERka+jIG8YeI5TTgKmBXdMoZnJa9WU5Hl/OZMzQ4LSJDR1+6mH4X+djMfg38JWoVDTLuzm9KSjltfC6TR2TFuhwRkX5zNP0hUwBdBRZ48/1atu1pVOtBRIacvoxBNBAeg7Dg3wrg61Gua9B4fFUpGSmJXDpzTKxLERHpV33pYlK/yUE0tXbwh7W7uPTk0WSm9mU4R0Rk8Djop5qZnXaoJ7r7m/1fzuDyzNrdNLV1qntJRIakQ33t/eEhtjlwQT/XMuj8pqSU4wqHcfqEvFiXIiLS7w4aEO5+/rEsZLDZtqeB1TtruPOS6ZhZrMsREel3feo4N7OTgBmEr4MAwN0fjVZRg8HjJWUkJhjzT9PUGiIyNPXlLKa7gbmEA+JZ4BLC10HEbUC0d3bx5JtlXDB9BIVZqbEuR0QkKvpyHcSngE8AFe5+E3AKkBPVqga4lZur2NvYxmc0rbeIDGF9CYiQu3cBHWaWDewB+vTJaGYXm9lmM9tmZnf0sn2Cmb1oZmvNbKWZFfXYnm1mZWb24778vmNl1Y5qUhITOG9qYaxLERGJmoMGhJk9ZGYfA94ws1zg58Bq4E3gtcO9sJklAg8R7pKaAVxnZjN67PYD4FF3nwncA9zfY/u9wMt9PJZjZl1ZHdNHZ2liPhEZ0g41BrEF+BdgDNAE/Bq4EMh297V9eO3ZwDZ33w5gZkuBecA7EfvMAL4SLL8EPLV/g5mdDowEngeK+3Iwx4K7s35XHZefoiunRWRoO+hXYHf/kbufTfj+0/uAJYQ/rK8ysyl9eO2xQOStSsuCdZHWAPOD5auALDMrMLMEwtdh3H6oX2BmC82sxMxKqqqq+lDSR7dzXzMNoQ5OHhvXwzAiEgcO20fi7jvd/XvufipwHXAlsKmffv/twBwzewuYQ/iOdZ3AF4Fn3b3sMLUtdvdidy8uLDw24wHryusAFBAiMuT15TTXJMLjCNcSPptpJfDtPrx2OR8ezC4K1nVz910ELQgzywSudvdaMzsb+LiZfRHIBFLMrNHdDxjoPtbWl9eRkpjA1JGaokpEhrZDzcV0IeEWw98CbwBLgYXu3tTH114FTDGzSYSD4Vrg73r8juFAdXCW1J2Eu7Fw9+sj9rkRKB4I4QDhFsS0URqgFpGh71CfcncCrwInuPsV7v5fRxAOuHsHsAh4AdgIPO7uG8zsHjO7IthtLrDZzLYQHpD+ztEcxLHi7qwvr+MkdS+JSBw41FxMH3kyPnd/lvDV15HrvhWx/ATwxGFe4xHgkY9aS394v7qZeg1Qi0icUD/JEdAAtYjEEwXEEVi3f4B6VGasSxERiToFxBFYHwxQpyYlxroUEZGoU0D0UXiAul4D1CISNxQQfVRa3UJdS7vGH0Qkbigg+kgD1CISbxQQfbSuvI7kRNMAtYjEDQVEH2mAWkTijQKiD9yddeV16l4SkbiigOiDsprwALXOYBKReKKA6AMNUItIPFJA9MH+AeppozTFt4jEDwVEH6wvr2PqSA1Qi0h8UUAchruztkwD1CISfxQQh6EBahGJVwqIw9AAtYjEKwXEYawrryMpQQPUIhJ/FBCHsX+AOi1ZA9QiEl8UEIegK6hFJJ4pIA6hrKaF2uZ2TipSQIhI/FFAHMJ6DVCLSBxTQBzC/gHq6RqgFpE4pIA4hA276pk8IlMD1CISlxQQh1Be28LEgmGxLkNEJCYUEIdQWRdiVE5arMsQEYkJBcRBNLZ20NDaoYAQkbilgDiIiroQAKOyFRAiEp8UEAfRHRBqQYhInIpqQJjZxWa22cy2mdkdvWyfYGYvmtlaM1tpZkXB+llm9pqZbQi2fSaadfamol4tCBGJb1ELCDNLBB4CLgFmANeZ2Yweu/0AeNTdZwL3APcH65uBz7n7icDFwANmlhutWntTWa8WhIjEt2i2IGYD29x9u7u3AUuBeT32mQH8KVh+af92d9/i7luD5V3AHqAwirUeYHddC7kZyboGQkTiVjQDYixQGvG4LFgXaQ0wP1i+Csgys4LIHcxsNpACvBulOntVUdeq7iURiWuxHqS+HZhjZm8Bc4ByoHP/RjMbDfwKuMndu3o+2cwWmlmJmZVUVVX1a2EV9S3qXhKRuBbNgCgHxkU8LgrWdXP3Xe4+391PBf4pWFcLYGbZwDPAP7n76739Andf7O7F7l5cWNi/PVBqQYhIvItmQKwCppjZJDNLAa4Fno7cwcyGm9n+Gu4ElgTrU4BlhAewn4hijb1q6+hiX1OrWhAiEteiFhDu3gEsAl4ANgKPu/sGM7vHzK4IdpsLbDazLcBI4DvB+k8D5wE3mtnbwc+saNXa056GEO46xVVE4ltSNF/c3Z8Fnu2x7lsRy08AB7QQ3P0x4LFo1nYo+09xHakWhIjEsVgPUg9Iu4OrqEcrIEQkjikgeqF5mEREFBC9qqwPkZqUQE56cqxLERGJGQVEL3bXhRidk4aZxboUEZGYUUD0orI+xEh1L4lInFNA9GJ/C0JEJJ4pIHpwd/bUt+oUVxGJewqIHqqb2mjr7NIZTCIS9xQQPegaCBGRMAVED91XUasFISJxTgHRwwctiPQYVyIiElsKiB4q60MkGAzPTIl1KSIiMaWA6KGiLkRhVipJifrTiEh806dgDxX1IUape0lERAHRU0VdiFHZqbEuQ0Qk5hQQPVTUhTRALSKCAuJDmlo7aGjt0CmuIiIoID6kIrgGYlSOuphERBQQET64UZC6mEREFBARugNC02yIiCggInV3MWkMQkREARGpoi5ETnoy6SmJsS5FRCTmFBARKupDaj2IiAQUEBEq6kIafxARCSggIqgFISLyAQVEoL2zi72NrWpBiIgEFBCBPQ2tuOsUVxGR/RQQgQ8uklNAiIiAAqKbLpITEfmwqAaEmV1sZpvNbJuZ3dHL9glm9qKZrTWzlWZWFGYxHu8AAAeBSURBVLFtgZltDX4WRLNO0EVyIiI9RS0gzCwReAi4BJgBXGdmM3rs9gPgUXefCdwD3B88Nx+4GzgTmA3cbWZ50aoVoKKuhdSkBHIzkqP5a0REBo1otiBmA9vcfbu7twFLgXk99pkB/ClYfili+98Ay9292t1rgOXAxVGslYr68BlMZhbNXyMiMmhEMyDGAqURj8uCdZHWAPOD5auALDMr6ONzMbOFZlZiZiVVVVUfqdjKupDuAyEiEiHWg9S3A3PM7C1gDlAOdPb1ye6+2N2L3b24sLDwIxWyu76F0RqgFhHplhTF1y4HxkU8LgrWdXP3XQQtCDPLBK5291ozKwfm9njuymgV6u5U1rVqgFpEJEI0WxCrgClmNsnMUoBrgacjdzCz4Wa2v4Y7gSXB8gvARWaWFwxOXxSsi4rqpjbaOrt0iquISISoBYS7dwCLCH+wbwQed/cNZnaPmV0R7DYX2GxmW4CRwHeC51YD9xIOmVXAPcG6qNApriIiB4pmFxPu/izwbI9134pYfgJ44iDPXcIHLYqoqgwCYqRaECIi3WI9SD0g7A6uotYgtYjIBxQQhE9xTTAozEyNdSkiIgOGAoJwC6IwK5WkRP05RET20yciulGQiEhvFBCEB6l1FbWIyIcpIAh3MWmAWkTkw+I+IJpaO2gIdegUVxGRHuI+IFo7urj8lDGcNCYn1qWIiAwoUb1QbjDIH5bCv113aqzLEBEZcOK+BSEiIr1TQIiISK8UECIi0isFhIiI9EoBISIivVJAiIhIrxQQIiLSKwWEiIj0ytw91jX0CzOrAnYeZrfhwN5jUM5AFc/HH8/HDvF9/Dr2Q5vg7oW9bRgyAdEXZlbi7sWxriNW4vn44/nYIb6PX8d+9MeuLiYREemVAkJERHoVbwGxONYFxFg8H388HzvE9/Hr2I9SXI1BiIhI38VbC0JERPpIASEiIr2Km4Aws4vNbLOZbTOzO2JdT7SZ2RIz22Nm6yPW5ZvZcjPbGvybF8sao8XMxpnZS2b2jpltMLPbgvVD/vjNLM3M3jCzNcGx/3OwfpKZ/TV4///GzFJiXWu0mFmimb1lZn8IHsfTse8ws3Vm9raZlQTrjvp9HxcBYWaJwEPAJcAM4DozmxHbqqLuEeDiHuvuAF509ynAi8HjoagD+Kq7zwDOAr4U/PeOh+NvBS5w91OAWcDFZnYW8D3gX919MlADfD6GNUbbbcDGiMfxdOwA57v7rIjrH476fR8XAQHMBra5+3Z3bwOWAvNiXFNUufvLQHWP1fOAXwbLvwSuPKZFHSPuvtvd3wyWGwh/WIwlDo7fwxqDh8nBjwMXAE8E64fksQOYWRFwKfAfwWMjTo79EI76fR8vATEWKI14XBasizcj3X13sFwBjIxlMceCmU0ETgX+Spwcf9DF8jawB1gOvAvUuntHsMtQfv8/AHwN6AoeFxA/xw7hLwN/NLPVZrYwWHfU7/uk/q5OBgd3dzMb0uc4m1km8DvgH929PvxlMmwoH7+7dwKzzCwXWAZMj3FJx4SZXQbscffVZjY31vXEyMfcvdzMRgDLzWxT5MYjfd/HSwuiHBgX8bgoWBdvKs1sNEDw754Y1xM1ZpZMOBz+092fDFbHzfEDuHst8BJwNpBrZvu/EA7V9/+5wBVmtoNwN/IFwI+Ij2MHwN3Lg3/3EP5yMJuP8L6Pl4BYBUwJzmZIAa4Fno5xTbHwNLAgWF4A/D6GtURN0O/8C2Cju/+/iE1D/vjNrDBoOWBm6cCFhMdgXgI+Few2JI/d3e909yJ3n0j4//E/ufv1xMGxA5jZMDPL2r8MXASs5yO87+PmSmoz+1vC/ZOJwBJ3/06MS4oqM/s1MJfwdL+VwN3AU8DjwHjCU6N/2t17DmQPemb2MeAVYB0f9EXfRXgcYkgfv5nNJDwQmUj4C+Dj7n6PmR1H+Ft1PvAW8Fl3b41dpdEVdDHd7u6XxcuxB8e5LHiYBPyXu3/HzAo4yvd93ASEiIgcmXjpYhIRkSOkgBARkV4pIEREpFcKCBER6ZUCQkREeqWAEDkCZtYZzJS5/6ffJvwzs4mRs++KxJqm2hA5Mi3uPivWRYgcC2pBiPSDYB7+7wdz8b9hZpOD9RPN7E9mttbMXjSz8cH6kWa2LLhvwxozOyd4qUQz+3lwL4c/BldDi8SEAkLkyKT36GL6TMS2Onc/Gfgx4av2Af4N+KW7zwT+E3gwWP8g8Ofgvg2nARuC9VOAh9z9RKAWuDrKxyNyULqSWuQImFmju2f2sn4H4Rv1bA8mCqxw9wIz2wuMdvf2YP1udx9uZlVAUeSUD8HU5MuDG7tgZl8Hkt39vugfmciB1IIQ6T9+kOUjETlHUCcaJ5QYUkCI9J/PRPz7WrD8KuGZRQGuJzyJIIRv/fgP0H2Dn5xjVaRIX+nbiciRSQ/u1rbf8+6+/1TXPDNbS7gVcF2w7svAw2b2f4Aq4KZg/W3AYjP7POGWwj8AuxEZQDQGIdIPgjGIYnffG+taRPqLuphERKRXakGIiEiv1IIQEZFeKSBERKRXCggREemVAkJERHqlgBARkV79f76mk7FBQoCgAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    }
  ]
}