Traffic_Sign_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Self-Driving Car Engineer Nanodegree\n",
    "\n",
    "## Deep Learning\n",
    "\n",
    "## Project: Build a Traffic Sign Recognition Classifier\n",
    "\n",
    "In this notebook, a template is provided for you to implement your functionality in stages, which is required to successfully complete this project. If additional code is required that cannot be included in the notebook, be sure that the Python code is successfully imported and included in your submission if necessary. \n",
    "\n",
    "> **Note**: Once you have completed all of the code implementations, you need to finalize your work by exporting the iPython Notebook as an HTML document. Before exporting the notebook to html, all of the code cells need to have been run so that reviewers can see the final implementation and output. You can then export the notebook by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission. \n",
    "\n",
    "In addition to implementing code, there is a writeup to complete. The writeup should be completed in a separate file, which can be either a markdown file or a pdf document. There is a [write up template](https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project/blob/master/writeup_template.md) that can be used to guide the writing process. Completing the code template and writeup template will cover all of the [rubric points](https://review.udacity.com/#!/rubrics/481/view) for this project.\n",
    "\n",
    "The [rubric](https://review.udacity.com/#!/rubrics/481/view) contains \"Stand Out Suggestions\" for enhancing the project beyond the minimum requirements. The stand out suggestions are optional. If you decide to pursue the \"stand out suggestions\", you can include the code in this Ipython notebook and also discuss the results in the writeup file.\n",
    "\n",
    "\n",
    ">**Note:** Code and Markdown cells can be executed using the **Shift + Enter** keyboard shortcut. In addition, Markdown cells can be edited by typically double-clicking the cell to enter edit mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Step 0: Load The Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import libraries\n",
    "\n",
    "import pickle\n",
    "import numpy as np\n",
    "import csv\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.utils import shuffle\n",
    "import cv2\n",
    "import tensorflow as tf\n",
    "from tensorflow.contrib.layers import flatten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Loaded Successfully.\n"
     ]
    }
   ],
   "source": [
    "# Load pickled data\n",
    "\n",
    "training_file = 'data/train.p'\n",
    "validation_file= 'data/valid.p'\n",
    "testing_file = 'data/test.p'\n",
    "\n",
    "with open(training_file, mode='rb') as f:\n",
    "    train = pickle.load(f)\n",
    "with open(validation_file, mode='rb') as f:\n",
    "    valid = pickle.load(f)\n",
    "with open(testing_file, mode='rb') as f:\n",
    "    test = pickle.load(f)\n",
    "    \n",
    "X_train, y_train = train['features'], train['labels']\n",
    "X_validation, y_validation = valid['features'], valid['labels']\n",
    "X_test, y_test = test['features'], test['labels']\n",
    "\n",
    "print(\"Data Loaded Successfully.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 1: Dataset Summary & Exploration\n",
    "\n",
    "The pickled data is a dictionary with 4 key/value pairs:\n",
    "\n",
    "- `'features'` is a 4D array containing raw pixel data of the traffic sign images, (num examples, width, height, channels).\n",
    "- `'labels'` is a 1D array containing the label/class id of the traffic sign. The file `signnames.csv` contains id -> name mappings for each id.\n",
    "- `'sizes'` is a list containing tuples, (width, height) representing the original width and height the image.\n",
    "- `'coords'` is a list containing tuples, (x1, y1, x2, y2) representing coordinates of a bounding box around the sign in the image. **THESE COORDINATES ASSUME THE ORIGINAL IMAGE. THE PICKLED DATA CONTAINS RESIZED VERSIONS (32 by 32) OF THESE IMAGES**\n",
    "\n",
    "Complete the basic data summary below. Use python, numpy and/or pandas methods to calculate the data summary rather than hard coding the results. For example, the [pandas shape method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.shape.html) might be useful for calculating some of the summary results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Provide a Basic Summary of the Data Set Using Python and Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples = 34799\n",
      "Number of validation examples =  4410\n",
      "Number of testing examples = 12630\n",
      "Image data shape = (32, 32, 3)\n",
      "Number of classes = 43\n"
     ]
    }
   ],
   "source": [
    "#Number of training examples\n",
    "n_train = len(X_train)\n",
    "\n",
    "#Number of validation examples\n",
    "n_validation = len(X_validation)\n",
    "\n",
    "#Number of testing examples.\n",
    "n_test = len(X_test)\n",
    "\n",
    "#What's the shape of an traffic sign image?\n",
    "image_shape = X_train[0].shape\n",
    "\n",
    "#How many unique classes/labels there are in the dataset.\n",
    "n_classes = len(np.unique(y_train))\n",
    "\n",
    "print(\"Number of training examples =\", n_train)\n",
    "print(\"Number of validation examples = \", n_validation)\n",
    "print(\"Number of testing examples =\", n_test)\n",
    "print(\"Image data shape =\", image_shape)\n",
    "print(\"Number of classes =\", n_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Include an exploratory visualization of the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the German Traffic Signs Dataset using the pickled file(s). This is open ended, suggestions include: plotting traffic sign images, plotting the count of each sign, etc. \n",
    "\n",
    "The [Matplotlib](http://matplotlib.org/) [examples](http://matplotlib.org/examples/index.html) and [gallery](http://matplotlib.org/gallery.html) pages are a great resource for doing visualizations in Python.\n",
    "\n",
    "**NOTE:** It's recommended you start with something simple first. If you wish to do more, come back to it after you've completed the rest of the sections. It can be interesting to look at the distribution of classes in the training, validation and test set. Is the distribution the same? Are there more examples of some classes than others?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Open sign name file to convert class to readable sign-type\n",
    "sign_names = {}\n",
    "\n",
    "with open('signnames.csv', 'r') as f:\n",
    "    reader = csv.reader(f)\n",
    "    for r in reader:\n",
    "        if r[0].isdigit():\n",
    "            sign_names[int(r[0])]= r[1]\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Speed limit (60km/h)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAEyCAYAAACbGke8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHZJJREFUeJzt3V+MXHd1B/DvuXdmdtf/YjsJkZuk5Y8ilTyUgKwIKRVKoUKBl4BUJCIV5QHJqAIJJPoQ8QKtWgmqAu1DRRVKRB7406iBElWobYSoUqQqxdAQEtyWNAqQxIoJJF5nvTsz997ThxnDErzne3ZnvLP+6fuRLK/nXt/5zZ27Z2dnv3uOuTtEREpRLXoBIiLzpKImIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSK0tvNO6uqyuu63nK7wfhByC5m/Bj0lyhmXwa/DwCJpSbEB3Hv6BG6Lt4nc07ZPpmHWrH7sa2vnQs6+nj5E8MeS+a3cNguczmnFT8fVW/2T/Ferx9ub1t+jY1Ho3C7JZ6X4cba8+5+NdtvpkdsZrcB+GsANYC/c/ePR/vXdY2jh48G2/lyqjp+cVn3+BPdkisuc8FV5EVupqhV5G7IQ53cTxfvtDGOLyYAGI3Wwu394AvRBT3y3PUSj2V5sBRut/5BeoyNhnzy+Jgegz2W4bihx2iaeJ8BKRQAUC2txMfYz8/HgSuvjHfo+IV69ZXHwu1nV8/TYzz79LPh9hpDeownHv+PH9GdMMO3n2ZWA/gbAG8DcCOAO8zsxp0eT0RkHmZ5T+1mAE+4+5PuPgLwZQC3z2dZIiI7M0tRuxbATzb9++npbb/CzE6Y2UkzO8neuxERmdUsRe1i7wj92jfo7n63ux939+NVpR+2isilNUuVeRrA9Zv+fR2A+N1AEZFLbJai9m0AN5jZq8xsAODdAB6Yz7JERHZmx5EOd2/M7AMA/gWTSMc97v549H8Mhrra+i6jDNsFnbfh9tGQRxg6lrfwTIYoXmudyBA1XfxYMhkzJ1+XmsT7mBVbq/GvfQ25G6t5dGCdxCCaEY8OdB4fozYexxj6Rrg9FdepSbYr8dz2yT77Erm9avWlcPvKkSvoMV5cPR1u3zgfny8AOHzwQLi9G88ltAlgxpyau38dwNfntBYRkZnpnXsRKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKLvaJNIRN9jL/MJ728bhyVQDP/KwPbEOB1lHFQdrAQAkfJtpaFjVcQ+yfqJvF8s89xKNBlvaeJOHouv+INzejePHCgAdCfB6x8O3TnqMsWAtwAPNVcWvsV4V936zdp0eo/X4nK6f5X3MGsSP9/CR36DHgMUh4NWf8+sjS6/URKQoKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFUVETkaLsak4NiBvsdR2fydiRbBdteAieuxqP+Toa0hWxbXlOrUfmi/YSeagBC5klGjwaGUC6b4XPl+wN4hmVTbtKjzFu4vNekYwiAFQVmenKD4GaNF+0xKyNfi8+p4PlffQYB1aWw+1Lyzy3t9bGObUl58e4/kg8P7i66jA9xk/PkhOfmPmbpVdqIlIUFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKLscvnV40PiwJeFLgE9X7xKNFZsRC84mRnCT6dl1IqB56GA8HbvzxJT3hjX54yHgfh2HPKseD2iORvE5S5xRGAssd7yRoCEOvXbGL/mKNNb0xDBxM3KNjfi1vkZCwF3is3ewtD/cfuTAMXqM5f3x9XF+xJ+XHuJw9qDHw8hZeqUmIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESmKipqIFGV3hxm7ow0aQSbmEKNjIaEmM8yYZMxqHkSqSVO7XpU5tfH91IkhwhvjjfgYiUxV53E+bDg8R4/h7LEYf17MyTm1OC8FAC3i/FeUk/zFMciFWJOmmgC/TruGZ7s2SAaxIdkvADhUxfu0/jN6jHPrcdbtxfXz9BjjJm4SSmJ92zJTUTOzpwCcwyTh2bj78XksSkRkp+bxSu333P35ORxHRGRmek9NRIoya1FzAP9qZt8xsxPzWJCIyCxm/fbzFnd/1sxeAeBBM/tvd39o8w7TYncCAKrEL3mLiMxipirj7s9O/z4D4KsAbr7IPne7+3F3P25kJJyIyKx2XNTMbL+ZHbzwMYC3AnhsXgsTEdmJWb79vAbAV6evvnoAvuju/zyXVYmI7NCOi5q7Pwngddv9f9GE9UyDR5KbRT8xoX2wEocJGxJGBUCbRCLxrXZDmiLWzhsJVl28Ds+8j8kCzWMeFK168f2QZU7XwZ67xHNLhtp7nz8vLVmsseceQDuO91le4o0395EHwxqmAsD6+lq4/QU/TY+xMjgUbh8mmrs2bRzgXh4cpMfI0jv3IlIUFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFF2eZjxpFHkVjKNBKs6ziotDQb8GCS71TU8h+Tk60H0OC8YjeMmgIZEPoxs75PBvADQI5dBlciptSPS0JANKkYmy8a/Blc1aQDa58eoLc6QDXr8GjPSnbPf4w0e9++Ph133l/lzO+yacHuXGBC9Pnwx3N6rE4O7D8XDiq+++hp6jG/RPSb0Sk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRl18O3URWtjTcBpMHZxBq8Ic0ZE80qjXw5MLYDgKaL78c63tBwuR+fMwuacv5iHcP1+BgtbwLY0TOfCM6S7ZlAc0WW0YwywWoy5b3PQ69LK/E+LWvMCQAeN2c8eOgIPcQ1VxyId+j4c/uzM/EU92rMn5crr3hFuP26636bHiNLr9REpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkVRURORoux6Ts2CnFlNGkBO/n8i30N0ZFixJXJqLA9liWHGA/I1pdfnzQi9jZv8jYcbiWOQxoo9fplUg7jpoSUaK4JkEC2RU/M2bopYZZpVNvExPNE0s/G4aSaWEk1Ehy/F62h4Tm2A+Do8P4wfKwCgip+7LjHceXU9fm7P90mebhv0Sk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRlV8O3BqCKQqmJXC3PtGaaAM62fbIOMtU8ERIeVHHYuGt4yHO8wcK1/NH0luKJ5CsHDtNj1MtxeNIGiSAxeXIzsWsj4VvnY+Ax3ogbJzZr8cRyAGhG58Lto43z9Bh+9vlw+4EDiXPaxNfH+oi/rlkfxcdY7vGmmatrq+H2J370A3qMLPqIzOweMztjZo9tuu2omT1oZj+c/s2jzSIiuyDz7efnAdz2stvuAvANd78BwDem/xYRWTha1Nz9IQA/f9nNtwO4d/rxvQDeMed1iYjsyE7fU7vG3U8DgLufNrMtpyqY2QkAJwA+NEVEZFaXvMq4+93uftzdj4c/JBARmYOdFrXnzOwYAEz/PjO/JYmI7NxOi9oDAO6cfnwngK/NZzkiIrOh76mZ2ZcA3ArgKjN7GsBHAXwcwH1m9l4APwbwrtS9maEKslmpHBKpw544SrSGCZ5lYo0TBzXP7nRNnIcabcRDhgHeOLG3bz89xr4rjobb66V99BhekcebaQDKnttEk0iQ8+6Jhob9fpzbq3v8tUC3Gj9elmMDgOF6nGX7+Zln6DHWBmfD7S3i5p4AwGZqVwd4g8c+aSI6PMezf1m0qLn7HVtsesvcViEiMif6caSIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCgqaiJSlN2f0G5BKLGaPVwJSwQjSZqwBz61mv0aq7f8sbTr8RTvzKT4/kocjN13+Ep6DNbgcZSZat7Gj6VX8ZAn63fQJS6PIWnwOFzjodcl8lmxb5k/lpWDV4fb184mro9RHEjdeGmNHqM6FDeSjM/WxJA8/70+D1bv338o3F6PMyvJ0Ss1ESmKipqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCi7Psw4nGUcZdim+oM4p8YTVQDGsw+8NY+Dal3LhgwD1sXDiquaPz3LZNBwvXyQHqNFfN5XE8N72yZuaHjk6DF6DCMZw/GIZ5lWX3z54LOXaeI8HQA0JIO4fp7nw6644opw+/Ih/rxsvBBfQ+Mhv8bW1uJrrL+fZ+5Y9LMZ8aHbDWl4WeU+c1P0Sk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRlV8O3DsCDxoe9RBfAnpOQHus0CAB1nK7MDAIHaeDYtTwoyqaFV4mp5vXScryORKB53MTndDzigdV+olEgwyawj9f5xPoemXp/6HAcVgYAR3x9rK6+QI+xthEHdA+S0DQA9AakeSd5rADQkcBy/zAvAXUV7zNINGbdIIHls+dfosfI0is1ESmKipqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCgLaBK5dQaI5bYAoO3iBo9Nw0NmLIdWkZzS5CDxPm1iHRX5mlL3yeBmAM6ybM4HMw9JpsrHvIHfYCVuelgnvn52HbmfjueyeqSxZlUnhir34vNe13HDQwBomrhxolsiHzaIB1X7+ll6DG/J89/wRpNOzodnoqH9+PPh4Eo87Hg76HLM7B4zO2Nmj2267WNm9oyZPTL98/a5rUhEZAaZbz8/D+C2i9z+aXe/afrn6/NdlojIztCi5u4PASCN30VE9oZZflDwATN7dPrt6ZG5rUhEZAY7LWqfAfAaADcBOA3gk1vtaGYnzOykmZ3sElOaRERmsaOi5u7PuXvrkx9XfhbAzcG+d7v7cXc/XmU6aIiIzGBHVcbMNg9xfCeAx7baV0RkN9GwjJl9CcCtAK4ys6cBfBTArWZ2EyZNxZ4C8L5LuEYRkTRa1Nz9jovc/Lmd3JkD6NqtQ3iJfoYYk1Brm+jw2Lbxe3u9RHPGHmk0mek0ySK+mfAte5dylJjivX6OND0kQVIAaDbiS2mdrhQYLMcNL3Pi+7E6Ec6u4n1I7nqyCtbM1Pg6atJ4M/V2Dgnfjoc80DwaxcfoEutgDS19qAntIiIXpaImIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESnKrjaJhMeNAIP+kb/QBjk3ALAqcRCaEOPHcDJ4uUsMZmb3kvlNWXYvoxHPmLVtvE+/5l/7huM4h3RuyIfVHnEy4Nd5bq8ljSYdvGmm0a/1mWnXZFA1eC7LWdYt87vUJFSXeSS9/iDcvrK0RI+xvBJnEF9a59dpll6piUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkVRURORouxq+NbhJByZqLGkuV4/MfmahhoTDR7ZOjwRAmYB3pY01gOAPjlG0/CQZ0tGbB89dJQeoyHn/YVzz9NjDEfx4+1XPHzbkDxqO+bPbc/i8+GZcLbHC/E2Eb4lzUwz16mTx9IlOl627Bpa4Z+3dT8O364kPm+z9EpNRIqioiYiRVFRE5GiqKiJSFFU1ESkKCpqIlIUFTURKcoCmkQGw4yNN72ryaBhT7S9Y/GetuX5MO/FmSljw47Bc0jtkDfOc5Jly/TMrMmAXzqYF4BZfD6cNG/MsESzys7i+xlurNJjOHn+vePXh5FPrS5xjTWjtfgYJAsHABUbzE1ybAAwHMUDsV86zy+yMWnu2iQGZmfplZqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESnK7oZvDaiqretoJjjLgrFtovkem1rdq3mo0UiTyKVBPNUaAJpxPC28a/k0cR8O43UE5/sX6yBB4dXz5+gxxs35cHudCCMvLcUB3h4JPANA1Y/v56VE+NbW42nyBhJoBXBw/6FwezfmYdPxKF5HJhRd1fF1mAmadyQYu77Gr9PRRhzg7SWC1Vn0SGZ2vZl908xOmdnjZvbB6e1HzexBM/vh9O8jc1uViMgOZcpjA+DD7v5aAG8E8H4zuxHAXQC+4e43APjG9N8iIgtFi5q7n3b3704/PgfgFIBrAdwO4N7pbvcCeMelWqSISNa23lMzs1cCeD2AhwFc4+6ngUnhM7NXbPF/TgA4AQBV4pdnRURmka4yZnYAwP0APuTu/N3WKXe/292Pu/txy7SMEBGZQaqo2aSvzP0AvuDuX5ne/JyZHZtuPwbgzKVZoohIXuannwbgcwBOufunNm16AMCd04/vBPC1+S9PRGR7Mu+p3QLgPQC+b2aPTG/7CICPA7jPzN4L4McA3sUOZGYY9Le+yybR9K5jzQY7fgyWU3OSQQOApomP0a9W+DoqNvA2zvYAwHj9hXgdB+K8FACsLF8Rbt9gE4IBVP34nC0vH6DHGAyWwu2WGGa8/9CV4fbxKM71AUBFuogOKv5p0yPX2Plz/B2chqy1SmT/QE5Z2/FrjJ2PRLwUHWlmOm7n99YUfXbc/VvAliOp3zK3lYiIzIF+HCkiRVFRE5GiqKiJSFFU1ESkKCpqIlIUFTURKYqKmogUZVebRBqAyrYO2dWJX3g3kvRrE0lAC6bEA7lfvO9IuBJsMjaAaikOm7brPCg63oibM2ZGtA/2HQ239/fvp8cAafLnW0Ydf8nIeU9kPGkjyV7NA7w1ab5oI97gcX31bLi9G67TYyD4XAGAemmZHsJJk9DUZHR2rSeeWxZob1mofhv0Sk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpyq7m1BxAE+RRMm3iKrZXYngvSGamSyzEPR7g2nRr9BgrK3FOzX0fPQbLqbVrPA/VNHEn9v4+3miyt3Qw3F4n8mEVa5oJnmUy1gA0MSC6GcXnbLyWGO68ER/DEo+lvxwPIq7q+PoBgGETP97E6UBFs46Z4d+kqWqP5zqz9EpNRIqioiYiRVFRE5GiqKiJSFFU1ESkKCpqIlIUFTURKYqKmogUZXfDt+5og2nPFWnemGGJ8G1FGhqO20TIs4n3GSUmxRuZfM3ClwDgiAO67QZvNNlsxI0Cu1E8BR4ARnUcNu71+WMx8rywpokA0LQkwEvCqJN9SOPERGK1quPntu7zBo9Wx+dsnFhHR5sv8s+5yuJgbKYBKMg6Ek9tml6piUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkXZ9WHG5lvniDwxRNhJtqtHtgNAxzJkibick50SMTWsk6G448SA16VBnFOzxNetdhyHhOj5AuCjOA/XksaLE7OHldhTR5uMAjQ0lclC9pfjppid8aaZo4YMACZDlwHA6T78fLTspCY+51gjSU80msyiz46ZXW9m3zSzU2b2uJl9cHr7x8zsGTN7ZPrn7XNblYjIDmVeqTUAPuzu3zWzgwC+Y2YPTrd92t3/8tItT0Rke2hRc/fTAE5PPz5nZqcAXHupFyYishPb+kGBmb0SwOsBPDy96QNm9qiZ3WNmR+a8NhGRbUsXNTM7AOB+AB9y91UAnwHwGgA3YfJK7pNb/L8TZnbSzE52qTcURUR2LlXUzKyPSUH7grt/BQDc/Tl3b929A/BZADdf7P+6+93uftzdj1fz/FV8EZGLyPz00wB8DsApd//UptuPbdrtnQAem//yRES2J/PTz1sAvAfA983skeltHwFwh5ndhEk06CkA77skKxQR2YbMTz+/hYsn9L6+o3sMJmi3QTD3l/8/3tyS6esAjxvWVabWk4BmIsHL3mMcjTLhynhC+3KPN2fs1/vJHonmjCRI7C1pvAjAM88/wZ7+uubPbcWmybNmlgCcTJsfN2N6jJZNm5/HLwRZIlhNAryW+Jxj2m5+v9ykX5MSkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRFRU1EirK7w4wRZ5FSv0ZFGklmMjNVRTJmqSaR9E74OlguK7GQhgzn3UgMiK6qOMtmVTzMFgDQI8/LgOflKnLO+gPeWNHJ421GPB/Wka6IbZcYiEwGYnvmIqvj897vJT59SYPPJpGXo/nBxENhn9m9zGNJ0is1ESmKipqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCgqaiJSlF0N384Dz7Re+infmX1YCHQiXmudaEZoJLDcNLzRZNPG09XZNPqJOKCZeSw1CfDuW17hx1haCre/eOZ5egwHC9fya8xJSNwz1ykNEm/QQ/BmpYnHwo6RutRZaD5xjCS9UhORoqioiUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSKsqs5NUPccDATVTE2RDh1ELJT4iAsh8YGwAJ8sG6Vac5IBiJXJB8E8KHK3iWGKpOmh+QuAPC19gf8cm37cU5tY8yH9/boLvwYNE9JBhVP9iGbMyeVXeqJlzU1eTCWaYhq8bVsmceSpFdqIlIUFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKLs/oT0IA3aJxopGpkWzKd8AUJPJ15n+joY4kJrreRffUUemawM8gJlp8MhOmSW+9rH7sUTKk02C90QY2chzm1kHnUiewALLNQmjAkBHQuBtIrDKBsGnQq9kFxbOTRwi9ViyeB9Zs2Uz+08z+56ZPW5mfzK9/VVm9rCZ/dDM/t7MBnNblYjIDmW+/RwCeLO7vw7ATQBuM7M3AvgEgE+7+w0AXgDw3ku3TBGRHFrUfOKl6T/70z8O4M0A/mF6+70A3nFJVigisg2pHxSYWW1mjwA4A+BBAP8H4EV3vzCh4mkA127xf0+Y2UkzO5n6BVwRkRmkipq7t+5+E4DrANwM4LUX222L/3u3ux939+Ns8pGIyKy2Felw9xcB/BuANwI4bGYXfnp6HYBn57s0EZHty/z082ozOzz9eAXA7wM4BeCbAP5gutudAL52qRYpIpKVyakdA3CvmdWYFMH73P2fzOwHAL5sZn8G4L8AfC5zh2H2KvGeG8uh5XJI8fa2ZcNsQcNsmW+0K9Zoskrk1GizysQ5JW8L9Ej2CwA6lrnL5NToPvysVvNoIkqOkclCsqxbm8ggzmPCb0fWYZlIHlmHtZmDzGEgchItau7+KIDXX+T2JzF5f01EZM/Qr0mJSFFU1ESkKCpqIlIUFTURKYqKmogURUVNRIqioiYiRbHd/CVzM/spgB9tuukqAM/v2gJmc7ms9XJZJ3D5rPVyWSdw+ax1J+v8LXe/mu20q0Xt1+580rnj+MIWsA2Xy1ovl3UCl89aL5d1ApfPWi/lOvXtp4gURUVNRIqy6KJ294Lvfzsul7VeLusELp+1Xi7rBC6ftV6ydS70PTURkXlb9Cs1EZG5UlETkaIsrKiZ2W1m9j9m9oSZ3bWodTBm9pSZfd/MHjGzk4tez2Zmdo+ZnTGzxzbddtTMHpzOY33QzI4sco3TNV1snR8zs2em5/URM3v7Itd4gZldb2bfNLNT0zm3H5zevqfOa7DOPXVeFzI32N13/Q+AGpOJVK8GMADwPQA3LmItibU+BeCqRa9ji7W9CcAbADy26ba/AHDX9OO7AHxij67zYwD+eNFru8hajwF4w/TjgwD+F8CNe+28BuvcU+cVkzbCB6Yf9wE8jMmMk/sAvHt6+98C+KN53eeiXqndDOAJd3/S3UcAvgzg9gWt5bLl7g8B+PnLbr4dkzmswB6Zx7rFOvckdz/t7t+dfnwOk3kc12KPnddgnXuKT+zq3OBFFbVrAfxk07+3nBu6BziAfzWz75jZiUUvJuEadz8NTC58AK9Y8HoiHzCzR6ffni782+SXM7NXYtLK/mHs4fP6snUCe+y8zjI3eCcWVdQuNslhr2ZLbnH3NwB4G4D3m9mbFr2gQnwGwGsA3ATgNIBPLnY5v8rMDgC4H8CH3H110evZykXWuefOq88wN3gnFlXUngZw/aZ/79m5oe7+7PTvMwC+ir0/bOY5MzsGANO/zyx4PRfl7s9NL/YOwGexh86rmfUxKRRfcPevTG/ec+f1Yuvcy+fVd2lu8KKK2rcB3DD9CcgAwLsBPLCgtWzJzPab2cELHwN4K4DH4v+1cA9gMocV2MPzWC8UiKl3Yo+cVzMzTMY9nnL3T23atKfO61br3GvndRFzgxf2GwXTHzX/FSY/Cb3H3f98IQsJmNmrMXl1BkzGCX5xL63TzL4E4FZM2rg8B+CjAP4Rk58s/SaAHwN4l7sv9E36LdZ5KybfIjkmP2F+34X3rBbJzH4XwL8D+D6ACwMtP4LJ+1V75rwG67wDe+i8mtnvYPKDgM1zg/90+rn1ZQBHMZkb/IfuPpzLfS6qqImIXAr6jQIRKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKP8PsAMa32BE9Z0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f02186a5cf8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualizations will be shown in the notebook.\n",
    "%matplotlib inline\n",
    "\n",
    "index = random.randint(0, len(X_train))\n",
    "image = X_train[index].squeeze()\n",
    "\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.imshow(image)\n",
    "sign_num = y_train[index]\n",
    "print(sign_names[sign_num])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEICAYAAAC55kg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFpBJREFUeJzt3X+U5XV93/HnqwtiWkRAFs66u2TRrgTwxFWnSA8xpSHFBa1Aqw1EZTXkrCZwqqemDdrTYjEktI2SQ5uQrrIHOEF+VES2ijUr1RB75McghB+uyIIExl13R5EfKR7axXf/uN8Jl907szNz78zdme/zcc49c7/v+/l+v5/7XZjXfD7f773fVBWSpHb6O8PugCRpeAwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqMUNArZRkSZK/SXLkINtKC40hoAWh+SU88fhZkp92Lb9npturqheq6sCqenyQbWcqySFJrkzywyTPJHkoye9Mc90/S/KJQfdJ7bLfsDsgTUdVHTjxPMljwG9W1dcma59kv6raNR9969NlwBLgF4BngKOBY4baI7WKIwEtCkl+L8n1Sa5N8izw3iT/MMntSZ5Ksj3JZUn2b9rvl6SSrGqW/6x5/StJnk3yrSRHzbRt8/qpSb6X5Okk/yXJ/07y/km6/g+Az1XVU1X1s6raUlVf6NrWsUm+luTJJN9N8s+b+m8DvwZ8vBkN3TTYI6q2MAS0mJwJfA54JXA9sAv4MHAYcCKwFvjgFOv/OvDvgEOBx4FPzrRtksOBG4B/3ez3+8DxU2znduAPkrw/yeruF5K8AtgMXA0cDrwH2JDk6Kr6k+Y9/n4zVXXmFPuQJmUIaDH5ZlX9j+Yv6p9W1V1VdUdV7aqqR4ENwD+aYv3PV9VoVf0/4BpgzSzavgO4t6publ67FPjRFNv5bTq/zP8lsCXJw0lOaV57J/C9qrq6eQ93A18E3jX1YZCmzxDQYvJE90KSX0jy5YmTrsBFdP46n8wPu54/Bxw4WcMp2r66ux/V+YbGsck2UlXPVdXvVdWbgFcBXwBuTPJK4OeBE5vprKeSPEVnCmjZFP2SZsQQ0GKy+1fi/jfgAeDvV9VBwL8HMsd92A6smFhIEmD5dFasqqeBP6ATKKvohMmtVXVw1+PAqjp/YpWB9lytZAhoMXsF8DTwf5Icw9TnAwblS8CbkvzTJPvROSexdLLGSS5MMpLkZUleTmda6EngYWATcFySX0+yf/M4PsnRzeo7gNfM7dvRYmcIaDH7KLAOeJbOqOD6ud5hVe2gM2XzaeDHwGuBe4Dnp1jtqqbtNuAk4O3NNNHTwNuA99IZYfyQzkjhgGa9zwJvSPKTJJ8f/LtRG8SbykhzJ8kSOr/c31VVfzns/ki7cyQgDViStUlemeQAOpeR7gLuHHK3pJ4MAWnwfgl4lM6loWuBM6pqqukgaWicDpKkFnMkIEktts9/gdxhhx1Wq1atGnY3JGnBuPvuu39UVZNemtxtnw+BVatWMTo6OuxuSNKCkeSvp9vW6SBJajFDQJJazBCQpBYzBCSpxQwBSWoxQ0CSWswQkKQW22sIJFmZ5OtJtiR5MMmHm/qhSTY3t8PbnOSQpp7mJtxbk9yX5E1d21rXtH84ybq5e1uSpOmYzkhgF/DRqjoGOAE4L8mxwAV07nq0Gri1WQY4FVjdPNYDl0MnNIALgbfQufH2hRPBIUkajr1+YriqttO5oQVV9WySLXRul3c6nRtgQOemGN8AfrepX93cW/X2JAcnWda03VxVTwIk2UznGxavHeD7GYhVF3y5Z/2xS94+zz2RpLk1o3MCSVYBbwTuAI5oAmIiKA5vmi3npTf8Hmtqk9V77Wd9ktEko+Pj4zPpoiRpBqYdAkkOBG4EPlJVz0zVtEetpqjvWazaUFUjVTWydOm0vgNJkjQL0/oCuST70wmAa6rqC015R5JlVbW9me7Z2dTHgJVdq6+gc3u9MV6cPpqof2P2Xe+PUz6SNL2rgwJcAWypqk93vbSJzk28aX7e3FU/p7lK6ATg6Wa66KvAKUkOaU4In9LUJElDMp2RwInA+4D7k9zb1D4OXALckORc4HHg3c1rtwCnAVuB54APAFTVk0k+CdzVtLto4iSxJGk4pnN10DfpPZ8PcHKP9gWcN8m2NgIbZ9JBSdLc8RPDktRihoAktdg+f3tJLRxecSUtPI4EJKnFDAFJajGngzQjTvlIi4sjAUlqMUNAklrM6SBJ6tK2KU9HApLUYoaAJLWYISBJLWYISFKLGQKS1GJeHdRSbbsCQlJvjgQkqcWmc3vJjUl2Jnmgq3Z9knubx2MTdxxLsirJT7te+9Oudd6c5P4kW5Nc1ty2UpI0RNOZDroS+K/A1ROFqvq1iedJPgU83dX+kapa02M7lwPrgdvp3IJyLfCVmXd53+UUi6SFZq8jgaq6Deh5L+Dmr/l/AVw71TaSLAMOqqpvNbefvBo4Y+bdlSQNUr/nBN4K7Kiqh7tqRyW5J8lfJHlrU1sOjHW1GWtqkqQh6vfqoLN56ShgO3BkVf04yZuBLyY5jt43qq/JNppkPZ2pI4488sg+uyhJmsysRwJJ9gP+GXD9RK2qnq+qHzfP7wYeAV5H5y//FV2rrwC2TbbtqtpQVSNVNbJ06dLZdlGStBf9TAf9KvDdqvrbaZ4kS5MsaZ6/BlgNPFpV24Fnk5zQnEc4B7i5j31LkgZgOpeIXgt8Czg6yViSc5uXzmLPE8K/DNyX5K+AzwMfqqqJk8q/BXwW2EpnhLCorgySpIVor+cEqursServ71G7EbhxkvajwOtn2D9J0hzyE8OS1GKGgCS1mCEgSS1mCEhSixkCktRi3k9gnvjlcpL2RY4EJKnFDAFJajFDQJJazBCQpBYzBCSpxQwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqMUNAklpsOreX3JhkZ5IHumqfSPKDJPc2j9O6XvtYkq1JHkrytq762qa2NckFg38rkqSZms5I4EpgbY/6pVW1pnncApDkWDr3Hj6uWedPkixpbj7/x8CpwLHA2U1bSdIQTecew7clWTXN7Z0OXFdVzwPfT7IVOL55bWtVPQqQ5Lqm7Xdm3GNJ0sD0c07g/CT3NdNFhzS15cATXW3Gmtpk9Z6SrE8ymmR0fHy8jy5KkqYy2xC4HHgtsAbYDnyqqadH25qi3lNVbaiqkaoaWbp06Sy7KEnam1ndVKaqdkw8T/IZ4EvN4hiwsqvpCmBb83yyuiRpSGY1EkiyrGvxTGDiyqFNwFlJDkhyFLAauBO4C1id5KgkL6Nz8njT7LstSRqEvY4EklwLnAQclmQMuBA4KckaOlM6jwEfBKiqB5PcQOeE7y7gvKp6odnO+cBXgSXAxqp6cODvRpI0I9O5OujsHuUrpmh/MXBxj/otwC0z6p0kaU75iWFJajFDQJJazBCQpBYzBCSpxQwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqMUNAklpsVt8iKmn+rLrgy3vUHrvk7UPoiRYjRwKS1GKGgCS1mNNBi5jTCJL2xpGAJLWYISBJLeZ00ALgtI6kubLXkUCSjUl2Jnmgq/afk3w3yX1JbkpycFNfleSnSe5tHn/atc6bk9yfZGuSy5Jkbt6SJGm6pjMddCWwdrfaZuD1VfWLwPeAj3W99khVrWkeH+qqXw6sp3Pz+dU9tilJmmfTucfwbUlW7Vb7867F24F3TbWNJMuAg6rqW83y1cAZwFdm2N+h6zU1A8ObnnGqSFI/BnFi+Dd46S/zo5Lck+Qvkry1qS0HxrrajDW1npKsTzKaZHR8fHwAXZQk9dJXCCT5t8Au4JqmtB04sqreCPwr4HNJDgJ6zf/XZNutqg1VNVJVI0uXLu2ni5KkKcz66qAk64B3ACdXVQFU1fPA883zu5M8AryOzl/+K7pWXwFsm+2+JUmDMauRQJK1wO8C76yq57rqS5MsaZ6/hs4J4EerajvwbJITmquCzgFu7rv3kqS+7HUkkORa4CTgsCRjwIV0rgY6ANjcXOl5e3Ml0C8DFyXZBbwAfKiqnmw29Vt0rjT6OTrnEBbcSWFJWmymc3XQ2T3KV0zS9kbgxkleGwVeP6PeSZLmlF8bIUktZghIUov53UHaw772gThJc8eRgCS1mCEgSS1mCEhSixkCktRihoAktZghIEktZghIUosZApLUYoaAJLWYnxjWouWtNxc//43750hAklrMEJCkFjMEJKnFDAFJarFphUCSjUl2Jnmgq3Zoks1JHm5+HtLUk+SyJFuT3JfkTV3rrGvaP9zcqF6SNETTHQlcCazdrXYBcGtVrQZubZYBTqVzg/nVwHrgcuiEBp37E78FOB64cCI4JEnDMa0QqKrbgCd3K58OXNU8vwo4o6t+dXXcDhycZBnwNmBzVT1ZVT8BNrNnsEiS5lE/5wSOqKrtAM3Pw5v6cuCJrnZjTW2y+h6SrE8ymmR0fHy8jy5KkqYyFx8WS49aTVHfs1i1AdgAMDIy0rONBH5YSOpXPyOBHc00D83PnU19DFjZ1W4FsG2KuiRpSPoJgU3AxBU+64Cbu+rnNFcJnQA83UwXfRU4JckhzQnhU5qaJGlIpjUdlORa4CTgsCRjdK7yuQS4Icm5wOPAu5vmtwCnAVuB54APAFTVk0k+CdzVtLuoqnY/2SxJmkfTCoGqOnuSl07u0baA8ybZzkZg47R7J0maU35iWJJazBCQpBYzBCSpxQwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqMUNAklrMEJCkFjMEJKnFDAFJajFDQJJazBCQpBYzBCSpxQwBSWqxWYdAkqOT3Nv1eCbJR5J8IskPuuqnda3zsSRbkzyU5G2DeQuSpNma1p3Feqmqh4A1AEmWAD8AbqJzO8lLq+oPu9snORY4CzgOeDXwtSSvq6oXZtsHSVJ/BjUddDLwSFX99RRtTgeuq6rnq+r7dO5BfPyA9i9JmoVBhcBZwLVdy+cnuS/JxiSHNLXlwBNdbcaamiRpSPoOgSQvA94J/PemdDnwWjpTRduBT0007bF6TbLN9UlGk4yOj4/320VJ0iQGMRI4Ffh2Ve0AqKodVfVCVf0M+AwvTvmMASu71lsBbOu1waraUFUjVTWydOnSAXRRktTLIELgbLqmgpIs63rtTOCB5vkm4KwkByQ5ClgN3DmA/UuSZmnWVwcBJPm7wD8BPthV/k9J1tCZ6nls4rWqejDJDcB3gF3AeV4ZJEnD1VcIVNVzwKt2q71vivYXAxf3s09J0uD4iWFJarG+RgLSdK264Ms9649d8vZ57onmiv/GC5MjAUlqMUNAklrMEJCkFjMEJKnFDAFJajFDQJJazBCQpBYzBCSpxfywmKSh8kNmw+VIQJJazBCQpBYzBCSpxQwBSWoxQ0CSWsyrgzR0Xh0ye72OncdNM9H3SCDJY0nuT3JvktGmdmiSzUkebn4e0tST5LIkW5Pcl+RN/e5fkjR7g5oO+sdVtaaqRprlC4Bbq2o1cGuzDHAqnRvMrwbWA5cPaP+SpFmYq3MCpwNXNc+vAs7oql9dHbcDBydZNkd9kCTtxSBCoIA/T3J3kvVN7Yiq2g7Q/Dy8qS8Hnuhad6ypvUSS9UlGk4yOj48PoIuSpF4GcWL4xKraluRwYHOS707RNj1qtUehagOwAWBkZGSP1yVJg9F3CFTVtubnziQ3AccDO5Isq6rtzXTPzqb5GLCya/UVwLZ++yBpZrwiSxP6mg5K8veSvGLiOXAK8ACwCVjXNFsH3Nw83wSc01wldALw9MS0kSRp/vU7EjgCuCnJxLY+V1X/M8ldwA1JzgUeB97dtL8FOA3YCjwHfKDP/UuS+tBXCFTVo8AbetR/DJzco17Aef3sU1qInH7RvsqvjZCkFjMEJKnFDAFJajFDQJJazBCQpBYzBCSpxbyfgKRp81LXxceRgCS1mCEgSS3mdNA+wCH21LyFYrv57z+3HAlIUosZApLUYk4HSQPitN6+xWmk6XEkIEktZghIUos5HSTtxmkdzcZC/e/GkYAktdisQyDJyiRfT7IlyYNJPtzUP5HkB0nubR6nda3zsSRbkzyU5G2DeAOSpNnrZzpoF/DRqvp2c7P5u5Nsbl67tKr+sLtxkmOBs4DjgFcDX0vyuqp6oY8+SPNqoQ75NRiL8d9/1iOBqtpeVd9unj8LbAGWT7HK6cB1VfV8VX2fzs3mj5/t/iVJ/RvIOYEkq4A3Anc0pfOT3JdkY5JDmtpy4Imu1caYJDSSrE8ymmR0fHx8EF2UJPXQ99VBSQ4EbgQ+UlXPJLkc+CRQzc9PAb8BpMfq1WubVbUB2AAwMjLSs420WOxrUwz7Wn80t/oaCSTZn04AXFNVXwCoqh1V9UJV/Qz4DC9O+YwBK7tWXwFs62f/kqT+9HN1UIArgC1V9emu+rKuZmcCDzTPNwFnJTkgyVHAauDO2e5fktS/fqaDTgTeB9yf5N6m9nHg7CRr6Ez1PAZ8EKCqHkxyA/AdOlcWneeVQdLc2Ze+O6ftU0xTvf9hH5tZh0BVfZPe8/y3TLHOxcDFs92nJGmw/MSwJLXYov7uoH1pOKx9y7CH4Bou//1f5EhAklrMEJCkFjMEJKnFDAFJajFDQJJazBCQpBYzBCSpxQwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqMUNAklrMEJCkFjMEJKnF5j0EkqxN8lCSrUkumO/9S5JeNK8hkGQJ8MfAqcCxdO5HfOx89kGS9KL5HgkcD2ytqker6v8C1wGnz3MfJEmNVNX87Sx5F7C2qn6zWX4f8JaqOn+3duuB9c3i0cBDA9j9YcCPBrCdxchjMzmPzeQ8NpMb9rH5+apaOp2G832P4fSo7ZFCVbUB2DDQHSejVTUyyG0uFh6byXlsJuexmdxCOjbzPR00BqzsWl4BbJvnPkiSGvMdAncBq5McleRlwFnApnnugySpMa/TQVW1K8n5wFeBJcDGqnpwnnY/0OmlRcZjMzmPzeQ8NpNbMMdmXk8MS5L2LX5iWJJazBCQpBZb9CHg11S8VJKNSXYmeaCrdmiSzUkebn4eMsw+DkOSlUm+nmRLkgeTfLipe2ySlye5M8lfNcfmPzT1o5Lc0Ryb65uLPVopyZIk9yT5UrO8YI7Nog4Bv6aipyuBtbvVLgBurarVwK3NctvsAj5aVccAJwDnNf+teGzgeeBXquoNwBpgbZITgP8IXNocm58A5w6xj8P2YWBL1/KCOTaLOgTwayr2UFW3AU/uVj4duKp5fhVwxrx2ah9QVdur6tvN82fp/A+9HI8N1fE3zeL+zaOAXwE+39RbeWwAkqwA3g58tlkOC+jYLPYQWA480bU81tT0UkdU1Xbo/DIEDh9yf4YqySrgjcAdeGyAv53uuBfYCWwGHgGeqqpdTZM2/7/1R8C/AX7WLL+KBXRsFnsITOtrKqQJSQ4EbgQ+UlXPDLs/+4qqeqGq1tD5lP/xwDG9ms1vr4YvyTuAnVV1d3e5R9N99tjM93cHzTe/pmJ6diRZVlXbkyyj89de6yTZn04AXFNVX2jKHpsuVfVUkm/QOW9ycJL9mr942/r/1onAO5OcBrwcOIjOyGDBHJvFPhLwayqmZxOwrnm+Drh5iH0ZimYe9wpgS1V9uuslj02yNMnBzfOfA36VzjmTrwPvapq18thU1ceqakVVraLz++V/VdV7WEDHZtF/YrhJ6D/ixa+puHjIXRqqJNcCJ9H5qtsdwIXAF4EbgCOBx4F3V9XuJ48XtSS/BPwlcD8vzu1+nM55gbYfm1+kc3JzCZ0/HG+oqouSvIbOxRaHAvcA762q54fX0+FKchLwO1X1joV0bBZ9CEiSJrfYp4MkSVMwBCSpxQwBSWoxQ0CSWswQkKQWMwQkqcUMAUlqsf8PEaENd+Vz2PcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bd65a0f0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEmBJREFUeJzt3XuQnXV9x/H3R/BWQQFZmADBpU7sSK2iE5Gp7RTFUSBqsBXFKqRWG6fiCFOcNvJHtc6gccbb2KF2UBnD1FtavMQGL5Rq1WlRA6KAkTFChJCYBAHBS7GBb/84z8ph2cvJnt2c5Jf3a2bnnOf33L7nt7uf8+zvPM+zqSokSe16xKgLkCQtLINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr32WUnGk1SSA7vpLyZZMciyc9jXhUk+Mky90qgY9BqZJF9O8o4p2pcn+enuhnJVnVZVa+ahrpOTbJm07XdW1euH3fYU+3pUkvcm2ZLkF0luSfL+Add9e5J/me+a1B6DXqP0MeDsJJnUfjbw8aratedL2uPeCiwFTgQOBp4HfHekFak5Br1G6XPAYcAfTzQkORR4MXBZN70syXeT3JPktiRvn25jSb6W5PXd8wOSvCfJHUluBpZNWva1STYmuTfJzUne0LU/DvgicFR3hP2LJEdNPnpO8tIkNya5u9vvU/vmbU7yliTfT/LzJJ9O8phpyn428Nmq2lo9m6vqsr5tHZXk8iQ7u6P9N3ftpwIXAq/savze7N2t/ZVBr5Gpql8Da4Fz+ppfAfywqiaC65fd/EPohfVfJzljgM3/Fb03jGfSO2J++aT5O7r5jwdeC7w/ybOq6pfAacDWqjqo+9rav2KSpwCfBM4HxoArgC8kedSk13EqcBzwdOAvpqnzauBvkrwxyR/0/3WT5BHAF4DvAUcDpwDnJ3lRVX0JeCfw6a7GZwzQJ9pPGfQatTXAmUke202f07UBUFVfq6rrq+qBqvo+vYD9kwG2+wrgA1V1W1XdCbyrf2ZVra+qH3dH0f8FfIW+vyxm8UpgfVVdWVX/B7wHeCzwh33LfLA7Sr+TXlifMM223gW8G3g1sAG4ve8D5WcDY1X1jqr6TVXdDHwYOGvAOiUA5nQGgjRfquqbSXYCy5N8m164/enE/CTPAVYDTwMeBTwa+NcBNn0UcFvf9E/6ZyY5DXgb8BR6Bzy/A1w/YNlH9W+vqh5Ichu9o+4JP+17/qtunYepqvuBi4GLuze7vwQu7friSfSGkO7uW+UA4BsD1ikBHtFr73AZvSP5s4GvVNX2vnmfANYBi6vqCcA/A5M/vJ3KNmBx3/SxE0+SPBq4nN6R+JFVdQi94ZeJ7c52S9et9EJ4Ynvp9nX7AHVNq6p+XVUXA3cBx9N7o7qlqg7p+zq4qk4fsE4JMOi1d7gMeAG9cfXJp0ceDNxZVf+b5ETgzwfc5lrgzUmO6T7gXdU3b+Ivg53Aru7o/oV987cDT0zyhBm2vSzJKUkeCVwA3Af894C1/VaS87vTOR+b5MBu2OZgemfefBu4J8nfdfMPSPK0JM/uq3O8G8uXpuUPiEauqjbTC8nH0Tt67/dG4B1J7gX+nl7IDuLDwJfpfZB5LfCZvv3dC7y529Zd9N481vXN/yG9zwJu7s6qeciwS1XdBLwG+EfgDuAlwEuq6jcD1tbv18B76Q313AGcC/xZVd3cDeu8hN74/i3d/I8AE29AE0NYP0ty7Rz2rf1E/McjktQ2j+glqXEGvSQ1zqCXpMbNGvRJFif5ane5+I1Jzuva357k9iTXdV+n963z1iSbktyU5EUL+QIkSTOb9cPYJIuARVV1bZKDgWuAM+hdefiLqnrPpOWPp3fGwon0LhL5D+Ap3RkEUzr88MNrfHx8mNchSfuda6655o6qGpttuVmvjK2qbfQuPqGq7k2ykYdeATjZcuBTVXUfcEuSTfRC/3+mW2F8fJwNGzbMVookqU+Sn8y+1G6O0ScZp3eTqG91TW/q7tB3aXdRCvTeBPovPd/CFG8MSVYm2ZBkw86dO3enDEnSbhg46JMcRO+y8fOr6h7gQ8CT6V3MsY3eRR8w9eXpDxsfqqpLqmppVS0dG5v1Lw9J0hwNFPTdZd6X0/tnEJ8BqKrtVXV/VT1A7yrEE7vFt/DQe4wcQ+/eIJKkERjkrJsAHwU2VtX7+toX9S32MuCG7vk64Kwkj05yHLCE3j07JEkjMMhtip9L766C1ye5rmu7EHhVkhPoDctsBt4AUFU3JlkL/ADYBZw70xk3kqSFNchZN99k6nH3K2ZY5yLgoiHqkiTNE6+MlaTGGfSS1DiDXpIa1/z/jB1ftf5hbZtXL5t1nqY3Vb+BfSftrTyil6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGHTjqAvZF46vWT9m+efWyGeeNwlT1DFLLXNeT9hX708+4R/SS1DiDXpIaZ9BLUuMMeklq3KxBn2Rxkq8m2ZjkxiTnde2HJbkyyY+6x0O79iT5YJJNSb6f5FkL/SIkSdMb5Ih+F3BBVT0VOAk4N8nxwCrgqqpaAlzVTQOcBizpvlYCH5r3qiVJA5s16KtqW1Vd2z2/F9gIHA0sB9Z0i60BzuieLwcuq56rgUOSLJr3yiVJA9mtMfok48AzgW8BR1bVNui9GQBHdIsdDdzWt9qWrm3ytlYm2ZBkw86dO3e/cknSQAYO+iQHAZcD51fVPTMtOkVbPayh6pKqWlpVS8fGxgYtQ5K0mwYK+iSPpBfyH6+qz3TN2yeGZLrHHV37FmBx3+rHAFvnp1xJ0u4a5KybAB8FNlbV+/pmrQNWdM9XAJ/vaz+nO/vmJODnE0M8kqQ9b5B73TwXOBu4Psl1XduFwGpgbZLXAbcCZ3bzrgBOBzYBvwJeO68VS5J2y6xBX1XfZOpxd4BTpli+gHOHrEuSNE+8MlaSGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS42YN+iSXJtmR5Ia+trcnuT3Jdd3X6X3z3ppkU5KbkrxooQqXJA1mkCP6jwGnTtH+/qo6ofu6AiDJ8cBZwO936/xTkgPmq1hJ0u6bNeir6uvAnQNubznwqaq6r6puATYBJw5RnyRpSAcOse6bkpwDbAAuqKq7gKOBq/uW2dK1PUySlcBKgGOPPXaIMhbG+Kr1D2vbvHrZHt3fxD7nOm8u+1zI16jpDfN9lGYz1w9jPwQ8GTgB2Aa8t2vPFMvWVBuoqkuqamlVLR0bG5tjGZKk2cwp6Ktqe1XdX1UPAB/mweGZLcDivkWPAbYOV6IkaRhzCvoki/omXwZMnJGzDjgryaOTHAcsAb49XImSpGHMOkaf5JPAycDhSbYAbwNOTnICvWGZzcAbAKrqxiRrgR8Au4Bzq+r+hSldkjSIWYO+ql41RfNHZ1j+IuCiYYqSJM0fr4yVpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjZv3n4NKoja9aP2X75tXLZpynNvg9Hp5H9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY2bNeiTXJpkR5Ib+toOS3Jlkh91j4d27UnywSSbknw/ybMWsnhJ0uwGOaL/GHDqpLZVwFVVtQS4qpsGOA1Y0n2tBD40P2VKkuZq1qCvqq8Dd05qXg6s6Z6vAc7oa7+seq4GDkmyaL6KlSTtvrmO0R9ZVdsAuscjuvajgdv6ltvStUmSRmS+P4zNFG015YLJyiQbkmzYuXPnPJchSZow16DfPjEk0z3u6Nq3AIv7ljsG2DrVBqrqkqpaWlVLx8bG5liGJGk2cw36dcCK7vkK4PN97ed0Z9+cBPx8YohHkjQas/5z8CSfBE4GDk+yBXgbsBpYm+R1wK3Amd3iVwCnA5uAXwGvXYCaJUm7Ydagr6pXTTPrlCmWLeDcYYuSJM0fr4yVpMYZ9JLUOINekho36xi9tDvGV62fsn3z6mUzztO+Y67fR7//o+MRvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ17sBRF7A/GV+1fsr2zauX7eFK9j6j6Jup9jmxv92dNzF/IV7HMNuc6XUsxHpzNdfXuLf83Cz0PoflEb0kNc6gl6TGGfSS1DiDXpIaN9SHsUk2A/cC9wO7qmppksOATwPjwGbgFVV113BlSpLmaj6O6J9XVSdU1dJuehVwVVUtAa7qpiVJI7IQQzfLgTXd8zXAGQuwD0nSgIYN+gK+kuSaJCu7tiOrahtA93jEkPuQJA1h2AumnltVW5McAVyZ5IeDrti9MawEOPbYY4csQ5I0naGO6Ktqa/e4A/gscCKwPckigO5xxzTrXlJVS6tq6djY2DBlSJJmMOegT/K4JAdPPAdeCNwArANWdIutAD4/bJGSpLkbZujmSOCzSSa284mq+lKS7wBrk7wOuBU4c/gyJUlzNeegr6qbgWdM0f4z4JRhipIkzR+vjJWkxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS44b9xyMjN75q/ZTtm1cv28OVaG801c/HqH42FuJndZhtLkTf7G2/j3N9jfO53sS6o+wbj+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjFizok5ya5KYkm5KsWqj9SJJmtiBBn+QA4GLgNOB44FVJjl+IfUmSZrZQR/QnApuq6uaq+g3wKWD5Au1LkjSDVNX8bzR5OXBqVb2+mz4beE5VvalvmZXAym7y94Cb5mHXhwN3zMN2WmTfTM++mZn9M71R982TqmpstoUOXKCdZ4q2h7yjVNUlwCXzutNkQ1Utnc9ttsK+mZ59MzP7Z3r7St8s1NDNFmBx3/QxwNYF2pckaQYLFfTfAZYkOS7Jo4CzgHULtC9J0gwWZOimqnYleRPwZeAA4NKqunEh9jXJvA4FNca+mZ59MzP7Z3r7RN8syIexkqS9h1fGSlLjDHpJalwzQe8tFx6U5NIkO5Lc0Nd2WJIrk/yoezx0lDWOSpLFSb6aZGOSG5Oc17Xv9/2T5DFJvp3ke13f/EPXflySb3V98+nuBIv9UpIDknw3yb930/tE3zQR9N5y4WE+Bpw6qW0VcFVVLQGu6qb3R7uAC6rqqcBJwLndz4r9A/cBz6+qZwAnAKcmOQl4N/D+rm/uAl43whpH7TxgY9/0PtE3TQQ93nLhIarq68Cdk5qXA2u652uAM/ZoUXuJqtpWVdd2z++l90t7NPYP1fOLbvKR3VcBzwf+rWvfL/sGIMkxwDLgI9102Ef6ppWgPxq4rW96S9emBx1ZVdugF3bAESOuZ+SSjAPPBL6F/QP8dmjiOmAHcCXwY+DuqtrVLbI//259APhb4IFu+onsI33TStDPessFqV+Sg4DLgfOr6p5R17O3qKr7q+oEeleznwg8darF9mxVo5fkxcCOqrqmv3mKRffKvlmoe93sad5yYXbbkyyqqm1JFtE7YtsvJXkkvZD/eFV9pmu2f/pU1d1Jvkbvc4xDkhzYHbnur79bzwVemuR04DHA4+kd4e8TfdPKEb23XJjdOmBF93wF8PkR1jIy3bjqR4GNVfW+vln7ff8kGUtySPf8scAL6H2G8VXg5d1i+2XfVNVbq+qYqhqnly//WVWvZh/pm2aujO3eaT/Ag7dcuGjEJY1Mkk8CJ9O7hep24G3A54C1wLHArcCZVTX5A9vmJfkj4BvA9Tw41nohvXH6/bp/kjyd3geKB9A7CFxbVe9I8rv0TnA4DPgu8Jqqum90lY5WkpOBt1TVi/eVvmkm6CVJU2tl6EaSNA2DXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXu/wGkA2ZFKkihlQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcdc13c8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAE6hJREFUeJzt3X+QnVd93/H3B//AFAfkH2vXI6kVBJXAtMW4iu3UmQy1IfUPipwZawJDguqKUaa1O2ZCmohMkwyddkZkJjFxpuOOBruROxQjTKhV8EAUYYZkWjussbExCrXwGGsrxVrjH0AcIIZv/7hn64u00t7V3tWujt6vmTvP85zn3Oeee3b3s2fPPs9zU1VIkvr1sqVugCRpcRn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvU5oSb479PhRkr8Z2n73Ao57X5JfmqPOv07yf9pr/VWS/5nkFSMc+8oke4+1bdJ8nbrUDZAWoqrOnFlP8gTw3qr608V+3ST/HPj3wJVV9UiSc4D1i/260rFwRK+uJTklyW8leTzJ00k+mmRF2/fKJHcmeSbJc0nuT3JWkt8Dfhr4SBut/94sh/5p4M+q6hGAqvpWVd1eVX/Tjv2KJB9Osq+N9v8wycvbL4RPAa8d+svjnOPTGzpZGfTq3b8Dfh74WWAV8LfAzW3fexn8VbsSOBe4EfhBVb0f+BKDvw7ObNuHug94R5LfTvIzSU4/ZP/N7fX+EfB64B8AW6rqW8AvAI+3Y5/ZyqRFY9Crd7/CIGD3V9X3gA8Cv5gkDEJ/AvjJqnqxqr5UVX89ykHb9NA7gUuAzwFPJ/lQkpclORX4V8BNVfVcVT0PbG31pePOOXp1q4X5auCeJMN373sZcA5wG/B3gbuSnAncAfxWVf1wlONX1U5gZ5KXAW8DPgF8DbgXOA14dNCEQXOAFxf8pqRj4Ihe3arBrVn/L3B5Va0YepxRVU9X1fer6rer6qeAnwM28NKoe+TbulbVj6rqc8AXgX8IHGAQ6j859JqvrqqZuXhvGavjyqBX7/4LsDXJaoAk5yX5F239rUne2Ebk32YQzjOj+aeA1x7poEmuS7IhyYoM/FPgMuC+qvpb4HbgD5Kc2/avTvK2oWOf1/6KkBadQa/e/S7wp8Dnk3wH+F/ARW3fSuBu4DvAV4F7gB1t383Ae5I8m+R3Zznus8C/Ab7B4JfE7cAHq+qTbf/7gP3AJPA88FngdW3fV4CdwDfb2T5nj+m9SrOKHzwiSX1zRC9JnTPoJalzBr0kdc6gl6TOLYsLps4999xas2bNUjdDkk4oDzzwwNNVNTFXvWUR9GvWrGFycnKpmyFJJ5Qk3xylnlM3ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUuWVxZexSWbPlM4eVPbH1miVoiSQtHkf0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1ruvz6Gc7Tx48V36hvP5AOrE4opekzs0Z9Elen+Shoce3k7wvydlJdiV5rC3PavWT5JYke5M8nOSixX8bkqQjmTPoq+rrVXVhVV0I/BPgBeBTwBZgd1WtBXa3bYCrgLXtsRm4dTEaLkkazXynbq4AvlFV3wTWA9tb+Xbg2ra+HrijBu4DViS5YCytlSTN23yD/p3Ax9r6+VV1AKAtz2vlK4F9Q8+ZamWSpCUwctAnOR14B/CJuarOUlazHG9zkskkk9PT06M2Q5I0T/MZ0V8FfLmqnmrbT81MybTlwVY+Baweet4qYP+hB6uqbVW1rqrWTUxMzL/lkqSRzOc8+nfx0rQNwE5gI7C1Le8eKr8xyZ3AJcDzM1M8OnF4rrx6dzJ9j48U9En+DvA24FeGircCO5JsAp4ENrTye4Crgb0MztC5fmytlSTN20hBX1UvAOccUvYtBmfhHFq3gBvG0jpJ0oJ5Zawkdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqXNcfPLJYTpQLLfzgFUngiF6SumfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM55Hv2Yee66pOXGEb0kdc6gl6TOGfSS1DmDXpI6N1LQJ1mR5K4kf5lkT5KfSXJ2kl1JHmvLs1rdJLklyd4kDye5aHHfgiTpaEYd0f8B8Nmq+ingTcAeYAuwu6rWArvbNsBVwNr22AzcOtYWS5LmZc6gT/Iq4OeA2wCq6gdV9RywHtjeqm0Hrm3r64E7auA+YEWSC8becknSSEYZ0b8WmAb+a5IHk3wkySuB86vqAEBbntfqrwT2DT1/qpX9mCSbk0wmmZyenl7Qm5AkHdkoQX8qcBFwa1W9GfhrXpqmmU1mKavDCqq2VdW6qlo3MTExUmMlSfM3StBPAVNVdX/bvotB8D81MyXTlgeH6q8eev4qYP94mitJmq85g76q/grYl+T1regK4GvATmBjK9sI3N3WdwLvaWffXAo8PzPFI0k6/ka9182/BT6a5HTgceB6Br8kdiTZBDwJbGh17wGuBvYCL7S6kqQlMlLQV9VDwLpZdl0xS90CblhguyRJY+KVsZLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnRgr6JE8keSTJQ0kmW9nZSXYleawtz2rlSXJLkr1JHk5y0WK+AUnS0c1nRP/PqurCqpr5kPAtwO6qWgvsbtsAVwFr22MzcOu4GitJmr+FTN2sB7a39e3AtUPld9TAfcCKJBcs4HUkSQswatAX8CdJHkiyuZWdX1UHANryvFa+Etg39NypVvZjkmxOMplkcnp6+thaL0ma06kj1rusqvYnOQ/YleQvj1I3s5TVYQVV24BtAOvWrTtsvyRpPEYa0VfV/rY8CHwKuBh4amZKpi0PtupTwOqhp68C9o+rwZKk+Zkz6JO8MslPzKwDPw98FdgJbGzVNgJ3t/WdwHva2TeXAs/PTPFIko6/UaZuzgc+lWSm/n+vqs8m+RKwI8km4ElgQ6t/D3A1sBd4Abh+7K2WJI1szqCvqseBN81S/i3gilnKC7hhLK3TnNZs+cys5U9sveY4t0QL4ddRi8krYyWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdG/WmZied2S5gWcyLV452wYwX00haCEf0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zvPoT2LH+1oBSUvDEb0kdW7koE9ySpIHk3y6bb8myf1JHkvy8SSnt/KXt+29bf+axWm6JGkU8xnR3wTsGdr+EHBzVa0FngU2tfJNwLNV9Trg5lZPkrRERgr6JKuAa4CPtO0AlwN3tSrbgWvb+vq2Tdt/RasvSVoCo47oPwz8OvCjtn0O8FxVvdi2p4CVbX0lsA+g7X++1f8xSTYnmUwyOT09fYzNlyTNZc6gT/J24GBVPTBcPEvVGmHfSwVV26pqXVWtm5iYGKmxkqT5G+X0ysuAdyS5GjgDeBWDEf6KJKe2UfsqYH+rPwWsBqaSnAq8Gnhm7C2XJI1kzqCvqg8AHwBI8hbg16rq3Uk+AVwH3AlsBO5uT9nZtv932//5qjpsRC+Ng/fq759f44VbyHn0vwH8apK9DObgb2vltwHntPJfBbYsrImSpIWY15WxVfUF4Att/XHg4lnqfA/YMIa2SZLGwCtjJalzBr0kdc6gl6TOGfSS1DmDXpI65/3otex5HrW0MI7oJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOjdn0Cc5I8lfJPlKkkeTfLCVvybJ/UkeS/LxJKe38pe37b1t/5rFfQuSpKMZZUT/feDyqnoTcCFwZZJLgQ8BN1fVWuBZYFOrvwl4tqpeB9zc6kmSlsicQV8D322bp7VHAZcDd7Xy7cC1bX1926btvyJJxtZiSdK8jDRHn+SUJA8BB4FdwDeA56rqxVZlCljZ1lcC+wDa/ueBc2Y55uYkk0kmp6enF/YuJElHNFLQV9UPq+pCYBVwMfCG2aq15Wyj9zqsoGpbVa2rqnUTExOjtleSNE/zOuumqp4DvgBcCqxIMvNRhKuA/W19ClgN0Pa/GnhmHI2VJM3fKGfdTCRZ0dZfAbwV2APcC1zXqm0E7m7rO9s2bf/nq+qwEb0k6fgY5cPBLwC2JzmFwS+GHVX16SRfA+5M8h+BB4HbWv3bgP+WZC+Dkfw7F6HdkqQRzRn0VfUw8OZZyh9nMF9/aPn3gA1jaZ0kacG8MlaSOmfQS1LnRpmjl0a2ZstnZi1/Yus1x7klWiyzfY39+i5vjuglqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknq3CgfDr46yb1J9iR5NMlNrfzsJLuSPNaWZ7XyJLklyd4kDye5aLHfhCTpyEb54JEXgfdX1ZeT/ATwQJJdwL8EdlfV1iRbgC3AbwBXAWvb4xLg1rbUSc4PJTm5+fVfOnOO6KvqQFV9ua1/B9gDrATWA9tbte3AtW19PXBHDdwHrEhywdhbLkkaybzm6JOsAd4M3A+cX1UHYPDLADivVVsJ7Bt62lQrkyQtgZGDPsmZwCeB91XVt49WdZaymuV4m5NMJpmcnp4etRmSpHkaKeiTnMYg5D9aVX/cip+amZJpy4OtfApYPfT0VcD+Q49ZVduqal1VrZuYmDjW9kuS5jDKWTcBbgP2VNXvD+3aCWxs6xuBu4fK39POvrkUeH5mikeSdPyNctbNZcAvA48keaiV/SawFdiRZBPwJLCh7bsHuBrYC7wAXD/WFkuS5mXOoK+qP2f2eXeAK2apX8ANC2yXJGlMRhnRS1pCnn+uhfIWCJLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc7z6I8jz4c+sl76Zjm9j4W0ZbbnnmhfC73EEb0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpc14wJc2TFxON37Fe3LWcLlBbzhzRS1Ln5gz6JLcnOZjkq0NlZyfZleSxtjyrlSfJLUn2Jnk4yUWL2XhJ0txGGdH/EXDlIWVbgN1VtRbY3bYBrgLWtsdm4NbxNFOSdKzmDPqq+iLwzCHF64HtbX07cO1Q+R01cB+wIskF42qsJGn+jnWO/vyqOgDQlue18pXAvqF6U63sMEk2J5lMMjk9PX2MzZAkzWXc/4zNLGU1W8Wq2lZV66pq3cTExJibIUmacaxB/9TMlExbHmzlU8DqoXqrgP3H3jxJ0kId63n0O4GNwNa2vHuo/MYkdwKXAM/PTPFIy83RzodfjHPll9s5314PcGSL8b2xlF//OYM+yceAtwDnJpkCfodBwO9Isgl4EtjQqt8DXA3sBV4Arl+ENkuS5mHOoK+qdx1h1xWz1C3ghoU2SpI0Pl4ZK0mdM+glqXMGvSR1zqCXpM4Z9JLUuRP+fvTL7dxkLR8nw/fGcnuPy6k9S9GW5fT+hzmil6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6tyiBH2SK5N8PcneJFsW4zUkSaMZe9AnOQX4z8BVwBuBdyV547hfR5I0msUY0V8M7K2qx6vqB8CdwPpFeB1J0ghSVeM9YHIdcGVVvbdt/zJwSVXdeEi9zcDmtvl64OtjePlzgafHcJwe2TdHZt8cmX1zdEvdP3+/qibmqrQYnzCVWcoO+21SVduAbWN94WSyqtaN85i9sG+OzL45Mvvm6E6U/lmMqZspYPXQ9ipg/yK8jiRpBIsR9F8C1iZ5TZLTgXcCOxfhdSRJIxj71E1VvZjkRuBzwCnA7VX16Lhf5wjGOhXUGfvmyOybI7Nvju6E6J+x/zNWkrS8eGWsJHXOoJekznUT9N524SVJbk9yMMlXh8rOTrIryWNtedZStnGpJFmd5N4ke5I8muSmVn7S90+SM5L8RZKvtL75YCt/TZL7W998vJ1kcVJKckqSB5N8um2fEH3TRdB724XD/BFw5SFlW4DdVbUW2N22T0YvAu+vqjcAlwI3tO8V+we+D1xeVW8CLgSuTHIp8CHg5tY3zwKblrCNS+0mYM/Q9gnRN10EPd524cdU1ReBZw4pXg9sb+vbgWuPa6OWiao6UFVfbuvfYfBDuxL7hxr4bts8rT0KuBy4q5WflH0DkGQVcA3wkbYdTpC+6SXoVwL7hranWplecn5VHYBB2AHnLXF7llySNcCbgfuxf4D/PzXxEHAQ2AV8A3iuql5sVU7mn60PA78O/Khtn8MJ0je9BP1It12QZiQ5E/gk8L6q+vZSt2e5qKofVtWFDK5ovxh4w2zVjm+rll6StwMHq+qB4eJZqi7LvlmMe90sBW+7MLenklxQVQeSXMBgxHZSSnIag5D/aFX9cSu2f4ZU1XNJvsDg/xgrkpzaRq4n68/WZcA7klwNnAG8isEI/4Tom15G9N52YW47gY1tfSNw9xK2Zcm0edXbgD1V9ftDu076/kkykWRFW38F8FYG/8O4F7iuVTsp+6aqPlBVq6pqDYN8+XxVvZsTpG+6uTK2/ab9MC/dduE/LXGTlkySjwFvYXAL1aeA3wH+B7AD+HvAk8CGqjr0H7bdS/KzwJ8Bj/DSXOtvMpinP6n7J8k/ZvAPxVMYDAJ3VNV/SPJaBic4nA08CPxSVX1/6Vq6tJK8Bfi1qnr7idI33QS9JGl2vUzdSJKOwKCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9Jnft/C0vtLWt2fLYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcc8b438>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Visualize Distribution of sign types\n",
    "unique_train, train_counts = np.unique(y_train, return_counts=True)\n",
    "\n",
    "plt.bar(unique_train, train_counts, align='center')\n",
    "plt.title(\"Training Set\")\n",
    "plt.show()\n",
    "\n",
    "unique_valid, valid_counts = np.unique(y_validation, return_counts=True)\n",
    "\n",
    "plt.bar(unique_valid, valid_counts, align='center')\n",
    "plt.title(\"Validation Set\")\n",
    "plt.show()\n",
    "\n",
    "unique_test, test_counts = np.unique(y_test, return_counts=True)\n",
    "\n",
    "plt.bar(unique_test, test_counts, align='center')\n",
    "plt.title(\"Test Set\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "## Step 2: Design and Test a Model Architecture\n",
    "\n",
    "Design and implement a deep learning model that learns to recognize traffic signs. Train and test your model on the [German Traffic Sign Dataset](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset).\n",
    "\n",
    "The LeNet-5 implementation shown in the [classroom](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) at the end of the CNN lesson is a solid starting point. You'll have to change the number of classes and possibly the preprocessing, but aside from that it's plug and play! \n",
    "\n",
    "With the LeNet-5 solution from the lecture, you should expect a validation set accuracy of about 0.89. To meet specifications, the validation set accuracy will need to be at least 0.93. It is possible to get an even higher accuracy, but 0.93 is the minimum for a successful project submission. \n",
    "\n",
    "There are various aspects to consider when thinking about this problem:\n",
    "\n",
    "- Neural network architecture (is the network over or underfitting?)\n",
    "- Play around preprocessing techniques (normalization, rgb to grayscale, etc)\n",
    "- Number of examples per label (some have more than others).\n",
    "- Generate fake data.\n",
    "\n",
    "Here is an example of a [published baseline model on this problem](http://yann.lecun.com/exdb/publis/pdf/sermanet-ijcnn-11.pdf). It's not required to be familiar with the approach used in the paper but, it's good practice to try to read papers like these."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-process the Data Set (normalization, grayscale, etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Minimally, the image data should be normalized so that the data has mean zero and equal variance. For image data, `(pixel - 128)/ 128` is a quick way to approximately normalize the data and can be used in this project. \n",
    "\n",
    "Other pre-processing steps are optional. You can try different techniques to see if it improves performance. \n",
    "\n",
    "Use the code cell (or multiple code cells, if necessary) to implement the first step of your project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Preprocess the data by augmenting data set to compensate for classes with few example images\n",
    "\n",
    "def norm(imgs):\n",
    "    img_array = np.asarray(imgs)\n",
    "    norm = (img_array - img_array.min()) / (img_array.max() - img_array.min())\n",
    "    \n",
    "    return norm\n",
    "\n",
    "def augmentBrightness(img):\n",
    "    hsv_img = cv2.cvtColor(img, cv2.COLOR_RGB2HSV)\n",
    "    rand_bright = np.random.uniform() + 0.25\n",
    "    hsv_img[:,:,2] = hsv_img[:,:,2] * rand_bright\n",
    "    rgb_img = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)\n",
    "    \n",
    "    return rgb_img\n",
    "\n",
    "def transformImage(img, angle_range, shear_range, trans_range):\n",
    "    angle_rot = np.random.uniform(angle_range) - angle_range/2\n",
    "    rows, cols, ch = img.shape\n",
    "    rot_matrix = cv2.getRotationMatrix2D((cols/2, rows/2), angle_rot, 1)\n",
    "    \n",
    "    x = trans_range * np.random.uniform() - trans_range/2\n",
    "    y = trans_range * np.random.uniform() - trans_range/2\n",
    "    trans_matrix = np.float32([[1, 0, x], [0, 1, y]])\n",
    "    \n",
    "    pts = np.float32([[5, 5], [20, 5], [5, 20]])\n",
    "    pt1 = 5 + shear_range * np.random.uniform() - shear_range/2\n",
    "    pt2 = 20 + shear_range * np.random.uniform() - shear_range/2\n",
    "    \n",
    "    pts2 = np.float32([[pt1, 5], [pt2, pt1], [5, pt2]])\n",
    "    \n",
    "    shear_matrix = cv2.getAffineTransform(pts, pts2)\n",
    "    \n",
    "    transform_img = cv2.warpAffine(img, rot_matrix, (cols, rows))\n",
    "    transform_img = cv2.warpAffine(transform_img, trans_matrix, (cols, rows))\n",
    "    transform_img = cv2.warpAffine(transform_img, shear_matrix, (cols, rows))\n",
    "    \n",
    "    transform_img = augmentBrightness(transform_img)\n",
    "    \n",
    "    return transform_img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f01bcb5f5c0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAE/CAYAAAAnhFRiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAH35JREFUeJzt3X2QZFd53/Hvc/tldnZmJe1KWnmRZEtgxQVO2Qu1pVAl4ijIYBknJagCSqqCiCpcSyirYqpwEoV/RByc4FSEbKccHGGprCQYieIlKImSoFA4iuNEQRChF4RjUIS10rKLvFrtanZeuvs++aPvxqNl5jxnpnu6Z45/n6qtne1759zTt3uevd39m+eYuyMiUopq2hMQERknFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKjJjmJmT5rZtZn7PmNmP7vFU5JtRkVNfsi5xcDMbjSzF83sr01zXgDu/pPu/gejjmNm15rZkTFMSbYZFTVJMrObgd8GfsHd/+sU59Ge1rFlZ1FRk3WZ2WHgduDn3P2PVt3+RjP7IzM7aWbfXP1y0MzON7O7zOyomT1nZh8zs1az7X1m9t/N7J+b2Utm9m0zuy5x/GfM7O+b2WPAgpm1V19Fmtmsmd3TXEU+ZWZ/b42rr4Nm9lhzvPvMbJeZzQH/EXiVmb3c/HnV2E6cTJWKmqzng8A/Aq5z90fO3mhmlwL/AfgYsA/4FeDzZnZxs8s9QB/4ceD1wFuBX1w17l8BngYuAm4DvmBm+xLzuAn4BeACd++fs+024Arg1cBbgPes8f3vBq4HrgR+Cnifuy8APw887+7zzZ/nE3OQHURFTdbzFuB/Ao+fc/t7gAfc/QF3r939QeAR4G1mdgnDYvEhd19w9+PAHcCNq77/OPAb7t5z9/uAP2ZYtNbzW+7+rLsvrrHt3cA/dvcX3f0I8FvrfP/z7n4C+HfAwfCey46moibr+dvAXwJ+18xs1e0/Bryreel50sxOAm8CDjTbOsDRVdv+JbB/1fc/56/sovA9IPXS79nEtleds32tfb+/6uszwHxiPCmAipqs5zhwHfBXgX+x6vZngX/t7hes+jPn7h9vti0DF63adp67/+Sq77/0nCL5o0DqpV+qjcxR4LJV/748437ljCs7mIqarKt5n+nNwPVmdkdz878B/qaZ/ZyZtZo33q81s8vc/SjwZeB2MzvPzCoze805UZD9wN8xs46ZvQt4LfDAJqf4WeAfmNne5r2+WzbwvceAC83s/E0eW7YpFTVJcvdnGRa2d5rZP2n+fQPwEeAHDK/O/i5//lz6W0AX+BbwIvA5hi9Nz3oYuAp4Afg14J3u/mebnN6vAkeA/wv8l+ZYy5n369vAZ4Cnm5fK+vSzEKYmkTIpZvY+4Bfd/U1bNP4HgRvdfeohYZkeXanJjmVmB8zsmuZl7k8AHwa+OO15yXQppS07WZfhp6tXAieBe3nlhxryF5BefopIUfTyU0SKoqImIkUZ6T01M7se+E2gBfxuE8BcV1VVXlUj1tExvFr2cBALtufsMSHBRF6Zc93UEFljROc0622OsbwTEp6Q0Q+Rc1+C4+Sc03GwavTjhI/dJB43oNdbfsHdL47223RRazov/DbD3xE8AnzNzO5392+t9z1VVbFnfv2sY87pt+AE1xlPuEFdR0cJx4iKs2U80tETO2uMViu5vdVObwdoB/el047/I6rrQXL7Sr+XMUZUGHMKdHA+WqN/NhY/fwALzmmnGw4R/f9vFj8/OjPBcYJ5AqyspB+7rBof/ExF2wGeffa734uPNNrLz6uB77j70+6+wvCTpxtGGE9EZGSjFLVLeeUvEB9pbhMRmZpRrsXXul78oQvRptHg4eHX+lxCRLbWKFXmCK/sinAZa3RbcPc73f2Qux+qxvCmpYhIyihF7WvAVWZ2pZl1GTYCvH880xIR2ZxNv/x0976Z3QL8Z4aRjrvd/cmxzUxEZBNG+nzb3R9go72wkq9AM16eBjEIH6SjBRDHanKiFHjwsX5Otiv6LDzjdFRjyENFbwvkvW0QjJExjzgoET8ubulR6uhxI47rDILoCUC7HTwuVRy1qYK4TtXKeGzDCEs8RjCNsWQQx5nb0zv3IlIUFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKFNYeCURssv4hXcPwpM5/erCfUbPmuaJgsQZ96YfhI1zeghYJ71T1v98Y+nNGD22OTNJ75MzRq8fBHgz2qlF88gJI0cNHnMCvDaG/nGtsM9dRig67IU3vrVSdKUmIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESmKipqIFGXCOTVLLr4yjvUDcwJR41hvMz5GvM84kjlR3smD9TiHO0ULTMZD1EF4K6c5Y5zLy1lLNVhvszsTjtFud5LbFxeXwzFaQcYs57G3aG3ZjIXBqyiomPHYDoLnWJWRl/M6+pnLCv9l0ZWaiBRFRU1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpyuSbRKaCfJ6xunrUoS8nwRuaTJfIKOSbs2p1Kwg+doKVwgEqS49RZ5zTOnrsMk5X2PTQ45BnK2iKmBO+bbXTY8xkrNDuQfPOaJ4Qn4+sppljCN8SBIlzOpFGoflxXl3pSk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpymRzagZVK9Ek0jIyZlFjvIwMUeZqtElRhizrvsRHCfeog336OadjkD4frYyWhrPt9OOyd9eucIyBp5sznlqOH7eVoOFlq9MNx2h30vOIzhdAP3wexpm7yoIfzyg/Rpx1y1nsulVHZWL0XGc1ltXBh0Yqamb2DHAaGAB9dz80jkmJiGzWOK7U/rq7vzCGcURERqb31ESkKKMWNQe+bGZfN7PD45iQiMgoRn35eY27P29m+4EHzezb7v7Q6h2aYncYoMpY/UZEZBQjVRl3f775+zjwReDqNfa5090PufuhnCW9RERGsekqY2ZzZrbn7NfAW4EnxjUxEZHNGOXl5yXAF5u8Vhv4fXf/T2OZlYjIJm26qLn708BPb+R7zCpanUQIM6NJZHhxmRM2DZseZqwEHoyR0+AxOo5nNGccBCHPOidc2UrPtZXOogIwY+nH7qKZOHw7N78/uf2FMAQKz710Krm96sZ3JnrvN+dtlGi1+SorOBusap7zdk4UEs9bKz44REaTyGiuphXaRUTWpKImIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESnKRJtEGkYr0fjOgkV1h/uk63Cd0SSyCvYZ1L1wjChC1qoyTm2QIcrKugUTGRDfl6qTnmt3V3xfzj9/Nrl9/4UHwjHanb3J7S+fWgrH2OPpuVYZzTut7ie396rFcIwocelVxjyCueZk3Vphhmz0xqxZObXg+qlqxT/7uXSlJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRFRU1EijLR8G1d1ywtLCT2yGlYFzVWzGi+FwRWnZyGdcHK6LaSMUTwf0oVryZeB6t4z19wUTjGhXvTodecwOqip8/Zd0/E53TgLye3L6UzsQDUpJtRdjJCr23SB6p3x9cCvWAFdqvi8xGFr1sZ4eyg/2eWMFybMY+8pqnjoSs1ESmKipqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCgTzakNl05dv32e13F2x4M8VF5mJtwjHCM+SEYTwCrIMrXinNru+YuT28+78JJwjEGQdVtcXA7HWO6lm1H2BzkLM6dbK+YsvNsO8l/9bsZj208fp9PdHQ4xvyd9Tn2QzuQN9wmCeVk9REdfuDvaJ+unJfq5zZlGJl2piUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkVRURORooThWzO7G/gbwHF3/8vNbfuA+4ArgGeAd7v7i/HhDILAaThC0AQyqxldsEveGOEg4RCdmXRDw/POTwdrAbp70uHa5Sp9DICFpXQwsteL70uQvWWQEb4dRwCzFzT4XO5lhKKDiXT70frrsGcu/aO1q3teOMagn16RfjAITjpQh9nbjFB0Px0CzsiZZ6zyPr4mkjlXar8HXH/ObbcCX3H3q4CvNP8WEZm6sKi5+0PAiXNuvgG4p/n6HuDtY56XiMimbPY9tUvc/ShA8/f+8U1JRGTztvwX2s3sMHAY4l/gFhEZ1Wav1I6Z2QGA5u/j6+3o7ne6+yF3P1RFq9KIiIxos1XmfuDm5uubgS+NZzoiIqMJi5qZfQb4H8BPmNkRM3s/8HHgLWb2J8Bbmn+LiExd+J6au9+0zqbrNnw0G8P7ahm5mlHVGYGpOsjVtGdmwjHO35deaPiii38kHONML30+T78cL6rcac0lt+/eE9+XxeXTye0vJxexHuoFGbIq44WFe3qfQdSsEMKMYUYvU3whanjZCcfotoJmlB6fU/d0li0nY1YP0nd4MIhze1Ur/TyNtm+E3uQSkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFmfAK7VBV6wcb66ABJOSsOB2LxujnHGNXuvninv2XhUPsvfhVye3Ly/E8XjqZDmD6ID6n51+4N7l9thMHIxfa6UaC/Wi1caA/SAeFPQiBDncKzlnGcywaI+OUshgFic/E96U9lz7vOTniqEtku50RaO50g0PEAe+qHZSaMf5euK7URKQoKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFUVETkaJMPKfmiXBNTgbNg8VqsxbEDY4TLWYL0K3Sp27PzAXxGO19ye3Hj30/HGMhyDtVFjfw6628lNw+Pxvfl9nZdG5vPmMB4CjKtrwU56E8Wr03a53q9E45z9NBMI+VXhwy69Xp51g7eA4C1HX6nA0ysn8WLETc6aZzbEC4iLlPeDFjEZEdQ0VNRIqioiYiRVFRE5GiqKiJSFFU1ESkKCpqIlIUFTURKcpkw7eebmyX05vRgyZ/ltM5L9inkxEE3BOEK+d6cdiUpeXk5jOL6e0Agzo9134UVgZeePFkcru14yaRe+bmk9vPnwtWGwc8CKS+lLESeK8fNGcMQqAAu3alV6RfPHMmHIN+sKp5FBIGVoLActXJaPAYPJd7QTgXoLL0OWtlrK5uUfg2HCGfrtREpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkVRURORoky8SWSdiu9kLDRrQZe/qMEfZCyanDGPKJlTDeKM2fLSqeT2fj8np9ZJbveMrohnltKZqhf+LJ1jA7AgD7dndzrHBnDBnvQ+vYyc2sJiOkPW7abPF8CFweLOx/vxwszLdXquOS0Re8Hj367ieXiwQPSgtxSOEZ2zqpXO9UH8/Aibe25AeKVmZneb2XEze2LVbR81s+fM7NHmz9vGNiMRkRHkvPz8PeD6NW6/w90PNn8eGO+0REQ2Jyxq7v4QcGICcxERGdkoHxTcYmaPNS9P029CiIhMyGaL2ieB1wAHgaPA7evtaGaHzewRM3ukzvllcxGREWyqqLn7MXcf+HC9u08BVyf2vdPdD7n7ocqUIBGRrbWpKmNmB1b98x3AE+vtKyIySWFOzcw+A1wLXGRmR4DbgGvN7CDDNkjPAB/YwjmKiGQLi5q737TGzXdt5mBOFLKLA3hxtjYn1pjeJ+edv+Wgo+XLvTg4Wy8tpLcHAc6h4CHMCBK7py/YFxfjRoInTqQDulXGPObmzktuv+D8OMDrlg6ktoLVxgHmd6dXmz81E2fWl4NAc05H1HrQC7bnrHqfPh+9XvzYmqXvS06TyPBnbpLhWxGRnURFTUSKoqImIkVRURORoqioiUhRVNREpCgqaiJSlAkvZux48vc/x5BVyWgSSfDrWjmzWB6kszsLK+mMEUDVCha8zZlIkHfKSe2FGSKPc0gLUS7r5OlwjEGQZZufT+fHADrtPcntK8txfrBdpfNfnYyfGgvycmbxg9tupc9H+mdpKMp/eUZebhDk4eqMJ2qV90QcC12piUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSKoqImIkVRURORokw2fGt5ocNRZGQJw1Xcc5pE9oMD9TP+v+h0uukd2vFMBivpeXRy1oUIzocFIVAA66SPs5IRij61nG6aOdONz8feubnk9sFsvEK7B00R60HGyujBsygnI97ppEPP3oubRNZBs9JWHZ/TFul55FwZVcEdzvm5zaUrNREpioqaiBRFRU1EiqKiJiJFUVETkaKoqIlIUVTURKQoE82pVVYx051Zd7sT526iPEuUQWv2Sh8jYwxrr38/hmPE/194cHdnoxwbsNhLN6PMaQIY3d9WRk5tNsh/7c5o8DjbSc+128porBhkzLqdOKcWNbzs9XJyalEj0jgfVgX3ZTBYDMcwT+fUovzYcIz0ec9ZdDtaEHmc8VVdqYlIUVTURKQoKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFUVETkaKE4Vszuxz4V8CPMOyfeKe7/6aZ7QPuA64AngHe7e4vpsaqKmN+dnbd7VFzPshp4BjX6SigmxOcrat0iNOiZC1gS+mmiPPE4VuCYOxSxurZUUA3pymiDdLz2OVxznt3lb6/0YrlAEv9ILC6HAdWV4K7u5LRnDF6XGa78WPbIR2cHQzi1ebxYKX4VjyPKJzdzwjfdoLnWF5oPk/OlVof+LC7vxZ4I/BLZvY64FbgK+5+FfCV5t8iIlMVFjV3P+ru32i+Pg08BVwK3ADc0+x2D/D2rZqkiEiuDb2nZmZXAK8HHgYucfejMCx8wP5xT05EZKOyf6HdzOaBzwMfcvdTua+BzewwcBig1ZrsOi8i8hdP1pWamXUYFrRPu/sXmpuPmdmBZvsB4Pha3+vud7r7IXc/1GqlV6URERlVWNRseEl2F/CUu39i1ab7gZubr28GvjT+6YmIbEzO68FrgPcCj5vZo81tHwE+DnzWzN4P/Cnwrq2ZoohIvrCoufsfsn5Xxes2dDSHQaLBnrUzGitG23O6zQWZGQ+PAnWd3qfVirM7LUs3eKxIL8wL0LV088VBxiv+XrCgbZ3RaPLMwlJ6h6U4U7XQDhbN7cT/B1fRWxwZ92UwSD92npHL2hUs7jzTirN/g+Uz6e05eTlPzyPvrfH086MfNCoF6HfS97eqxvd7APqNAhEpioqaiBRFRU1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUpSJ/oZ5XdecWVk/hFmv5IwSNJsLVl8fjhA0RczK7wYrTmesJt6fSa/yPjsb35dO0CQgJ4u8FCQwlzLGiAK8Cxk50aWVoAVoEFYG8OCU5QSrK0tPdnc3vhaYCebaX3gpHKPupRtaRiunD3dKn5A6eNwAquA4g3gIFtP9UGm3FL4VEVmTipqIFEVFTUSKoqImIkVRURORoqioiUhRVNREpCiTzanhrPRTzeIycjcByxoj2mf0hVWzGisO0o0V+/WJcIy52XQeqmMZi9W203m5qpXeDpB8WAHvxWEmD857TgbRqvRxcuJQ7U46p9YlCF0B/YXTye3LSy+HY1gQMsxa8yPIUwb9MIeCBo51ovHrWb2l9HM95+cll67URKQoKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFUVETkaKoqIlIUSYavsXBkyG7OIAXrSidHj/3ODljREHQjGaVwVyXl06FY9T9dIhzZmZPOEY32Ge2kxFG3pV+Knm3Ew5hFjS8zDingzpaCT5uNFlZOii6ePp4OMbKYnqMnHh3FYWRM56mURPIQU5H1HD19HgMt/Q8cqaRS1dqIlIUFTURKYqKmogURUVNRIqioiYiRVFRE5GiqKiJSFEmm1MjvQiwRSE04mZyWe0dx9EjMszLZQwR3d86nkgvWAB4UJ8Jx1hZSme7Zs/EebmZ2XQjybn5feEYF+y9NLm9asVZtx+88L3k9oXTx8IxFpfT56zXi7JwcQYx57kePU9zFjOO5lHXcZfIaIHoVjsuIx4dJid0lym8UjOzy83sq2b2lJk9aWa/3Nz+UTN7zswebf68bWyzEhHZpJwrtT7wYXf/hpntAb5uZg822+5w93+2ddMTEdmYsKi5+1HgaPP1aTN7Cki/ThARmZINfVBgZlcArwcebm66xcweM7O7zWzvmOcmIrJh2UXNzOaBzwMfcvdTwCeB1wAHGV7J3b7O9x02s0fM7JHUhwQiIuOQVdTMrMOwoH3a3b8A4O7H3H3gw0r1KeDqtb7X3e9090PufshMCRIR2Vo5n34acBfwlLt/YtXtB1bt9g7gifFPT0RkY3I+/bwGeC/wuJk92tz2EeAmMzvIME3zDPCBLZmhiMgG5Hz6+YesHTV9YDMHTOb4xrJK8xjGGEOfyWh1bSCjM15GuDJIAfdXVuJpBNt7K/Fq4rP99EX/rnYcNr3wvB9Pbp+ZixteHvv+k8ntKysvhWMMgsaKVdg0MSN8m/P+cjCG1znzSG/PuS/WTq8EbzljBBOpwmdhPr3JJSJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBRlwk0iPbm4albjvCC7FTWRbA6U3pwxi3BR5XGszprTBDCcbTxGHeyT87AsL6W7AA6W47zc3vlucvt5F8Y5tcerqClixvkIuiJGiwxDzlLXGdmuaIesp1jwXK9yGrMG0xjEjSZ7vfTjP85fodSVmogURUVNRIqioiYiRVFRE5GiqKiJSFFU1ESkKCpqIlIUFTURKcrEV2hPBTmjxnrNXpsefyP7hLMI5zr66tmZ682nR8i6s6MHeKMxckKvgzpY+dx64RjtoBllu50O+ALUK/30NDKColGo1YNGlBBnrz0nwBvMo9XKuK4JxhgM4selChpNjrFHpK7URKQsKmoiUhQVNREpioqaiBRFRU1EiqKiJiJFUVETkaJMPKeWyjzl5cdGz6lNYtHkvMzdKEcYinJoeQmzqONlTrPKtH6dzn4BDAZRI8G4GWGn00lu73bjnFqUZfOgieRwnyhkNnqOMWc9ZIKFhlutID8G4aWPVfEYdZ1+7MLn4AboSk1EiqKiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBQlDN+a2S7gIWCm2f9z7n6bmV0J3AvsA74BvNfdk+lJd082lBtHk8gcUWA1b534cTRWDMKVGWNU0X3JaGgY9W+0jHksB+fjhMfh22iffcHq6wBzu2aS223v3nCMlV461Vp7xjkNzlmd0SSSYJ96EJ+PXrB6ep3xMxc2msxovBn9bI8vept3pbYMvNndfxo4CFxvZm8Efh24w92vAl4E3j/GeYmIbEpY1Hzo5eafneaPA28GPtfcfg/w9i2ZoYjIBmS9p2ZmLTN7FDgOPAh8Fzjp/v9fLxwBLt2aKYqI5Msqau4+cPeDwGXA1cBr19ptre81s8Nm9oiZPTKO98NERFI29Omnu58E/gB4I3CBmZ39oOEy4Pl1vudOdz/k7ofG+3agiMgPC4uamV1sZhc0X88CPws8BXwVeGez283Al7ZqkiIiuXL6qR0A7jGzFsMi+Fl3//dm9i3gXjP7GPC/gbu2cJ4iIlnCoubujwGvX+P2pxm+v7YBRvriMG4COBEWN71rBc33qmA7gAX71BldAKvofcqMPFSUU/OMefSDeXQvujgco3Px/vQxMjJ3u7vpnNrc7K5wjCCmxkpGPmwlyodlPC7tIIOY0+AxOko/mCfAIMy6xWPE91dNIkVE1qSiJiJFUVETkaKoqIlIUVTURKQoKmoiUhQVNREpioqaiBTFxrGSePbBzH4AfG/VTRcBL0xsAqPZKXPdKfOEnTPXnTJP2Dlz3cw8f8zdwxT3RIvaDx3c7JHhL7pvfztlrjtlnrBz5rpT5gk7Z65bOU+9/BSRoqioiUhRpl3U7pzy8Tdip8x1p8wTds5cd8o8YefMdcvmOdX31ERExm3aV2oiImM1taJmZteb2R+b2XfM7NZpzSNiZs+Y2eNm9uhwnYXtw8zuNrPjZvbEqtv2mdmDZvYnzd/xQpdbbJ15ftTMnmvO66Nm9rZpzvEsM7vczL5qZk+Z2ZNm9svN7dvqvCbmua3Oq5ntMrP/ZWbfbOb5D5vbrzSzh5vzeZ+ZxYuH5nL3if8BWgxXpHo10AW+CbxuGnPJmOszwEXTnsc6c/sZ4A3AE6tu+6fArc3XtwK/vk3n+VHgV6Y9tzXmegB4Q/P1HuD/AK/bbuc1Mc9tdV4Zdn+cb77uAA8zXOPks8CNze2/A3xwXMec1pXa1cB33P1pH67qfi9ww5TmsmO5+0PAiXNuvoHhOqywTdZjXWee25K7H3X3bzRfn2a4HselbLPzmpjntuJDE103eFpF7VLg2VX/3s7rhjrwZTP7upkdnvZkMlzi7kdh+MQH0j2yp+sWM3useXk69ZfJ5zKzKxi2sn+YbXxez5knbLPzOul1g6dV1NZqSL5dP4a9xt3fAPw88Etm9jPTnlAhPgm8BjgIHAVun+50XsnM5oHPAx9y91PTns961pjntjuvPsK6wZsxraJ2BLh81b/XXTd02tz9+ebv48AX2fBiMxN3zMwOADR/H5/yfNbk7seaJ3sNfIptdF7NrMOwUHza3b/Q3Lztzuta89zO59U3sW7wZkyrqH0NuKr5BKQL3AjcP6W5rMvM5sxsz9mvgbcCT6S/a+ruZ7gOK2zj9VjPFojGO9gm59XMjOFyj0+5+ydWbdpW53W9eW638zqNdYOnFr5tPmr+DYafhN7t7r82lYkkmNmrGV6dwXA5wd/fTvM0s88A1zLseHAMuA34tww/WfpR4E+Bd7n7VN+kX2ee1zJ8ieQMP2H+wNn3rKbJzN4E/Dfgcf58hbmPMHy/atuc18Q8b2IbnVcz+ymGHwSsXjf4V5ufrXuBfQzXDX6Puy+P5ZjTKmoiIltBv1EgIkVRURORoqioiUhRVNREpCgqaiJSFBU1ESmKipqIFEVFTUSK8v8AP1aUG738YVwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcd4ceb8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAE/CAYAAAAnhFRiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGbdJREFUeJzt3X2sZHV9x/H3Z+7drUaoiopdEV1RajVGV7MiDY21tlpKbZFErUQtJtolRhJt7AOxsdCWVm0rVhOjrkqljQoqWvBZ6kPQPiBgV1hdn7MisrIiIIs87b3z7R/n3GRY7z3f371z7szcH59XcrNz55w55ztn5n72zMx3fj9FBGZmtRhMuwAzsz451MysKg41M6uKQ83MquJQM7OqONTMrCoONRuLpHlJIWnrhPb3ZUkvncS+bGNyqG1Qkm4b+RlKumPk9xdNu77lSHqMpHVrjJR0jqT3rtf2bWOYn3YBtjYRcdjSZUl7gZdHxH+utL6k+YhYmERtZtPkM7VKtWctF0r6gKQDwIsl/bqk/5V0i6R9kt4qaVO7/tLLyNMlfVfSzZLeOrK9X5V0maSfSbpR0vtX2O8fStol6YCkayW9bmTxZe06S2eUT21/f7mkb7b7/JSko0e2d6Kkb7X7fQugwvu/dH9eIel7bT1nSTq2PQa3tsdm6f4/SNInJf2kreNjko4a2d6j25e+ByR9VtLbR88KJZ0wcmx3SXp6SZ22DiLCPxv8B9gL/M4h150D3A38Ac1/XvcFngo8jeYM/Rjg28AZ7frzQAAXA/cHtgI3LW0X+BDwl+227gOccMjttra/PxN4Qrvek4Abgee0yx7TPOXuUefzgG8Bj223dTbwpXbZkcBtwCnAJuDPgQXgpSsch3OA9x5S10eAw4Entsfj0va+PRD4JvCidv2HtPu5L/DL7e0+PLLtrwBvBDYDTwcOjOzraOCnwO+29/vE9n4/aNrPjXvjj8/U6vbliPhYRAwj4o6IuCIiLo+IhYj4PrAT+M1DbvP6iPhZROwFvghsa68/SBMGWyLizoj4r+V2GBGfj4jd7T6/BlywzD5GnQ78Q0R8K5qXx+cAx7VnSc8BdkXERyPiIPAm4CerPAZvjIgDEXE1sAf4dETsjYibgc8AT27r/km7nzsi4lbgH5bqlnQMTUCfHRF3R8RlwCdG9vHHwCUR8Zn2fn8a+BpNuNmEOdTq9sPRXyT9mqRPSPqxpFuBvwUefMhtfjxy+XZg6b2719CcLV0p6RpJpy23w/Yl7hfbl3E/A16+zD5GPRJ4W/uy7RaaM5wh8HDgYaP3ISKGwHXdd/kX3DBy+Y5lfj+srft+kt7dvmS+Ffj8SN0PA34aEXeM3Hb02D4SOHXpPrT34/j2djZhDrW6HfpJ4zuB3cBjIuKXgb+m8D2qiNgXES+PiC3AK4Gdkh61zKoXABcBR0fE/YF3j+xjuU8+fwi8LCIeMPJz34i4HNhH89IOAEkDmrBbD38BPAo4rj02zxxZtg94kKT7jFx39MjlHwL/esh9uF9E/NM61WodHGr3LocDPwN+LulxNC/9ikh6wcgb57fQBNTiCvu4KSLulHQ88MKRZfuBaF/OLXkH8FdtPUh6gKTntcs+DmyTdLKkeeBPad77Wg+H05yZ3izpQTSBD0BEfA+4BjhL0mZJvwH8/sht/x04RdKzJM1Juo+k35LkM7UpcKjdu7wGOI3mTe53Aheu4rZPA66Q9HOaN9FfGRHXLrPeK4DXt5+4vhb44NKCiDgAvB64vH2Ztj0iPgScC3yofdl3Nc0b7kTEDcAfAf9E80b8I4DLV1HzapxL8wHJT4H/Bj51yPJTaT4g+ClwFs2xu6utcy/Nhwyvo3nP71qaY+2/rylQhAeJNFstSRfRfIjxd9Ouxe7J/5OYFZB0nKRHSRpIOonmk9mLp12X/SJ/o8CszMNoPgA5guYT2D9p20Rsxvjlp5lVxS8/zawqDjUzq8pY76lJOhF4CzAHvDsi3pCs79e6I6SivldblR6OabaJHp7F/Tz0G+f503wZpGt50UG9MSLSPsU1h5qkOeBtwLNo3ji9QtIlEfGNtW6zJoNBfhI8NzfXuVwFT9rBoHudPv54Sp5w2TplOZDdl/yYZusU/UeSrVN0PLqXlzw/sjpK7ku2Tsk2Inn0Sp5id95xe+fygwfvLtgKPyhZaZyXn8cB342I70fE3TRfjzl5jO2ZmY1tnFA7int+qfe69jozs6kZ5z215c46f+E8VdIOYMcY+zEzKzZOqF3HPUcqeDhw/aErRcROmnG7/EGBma27cV5+XgEc2351ZDPNaAyX9FOWmdnarPlMLSIWJJ1BM3roHHBeRHy9t8rMzNZgol+TmsTLzz4+5h4UtA5Moo6ybXQvz1o+gLT/YFjSwpB87F/2NEsel0F3CwzkLR2DpI0G8lqznqu+ZM9DFTy2fTzHsn6c7LEHWFzonsjs9ttvy+uAqyJie7aSv1FgZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWFYeamVVl4hOvdDX7FTWbpmNujd+QOClp43NBr3J2PMqaXpPxsgoO1yD5/7GsUbS7MTZKmqLThtV8G9kaw2HBGGR9NLWXNE6Pa0K999nYgfPzm9JtLCwcLNqXz9TMrCoONTOrikPNzKriUDOzqjjUzKwqDjUzq4pDzcyqMvE+tbmOgf5KWmb6mGd2NrrUChTcmXSVkmkus8EZC7axKRnAcVNBH5LmutdZLPg/eGE4/vyjJTNh5mtk6xT0IPbxRO1hPta0jJLnabLOpk2b0224T83M7pUcamZWFYeamVXFoWZmVXGomVlVHGpmVhWHmplVxaFmZlWZcPOtOjsKS3oNs8H3NkxjLaTdlWWTmmf3uGBwxqS7dlAwsOL85u7GWSlvvl2M7v0sFByQxcgGzRy/UzSSfTSSdYoG3kw2UVRGdl8KZptPj1nJAKDdy0sGiSzlMzUzq4pDzcyq4lAzs6o41MysKg41M6uKQ83MquJQM7OqTHyQyHH1MRFxH/O3ZnWU1dlHV13WD5XvYzDofhoomYgW4OCwe51k7EagYIDHgv6wYdpT1cfIm+MreeSzOZMHRRNEZ3Xk5zXBYrpGWke6hf4O+lihJmkvcABYBBYiYnsfRZmZrVUfZ2q/FRE39rAdM7Ox+T01M6vKuKEWwGclXSVpRx8FmZmNY9yXnydExPWSjgQulfTNiLhsdIU27Bx4ZjYRY52pRcT17b/7gY8Cxy2zzs6I2N58iLChxtAwsw1ozaEm6X6SDl+6DDwb2N1XYWZmazHOy8+HAh9t+7HmgfdHxKd7qcrMbI3WHGoR8X3gSau5jdRP82yyl7FXyWYsL6qih+bbskEiu0+2B8ms55DPjD4sOKFfXOyudljQfZv1zfbSoFnQwNvHftJm06Lu26wZOd9E1qBb0L9Lem8KChlG1sDbH7d0mFlVHGpmVhWHmplVxaFmZlVxqJlZVRxqZlYVh5qZVWXig0SqY2Lckv6wXoaSSzcy/l6Gw4JJYpMeMykfnDHrQ9Pc5nQbkeyn5K6kvX+D8SfNLZuHOJnsuqSNMZ27d/zBGUvkfXu5bNDMkp68QbpOSZ9admf6GyTSZ2pmVhWHmplVxaFmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUm33zb2aVZ0LHYwwzcWYNmiXQbJTOjJ+vMzecPz9x8d/NtJA2+jaSOuXwbc+n9zY/5YtLlmy2HfDDKosc+fejyx3Z+Pm+czgwXuwdWHEZ+PPL7W/D3Mv5DO5GG9yU+UzOzqjjUzKwqDjUzq4pDzcyq4lAzs6o41MysKg41M6vKxPvUigZP7NBHj1kPcxWjZBZYFfSHDZJepvlN+cOT7WexaBLh7sekpNWta/BPKJvcObsvJf1y2fOrZFLlrB+uZADIfBLhfCPDZJWDBwv61HoY4DGXb0PJOr38Xbd8pmZmVXGomVlVHGpmVhWHmplVxaFmZlVxqJlZVRxqZlYVh5qZVSXt7pR0HvAcYH9EPKG97gjgQmArsBd4QUTcnG0roocmux4GrCuZCT4TyTayZlSAufnu2dOVzL4OEEmHZtnAit2DESpZ3mxj/ObbrLm2j20U9O8yF9lKBbOaJ7VmzduQP08X5/JtJONM9qLkrylrAu5jRvslJWdq7wVOPOS6M4HPRcSxwOfa383Mpi4NtYi4DLjpkKtPBs5vL58PPLfnuszM1mSt76k9NCL2AbT/HtlfSWZma7fuX2iXtAPYsd77MTODtZ+p3SBpC0D77/6VVoyInRGxPSK29zI8hplZh7WG2iXAae3l04CL+ynHzGw8aahJ+gDwP8BjJV0n6WXAG4BnSfoO8Kz2dzOzqUvfU4uIU1dY9Ns911LUh5RvZPxN9DFcXckgkUr+TxkUnEgv9DB5b7ZKFAysmPa6pVuAhcXxB1acS/oDBwWNatkAj3Mlj21WaslzPZlFuKTfMlunpIxsgMeyPrVxVyjnbxSYWVUcamZWFYeamVXFoWZmVXGomVlVHGpmVhWHmplVxaFmZlWZ+AztXc2gJQMrZs2kRY2APcwGnfZWFozOp8FCtkJeSC9djUkDb1GjaLa4YBbvZDzLdOxGIJJBMQvGVSx6HqbbSJ4hw6Km6B6ep8n9zRprm0K6j2kky9uVksWeod3MbFkONTOrikPNzKriUDOzqjjUzKwqDjUzq4pDzcyqMuE+taC7oamgl6m3WsaT9dUMC3p3Fhe7+9SyCZMBYjCXrjO2khaitNSSRy4b0LBgUMSkES0bABJgkPWYFQyauZA8/sPFgudH9hwr6YWkez8i30YMk+dpwWTXvY4CmfCZmplVxaFmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWlYkPEtnZlDosaK7MBk4s6PHLmjj7GLCuZFbzRXU3LUbRTODd6xTNFJ8c9qKj0cu09ulIk3kZi90rFfS8shjJ41Lw/CgZBDKXDIiajapJ3lw7XDyYV5E11/Y4wGMffKZmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWlZmazLifXqceNlEwGGFWasldSXuZCgaaHCTrlEzMm03wWzRB9JjLm5WSQRELev8y2cCLkPftlUgnuy7YxyDt28sHZxwOu/vQSgZ4zPvySgZ3ndzwrukzXtJ5kvZL2j1y3dmSfiRpV/tz0vqWaWZWpuTl53uBE5e5/s0Rsa39+WS/ZZmZrU0aahFxGXDTBGoxMxvbOB8UnCHp6vbl6QN7q8jMbAxrDbW3A48GtgH7gDettKKkHZKulHTlGvdlZlZsTaEWETdExGJEDIF3Acd1rLszIrZHxPa1FmlmVmpNoSZpy8ivpwC7V1rXzGyS0j41SR8AngE8WNJ1wFnAMyRto2lQ2Qucvo41mpkVUx8DIhbvTN3dhIOC2cb7GBQx6wMsahNMuidLmg2zxtiiAR6TY1a0jaTWiPy+lAxXmMmeiwVl9FBF3hg7KOicncuOe8EAj9kAjpE01gIMs9nVexnLcjJd83fdfddVJW9j+WtSZlYVh5qZVcWhZmZVcaiZWVUcamZWFYeamVXFoWZmVZn4IJFdomBQxKzvajAYv5mpZJDItE+taKTBpD+sYPC9SPqQSnqIslrLet2yvqx0E6lBSe9fetzz45H1mM3P5/2Uc3PZOvkxPUj3Y3vXYh8DPPYyqmq+yiwNEmlmtpE41MysKg41M6uKQ83MquJQM7OqONTMrCoONTOrikPNzKoyY823JYPNda9T0vQ6KJi1fLwqSlbIm41LjkfaoFtyTNND1kfTa0kZWTPy+JJxShvJ82Nu8EvpJjZt7v7TGswX/OlFd/PtwmK+jSHdDbpFf3PJY7u4kDSAA4tJo/BwmDfel/KZmplVxaFmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVVmqk+tRCT9LNlygEj6oYYlgzMm/T0lXVvp5L2Tmmi6h6a7dI2iA1KwzgQ2Mpf0qQ0L+hwjNncuHxScT2SDUW4q6FO7/e67O5cfPJhPiDzJCc/74DM1M6uKQ83MquJQM7OqONTMrCoONTOrikPNzKriUDOzqjjUzKwqafeepKOBfwN+BRgCOyPiLZKOAC4EtgJ7gRdExM3rV2pjmAysuDgsmLV6zOWQ95ION1jD4nqL4Wwcj5Ie4MWkgXuhYGb0O++8o3P5bT+/Ld3G7Xfc3rm8z4EVa1JyprYAvCYiHgccD7xS0uOBM4HPRcSxwOfa383MpioNtYjYFxFfbS8fAPYARwEnA+e3q50PPHe9ijQzK7Wq99QkbQWeDFwOPDQi9kETfMCRfRdnZrZaxV9ol3QYcBHw6oi4tXSiDUk7gB1rK8/MbHWKztQkbaIJtPdFxEfaq2+QtKVdvgXYv9xtI2JnRGyPiO19FGxm1iUNNTWnZO8B9kTEuSOLLgFOay+fBlzcf3lmZqtT8vLzBOAlwDWSdrXXvRZ4A/BBSS8DrgWevz4lmpmV0yQHgFPRTLJj7yNdZzDozvI+JuatyUYaJDB76IomiE76v7JeSVs3V5W8jeVvFJhZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWFYeamVXFoWZmVdlwM7Rnipork+ZJqXtmbOhnMvGsxXdWml5LepHzUsef9T573Eq2YfXzmZqZVcWhZmZVcaiZWVUcamZWFYeamVXFoWZmVXGomVlVqutTK5H1MpVMEtvHQJJZR1XRHnoYz7KP4+GJdW1W+EzNzKriUDOzqjjUzKwqDjUzq4pDzcyq4lAzs6o41MysKg41M6vKvbT5NhsksqSjdfyu13SQyIKBFYeL3felZGBFs5r4TM3MquJQM7OqONTMrCoONTOrikPNzKriUDOzqjjUzKwq98o+tUwfvV19TKpsZquXnqlJOlrSFyTtkfR1Sa9qrz9b0o8k7Wp/Tlr/cs3MupWcqS0Ar4mIr0o6HLhK0qXtsjdHxD+vX3lmZquThlpE7AP2tZcPSNoDHLXehZmZrcWqPiiQtBV4MnB5e9UZkq6WdJ6kB/Zcm5nZqhWHmqTDgIuAV0fErcDbgUcD22jO5N60wu12SLpS0pU91Gtm1kkln9JJ2gR8HPhMRJy7zPKtwMcj4gnJdvKdzYCSUTqk7v8P/OmnWe+uiojt2Uoln34KeA+wZzTQJG0ZWe0UYPdaqjQz61PJp58nAC8BrpG0q73utcCpkrbRzMm7Fzh9XSo0M1uFopefve1sg7z8NLOZ1M/LTzOzjcShZmZVcaiZWVUcamZWFYeamVXFoWZmVXGomVlVHGpmVhWHmplVxaFmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWFYeamVXFoWZmVXGomVlVHGpmVhWHmplVxaFmZlVxqJlZVRxqZlYVh5qZVcWhZmZVcaiZWVUcamZWFYeamVXFoWZmVXGomVlVHGpmVhWHmplVJQ01SfeR9BVJX5P0dUl/017/KEmXS/qOpAslbV7/cs3MupWcqd0FPDMingRsA06UdDzwRuDNEXEscDPwsvUr08ysTBpq0bit/XVT+xPAM4EPt9efDzx3XSo0M1uFovfUJM1J2gXsBy4FvgfcEhEL7SrXAUetT4lmZuWKQi0iFiNiG/Bw4DjgccutttxtJe2QdKWkK9depplZmVV9+hkRtwBfBI4HHiBpvl30cOD6FW6zMyK2R8T2cQo1MytR8unnQyQ9oL18X+B3gD3AF4DntaudBly8XkWamZWaz1dhC3C+pDmaEPxgRHxc0jeACySdA/wf8J51rNPMrIgiln0rbH12Jk1uZ2ZWm6tK3sbyNwrMrCoONTOrikPNzKriUDOzqjjUzKwqDjUzq4pDzcyq4lAzs6qUfKOgTzcCPxj5/cHtdRvBRql1o9QJG6fWjVInbJxa11LnI0tWmug3Cn5h59KVG+WL7hul1o1SJ2ycWjdKnbBxal3POv3y08yq4lAzs6pMO9R2Tnn/q7FRat0odcLGqXWj1Akbp9Z1q3Oq76mZmfVt2mdqZma9mlqoSTpR0rckfVfSmdOqIyNpr6RrJO2atXkWJJ0nab+k3SPXHSHp0nY+1kslPXCaNbY1LVfn2ZJ+1B7XXZJOmmaNSyQdLekLkva089y+qr1+po5rR50zdVynMm9wREz8B5ijmZHqGGAz8DXg8dOopaDWvcCDp13HCrU9HXgKsHvkun8Ezmwvnwm8cUbrPBv4s2nXtkytW4CntJcPB74NPH7WjmtHnTN1XAEBh7WXNwGX08xx8kHghe317wBe0dc+p3Wmdhzw3Yj4fkTcDVwAnDylWjasiLgMuOmQq0+mmYcVZmQ+1hXqnEkRsS8ivtpePkAzH8dRzNhx7ahzpkRjovMGTyvUjgJ+OPL7LM8bGsBnJV0lace0iynw0IjYB80THzhyyvV0OUPS1e3L06m/TD6UpK3Ak2nOLmb2uB5SJ8zYcZ30vMHTCjUtc92sfgx7QkQ8Bfg94JWSnj7tgirxduDRwDZgH/Cm6ZZzT5IOAy4CXh0Rt067npUsU+fMHdcYY97gtZhWqF0HHD3y+4rzhk5bRFzf/rsf+CjNgzLLbpC0BaD9d/+U61lWRNzQPtmHwLuYoeMqaRNNULwvIj7SXj1zx3W5Omf5uMYa5g1ei2mF2hXAse0nIJuBFwKXTKmWFUm6n6TDly4DzwZ2d99q6i6hmYcVZng+1qWAaJ3CjBxXSaKZ7nFPRJw7smimjutKdc7acZ3GvMFTa75tP2r+F5pPQs+LiL+fSiEdJB1Dc3YGzYgm75+lOiV9AHgGzYgHNwBnAf9B88nSI4BrgedHxFTfpF+hzmfQvEQKmk+YT196z2qaJP0G8CXgGmDYXv1amverZua4dtR5KjN0XCU9keaDgNF5g/+2/du6ADiCZt7gF0fEXb3sc1qhZma2HvyNAjOrikPNzKriUDOzqjjUzKwqDjUzq4pDzcyq4lAzs6o41MysKv8PzWh7LowR9W4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcb9cbe0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Visualize image preprocessing\n",
    "index = random.randint(0, len(X_train))\n",
    "image = X_train[index].squeeze()\n",
    "\n",
    "plt.figure(figsize=(5,5))\n",
    "sign_num = y_train[index]\n",
    "plt.title(sign_names[sign_num])\n",
    "plt.imshow(image)\n",
    "\n",
    "trans_image = transformImage(image, 20, 10, 5)\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.title(\"Translated Image\")\n",
    "plt.imshow(trans_image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beginning Data Augmentation...\n",
      "Augmenting Class  1 / 43\n",
      "Augmenting Class  2 / 43\n",
      "Augmenting Class  3 / 43\n",
      "Augmenting Class  4 / 43\n",
      "Augmenting Class  5 / 43\n",
      "Augmenting Class  6 / 43\n",
      "Augmenting Class  7 / 43\n",
      "Augmenting Class  8 / 43\n",
      "Augmenting Class  9 / 43\n",
      "Augmenting Class  10 / 43\n",
      "Augmenting Class  11 / 43\n",
      "Augmenting Class  12 / 43\n",
      "Augmenting Class  13 / 43\n",
      "Augmenting Class  14 / 43\n",
      "Augmenting Class  15 / 43\n",
      "Augmenting Class  16 / 43\n",
      "Augmenting Class  17 / 43\n",
      "Augmenting Class  18 / 43\n",
      "Augmenting Class  19 / 43\n",
      "Augmenting Class  20 / 43\n",
      "Augmenting Class  21 / 43\n",
      "Augmenting Class  22 / 43\n",
      "Augmenting Class  23 / 43\n",
      "Augmenting Class  24 / 43\n",
      "Augmenting Class  25 / 43\n",
      "Augmenting Class  26 / 43\n",
      "Augmenting Class  27 / 43\n",
      "Augmenting Class  28 / 43\n",
      "Augmenting Class  29 / 43\n",
      "Augmenting Class  30 / 43\n",
      "Augmenting Class  31 / 43\n",
      "Augmenting Class  32 / 43\n",
      "Augmenting Class  33 / 43\n",
      "Augmenting Class  34 / 43\n",
      "Augmenting Class  35 / 43\n",
      "Augmenting Class  36 / 43\n",
      "Augmenting Class  37 / 43\n",
      "Augmenting Class  38 / 43\n",
      "Augmenting Class  39 / 43\n",
      "Augmenting Class  40 / 43\n",
      "Augmenting Class  41 / 43\n",
      "Augmenting Class  42 / 43\n",
      "Augmenting Class  43 / 43\n",
      "Data Successfully Augmented\n",
      "6661  images added to training set\n"
     ]
    }
   ],
   "source": [
    "#Run data augmentation on the Training set\n",
    "\n",
    "def augment_data(train_data, train_labels):\n",
    "    print(\"Beginning Data Augmentation...\")\n",
    "    unique_labels, label_counts = np.unique(train_labels, return_counts=True)\n",
    "    unique_labels, label_index = np.unique(train_labels, return_index=True)\n",
    "    \n",
    "    total_added = 0\n",
    "    data = train_data\n",
    "    labels = train_labels\n",
    "    class_index = 1\n",
    "        \n",
    "    for i in label_index:\n",
    "        print(\"Augmenting Class \", class_index, \"/\", n_classes)\n",
    "        class_index +=1\n",
    "        \n",
    "        image_class = labels[i]\n",
    "        image_class_arr = np.expand_dims(image_class, axis=0)\n",
    "        \n",
    "        indices_in_class = np.where(labels == image_class)\n",
    "\n",
    "\n",
    "        while label_counts[image_class] < 600:\n",
    "            rolling_index = random.randint(0, len(indices_in_class[0])-1)\n",
    "            image = data[indices_in_class[0][rolling_index]].squeeze()\n",
    "    \n",
    "            trans_img = transformImage(image, 20, 10, 5)\n",
    "            trans_img_arr = np.expand_dims(trans_img, axis=0)\n",
    "            \n",
    "            data = np.append(data, trans_img_arr, axis=0)\n",
    "            labels = np.append(labels, image_class_arr, axis=0)\n",
    "            \n",
    "            total_added += 1\n",
    "            label_counts[image_class] +=1\n",
    "\n",
    "     \n",
    "    print(\"Data Successfully Augmented\")\n",
    "    print(total_added, \" images added to training set\")\n",
    "    \n",
    "    return data, labels\n",
    "\n",
    "\n",
    "X_train, y_train = augment_data(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEICAYAAAC55kg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAF6hJREFUeJzt3X+0ZWV93/H3JwNiEkQgXCzOQGa04w/IiqPeIF1EQ2OKAzGCjSYQfxBrO2qgxVVti3a1WH/UpImaRRtJR50CDSJUVKhLa0aqoF2i3FHkhyMyIIHrTGZGUcBiaQe+/WPvK8eZc+/cOefMPXPvfr/WOuuc/exn7/2cfWfu5+5n/3hSVUiSuunnxt0ASdL4GAKS1GGGgCR1mCEgSR1mCEhShxkCktRhhoC0j5IsS/LjJMeNsq40DoaAFlSSe5JsT/KLPWX/OMkX9+M2f9zzeizJT3qmX7Wv66uqR6vq0Kq6d5R191WSI5JckuRvkzyY5I4kb53nsn+V5B2jbpMWH0NA43AQcP5Cbaz9JXxoVR0K3Av8Tk/Z5bvXT3LQQrVtSBcBTwCeBRwOnAncPdYWadExBDQOfwq8Ncnh/WYmeVaSjUnub/+6/b22fFWSHyX5uXb6w0l29Cz3V0nevK+NSfLuJFcmuSLJQ8Crk/y9JDe229uW5KIkB7f1D0pSSVb2bPeiJJ9N8lCSryRZta912/mnJflOkgeS/Mck/yvJH87S9F8DPlpVP6qqx6pqc1V9omddxyf5fLsfv53kd9vyPwJ+H3h7ezT0yX3dZ1o6DAGNwxTwRWCProu2m2gj8FHgaOBs4INJTqiq7wIPAs9tq78Q+HGSZ7fTLwKuH7BNL2+3+WTgSmAXzdHKUcDJwFrgDXMs/wfAvwGOpDnaeNe+1k1yNHAV8C/a7X4XOHGO9dwIvDfJHyZZ3TsjyZNo9uNlNPvxVcD6JM+sqg+23/Hft0dDL59jG1riDAGNy78F/mmSid3KXwrcU1X/pap2VdXXgauBV7Tzrwd+I8nfaac/3k6vAg4Dvjlge75cVf+9/Yv6J1V1U1V9tW3D3cB64DfmWP7jVTVVVf8PuBxYM0DdlwI3V9U17bwPAN+fYz1/RPPL/J8Bm5PcmeTUdt7LgO9U1WXtd9gEfIrH96MENH2z0oKrqtuSfBq4ANjcM+uXgRck+VFP2UHAf20/X0/zC24auIHmiOI1wP8BvlRVjw3YpPt6J5I8C3gf8HzgF9o2fHWO5f+25/PDwKED1H1qbzuqqpJMz7aSqnoYeDfw7iRPBt4OXJ1kBc1+PLnPfrxkjnapgzwS0DhdCPwTYHlP2X3A9VV1eM/r0Kp6Uzv/eppuoFPaz1+m6a75DQbvCgLY/XG6/xm4Dfi7VXUYzZFLhlj/fGwDVsxMJAk/u29mVVUPAO+lCZSVNPvxuj778byZRUbaci1ahoDGpqq28Hh3xoxPA89I8pokB7evX5vp96+qO4GfAK8GbqiqB4HtwO8yXAjs7knAA8D/brc91/mAUfk08Lwkv9NeoXQ+sHt32U8luTDJZJInJHkizX68H7gTuBY4Ickf9OzHE5M8s118O/C0/ft1tBgYAhq3dwI/vWegqh4CTgXOArbSdJ38CXBIzzLXAz/oufb+epq/0r8xwna9BTgHeIjmqODKEa67r6raTnPVzvuBHwBPp/lOj8yx2KVt3a00R0e/XVUPt0cGL6EJy200+/G9PL4fPww8J8kPk3x89N9Gi0UcVEY6MCVZRvPL/RVV9aVxt0dLk0cC0gEkydokT05yCM1lpLuAr425WVrCDAHpwPLrNHf9fp/m3oQzq2qu7iBpKHYHSVKHeSQgSR12wN8sdtRRR9XKlSvH3QxJWjQ2bdr0/aqa9fLiXgd8CKxcuZKpqalxN0OSFo0kfzPfunYHSVKHGQKS1GGGgCR1mCEgSR1mCEhShxkCktRhhoAkddheQyDJsUm+kGRzktuTnN+WH9kOBn5n+35EW552IO0tSW5J8ryedZ3T1r8zyTn772tJkuZjPkcCu4C3VNWzgZOAc5McTzMs4HVVtRq4rp0GOA1Y3b7WARdDExo0I0m9gGbw7AtngkOSNB57DYGq2tYO9j0z4MdmmiHvzqAZ0IL2/cz28xnAZdW4ETg8yTE0A1xsrKr7q+qHwEaapyQeeJL+L0laYvbpnECSlcBzaQbcfkpVbYMmKICj22rL+dlBu6fbstnK+21nXZKpJFM7d+7clyZKkvbBvEMgyaHA1cCb23FdZ63ap6zmKN+zsGp9VU1W1eTExLyegSRJGsC8QiDJwTQBcHlVfaIt3t5289C+72jLp4FjexZfQTNE3mzl42GXjyTN6+qgAB8BNlfV+3tmXUszEDft+zU95a9trxI6CXig7S76HHBqkiPaE8KntmWSpDGZz6OkTwZeA9ya5Oa27O3AHwNXJXk9cC/wynbeZ4DTgS3Aw8DrAKrq/iTvAm5q672zqu4fybeQJA3kgB9ecnJysvbLeAKzdf1UzT1Pkg5wSTZV1eR86nrHsCR1mCEgSR12wA8vqUXEbjRp0fFIQJI6zBCQpA6zO0j7xi4faUnxSECSOswQkKQOsztIknp1rMvTIwFJ6jBDQJI6zBCQpA4zBCSpwwwBSeowrw7qqo5dASGpP48EJKnD5jO85IYkO5Lc1lN2ZZKb29c9MyOOJVmZ5Cc98/6yZ5nnJ7k1yZYkF7XDVkqSxmg+3UGXAP8JuGymoKp+f+ZzkvcBD/TUv6uq1vRZz8XAOuBGmiEo1wKf3fcmH8DsYpG0yOz1SKCqbgD6jgXc/jX/e8AVc60jyTHAYVX1lWrGs7wMOHPfmytJGqVhzwm8ENheVXf2lK1K8o0k1yd5YVu2HJjuqTPdlkmSxmjYq4PO5mePArYBx1XVD5I8H/hUkhOAfv0ks/aRJFlH03XEcccdN2QTJUmzGfhIIMlBwD8Erpwpq6pHquoH7edNwF3AM2j+8l/Rs/gKYOts666q9VU1WVWTExMTgzZRkrQXw3QH/Rbw7ar6aTdPkokky9rPTwNWA3dX1TbgoSQntecRXgtcM8S2JUkjMJ9LRK8AvgI8M8l0kte3s85izxPCLwJuSfJN4OPAG6tq5qTym4APA1tojhCW1pVBkrQIpQ7wyxcnJydrampq9Cue63LOQS/1XEyXiO6P77iYvr80myXw7zjJpqqanE9d7xiWpA4zBCSpwwwBSeowQ0CSOswQkKQOczyBhbIErjiQtPR4JCBJHWYISFKHGQKS1GGGgCR1mCEgSR1mCEhShxkCktRhhoAkdZghIEkdZghIUocZApLUYfMZXnJDkh1Jbuspe0eS7yW5uX2d3jPvbUm2JLkjyUt6yte2ZVuSXDD6ryJJ2lfzORK4BFjbp/wDVbWmfX0GIMnxNGMPn9Au88Eky9rB5/8COA04Hji7rStJGqO9PkW0qm5IsnKe6zsD+FhVPQJ8N8kW4MR23paquhsgycfaut/a5xZLkkZmmHMC5yW5pe0uOqItWw7c11Nnui2brbyvJOuSTCWZ2rlz5xBNlCTNZdAQuBh4OrAG2Aa8ry3v99D8mqO8r6paX1WTVTU5MTExYBMlSXsz0KAyVbV95nOSDwGfbiengWN7qq4AtrafZyuXJI3JQEcCSY7pmXw5MHPl0LXAWUkOSbIKWA18DbgJWJ1kVZIn0Jw8vnbwZkuSRmGvRwJJrgBOAY5KMg1cCJySZA1Nl849wBsAqur2JFfRnPDdBZxbVY+26zkP+BywDNhQVbeP/NtIkvZJ6gAf43ZycrKmpqZGv+K5xvwddDzg/bHO/aUL31EaxBL4d5xkU1VNzqeudwxLUocZApLUYYaAJHWYISBJHWYISFKHGQKS1GGGgCR1mCEgSR1mCEhShxkCktRhAz1FVNIC6vcYg0X0CAMd2DwSkKQOMwQkqcPsDlrK7EaQtBceCUhShxkCktRhdgctBnbrSNpP9nokkGRDkh1Jbusp+9Mk305yS5JPJjm8LV+Z5CdJbm5ff9mzzPOT3JpkS5KLktmG75EkLZT5dAddAqzdrWwj8CtV9avAd4C39cy7q6rWtK839pRfDKyjGXx+dZ91SpIW2F5DoKpuAO7freyvq2pXO3kjsGKudSQ5Bjisqr5SzaDGlwFnDtbkMUv6vw6k9kjSPI3ixPA/Aj7bM70qyTeSXJ/khW3ZcmC6p850W9ZXknVJppJM7dy5cwRNlCT1M1QIJPnXwC7g8rZoG3BcVT0X+OfAR5McBvT783TWM5tVtb6qJqtqcmJiYpgmSpLmMPDVQUnOAV4KvLjt4qGqHgEeaT9vSnIX8Ayav/x7u4xWAFsH3bYkaTQGOhJIshb4V8DLqurhnvKJJMvaz0+jOQF8d1VtAx5KclJ7VdBrgWuGbr0kaSh7PRJIcgVwCnBUkmngQpqrgQ4BNrZXet7YXgn0IuCdSXYBjwJvrKqZk8pvornS6OdpziH0nkeQJI3BXkOgqs7uU/yRWepeDVw9y7wp4Ff2qXWSpP3Kx0ZIUocZApLUYT47SHua7YYzn1ckLTkeCUhShxkCktRhhoAkdZghIEkdZghIUocZApLUYYaAJHWYISBJHWYISFKHecewlq5+dz571/PS4s94aB4JSFKHGQKS1GGGgCR1mCEgSR02rxBIsiHJjiS39ZQdmWRjkjvb9yPa8iS5KMmWJLckeV7PMue09e9sB6qXJI3RfI8ELgHW7lZ2AXBdVa0GrmunAU6jGWB+NbAOuBia0KAZn/gFwInAhTPBIUkaj3mFQFXdANy/W/EZwKXt50uBM3vKL6vGjcDhSY4BXgJsrKr7q+qHwEb2DBZJ0gIa5pzAU6pqG0D7fnRbvhy4r6fedFs2W/kekqxLMpVkaufOnUM0UZI0l/1xYrjf2IQ1R/mehVXrq2qyqiYnJiZG2jgtMcmeL0nzNkwIbG+7eWjfd7Tl08CxPfVWAFvnKJckjckwIXAtMHOFzznANT3lr22vEjoJeKDtLvoccGqSI9oTwqe2ZZKkMZnXs4OSXAGcAhyVZJrmKp8/Bq5K8nrgXuCVbfXPAKcDW4CHgdcBVNX9Sd4F3NTWe2dV7X6yWZK0gOYVAlV19iyzXtynbgHnzrKeDcCGebdOkrRfecewJHWYISBJHWYISFKHGQKS1GGGgCR1mCEgSR1mCEhShxkCktRhhoAkdZghIEkdZghIUocZApLUYYaAJHWYISBJHWYISFKHGQKS1GEDh0CSZya5uef1YJI3J3lHku/1lJ/es8zbkmxJckeSl4zmK0iSBjWvkcX6qao7gDUASZYB3wM+STOc5Aeq6s966yc5HjgLOAF4KvD5JM+oqkcHbYMkaTij6g56MXBXVf3NHHXOAD5WVY9U1XdpxiA+cUTblyQNYFQhcBZwRc/0eUluSbIhyRFt2XLgvp46022ZJGlMhg6BJE8AXgb8t7boYuDpNF1F24D3zVTts3jNss51SaaSTO3cuXPYJkqSZjGKI4HTgK9X1XaAqtpeVY9W1WPAh3i8y2caOLZnuRXA1n4rrKr1VTVZVZMTExMjaKIkqZ9RhMDZ9HQFJTmmZ97Lgdvaz9cCZyU5JMkqYDXwtRFsX5I0oIGvDgJI8gvAPwDe0FP8H5KsoenquWdmXlXdnuQq4FvALuBcrwySpPEaKgSq6mHgl3Yre80c9d8DvGeYbUqSRsc7hiWpw4Y6EpDmLf0uDgOq7wViWoz8GS9KHglIUocZApLUYYaAJHWYISBJHWYISFKHGQKS1GGGgCR1mCEgSR22tG8W63fzyoF440rXb7LZ2/ffHz/HubZ5IM3bm7n2zb7Om097Bm3rMOsc9Oc/6Pff13XOLDeOn/8IeCQgSR1mCEhShxkCktRhhoAkdZghIEkdZghIUocNHQJJ7klya5Kbk0y1ZUcm2Zjkzvb9iLY8SS5KsiXJLUmeN+z2JUmDG9WRwN+vqjVVNdlOXwBcV1WrgevaaYDTaAaYXw2sAy4e0fYlSQPYX91BZwCXtp8vBc7sKb+sGjcChyc5Zj+1QZK0F6MIgQL+OsmmJOvasqdU1TaA9v3otnw5cF/PstNt2c9Isi7JVJKpnTt3jqCJkqR+RvHYiJOramuSo4GNSb49R91+90fvcW90Va0H1gNMTk525NkJkrTwhj4SqKqt7fsO4JPAicD2mW6e9n1HW30aOLZn8RXA1mHbIEkazFAhkOQXkzxp5jNwKnAbcC1wTlvtHOCa9vO1wGvbq4ROAh6Y6TaSJC28YbuDngJ8Ms1T8A4CPlpV/yPJTcBVSV4P3Au8sq3/GeB0YAvwMPC6IbcvSRrCUCFQVXcDz+lT/gPgxX3KCzh3mG1KkkbHO4YlqcMMAUnqMENAkjrMEJCkDjMEJKnDDAFJ6jBDQJI6zBCQpA4zBCSpwwwBSeowQ0CSOswQkKQOMwQkqcMMAUnqMENAkjrMEJCkDjMEJKnDBg6BJMcm+UKSzUluT3J+W/6OJN9LcnP7Or1nmbcl2ZLkjiQvGcUXkCQNbpjhJXcBb6mqr7eDzW9KsrGd94Gq+rPeykmOB84CTgCeCnw+yTOq6tEh2iBJGsLARwJVta2qvt5+fgjYDCyfY5EzgI9V1SNV9V2aweZPHHT7kqThjeScQJKVwHOBr7ZF5yW5JcmGJEe0ZcuB+3oWm2aW0EiyLslUkqmdO3eOoomSpD6GDoEkhwJXA2+uqgeBi4GnA2uAbcD7Zqr2Wbz6rbOq1lfVZFVNTkxMDNtESdIshgqBJAfTBMDlVfUJgKraXlWPVtVjwId4vMtnGji2Z/EVwNZhti9JGs4wVwcF+Aiwuare31N+TE+1lwO3tZ+vBc5KckiSVcBq4GuDbl+SNLxhrg46GXgNcGuSm9uytwNnJ1lD09VzD/AGgKq6PclVwLdoriw61yuDJGm8Bg6Bqvoy/fv5PzPHMu8B3jPoNiVJo+Udw5LUYYaAJHWYISBJHWYISFKHGQKS1GGGgCR1mCEgSR1mCEhShxkCktRhhoAkdZghIEkdZghIUocZApLUYYaAJHWYISBJHWYISFKHGQKS1GELHgJJ1ia5I8mWJBcs9PYlSY9b0BBIsgz4C+A04Hia8YiPX8g2SJIet9BHAicCW6rq7qr6v8DHgDMWuA2SpNbAA80PaDlwX8/0NPCC3SslWQesayd/nOSOEWz7KOD7JHPXmmv+0p3X7JsDpz3jmdd/fnf2zb4vu7T2zWjXu3/3zd798nwrLnQI9PtWtUdB1Xpg/Ug3nExV1eQo17lUuG9m576Znftmdotp3yx0d9A0cGzP9Apg6wK3QZLUWugQuAlYnWRVkicAZwHXLnAbJEmtBe0OqqpdSc4DPgcsAzZU1e0LtPmRdi8tMe6b2blvZue+md2i2Tep2qNLXpLUEd4xLEkdZghIUoct+RDwMRU/K8mGJDuS3NZTdmSSjUnubN+PGGcbxyHJsUm+kGRzktuTnN+Wu2+SJyb5WpJvtvvm37Xlq5J8td03V7YXe3RSkmVJvpHk0+30otk3SzoEfExFX5cAa3cruwC4rqpWA9e1012zC3hLVT0bOAk4t/234r6BR4DfrKrnAGuAtUlOAv4E+EC7b34IvH6MbRy384HNPdOLZt8s6RDAx1TsoapuAO7frfgM4NL286XAmQvaqANAVW2rqq+3nx+i+Q+9HPcN1fhxO3lw+yrgN4GPt+Wd3DcASVYAvw18uJ0Oi2jfLPUQ6PeYiuVjasuB7ClVtQ2aX4bA0WNuz1glWQk8F/gq7hvgp90dNwM7gI3AXcCPqmpXW6XL/7f+HPiXwGPt9C+xiPbNUg+BeT2mQpqR5FDgauDNVfXguNtzoKiqR6tqDc1d/icCz+5XbWFbNX5JXgrsqKpNvcV9qh6w+2ahnx200HxMxfxsT3JMVW1LcgzNX3udk+RgmgC4vKo+0Ra7b3pU1Y+SfJHmvMnhSQ5q/+Lt6v+tk4GXJTkdeCJwGM2RwaLZN0v9SMDHVMzPtcA57edzgGvG2JaxaPtxPwJsrqr398xy3yQTSQ5vP/888Fs050y+ALyirdbJfVNVb6uqFVW1kub3y/+sqlexiPbNkr9juE3oP+fxx1S8Z8xNGqskVwCn0DzqdjtwIfAp4CrgOOBe4JVVtfvJ4yUtya8DXwJu5fG+3bfTnBfo+r75VZqTm8to/nC8qqremeRpNBdbHAl8A3h1VT0yvpaOV5JTgLdW1UsX075Z8iEgSZrdUu8OkiTNwRCQpA4zBCSpwwwBSeowQ0CSOswQkKQOMwQkqcP+P0HsaWm/kVzpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcbc0748>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples = 41460\n"
     ]
    }
   ],
   "source": [
    "#Visualize new distribution of sign types\n",
    "unique_train, train_counts = np.unique(y_train, return_counts=True)\n",
    "\n",
    "plt.bar(unique_train, train_counts, color='r')\n",
    "plt.title(\"New Training Set\")\n",
    "plt.show()\n",
    "print(\"Number of training examples =\", len(y_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Successfully Normalized\n"
     ]
    }
   ],
   "source": [
    "#Shuffle training set\n",
    "X_train, y_train = shuffle(X_train, y_train)\n",
    "\n",
    "#Normalize Images in Dataset\n",
    "X_train = norm(X_train)\n",
    "X_validation = norm(X_validation)\n",
    "X_test = norm(X_test)\n",
    "\n",
    "print(\"Data Successfully Normalized\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Architecture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Define your architecture here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "\n",
    "EPOCHS = 50\n",
    "BATCH_SIZE = 80\n",
    "KEEP_PROB = tf.placeholder_with_default(1.0, shape=())\n",
    "\n",
    "def covnet(x):    \n",
    "    # Arguments used for tf.truncated_normal, randomly defines variables for the weights and biases for each layer\n",
    "    mu = 0\n",
    "    sigma = 0.1\n",
    "    \n",
    "    \n",
    "    #Layer 1: Convolutional. Input = 32x32x3. Output = 28x28x16.\n",
    "    conv1_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 3, 16), mean = mu, stddev = sigma))\n",
    "    conv1_b = tf.Variable(tf.zeros(16))\n",
    "    conv1   = tf.nn.conv2d(x, conv1_W, strides=[1, 1, 1, 1], padding='VALID') + conv1_b\n",
    "    #Activation.\n",
    "    conv1 = tf.nn.relu(conv1)\n",
    "    #Pooling. Input = 28x28x16. Output = 14x14x16.\n",
    "    conv1 = tf.nn.avg_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    #Layer 2: Convolutional. Output = 10x10x64.\n",
    "    conv2_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 16, 64), mean = mu, stddev = sigma))\n",
    "    conv2_b = tf.Variable(tf.zeros(64))\n",
    "    conv2   = tf.nn.conv2d(conv1, conv2_W, strides=[1, 1, 1, 1], padding='VALID') + conv2_b\n",
    "    #Activation.\n",
    "    conv2 = tf.nn.relu(conv2)\n",
    "    #Pooling. Input = 10x10x64. Output = 5x5x64.\n",
    "    conv2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "    \n",
    "    #Flatten. Input = 5x5x16. Output = 1600.\n",
    "    fc0   = flatten(conv2)\n",
    "    \n",
    "    #Layer 3: Fully Connected. Input = 1600. Output = 800.\n",
    "    fc1_W = tf.Variable(tf.truncated_normal(shape=(1600, 800), mean = mu, stddev = sigma))\n",
    "    fc1_b = tf.Variable(tf.zeros(800))\n",
    "    fc1   = tf.matmul(fc0, fc1_W) + fc1_b\n",
    "    #Activation.\n",
    "    fc1    = tf.nn.relu(fc1)\n",
    "    #Dropout\n",
    "    fc1    = tf.nn.dropout(fc1, KEEP_PROB)\n",
    "    \n",
    "    #Layer 4: Fully Connected. Input = 800. Output = 256.\n",
    "    fc2_W  = tf.Variable(tf.truncated_normal(shape=(800, 256), mean = mu, stddev = sigma))\n",
    "    fc2_b  = tf.Variable(tf.zeros(256))\n",
    "    fc2    = tf.matmul(fc1, fc2_W) + fc2_b\n",
    "    #Activation.\n",
    "    fc2    = tf.nn.relu(fc2)\n",
    "    #Dropout\n",
    "    fc2    = tf.nn.dropout(fc2, KEEP_PROB)\n",
    "    \n",
    "    #Layer 5: Fully Connected. Input = 256. Output = 43.\n",
    "    fc3_W  = tf.Variable(tf.truncated_normal(shape=(256, 43), mean = mu, stddev = sigma))\n",
    "    fc3_b  = tf.Variable(tf.zeros(43))\n",
    "    logits = tf.matmul(fc2, fc3_W) + fc3_b\n",
    "    \n",
    "    return logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train, Validate and Test the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A validation set can be used to assess how well the model is performing. A low accuracy on the training and validation\n",
    "sets imply underfitting. A high accuracy on the training set but low accuracy on the validation set implies overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Train your model here.\n",
    "### Calculate and report the accuracy on the training and validation set.\n",
    "### Once a final model architecture is selected, \n",
    "### the accuracy on the test set should be calculated and reported as well.\n",
    "### Feel free to use as many code cells as needed.\n",
    "x = tf.placeholder(tf.float32, (None, 32, 32, 3))\n",
    "y = tf.placeholder(tf.int32, (None))\n",
    "one_hot_y = tf.one_hot(y, 43)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "rate = 0.001\n",
    "\n",
    "logits = covnet(x)\n",
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=one_hot_y, logits=logits)\n",
    "loss_operation = tf.reduce_mean(cross_entropy)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate = rate)\n",
    "training_operation = optimizer.minimize(loss_operation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_y, 1))\n",
    "accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "def evaluate(X_data, y_data):\n",
    "    num_examples = len(X_data)\n",
    "    total_accuracy = 0\n",
    "    sess = tf.get_default_session()\n",
    "    for offset in range(0, num_examples, BATCH_SIZE):\n",
    "        batch_x, batch_y = X_data[offset:offset+BATCH_SIZE], y_data[offset:offset+BATCH_SIZE]\n",
    "        accuracy = sess.run(accuracy_operation, feed_dict={x: batch_x, y: batch_y, KEEP_PROB: 1.0})\n",
    "        total_accuracy += (accuracy * len(batch_x))\n",
    "    return total_accuracy / num_examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "\n",
      "EPOCH 1 ...\n",
      "Training Accuracy = 0.841\n",
      "Validation Accuracy = 0.861\n",
      "\n",
      "EPOCH 2 ...\n",
      "Training Accuracy = 0.922\n",
      "Validation Accuracy = 0.942\n",
      "\n",
      "EPOCH 3 ...\n",
      "Training Accuracy = 0.945\n",
      "Validation Accuracy = 0.957\n",
      "\n",
      "EPOCH 4 ...\n",
      "Training Accuracy = 0.962\n",
      "Validation Accuracy = 0.964\n",
      "\n",
      "EPOCH 5 ...\n",
      "Training Accuracy = 0.971\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 6 ...\n",
      "Training Accuracy = 0.979\n",
      "Validation Accuracy = 0.969\n",
      "\n",
      "EPOCH 7 ...\n",
      "Training Accuracy = 0.981\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 8 ...\n",
      "Training Accuracy = 0.987\n",
      "Validation Accuracy = 0.967\n",
      "\n",
      "EPOCH 9 ...\n",
      "Training Accuracy = 0.990\n",
      "Validation Accuracy = 0.969\n",
      "\n",
      "EPOCH 10 ...\n",
      "Training Accuracy = 0.991\n",
      "Validation Accuracy = 0.964\n",
      "\n",
      "EPOCH 11 ...\n",
      "Training Accuracy = 0.993\n",
      "Validation Accuracy = 0.967\n",
      "\n",
      "EPOCH 12 ...\n",
      "Training Accuracy = 0.993\n",
      "Validation Accuracy = 0.967\n",
      "\n",
      "EPOCH 13 ...\n",
      "Training Accuracy = 0.996\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 14 ...\n",
      "Training Accuracy = 0.995\n",
      "Validation Accuracy = 0.968\n",
      "\n",
      "EPOCH 15 ...\n",
      "Training Accuracy = 0.997\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 16 ...\n",
      "Training Accuracy = 0.998\n",
      "Validation Accuracy = 0.974\n",
      "\n",
      "EPOCH 17 ...\n",
      "Training Accuracy = 0.997\n",
      "Validation Accuracy = 0.965\n",
      "\n",
      "EPOCH 18 ...\n",
      "Training Accuracy = 0.998\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 19 ...\n",
      "Training Accuracy = 0.998\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 20 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 21 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.975\n",
      "\n",
      "EPOCH 22 ...\n",
      "Training Accuracy = 0.998\n",
      "Validation Accuracy = 0.965\n",
      "\n",
      "EPOCH 23 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.976\n",
      "\n",
      "EPOCH 24 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 25 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 26 ...\n",
      "Training Accuracy = 0.998\n",
      "Validation Accuracy = 0.963\n",
      "\n",
      "EPOCH 27 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 28 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 29 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 30 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 31 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.968\n",
      "\n",
      "EPOCH 32 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.976\n",
      "\n",
      "EPOCH 33 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 34 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.976\n",
      "\n",
      "EPOCH 35 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 36 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.974\n",
      "\n",
      "EPOCH 37 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.975\n",
      "\n",
      "EPOCH 38 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.976\n",
      "\n",
      "EPOCH 39 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 40 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 41 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 42 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 43 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.978\n",
      "\n",
      "EPOCH 44 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.968\n",
      "\n",
      "EPOCH 45 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.971\n",
      "\n",
      "EPOCH 46 ...\n",
      "Training Accuracy = 0.999\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 47 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 48 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.972\n",
      "\n",
      "EPOCH 49 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "EPOCH 50 ...\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.973\n",
      "\n",
      "Model saved\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    num_examples = len(X_train)\n",
    "    \n",
    "    print(\"Training...\")\n",
    "    print()\n",
    "    validation_accuracy_chart = []\n",
    "    train_accuracy_chart = []\n",
    "    for i in range(EPOCHS):\n",
    "        X_train, y_train = shuffle(X_train, y_train)\n",
    "        for offset in range(0, num_examples, BATCH_SIZE):\n",
    "            end = offset + BATCH_SIZE\n",
    "            batch_x, batch_y = X_train[offset:end], y_train[offset:end]\n",
    "            sess.run(training_operation, feed_dict={x: batch_x, y: batch_y, KEEP_PROB: 0.4})\n",
    "            \n",
    "        validation_accuracy = evaluate(X_validation, y_validation)\n",
    "        validation_accuracy_chart.append(validation_accuracy)\n",
    "        \n",
    "        train_accuracy = evaluate(X_train, y_train)\n",
    "        train_accuracy_chart.append(train_accuracy)\n",
    "        print(\"EPOCH {} ...\".format(i+1))\n",
    "        print(\"Training Accuracy = {:.3f}\".format(train_accuracy))\n",
    "        print(\"Validation Accuracy = {:.3f}\".format(validation_accuracy))\n",
    "        print()\n",
    "        \n",
    "    saver.save(sess, './model')\n",
    "    print(\"Model saved\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VNX5+PHPk30nkLDvuBIVESMVRaFiLVArrlXUulRLF5faaltt/bnQr11tq612UYtrK1qqFi0uFEHUuhBEkEX2LawhkD2TZGae3x/nJgxZJ2RCYOZ5v8grM/feuXPuMHnuuc859xxRVYwxxsSGuK4ugDHGmEPHgr4xxsQQC/rGGBNDLOgbY0wMsaBvjDExxIK+McbEEAv65pASkftE5LlO3P8KERnvPRYReVJE9onIxyJylois7qz3NuZIYEHfRJyIXCkiBSJSISI7ROR1ERl7KN5bVU9Q1QXe07HAl4ABqjpaVd9V1eMi/Z7eiUxFZHSk921MpFnQNxElIj8AHgJ+DvQGBgF/AqZ0QXEGA5tUtbKjOxKRhBaWC/B1YC9wbUffp51lEhGxv2HTLvaFMREjIt2A6cBNqvqSqlaqap2qvqqqP2zhNf8UkZ0iUioiC0XkhJB1k0VkpYiUi8g2EbnDW54rIq+JSImI7BWRd+uDn4hsEpFzReQG4AlgjHfFcb+IjBeRwpD99xORf4lIkYhsFJFbQ9bdJyKzROQ5ESkDrmvhsM8C+gHfA64QkaRGx/dNEVnlHcNKERnlLR8oIi95710sIo+EvO9zIa8f4l1FJHjPF4jIAyLyPlAFDBOR60PeY4OIfKtRGaaIyKciUiYi60VkoohcJiKLG213u4i80sJxmihhQd9E0hggBXi5Ha95HTgG6AV8Avw9ZN3fgG+paiZwIvC2t/x2oBDoibua+AlwwHgiqvo34NvAB6qaoar3hq73ThKvAkuB/sAE4DYR+XLIZlOAWUB2o3KFutbbzwve8/ND3uMy4D7gGiALuAAoFpF44DVgMzDEe/+ZLey/OV8HpgGZ3j52e++bBVwP/D7k5DIaeAb4oXccZwObgNnAUBEZHrLfq4Fn21EOcwSyoG8iKQfYo6r+cF+gqjNUtVxVa3AB8mTvigGgDsgTkSxV3aeqn4Qs7wsM9q4k3tX2DyJ1GtBTVaeraq2qbgAeB64I2eYDVX1FVYOqWt14ByKSBlwG/ENV63AniNAUz43Ar1V1kTrrVHUzMBp3dfBD72rIp6rvtaPsT6nqClX1e8f/H1Vd773HO8BbuCsQgBuAGao61zuObar6ufd5v4AL9HhXWENwJyMTxSzom0gqBnJbyn83JiLxIvJLL+VQhquBAuR6vy8BJgObReQdERnjLf8NsA54y0tn3HkQZR0M9PNSRCUiUoK7Yugdss3WNvZxEeAH5njP/w5MEpGe3vOBwPpmXjcQ2Nyek2MjB5RLRCaJyIdeqqsE95nVf4YtlQHgaeDKkHaJF72TgYliFvRNJH0A+IALw9z+SlwK5VygG66mCSAAXg15Ci718wrwore8XFVvV9VhwFeBH4jIhHaWdSuwUVWzQ34yVXVyyDZtXT1cC2QAW0RkJ/BPIBGYGvIeR7Xw3oNaODlWAmkhz/s0s01DuUQkGfgX8CDQW1WzcSchaaMMqOqHQC3uquBKLLUTEyzom4hR1VLgHuBREblQRNJEJNGrif66mZdkAjW4K4Q0XI8fAEQkSUSuEpFuXuqkDAh4684XkaO9Gmr98kA7i/sxUCYiPxaRVO+q40QROS2cF4tIfTvA+cBI7+dk4FfsT/E8AdwhIqeKc7SIDPbeewfwSxFJF5EUETnTe82nwNkiMshLc93VRlGSgGSgCPCLyCTgvJD1fwOuF5EJIhInIv1F5PiQ9c8AjwD+dqaYzBHKgr6JKFX9HfAD4G5cINoK3IyrqTf2DK4hchuwEviw0fqvA5u81M+38fLPuIbf/wIVuKuLP4X0zQ+3nAHcVcJIYCOwBxeku7X2ukZl+1RV31LVnfU/wB+AESJyoqr+E3gA+AdQjvsMeoS899HAFlyj9OVeuebicu3LgMW0kWNX1XLgVtxV0D5cjX12yPqP8Rp3gVLgHVxqq96zuEZyq+XHCLFJVIyJXSKSiuv9M0pV13Z1eUzns5q+MbHtO8AiC/ixI6xeFsaY6CMim3ANvuE2vJsoYOkdY4yJIZbeMcaYGHLYpXdyc3N1yJAhXV0MY4w5oixevHiPqvZsa7vDLugPGTKEgoKCri6GMcYcUURkczjbWXrHGGNiiAV9Y4yJIRb0jTEmhljQN8aYGGJB3xhjYkibQV9EZojIbhFZ3sJ6EZE/iMg6EVlWP2OPt+5aEVnr/RzS+UONMcY0FU5N/ylgYivrJ+FGPTwGN4XbnwFEpAdwL/AF3ExB94pI944U1hhjTMe02U9fVReKyJBWNpkCPONNV/ehiGSLSF9gPDBXVfcCiMhc3Mnj+Y4W2hhz8FSV7aU+1u2uYNOeSmr8AVQhqKAo9SOz9EhPYkhOOsN6ptMrMxk3fUFTvroAu8tq2FdV2+J7JsbHkZQgJMbHeY/d77SkeBLjW657VtT42VlazfYSHztLfVTV+umWlki31ES6pSZ5vxPJSE6gLhikzh+kLqDUBYLUBoL4A0pCvJCc4N4zOSHePY6Pw+cPUO7zU+6ro8znp9znp8LnRwSyUxPJSk0kO23//kUEX12Asuo6SqvrKKmuo7Sqjqq6AGmJ8aQnJ5CZkkBGcgLpyQmkJcVTWeNnb1Uteytr2VdZx96qWvZV1uIPKvEixAnExQlxIsTHQe+sFKaM7N+h/9+2ROLmrP4cOH1bobespeVNiMg03FUCgwYNikCRzJEuEFRWbC/lww3FfLK5hKzUBIb1zGBobjrDctMZlJNGckJ8m/vxB4J8vrOcxZv3sXjzPjYXV9IjPYmemcn0ykyhV1YyvTKTyclIJsH74xMBwf2OEyEjOYFuaYlkpSS0GPiaK/+2fdWsL6rwfiop3FdFrT+IP6j4Ay44+YPueSCo+ANKULXheSCoZKcl0j871f103/87PSmBqtoA1XV+KmsCVNcGqKr1UxsINlseX12QjXsqWbfblaeqtn1zzqQlxTM4J52huWmkJiawu9zHrjIfu8pqKK2ua9e+GkuKjyMtOZ70pATSk+NJS0qgqtbPjlIf5b6DnVEysuLjhIQ4ocbf/OcbKacMyj4ign5zfwXayvKmC1UfAx4DyM/PtxHgopSqsr6okjW7ykmIk4aal/sdR10gyOLN+/hgfTEfb9xLeY37gx/UI42q2gAvFhQ27CtOYED3NPpkpZCVmkBWiquZZaUkkJWaSGl1HYs37+PTrSUNAa5PVgrH9M6gqKKGFdvL2FNRQ7Ad37b4OKGbV/vLTk0kOSG+oWasAF5NubS6jk3FLsDX65GexMAeaSQnxJGSGEdCcgKJ8UJCXBzx8UJinBDnBZb4uDjvBATFlbVsK6nmnTVF7C7v2PS1/bqlcFSvDL6WP5Cje2VwdK8MhvVMJy0pgbiQE129ovIaNhVXsmlPJRv3VLGpuJJVO8rx1QXonZXC0Nx0Th+WQ++sFHplJtM9LYm4ZirtquyvffuD1AXcT40/SHVtgMraAJU1fipr/e53TYCemcmMGZZD3+xU+nZLoW839zstKZ4yn5+SqlpKvRp3aXUdlTUBEuOl4QrC/bjP1x9071Xjd+9f6w9S4w+QkhhPZkoCmSmJZHq19MyURILq/g9LqupCavW11AWUbvVXAN4VRnZaImlJ8VTVBqiocVcKlbX1vwNkJCfQIz2J7mlJ7nd6ItmpSSTGC6oQUHeiDwYheIgGv4xE0C/ETb5cbwCw3Vs+vtHyBRF4P9PFqmsDfLq1hMWb9/LJlhJSE+PJ65dFXr8sTuiXRa/MlIZty311vL+umHfWFLFwTRHbSqrb3P/Q3HTOP7kfpw/rwZhhOfTKcvsr89WxaU8lG4oq2bCnkg1FFeypqGF7iY/PfeWUVddRXuNH1QXovL5ZfC1/IKMGdyd/cHf6Zace8D6BoLK3spbd5T6KK2oJBBXF/QEq7iQVVKWiJkBJVS0lVXXsq6qlpLqOkqpaav1BBMH7h8SBEMegHml88bheHNXTBdVhPTPokZ7U4c+9xh9gR4mPbSXV+OoCpCa5WnF6UnzD46SEuGZrW/FxQkpi21dGoQb2SGNgjzTOOqbN4VwOqZyMZCC9q4vRYSLQ/P9WJ79vOEMrezn911T1xGbWfQU3Hd5kXKPtH1R1tNeQuxio783zCXBqfY6/Jfn5+Wpj7xxefHUBFq4p4qONeynYvI8V20rxe1XkY3pl4PMH2Lp3fzDPzUgmr18WvroAn2zehz+oZCQncObROYw7thcjB2YTVG2oedX4Aw2XzScPyKZPt5RmyxGOYFCpqPWTGBdHalL7gpwxRzIRWayq+W1t12ZNX0Sex9XYc0WkENcjJxFAVf8CzMEF/HVAFW4+TlR1r4j8DFjk7Wp6WwHfdL5gUHljxU4+21bKqYO6M3pYD7JSEptsp6p8urWEWYsLeXXpdsp8fpIT4jh5QDbTzh5G/pDujBrUnew0V4Mtra5j1Y4yVm4vY+WOMlZsLyM+DqadPYxxx/Zk1ODurTbYRUpcnDR7PMYY57CbRMVq+p3n3bVF/PqN1Xy2rRQRl2uNEzhpQDZjhuVwxlE5DOqRxpzlO5i1uJANRZWkJMYx8YQ+XDxqAKcPyyEpwe7nM+ZwFLGavjnyfbq1hF+/8Tn/W19M/+xUfnvZyUw+qS+fbi3hgw3FfLB+D0+8u4G/vLO+4TWnDenOt84exuST+pJpNWdjooYF/ShS6w9S7nO9Dcp8fvZV1fLioq28vnwnOelJ3PvVPK78wqCGro5jjsphzFE58KVjqazxU7B5HxuLKhh/XC+G5B75DWXGmKYs6B+hAkFlyZZ9vLliJ/M+382OEh/VdU37XqcnxXPbucdw41nDyEhu+b87PTmBccf2ZNyxh1dPDWNMZFnQP4LU+AP8b30xb63YydyVu9lTUUNivHD6sBwmHN9rf191r996t9REju6V0dDYaowxFvQPc8GgsmjTXv71SSGvf7aT8ho/6UnxjD++F+fl9eaLXrA3xphwWNA/TG3cU8nLnxTy0pJtFO6rJj0pnokn9uX8EX0Zc1ROu2+0McYYsKB/2Fmwejd/fHsdizfvQwTGHp3LHecdx3kn9CYtyf67jDEdY1HkMLF1bxU/e20lb63cxeCcNO6cdDwXjuzfobtTjTGmMQv6XcxXF+DxhRt4dME6BOFHE4/jhrFDwxpB0hhj2suCfhdasHo3981ewabiKiaf1IeffiWP/o0GBTPGmEiyoN8FVJWfvbaKGe9vZFhuOs98YzRnW/94Y8whYEG/Czz+7gZmvL+Ra8YM5qdfGW6pHGPMIWNB/xB7bdl2fj7nc74yoi/3ffUE4uIO/XjaxpjYZUMmHkIfb9zLD15YymlDuvPby062gG+MOeQs6B8i63ZX8M1nChjQI5XHr8m3m6uMMV3Cgv4hUFRew3VPfkxivPD09aNtLBxjTJcJK+iLyEQRWS0i60TkzmbWDxaReSKyTEQWiMiAkHW/FpEVIrJKRP4gIjGV06iq9XPD04sorqhlxnWnMbBHWlcXyRgTw8KZLjEeeBT4Em6y80UiMltVV4Zs9iDwjKo+LSLnAL8Avi4iZwBnAiO87d4DxhHlE6SrKmt2VTB/9W5mf7qdz3eW8djX8xkxILuri2aMiXHh9N4ZDaxT1Q0AIjITmAKEBv084Pve4/nAK95jBVKAJEBwc+vu6nixDz9VtX7+t66Y+at3s2B1EdtK3ETheX2z+P3lIzk3r3cXl9AYY8IL+v2BrSHPC4EvNNpmKXAJ8DBwEZApIjmq+oGIzAd24IL+I6q6qvEbiMg0YBrAoEGD2n0QXW1PRQ2THn6XovIa0pLiGXt0LrecczTjj+tlY+cYYw4r4QT95nLwjWdTvwN4RESuAxYC2wC/iBwNDAfqc/xzReRsVV14wM5UHwMeAzcxevjFPzz8bu4a9lXW8sQ1+Zx1bK7dbGWMOWyF05BbCAwMeT4A2B66gapuV9WLVfUU4KfeslJcrf9DVa1Q1QrgdeD0iJT8MLFqRxkzP97C18cM5ty83hbwjTlclG6Dt/4fVOzu6pIcVsIJ+ouAY0RkqIgkAVcAs0M3EJFcEanf113ADO/xFmCciCSISCKuEbdJeudIpapMf3UlWamJfG/CMV1dHGNMva2L4PEvwv/+AP+9r6tLc1hpM+irqh+4GXgTF7BfVNUVIjJdRC7wNhsPrBaRNUBv4AFv+SxgPfAZLu+/VFVfjewhdJ23Vu7igw3F/OBLx1rf+yNFoK6rS9B5Pn4cHj8HynaEt31NObz+Y1jxCgSDkS9PwA++Uqguify+W/PpP+CpyZCYCide4p7v/OzQluEwJqqHVwo9Pz9fCwoKuroYbarxBzjv9wtJio/j9e+dRUK83ed2yJUWQnImpHQLb/vFT8Ebd8GFf4ITLurUoh1yxevhz2eA3we9ToDr/wOp3VvevrYK/n4pbH7fPe+VB+N+DMMvgLh2fJfLd8GG+bD+bdixDGorobbC/Q7U7N9u1DXwpemtl6mjAn74773wwSMw9Gy47GkQgYdHQv9R8PWXO++9DwMislhV89vazgZcO0hPvb+JzcVVPPON0bEd8GsrISEF4g5hW0bRGlj4G1g+C4aOg2teafs1gTpY+KALiv+8DvZthjO/54LCkU4V/vMDiE+C8x+C2bfAP65wQS6pmZsB/TXwwlWw+X9w8eMgcfDOr+Cf17Ye/INBqN4HO5e6IL9+Puxa7tal5cLA0ZCSDUnp7n2TMiAxDUo2w6InYM2bMOnXkDcl8p97dQnM+gasnwejvwVffgDiE926cT+CN38C6+bB0RMi+75HIAv6B6GovIY/vr2Oc47vFZvj4PtrYM0bsPQFWPsWnHwFTHmk8983NNgnpMDgM10tc+vHLuC0ZsXLULrV1f5W/tvVCPdthMm/hfgj/M9g2YuwYQFMfhBGToWEZBcAZ10Plz+3P/iBO/n983oXtC94BEZ8zS0/4SJY/tKBwb/fKVCxyzWEVha5n6DfbR+fBINOh3Pvg6POgd4ntX6FMPJKmH2r2/dxk11Zu/Xv+LHXVMC6/8LbP3Mn8q8+DKded+A2p90IH/0V5t4Lw8Yf2gpKY8Eg7PoM/LWQ0cv9JB7aiZMsvXMQ7nppGf8sKOTN75/NUT0zuro47Tf/57ByNlz1ImSHeV+EKmz9CJbOhBUvuVxtRm/oPhS2fgjfnO8uoQ+WvwbKtoM2k1v2lcCHf9kf7Ed/E8641f2xPHQS9BsFV89qvex/OQsCtfDdD92yt6fDe7+HoybAZU9BStbBlz1UyRbYvgSOPtfVeFuzdyN8+CdY9gIkZ0H3IdBjqPtMewyFHsOg94mtB6mqvfBIvnvNDW/t33bRE/Cf2+HkK2HKoy4gBwPw0jT3OU76DXxhWtP9BQMu+L//kKvVp/fcH5zSvd85R8PgM9o+vsYCfvjwUZj/C4hLgHPvdSeM5iSlu6uH5k7IVXth9evw+Wvu5OX3QWY/uPRvrlzN+WwW/OsGuPDP7gTUHnvWwbKZsHcD9BwOfU6E3idAt4HhXbEE/C6N9vlrsOo1KN9+4PqkzP2fcb9RMPHn7SufJ9z0jgX9dlqxvZTz//ge158xlHu+mtf2C+o/38MljbD8JVcDRFyQ+cYbkNmn9dfs/MzVDovXusv148+Hky+HoeOhrgr+cAr0PA6u+0/bx1lX7a4Site7mvbeTe532Xaa3v4RIjFtf7BPz92//N3fwrzprZ901r8Nz17karajvr5/+eKn4LUfQK/hcOWLHa95qsITE2DbYkhMh7wLYMTlLr8cGri3LYb3/wCrZoPEe+mOOO/z2AhVe/ZvO/ALrrae0av59/z3Te5EPO0dF4xCLfgVLPg5nHELnDsdXr0Vljzraudjv9/c3g6NvRvhtdvc1UmrBNJyvBOOd/Kp2AWb3gcNQNYAGP5V9zPo9NZPjsEgPHGOu2q5ZXHbtevKYle5WToTthW4/5+s/u5qsV5KN3dS7nmcO2knZXhprXT3GNx3b/XrUL0XElJdeun4891xVe525anYvf9x9iDX5nQQLOh3AlVl6uMfsnpnOQvu+CLd0hJb3jhQ574w7/4WsgfCpU8eGKy6wu5V8PgEFxwm3At/v8yV7br/tFy2tXNdDjw5Cyb8P/cHlpx54DYFM+C178PXnnWBriXBIPzja7Burnue3uvAmm23gQemIupJnMvdZzSTSvOVudr+4DNh6j+af99nLnTHftsyl/oItW4evHit+2M95rwDy9N9KKS2Y7yk+hPq2O9D5R6XRqopc7XQky6Fvie7z2rz+5DcDfKvhy98C7L6HbifmnLYt8ldWb15N6T1gCv+Af1GHrjdpvfgqa/AmbfBl+5vWh5VeP1H8PFjMOA0KFwEZ/8Qzrk7/GPqLKouNVe5p/l1teUhAbFof5opMQ2On+y+h31Htq8yVf95TbgXzvpB8++79i1XGVj7lktl9T7RnbhPugyy+rr/m10rXVvGruWwawXsWesarwO1TfeZ3A2Om+jKe9SE5ttYIsSCfif43/vz6fbm90gaOIpjxl0Jw8Y1DSL1wX7hb1wDVu+TXA05vZcLSn1O6prC+0pddz5fGXxrofsCb3zX9eDIPRaufbVpgCuYAf+5A3rnuZpw4+BUL+CHv4x1l9k3fdT0M6lXXys/7wGXd02OUGpswS9hwS/g2+83re3uWAp/PbvlP3Rwf7iv/xiKVrsaV6i0HJj4y/2575b4a+DR0a6G/+13Xa2zrtrV8pa94PLOQb87sZ3+XXfF0fjk2ZwdS+H5K6GqGC581HVBrH+/P5+5P2XVUjAJBuGlG2H5v+D0m1wD5+Fy1dkV/nGFO+neumR/RUfVXX0u+IX7vDP6wIjLYMQVTb9PrQnUeb2XKt0VsN8HucdBwqHpzh1u0EdVD6ufU089VQ9HtWVFuv2+o7X03v4a/Hl/1XuzVB/or/rP61WXv6RatVd18TOqvz/JrfvrONXVb6gGg6qFi1UfPF71//qornjl0Bc+EFB9/krV+7qrbnzvwHVr3lK9P0f18XNVfeX7t3/r/7njeO5SVV9Z2++xdq7b/v0/Nr9+47uq92W7zysY7NjxNFa11/1fvHBN03WzblB9oJ9q1b7w9uUrV93xmerK2arvPaz61/GqP+utunt166/736Pu+NfObX59RZHqhoWq/trwyhGqfJfqE+e5/f93uvv/mf+L1t8vlL9WdfOHkf/cj0S7Vrnv4X9+6D6Pz+eo/uUs91k+NEL1k+cO7v/oMAAUaBgx1mr64QgG2Pan88kt+phPz32eL4wZBxsXwqpX4fP/HJiD7XcKjL/LpQpCa1TlO2HmVS4/OO5O1y0utLeDrxTWvAWfv+purjn/9+2rZbTm3d/BvPvhy7+AMd9tun7lbJfCGXwGXP4svHobrHwF8r/hGvzC7d3y3CXuTshbl0B6zv7lFbvdlUByJkxbEF4Nt73mTXfHedNHLscKrlH14ZFw+ndcDfdglO1w/d+zB8IN/22+1lZdAn8Y6dI3X3+lc2rS/lqYczt88gwM+6KrrQ6/wDVemvZ59Xuw5DnXGLtjqWvbOvtH7mquufTiEcLSOxFU+9Z0kv73W/7a7Vam3TadA+aBCQZc7nX929A/H479cst/9HU+14C19HmX4/vSz1xj1uevwYZ3IFjnesSousv2r78E/U/tWOHXz4fnLnZd8i75W8tlW/oCvPwt1whVW+HKdsYt7Qtgu1e5lMNpN8Dk37hlwQA8e6HrVvnNt90fWmeoLHa5/eO/Apc87pa9ficsehy+txS6DWj99a1Z9Zrr1z72+64RtLG597iG2W8thL4jmq6PFFV31+0bd7rU2M0FLTfwmpaV74JHToO07q6NY8TlR3Swr2dBP1I+nwMzp/KCfzzHTXuKkYM6eEehKnzwKMz9f/u7J3Yf4vVCuMCdOEq3wNMXuK5pV/0TBo9pfX8bF7reAUkZLmgnejfG1FXBM1PcieTG/7adQ1/8tOvvPPlBOOHCgzu+137gGsK++yH0PNZ1D33nV67b4ClXH9w+w/XW3e6zvbnANX7+7gQYfj5c/FjH9z37VlfLvvZVGHrW/uUlW+CP+XDixXDRXzr+PuHY9olr3G7csGvCV13i/kaO9Hs0QljQj4Ti9QT/Oo6VNT154pg/89DVERwgdNN7rjfF0V9ytd/GNerSbS5glxa6BuDm+jNv+cidPLZ+1PL7JGe5lErOUeGVS7Vj6YmKIvjjKJcqGj3NpXxGXnnQ3dDapXwXPDwCTrzU9b55+2fw7fci03heW+n6+vtr4Dvv7R9O4KVvuRu/blnsUkDGdBEbhqGjaivhhaupDsRxk/82np0U4cv2IWPdT0u69Yfr57j+5f+4HL72DBw3ya3bsw7m3efaFDJ6u1vvB37B6zVQub8HQW0FDBoTfsCHjuejM3rCWbe7O143vef6wE9+sGP7DFdmb9craNET7mR31ITI9ZZKSndpo7+d565mLp0BO5e5njlnfs8CvjliWNBvjirMvgUt+pzv1PyYCWNOY1BOF0xontHLpROeuwReuNoFz13LoeBJd3PJF38KY25q/52Rne0L34aCv7k8+2VPd2rf5CbO/J7ralq9F868NbL77n+qa6R/+2euoX7p867G31JXUGMOQxb0m/Px47D8X/wr+xssKTmFh885uuvKktYDrvm3q+2/dpu7fT3/etf753BtxEtMcSerumqX1z+Usvq5u3Z3LHU3dEXa2O+7G7pevdU1tk/8ZfijfBpzGLCcfmM7lsIT57K3z5mcuv4G7pyUx7fGtSM90llqq1wN9tiJkNuFJyHjGm//PNb1/rhp0SG7+caY1lhO/2DUVMCsb6BpOdxUeSP9stO49owhXV0qJykNzri5q0thwI2P8s233RWNBXxzhAlrIHgRmSgiq0VknYjc2cz6wSIyT0SWicgCERkQsm6QiLwlIqtEZKWIDIlc8SNszg+heD0fjvwlH+wU7vjysaQk2py3phm5R3es778xXaT0pnIkAAAdxklEQVTNoC8i8cCjwCQgD5gqIo2Hl3wQeEZVRwDTgV+ErHsG+I2qDgdGA4fnLMVLX4Cl/4Czf8gzOwbSOyuZKSdHYLxvY4w5jIRT0x8NrFPVDapaC8wEpjTaJg+Y5z2eX7/eOzkkqOpcAFWtUNWqiJQ8korXu5mHBo2hduwPWbimiHOO701cXAwPTGWMiUrhBP3+QMgg0hR6y0ItBbzh/7gIyBSRHOBYoEREXhKRJSLyG+/K4QAiMk1ECkSkoKioqP1H0RH+GjccblwCXPIEH20upbI2wITjD9OeMcYY0wHhBP3mqruNu/zcAYwTkSXAOGAb4Mc1FJ/lrT8NGAZc12Rnqo+par6q5vfseYinH/zv/a7HzpRHodsA5q3aTXJCHGce3cVj3xtjTCcIJ+gXAqG3Gw4ADpjvS1W3q+rFqnoK8FNvWan32iVeasgPvAJ0YE69CFvzlpu+7bRvwvDzUVXmfb6LM4/OJTXJGnCNMdEnnKC/CDhGRIaKSBJwBTA7dAMRyRWR+n3dBcwIeW13Eamvvp8DrOx4sSPknV+5+T7P+z8A1u2uYOveaiYMt9SOMSY6tRn0vRr6zcCbwCrgRVVdISLTRaR+brzxwGoRWQP0Bh7wXhvApXbmichnuFTR4xE/ioNRssWNbT/yKtffGpj3uetYdI7l840xUSqsm7NUdQ4wp9Gye0IezwJmtfDauUAnDjJ+kFb+2/0OGUJ43qpdnNAvi77d2pg02RhjjlBh3ZwVlVa84mY66jEMgH2VtSzevM967RhjolpsBv361E7e/lr+gjW7CSpMGN67CwtmjDGdKzaDfrOpnd30zEzmpP42YqIxJnrFZtBf8Qr0GdGQ2qkLBHlnTRHnHNfL7sI1xkS12Av6JVtdaueEixoWLdq0l3Kfn3Osq6YxJsrFXtBvIbWTlBDHWLsL1xgT5WIv6K94+YDUDsDbn+9mzLAc0pNtegFjTHSLraDfkNrZX8tfX1TBxj2VdheuMSYmxFbQr0/thHTVfHuV3YVrjIkdMRb0vV47OfvnvP3vql0c3yeTAd3TurBgxhhzaMRO0C/ZCoWLDkjtlFbVUbB5n6V2jDExI3aCfjOpnQVrdhMIKuccb3fhGmNiQwwF/aapnQWri+iRnsTIgdldWDBjjDl0YiPolxY2Se0ALN68j9FDehBvd+EaY2JEbAT9ZlI7eypq2LK3ilGDrZZvjIkdMRL0ZzdJ7Xy6pQSAUwZ176pSGWPMIRdW0BeRiSKyWkTWicidzawfLCLzRGSZiCwQkQGN1meJyDYReSRSBW+Xveuh/4FT8y7Zuo+EOOHEfjaqpjEmdrQZ9EUkHngUmATkAVNFJK/RZg8Cz6jqCGA68ItG638GvNPx4h4kXxmkHJjGWbKlhOF9s2wCdGNMTAmnpj8aWKeqG1S1FpgJTGm0TR4wz3s8P3S9iJyKmzf3rY4X9yDU+SBQAylZDYsCQWXp1hJOGWT5fGNMbAkn6PcHtoY8L/SWhVoKXOI9vgjIFJEcEYkDfgv8sLU3EJFpIlIgIgVFRUXhlTxcNWXud8r+NM6aXeVU1gYs6BtjYk44Qb+5/oza6PkdwDgRWQKMA7YBfuC7wBxV3UorVPUxVc1X1fyePXuGUaR28JW638n7g/6S+kbcgdaIa4yJLeGMJVwIDAx5PgDYHrqBqm4HLgYQkQzgElUtFZExwFki8l0gA0gSkQpVbdIY3Gl8TWv6S7bso0d6EoNzbLwdY0xsCSfoLwKOEZGhuBr8FcCVoRuISC6wV1WDwF3ADABVvSpkm+uA/EMa8AF8rlZ/QNDfWsIpA7MRsZuyjDGxpc30jqr6gZuBN4FVwIuqukJEpovIBd5m44HVIrIG12j7QCeVt/3q0zteQ25pdR3rdldYPt8YE5PCmipKVecAcxotuyfk8SxgVhv7eAp4qt0l7KhGDblLt9pNWcaY2BX9d+Q2NOS6mv6SLSWIwIgBdlOWMSb2xEDQLwOJh6R0wN2Je2yvTDJTEru4YMYYc+jFQNAvdakdEYJBZckWuynLGBO7YiTou9TOxuJKSqvrLOgbY2JW9Af9mrKGRtz6m7JGWSOuMSZGRX/Q95WGNOLuIzM5gaN6ZnRxoYwxpmvEQNA/sKY/clA2cTZTljEmRsVA0HcNuVW1fj7fWcYpNh+uMSaGRX/Q93L6ywpLCardlGWMiW3RHfSDgYagX9+IO9Jq+saYGBbdQb9+CIbkLD7Zso+huel0T0/q2jIZY0wXiu6g7w2rrClZdlOWMcYQ9UHfjbtT7E9hT0WN5fONMTEvuoO+l95ZXeomP7eeO8aYWBfdQd+r6S8vVlIS4zi+T2YXF8gYY7pWTAT9T3YFGTEgm4T46D5cY4xpS1hRUEQmishqEVknIk2mOxSRwSIyT0SWicgCERngLR8pIh+IyApv3eWRPoBWeQ2568riOapn+iF9a2OMORy1GfRFJB54FJgE5AFTRSSv0WYPAs+o6ghgOvALb3kVcI2qngBMBB4SkUOXWPdq+jt8iWSl2vj5xhgTTk1/NLBOVTeoai0wE5jSaJs8YJ73eH79elVdo6prvcfbgd1Az0gUPCw1ZWhSBlV+IcsmTTHGmLCCfn9ga8jzQm9ZqKXAJd7ji4BMEckJ3UBERgNJwPrGbyAi00SkQEQKioqKwi1723wlBJNc421WSljTARtjTFQLJ+g3NySlNnp+BzBORJYA44BtgL9hByJ9gWeB61U12GRnqo+par6q5vfsGcELAV8pgSQ3rLKld4wxBsKp/hYCA0OeDwC2h27gpW4uBhCRDOASVS31nmcB/wHuVtUPI1HosPnKqE2sr+lb0DfGmHBq+ouAY0RkqIgkAVcAs0M3EJFcEanf113ADG95EvAyrpH3n5Erdph8pdTEuwlTMi29Y4wxbQd9VfUDNwNvAquAF1V1hYhMF5ELvM3GA6tFZA3QG3jAW/414GzgOhH51PsZGemDaFFNGT4v6Ft6xxhjwkvvoKpzgDmNlt0T8ngWMKuZ1z0HPNfBMh48XylVGa5/vqV3jDEmmu/IVQVfGRWkAZCVaukdY4yJ3qBfVw3BOspIJz5OSE2M7+oSGWNMl4veoO/djVsSTCUrJQERmwzdGGOiN+h7wyqXBFOtEdcYYzzRG/RDJlCxRlxjjHGiOOi7mn5RXYr10TfGGE8UB/0SAHbXJVtN3xhjPFEc9F16Z1dtsnXXNMYYT/QGfa8hd7vPavrGGFMveoO+rxSNS2RvbRyZFvSNMQaI6qBfhqZ0A8TSO8YY44nioF9KwIZVNsaYA0Rv0K8po64+6NvNWcYYA0Rz0PeVUptgY+kbY0yoqA76DWPpW3rHGGOAqA76ZVTF1U+gYjV9Y4yBMIO+iEwUkdUisk5E7mxm/WARmSciy0RkgYgMCFl3rYis9X6ujWThW+UrpVLqx9K3mr4xxkAYQV9E4oFHgUlAHjBVRPIabfYgbh7cEcB04Bfea3sA9wJfAEYD94pI98gVvwUBP9RVUkY6IpCRZDV9Y4yB8Gr6o4F1qrpBVWuBmcCURtvkAfO8x/ND1n8ZmKuqe1V1HzAXmNjxYrfBuxu3LJhKRnICcXE2lr4xxkB4Qb8/sDXkeaG3LNRS4BLv8UVApojkhPlaRGSaiBSISEFRUVG4ZW+ZN9javmCqNeIaY0yIcIJ+c9VkbfT8DmCciCwBxgHbAH+Yr0VVH1PVfFXN79mzZxhFaoM3rPLeQJrl840xJkQ4ye5CYGDI8wHA9tANVHU7cDGAiGQAl6hqqYgUAuMbvXZBB8obHm+EzT11ydZH3xhjQoRT018EHCMiQ0UkCbgCmB26gYjkikj9vu4CZniP3wTOE5HuXgPued6yzuXl9HfX2axZxhgTqs2gr6p+4GZcsF4FvKiqK0Rkuohc4G02HlgtImuA3sAD3mv3Aj/DnTgWAdO9ZZ2rfiz9GhtL3xhjQoUVEVV1DjCn0bJ7Qh7PAma18NoZ7K/5Hxpe0N9Zm8RJVtM3xpgG0XlHrteQu7MmkSzL6RtjTIMoDfqlaHImAY2z3jvGGBMiOoN+TRmBpCzABlszxphQ0Rn0faX4G8bSt/SOMcbUi9qgX5vggr7Nj2uMMftFbdCvsbH0jTGmiagN+jaWvjHGNBWdQb+mjKq4dMBq+sYYEyr6gr4q+Moox02gkmH99I0xpkH0Bf3aStAAZZpGWlI8ifHRd4jGGHOwoi8iekMwlNhY+sYY00TUBv19wVRrxDXGmEaiL+h7wyoX+21YZWOMaSz6gr5X099dm2ITqBhjTCNRGPRdTX9XXbINtmaMMY1EYdB3k6Lvqkm29I4xxjQSVtAXkYkislpE1onInc2sHyQi80VkiYgsE5HJ3vJEEXlaRD4TkVUiclekD6AJL6e/3ZdoDbnGGNNIm0FfROKBR4FJQB4wVUTyGm12N24axVNwc+j+yVt+GZCsqicBpwLfEpEhkSl6C3ylaHwyVcFEG2zNGGMaCaemPxpYp6obVLUWmAlMabSNAlne427A9pDl6SKSAKQCtUBZh0vdGl8pwWQbS98YY5oTTtDvD2wNeV7oLQt1H3C1iBTi5tK9xVs+C6gEdgBbgAebmxhdRKaJSIGIFBQVFbXvCBrzhUygYukdY4w5QDhBX5pZpo2eTwWeUtUBwGTgWRGJw10lBIB+wFDgdhEZ1mRnqo+par6q5vfs2bNdB9CEr5S6BBtW2RhjmhNO0C8EBoY8H8D+9E29G4AXAVT1AyAFyAWuBN5Q1TpV3Q28D+R3tNCtqimjpmECFavpG2NMqHCC/iLgGBEZKiJJuIba2Y222QJMABCR4bigX+QtP0ecdOB04PNIFb5ZvlKq64dVtn76xhhzgDaDvqr6gZuBN4FVuF46K0Rkuohc4G12O/BNEVkKPA9cp6qK6/WTASzHnTyeVNVlnXAc+4VOoGLpHWOMOUBY+Q9VnYNroA1ddk/I45XAmc28rgLXbfPQ8ZVR4Y2lb+kdY4w5UHTdkeuvBX81ZaSRlBBHSmJ8V5fIGGMOK9EV9L27cUuCaZbaMcaYZkRX0Lex9I0xplVRFvTdYGt7AzZrljHGNCfKgr5L7+ypTbZGXGOMaUaUBX2X3rGx9I0xpnnRFfS9htydNpa+McY0K7qCvlfT31GTZA25xhjTjCgL+mWoxLHPn2Q1fWOMaUaUBf1SNCkTJY4sa8g1xpgmoi7o7x9L32r6xhjTWHQF/Zoy/IluWGVL7xhjTFPRFfR9pdR4E6hYP31jjGkqyoJ+Gb54r6Zv6R1jjGkiuqrDvlKqMoYClt4x5nBQV1dHYWEhPp+vq4sSNVJSUhgwYACJiQcX48IK+iIyEXgYiAeeUNVfNlo/CHgayPa2udMbgx8RGQH8FcgCgsBpqto53wBfKZWZ9bNmRdf5zJgjUWFhIZmZmQwZMgSR5qbbNu2hqhQXF1NYWMjQoUMPah9tpndEJB43A9YkIA+YKiJ5jTa7Gzej1im46RT/5L02AXgO+LaqngCMB+oOqqRtCQahpoxyTSU+Tki1sfSN6XI+n4+cnBwL+BEiIuTk5HToyimcnP5oYJ2qblDVWmAmMKXRNoqryQN0Y//E6ecBy1R1KYCqFqtq4KBL25rackAp1XSyUhLsS2bMYcL+FiOro59nOEG/P7A15HmhtyzUfcDVIlKIm1bxFm/5sYCKyJsi8omI/KhDpW1NMADHTWZLXH9rxDXGmBaEE/SbO61oo+dTgadUdQAwGXhWROJwbQZjgau83xeJyIQmbyAyTUQKRKSgqKioXQfQIK0HTH2ejxLyrRHXGANAcXExI0eOZOTIkfTp04f+/fs3PK+trQ1rH9dffz2rV69udZtHH32Uv//975EocqcLp7WzEBgY8nwA+9M39W4AJgKo6gcikgLkeq99R1X3AIjIHGAUMC/0xar6GPAYQH5+fuMTSruUVddZH31jDAA5OTl8+umnANx3331kZGRwxx13HLCNqqKqxMU1Xwd+8skn23yfm266qeOFPUTCiY6LgGNEZCiwDddQe2WjbbYAE4CnRGQ4kAIUAW8CPxKRNKAWGAf8PkJlb1aZr45huRmd+RbGmINw/6srWLm9LKL7zOuXxb1fPaHdr1u3bh0XXnghY8eO5aOPPuK1117j/vvv55NPPqG6uprLL7+ce+65B4CxY8fyyCOPcOKJJ5Kbm8u3v/1tXn/9ddLS0vj3v/9Nr169uPvuu8nNzeW2225j7NixjB07lrfffpvS0lKefPJJzjjjDCorK7nmmmtYt24deXl5rF27lieeeIKRI0dG9DNpS5vpHVX1AzfjAvgqXC+dFSIyXUQu8Da7HfimiCwFngeuU2cf8DvcieNT4BNV/U9nHEi9smq/ddc0xrRp5cqV3HDDDSxZsoT+/fvzy1/+koKCApYuXcrcuXNZuXJlk9eUlpYybtw4li5dypgxY5gxY0az+1ZVPv74Y37zm98wffp0AP74xz/Sp08fli5dyp133smSJUs69fhaElZ09Prcz2m07J6QxyuBM1t47XO4bpuHRLmvznL6xhyGDqZG3pmOOuooTjvttIbnzz//PH/729/w+/1s376dlStXkpd3YO/01NRUJk2aBMCpp57Ku+++2+y+L7744oZtNm3aBMB7773Hj3/8YwBOPvlkTjihaz6PqKoS+wNBKmsDZFrQN8a0IT09veHx2rVrefjhh/n444/Jzs7m6quvbrYvfFJSUsPj+Ph4/H5/s/tOTk5uso1qh5orIyaqxt4p97kP19I7xpj2KCsrIzMzk6ysLHbs2MGbb74Z8fcYO3YsL774IgCfffZZs+mjQyGqomOZz93sa+kdY0x7jBo1iry8PE488USGDRvGmWc2m63ukFtuuYVrrrmGESNGMGrUKE488US6desW8fdpixwulxz18vPztaCg4KBeu3xbKef/8T0evyafL+X1jnDJjDHttWrVKoYPH97VxTgs+P1+/H4/KSkprF27lvPOO4+1a9eSkND+undzn6uILFbV/LZeG101/WpX07d++saYw01FRQUTJkzA7/ejqvz1r389qIDfUVEVHS29Y4w5XGVnZ7N48eKuLkZ0NeSWVVtDrjHGtCa6gn59Td8GXDPGmGZFWdD3IwIZSVbTN8aY5kRX0K+uIyM5gbg4G7/bGGOaE11B34ZgMMaEGD9+fJMbrR566CG++93vtviajAw3YOP27du59NJLW9xvW13LH3roIaqqqhqeT548mZKSknCL3mmiKuiX+/yWzzfGNJg6dSozZ848YNnMmTOZOnVqm6/t168fs2bNOuj3bhz058yZQ3Z29kHvL1KiKvldVl1HlvXRN+bw9PqdsPOzyO6zz0kw6Zctrr700ku5++67qampITk5mU2bNrF9+3ZGjhzJhAkT2LdvH3V1dfzf//0fU6YcOAvspk2bOP/881m+fDnV1dVcf/31rFy5kuHDh1NdXd2w3Xe+8x0WLVpEdXU1l156Kffffz9/+MMf2L59O1/84hfJzc1l/vz5DBkyhIKCAnJzc/nd737XMELnjTfeyG233camTZuYNGkSY8eO5X//+x/9+/fn3//+N6mpqRH9yKKqpl/m89tga8aYBjk5OYwePZo33ngDcLX8yy+/nNTUVF5++WU++eQT5s+fz+23397qgGh//vOfSUtLY9myZfz0pz89oL/9Aw88QEFBAcuWLeOdd95h2bJl3HrrrfTr14/58+czf/78A/a1ePFinnzyST766CM+/PBDHn/88YZhlteuXctNN93EihUryM7O5l//+lfEP5OoqhaXVdcxvG9mVxfDGNOcVmrknak+xTNlyhRmzpzJjBkzUFV+8pOfsHDhQuLi4ti2bRu7du2iT58+ze5j4cKF3HrrrQCMGDGCESNGNKx78cUXeeyxx/D7/ezYsYOVK1cesL6x9957j4suuqhhlM+LL76Yd999lwsuuIChQ4c2TKoSOixzJEVZTd8aco0xB7rwwguZN29ew6xYo0aN4u9//ztFRUUsXryYTz/9lN69ezc7lHIokaa9Ajdu3MiDDz7IvHnzWLZsGV/5ylfa3E9rVxT1QzJD60M3d0RYQV9EJorIahFZJyJ3NrN+kIjMF5ElIrJMRCY3s75CRO5o/NpICQaVihpryDXGHCgjI4Px48fzjW98o6EBt7S0lF69epGYmMj8+fPZvHlzq/s4++yzGyY+X758OcuWLQPckMzp6el069aNXbt28frrrze8JjMzk/Ly8mb39corr1BVVUVlZSUvv/wyZ511VqQOt01tpndEJB54FPgSbqLzRSIy25stq97duGkU/ywiebhZtoaErP898DqdqKLWjyrWkGuMaWLq1KlcfPHFDT15rrrqKr761a+Sn5/PyJEjOf7441t9/Xe+8x2uv/56RowYwciRIxk9ejTgZsA65ZRTOOGEE5oMyTxt2jQmTZpE3759D8jrjxo1iuuuu65hHzfeeCOnnHJKp6RymtPm0MoiMga4T1W/7D2/C0BVfxGyzV+BDar6K2/736rqGd66C3FTKVYCFar6YGvvd7BDK5dU1XL3K8u5LH8g447t2e7XG2Miz4ZW7hydPbRyf2BryPNC4AuNtrkPeEtEbgHSgXO9QqQDP8ZdJXRaagcgOy2JR64c1ZlvYYwxR7xwcvrNjWnQ+PJgKvCUqg4AJgPPikgccD/we1WtaPUNRKaJSIGIFBQVFYVTbmOMMQchnJp+ITAw5PkAYHujbW4AJgKo6gcikgLk4q4ILhWRXwPZQFBEfKr6SOiLVfUx4DFw6Z2DORBjzOFJVZvt+WIOTkdnOwynpr8IOEZEhopIEnAFMLvRNluACQAiMhxIAYpU9SxVHaKqQ4CHgJ83DvjGmOiVkpJCcXFxhwOVcVSV4uJiUlJSDnofbdb0VdUvIjcDbwLxwAxVXSEi04ECVZ0N3A48LiLfx6V+rlP7XzYm5g0YMIDCwkIsbRs5KSkpDBgw4KBfH1UToxtjTKwKt/dOVN2Ra4wxpnUW9I0xJoZY0DfGmBhy2OX0RaQIaH0gjNblAnsiVJwjiR13bLHjji3hHPdgVW1zOILDLuh3lIgUhNOYEW3suGOLHXdsieRxW3rHGGNiiAV9Y4yJIdEY9B/r6gJ0ETvu2GLHHVsidtxRl9M3xhjTsmis6RtjjGmBBX1jjIkhURP025rHN5qIyAwR2S0iy0OW9RCRuSKy1vvdvSvLGGkiMtCbh3mViKwQke95y6P9uFNE5GMRWeod9/3e8qEi8pF33C94I+BGHRGJ9+befs17HivHvUlEPhORT0WkwFsWke96VAT9kHl8JwF5wFRvrt5o9RTe/AUh7gTmqeoxwDzveTTxA7er6nDgdOAm7/842o+7BjhHVU8GRgITReR04Fe4CYqOAfbh5rSIRt8DVoU8j5XjBviiqo4M6Z8fke96VAR9YDSwTlU3qGotMBOY0sVl6jSquhDY22jxFOBp7/HTwIWHtFCdTFV3qOon3uNyXCDoT/Qft4bMPJfo/ShwDjDLWx51xw0gIgOArwBPeM+FGDjuVkTkux4tQb+5eXz7d1FZukpvVd0BLkACvbq4PJ1GRIYApwAfEQPH7aU4PgV2A3OB9UCJqvq9TaL1+/4Q8CMg6D3PITaOG9yJ/S0RWSwi07xlEfmuhzNd4pEgnHl8TRQQkQzgX8BtqloWC9PwqWoAGCki2cDLwPDmNju0pepcInI+sFtVF4vI+PrFzWwaVccd4kxV3S4ivYC5IvJ5pHYcLTX9cObxjXa7RKQvgPd7dxeXJ+JEJBEX8P+uqi95i6P+uOupagmwANemkS0i9ZW2aPy+nwlcICKbcOnac3A1/2g/bgBUdbv3ezfuRD+aCH3XoyXohzOPb7SbDVzrPb4W+HcXliXivHzu34BVqvq7kFXRftw9vRo+IpIKnItrz5gPXOptFnXHrap3qeoAb37tK4C3VfUqovy4AUQkXUQy6x8D5wHLidB3PWruyBWRybiaQP08vg90cZE6jYg8D4zHDbe6C7gXeAV4ERiEm6j+MlVt3Nh7xBKRscC7wGfsz/H+BJfXj+bjHoFrtIvHVdJeVNXpIjIMVwPuASwBrlbVmq4raefx0jt3qOr5sXDc3jG+7D1NAP6hqg+ISA4R+K5HTdA3xhjTtmhJ7xhjjAmDBX1jjIkhFvSNMSaGWNA3xpgYYkHfGGNiiAV9Y4yJIRb0jTEmhvx/ToZ5MH1T4jEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcbe9cc0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Visualize training and validation accuracy across Epochs\n",
    "plt.plot(train_accuracy_chart, label='Training')\n",
    "plt.plot(validation_accuracy_chart, label='Validation')\n",
    "\n",
    "plt.legend(loc='lower right')\n",
    "plt.title(\"Classifier Accuracy\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model\n",
      "Training Accuracy = 1.000\n",
      "Validation Accuracy = 0.973\n",
      "Test Accuracy = 0.954\n"
     ]
    }
   ],
   "source": [
    "### Final model accuracies\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './model')\n",
    "    training_accuracy = evaluate(X_train, y_train)\n",
    "    print(\"Training Accuracy = {:.3f}\".format(training_accuracy))\n",
    "    validation_accuracy = evaluate(X_validation, y_validation)\n",
    "    print(\"Validation Accuracy = {:.3f}\".format(validation_accuracy))\n",
    "    test_accuracy = evaluate(X_test, y_test)\n",
    "    print(\"Test Accuracy = {:.3f}\".format(test_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 3: Test a Model on New Images\n",
    "\n",
    "To give yourself more insight into how your model is working, download at least five pictures of German traffic signs from the web and use your model to predict the traffic sign type.\n",
    "\n",
    "You may find `signnames.csv` useful as it contains mappings from the class id (integer) to the actual sign name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load and Output the Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAH8CAYAAAD14ypOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XmYHVWZP/Dve5fe0unsC9nZkrCDgGEVFAFxxR0EEUfcZ5zf6CgzjgoobriPOjrjhoKgLKLsmxC2hBACBAiBsISQfeslvfe9t87vj6ropb/vTW6nSHci38/z5En3e6vqVNWt5XTVe86xEAJEREREZMdkhnoFRERERHZnqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDK1CvAzOaa2XkVPrvQzC5Pfp5mZh1mlt3BcjrMbK8BznOlmZ2+I+X1W86JZrYq7XIqLPtv+6jC5w+Z2QE7o2yRV5KZXWpmFw/1esiubTCPEzN70czeuBOWW/G+92q0W1emzOw4M5tnZm1m1mxmD5jZkUO9XpWEEF4KITSGEEo7OH9jCOEFoLqT0cwOBnAIgL8kv7/ezJ4ws1Yz22xm15nZ5LLpa83s12a2xczWmdlnd2Q9q2Vmt5vZKVVM+l0AX92Z6yIDk1ygu5MKfouZ3WRmU4d6vUR2JUmFo8XMaod6XWTn2m0rU2bWBOBGAD8GMBrAZAAXAegdyvXaxXwcwO/D33tmfQrAqSGEkQAmAXgWwM/Kpr8QwL4ApgN4PYAvmNmbdsaKmdkwAIcDuKeKya8H8Hoz22NnrIvssLeFEBoB7AFgPeJzcbeyo0+JRbbHzGYAOB5AAPD2IV0Z2el228oUgJkAEEK4MoRQCiF0hxBuDyE8DgBmdm7ypOrHyZOrp83spK0zm9kIM/uVma01s9VmdnH5hdXM/snMliZ/VdxmZtPLPjs5WV6bmf0EgFWzwmY2w8yCmeWS3+cm5c5L/sK/wczGmNnvk6dDC5MTcuv8wcz2MbOPATgLcWWnw8xuqFDkaSirrIQQ1ocQ1pR9XgKwT9nv5wD4WgihJYSwFMAvAJxbYVs+Y2ZPmdmUra8AzewLZrYh2aenm9mbzWxZ8tTwi/0WcRKAB0IIWyu/NWb2OzNrN7MlZnZE2Xr3AFgEoJqnWDLIku/nGgD7b431fwWQnI/3l/0ezOxTZvZs8p1/zcz2NrP5ybF/lZnVJNNuPb6+aGabkqdiZyWfHWlm67eeU0ns3Wb2mLeuyRPdn5nZzWbWibiSPiI59jaa2Qoz+5KZZZLp9zazu5InuZuSc3Nk2fIOM7NHkm34I4C6V2q/ym7vHAAPArgUwIecz0clT3TbzWyBme299QMzm21mdyTXzmfM7H1ln73FzB5NzpOVZnZh+ULN7IPJcbzZzP5rWyu4nWP/XDO738y+m9wHl5vZac4yapP1PKgsNt7iJ9fjqttVu7/duTK1DEDJzH5rZqeZ2ShnmjkAXgAwFsAFAP5kZqOTz34LoIi4MnEY4hv1eQBgcY7RFwG8C8A4APcBuDL5bCyAawF8KVnu8wCOTbEdZwD4IOIna3sDmA/gN4ifti1N1vtlQgj/B+D3AC5JXv29rf80Fj/52RPAM/3i08ysFUA3gH8HcEkSH4X4adXisskXA6BcJTP7MuJK1gkhhK15VBMR30gmA/gK4orY2YifPh0P4Cv28nyvNwO4qez3twP4A4CRiJ9E/aRfsUsRv7KUXYyZNQB4P+Ibx0C8CfHxcRSALwD4P8R/JEwFcCCAM8umnYj4fJuM+Mb0f2Y2K4SwEMBmACeXTXs2gMu2Ue4HAHwdwHAA9yN+ojYCwF4ATkB8E/zw1s0D8E3E58Z+ybpdmGx3DYA/J2WNBnA1gHcPaA/IP7JzEF+nfw/gVDOb0O/zMxG/TRkF4DnEx+TWa/cdAK4AMD6Z7n/s73mjncmyRwJ4C4BPJvcsmNn+iN82fBDxMTsGwJRtrOO2jn0gvoc+g/jcuwTAr8zsZQ8Pkj+I/4D4vCvftjtDCBu3UfY/lhDCbvsP8cXtUgCrEFeMrgcwIfnsXABrAFjZ9A8hPsgmIH4dWF/22ZkA7k5+vgXAR8o+ywDoQvz66xwAD5Z9Zkn551VYxwsBXJ78PAPxI99c8vtcAP9VNu33ANxS9vvbADxW9nsAsE/y86UALt7GvpmcTF9X4fPRAM4HcFTy+9T+0yO+Qb2Y/HwigNUAvo/4BjSibLoTEVfOssnvw5NlzSmbZhGA08t+XwFgatk+urPss/0BdPdb368D+PVQH3P697fv40UAHQBak3NvDYCDyj6fW35OJOfj/WW/BwDH9js+zi/7/XsAflh2fBUBDCv7/CoAX05+Ph/x6+ytx3UXgD0qrPelAH5X9nsW8bVg/7LYxwHMrTD/6QAeTX5+HfgaM29b56X+vTr+ATgOQAHA2OT3pwH8W9nnlwL4ZdnvbwbwdPLz+wHc1295/wvgggpl/RDAD5KfvwLgD2WfDQPQB+CNznzbPPaTc/a5ss8akvN2YvL7385xxJWulQAyye8PA3jfUH8Pg/lvd34yhRDC0hDCuSGEKYj/kp2E+MDaanVIvtnEimSa6QDyANZanIzdivhgHZ9MNx3Aj8o+a0ZcaZqczL+ybB1C+e87YH3Zz93O7407uNzW5P/h3ochhGbET+f+krwi6Ug+aiqbrAlAe9nvIwF8DMA3Qwht/Ra5Ofw9sb47+d/dluRx8JYQQvl+W1f2cxeAuvJXN8l2tEJ2JaeHOP+uFsA/A7jHzCYOYP6BHPstIYTOst+3nssAcDmAt5lZI4D3Ib4Rrd1GueXH3VgANcnyypc9Gfjb64o/WJwKsCUpa2wy3ST41xiRDwG4PYSwKfn9CvCrvv7XvK3H+3QAc7bef5J70FmIn87CzOaY2d3Jq7k2AJ/Ay4/J8vtTJ+Int55tHvv91zGE0JX8SPekEMICxE/MTjCz2Yjf+Fxfodx/SLt1ZapcCOFpxLX9A8vCk/s9kpyG+C/JlYhr5GNDCCOTf00hhK2PUVcC+HjZZyNDCPUhhHkA1iJ+igMASJY/FK2YwjY/jE+i55HkllWQQ1yBbAohtCDetvJXaYcAWFL2ewuAtwL4jZmlebXZ/xVfNfbDy19Byi4ixDmLf0Kcg3dcEu5E/JfsVgOpZHlGJa8/ttp6LiOEsBrx6/F3In7yvK1XfMDLz51NiJ8gTC+LTUP8FBaIX/EFAAeHEJoQv8rYek1ZC/8aI69iZlaPuFJ/gsWtotcB+DcAh5hZNakKKwHc0+/+0xhC+GTy+RWIKypTQwgjAPwcLz8my+9PDYhf9Xm2d+wP1G8Rnx8fBHBNiHMpXzV228pUkqD3OTObkvw+FfGruvK8jfEAPmNmeTN7L+Ib8s3JX623A/iemTWZWSZJND0hme/nAP5z6zvqJEnvvclnNwE4wMzelTw5+QzS3yh2xHrE77m35WbE78EBAMk6z0q2dxziV3aPJk+pAOB3AL5kZqOSvy4+iriC+jchhLmI/0q6zszm7OC6vyVZt6pY3Kz4cMR5BLKLsdg7EOd+LE3CjwF4l5k1mNk+AD7yChR1kZnVmNnxiCv1V5d99jvEeVcHAbiu2gUmT1OvAvB1MxtucUOTzyJ+AgXET0Q7ALRa3I3I58tmn4/49eNnzCxnZu8C8Nod2zT5B3I64j8s9gdwaPJvP8S5t+dUMf+NAGYmieT55N+RZrZf8vlwAM0hhB4zey3iHMCtrgHwVou7DapB3KWMe5+v4tgfqMsQ/0FzNuLz8VVlt61MIX79NAfAAotb5TwI4EkAnyubZgHipv6bEOfcvCeEsPWR5zmIH3E+hfiJyzWIm3gjhHAdgG8D+EPyaP9JxC3jkDy2fS+AbyF+fLovgAd22lZW9isA+yePgf9cYZr/A3BW2V/OkwHcinjfPQEgQnzwb3UB4qdZKxC3AvxOCOHW/gsNIdyBOEnxejM7fCArbWYjEF9Y5g1gtrcjfo+/ZrtTymC6wcw6AGxBfH59KISw9UnmDxDnaqxH/Bfr71OWtQ7xebomWdYnkqfRW12H+C/s6/q9DqzGvyB+kvYC4nzAKwD8OvnsIgCvAdCG+A+pP22dKYTQh7iRyrnJur2//HN51foQgN+EuF/BdVv/IW5Uc1a/9AUSQmhH3CDqDMTH+zrE96OtfVV9CsBXzawdcY7UVWXzLgHwacTH8FrEx+W2Olve1rE/ICFujPQI4ie59+3IMnZn9vLX/f84zOxcxMlxx21v2n9kZnYFgKtCCJUqXIPK4ia+7wkhvG+7E/99ngWIGwQ8ufPWTHZVZnYi4kYc22qVBDN7HvHr+TsHZcVE5GXM7NcA1oQQvjTU6zLYtllDlt1fCOED259qULUifmpRtRDCjr5OlFcJM3s34r+I7xrqdRF5NbK4T8R3Ie5q6FVHlSkZVCGE24d6HeQfi5nNRZyf8sEQQjTEqyPyqmNmX0OcZP/NEMLyoV6fofAP+5pPREREZDDszgnoIiIiIkNOlSkRERGRFAY1Z+r44/ald4qd7QWaLl/H4wav39BLMQDI5/g1ZW1dA8XO/fCHKfbxD3+SYvnbuIua/I03umVn+px1cjM2nFepxtvoThdVOy/gvrIt8QpZhfn7K2Z5uuitp7vTNp9wMsV++YtvUWzewvkUG9aQp1hXR5Fik/eeRDEAWLeK+5irHeYc2tl6Cl3zhwer2xk7STHqoS/Ne/FuzljatlPe0Hu7o/pdZN5KObP72QXV/W3X7aRFrd/Y4k47ZmQTxZpqshQLwSvb2+7dNyXLP+95uzOZzJCeE2d88IN0dLz99HfSdCXn2pjN+MdQxrs0osQx59iKAu+O7h6+PtXUecPDAsUiX9/a2/sotmI1d1Te3NZNsfYtHOvp5vWJKhyq+Twf/00j+J7ZOJyvl9OmjKbYqBFcRk3O/x5CxPu8ro6v1cUi1wuiEsfMOR9bW5opBgCLFyyi2Jf/8wsUm72f032jdwHuR0+mRERERFJQZUpEREQkBVWmRERERFIY1JypnPPOvqaG63MdnZyLVFvjr2pjA7/XPet9nAv1qQ9wh9uZ717M6/jMUxSzSmkSbtqTEzSnzuomxnjzpsxXyXLZXk88xq+ykSvye/hwjT9axthH+H30v336sxRrauBRRR5dcjfFNm/cSLEtLVvcsmv6eLvrh9HA5sg77+aHWnDe+fvdlQwkl6nKfLyq502ZQuMU7W8jxwrOuXPnPB69aeUjj7tFj57Cnaa/6+1voViNkx9YIbFrUHj5TWm7sfFn3/W6xglOsk9Uci5Q7sz+seplgXnpqF7uXBQ5OXaoodjy5Rvcsl9Yvp5iza08BvDmvjqKdRU438q7bRuGUaxQ8PdZTS0vM3JyuLI5ji1Zwds4aQRvy57T/fzWsaOGU+zFxc9SrLWZy8k5N+LGet5nla5Wp57Ceb1Tpu5Bsci5QWa8e3j/abY7hYiIiIhUpMqUiIiISAqqTImIiIikoMqUiIiISAqDmpHb0c2dbpWKTjZ0ket42QoJYG9+0xkU+/S5nGxuF19AsfzKVbxAL4HRSeJOlsohJ9OxWMvJgbb3gVz09Om8uJEjeTqnQzMAwLp1FIqe5YT63Bre7gBONrTI6SjSy9oEkF32AsXqLvgvin38i1+m2K9ynOC5auXVXLZ3rADocDqs6yi2UizXvusloHudQLoNCZzjMlRItay2U1Z3Xic2kBTldOnMvC9eXMvJu6vvvZdidtk17hI7DtyPYk/vP5tiB83al2LeWe/t80r7O03C+M4ZM7Xahg1Da+LIiRTbsJKPg5yTSN3b0+UuMxT4+tbR43Qg6ZRdM5yTqR9/eg3FVqz2G8i08qUI2fxYipVy3Jiq4HxnOePk974ePneyOS95HfDy0iPja2iv0xCgs6uWYu0dHFu9uc0te6/JXPjUiZMp9pZT3kCxfaeNo1hdnq/pDXW8PgAwYgQnq9c2eA09vMR9JaCLiIiI7FSqTImIiIikoMqUiIiISAqqTImIiIikMKgZuV4vosNHcrJYbxcnWE+ffoS7zM9/7p8pZt9yejZf5SRde/mYTgJ5qJRgOswZJfw0Ht3cTjmJixnDo9l7yaBp00OzzkYWl3KPs7iOE3gzC+dzrFAh+d1J2rMWzrzMfu0iip3z1W9Q7LkXllOsue15t+iM8/1s6eSE0/2mcXLp0PMSIJ2pBpBUHqpONN7xHvd3RoJ0j3MMLVq0kGLR7XN5fbrb3WX2PbqYYkvuuYdis/fmkeLzWW4Y4R1r8so67eRTKHbLHbdRLHIaajQ4I2IAQNNwvt5mctwwqC/i2KJHuHHNmk1cdlfBu6YDoY6X2VviY6u3q4Ni9fWcRN7XzSOEDK9voFixQg/oIcO3/a4CJ5uXnAZI9Q3juZxenndLt5+Avmw53xPaW3l7Jjo9pR+07wyKzZjCifxW4ms/AIQc37vchlfmJ7Bvj55MiYiIiKSgypSIiIhICqpMiYiIiKSgypSIiIhICqpMiYiIiKRgO2fYAt/rjptNhfX2cDb96CZuJfft7//OXebMBXdSLH/zzRTLOPXG4I1OkuOWDtG+h7hlh3/9NM8+aYIzZXUtqQaroZD7nXut/h54kGKZn/7QXWamdRMHS7yDQ8kpZ68ZFFv9mc9S7HPnf9wte9VaHkZn9LgRFBvWxMMJXHvtgiFtnlWIOmmHeN9PmiFikiVUGXvl+dvD0z28fAXFHvqfX1KseOW1FKvt869jvc6IGsPmHE6xo7/+dYrtN3sGxbzrSKWvptpL69C2D+TSs9maoV2lEo9ZVfQu1k7L654KLdg2bOym2G13PUSxG//6MMVaunj4lu4itzbrK3GrPQAI4OtOoZfXs76W74XF3haKHbg/D300YRzfMx9/4gl3fTZt5laDyDVSyGq4dWJnN693NssnWW2uwrBjJW51W2ccG1XfSbETjplJsTPew8POTJvM134AiCIeashvncvbk8vWbvec0JMpERERkRRUmRIRERFJQZUpERERkRRUmRIRERFJYVCHk+no6qFYiLg+d/yJ76DYrLw/jEn2Vk5At8hJVvQSzZwExuKBPGxN9vwv+GU3cGLhUCaWV8tLZg7e7jn2KIoVJ3/LX+aXv8zzr19JsayTSJp/noeJmXQPf69veetZbtm33fwbiuXqOIkwMh7CYfexMxqKVDvsjDPnQFbHWWRnoUixJ+Zz8m/vHXdRrKbbuRYccbBbdOmppynW9TAn5j4x726Kzdr7wxTL5DkBNwT/b1LvvPeS8SsNV8UTVh2sOqvdhjj93dNX4OTjkOV9XHL2e1fR356/zn+SYrfe8xjFmrv4GtHWxbfJYHzt7yv4ZedzvJ61tXx9ykacGD5j6hiK7TONk8UztoViRx0+yV2fxYufo9jqdTzMSybwNtY6ydkl8DkRRU7LDwClEifum/EQQM2dvD73LOQhxkZNfJRip7+F71sAMLqRh4nJOueeN4xONfRkSkRERCQFVaZEREREUlBlSkRERCQFVaZEREREUhjUBPR8noszJ8ntjDPP5pl//WN3mZki9xoLJzExyjiJn1Nm8/L+/d94HYd5ieavRK/Uuw5zMtCd/HzU7LmnO3/xoq9SLDj7MrRtdsrm5WX/fAPF3vu977plP7HorxTb0LqWl+n01DvUqu/t3Es0HkgWeJoE9gH0+l1lUvuip5+hWPt991GstHINxYp1Tg/HJ5/grk/OWdHcw4sp1nrrPRR7+pjjKXbgrL0oFrwDGK98D+j+vq2eX87gjYBRrXwtJ4F3O6Mp9BT42v/w45ykDAB3zV9KsfXtvEe6ig0Uszwfb1HE94Rc3n82kTXnflTkxlgjGnl9Zk4fTbHGPCfo53IcA7whPoCD9xtPsby1UWz1Gu6ZvMY4gbyQ4++rWPTLDiVONs/WcGJ45FRNmrv5WL1zHjcsaBzDZQDAqcccSrG+Nt7ul17ihlPHHP1ad5nl9GRKREREJAVVpkRERERSUGVKREREJAVVpkRERERSGNQE9MZ6TlQ76MDXU2yPUhfF8kufdZeZAS8zOJnTUZaT3PCJT1Ao28QJdsHptRsAFj7CPTbfe/8dFMtnayh20hveRLH9Zh9IsYyXBV4pZdXJeC2WuHfaRYsWUGzZc09RbOa+nKB/+GuOdovO7MWJ6aWPf5on/O43OVbkdcw4sZpr/+SWfepp76LYNdf+gmKdvdzr9lBze6N3k9IHY23Si5zekNs6OVF42Tw+Brvu4CRw6+ZzL3sq93BcqOfzFgCio3hEg56HH6dY3QLuSXnp/fMoNmtfPs5zWd5mAAjO36peb+fmJAp7Mf+0r/T3sHNcecnmu17+OUrOPso41++WZm7Mctf9i9xlrm7hc7+7xD2JFyInQTrPx1ZvByeQ1+YrpPgHTg4fVsc7fr99J1Js3Ci+RWcznBieNe493b93AONGcvJ8zQFTKVbqW0extZu5bDiNyCzrVy36vF7r+3hfRBHf10sZ/m5e2sDb/eAjfiOE5lXcmGXt89wwwYzPPSWgi4iIiOxkqkyJiIiIpKDKlIiIiEgKqkyJiIiIpDCoCeh1tZyodvIpnIhtczkRFSU/eTg4PXd7SZ7RHE6czh2wr7M8Tob701/+6Jb9nR9+m+cvOT2tO7mkf7yWl/nl/7yIYqee/BaKVUpGLkWcCPvtSy6k2J9v4ETuotPDsNd79Cknn+qWfcFXLqFY9uSTKNZ3x+0Uyz/2CMXMSWTOzeeEYAA4/D3vodi1gZP+i4UWd/7dlXesDoSb/O6VU22H7AAiZ5n3PfIYxdrvfoBnXssJxZlR4yhWc4TTk7G/OsiMHksxmz2TYj1LllBs8613UezZ4/k6MnvWjEqlO7Fqe7Kv9rsdwDGQ8ngZLFHgkQp6i7wvH1rEveg/89wGd5kdPZzcXQBfIwrgsgu9fC3K552GT0XuTRsAamt6KXbArGkUmzSRb8f5LDfGiqJuitXV8Lw9PVxuvACOjxzBSfaHHT6FZ33seYqtd5LSM1nePwDQ5xyvUZYTy83ZnkKGGyG0dfJ3uPRZ/3sYXzeBYqNGc8ys4M6/PXoyJSIiIpKCKlMiIiIiKagyJSIiIpKCKlMiIiIiKQxqArplOLnvsEMP4un+cjXPHA2gC2hzNuvU07wJKdK2hZOUf/KzH7rFTBzDCXqf/3//xaU4q/6dH36NYt/7ASe0zznyWIqNHDnSXZ8FD3GC9vU3crL5oQfNodh73/F+is1bcB8v71a/F/LDD+ceYt95+pkUy5zBsfAEJ6BnSrzTsn1+YmDTU09S7KDDeH3mPXCnO//uYGfkDrsJ7F4P3W7Z/gptaNlCsbUPL6ZYzz18rHr9iNeeysd/X94ZzaDoj1LgNUbJvN7pQf3JpynWuXAhxR5/iHvYnr3PXm7ZyPI6ZSqNXuBMmY733e4eCejBGdVig5Pk/Nhi7um6pdU/DnpKfE8oRnwchcBlR07jnJocN4iyDPd0DgD7zxxPsamT+V6Yz/P8pSInm1vgM6XQ4523XAYAFHq5uUbWeP8Ob+LE8EMP4Z7Slz7VTLGX1js9pQPIOqMFlLJ8XHrXgqJzTGcD92K/udkfkaClhe8f+0zie2lHJ29PNfRkSkRERCQFVaZEREREUlBlSkRERCQFVaZEREREUlBlSkRERCSFQW3NN3Y8d90+LuvU55p5WImQ8VuimFMfLI3hISiy+/PQMV4ruzWr11PskAN5CAkAeNc73k2xY4/nlkJeq8HP4XyKffb8f6bYw4seotgbTzrFXZ/7H7iXgxlunXLx175JsfHOd3PSm95IseeWL3PLvvmWWyjmtuY77BCKRRMm8XSrXqSYlfxWGpkH5lNsvzefSLG5993qzr974OPfGw6m4txVNgf0pvIagXktawBg4cJHKdZ211yev80ZgmL8HhTLH7A/xfoip8VWhe0z59yLxvPQIg2z9+HpnuAWfs0330ax5Ucf45a95z6T3bhsW3C+s2eWraLY88v5Wh1hmLvMyBleqhTxvSMEjuWce0+Ieii29zQeuggApk8ZQbFhtdxKD84wMavWvESxaZO4RV2h4FwfMv7t/cUV3Apy/FS+BjeO4nlHjOT9e9DsGbw+gcsAgGXr+N5eCLwvoyy3tCwV+LupAw9R19fj3yc2bOig2N57jKZYb4VW49ujJ1MiIiIiKagyJSIiIpKCKlMiIiIiKagyJSIiIpLCoCagT5+6J8VKL71IsZrITYN1l+mOiDF7Pw7mOAHRs//+PO8Pv/eDClM7XfhXmRQ8YSInwXqaW3h4m0q2bOGhPLJZHlJg3DhO0PfWO5vl5PVZs2a6ZS94aIET5S8nk+Nllg4/gmddvYKXlvHr/uHppRSb+iFOfm9p63LnH1re8BdVJosP2uggvI7L129wp1yxgIcG6r73YYoFHo0DDafw0DHdeT5eMsbrEyoMN5Vx9mUUnCFmXnckxTqcBPTee7ixw8IFHAOAPfd5rxP1kued49rZxp3BS/Yeaj0FHu7kmeUrKbapg4+NHmd4EQAoGE/rDVvjjACEWvRSbPRwTnKeuddwt+yGWr7u5HJ8XC5/kROxX1zODTUmT+DvLOd8jdm8vy9WbuKNXNPZRrG9ZvJQK5PH8xAzDU2csD1hD38omxc3cZJ95AwzVwQnoHvnrdewJjjfNQCs3NBKsQP6nHvhDp4TejIlIiIikoIqUyIiIiIpqDIlIiIikoIqUyIiIiIpDGoC+uhxTk/Xm5t5QicBvVI+ZpR1EkwnT6dY3skpq74H6XQ9TXuxG2++3pmZQ7Nmzqq67HyeE/kip7fo4CX4V1mtNvMn7O3h5ElvgyIvaX9/7uUa11/nlO0fBNkiJ6yOjjhBtNDrZD3Ly2QC7+Ne53B58EHumR8Aeu64h2K5Xqe35715RIL8LOdYLzo9mOf5GIzg93rs5vJ7x/9U7q08uy83mAnLuGfnTTf91S16+dGvpdiee09xpnTWZ5BaFwykF/3BsqWTk66Xr1pDsc5CA8V60eQu0+vTOutc9HLOfs9FfH0ZP4qvtWOa/NtpLs+lr3V6An98CSeBl3o44bvo9AReU8NllOAngRcyYyi2ZgNfGwvmJIsXuBfxCeOchPg6/1pdLPAyczWc6B4iTiLPGW9P1r2q0XHdAAAgAElEQVSv+2V3FHmZXSVOdB9ey8dVNfRkSkRERCQFVaZEREREUlBlSkRERCQFVaZEREREUhjUBPRhjdwja9TByYbm5V5Wysf0ejMeyT3RDlai5bXX/ZFi1/zpSootW7aMYm844U0UO+jAgylWaVvqGzhZMQqcmNvbxz36NuS8Q4HLqa2tc8suljiBMTjJzBmnd9rSeO6F1s/J9RMLM06ScqMTK6G6XvBfzbw07qdeWk2xtnmL3Pl7Fj5GsaJzjjaedDTFup3GEpksJ//mCk5ibYU/C73GH5HTOCEEPi5HnngcxdY/zedt3733u2U/Nn8hxWbsOZVi5l7wBoe3f4Za2xbuMbx5Myc+R0XnWpTxb2nmdLlvTm/0bnsFp9FNVzcfQ1HwE5c3N3dS7IknOaG+tzCaYnU5PtYzxknTGeeWUHJHEgEQ8fb0Oq1M1q3nHsOLPZwkn81w4426uvFu0bncWor1OAn13pU+74ye4W23d34DQHc3X0vaWvhYG9akHtBFREREBp0qUyIiIiIpqDIlIiIikoIqUyIiIiIpDGoCel2eE8hQcpLFSk76WaVESScDzdxk6sHR08M90ba2cpKbl+mYyVTXe3qlpNE6Lznc2ZV9fZyI19AwzFkil9PY6E0HFAq83YUiJ33W5Pm7yY7kXouDU7ZVSqh0wvkeTrKvr+PEzaHn/T3jJVB62+4nSrp7yZnU7e28yHM/eD8nm7fffpdbdnuR9/uI2QdQLHISsbPOZgfnWlCIuIxs1rm2AMhknKRVZ2fknQT0aMY0jjk9mEfPrXDLXnnT7RR78fhjKLbn1Ak88yB1TL7r9X8OtLfxaArdnXxcZjN8LSoV/ecD5oyUYVWeU17jhOA0Zuns8Ru4LH58HcWaW3naXiehPuP0nt7Xx+vYVM+J6iWnsQ8AZJyexL1GTcUSz7+5lXswf2IJJ9OP32OGW3YuO4JipR6nEZkzmod323PvExUaaJmz3b09PH9mxI41VNKTKREREZEUVJkSERERSUGVKREREZEUVJkSERERSWFQM7U7e7gX28wYp6dUt6frSj31crJZtIXL8VPxXnlnnXk2xc543xkU+8Uv/49iv/rNzyh219w7KHbySdxTOgBkjL/O4CRte8ni1co7iYEA0FfgpPZSwekV3ZvdzSx0kggrdtbsdpdOoUzlBQwhL1nSi1W/7tWmpRecCR9Y9DjF2uY/wLGlz7ql5JxLSu5k7u08E5xLj/F5b87WZHJOj8lO7+lxnLPas1kuu+TM73TMjFFvOJ5irc+ucstum7uAYo8t4Nj0aafzOjq9ou+M3sp3xTOis5MT0Nu3cAyRk0jtHVcAMuD5vR7QASfx2bl79JW4nCXPcKI5AKzb7CW1c6ODEnjkjhD4XtbTy8d0seD05p7zv13vMCo5vaJHxonYUcTbsnod9/De1rXJLTuKuJf4cWO45/f2Tk50L5X4vhU514ysc30AgFyOGyB5+y0qqQd0ERERkUGnypSIiIhICqpMiYiIiKSgypSIiIhICoOagL5pPSdqRnsdyhM6SWWVekA3J3EOG9ZzOc78bupvhd5Tq+XNn8txUuOHz/0Ixa686rcUm/8gJ/+efNKpbtnZnNOLs5Ni6iXresmt3rZ4PUoDQOT0ZO/tc+9rLDa3UazWm9dJyo0/4FCp1untvG7XS7f19od/CHpJrAMphyfucpI8F97Lx1v3bdzbedHrpB3AsCO4t/PSaKeRiXHjBK/jd+8YLHnHWoUE9EyGrw+FAicj19Rwsq05icfRPvtQLDtjslt2z4svUeyFP3ODktVHH0WxaU6v6G5/3dW3y9mN8MrX19dTrI07wkdwGy8BWec2Yc7Oq2/g60b3Fh7BYpPTE/jGVr9hTwSn1+/g9N4O7gG9L+JjNTi37WzGGYEi8tdnRFMjz9/sJHxneH2C8f7J5jipvLvPf04TOffr+jreFz3dvO5ej/VeAnrw6g8AIqcxlnePytf4jay2R0+mRERERFJQZUpEREQkBVWmRERERFJQZUpEREQkBVWmRERERFIY1NZ86zasoZhNmUaxKMOrlSlWaM3ntYZ69mlepjOshNcyraO9nWKr13CrHACYMH4Pio0cOYrXx2mRVOO0GPBaEXR08HAClZrqNDU1OVPyDurtdYZm8AyguZjXUqJYdFpsOetj3v4NTtMuZygDwG+p1uEMe9PXxcMeDDWvRVFwhtLxW5n6rVZcgf9uevDRxRQrPfEYxXpXvEixQi23rgKAxuN5uJU+4/MsuA2NqmtR6qs0ndMKMsuxjl5uzVTvtAjNOtem4a/n1ngAsPk3KyjWPpdbSy566BGKTZvEQ0ZFWT4ncl5rZgAhZavkoZTNOucEeLiqYolb2Znz/QBAxtkf5rT0DM455Q1fVHR2b1ThdloCH0cl8PkTOUPZFJ3z1hv6KDgnlDe8WBzn48i70pec1qwlb8LA1/naPLeOBYCol9dz88aNvEjnml5yvpvIvX66RSPvbGVtrTNUUB8fV9XQkykRERGRFFSZEhEREUlBlSkRERGRFFSZEhEREUlhUBPQVyx/gWJdI7ir/WFecmuJE8MBIOMkycFJmLVNzTzd+HEUamnZRLHzPnmWW/ZxR7+BYhdf9G2KZbO8mxcuWkCx1pYWiu05fU+nZD/Dbq89eagLL5Hv+huupdinPvn/KFaKOLHw/vvvccseNYoT74cN42EC3HVf/DhP5Q4LUWGYgDoe9mBNxGNN1E3kJNbdWeXkbN7HWzr5/Jl3B3+XPXfeR7FuJ4l71BFHuCX3jubz2Wsk4g0JU20CupmTOFxhGBFv/qyT1JvNccKsm5Tb5zSq2GeWW3Zm0iSKFVfxkFrP/vkGir105GEUmzp1LMUiJ1kbAMxpxLC7qG/g62VtvdfSiBsNZDJ+w4iC0xgmU3KSj4vcOKfkjHOUcY63QoXzsegcryWnUQace1nRadjTV+LrWBSca1uFZHzLOAnozknqFI0o42yj90gm8q+1Gfescr4bZ59lnGM6OIVXOvQj5/4xrJHP+yjwfbgaejIlIiIikoIqUyIiIiIpqDIlIiIikoIqUyIiIiIpDGoCelc39yz6zPLnKHboAZzQmXV6Ca4k28vl9N13P8Xy734nxaZMmU6xOUdyr84AcMddt1Fsw4b1FNtjD+4p/Z77OPm3oZ57MH/zaW93SvYz7I54DScFz551AMV+edkvKfbIYt6/7W1bKPbM08+4ZX/0ox+nWC7HPfqWupzeZedzMr6521ihB/S9OEl/6So+rrr6qu/RffBUuU7OZF7SKOAnfD/y2JM8/yLu7bxj02aerp4bEtQe7/f6jcjrNZkTXiMngTfr9Ejt9faccXs99vdFX8FJhM3xOnrHW8bp7nkged21r+N91H4FN/7YMvdBij10P/eUPuV97+ZCKiSge/sjcval1zhmqNXW8XFQ3+D0og9OFg9eIjbgf3HOrisWuYduJxcaJacX8VKFZxMlp+iCd7w58xac76zkJaA7DW7MaVQBAJZxelA3pxznGArOzrCMN/qF18AEqHH2hTdKh9uNufMdRk7SfnAamABATa0zesEIbrzU26Ye0EVEREQGnSpTIiIiIimoMiUiIiKSgipTIiIiIikMavZhX4F7Or33Pk7iPuQ4Tvi2hx/1F+rlqTmdIduNN3PwtFN5uoYGin3pPy50i+7r4e158CFOdH/8CU70HT1yDMX+8z8uoNj06TN4HSv0tJuv4YTDb339uxS78Ktfpthjjy7mcpxExXe/871u2ed95FNOlNezcPvtFKtpd3q3r7K3WwDA0cdS6PHH51OsLssJ8UMtBD6GvORhc3pMrpQM3dnJibnzb7uDYqsf4MTn4CSLjz9mDsX6cpy4CQBZZ3vatnBDhuYWHpFgj4ncUKO2tpZiXmJsxkleBwBz4l7PzoU+3mc5p7dn79wLeT/RNz9rJk87kXsxD2vXUWzp7XMptuYYvi5OnsbXkXihXsL1rtgAg61b8xLFGht4H5eKfFyFrN8TPoyP1+B8v5FzzfMOLS/ZvNL1KTg9qCPj9N7uHJheh+M9fdwgpOQcl/msn4xfm+F43ukNvhfVjQqQjfi6WqkH/pJzfegocvJ8NuM0EjEup+DUKeqzfi/4jcaNCxqd5Pl2Z19UQ0+mRERERFJQZUpEREQkBVWmRERERFJQZUpEREQkhUFNQPfyH+fN48TY8z74SYplm7h3cADIt7Z5JfH861ZRrHj1nyiWOecsio0cOcot+4ff/wnFnnuOe97u6Oig2MyZnJza2NhIsUrJ5j6edurUGRT7xc8vpdi6dZwEW1vHSZujR492S/YSE4tOYrld8XuORdUly4asX/ff8trDKPbwtT/iCet3vd6eI6enYPMSY8GJltminyh5y+13Uqzz7vt4/k7+fqJR/P3WHXkIxfIVcpmfe+kFil175R8o1tPFCbh77cPnxDve/S6KeUnplc6TqMj7zesJ/K7buSHM0qVLKHbAgQdT7I2nvsktu5TjdRpx/NEU23LNDRTL3c2NAx445V6KvXkUN6IBgIZhnITb5Yw+UFfL53hNw3B3mYPlmaeeotiEMRMoVpfn7emu0Pt1rxMPzlXLG1XAO14i864l/jFoTm/pOeN1z2Y5ETvn9PDdU+D7RHBGm8jkNrrrU5fj7akpcbJ5Fl6DHZ4374wUUKEZAIrOeRp5jUScpP2Mcw3MO5fAbMTXFgCYOprPienjRlBsxpHeqCPbpydTIiIiIimoMiUiIiKSgipTIiIiIimoMiUiIiKSwuAmoEecVNbatolit93NyaDvfPs7/GVedhnFzOlJ1rwU6T9dQ6HiAbMoljn8CLfsjNNL68yZPL+XG+v1cj1YMlle70mTJ1c1b6X19nrOLv70vylWu3atNzOFSk7PtIVjj3PLfmDl41z2ME7w7HWSvYdasci98ua9ZFIn+X75amdfAljy17sp1rHoSYpxuisw/kTuZTtyeo9esZ4bdADADVddzeU4ic+e5c8/S7E7buWRC046hRO+6+t55ALAP/fuuoN74V+4gHvM9w71Bx94gGLZrH8ZfePJJ3PwwNkUytzDjQN6Vq+k2Au383TP7L23W/aMPfl89nqDLxU4VXiYvysHjZcUX+vcqkY18Yp2t/D5BAAwp1ds7wuuduSFEu+3Sld0M57WnBTtrJP8DmderyCv0UrwhgIBkHMaRrjr6CTOw405+7zS8AxOUrs5Pahnsk7jgBJfR2pqnMZmETf4AoDJk8ZRbP58vlYuXswNXObM+Td3meX0ZEpEREQkBVWmRERERFJQZUpEREQkBVWmRERERFJQZUpEREQkhUFtzRc53c6X8lyf++Mff0GxN/7kj+4yh9/Jw9Hk13ArJ6/WaL09PN0l36VY8ctfcsvOHnggL9OckryhUZzYwIaO2bX0Xc7DxORv4+/G28LgbHfIcYvD6Iz3uWXfctn5XHaOW2Tke3a94WRat2ymWGO9M6yQ0wLzhptu8pd5F7f66uvjtnuNU6dybBa3Rr373rsotmjhw27ZvT18TlXLa3205HFuqdndxWXM2v8Ad5nPP8/DOy1b8oRTdrWta3m6+fff407ptbB67dE8nEzjMXMo1vaXWyjWde8Cij1+zJFu2cMa+fjfa58ZFOvp6nPnH0qdnU6rrboxFNtrGg8xs6Gl2V1mxvl+I7flnneFcoaicZbnNEAG4LfccwdcyfFwKSVnHYsRt56LnHOnVKE1X9a554asM63x+gTj64jXkhsRD08TT8xxi/jaVuMME2NZZ+itIl8/J+/htNwEMKKJt3FjCw+ptcZpSVsNPZkSERERSUGVKREREZEUVJkSERERSUGVKREREZEUBnc4GXBWWdbJ92tuWU+xy6+81F3mpz/1KYpFF1xIMevjpD0v1TDb1kqx4CwPAKKPncfzn3yqs1Cus+5qyeZeQmWxu5NjP/+5O3/t9TdQzIreUAjOvnCWF73zPRS74TlOwAWA55c7Q5sU+Fhr2VzdsCaDqc0ZTqm+lodXWLLkBYqtup+HNgGAwjJOui7k+FTf46QTKVZyjtVNm3gd0ySaD4R3XD7/7DNObNlAlppijVgU+Ym+D9zDift1zrA3x8w5nGKZex+iWOfK5RRb7SSlA0B03DEUa3OSbbs6t1Bs7BhumDCYigVOfM7lOfF53KgRFBvT5GQuA+jZwvMXnfuRN3SMG3OGLAtuormfmO4lpUcRb3exxFfHviJPV/BGeXGGZInjvN2W4WXCiQVwg4XIa7zhDBEDAJnI25eclF50GofVZvk7HFbLsb2mjnXLbmtZ4ZTN50Qms2ONMvRkSkRERCQFVaZEREREUlBlSkRERCQFVaZEREREUhjUBPTaOk5Ky+U5ya2zl5Pzbr3janeZhx3BvQcfffZZFLPfXU6xTMnrudXpmXwLJ2kCQPYH36dY6S7uDTmceSbPezD32JzJDs7XUezm5L7w4DyKRb/9LcVql3MiNACYsyu9PN/ISbwszd6fYqve8DqKXfmNc92yezdz4ZubOXm+xjn+hlrrxhaKjRwzmWI333w7xTbfxd9ZjPdx/X77Uiw7nnuQLjmNBhqcpOnhTU1uycUifxdegrbXY7M3UoA3r99beYWkcm9ap1dpd5SC6kupwGlw4/Tsb8YJuPnjj6JY4S/cyKN7/kK35Hvv5uT3k044lmLDhtW58w+l+nruwbq30E2xhvwwiu07jXtKB4DWpzZSrKvAzxKyWafsktcdt1uMy5yJzWmI4/W+nq3hY6Ov4JwTcHocN/96l81xvLfI94Sohs/lYuCGXDnnfMrn/Oc0Red8zjo9m+edG0robaPYvrP4+x413G8IsN5pqJQDH1fZ3I41UNGTKREREZEUVJkSERERSUGVKREREZEUVJkSERERSWFQE9C3dHBScH29kwRonPDX0+f3uPyD719AsfHf+gXF9nn76RSzv/yJYhm31263aMDpNDb36CM8+xNPUCyaNIUXt99sXsepnIxsNX5iYdTNyXR44UWe/3Fen+zmzRTLOd3qBidxMuZ1wcuJidGkSRTr+sJnKfbt7/47xdpb+fgBgNJwp3f7dv4ei143wUPspr9eQ7HSrbdSrHU+9/q9vpmTagFgXA0njI+cw8nHHU7SdcZJDH/Dm06m2OtKb3DLNi/30zl/ck4idjbDx1ZfL/dwXPISyJ1rBgAUnUYmJaen6ZIzXXCS6b3t6yvx8Qe4hz9GjeHemQs5nnD4wQdRrPM+btzS/tKLbtnrH3qMYptfw41eNm3m/Ttj74PdZQ6WEcO5Z/PmNr7+v7BsCcWmTufe5AFg84Raii1bsYFifc51LJvhRHc/udu/nZpzTmXcY5jnLxSckSmKfKxnsnzOh+AkpQMoOb2Qm9P4yV1Fc67/Ea9PodI5kfEaf/B3mzUerWLSFN7GKRM4tnk1XysBoLuDG/sg4nKamvj4q4aeTImIiIikoMqUiIiISAqqTImIiIikoMqUiIiISAqDmoCez3Bvu329fRTL5jhhL18hwbStnROnz//8Jyl2yXd+TrE9G3l9sr+/gmMlPwPd3C6+nR7U+zjhNeMkjtrKlygWvCxWr/doAFmnbL8bZ57fSl7ivTNdha5/S07ycLTXnhTr+uL5FPvGjy+mWFv7Oop1NPvfQzHHSaON4zhBtKHBSSQdYvPuXUyxfSdyj/BLFiyiWG2F47L+EG7I0FfD509DyUnEDnxJKDm9Fjdk/EYQXV7iqXNsZLKcHOslwWadS5SXJF9XywnGAFAo8Ppks7wvikXeFzlvfbx5K5yP3sXVHeUg4nOqx/m+muYcR7GWW7lXdADoePAhij24/94Ue9vpb3bnH0ozZ86i2LXX/pliOWcPZ4tOkjGAQ/bhnrJ7e7hH7Rc3d1Cs5CSb9zrfmdfjPQBETkOPrHfqOueU1yt6KfB0UcQxy/m92+eyzkgkOT5PSt69LMf7PPK+h6x/n8iZ06Ckj0cYyeW5MdWsffn4XbPycYq1bfBH6aiv4bIz4O3e0uIfQ9ujJ1MiIiIiKagyJSIiIpKCKlMiIiIiKagyJSIiIpLCoCagl9xeUTl5Mzi9pHZ2ON2NAyg6GdZb2tZS7BP/8kGKffmL36TYcV/lZOjw0/9xy86u5XLM2R7zEvnc3GFnXnfCSl2yMy+x3MmHdJPNg9Pdc3B66QWA4ltOpdiWM95FsR//71cptmj5A7zALNfz67kzeABApoaTmQtdTnJ10e9Ffyht2sDrVLuJe3bu6uWETKvQG/2jTg/3+Sd5mUWnN+OejJP4nOfpZtXUu2U3OEnxkZOMWnCSbbM5LqfOWR84sSjv9/accc6VkldOjten5DT+KGX4+M85CfYAUPQaZTixQuT0zF/H5dT28fnop74DXc8vp9iaJ5dSbOl+e1Fsv/2OqLDUwfGzn3NjoZFOr9R778kNLfJuF/xAodBOscNm8wWl9Dw3aHpmPfeSnavh5O4S/EYZ/iXcOS6LnCBdm+XzrKebj5e1q3m9YdyIBwA2t/K6RxE34HBHu3Cu/1lnHRE5o3EAKJRaKTZmNE933BHcW/+sKU0UWx7xcdHX4p+PeefcLZX4WtLbw43iqqEnUyIiIiIpqDIlIiIikoIqUyIiIiIpqDIlIiIikoIqUyIiIiIpDGprvshp1dZQz6vQ67T6q5RgX3Raq9VkOdbWvZFin/+Pf6HYu97+AYp94htfc8tuWrCQYtm/8PAOtoZb/cEZoqMzw600GiKu74Y6f+gMdHPLsJ6Ms8+dBi9R3mmJcvjhFCqe9X636Ic6V1Ls1z/4CMW6mnkIh+C0sqgZza0sutt4qAcAQLvTKq3gHBf+CAdDixvwIFfDx0Gd09oyqtCWKxR5/i2BN77Paa1WdP6+6uricjrAQ0AAwCzjVnVjjFtWZp31AXi9vRZF3pBGhQrfbXBa83nDewT370qeLuOU3eusNwBEzvAiXutjr51y5EzX7pRdqTVfn9OqLffsCo5l/FaQQ2nKpCkUG9k0nGLdnXwMdvbx9QUAir28P0o5bh32GqeFn+U2UezF9TzkSE+hwS0bgVu7BacVmTm3Y284sXXN3CKu4zFuzVeK/NZ8yIykUK+Np1jWaVXvtcj3hjGrzfs37LETeB/N3JO/22y0gWJd7dwiMxvxBfQ1Bx/klr34cR5iyRsyp1RhmK7t0ZMpERERkRRUmRIRERFJQZUpERERkRRUmRIRERFJYVAT0OvqOekul3eGhnASyNu3+EmeJSfZNnKS0nPmJIMWucv7y674NcVuve0mt+yPfeocir3+q1+i2B7tnBgePbKYYn2rn6dY3XpONszk/K+tVHLK2YMTC6PZ3FV/z6EHUuyxVU9T7Jo/8HA7APD0Ch7CxIpO0nSB6++Zek509EaFGD96rFv26md4H21Zx2m9TSP94R6G0rpmTrSccuhhFJs+ZQ+e2U3iBoKTYN3bzUNiZJ2hTbx5nbxn9DjnHQD0FZwho5zGDcFJzq6r5WToXNYZvsU5roJzzgNAyRmqxU1KL/F0WaeRiDddxh2fyb8OZZz1yXtlO/N6W9hbYWipOq+xz+xpFGtqHObOP5TyzpAlzRubKdZX4CTnTIVzIu8cb4WIl2k9fD7O2Xs6xaaP4qTyZ57zG2W0tvJ6FiNOfu91LutRhq/p2QY+b7sDLy+qMLxN8Bp6gBv3ZJ3jsta5ME9oaqTY5IkcA4Apk7nxVFcbD31U6+yLptoJFFvTzd/hsNFT3bIPdBLTV63lRhkvrXjRnX979GRKREREJAVVpkRERERSUGVKREREJAVVpkRERERSGNQE9O6i09evl+TpJLllK1T7+pyszIzTa2yhl5PpSk7v4HB6HF+9dpVb9p9u/iPF/nz9zRSbOnEvis05kpOMJx11FMXGjOOkuwo5p2jv5J56l7/wLMWemHcbxZ676hsU29jMyX1N4/wvImMc37yaE/xrh3MCYjY4vQFnnSTJiJOoAcDyTq/SzrFWKA7q4V6Vzg7epgcf4p71nc6IUazQU6/Xm3euhre9cTgniY4Zwz0U77cfJ+AuXfSCW/ZGp3fmfJYTYXO1dRSzLG9k8BKKnc12E+cB1NVxUnvGnAR05zpS4zT0yDmx2hq/F/HIafSSq+PE5UI3J+3XNfL+qanj/ThuBCceA8CcI7hByXHHnkixw1/LoxwMtb4+Ttju6eVE7N5e7v06W+Hi2FXkafO1vD/zgacr9Dr7fdQ4ijUdwo19AKC9g7/z1Ws44bvXaTSQcc6JDB9WyNfwdTVfw8cQAOTzTiOgHJdtTpMHp80Kmhr5OlJT4/eA3tuzmtcnx/tiSwf3ZL/see7Rvde5JyxbsdQt2zs0Ro7k3uCHN/K5Uw09mRIRERFJQZUpERERkRRUmRIRERFJQZUpERERkRSsUuKmiIiIiGyfnkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiBXVl9YAACAASURBVIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkyJiIiIpKDKlIiIiEgKqkzthszsXDO7fxuf32JmH6piOS+a2Rtf2bUTeWWZ2RIzO7HKaYf0mDazC83s8iqnnWBm95pZu5l9b2evm/zjM7O5ZnbeEK/DNDPrMLPsUK7HYFNlagCSC/V6MxtWFjvPzOYO4WqREMJpIYTfDvV6yK7LzM4wswVm1mlmG5KfP2Vm9gqWMcPMgpnl0iwnhHBACGHuK7A+J5rZqrTLeQV9DMAmAE0hhM8NpCImg2cwzpXdWf8/YEIIL4UQGkMIpaFcr8GmytTA5QD861CvhMiOMrPPAfgRgO8AmAhgAoBPADgWQM0gr0uqitZQegXWfTqAp0II4ZVYH3nl6VyRaqkyNXDfAfDvZjbS+9DMjjGzhWbWlvx/TIXp/sPMrukX+5GZ/Xfy8wgz+5WZrTWz1WZ2cf/Hpmb2XTNrMbPlZnZaWfxlj3rN7KNmtjR5nfCUmb3GWZ9Msk7Pm9lmM7vKzEYnn9WZ2eVJvDXZrgkD2GeyizCzEQC+CuBTIYRrQgjtIfZoCOGsEELv1unM7HdmttHMVpjZl8zMvV6Y2WvN7GEz25I8uf1+8tG9yf+tyWP/o5NX1A+Y2Q/MrBnAhWa2t5ndlRxfm8zs9+XnV/lfvmZWb2a/TY77pWb2Bedp06Fm9nhyDv4xOX6HAbgFwKRkXTrMbJK3fyptt7fuVezvo8xsXnLeLLbkdaWZXQrgQwC+kKzLWwF8EcD7k98Xb2/ZsnMN4FypTa7FLyXH/8/NrD757EQzW2Vmn0ueaq01sw+XlVHNvOeb2ToAvzGzUWZ2Y3J8tiQ/T6lye7Jm9sXkGt9uZovMbGry2Y/MbGVyDi8ys+PL5rvUzC4u+/1vT3jN7DIA0wDckBy3X7B+T6TNbJKZXW9mzWb2nJl9tGxZFyb3mt8l67TEzI7Ywa9sSKkyNXAPA5gL4N/7f2Bx5eMmAP8NYAyA7wO4yczGOMu5EsCbzawpmTcL4H0Arkg+/y2AIoB9ABwG4BQA5e/C5wB4BsBYAJcA+JUZP3Y2s/civuifA6AJwNsBbHbW5zMATgdwAoBJAFoA/DT57EMARgCYmmzXJwB0O8uQXd/RAGoB/GU70/0Y8Xe+F+Jj4hwAH64w7Y8A/CiE0ARgbwBXJfHXJf+PTB77z09+nwPgBQDjAXwdgAH4JuLjbj/Ex9mFFcq6AMCMZL1OBnC2M837ALwJwJ4ADgZwbgihE8BpANYk69IYQlizA9vdf90rMrPJiK8HFwMYjfiaca2ZjQshnAvg9wAuSdblRgDfAPDH5PdDtrVsGRTVnivfBjATwKGIr9eTAXyl7POJiI+pyQA+AuCnZjZqAPOORvwU82OI79m/SX6fhvg6/JMqt+ezAM4E8GbE94J/AtCVfLYwWYfRiO9BV5tZ3fYWGEL4IICXALwtOW4vcSa7EsAqxOf3ewB8w8xOKvv87QD+AGAkgOsHsD27lhCC/lX5D8CLAN4I4EAAbQDGIa7gzE0+/yCAh/rNMx/xxdxb3v0Azkl+PhnA88nPEwD0Aqgvm/ZMAHcnP58L4LmyzxoABAATk9/nAjgv+fk2AP+6re1Jfl4K4KSyz/YAUED8WvOfAMwDcPBQfwf6l/oYPhvAun6xeQBaEV+YXwcgmxx/+5dN8/Gtx7mzzHsBXARgbL/4jOS4zJXFzgXw0nbW8XQAj5b9Xn6cvgDg1LLPzgOwqt+0Z5f9fgmAnyc/n1g+rVPuNre7ynW/EMDlyc/nA7is3+e3AfhQ8vOlAC725tW/of9X5bliADoB7F02zdEAloe/H3Pd/c6BDQCOqnLePgB121jHQwG0lP0+F8m135n2GQDvqHLbWwAckvzc/zh92XlUfn4mv//tvEf8h1EJwPCyz78J4NLk5wsB3Fn22f4Auof6u9+Rf3oHuwNCCE+a2Y0A/gNxJWSrSQBW9Jt8BeK/NjxXIK4k/Q7AB/D3p1LTAeQBrC172JQBsLJs3nVl69OVTNfolDEVwPPb3qK/lXmdmUVlsRLiit1lyXL+kLx+uRzAf4UQClUsV3YtmwGMNbNcCKEIACGEYwAgeXSfQfy0swYvP5a3dRx/BPHrkKfNbDmAi0L8pKWS8uMYZjYe8dPc4wEMT9ahpcK8k/rNv9KZZl3Zz13JPNWoZru98iqZDuC9Zva2slgewN0DWIYMnWrOlXGI/5hdVHatNsQV878tZ+v8iS7E1+pq5t0YQuj524dmDQB+gPjJ69anW8PNLBu2n/Bd8V5gcW7YeYjPlYD4ydXY7SyvGpMANIcQ2stiKwCUv8rrf77Wle/z3YVe8+24CwB8FC+/0K5BfAEtNw3A6grLuBrAick773fi75WplYj/Qh4bQhiZ/GsKIRywA+u5EvGrl2qmO62svJEhhLoQwuoQQiGEcFEIYX8AxwB4K+LXH7L7mY/42HrHNqbZhPipZPmxXPE4DiE8G0I4E/Grr28DuCbJUaqUWN0//s0kdnCIXxWejfim4lkLoDxHZOo2tmN75fZXzXYPJFl8JeInU+Xn1LAQwrd2cP1kcFV7rnQDOKDsOx4RQvD+sN2RefsfE58DMAvAnORc2foqvZqWhe69IMmPOh/x6/FRIYSRiN+8bF1mJ+JK31YT+y1iW8ftGgCjzWx4WWxb98TdlipTOyiE8ByAPyLONdrqZgAzzewDZpYzs/cjfmzp/pUeQtiI+LHsbxA/2l2axNcCuB3A98ysyeLk8L3N7IQdWNVfIk6YP9xi+5hZ/wofAPwcwNe3fmZm48zsHcnPrzezg5K8ri2Ibzivqmav/yhCCK2IX8n9j5m9x8wak+PrUADDkmlKiPOevm5mw5Nj4rOIn0gSMzs7yQOKEL8CAeLjYyOACHH+0bYMB9CBOFF9MoDPb2PaqwD8Z5KIOxnAP1ex2VutBzAmSSwmA93uKlwO4G1mdmqS/FuXJO9WShheD2CGVUj0l8FV5bkSAfgFgB8kT1hhZpPN7NQqlr8j8w5HXAFrTXJ0LxjAJv0SwNfMbN/kXnBwks87HHF+7kYAOTP7CuInU1s9hji/d7SZTQTw//otdz0qnOMhhJWIX41+Mzn+D0b8JPv3A1jv3YJO2nS+iuSkAoAQwmbET20+h/gR8RcAvDWEsGkby7gCcR7WFf3i5yB+5fAU4lce1yDOYxqQEMLViBNlrwDQDuDPiJMM+/sR4uS/282sHcCDiJNtgfgvkWsQV6SWArgHO36DkSEW4iTRzyI+Pjcgvhj+L+K/Tuclk/0L4r9IX0Cc23cFgF9XWOSbACwxsw7Ex9EZIYSeEEIX4mPvAYtbsx1VYf6LALwG8V/DNwH40zZW/6uIk1mXA7gT8XHZu71tBoAQwtOIk2FfSNbHe/03kO3eXnkrET/V+CLiG9VKxBXFStfdq5P/N5vZIztSpryyqjxXzgfwHIAHzWwL4uNyVpVFDHTeHwKoR/xU60EAtw5gc76P+I+F2xFfy3+VLOs2xC1dlyF+BdeDl7/OvgzAYsS5UbcjfohQ7psAvpScU9QwC3EqywzET6muA3BBCOGOAaz3bsGSpC8Rkd2OmX0SceVtR57aioi8IvRkSkR2G2a2h5kdm7xumYX4KfB1Q71eIvLqptZ8IrI7qUH8mmVPxPlZfwDwP0O6RiLyqqfXfCIiIiIp6DWfiIiISAqqTImIiIikMKg5Uwvu+xm9U6xf/SxN17eRO159tMV/HXnEP32EYjUNwylW7cvMrNv3md8fWuS8Ig2InBhPZ1F13TS98NxSJ5p1YkChl4fLa25rp9gLK1+iWEszx1DaQqHeYptbdhQ6KdYXdVGs5G63E+NhBmHBr/vX1/CYy4313IvEhi3LKHbVt2+oprO7neawY2fRwdHZ4ey3Pu4MeOw4t7skrFnF31F3J/ce4B2/GeNjq9TL38+4Cd5wk8ApbzqNYke//i1cThN3ddZSbKDYuk09FNvY4ZxjJf9rHNbA004YXkuxmZPqKTZtLG/3Uwvvpdidt1xNMQBYsvwpim3axB2793Tw91Dq47L7in0UC8Hf7sCbDTjnT1TiCTs6Oob0nPjOTy+lHdLa2UrTLXv2CYo1BL+XjJph3Idmzvj2l6nl6RoahlGsdTMP65jN+kPZrd24jmJZ54bU51wbJ4zh8yyb46/n4ANeS7F169e66/P083xPsYiPgwbnPtpQx/ti5ZrlFGtv5/sBAORreZmjRjTxhM61qVTk77a+ns/bMSP8a1NndwfFmrfwcRWce8+fL7t8u+eEnkyJiIiIpKDKlIiIiEgKqkyJiIiIpDCoOVOlkTMp1lfP70a7R/DA7OOeanaXeeUlX6bYGz76LxSbPInH+s06b0Hd/CbzM6662zn/Ie8MdJ3r5vfH2ZdeoFjGecfd0M7veQu1/J4YAOpz/M7+wD04/+w1xxxJseebeWSNx5cupNjTy/n9OAAE4/fZJWcQ84zzLtwy/EVkclzPr8/z+3YA2KPpCIqZkw+RGzWkqSCuYWM4R2nEWB7tp+jky+T+f3tnHq5XWZ77Z03ftOe9M5CQhEyQQJhDggQCBNAig1AP4oB6UFFO1bbaetRqqaetWmtbvehpOfUoUEXgiChCURGQwQAhgQABEhKmhOzM2Tt7+vb+hjWdP/Cf432viw/Xdfam13X//rz3Wut911rv+653f9f9PA/xN5mZFTrw3msh+ulcYlVbMA+9TJ//wpdB6160irZ9zzYc6995CedELUW/VxiTeZYWQPITHBthkuGKJP4Sa+JYnbEF+12tDoJ2/KJTQXvPp9fQpq+oPg/at//xb0F7dMPT2MV6CJpD/veNIn7fLlncXJd4PN+CqXGaCa55UYjPY/GiY0CbGMN3ZmYW1vD7EUd4zUqAzyNwcKLUyXya1t5N2zZyflsbjv/6EPZ9aBh9q5ee8y5s469/BtLcy1bT7vQXcX3ZcwDrDkdk7g0Mo8fI93BcdjMflJl1dqKfqa0N13UnxucbpOh1rEc4zuME10ozs0IQgNbRif0cq6LPuBX0y5QQQgghRA60mRJCCCGEyIE2U0IIIYQQOdBmSgghhBAiB5NqQG96C1CcdixIpRlo2DvsCJJU0sxO3/wQaBsfvB60kVUfAm1O7wzQOktockuG9tO2K0+tB6386GN44EvbQGJJ28zFve2SiCQoJAnWzMyiLjQ1FkYxgWOxgEa8nrmHg3bSGWeCdvC9mJTRzGztlgdAe3H7WtBqxEiaEhNt1MTjkpAnxQu68HlEIZoQA0Pj5VSz44WDoHV1oSGzqxuDDord/HkUS5j0M3DxnX/qk58C7eyLrwbt5sfRBPvUy/h+zMxCD5P60VyrxESeOvgeWchAzEzlZO6Y8eCGQogG9M9c1APaz9eiCXZdPz7HR1/G48zMju3D9eVL//hT0J785XdB+x9/83XQmglJZOvw95ASQz4LrnkrcsxRy0C74xc/Aa2nE9+ZSwJPXtdRK5QwSWy9jnOn4OM3IYwwqGKsisZ5Mz73ZvZOB40lNO4gRvXutTtAG+7HJKIj3/s57c/512Ki6zvX3gXatG5se7yOz3e0iusDCxwxMyuXUe+s4NpW8vC4Wo2sa6Sd8SZPGOo5uEaMjWHfDxzgianfCP0yJYQQQgiRA22mhBBCCCFyoM2UEEIIIUQOtJkSQgghhMjBpBrQn9y4AbQXX0VjOcs4XipwQ1u3oblvzZzloG16GStl98xCo9qM/pdB8+9Gc56ZWbD3AGguyaprMRoLU5KNNSmi0dEhWcRJUWszM/PC1szqHjF3+1tfwnZeeBG0WTNn0rb/8KILQdu25rOg/fLem0AbamLG2biBpt5jF82lbScuvsf+QWJgfwv+7+CS7MFRgn3ftRufUU+dG+q7u9CY+9W//CfQhkongnbNHWi+HC+QjOwZmbNdYvJMWHZy4gj2yJxgsOulZI6ZmbnEtH3qPNROmYPnTz8LTcKbfoSZtMfLPPP1tirO58/egFUOPnj2+0C74SY0YX/yE58AbXAIqzCYmTktvgeXObOnmCc2PA7agpmYmd9SDCSoEgO5mVmzgXOqqxfH9QjJfn1wYB9onoPfo1FSEcPMLIqwnwcGSab2CCsarDoGM+6P/dm/g+aTtS0d40bsjt/gN27RwiNB27MHTe11EtjDRlC5yOdyRxsGqDQa+M5qZDpXSmhUZ6Z08sk0M7PdQzh3n38eg8P6erGPrfDW+7oIIYQQQvwnQpspIYQQQogcaDMlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA4mNZpv326MZKk3MNLBJWUPGrU6vabLoooOYImO8953CWjFn/4YtNJ6jCRx67xcBIs6TEntDFbSIiUheU5EIvdIWv2scjIe6afjYISIE/MSFNgQSn6VR4gUbrwRtOOWLQWt730fBu2+p18FbWQYI2M6O3mUxdbX8H3XyS12kjItU02hhOOl3I5aey/2nZXhMTP78l9gKZLnBuaDdvt6jMxJA2wnbuIcdXweXUsD9wLyP5tLDkxJNCpW7TAj88RCHsLjRxjt89GzscyLF2FZiaNmYbmeC47BJfOOV/l7YNGjNfJ8r38In++ly3Hu3PCDH4L2kSveT9seruJ981XjrRfNN3AAo8g6O3Du+2QNHR0npU3MzCHlhpoxPpGuDoyErR7YDVpKxuDEBP9OtJNyKezbMa0Po0f7NvSDNjKOa7DjYPRcmvFuR370IGgrvvNHoP1s5y7QOsr4HsbIs5jezSNcC6SM1EQD76cY4Hfr4CB+E+ox3mP/a7jPMDMbHsNovkULMUp0Wg+W0WkF/TIlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA60mRJCCCGEyMGkGtCJB9AqpAcdJTSfpQ63T87pw/ImM44/FrT2v/sqaP4+YlRjnr0Ay0KY8fIMjo99b/ag4TU54RTUjliIjfR0gpSm3FjoENO2u2M7aN7TT4IWHCLPgpX8CLl5PSHPqPAKlgo6/DvfAe3dn/kcaNff9RBojQYPQqhU0CjseWiu7iq2aLyfRBwXx0ttAsdVfQDLXHzzm/9Mr/n8QRxHtz2N14x8fG4uKcvikJI3rFxJps5KlpCaD2mMfYyZyZ5oHgn8MDO74BhcN+a0oznbifE9JCmOt/echsbadS/vp23vsWnYT2KsTQzfw0+ewj6WVywA7dvXfou2fdXHrgaNBsKwKJMpplrFdaxcxOc2VkPj/mijSq/ZXsDzD5FSPD1dWGKGlaKplDAAI04qtO0GKTHmOKidfvwK0EY+/W/YDvkNhJUYi1iQh5kV6rg2unc9BdrC5YtBe3UHll+Z0YfPbOZM/OaZmdUbOO/rg1iuZ7iKxw0cwlJXURPnaIVtKszssMNw/nS0kaCkrHo0b4B+mRJCCCGEyIE2U0IIIYQQOdBmSgghhBAiB9pMCSGEEELkYFIN6O0k23PSQOdctYYmwnafZ95ec9zZoLV99S9BCwYw+2lSQBOhQ7KLR91oJDUzi6ej+T287L2gpcctw7Y9YrInBlGSKNeMZX03M+pLJ8d6yVWghRueAS24/VbQCjtfpm07DjEUEzd/YR+adTu/icEBH/7v14D2wGMP07YLIb7beYdhJuOOznZ6/lQyPo5m0EYNzalXf+wToKVtq+g173yEZGIOMHMxM626zGzOxiptmWfnT0NyNIknYYEVDgvyIJEsPQ6OATOz956OWaXdCJ8Pux8ypK23Dd/Nu1fwAJV/e5wETHh4LLtv10Ez821P4rr4uYvX0Lav+tgHQbvlNqz40IhZivmppbuEhvyIBEYUiphZ3GvyLORJE+8zJeMoJmbx3k5cS2qkcofn81lRCjA7+cLZmHm77VGsBDE0hoEIMVlXu1YcDdrYcxh8ZGaWEgP64J2PgHbyBZ8GbfurL+IFSfb1kYxKGUmK43/PEAbXFHx8N71dGIwVRzhPCuR5m5l55JoBCUxwkt9vW6RfpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA5mFQDekjMfbUQs9DG1V2gnX/xh+k1u667FrTifpKR2GdmczQMJiU06oYX/iFtO3zXBaClJKM1s7dyXzk5jmgsk3HmRQmxh+fHq5aDFq04HrXbfkavWbz7dtCcAjEzj6OhMhhGs2LPj24G7eTLr6Btr197L/angpltRxrcmDiVVMo43ubOwoCHd7/3U6B97vs4d8zMxv0+0JiJnAwDmhGbJwTmxmVq5CZOdxYswaoHUEM8MRlfuhLNqWZmM8oko3uUMX9+B1pzgRjs33kymtzNzO5/vh+0reNowPVJVvSYZK+OPQyg+Je7efb1az/5ZdCeew6zXA830fw71dCxkeC3wyUDuFjkn7TxIVxjwgQDBAIyT5okgCJMcRxM1NGUbmbWVsZ+Ll+Ca+vYtdeDxr4mSYDfso7Pvwc094cP0f5U/2MdthOhKd1+/ARIR561FLT1T28krfCPUbmAa/DMXlwDAw+DEJjpPyYZ0Is+z0TPKjk0mrg+FMl8bAX9MiWEEEIIkQNtpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIwSQb0NGQ2WloDJwz7yTQ5m/dRq9ZfvZZ0BKSGZfZSZMCmkGbf/p50KKVK2jbLMM3h2R2JgY9lpE6YdnOM43mpB1qiG+NiJgxkw9cxg+ePQsk7+YbQQvKxAjdRGNh+fkXQJt/4hba9N5FmP03re0DrbebZ6qeSmpVvPerPvtJ0O54FI37w34vvyjJ9JuQoAXicbaUZJp2iMayR7+uE42MI5+ZzUm/WZDIrA40oF90Aq9S4EVoUKXBH0Rjs9sh7mjfwczkZmYfXo2Zs796D77HcQeN5cy07we4XI+H3Pz+kw143+++HIN4rrvuW/T8qaRA1uXhMXzGpQoxi5Ps3mZmhQANzfU6jqPhcTTk+w4avhkljy/MPR34jiqP4bewNob98Y5eiNc7aj5oxQW4/jpXnkv7M773AGj+M6/gcb/GgIXlf4hr00tdL4F22AweELJvcBjFBOf9wKEB0Lo6MKjIq+BYacQYrGBm5rHvo0MyoBd/v0oZ+mVKCCGEECIH2kwJIYQQQuRAmykhhBBCiBxoMyWEEEIIkYNJNaDvH34atJ5uzIi66nQ0Sha+9CV+UQ9vwSXZaWNiKmt8DM10zZUrQWPG8NdhhkPmwG0tNXkcY789cn8tpzrPaJqZW9k1XWJeZxmyzcyaZ58BWmF4BLTinT/Ck0lGX4cYnIN7MNO5mdlxX8Jsz09u3Ata/56D9PyppK8Xs5UvPfkdoP3gR/gsgyI31IckSzczcqdknrAB41DjZsakIEb3lGnU/E7600TD9uWr0Nza6aHh2swsinAcucQo7LDqA+Q5psS1z6oUmJmdvBhNz6fNQSP1Q/swm3zSYiUFUtjBzMx+/Sw+t2uvvAi04nX/yi8wlbCU++S5JzE+t6zACK+Az7PIxhvrDpk7jkMyoDdoqQBbfswpoI184SegucQM7e1Cw3bn364BrdHA8e/O7KL9mbHiGND2vvwaaCG5n/Dm34B2zsfPAu3JpzHLuplZMcA1qzqBpv/pfWg2jyN83yEJDmiQ48zMdu3cA1pAsqLPmsXf4xuhX6aEEEIIIXKgzZQQQgghRA60mRJCCCGEyIE2U0IIIYQQOdBmSgghhBAiB5MazdfTgxEHPcUloLU9vQk0v8mjdVJSlsI8jBCMTlgOWnzueaC1Gun226OJwsLnyBXJYS6JeqKtkpIWr/+B6A7rD4tiocUzWjrKzCwl7UQXvxO05gtYJiZ4BTUjkToBKbdgZtb21AbQZh2+ALQnnyPtTDGXXHAhaE++hCUxJkjpGBaMZ2ascpKxN+eyckHkzJhEtXkZc8IjUWgRK0eTkgi2GCNzFnXhszj3SIzm81I+NmLy7yId/SyKkdxiRPqYtYoWSBTYB1bPAO2J2w+BNmYYzcSiBp0A1zozs6iMpWw2bMdnvuoMjMKdagZGR0Gj5YfIrbcV+PNokLWj4JPyPE0yjnyMyqwEWLJs5kx8t2ZmszcPgTYwihFs9QDf77SzjsULzsW14N4HHgNtwbw5tD8nXn4maM277gctNJx7wRObsZ0P4PXW84ouVi7iexyfQK1aw2cxOISl5yYaOHdm9/EoxuOOwtI8bA0cr2E7raBfpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA5mFQD+p49aP1ccz6awAs3YskRN+Qp4i1Cc2viowmx+b4rQGOmaW6wzjJ8E63F0jHUK+60trfN9J9TjZTyaLFDtJms2yN6TEyjjYuwpIX3vZ2guXV0MKa0xIZZsPFZ0KaffCVoC/Ztp+dPJavPRpP+jRuwdEzqoKE4SRr0muwVtTpUmSGTjVWXmKvNzBIyTx0WWJGQMi+G73xGJy5Re2uobXyVl4B4OfNLCgAAIABJREFUciuW4/jixbNBayflaNj68OAWvN7DO/mEPGEumpR7erCchk+eeUr+z3VS1JKMCZmW0TT9yItorL3sPCxdNNUUiYe8GeG3IyDG8GqTm4cjUqolSXDMuOyTSI4LSSmatx2PQU5mZiN/fjNpB8eMR0rhdH7obNBuvQPLat3y81+BViq20f5c//d/AVrHpatAG7gNTek1Uqpl5Pr7QDvx6lNp2/c+iH2v1nDe79mHpV8WLkBD/fzDZ9J2GA4JPZkggW2NBg9meSP0y5QQQgghRA60mRJCCCGEyIE2U0IIIYQQOdBmSgghhBAiB5NqQN//KprXFrcdBlpx5w7QUmL4MzNzfNwPhsccD1q0ADNi82znbwKWXJyYyFNyYEoMvLw77LiMPTA1v7doiM9yteORLTedEMNfuuxI0KK+w0ErDezDcx0+XL2RCdDKEfZz4wv7QfsgveLkMXfRCaAdvG8ADyygUdLz+DhwjJhtIzYGiQGXDEKWeTsmgR+v94m4h9nYYJnJSTtP9OP1Nt06iOcWMGO4mVlKPPoxyXzNsp3HxLRcJ8924whve+MB8txizO5tfjv2h5nS2ZpBWzZzSHr8vaO4/s5exE3TU0lC5rnn4ngbJpnSe7owO76ZWT3GNaLmYIbveojtRE18br0900Gb/vRB2vb+Icx2nnr4LjvOOhk0bwF+H1+5Gw3f7HeRMMb7MzPr341r66J3omF88N51oDUcNPgf2rIVtMXDq2nbT/f1gTaxD/uzbMk80MbGse2eCmadrxOTvJnZeB3nhBPjAlEpkioHLaBfpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA5mFQD+lxiNPb3HwDNbRLXaBGz3ZqZJSRrbLz8FNBazi5OTNMsi3iWzky0NDN5i/1hpMRcmqV7NGs4y3bemvE4C5bl2iUG3pgZ9JctRW3dELZRwOzRZmZegHp5gJi4M+26U8dwFQ2vVcP7KVTQFBk2uQk8ahLjPw3gYMfhUS6ZYyxbs5lZJcXnvnIhZmJeTzKWT7hoxPZcXKJCvwJa1nxKifE+YUZucjobLSyZO8uUbmaWFtA8nzbZPMP34La4PiQJXwuSGE24TTIGvApm1p9qggKOg3IBx0EhwG/CyATPgO6RqhgW4nvziTHcD3Dunb78dNCG/ztW7jAzc8lvFnXicZ515bkokqHV04UBD2yddzOCj7racf4EHThHey85E7RDP/glaDGZE0Pf/QVte+VfnI/XfACv2VHG6gFRA8f04Ci+7yzjPQvw8nwUY76sviH6ZUoIIYQQIgfaTAkhhBBC5ECbKSGEEEKIHGgzJYQQQgiRg8k1oM/D7NfuK9vwQGbIHMUssmZmTieaFaMlR4PGbJpZtnLoDj3OzCF/cVm2Z3I/DjHwMr83ayMLl2QOTtn9sBtnzTC3LTHvmpnFxLXnkyzOCWkonncEaM4zz6Hm8eHq+NiON0yyI3e/9cy2g4P43JoeGtC9Bh7HnqWZ8fdG3JcOcY6yzNmBg9mjzz6az57/ehZmLp7Ti8dedV0/aC/ViCGeGIoDrwu0MKNCQjiG4yCJWcZx9iyIxp4tGedmfO6xd5PGxCRP7ofO5QwDOl1fSD8neLLoKWVwGDOJ97ah2XwgRKNxdxdmJjcz27sfs+ZPJ1nMHZLBv1LCrOpzNx8C7cChEdq2kXdRXn0ctj13Gmpk3rZV8FmkLACI98ZKZXS/RyRreMc5J4E29ov1eMEGPovxnbto24f3YyWHad143/U6fu+7e/A4z8F1sZjyDOYuOZZN0dHxGj3/jdAvU0IIIYQQOdBmSgghhBAiB9pMCSGEEELkQJspIYQQQogcTKoBfdbM2aC5u9Fs6BBjoRGTsZmZ0whBSzvRoMqyn2bkJidalgkc9ZSYDR3iEWXZxZnZkCU7Z+bSrP44NIt5awZlSsZxboYJF45jGdC70eBpAclY7HNjoVPCjL4euUe30FofJ5MmMe67Ru49JeOKZPc2M0uI7pHzrYnm7mWzsPrAJy84DLSls7jxOTA8f98Avos9h0jwRgGv2Te6BbSOjiXYxoHdtD+Lu9Cs67tYiYFNcVYVgPnPjWScf70dkr2dzB/mIWeBLC7J1pySzOBmZklEAhZIluy01Xk/iQSkokFn1wzQmsNYJcHN+KT1keCTag0DK9rK2PY5K08D7dBnbgKNjwKzqIDPeNFVF4NWIN+tRh3nU28nZitnSfhjOljNSkW8R7p+VzALed/la0A7dO0doKUBfxoHvo+Z0VddcxloDz/+IGiFAO8nTHCtdDJSmMckoCQi+4d2kn29FfTLlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA50GZKCCGEECIHk2pAZ1laKSzLtpdhtiXubpbNmKcXZ3tJ1g5vO6NHVIWm6aMgBnK63c3Oyd7qkXBmiwcy47wZzwydkPfIzPOOT0yExIDuFLkxMC2hodJpwyzX09t5duSpxPeIIZ+Z0n0yVRNutPRjzGbcVxwA7Y8uQVPvuUsxy7CbYNbiLN8yM78/9cIwaLFh34MIs5UfSmaBNjCCz8JpW0D788IomowTlg2exWSQ6wXk/8805GnE44D0k0xoxyVrGGk9IU51jwVqmFkS4bFJgmbbMlsrpxi2/B8cwYzYHqmIUAtxrJplzB/yTalUMJhl5qYDoA0OoPk99flvE4XTjgJtfAauZUkN+14qYwDFjGk4R2kGdLKumpm1kfUypZFOJPv66ceDNvKTR0CLdu+jbacHMRP9rM0YhDZ9OgaJTNSqqA1j9nU3I1jNZ4FbJNIjbCgDuhBCCCHEpKPNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA50GZKCCGEECIHkxrNt+iok0BLD2IkACurwkqymJm5LPSjipEfjoMO/4RF8LQa/mYZJWFIFE6TRGkERRJpQcpPuKxkDQ/xax1yj6zaCCUjjIs9tzQl0WasTEadRIuxCL8iRraYmTkkAsefNhO0cBDHxVTT04ElcvwQI9BS8n9PX8Dv55JTcRxddtoRoLV7pGyTYcSXkSgYFqlpZpaSshTnrMSowXNWsMgalCbGceK/uB/Hy0NbeATO2hewHAeL02NRpi4pP1EgJVmy4uFYaSmPRbOS+RyHOHdSthawWjRmFpBIQi/CvhffzII3SbSXSVmVJkZ69s3oA23w0Ai9Zk8bRrBVY3zuJy97G2ijn78F+0OiwOKAL6LTPnguaDQSsY5j2CHvp0DKCrGFtZN9Y8ysEOA4aIQkUpR9Z0j0aO8Vbwdt37dvpm17pMzR0L/fD9qJ374StF89fC9obaT0y9AwRg+bmZU68PtRausGLQ3ZmvHG6JcpIYQQQogcaDMlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA4m1YD+av9e0I467DDQEpIOnhmxXz8YSzl4O3fiYUuX4TWJwZSaPLNKqFAzNmpBCY1vKek3NZuztt/EFpgaw7nY4hUz3gM5PQ7RzBwUiQH3AJYToOb+IppIzczczg7Qkmm9oPX/chc9fyrp7cDn0eGOg3biEhwvV5+Hxm4zs3mdqMUxmrZTZsRmFyRG1DQjIoSVPCk4xOje4nArtuOBp7bjsvW2I8lNm9kHVuHYaCfmYRZY4TADOin/w4JOzMwSMoZjUraDrjm0Iha5Hik9ZGaWkvIpHQW86N7+5/HkJWvoNSeLg8NY+mjeTBzrjoP3OFbl5WQswW9KXzca3RdsQ/PyoYOoOcQEXlm5lDYdHDUHu0PeZUDK0bDlv7sDTdfsU9TZjoE5ZmZxjGPQJ4EVCSmNxsZq9+pjQRv6+VzadrgZv83RMAYNTHtkO2iHzZ4N2sQYlvUplbjxvllDg39Egp9mTsM9SSvolykhhBBCiBxoMyWEEEIIkQNtpoQQQgghcqDNlBBCCCFEDibVgP7yJjQ7hhdgdti0iAa7NOQZjl1iLPSf2YDtvP0PyNl4+8xMmhLTqFmGAZ2ZdbGL5rqY+ZrBjeoZxzLTKjOMs6zJrWaDzzQOk14xny/JnB288io5EBtys7I1k4CFZolkPN76JD9/CqmO7wGtu4TvfGgU7+fAGDcfzyRe7MBI5u5Ws1+/iXgFWr2AauSarB3SBvOPxxmBEXOmoeYkGBiRkg7xCgfkellVAZjOMsezZ8G0hJjfcaj89nxs55g5OIY2b1wL2qXnTa0BvVxC4/RwDd/ZjBJ+J8KQZfU3sxTX25UnrABt9Iu3g9YkRuyEVKvo/DB+y8zMIpL12y+gSTolxvCEVJFoa8eAJlaRoNLGDehsprDvRJM8S49UEnFINvfD3n8+bXv3Nd8FLSxi24duwWznK6+7GrSfPnAnNuLy77Uf4PjvIF71JMwIYngD9MuUEEIIIUQOtJkSQgghhMiBNlNCCCGEEDnQZkoIIYQQIgeTakB3iJluiBiF29oxa3H54AS9Zkq2g94TaDT2RjDLatyFWbK5E5vvOVv1Zztkz+qQDNLUbEsyGVuWIZ6a58npbmvZno0cl5UAPYrQCesX8N16+w+gtgszk7sBGkadJnfbJmU0ZFb7d4P24U7MeDzVrH/0AdBOXPgO0O7cimbbL9yEY9rM7LTFOD4+cHYfaEunk/cbNkDigRa0aXM9/MPm1/Ca615jJnCcJ7P70CG6fD6uD31lbhrlCxwz+raWhdwh95eSde23F8XzqVkdj3NZPEfSoqHdzOI6PvPTF+H53//BQ6B9+Qt/Ra85WcQh3lNXO45/j9x7XzeJODCz7g6c+0e8WAWtuo9UYyDroHvKYmxkEc+c7Qc4Cj3yTWCBSoxigHOiRDLz97W30/PZsh4R83uJrMGMiFS6KJ+wiB7rnbQAz9/wEmgpyWTv/WITaPOPxnYO7OOVLqIm7iF2DZN1qIlZ1VtBv0wJIYQQQuRAmykhhBBCiBxoMyWEEEIIkQNtpoQQQgghcjCpBvTNz2AG9BOPQyPf7FVngpb+7DZ6TYeYSYNRNOY2f3oLaNFHP4nXI67TzKTfrD9ZZtTfgZrXW00LnZFxmfWUZp8mz4ynO2dN8LZd4phlhwZ3/gdofnUUD2xDw2hCTLVmZhNzZ4LW9R8/B23adsyMP9XcedcdoP31Ny4D7VfPDoNWD9BUbmb28DbUNuwYAO28Zeh4vWxVD2hziW/fT9G4acaHaxsJEPjlc3h+zUeTcdxAI2qvvxe0D57Fsz2/YxnqHq0KgBKLO3HIOHcz0pCnpNKA4xGXMXlo7L9cZkoPY/7/8OziOGh9ERprN2x8ip4/lVRIRuwowrk/NkGCkmK+/h6/7ATQxv8Ss2ezQITYxXfW/SHMEu8yZ7eZuSwwiB6KYpyQDOgVnE+lAprFK2UMADIzC0lGdofMiZSMwoSMaXbfGXERNv39GFyze9PLoNVD7GN8xyOgrbrgT0C7f5gbyMeJyX46SYG+YycGSbWCfpkSQgghhMiBNlNCCCGEEDnQZkoIIYQQIgfaTAkhhBBC5GBSDejjNTRFbnh8PWgr3nsFaMVfoFnQzMyv4zWNmPHKxPicrDobtGjJMaQV7qZjmc1TdixxYrvskixbOW05i1aPZntokq25ZeOkUZe+99hjoBWf3ogHFol5uInG4yhp0qZHuztBG77/ftB6i2h05/mSJ48tL6BbvLp/C2gr5x8B2tqdrYdGhAkay+9+Dk29v3lhP2gXnozG8He/DbOQm5lNK6HJc+FsNMIeUUFD/NYQx4Hj4RI1kLaB9r/W8uCEdmLWPXMBmk6dFE3kbDaRBOh8Lr9+VbwmLUlATiUO3pQ40P2AV0O4eAU+o9u+/w3Q6gk3z08lRZIxvFTCNd0P0BjOTPpmZot34HoyugfHekyiDkonHYl9PBYzeTsZlTIiYiJn44jmxo9IpQxiAr9wzemgLTtyPu1PoYDjn363yFhl95jEOIYcYto3M2s/eiFoldOPB6320DN4zRqu/83b1oG24B3YhpnZ1q14zfE6BsL09bLKKG+MfpkSQgghhMiBNlNCCCGEEDnQZkoIIYQQIgfaTAkhhBBC5ECbKSGEEEKIHExqNN/cebNA6+jGWhVP9u8EbfUfXEyv2XbfXSiSSAKnidE+lW/8DWgTX/sH0KLZc2nbKYuEYVEMNCqClQ4gZSUyShQwWKp/Fn3BovRYxOGbKaNTevU10Np++H9AcyOMnkgKJOqJtNE8/ija9qYXXwHNG6uC5pffepFLrFzEN//x71D71g9B23gjL5tQ9TAaJQxJ2aUiRvWMRd2g/fgp7OMDz/GSC+86FSPyzj4WI+raSRkHr4pzJybRTE5Eomh9Hl1466OHQDt1Ia5DFQfbSRy8b9cj8ynl/5M6Lou0w3tMWLSvj8exOTG7iGWGzMwWFfD9fP1X94BW8Cf1E9ASjRDXiEIBI7lSsratOBGj2szMxv7qdtAiw/fbdHFczr76IjyOvA2PheiZmSWoN5t4P0UShc7KF7GKZbOnY0RzmlHaLCJtxymZZ+TclJQd8z0yl0lEppnRKNW+970dtNeewNJzTcP1e/B+jBhfdvnnaNNPjGPmgEoJI5Ujcj+toF+mhBBCCCFyoM2UEEIIIUQOtJkSQgghhMiBNlNCCCGEEDmYVPfhgqVLQIsaaIbbsXM3aMe/61J6zcJGTCfvjWPZECOGOP8glhOo/P3XQBv/48/Stptz54PmFnka/d+FmUkdoibEgOtmmEZdWkvhTZSEaeHcwjOYkt/MrHLTLdgfUhLGYjRFOnU0i0d9aCjefTSOHzOzV+65F7QlS7AERLQPx8pUw8qLPPscPuPH194G2ofOvIRe84a1WGIpIiVYkhjfb0L6E8c4pg8YlqcxM/veA/jOb3lkH2iNFEvMtHUS03XMTNyk38TIb2a2bwjN70M1bKetSII/HDzOI6ZyFkxixk24IekmCwhxSY2aIBwB7VNrMIDHzOxv/uhDoEUk+KOji5RymmLCkJQn8dGc3dGOfZ//Gq4lZmaju8kYJO8tPRlLkTjHzQOtTM5lc8fMLCKBSh4JLGJrAbvm2g3PgXb9bT/Dhh3+nfjrP74StEUL5oDGPicp+XjQuZfwtj0P51Rx7gzQ2lefDFrtXjSQO8TQXr3hAdr2ysvPBK3/FTS67z/EA3veCP0yJYQQQgiRA22mhBBCCCFyoM2UEEIIIUQOtJkSQgghhMjBpBrQe3swMzMzubnEiL3p5a30mqv+7Iuglf/pm6AVhzAjcEIMpsFOzL5e/tuv0Ladd78HxfPeAVLchmZqZjZkWcjZdjfLbMuMsPSSrOnBQdAKP8WswZVHMeOsmZkTkyzvxIvvEAdusw1NwiNXvBe0deufpG17AZpTdx+1GLR5/U/R86cSluE+ImPjG3+PWdFvvfVt9JrnL8VsyHdvwYEUNfG5u36LGfOJ8dPMzDw0lk/E+H5cn1QAIFnamf3cJVmYPYdnLW6maIgfquFVD6+Q/pCE+cw4HGRke2ZZstl9swzZQXMCtI+fic/2npu+TtveTqpIFMqknx57wlNLQMzmAXnup5x8GmiNz9xErxmzwAqSzX7WR8/H40jQTEr6U29glQ0zM99r7TPbIFU62ioYOLJ9FwZOJWSOsnXEzOzAMAYyLA3mY3/qOAZdci8eWehTGmJlFpIgCJ9ksp92+Tmg7X14E7ZTq4E28jBf549+/2rQtqU41mokMKcV9MuUEEIIIUQOtJkSQgghhMiBNlNCCCGEEDnQZkoIIYQQIgeTakDfTzKONybQIOqQPV7/a3vpNZtL8PyzPv9XeM2vXgNaaXgAtDRC12kh5SZN7wfXgxb/BDNVRytPRW35Cjx3LmbadToxw3GS4Rl1Rw5hH7fvAK2wEY3cPsls7pNnYS7P8B67OJScFN9j3NEOWvXjV4P2wOYXQRs+iEEEZmaFIhpzZ5UxO/LE4O9nLPz/SaGAzy0htus6een/7VMfodf8/vVkDMZjoN33IjHWlvD9MLMtNaUbD4JIyf24LDKCzTNy3ynLPh2isdXMLCDDNSRGd+JpN5+ZzYtodI8TNOqa8Qzq7Pn4MWbtvmo1GmP7H7sBtNtvv5W2HRLzccnDvldHed+nEhpe4+Lz6NuCQTNDr/XTa0bkuReOWwRacBJqDWKaLjgkqCJrTpCfLGISiOORaKyJCaxmsOZtJ4L21JZtoM3s41UKTlyK95iQSgMeMZuzjOwe+SZ4GUttk7STkHlfmIHBaj0XnAHa6I/ux+tlBFWM/+/7QFvy8ZWg7R7g35k3Qr9MCSGEEELkQJspIYQQQogcaDMlhBBCCJEDbaaEEEIIIXIwqQb0tlIZtJ7ObtCYGY75Vc3M9g8Og7ahYx9oK7+IBnTnnzCrdGEYTY1pxA1tKTFFFkewP8EDD4CWPPwo9idBo2NCzH3M0GhmFhXRdO0Ts63bRFOjw9KVM3xiSjczr9kErTm9D7SxP/0saL989jnQdhMjabGIz9vMLGxin14izuPvFzGbMObZnVxYtv9iGe8zJM/3IAnoMDP7yCfeB9q//vN3QZszE437N68fAq3moxnUIeZsMzOHZJV2yD36JA0/M616JIjBWGbnAp8UJLm4TZAMx8MJPvOUmN9rJNN/VhLxoIhjsNfFQICPrMFM9L++5Vug/fsPvodtZ2Rfd4iNu1bHMfRW/H+62cB+Hnv0CaCNfvUO0BoZwUIxMd/3XXUBaCxAIKAZzMk3KmNOpKxaAGknJgE/DpkURxw+DbR/+cofs4Zpf1zSH498c+M6BnelEY6rkARgJBHPgB7VMMt7fQIDMFiVjuLi2aA1Cvh8irxAiI1s2gzasuZ5oD3Tid+tVnjrzSQhhBBCiP9EaDMlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA4m1YBeIgb0lGSN9YkZOnW4mY5lYj5AjLmvTEOj++Ff/wfQOm77MWildQ/Ttt0QDXoJMei5ITEWusTc1+KzsDdhdPRIZmiHZpUmjj+y1Y48bgJvnHMuaKOXXgra9n2YXXb7y6+QtvG+/YQ7C5nZMQ3xGY2ddAo9fyo5eukC0LbueA20cnsHaOEEMxSbDQ4eBO2jH/0QaF+55iug/cMHzwfthntwPj0/iJnSzcwSrxM0ktjZfB/fj0vmuEfM6wkZmDEzpZuZ52HQwbfvHsX+OPgsSeyGkeltxRKfE+csQGPuqtmoXfu1T4P2EAlQYdmj/YzInEoZgwvqxPwb+Gh+n2raKhhIs3g/rmMDr+wAja2hZmZJET914+ufB21k7dN4TZKt3Glif5rGg3Mi4qZmASXNJr6fOMZ2QmJ+L41hH2OSuf23FwWJGdCZUZ1FiQQ1Yl7PCBgLyKSqF8n6Tc4theQ5FvB6xSgjKIN8J4b+7S7QVn0BAxNaQb9MCSGEEELkQJspIYQQQogcaDMlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA4mNZovIZEWzPSfkDT4LNjMjEfxVCoYNbh58zbQtrkvgbb0D94B2vx3cnd/6b5fgRb85teoNTByzyF1LlgpAxZl55KSFmZmfr2G55NQKodoYRmjeqJVZ4A2/naM9jIz+83mLaCN/CuWxFhy6XtAI4F7FpCyJBlDwBIWnUjGhV/FUh5Tza7BvaCFNYwKSut4Pw6JiDMzK1UwkqtGSgh98Zovgnbq8ttB+/M//wJoE04Pbfv+TVjKafNeXGaaKZb3cB3UHFLKyWeVjzJKSBTIsWGCz4eV3uguYMTVacfh2rJiPo80vu9nN4J25aexrM/YBEb4xSQCzCeRtBlBjDY6imuOS6KA2dybapafdBpojWt/AZpLvx789wG/hs+jeguW+YpIGSraDpl7UYU/TC8lx3aRY1nZsi6MRg1I6SSnimt/kBH97PdiOZp0P0YApyWyBpPyThEp6eJnlJNp4hSnZaSCJj6f2GXldshxGbsah/Qz3rYLtMOrv9+2SL9MCSGEEELkQJspIYQQQogcaDMlhBBCCJEDbaaEEEIIIXIwqQZ0nxj5+qb3gTYwOARakuG0DIipcqxKDN/E3M1soy88j0bql4kZ2sxs7hmrQJt1CZZQ6dmzBzR/+w7QDuzAsioF0nZPHcsOmJm5PVgyJ5l9OGjRosWg7SuiKbd/4BBod/7P62jb6SiW6FjWhobBXS+i6Z/VHkg88r5ZeQPjYyMlhuJCA02aU82hQXxuroPTsk7eeaFEjNRm5gc4Zjrb0Tg9PorG57WPrQPtwQcvBG316tW07QsvwGPf/1/OBm0ixXI0u9EDa7sP4H2Pp6i5PnG2mllfF5q25/SgMffwXgzAGBvEebtuLZaW+pOvoWnfzKz/tR2gpWRcJwEpV0K8w+y9OmSsmJmFTRz/zQifm5/lYJ9C5tfxXUYv7gCN2b0zCqhQEzj7ALhkLWIBO+zb4TIDuZl5Dgn4IZWgyqQE1ngVA0dqAbZTiXGcxxmlhuqj+H31KjiOiqSPTfK9LbBgKtqyWUrc5qxkWlwiGvlGFQo4b73pfC3wiqhXAjTjj3fz898I/TIlhBBCCJEDbaaEEEIIIXKgzZQQQgghRA60mRJCCCGEyMGkGtCPXLoQtF27MWOyw0xuxMRnZuYSvaOrC7SUmPEiYkiuTaBWD3km2W1b0Uy9OUQDu0tyd/fNOgwveMR5IPkk3XMpIx28R8ykI0PDoIVPbQBtz8aNeL2lR4OWFQjglNEAOd6ogtbcsR00v9iB1yMOxjQzB3qW3fH/JWwSR+VUE+H4LRKjZSlAo+V4jRvqQ5L9Og6JabVcweMivKZfwGVi3bpHadtPPPE4nk+yRZ9w/DGgzZk7D7S2NpzL5RKa6ZOYm393E7PtYxNoUx4YGgGtSQIeWBWHiQYGWpiZOSSbdkLM80csmAPa/gHsd+CiMXb4EM4xM7OYLFlphKJL3u1Us49kuj7ijFNA8yI8zg8y7oeNmRIey4zPfhHPrZdwvLkZ6eRLHpnPnfguUxJsVCnjNUOSHd8Mr9eI+bgkfnqLyNo6EuI60kxIdQayWFdJ1Q8zs0IH9rP6GbDJAAABIElEQVSTfK/jJp6fkIzuCbmZjAIh1qjjXCmSd/PMnTeBdtF7MLDmd9EvU0IIIYQQOdBmSgghhBAiB9pMCSGEEELkQJspIYQQQogcOOlbMAOuEEIIIcR/FvTLlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA50GZKCCGEECIH2kwJIYQQQuRAmykhhBBCiBxoMyWEEEIIkQNtpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA50GZKCCGEECIH2kwJIYQQQuRAmykhhBBCiBxoMyWEEEIIkQNtpoQQQgghcqDNlBBCCCFEDrSZEkIIIYTIgTZTQgghhBA50GZKCCGEECIH2kwJIYQQQuRAmykhhBBCiBxoMyWEEEIIkYP/CzD5+i9SN5d7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01bcdf8828>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import glob\n",
    "\n",
    "images = glob.glob('data/TrafficSigns/img*.jpg')\n",
    "images.sort()\n",
    "fig = plt.figure(figsize=(10,10))\n",
    "\n",
    "index = 1\n",
    "\n",
    "my_images = []\n",
    "my_labels = np.array([1, 22, 35, 15, 37, 18])\n",
    "\n",
    "for fname in images:\n",
    "    img = cv2.imread(fname)\n",
    "    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n",
    "    fig.add_subplot(2, 3, index)\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img)\n",
    "    plt.title(sign_names[my_labels[index-1]])\n",
    "    my_images.append(img)\n",
    "    index += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict the Sign Type for Each Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Bumpy road')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAE/CAYAAAAnhFRiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuQXPWVH/Dvt7vnoRdIQhIIoReSVkJgLPAgnsICzCPObsAQe012CbvllJxknawrm6SIKynjZF1rb2J7N6mUU3IgxrtgzPImYJBAQkIgCc0IoYeFEQ8BErKkEXprXt198kdfZcfyzO/cmb6a7vn5+6lSaabP7XN/fbv7zO3u078fzQwiIrHI1XoAIiJZUlETkaioqIlIVFTURCQqKmoiEhUVNRGJioqayClI/pjkn9d6HDI4Kmq/xUjuJNlB8hjJgySfJTm11uMSqYaKmvyemY0GMBnAXgD/o8bjGTCS+VqPQeqHipoAAMysE8CjAOafvIzkyyT/Wa/f/4jkml6/G8l/SXIHyaMk/wvJWSTXkjxC8hGSjcm2i0nuIvkNku3JWeIfJLHLSO4lWeiV+w6Sm/oaa/Ly8IcknyN5HMB1JM8k+ROS+0l+QPI/kswl288iuYLkgWTfD5Ic2yvfJSQ3JrfhZwCaszquMvRU1AQAQHIkgN8HsG6AV70FwGcAXAHg3wNYCuAPAEwFcBGAO3ttew6ACQCmALgbwFKSc81sA4ADAG7ste0fAvibwH7/CYBvAxgDYA0qZ5hnAjgfwGcB/FMAf3zy5gH4CwDnArggGdu9ye1uBPBksq/xAP4OwB0DOgJSV1TU5EmShwAcQaWo/NcBXv+7ZnbEzLYB2ApgmZm9Z2aHAfwcwCWnbP+fzKzLzFYBeBbAl5LLH0ClkIHkeAA3A3gosN+nzOxVMysD6EGlIP8HMztqZjsBfA/AXQBgZu+Y2fJkv/sBfB+VwgdUinEDgL8ysx4zexTAhgEeA6kjBX8TidxtZvZi8r7UrQBWkZxvZr9Kef29vX7u6OP3c3r9ftDMjvf6/QNUzp4A4G8BbCc5GpVC94qZ7Qns96NeP08A0Jjk6517CgCQnATgvwNYhMqZXQ7AwWS7cwHstl+f2aF3HhlmdKYmAAAzK5nZ4wBKAK5JLj4OYGSvzc75jSsOzDiSo3r9Pg3Ax8n+dwNYC+ALqJxhhV56AkDvItSOytna9FNy705+/otk+4vN7AxUzgiZxPYAmEKSp1xXhikVNQEAsOJWAOMAbE8u3gTgdpIjSc4G8JUMdvUtko0kFwH4XVTewzrpJ6i8L/cpAE+kTWhmJQCPAPg2yTEkpwP4N6ic/QGVs7NjAA6RnALg3/W6+loARQD/mmSB5O0AFg7upkk9UFGTZ0geQ+U9tW8DuDt5fwwAfgCgG5WXlA8AeLDKff0KlZd9Hye5/rmZvdUr/gQqZ1tPnPIyNY1/hcqZ5XuofHDwEID7k9i3AFwK4DAq7+M9fvJKZtYN4HYAf5SM7fd7x2X4oSaJlKFAcjGAvzWz85zt3gXwVTN7cUgGJtHRmZrUDZJ3oPLe14paj0WGL336KXWB5MuoNP7elbRpiAyKXn6KSFT08lNEoqKiJiJRqeo9NZK3APhrAHkA/9vMvhPafsKECTZ9xvTQJlWjv4nUQJo3yXp6isF4oeBPxpFnFo8AL4fessmef7+1tbW1m9lEb7tBF7XkazX/E5XvC+4CsIHk02b2i/6uM33GdKx//dV+c6Z5qNC58aybx1uaJ1f1T0B6NzjFLvy3Vas/oe9I8d7/3v0Hg/Gzxp7h5jijMVz4zNLcFu+g6XOMgaL7x8a/X/L5fKqvr1XzaF0I4J3ky8vdAB5G5buDIiI1U01Rm4Jf/1LxruQyEZGaqaao9XU++RsvZEguIdlKsrV9//4qdici4qumqO1CZbK9k85DMuNCb2a21MxazKxlwkT3PT4RkapUU9Q2AJhDcmYye+iXATydzbBERAZn0J9+mlmR5NcAvIBKS8f9vWZ3EBGpiar61MzsOQDPDeAasMDH4em+suV9NJxFK0UWfSFD09LhSnFT/OPuJ+lh+KT/xdf6b+U56aONm4Px8ecFJ/gAANz+j/5hMN6Yz6THZdjwWimG6muS/m6yG4e+USAiUVFRE5GoqKiJSFRU1EQkKipqIhIVFTURiYqKmohERUVNRKJSg4VX+m++decGAwBzmglTNLT6cztVL80esmg3HJrWSX/+sJ179gbju1evdnPwbx4Nxo9ddIGb463584LxT82d4+bw/tJn8RgbuqbXemkk9saR3XNSZ2oiEhUVNRGJioqaiERFRU1EoqKiJiJRUVETkaioqIlIVGrQp9Z/P0qalposeswsk54ZZ5ssxlknPUadVnK3aWvbEIyXl73s5rCOo8F49xtvujm2rVoVjM+bdb6boyEfXjs0NwR9jjJ4OlMTkaioqIlIVFTURCQqKmoiEhUVNRGJioqaiERFRU1EoqKiJiJRqUHz7W+HemmcTTMOr5d0686P3BwHVr0WjBd3vO/maOoOj7WrodPNcez5F4Pxtxdd5+a4YN6M8AYZHNMsHh1xtQBrhXYRkT6pqIlIVFTURCQqKmoiEhUVNRGJioqaiERFRU1EojIM+9Tqo/9rKBZnzaTVLcUwjvcUg/Eta1vdHF3LVwTjjR09/kBaLg6GS794y01xonVLML7ltZVujrmz/jgYzzX4k2aahc8X0swz6fUYWhaTVaZ6jDkbZTAMZth1V1VRI7kTwFEAJQBFM2vJYlAiIoOVxZnadWbWnkEeEZGq6T01EYlKtUXNACwj2UZySRYDEhGpRrUvP682s49JTgKwnORbZra69wZJsVsCANOmTa1ydyIiYVWdqZnZx8n/+wA8AWBhH9ssNbMWM2uZMHFCNbsTEXENuqiRHEVyzMmfAdwEYGtWAxMRGYxqXn6eDeCJZHHhAoCHzOz5TEYlIjJIgy5qZvYegE8P4nr9xtKtvu51C2bRsTpUDb4ZNGhm0ATc9tYvg/Gjr7zi5ih99HEwXmw+082BGz8bDBdSHJBCa3gV90PPh1dwB4C3rloUjF8011/l3eg0zmbwEMuiXdV//GTDH6smiRQR6ZOKmohERUVNRKKioiYiUVFRE5GoqKiJSFRU1EQkKkM+SWSoFy2LhXd/25QRnrDw8PFuN8fbr60Pxk8s93u72FEOxvM3X+Hm6BkxJhgvX+FP19fZujkYb17/hptj+5rwwsxz58x0cxTy4fvFUpxPeJNAEuFjnmqbVM8nb6xputCc53aG7XI6UxORqKioiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlEZhiu0h6Vp4M2CN6FlmlGY17OYIknZGccrGze5OY6ufDW8wZ4Dbo7cuInBeGPLAjeH1yacG++vccF5vxOMd27b5uY48Hx4tfkdi650c8ybO8PZIs35RBYTotbJpKlD9LwEdKYmIpFRURORqKioiUhUVNREJCoqaiISFRU1EYmKipqIRGXY9akNYbtLkNsPl2I2S2e9W6TpD9p38EgwvsdZ3BcAOleFJ0UMT3dY0XTz1cF4d0OTn6QYntDQmzQRAHLXhSej7Nn6lpvj+IYNwfjm19vcHPNmOwse5/0JHnOZLFc8FOctKZ6U/oM9MzpTE5GoqKiJSFRU1EQkKipqIhIVFTURiYqKmohERUVNRKKioiYiUXGbb0neD+B3Aewzs4uSy8YD+BmAGQB2AviSmR08fcPsLdzE503emGoPGXT4psng9SMWU2TZsCG84vjhFS+7OYqHjwbjuUmT3RwNF84PxrvLfrOp11nNFM2o5UnnBOMj5832c2wJN+h+8twLbo73r7wqGJ85e4qbQwYnzZnajwHccspl9wB4yczmAHgp+V1EpObcomZmqwF8csrFtwJ4IPn5AQC3ZTwuEZFBGex7ameb2R4ASP6flN2QREQG77R/UEByCclWkq3t+9tP9+5E5LfcYIvaXpKTASD5f19/G5rZUjNrMbOWCRP91YBERKox2KL2NIC7k5/vBvBUNsMREamOW9RI/hTAWgBzSe4i+RUA3wFwI8kdAG5MfhcRqTm3T83M7uwndMPgdhnqV8qgP6xOJpFMJ9y79f7efl/V/38frN8YjHesbnVzWDEcH3lTeAJIAOhoyAfjOfp9alYO96HlUjw+ys4K0blrL3NzHHP61LpWrXVzbFgf3mbm7C+6ObzHByzFC60Ux70eWCYTYlboGwUiEhUVNRGJioqaiERFRU1EoqKiJiJRUVETkaioqIlIVFTURCQqw26F9uEiZ37TY5fTS7pu3etujs7lq4LxQleHmwOz5gTDDXPn+jmK4ebJcoP/97PsrQWfprG67Gw01Z+cMT9nZngYb7/v5mh/9qVg/P0rF7o5Zs46z9kixQEZJt3oWUzuepLO1EQkKipqIhIVFTURiYqKmohERUVNRKKioiYiUVFRE5GoqE/tNHE6rgAAv/hwdzB++LU2N0fnhk3BeNGZNBEARt9wZTDekWIh4ly+Oxgv9DS4Obw/sWkWmS6Xw0feLDyZJQCMXXxNML73rbfdHN2r1wTjm9ZucHPMmDk1GKe3GvYwksUC4ifpTE1EoqKiJiJRUVETkaioqIlIVFTURCQqKmoiEhUVNRGJioqaiESlBs23oTqapmXVa9Lzm03dNr8U89V5k0B2Ff1mwnVrws21R5etcHMcLXYF42fOu9DNUXaaPPMp7hYrhY9HTzk8TgDI551V3nN+42zOufMaUjTflmdMC8fdyRuB8jsfBOMfPbvMzbFz0VXB+MypZ7s5Mlz4/LTKcpg6UxORqKioiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSiUoM+tVBHSppuleonk8ugkw09TpJX2za7OQ6vfTUc377DzVFw7sLCjeEJIAEgZ87DgP4kkXSOaq6QYjFjZzJKbwJIAMjnw7ellGLCyx5nqOOuX+TmOLRjVzB++OX1bo5N68PbTJ92m5sj70wkmeXkjNXIchTuI43k/ST3kdza67J7Se4muSn59/kMxyQiMmhpXn7+GMAtfVz+AzNbkPx7LtthiYgMjlvUzGw1gE+GYCwiIlWr5oOCr5HcnLw8HZfZiEREqjDYovZDALMALACwB8D3+tuQ5BKSrSRb2/e3D3J3IiLpDKqomdleMyuZWRnAjwAsDGy71MxazKxlwsQJgx2niEgqgypqJCf3+vULALb2t62IyFBy+9RI/hTAYgATSO4C8E0Ai0kuQKW9ZCeAr57GMYqIpOYWNTO7s4+L7xvsDkO9fkw1U1x4oyx6CdM0JJ443hGMb1gdbqwFgI4XwpNAFlNMzjiqJTwJZGn8JD8Ji+F4inHQufNKJT+J13yby/kvLHp6OoPxxsZGNwcRnkiyPHu2myM/Y0ow3rnzQzfHe08uD8Z3X3mFm2OaM5FkmqeLuxB8nU1Eqa9JiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlEZ8kkiGWh6MfMbXrx+KMCfBNBlfq1f98abwXhpyyY3R9cHO4PxnqYRbo7Ri8ITFnbTX7zXetwt3Bz+/ZKGlyPF9J758DbHusL9hQAwoqkpGM/n/KfNmOvCPWQH/k94sWMAOPpyuNex7fWNbo5p5/Y1a9jfK6dYqbpQDj8fLJP7Pjs6UxORqKioiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlGpwQrtp1e6JtBwM+mR40fdDK8tXxWMd774ipujw2kmHdfS4uboGn9mMO5O8Ic0EzhW33xL+n8/K0teDH4fAJB3GqfzBX+SSO/WlrqdSTUBYPbcYDh37rluiuKu8CrvO558xs3x4WWXBONTp/rrhpTz4SPCFE3zQ0lnaiISFRU1EYmKipqIREVFTUSioqImIlFRURORqKioiUhUatCnVuVqw87VLUVjlrfJxk1b/WG0hSeBPNZ+wM8xYlQw3rTIX6wW5fAkkCVLsYiw0x+WT7GIcMlbiDjF/eItIt3d0+3mQCF8PJhioslcKTyOLNqymq7179ujDz0WjB95eZ2b4/U14Ykmz/vSHW4OOH1qaRb/9haqzuezK0U6UxORqKioiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlFxO95ITgXwEwDnoLL8+VIz+2uS4wH8DMAMADsBfMnMDoazGcz6n2AvTRMfnRXH0zRGHj/eGYyvfWG5m2P3q+HGR0vR9DrpqsuD8e5Cs5sjHzieAHD4yBE3xycHPwnGJ58z2c3R5KxqXkpx3+acJl+maAIuO7vp6Q7f9wBQyHkTXqZYKb4hPBllw9zf8XOcE57A0fb8ys2xfdnLwfjHVy1yc0yZdlZ4gzRPuhT3f1bSnKkVAfyZmV0A4AoAf0JyPoB7ALxkZnMAvJT8LiJSU25RM7M9ZrYx+fkogO0ApgC4FcADyWYPALjtdA1SRCStAb2nRnIGgEsArAdwtpntASqFD8CkrAcnIjJQqYsaydEAHgPwdTPz36j5++stIdlKsrV9f/tgxigiklqqokayAZWC9qCZPZ5cvJfk5CQ+GcC+vq5rZkvNrMXMWiZM9FeuERGphlvUWPmo5z4A283s+71CTwO4O/n5bgBPZT88EZGBSTOJ0dUA7gKwheTJScS+AeA7AB4h+RUAHwL44ukZoohIem5RM7M1QL8z690woL2ZoVzsv3+LTn8QAJQR7svKF8N9bADw82UvBuPHV/oLEeedBY/L48a7OZov+3Qw3pCiteedD98Lxh/76cNujs4THcH4+bP9nqpb77g9GPf62AC//6tc9BcR9iYjXLHsBTfH9u3bgvELL7rYzfG5m28JxksF/7F+5qIrg/Ejj/qLGRdWhvspX71ptZvj8+NuDsZHjhrh5jhx4kQw3tzk92SmpW8UiEhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlFRURORqAzpCu1mhmKxp994Q6HBzZHLh+vw+7v3uDm2vbQyGD/W5q/Q3uXEJy32J98rM9xw+MHeXW6OZx75u2C8y2l6TOP9d3e42yx//rlg/Iabws2oADBixMhgPMXcjFixfFkwvmH9WjeHN5/hulfDq54D/orjn7vxRjcHLpoXDOdW+U3inbs/CsbfW+bn+OWsWcH4jJlT3BzeBJ+lnnDT9EDoTE1EoqKiJiJRUVETkaioqIlIVFTURCQqKmoiEhUVNRGJypD2qRVLRRw6cqDf+OgRo90czIcngXzm2WfdHIdWhHtzuru9LjRg9NSp4fjcuW6OlatXBONtG1rdHF2d/uK81TLze4i2bd4cjHec8Mc5d/6Fwfi7777j5nh725ZgPM2C2T4/x9o1q4LxNBOiLrwyPEnkaGcxbAA4/NTPg/ETq9e7OTZfdVkwPmq0PwHo+bNnBOOdJ7rdHGnpTE1EoqKiJiJRUVETkaioqIlIVFTURCQqKmoiEhUVNRGJioqaiERlSJtvS6UiDh9u7zc+osmfJHLbtvCK5LvW+BP49bwdbuLsKfiHZfINi4PxkjOZJQC0t/d/LIChaazNitfU+u6OX7o53t3xdhYjySBH9byV4l9dFW68BoBmZ9LMqy7/jJsjt/r1YPz4R++7OXY7Dbrla65ycxw+eDQYP3H8iJsjLZ2piUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlEZ2j61YgmH9h/sNz72LH9R1OeeCy9We2DFaylGEu4hGnHBHDdDftLZwXip6E+sONLpQxpzxhlujmKxFIx7/VIAUPYmgUwxsaK3n3STMzrbpMlhzuSLKXJ4W2TTCRee7BQA8oXwNmSjm6Nh0RXBeM9Tz7g5OtZuCMZXr/R77m747NXB+KhR4YW9B8I9UyM5leRKkttJbiP5p8nl95LcTXJT8u/zmY1KRGSQ0pypFQH8mZltJDkGQBvJ5UnsB2b2307f8EREBsYtama2B8Ce5OejJLcD8F8niojUwIA+KCA5A8AlAE5+GexrJDeTvJ/kuIzHJiIyYKmLGsnRAB4D8HUzOwLghwBmAViAypnc9/q53hKSrSRbDx3K7kurIiJ9SVXUSDagUtAeNLPHAcDM9ppZySrrp/0IwMK+rmtmS82sxcxaxo71P80TEalGmk8/CeA+ANvN7Pu9Lp/ca7MvANia/fBERAYmzaefVwO4C8AWkpuSy74B4E6SC1Bp29kJ4KunZYQiIgOQ5tPPNQD66mh8bqA7O3zkIJ596dF+46Xnn3dzHFobnmxw7yf73RwTG8NNr2MvDzcKAsAxp4kzl2JV8+tvuTEYv7Z0vZuD3m5SdIoWnCbPfM5/l6K7K7yqfclrigUAhsdRLIUbjQGgVC6G4ylymNPQ7B5zAN2lnnCOFIdj3FkTgvGegp9kzMWfCsaPvxJeSR4Ajn64Mxjf+/qmYBwADlx6YTDefiD8+BkIfU1KRKKioiYiUVFRE5GoqKiJSFRU1EQkKipqIhIVFTURicqQThJ57NgJvLb6zX7jc86Z7+bYtr4tGG8q+Y1ZIz49LxjvbvQn8BtZcvqhzD+0JWdixZE5f3HnE04/FFL0mOXy4ckGmfdvS955KKXp22tuagrGe3qc2wogn3d63Yrh+w0ACs7t9fYBAEXn9qZ54uW84172+9Q6ncfyGZdf4+Y4+Hx4Islj68ILJgPAuvmzgvHfuy276Rh1piYiUVFRE5GoqKiJSFRU1EQkKipqIhIVFTURiYqKmohERUVNRKIypM23xR5D+77OfuNN7dvcHCe6OoJxml+n39i8JRhv2OqPo8jwfjpzfoPmGQ3hHHMbR7g5RjrNxuW836DZ4zT55gv+MW32bm+K41FuCDcB51LMeFlyxtpc8BuaS84MjqWc/7QpOE3PxRRN0WVnm56yP+ElmsNjber2m6K9LU68+76b4+Ot24Px7Rec7+ZIS2dqIhIVFTURiYqKmohERUVNRKKioiYiUVFRE5GoqKiJSFSGtE8NBiCwZmmh0e+7aXYm3yu7XTX+YrVHUiy82+30MhVT/L04cSI81mM44uaYy3Bv11n0J0XMu7c3xQLATn8g+1wP+9f1OJtYij61srPItKX6O+4sVJ3itnQ5x6yMFBNNOuPw71mg7OQ4muK2eM+o7hSrOxd2fBCO58KP44HQmZqIREVFTUSioqImIlFRURORqKioiUhUVNREJCoqaiISFRU1EYmK23xLshnAagBNyfaPmtk3Sc4E8DCA8QA2ArjLzLpDuXqKPfjVJ/v6jZ+34BJ3wNPPmxzeIEXjrDkNml0dJ9wceWcCP28fAOD1PXY6TcIA0O2tWt7gT4poTiNoc5PfGOmtal4opmiKLoW3KaWYFNFr0C2X/Bz5stPgnSJHzrn/y85tBYCcc3sb0twWZz/+KIAu55g2l/3H+sh504LxM0aPSjGSdNKcqXUBuN7MPg1gAYBbSF4B4LsAfmBmcwAcBPCVzEYlIjJIblGzimPJrw3JPwNwPYBHk8sfAHDbaRmhiMgApHpPjWSe5CYA+wAsB/AugENmdvLrZ7sATDk9QxQRSS9VUTOzkpktAHAegIUALuhrs76uS3IJyVaSraneZxIRqcKAPv00s0MAXgZwBYCxJE++O3wegI/7uc5SM2sxsxY6M1uIiFTLLWokJ5Icm/w8AsDnAGwHsBLAP042uxvAU6drkCIiaaWZT20ygAdI5lEpgo+Y2f8l+QsAD5P8cwBvALjvNI5TRCQVDuX7XLkcramh/zra1Oj3Q+WcV7BFZ3FfwJ8osNDo1/rRY0YH42edNdLNccEF04PxN9rec3Ps/+RQMN6Q9/vUCk3NwThTLIhsXn9gioeZ91hsbk7z+HD6w1I0ZjUWnJ47Jw74j+Vyin7KgrMQcU+H06MIoHl0+L5tbPYfHxPPPCMYv7zlIjfHNVcvDsY/s/Azbo6mprPazKzF207fKBCRqKioiUhUVNREJCoqaiISFRU1EYmKipqIREVFTUSioqImIlEZ0uZbkvsB9F6qeQKA9iEbQHWGy1iHyziB4TPW4TJOYPiMdTDjnG5mE72NhrSo/cbOKzN3uB3C9WC4jHW4jBMYPmMdLuMEhs9YT+c49fJTRKKioiYiUal1UVta4/0PxHAZ63AZJzB8xjpcxgkMn7GetnHW9D01EZGs1fpMTUQkUzUraiRvIflLku+QvKdW4/CQ3ElyC8lNJFtrPZ7eSN5Pch/Jrb0uG09yOckdyf/jajnGZEx9jfNekruT47qJ5OdrOcaTSE4luZLkdpLbSP5pcnldHdfAOOvquJJsJvk6yTeTcX4ruXwmyfXJ8fwZSX+yvLTMbMj/AcijsiLV+QAaAbwJYH4txpJirDsBTKj1OPoZ27UALgWwtddlfwngnuTnewB8t07HeS+Af1vrsfUx1skALk1+HgPgbQDz6+24BsZZV8cVlSW7Ryc/NwBYj8oaJ48A+HJy+f8C8C+y2metztQWAnjHzN6zyqruDwO4tUZjGbbMbDWAT065+FZU1mEF6mQ91n7GWZfMbI+ZbUx+PorKehxTUGfHNTDOumIVQ7pucK2K2hQAH/X6vZ7XDTUAy0i2kVxS68GkcLaZ7QEqD3wAk2o8npCvkdycvDyt+cvkU5GcAeASVM4u6va4njJOoM6O61CvG1yrotbXBO31+jHs1WZ2KYB/AOBPSF5b6wFF4ocAZgFYAGAPgO/Vdji/juRoAI8B+LqZHan1ePrTxzjr7rhaFesGD0atitouAFN7/d7vuqG1ZmYfJ//vA/AEKndKPdtLcjIAJP/vq/F4+mRme5MHexnAj1BHx5VkAyqF4kEzezy5uO6Oa1/jrOfjaoNYN3gwalXUNgCYk3wC0gjgywCertFY+kVyFMkxJ38GcBOAreFr1dzTqKzDCtTxeqwnC0TiC6iT48rKitv3AdhuZt/vFaqr49rfOOvtuNZi3eCaNd8mHzX/FSqfhN5vZt+uyUACSJ6PytkZUFkj9aF6GifJnwJYjMqMB3sBfBPAk6h8sjQNwIcAvmhmNX2Tvp9xLkblJZKh8gnzV0++Z1VLJK8B8AqALQBOLqj3DVTer6qb4xoY552oo+NK8mJUPgjovW7wf06eWw8DGI/KusF/aGZdmeyzVkVNROR00DfDUYD+AAAALElEQVQKRCQqKmoiEhUVNRGJioqaiERFRU1EoqKiJiJRUVETkaioqIlIVP4fSgdg217qHrcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f01b98fb6a0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Run the predictions here and use the model to output the prediction for each image.\n",
    "### Make sure to pre-process the images with the same pre-processing pipeline used earlier.\n",
    "### Feel free to use as many code cells as needed.\n",
    "my_images = np.array(my_images)\n",
    "\n",
    "#Normalize Images\n",
    "my_images = norm(my_images)\n",
    "\n",
    "index = random.randint(0, len(my_images)-1)\n",
    "image = my_images[index].squeeze()\n",
    "\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.imshow(image)\n",
    "plt.title(sign_names[my_labels[index]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model\n",
      "My Images Accuracy = 0.833\n"
     ]
    }
   ],
   "source": [
    "#Show accuracy on test images\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './model')\n",
    "    my_images_accuracy = evaluate(my_images, my_labels)\n",
    "    print(\"My Images Accuracy = {:.3f}\".format(my_images_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model\n",
      "\n",
      "Image  1\n",
      "Accuracy: 0.000\n",
      "\n",
      "Image  2\n",
      "Accuracy: 0.500\n",
      "\n",
      "Image  3\n",
      "Accuracy: 0.667\n",
      "\n",
      "Image  4\n",
      "Accuracy: 0.750\n",
      "\n",
      "Image  5\n",
      "Accuracy: 0.800\n",
      "\n",
      "Image  6\n",
      "Accuracy: 0.833\n"
     ]
    }
   ],
   "source": [
    "### Calculate the accuracy for these 5 new images. \n",
    "### For example, if the model predicted 1 out of 5 signs correctly, it's 20% accurate on these new images.\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './model')\n",
    "    image = []\n",
    "    label = []\n",
    "    \n",
    "    for i in range(len(my_images)):\n",
    "        print(\"\\nImage \", i+1)\n",
    "        image.append(my_images[i])\n",
    "        label.append(my_labels[i])\n",
    "        single_image_accuracy = evaluate(image, label)\n",
    "        print(\"Accuracy: {:.3f}\".format(single_image_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output Top 5 Softmax Probabilities For Each Image Found on the Web"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each of the new images, print out the model's softmax probabilities to show the **certainty** of the model's predictions (limit the output to the top 5 probabilities for each image). [`tf.nn.top_k`](https://www.tensorflow.org/versions/r0.12/api_docs/python/nn.html#top_k) could prove helpful here. \n",
    "\n",
    "The example below demonstrates how tf.nn.top_k can be used to find the top k predictions for each image.\n",
    "\n",
    "`tf.nn.top_k` will return the values and indices (class ids) of the top k predictions. So if k=3, for each sign, it'll return the 3 largest probabilities (out of a possible 43) and the correspoding class ids.\n",
    "\n",
    "Take this numpy array as an example. The values in the array represent predictions. The array contains softmax probabilities for five candidate images with six possible classes. `tf.nn.top_k` is used to choose the three classes with the highest probability:\n",
    "\n",
    "```\n",
    "# (5, 6) array\n",
    "a = np.array([[ 0.24879643,  0.07032244,  0.12641572,  0.34763842,  0.07893497,\n",
    "         0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.08594638,  0.0178669 ,  0.18063401,\n",
    "         0.15899337],\n",
    "       [ 0.26076848,  0.23664738,  0.08020603,  0.07001922,  0.1134371 ,\n",
    "         0.23892179],\n",
    "       [ 0.11943333,  0.29198961,  0.02605103,  0.26234032,  0.1351348 ,\n",
    "         0.16505091],\n",
    "       [ 0.09561176,  0.34396535,  0.0643941 ,  0.16240774,  0.24206137,\n",
    "         0.09155967]])\n",
    "```\n",
    "\n",
    "Running it through `sess.run(tf.nn.top_k(tf.constant(a), k=3))` produces:\n",
    "\n",
    "```\n",
    "TopKV2(values=array([[ 0.34763842,  0.24879643,  0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.18063401],\n",
    "       [ 0.26076848,  0.23892179,  0.23664738],\n",
    "       [ 0.29198961,  0.26234032,  0.16505091],\n",
    "       [ 0.34396535,  0.24206137,  0.16240774]]), indices=array([[3, 0, 5],\n",
    "       [0, 1, 4],\n",
    "       [0, 5, 1],\n",
    "       [1, 3, 5],\n",
    "       [1, 4, 3]], dtype=int32))\n",
    "```\n",
    "\n",
    "Looking just at the first row we get `[ 0.34763842,  0.24879643,  0.12789202]`, you can confirm these are the 3 largest probabilities in `a`. You'll also notice `[3, 0, 5]` are the corresponding indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model\n",
      "--------------------------------------------\n",
      "Image  1  -  Speed limit (30km/h)\n",
      "\n",
      "Probability - 0.852\n",
      "Prediciton -  Speed limit (20km/h)\n",
      "\n",
      "Probability - 0.148\n",
      "Prediciton -  Speed limit (30km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  No vehicles\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (50km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (70km/h)\n",
      "--------------------------------------------\n",
      "Image  2  -  Bumpy road\n",
      "\n",
      "Probability - 1.000\n",
      "Prediciton -  Bumpy road\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Dangerous curve to the left\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Bicycles crossing\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Road narrows on the right\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Double curve\n",
      "--------------------------------------------\n",
      "Image  3  -  Ahead only\n",
      "\n",
      "Probability - 1.000\n",
      "Prediciton -  Ahead only\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Turn left ahead\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Go straight or right\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Roundabout mandatory\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (60km/h)\n",
      "--------------------------------------------\n",
      "Image  4  -  No vehicles\n",
      "\n",
      "Probability - 1.000\n",
      "Prediciton -  No vehicles\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (20km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (70km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Yield\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Keep right\n",
      "--------------------------------------------\n",
      "Image  5  -  Go straight or left\n",
      "\n",
      "Probability - 1.000\n",
      "Prediciton -  Go straight or left\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (20km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (30km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (50km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (60km/h)\n",
      "--------------------------------------------\n",
      "Image  6  -  General caution\n",
      "\n",
      "Probability - 1.000\n",
      "Prediciton -  General caution\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (20km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (30km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (50km/h)\n",
      "\n",
      "Probability - 0.000\n",
      "Prediciton -  Speed limit (60km/h)\n"
     ]
    }
   ],
   "source": [
    "### Print out the top five softmax probabilities for the predictions on the German traffic sign images found on the web. \n",
    "### Feel free to use as many code cells as needed.\n",
    "k = 5\n",
    "softmax = tf.nn.softmax(logits)\n",
    "top_k = tf.nn.top_k(softmax, k)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, './model')\n",
    "    my_softmax = sess.run(softmax, feed_dict={x:my_images})\n",
    "    my_top_k = sess.run(top_k, feed_dict={x:my_images})\n",
    "    \n",
    "    \n",
    "    for i in range(0, len(my_images)):\n",
    "        print(\"--------------------------------------------\")\n",
    "        print(\"Image \", i+1, \" - \", sign_names[my_labels[i]])\n",
    "        \n",
    "        for j in range(k):\n",
    "            print(\"\\nProbability - {:.3f}\".format(my_top_k[0][i][j]))\n",
    "            print(\"Prediciton - \", sign_names[my_top_k[1][i][j]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Writeup\n",
    "\n",
    "Once you have completed the code implementation, document your results in a project writeup using this [template](https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project/blob/master/writeup_template.md) as a guide. The writeup can be in a markdown or pdf file. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Note**: Once you have completed all of the code implementations and successfully answered each question above, you may finalize your work by exporting the iPython Notebook as an HTML document. You can do this by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 4 (Optional): Visualize the Neural Network's State with Test Images\n",
    "\n",
    " This Section is not required to complete but acts as an additional excersise for understaning the output of a neural network's weights. While neural networks can be a great learning device they are often referred to as a black box. We can understand what the weights of a neural network look like better by plotting their feature maps. After successfully training your neural network you can see what it's feature maps look like by plotting the output of the network's weight layers in response to a test stimuli image. From these plotted feature maps, it's possible to see what characteristics of an image the network finds interesting. For a sign, maybe the inner network feature maps react with high activation to the sign's boundary outline or to the contrast in the sign's painted symbol.\n",
    "\n",
    " Provided for you below is the function code that allows you to get the visualization output of any tensorflow weight layer you want. The inputs to the function should be a stimuli image, one used during training or a new one you provided, and then the tensorflow variable name that represents the layer's state during the training process, for instance if you wanted to see what the [LeNet lab's](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) feature maps looked like for it's second convolutional layer you could enter conv2 as the tf_activation variable.\n",
    "\n",
    "For an example of what feature map outputs look like, check out NVIDIA's results in their paper [End-to-End Deep Learning for Self-Driving Cars](https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/) in the section Visualization of internal CNN State. NVIDIA was able to show that their network's inner weights had high activations to road boundary lines by comparing feature maps from an image with a clear path to one without. Try experimenting with a similar test to show that your trained network's weights are looking for interesting features, whether it's looking at differences in feature maps from images with or without a sign, or even what feature maps look like in a trained network vs a completely untrained one on the same sign image.\n",
    "\n",
    "<figure>\n",
    " <img src=\"visualize_cnn.png\" width=\"380\" alt=\"Combined Image\" />\n",
    " <figcaption>\n",
    " <p></p> \n",
    " <p style=\"text-align: center;\"> Your output should look something like this (above)</p> \n",
    " </figcaption>\n",
    "</figure>\n",
    " <p></p> \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Visualize your network's feature maps here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "\n",
    "# image_input: the test image being fed into the network to produce the feature maps\n",
    "# tf_activation: should be a tf variable name used during your training procedure that represents the calculated state of a specific weight layer\n",
    "# activation_min/max: can be used to view the activation contrast in more detail, by default matplot sets min and max to the actual min and max values of the output\n",
    "# plt_num: used to plot out multiple different weight feature map sets on the same block, just extend the plt number for each new feature map entry\n",
    "\n",
    "def outputFeatureMap(image_input, tf_activation, activation_min=-1, activation_max=-1 ,plt_num=1):\n",
    "    # Here make sure to preprocess your image_input in a way your network expects\n",
    "    # with size, normalization, ect if needed\n",
    "    # image_input =\n",
    "    # Note: x should be the same name as your network's tensorflow data placeholder variable\n",
    "    # If you get an error tf_activation is not defined it may be having trouble accessing the variable from inside a function\n",
    "    activation = tf_activation.eval(session=sess,feed_dict={x : image_input})\n",
    "    featuremaps = activation.shape[3]\n",
    "    plt.figure(plt_num, figsize=(15,15))\n",
    "    for featuremap in range(featuremaps):\n",
    "        plt.subplot(6,8, featuremap+1) # sets the number of feature maps to show on each row and column\n",
    "        plt.title('FeatureMap ' + str(featuremap)) # displays the feature map number\n",
    "        if activation_min != -1 & activation_max != -1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmin =activation_min, vmax=activation_max, cmap=\"gray\")\n",
    "        elif activation_max != -1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmax=activation_max, cmap=\"gray\")\n",
    "        elif activation_min !=-1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmin=activation_min, cmap=\"gray\")\n",
    "        else:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", cmap=\"gray\")"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}