Traffic_Sign_Classifier-Feb18-FINAL.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Self-Driving Car Engineer Nanodegree\n",
    "\n",
    "## Deep Learning\n",
    "\n",
    "## Project: Build a Traffic Sign Recognition Classifier\n",
    "\n",
    "In this notebook, a template is provided for you to implement your functionality in stages which is required to successfully complete this project. If additional code is required that cannot be included in the notebook, be sure that the Python code is successfully imported and included in your submission, if necessary. Sections that begin with **'Implementation'** in the header indicate where you should begin your implementation for your project. Note that some sections of implementation are optional, and will be marked with **'Optional'** in the header.\n",
    "\n",
    "In addition to implementing code, there will be questions that you must answer which relate to the project and your implementation. Each section where you will answer a question is preceded by a **'Question'** header. Carefully read each question and provide thorough answers in the following text boxes that begin with **'Answer:'**. Your project submission will be evaluated based on your answers to each of the questions and the implementation you provide.\n",
    "\n",
    ">**Note:** Code and Markdown cells can be executed using the **Shift + Enter** keyboard shortcut. In addition, Markdown cells can be edited by typically double-clicking the cell to enter edit mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Step 0: Load The Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load pickled data\n",
    "import pickle\n",
    "\n",
    "# TODO: Fill this in based on where you saved the training and testing data\n",
    "\n",
    "training_file = 'train.p'\n",
    "testing_file = 'test.p'\n",
    "\n",
    "with open(training_file, mode='rb') as f:\n",
    "    train = pickle.load(f)\n",
    "with open(testing_file, mode='rb') as f:\n",
    "    test = pickle.load(f)\n",
    "    \n",
    "X_train, y_train = train['features'], train['labels']\n",
    "X_test, y_test = test['features'], test['labels']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 1: Dataset Summary & Exploration\n",
    "\n",
    "The pickled data is a dictionary with 4 key/value pairs:\n",
    "\n",
    "- `'features'` is a 4D array containing raw pixel data of the traffic sign images, (num examples, width, height, channels).\n",
    "- `'labels'` is a 1D array containing the label/class id of the traffic sign. The file `signnames.csv` contains id -> name mappings for each id.\n",
    "- `'sizes'` is a list containing tuples, (width, height) representing the the original width and height the image.\n",
    "- `'coords'` is a list containing tuples, (x1, y1, x2, y2) representing coordinates of a bounding box around the sign in the image. **THESE COORDINATES ASSUME THE ORIGINAL IMAGE. THE PICKLED DATA CONTAINS RESIZED VERSIONS (32 by 32) OF THESE IMAGES**\n",
    "\n",
    "Complete the basic data summary below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples = 39209\n",
      "Number of testing examples = 12630\n",
      "Image data shape = (32, 32, 3)\n",
      "Number of classes = 43\n"
     ]
    }
   ],
   "source": [
    "### Replace each question mark with the appropriate value.\n",
    "\n",
    "# TODO: Number of training examples\n",
    "n_train = len(X_train)\n",
    "\n",
    "# TODO: Number of testing examples.\n",
    "n_test = len(X_test)\n",
    "\n",
    "# TODO: What's the shape of an traffic sign image?\n",
    "image_shape = X_train[0].shape\n",
    "\n",
    "# TODO: How many unique classes/labels there are in the dataset.\n",
    "import csv\n",
    "dict_signs = {}\n",
    "with open('signnames.csv', 'r') as csvfile:\n",
    "    lines = csvfile.readlines()\n",
    "    n_classes = len(lines) - 1 \n",
    "    count = 0 \n",
    "    for row in lines:\n",
    "        if count > 0: \n",
    "            dict_signs[count - 1] = row.split(',')[1]\n",
    "        count += 1\n",
    "\n",
    "print(\"Number of training examples =\", n_train)\n",
    "print(\"Number of testing examples =\", n_test)\n",
    "print(\"Image data shape =\", image_shape)\n",
    "print(\"Number of classes =\", n_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the German Traffic Signs Dataset using the pickled file(s). This is open ended, suggestions include: plotting traffic sign images, plotting the count of each sign, etc.\n",
    "\n",
    "The [Matplotlib](http://matplotlib.org/) [examples](http://matplotlib.org/examples/index.html) and [gallery](http://matplotlib.org/gallery.html) pages are a great resource for doing visualizations in Python.\n",
    "\n",
    "**NOTE:** It's recommended you start with something simple first. If you wish to do more, come back to it after you've completed the rest of the sections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "No passing\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEB5JREFUeJztnE2MHMd1x3+vu2dmv7m7/NKKor5CBZEly1JsK5F0cGLH\nQJCL40OC6BDEQADnYiABcoiRU44+JLkGUBADOQQwgjiIDcNGIigfcGxEosUwlCVKlESTlEiK5HKX\n+zW7M91VL4f3qmf1uSMu3SLEeYI4O93VVTWvX73Pf5WoKiNqhrKPegK3Eo2Y3SCNmN0gjZjdII2Y\n3SCNmN0gjZjdIO2K2SLymyLyioi8JiJfv1GT+riSXG9QIyI5cAr4IvAmcBR4UlVfunHT+3hRsYtn\nHwVeU9XTACLyLeBLwPsye25+Xm8/dBiA+h2Lfw770iU1V/8qg2dF6maD+0Zx23j1kO8xBRH1e4O+\ndpgKFy+c59ry0o4P7IbZh4A3tn1/E/iVd01I5KvAVwEWbj/Et/7lB6jkhLKy+5lrMrXvIYAkNmSJ\nYUrwS3lu7csqWBPJkBC8r9wHzahCafd9Hr3Sh8kCeWZ8if4GcrFWMShZHu1v9b40Eusfk3mf4s/Z\n/a88+eUPYNOAdsPs93qT7xJPVX0KeArggQcfUqkqArGWvOCMCsF+Up5tk0L/lVEjiduxTFKcmBKQ\naH1El8ooQhaNu5W3z4oWAKUqsbJrbX/RGqt6+lt9l+zc2hcoufdbRR/TP6UQQhkYVhXvhtlvAoe3\nfb8DuPBBDyhQqRJCmYQW1eB/JGkLxMymVUuvKpnfV2d6kTsTRemVLsUulZpllKX36y9Mqr6PR612\neq52QpbeakS1VfcLELVC0oKpbJzoK6MMAtnO6ibRbryRo8B9InKPiLSB3wO+u4v+PvZ03ZKtqpWI\nfA34VyAHvqmqL+7wDP2qJARFar3pS9LFR0OoVcaApDZY6uqjqjbs+1aXjcUlAE698SYAr29scGB+\nPwAH5+cB2DM5DcDUxAztzhgAua8OSUo5DIyt6qa1QQlJt+fiY5vaUZRW1kbfrT3fk3ajRlDV7wPf\n300ftxLtitkfmlTRsiRkOaJJJ7rRCSYtrUzAjVvm0t7vV0hpOnfjipmF547+FwBnzr7O1Tcu2b3V\nrrWPkBf20zoTUwBMzO8FYP7Oe7nr/gcAOHL3PQDsnZ0FoEDIknfkqrgvgrgkx+iry72eHEUiyJBe\n6yhcb5AalWxVNSktlNIlOS/sfYt7Jf0+FC5dMZqkti9d5N9+9EMAXjz+YwAuvXEZgM2NTTREtlPG\nwAfdcBFdOncGgAunXuL0sWcBePkTDwPwiYcfBeD+I0fYMzlufbgtyXIluTT9mPxs+x6iEMmG1NhN\nMxuhokBCJPNF1esl188+WlmO9nsAbJz9KQBP/+A7HDthgenaNTOM+GPCIIBMXphde/sLkOS8r62x\ntrYOwMqivbDF82cAuPbo53nokUcAOLBvzvosKwpXG6lPTX695iBx6OB3pEYapGYNJIpKRRUqROsk\nR30PIA+wfO4FAJ753j8BcPT5E5Sb7qalSNLbV1kGbXPlWm4MO50O0cP//taWtd8yV46qT57czY01\nAK6+YuM9311jZd1U12OfewKAhblpMkkphXdkU6RwV3Q40R5JdoPUuIEMZUVURT1Uzt0w5q4HVxdf\n5sfPfA+Ao8dM4vobFckGRtefU3tvA2Dv7XfQnjTJFk9SBSD3fEAyZrplruPa0iJXL5y3zrbMNmQ9\nu7d67mecDM8A0BpvA/DYE4+xb2rS+vVJtDy8V0qQYUOaj4DZZVWRFy2q2oNIS/oaAKeO/5DjzuTe\nuuczImjHvITZ2+4FYH7BDVi7jaY8xvaxfOmnYDQbnwBg6rZDdGb2ALBy3pKW60uL1qZX0j1/FoBX\nn3/O2u9b4LMP3g9AJ+v7OMY2EbHM4chA3nzUsOtnacoYQu1GxWBLeenCSQBeOHqC5SVzzdSNaMjb\n7LntTgDmFvYBIG3vNNM6xZlUU8bAPVOXp1jnyHN00gzp7OG77Jqrn9VLl8n6ZlCvnXkFgFMnnufO\nQ5ZnObx3yuflxjxTcsm31yw+kEaS3SA17PpZYKBVCW4Y49YKACdfOg7AG+cXqcpkgGx64/P7mD1g\n0iVFSoTbRwiB3CNOcZcuMPDScs+vxJiiIEWSKLoen92/AEDV7dJdtVVVrZsNWTn9Mq+9fh8A+6Yt\n4mx1fMXFiqLdGfq3jyS7QWpUskWVrCqJkiEp6Fi8CMCls+4ZrG6iHlqLS83E3Dx5J3kAqYQl/m+G\nVqlC4zo7ywahtaZapUtzNnAd6oh+wly7mQMH6HUt+Ile/elevsjZM2cAeOC+XwSgnZtn1CoyJJTN\n5LM/NIkgRRstq7pGeG3xLQAWLy0DUFWBqLbginFjQmtmD/QsstvsehrVC8ZFq1UXQzMvChSTkyRH\nMKuMeetrZvj6UdGUWCnMynb8ucmpWYpxy5eE0qLLsL7GyoVzAFxZMdVyYK8VIjqtoi4sD0MjNdIg\nNRzUQIi2kreumSQvX70KwPqGuYAxRCJm1IoxX64S6K5Y+9bEDAAT7r6hSuVLvvKCbLbVZbxlcnR1\nedH7sgLBeKudPD3UCxLdrmUSq7EZ2hNmNMtVL7tVfXpLNselVTPm3f5BANoorVyGQJgYjSS7QWrW\nQBLJwhZh+S1On7La8P+dfB2AddfBISriUIbMpTNurdP1DJ2smM7Os+3ZQvvbPUbK2T3QtmevXFsF\noJeZ1I9pTKgJxC3klvc0M50x2TYoQ+0eqhI2TMp7yybZCVYRsFL0TZkb6XU3OHvif3jx5Wd57iWL\n0DIsx5HSnkA9fUkVkvUV1hetzhhTqrUuicvgx3qSalwUGTeGrCxeAWC1Z99bIgySGZ6mbZmhLIpx\nJlLF3XkdVcgdv9JPLr4LAyoMr0RGaqRRalSy19eW+dG/f5vjZ86zEc2tm542CZ3KE+wIosPBEkys\nmBhjyqO9vkshxdtqYAAEl8DOxARjLbs/O+Fu2pj91LzIaxWhwbOKvoJm2h0yv5ayhVFj7bPXU5SE\nymohGoeW7pFkN0iNSvZWr+Tk6fNslS1qo1Z6xJYQUQKpUlBumjsoe2doTZurNzVtmbrxPXXajxjM\ngF32/HR7cpy2Aynn9pn5m2+buzY3P13j+PpblgdZWrRiwsxki+6ijZnAlkJONmarqjVmc8hqvS5k\nNxLrJyKHReQ/ROSkiLwoIn/s1+dF5GkRedU/54Ye9RalYSS7Av5UVY+JyDTwvIg8DXwFeEZVv+Fb\nPL4O/NkHdRRUWdlSKkKtl8uew3ZdJ0urQCqTrmrT3LYyLjB18AgAuQt0QpIKkBUmcbfdadm5IFA4\nLm/OpTLhrUOE6FnCrGN2Y/6gIaPysmRlw1daqiRlBTJhwdXUnOn/lof5eaGGURxSuHdktqpeBC76\n32sichIDwn8J+DVv9vfAf7IDs1EBFfIiq0GN6lDeEI0p2m6jXhMMXhFfvXqFlv/g5CEmmJhBZPyF\n1aFhpHwHVEwYYD/qQoIzqQZMrqzS3+j6fFxVdFqMHTAVdNDrnmNFx5/LyPM48Ml3oA+ls0XkbuAR\n4FngoL8IVPWiiBx4n2fqnQd5Mue3KA3NbBGZAr4N/Imqrg77NrfvPOh02qoCVRnI8qQGTILWXJqn\nx8bIN82oBd+b0V1apPA06NxBe6cprRqzbBAEeZF3+8wSPiW5d+r/wSDbWnla9drlt+h7FT64wR6b\n3svtR34JgIU5KxR7YEunVRIqubEFXxFpYYz+B1X9Z798SUQW/P4CcHm4IW9d2lGyxUT474CTqvrX\n2259F/gD4Bv++Z2d+kpZP1WlCo4b8XtbXmid6EyTjztY3aG6oddl9aIB3TsOBZ7eZxBgRerNT2/b\n4ZV2iyVxqreRKHlqtmUu48ZlgyGvLa3U7cRRVtOH7+W+e804z0yYOzlWpNWSESS7cQYSeAL4feAF\nETnu1/4cY/I/isgfAueA3xluyFuXhvFG/pv3f3df+DCDKUoZbKdY5qiitPEt7cjqbm0yNe6YPdeF\nvdUV6FoAsnjWsoTBIccTc7MUHZO4hK4SlJhAOvXWs6xuo55BXL1kVaKlt+xTNKNyj6btReDDn3mU\nB+47sq0HqBI7KiGL1dA6u+EUq01Y8qyO4lKNsNdLlfGM6WkzRFMtd9dixdaauWRs2tJfPn0KgLXJ\nKab2G5Zkcsb84HanTXQDngy59B1idvEiK15QCG6I1d3EqhCYNfV09yO2pfPTj3yK/RPmV6dSXkGC\nuWWQhVHx4GakhnEjAnlOkWXEBKhMu8YY5Bs2HeZ7rWXT04lJCpefvgPZxQut/dVrLK1ZUn/V/fi8\nVdTSpglL4iUzjbHe1Jqyderj5PP7OfypxwF4/HO/AcCDt+0nV4tog6/GssbvF+TDwqEYSXaj1KzO\nFui0c7IIXUcASF2VMdIQKFzSgruD61XF5JTp8Ykxc8k2VyxvEquS3EW1qlJeuxro0XcB2LX2ByvP\nm0zvt3B8+q4H+OXHfx2AI4ctNG8Vau4dkHt+JSFXogaIN2lZTFWp+qX52n5tUOqrN6pDZWqh5SnW\nzTLWFfTxMTOC4wc8T1Ft0F8xoxk8qqyiDvbQpP4LjwjHhLExq9CPTZthvezFywNz+9GepWk3l83D\n6U/dg1RuZN9x+IPtvRrUQHeikRppkJqHDAelipHCI8GgKU9h4lJIRrfrEt1P0LGCnrtpXf9seRV8\nZnKauYXbAWi7+okh1FKUibWLDlsrQ4+1dTOyly+Zz77hgrl89DJlaXtpDi0Y6H5vFetKfipwSDoe\ngwyt4XA700iyG6TGIcMI5Fk2iOxS5i1BgMtI/+1bIykyrXPOyW9Tz5usrCyz6mipPJ0RUhQ1GH6Q\nI3GpzDIKT9sVHQtW5n1FUEwwnlmuemvT7MZb516hdMXf8mBrfo/p+nFp+wbZUcH3pqPmw3URQ7Om\nix6I7J8z72J6ZpzL1zzv0faAR0O9ZbrTfjtkOEYdeHdhAIZPRwvFGjqcwvdt84npsAIPbkrl1WNH\nAXj1f38CPt+UOZw8dAcAn/+iHVP0q5/9JLOt1s+nUnOjKIRIlk5NmPL9LXvM5+31NulMGggyoZPI\n+mS5R3n+cqT2wzIyT3m2soQNKWo8djrHJJXRhFgXGVqpcuRRbJHn9UkQbCswhJ4VFy4tm4v5s3NW\njX/o059kejsiawcaqZEG6brP9buuwUSuABvAYmODXj/tY/h53qWq+3dq1CizAUTkJ6r6mUYHvQ76\necxzpEYapBGzG6SPgtlPfQRjXg/d8Hk2rrNvZRqpkQapMWbfzGdtfwBS9y9E5LyIHPf/f2tX4zSh\nRm72s7Yd0bWwHakL/Dbwu8C6qv7ljRinKcmuz9pW1T6Qztq+KUhVL6rqMf97DUhI3RtKTTH7vc7a\nvuE/5kbQO5C6AF8TkRMi8s3dAv6bYvZQZ21/1PROpC7wN8AvAA9jGPW/2k3/TTH7Q5+13TS9F1JX\nVS+palCrRv8tpg6vm5pi9k191vb7IXUTJNrpy8BPdzNOI/ns6zlru2F6P6TukyLyMKbyzgB/tJtB\nRhFkgzSKIBukEbMbpBGzG6QRsxukEbMbpBGzG6QRsxukEbMbpP8HJ6kKeojh6ogAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0b96fe8d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Data exploration visualization goes here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "import matplotlib.pyplot as plt\n",
    "# Visualizations will be shown in the notebook.\n",
    "%matplotlib inline\n",
    "\n",
    "import random\n",
    "index = random.randint(0, n_train)\n",
    "image = X_train[index]\n",
    "plt.figure(figsize=(1,1))\n",
    "plt.imshow(image)\n",
    "print(y_train[index])\n",
    "print(dict_signs[y_train[index]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "## Step 2: Design and Test a Model Architecture\n",
    "\n",
    "Design and implement a deep learning model that learns to recognize traffic signs. Train and test your model on the [German Traffic Sign Dataset](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset).\n",
    "\n",
    "There are various aspects to consider when thinking about this problem:\n",
    "\n",
    "- Neural network architecture\n",
    "- Play around preprocessing techniques (normalization, rgb to grayscale, etc)\n",
    "- Number of examples per label (some have more than others).\n",
    "- Generate fake data.\n",
    "\n",
    "Here is an example of a [published baseline model on this problem](http://yann.lecun.com/exdb/publis/pdf/sermanet-ijcnn-11.pdf). It's not required to be familiar with the approach used in the paper but, it's good practice to try to read papers like these.\n",
    "\n",
    "**NOTE:** The LeNet-5 implementation shown in the [classroom](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) at the end of the CNN lesson is a solid starting point. You'll have to change the number of classes and possibly the preprocessing, but aside from that it's plug and play!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementation\n",
    "\n",
    "Use the code cell (or multiple code cells, if necessary) to implement the first step of your project. Once you have completed your implementation and are satisfied with the results, be sure to thoroughly answer the questions that follow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([  210.,  2220.,  2250.,  1410.,  1980.,  1860.,   420.,  1440.,\n",
       "         1410.,  1470.,  2010.,  1320.,  2100.,  2160.,   780.,   630.,\n",
       "          420.,  1110.,  1200.,   210.,   360.,   330.,   390.,   510.,\n",
       "          270.,  1500.,   600.,   240.,   540.,   270.,   450.,   780.,\n",
       "          240.,   689.,   420.,  1200.,   390.,   210.,  2070.,   300.,\n",
       "          360.,   240.,   240.]),\n",
       " array([  0.        ,   0.97674419,   1.95348837,   2.93023256,\n",
       "          3.90697674,   4.88372093,   5.86046512,   6.8372093 ,\n",
       "          7.81395349,   8.79069767,   9.76744186,  10.74418605,\n",
       "         11.72093023,  12.69767442,  13.6744186 ,  14.65116279,\n",
       "         15.62790698,  16.60465116,  17.58139535,  18.55813953,\n",
       "         19.53488372,  20.51162791,  21.48837209,  22.46511628,\n",
       "         23.44186047,  24.41860465,  25.39534884,  26.37209302,\n",
       "         27.34883721,  28.3255814 ,  29.30232558,  30.27906977,\n",
       "         31.25581395,  32.23255814,  33.20930233,  34.18604651,\n",
       "         35.1627907 ,  36.13953488,  37.11627907,  38.09302326,\n",
       "         39.06976744,  40.04651163,  41.02325581,  42.        ]),\n",
       " <a list of 43 Patch objects>)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADlNJREFUeJzt3X+s3fVdx/Hny8KccRrAXkjTH15M+geYOLY0jAT/wGGg\nwGIxEQNRV5Gk/lESlsyYsn+qEEz3h9tcMolVGkoyYcRt0oxGbCoG/QNGYcjASqhYobahnWWMhQQD\ne/vH+RYO7b333J/nnN7P85HcnPN9n88553M+t72v+/l8f9xUFZKk9vzUqDsgSRoNA0CSGmUASFKj\nDABJapQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUqHNG3YGZrFy5siYnJ0fdDUk6qzzzzDM/qKqJQe3G\nOgAmJyc5cODAqLshSWeVJP89m3YuAUlSowwASWqUASBJjTIAJKlRBoAkNcoAkKRGGQCS1CgDQJIa\nZQBIUqPG+kzgUZnc9ui0jx3eccMQeyJJS8cZgCQ1qtkZwEy/5UtSC5wBSFKjDABJapQBIEmNMgAk\nqVEGgCQ1qtmjgLT4PH9COrs4A5CkRhkAktQoA0CSGmUASFKjDABJapRHAWlOvIaStHw4A5CkRjkD\nkKROa+eyOAOQpEYZAJLUqIEBkGRtkseTHEzyYpI7uvoFSfYlebm7Pb+rJ8lXkxxK8nyST/a91uau\n/ctJNi/dx5IkDTKbGcC7wOer6hLgCmBrkkuBbcD+qloP7O+2Aa4D1ndfW4B7oRcYwHbgU8DlwPZT\noSFJGr6BAVBVx6rq2e7+W8BBYDWwCdjdNdsN3Njd3wQ8UD1PAuclWQVcC+yrqpNV9QawD9i4qJ9G\nkjRrczoKKMkk8AngKeCiqjoGvZBIcmHXbDXwWt/TjnS16eoaskHH8i/Hox0knWnWO4GTfAz4JvC5\nqvrRTE2nqNUM9dPfZ0uSA0kOnDhxYrbdkyTN0awCIMm59H74f72qvtWVX++Wduhuj3f1I8Davqev\nAY7OUP+QqtpZVRuqasPExMRcPoskaQ4GLgElCXAfcLCqvtT30B5gM7Cju32kr357kofo7fB9s1si\negz4s74dv9cAdy7OxxgPrZ1EIunsNpt9AFcCvwd8P8lzXe0L9H7wP5zkNuBV4Kbusb3A9cAh4G3g\nVoCqOpnkbuDprt1dVXVyUT6FJGnOBgZAVf0rU6/fA1w9RfsCtk7zWruAXXPpoCRpaXgmsCQ1ygCQ\npEYZAJLUKANAkhplAEhSowwASWqUASBJjTIAJKlR/k3gIfEyEZLGjTMASWqUASBJjTIAJKlRBoAk\nNcoAkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKj\nDABJapQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWqUASBJjTIAJKlRBoAkNWpg\nACTZleR4khf6an+S5H+SPNd9Xd/32J1JDiV5Kcm1ffWNXe1Qkm2L/1EkSXMxmxnA/cDGKepfrqrL\nuq+9AEkuBW4Gfrl7zl8mWZFkBfA14DrgUuCWrq0kaUTOGdSgqp5IMjnL19sEPFRV7wD/leQQcHn3\n2KGqegUgyUNd23+fc48lSYtiIfsAbk/yfLdEdH5XWw281tfmSFebri5JGpGBM4Bp3AvcDVR3++fA\nHwCZom0xddDUVC+cZAuwBWDdunXz7J60PExue3Taxw7vuGGIPdFyNK8ZQFW9XlXvVdVPgL/mg2We\nI8DavqZrgKMz1Kd67Z1VtaGqNkxMTMyne5KkWZhXACRZ1bf5m8CpI4T2ADcn+ekkFwPrge8CTwPr\nk1yc5CP0dhTvmX+3JUkLNXAJKMmDwFXAyiRHgO3AVUkuo7eMcxj4Q4CqejHJw/R27r4LbK2q97rX\nuR14DFgB7KqqFxf90+h9My0dSBLM7iigW6Yo3zdD+3uAe6ao7wX2zql3kqQlM9+dwBoSdwJKWipe\nCkKSGmUASFKjXAKao3HauTpOfZF09nEGIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhplAEhSozwP\nQGfw/AKpDc4AJKlRBoAkNcoAkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKS0Fo\nWZrpchaHd9wwxJ5oKfj9XRzOACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjDABJapQBIEmN8kQw\nnbX828XSwjgDkKRGGQCS1CgDQJIaZQBIUqMMAElq1MAASLIryfEkL/TVLkiyL8nL3e35XT1Jvprk\nUJLnk3yy7zmbu/YvJ9m8NB9HkjRbs5kB3A9sPK22DdhfVeuB/d02wHXA+u5rC3Av9AID2A58Crgc\n2H4qNCRJozEwAKrqCeDkaeVNwO7u/m7gxr76A9XzJHBeklXAtcC+qjpZVW8A+zgzVCRJQzTffQAX\nVdUxgO72wq6+Gnitr92RrjZdXZI0Iou9EzhT1GqG+pkvkGxJciDJgRMnTixq5yRJH5hvALzeLe3Q\n3R7v6keAtX3t1gBHZ6ifoap2VtWGqtowMTExz+5JkgaZbwDsAU4dybMZeKSv/tnuaKArgDe7JaLH\ngGuSnN/t/L2mq0mSRmTgxeCSPAhcBaxMcoTe0Tw7gIeT3Aa8CtzUNd8LXA8cAt4GbgWoqpNJ7gae\n7trdVVWn71iWJA3RwACoqlumeejqKdoWsHWa19kF7JpT7yRJS8YzgSWpUQaAJDXKAJCkRhkAktQo\nA0CSGmUASFKjDABJapQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWrUwL8HIC2G\nyW2PTvvY4R03DLEnWgp+f89OzgAkqVEGgCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWqU5wFIGhnP\nHxgtZwCS1CgDQJIaZQBIUqMMAElqlAEgSY3yKCCNnEeCSKPhDECSGmUASFKjXAKSzlIunWmhnAFI\nUqMMAElqlAEgSY0yACSpUQvaCZzkMPAW8B7wblVtSHIB8A1gEjgM/HZVvZEkwF8A1wNvA79fVc8u\n5P0lzZ07j3XKYswAfq2qLquqDd32NmB/Va0H9nfbANcB67uvLcC9i/DekqR5WooloE3A7u7+buDG\nvvoD1fMkcF6SVUvw/pKkWVjoeQAF/GOSAv6qqnYCF1XVMYCqOpbkwq7tauC1vuce6WrHFtgHaazN\ntOQCLrtodBYaAFdW1dHuh/y+JP8xQ9tMUaszGiVb6C0RsW7dugV2T5I0nQUtAVXV0e72OPBt4HLg\n9VNLO93t8a75EWBt39PXAEeneM2dVbWhqjZMTEwspHuSpBnMOwCS/GySnzt1H7gGeAHYA2zumm0G\nHunu7wE+m54rgDdPLRVJkoZvIUtAFwHf7h3dyTnA31bVPyR5Gng4yW3Aq8BNXfu99A4BPUTvMNBb\nF/DekqQFmncAVNUrwMenqP8vcPUU9QK2zvf9JI3WoJ3ZOvt4JrAkNcoAkKRG+fcAxoBT6+k5Nu3y\ne7/0nAFIUqMMAElqlEtA0iJwuWJ8eOmN2XMGIEmNMgAkqVEuAUl9/GMpmo+z9d+NMwBJapQzAGmW\n3NGr5cYZgCQ1ygCQpEa5BCRJs7AclwCdAUhSowwASWqUS0DSiI3T0sI49UVLzxmAJDXKAJCkRrkE\nJC1D47SUM059GYX5fv5hXELCGYAkNWpZzwBa/81DU/PfRdv8/n/AGYAkNcoAkKRGGQCS1CgDQJIa\nZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjDABJapQBIEmNGnoAJNmY\n5KUkh5JsG/b7S5J6hhoASVYAXwOuAy4Fbkly6TD7IEnqGfYM4HLgUFW9UlX/BzwEbBpyHyRJDD8A\nVgOv9W0f6WqSpCEb9t8EzhS1+lCDZAuwpdv8cZKXFvB+K4EfLOD5y53jM5hjNDPHZ7B5jVG+uKD3\n/MXZNBp2ABwB1vZtrwGO9jeoqp3AzsV4syQHqmrDYrzWcuT4DOYYzczxGWycx2jYS0BPA+uTXJzk\nI8DNwJ4h90GSxJBnAFX1bpLbgceAFcCuqnpxmH2QJPUMewmIqtoL7B3S2y3KUtIy5vgM5hjNzPEZ\nbGzHKFU1uJUkadnxUhCS1KhlGQBebuJMSXYlOZ7khb7aBUn2JXm5uz1/lH0cpSRrkzye5GCSF5Pc\n0dUdo06Sjyb5bpJ/68boT7v6xUme6sboG90BHs1KsiLJ95J8p9se2/FZdgHg5SamdT+w8bTaNmB/\nVa0H9nfbrXoX+HxVXQJcAWzt/t04Rh94B/h0VX0cuAzYmOQK4IvAl7sxegO4bYR9HAd3AAf7tsd2\nfJZdAODlJqZUVU8AJ08rbwJ2d/d3AzcOtVNjpKqOVdWz3f236P0HXo1j9L7q+XG3eW73VcCngb/r\n6k2PUZI1wA3A33TbYYzHZzkGgJebmL2LquoY9H4AAheOuD9jIckk8AngKRyjD+mWN54DjgP7gP8E\nflhV73ZNWv//9hXgj4GfdNu/wBiPz3IMgIGXm5Cmk+RjwDeBz1XVj0bdn3FTVe9V1WX0zuK/HLhk\nqmbD7dV4SPIZ4HhVPdNfnqLp2IzP0M8DGIKBl5vQ+15PsqqqjiVZRe+3umYlOZfeD/+vV9W3urJj\nNIWq+mGSf6a3v+S8JOd0v+W2/P/tSuA3klwPfBT4eXozgrEdn+U4A/ByE7O3B9jc3d8MPDLCvoxU\nt1Z7H3Cwqr7U95Bj1EkykeS87v7PAL9Ob1/J48Bvdc2aHaOqurOq1lTVJL2fO/9UVb/DGI/PsjwR\nrEvgr/DB5SbuGXGXRi7Jg8BV9K5M+DqwHfh74GFgHfAqcFNVnb6juAlJfhX4F+D7fLB++wV6+wEc\nIyDJr9DbibmC3i+PD1fVXUl+id7BFhcA3wN+t6reGV1PRy/JVcAfVdVnxnl8lmUASJIGW45LQJKk\nWTAAJKlRBoAkNcoAkKRGGQCS1CgDQJIaZQBIUqMMAElq1P8DIHjSuJtvMNAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0b700c3ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Preprocess the data here.\n",
    "##Constrast Limited Adaptative Histogram Equalization\n",
    "#import cv2\n",
    "#clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))\n",
    "#for i in range(len(X_train)):\n",
    "#    X_train[i][:][:][0] = clahe.apply(X_train[i][:][:][0])\n",
    "#    X_train[i][:][:][1] = clahe.apply(X_train[i][:][:][1])\n",
    "#    X_train[i][:][:][2] = clahe.apply(X_train[i][:][:][2])\n",
    "#for i in range(len(X_test)):   \n",
    "#    X_test[i][:][:][0] = clahe.apply(X_test[i][:][:][0])\n",
    "#    X_test[i][:][:][1] = clahe.apply(X_test[i][:][:][1])\n",
    "#    X_test[i][:][:][2] = clahe.apply(X_test[i][:][:][2])\n",
    "\n",
    "### min-max scaling\n",
    "import numpy as np\n",
    "X_train = -1 + 2*(X_train/255)\n",
    "X_test = -1 + 2*(X_test/255)\n",
    "plt.hist(y_train, bins=n_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 210 2220 2250 1410 1980 1860  420 1440 1410 1470 2010 1320 2100 2160  780\n",
      "  630  420 1110 1200  210  360  330  390  510  270 1500  600  240  540  270\n",
      "  450  780  240  689  420 1200  390  210 2070  300  360  240  240]\n",
      "The max class is 2\n",
      "The number of training samples of class 2 is 2250\n"
     ]
    }
   ],
   "source": [
    "hist = np.histogram(y_train, bins=n_classes)\n",
    "hist_counts = hist[0]\n",
    "max_class = np.argmax(hist_counts)\n",
    "max_count = np.amax(hist_counts)\n",
    "print(hist_counts)\n",
    "print('The max class is ' + str(max_class))\n",
    "print('The number of training samples of class ' + str(max_class) + \" is \" + str(max_count))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating new data to balance the training dataset ... \n",
      "\n",
      "Class 0\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 1\n",
      " Percentage: 100.0%                           \n",
      "\n",
      "Class 2\n",
      "\n",
      "\n",
      "Class 3\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 4\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 5\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 6\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 7\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 8\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 9\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 10\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 11\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 12\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 13\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 14\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 15\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 16\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 17\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 18\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 19\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 20\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 21\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 22\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 23\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 24\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 25\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 26\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 27\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 28\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 29\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 30\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 31\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 32\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 33\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 34\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 35\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 36\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 37\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 38\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 39\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 40\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 41\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Class 42\n",
      " Percentage: 100.0%                                   \n",
      "\n",
      "Data Generation Completed !\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "import os\n",
    "from scipy import ndimage\n",
    "\n",
    "n_extra_data = 0\n",
    "diff_list = []\n",
    "for i in range(n_classes):\n",
    "    diff = max_count - hist_counts[i]\n",
    "    diff_list.append(diff)\n",
    "    n_extra_data += diff \n",
    "    \n",
    "X_new = np.zeros([n_train + n_extra_data,32,32,3])\n",
    "y_new = np.zeros(n_train + n_extra_data)\n",
    "X_new[0:n_train, :, :, :] = X_train \n",
    "y_new[0:n_train] = y_train\n",
    "\n",
    "print(\"Generating new data to balance the training dataset ... \\n\")\n",
    "count = 0\n",
    "for i in range(n_classes):\n",
    "    diff = diff_list[i]\n",
    "    indices_same_class = np.where(y_train == i)[0]\n",
    "    print(\"Class \" + str(i))\n",
    "    for k in range(diff):\n",
    "        percent = 100.0*k/(diff-1)\n",
    "        if k%50 == 0 or percent == 100.0:\n",
    "            sys.stdout.write(\"\\r Percentage: \" + str(percent) + \"%                           \\r\")\n",
    "            sys.stdout.flush()\n",
    "        random_index = random.choice(indices_same_class) \n",
    "        random_sample = X_train[random_index]\n",
    "        random_rotation = random.uniform(-10, 10)\n",
    "        random_sample = ndimage.rotate(random_sample, random_rotation, reshape=False)\n",
    "        random_sample = random_sample.reshape(1,32,32,3)\n",
    "        random_value = random.uniform(-0.2, 0.2)\n",
    "        if random_value >= 0:\n",
    "            random_sample = np.where((random_sample + random_value) > 1,1,random_sample+random_value)\n",
    "        else:\n",
    "            random_sample = np.where((random_sample + random_value) < -1,-1,random_sample+random_value)\n",
    "        X_new[n_train + count, :,:,:] = random_sample\n",
    "        y_new[n_train + count] = i\n",
    "        count += 1\n",
    "    print(\"\\n\")\n",
    "X_train = X_new\n",
    "del X_new\n",
    "y_train = y_new\n",
    "del y_new\n",
    "print(\"Data Generation Completed !\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "balanced_data = {\"features\":X_train, \"labels\":y_train}\n",
    "filename = 'balanced_train_data.pickle'\n",
    "if os.path.exists(filename):\n",
    "    os.remove(filename)\n",
    "with open(filename, 'wb') as f:\n",
    "    pickle.dump(balanced_data, f, pickle.HIGHEST_PROTOCOL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The new size of the training dataset is 96750\n",
      "96750\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADVVJREFUeJzt3H+o3Xd9x/Hna4nOMTearrelJOnSjfxhB7NKqAX3R6ej\nplWWDlawbDNIIfujhQqOEf2nmyLoH1MRXKGbwQhOV6auYQvrQubo9ke1qXa1XSfNnGuzhCYu/kRw\nVN/743xij+nNvTf33txzve/nAy7nnM/9nHM+50OSZ873/EhVIUnq52dmvQBJ0mwYAElqygBIUlMG\nQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTW2e9QIWcsUVV9SOHTtmvQxJ+qny2GOPfaOq5habt64D\nsGPHDo4dOzbrZUjST5Uk/72UeR4CkqSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQ\npKbW9SeBV2rH/r+f9RIkaVm+/v43X/L78BmAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQB\nkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqalFA5Bke5LPJ3k6\nyVNJ7hnjlyc5kuSZcbpljCfJR5IcT/JEktdO3dbeMf+ZJHsv3cOSJC1mKc8AXgDeWVWvAm4E7kpy\nHbAfOFpVO4Gj4zLALcDO8bMPuA8mwQDuBV4H3ADcey4akqS1t2gAqupUVX1pnP8u8DSwFdgDHBzT\nDgK3jfN7gE/UxCPAZUmuBt4EHKmqs1X1TeAIsHtVH40kacku6jWAJDuA1wBfAK6qqlMwiQRw5Zi2\nFXhu6monxtiFxiVJM7DkACR5JfAZ4B1V9Z2Fps4zVguMn38/+5IcS3LszJkzS12eJOkiLSkASV7G\n5B//T1bVZ8fw8+PQDuP09Bg/AWyfuvo24OQC4z+hqu6vql1VtWtubu5iHosk6SIs5V1AAT4GPF1V\nH5z61SHg3Dt59gIPTo2/bbwb6Ebg2+MQ0UPAzUm2jBd/bx5jkqQZ2LyEOa8H/gD4SpLHx9i7gfcD\nDyS5E3gWuH387jBwK3Ac+D7wdoCqOpvkvcCjY957qursqjwKSdJFWzQAVfWvzH/8HuCN88wv4K4L\n3NYB4MDFLFCSdGn4SWBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMG\nQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkD\nIElNGQBJasoASFJTBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQB\nkKSmFg1AkgNJTid5cmrsT5L8T5LHx8+tU797V5LjSb6a5E1T47vH2PEk+1f/oUiSLsZSngF8HNg9\nz/iHqur68XMYIMl1wFuBXxvX+fMkm5JsAj4K3AJcB9wx5kqSZmTzYhOq6uEkO5Z4e3uAT1fVD4D/\nSnIcuGH87nhVfQ0gyafH3H+/6BVLklbFSl4DuDvJE+MQ0ZYxthV4bmrOiTF2oXFJ0owsNwD3Ab8K\nXA+cAv5sjGeeubXA+Esk2ZfkWJJjZ86cWebyJEmLWVYAqur5qvphVf0I+AtePMxzAtg+NXUbcHKB\n8flu+/6q2lVVu+bm5pazPEnSEiwrAEmunrr4O8C5dwgdAt6a5GeTXAvsBL4IPArsTHJtkpczeaH4\n0PKXLUlaqUVfBE7yKeAm4IokJ4B7gZuSXM/kMM7XgT8EqKqnkjzA5MXdF4C7quqH43buBh4CNgEH\nquqpVX80kqQlW8q7gO6YZ/hjC8x/H/C+ecYPA4cvanWSpEvGTwJLUlMGQJKaMgCS1JQBkKSmDIAk\nNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTBkCS\nmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJ\nTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1tWgAkhxIcjrJk1Njlyc5kuSZcbpljCfJR5IcT/JE\nktdOXWfvmP9Mkr2X5uFIkpZqKc8APg7sPm9sP3C0qnYCR8dlgFuAneNnH3AfTIIB3Au8DrgBuPdc\nNCRJs7FoAKrqYeDsecN7gIPj/EHgtqnxT9TEI8BlSa4G3gQcqaqzVfVN4AgvjYokaQ0t9zWAq6rq\nFMA4vXKMbwWem5p3YoxdaFySNCOr/SJw5hmrBcZfegPJviTHkhw7c+bMqi5OkvSi5Qbg+XFoh3F6\neoyfALZPzdsGnFxg/CWq6v6q2lVVu+bm5pa5PEnSYpYbgEPAuXfy7AUenBp/23g30I3At8chooeA\nm5NsGS/+3jzGJEkzsnmxCUk+BdwEXJHkBJN387wfeCDJncCzwO1j+mHgVuA48H3g7QBVdTbJe4FH\nx7z3VNX5LyxLktbQogGoqjsu8Ks3zjO3gLsucDsHgAMXtTpJ0iXjJ4ElqSkDIElNGQBJasoASFJT\nBkCSmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkp\nAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTBkCSmjIAktSU\nAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaWlEAknw9yVeSPJ7k2Bi7PMmRJM+M0y1jPEk+\nkuR4kieSvHY1HoAkaXlW4xnAb1bV9VW1a1zeDxytqp3A0XEZ4BZg5/jZB9y3CvctSVqmS3EIaA9w\ncJw/CNw2Nf6JmngEuCzJ1Zfg/iVJS7DSABTwj0keS7JvjF1VVacAxumVY3wr8NzUdU+MMUnSDGxe\n4fVfX1Unk1wJHEnyHwvMzTxj9ZJJk5DsA7jmmmtWuDxJ0oWs6BlAVZ0cp6eBzwE3AM+fO7QzTk+P\n6SeA7VNX3wacnOc276+qXVW1a25ubiXLkyQtYNkBSPLzSX7h3HngZuBJ4BCwd0zbCzw4zh8C3jbe\nDXQj8O1zh4okSWtvJYeArgI+l+Tc7fxVVf1DkkeBB5LcCTwL3D7mHwZuBY4D3wfevoL7liSt0LID\nUFVfA149z/j/Am+cZ7yAu5Z7f5Kk1eUngSWpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAk\nNWUAJKkpAyBJTRkASWrKAEhSUwZAkpoyAJLUlAGQpKYMgCQ1ZQAkqSkDIElNGQBJasoASFJTBkCS\nmjIAktSUAZCkpgyAJDVlACSpKQMgSU0ZAElqygBIUlMGQJKaMgCS1JQBkKSmDIAkNWUAJKkpAyBJ\nTRkASWrKAEhSUwZAkppa8wAk2Z3kq0mOJ9m/1vcvSZpY0wAk2QR8FLgFuA64I8l1a7kGSdLEWj8D\nuAE4XlVfq6r/Az4N7FnjNUiSWPsAbAWem7p8YoxJktbY5jW+v8wzVj8xIdkH7BsXv5fkqyu4vyuA\nb6zg+hud+7M492hh7s/ilrVH+cCK7vOXlzJprQNwAtg+dXkbcHJ6QlXdD9y/GneW5FhV7VqN29qI\n3J/FuUcLc38Wt573aK0PAT0K7ExybZKXA28FDq3xGiRJrPEzgKp6IcndwEPAJuBAVT21lmuQJE2s\n9SEgquowcHiN7m5VDiVtYO7P4tyjhbk/i1u3e5SqWnyWJGnD8asgJKmpDRkAv27ipZIcSHI6yZNT\nY5cnOZLkmXG6ZZZrnKUk25N8PsnTSZ5Kcs8Yd4+GJK9I8sUk/zb26E/H+LVJvjD26K/HGzzaSrIp\nyZeT/N24vG73Z8MFwK+buKCPA7vPG9sPHK2qncDRcbmrF4B3VtWrgBuBu8afG/foRT8A3lBVrwau\nB3YnuRH4APChsUffBO6c4RrXg3uAp6cur9v92XABwK+bmFdVPQycPW94D3BwnD8I3Lami1pHqupU\nVX1pnP8uk7/AW3GPfqwmvjcuvmz8FPAG4G/GeOs9SrINeDPwl+NyWMf7sxED4NdNLN1VVXUKJv8A\nAlfOeD3rQpIdwGuAL+Ae/YRxeONx4DRwBPhP4FtV9cKY0v3v24eBPwZ+NC7/Eut4fzZiABb9ugnp\nQpK8EvgM8I6q+s6s17PeVNUPq+p6Jp/ivwF41XzT1nZV60OStwCnq+qx6eF5pq6b/VnzzwGsgUW/\nbkI/9nySq6vqVJKrmfyvrq0kL2Pyj/8nq+qzY9g9mkdVfSvJPzN5veSyJJvH/3I7/317PfDbSW4F\nXgH8IpNnBOt2fzbiMwC/bmLpDgF7x/m9wIMzXMtMjWO1HwOerqoPTv3KPRqSzCW5bJz/OeC3mLxW\n8nngd8e0tntUVe+qqm1VtYPJvzv/VFW/xzrenw35QbBR4A/z4tdNvG/GS5q5JJ8CbmLyzYTPA/cC\nfws8AFwDPAvcXlXnv1DcQpLfAP4F+AovHr99N5PXAdwjIMmvM3kRcxOT/zw+UFXvSfIrTN5scTnw\nZeD3q+oHs1vp7CW5CfijqnrLet6fDRkASdLiNuIhIEnSEhgASWrKAEhSUwZAkpoyAJLUlAGQpKYM\ngCQ1ZQAkqan/B04AbPTHFIZeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0b151eb860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with open(filename, 'rb') as f:\n",
    "    data = pickle.load(f)\n",
    "    X_train = data[\"features\"]\n",
    "    y_train = data[\"labels\"]\n",
    "plt.hist(y_train, bins=n_classes)\n",
    "print(\"The new size of the training dataset is \" + str(len(y_train)))\n",
    "print(len(X_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 1 \n",
    "\n",
    "_Describe how you preprocessed the data. Why did you choose that technique?_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** I used min-max scaling to normalize the training and test data to a range that goes from -1 to 1. This is very helpful as we would like our features to be of similar magnitude in order for gradient descent not to be biased towards certain features. Moreover, it's much easier for an optimizer to work on a \"well-conditioned\" dataset, i.e. 0 mean and equal variance. I chose not to convert the training and test images from colour to grayscale as I believe that colour might be important when detecting traffic signs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "### Generate additional data (OPTIONAL!)\n",
    "### and split the data into training/validation/testing sets here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_validation, y_train, y_validation = train_test_split(X_train, y_train, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 2\n",
    "\n",
    "_Describe how you set up the training, validation and testing data for your model. **Optional**: If you generated additional data, how did you generate the data? Why did you generate the data? What are the differences in the new dataset (with generated data) from the original dataset?_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** As soon as I plotted my original training dataset using a histogram, I realized how imbalanced it was. Hence, I decided to generate additional training data by rotating and changing the brightness of enough random images from each class so that all classes have the same number of instances as the class corresponding to the tallest peak of the above-mentioned histogram. <br> \n",
    "I split my original training data into a training set and a validation set, i.e. 80% and 20% of the original training data respectively. I retained the original test set (see above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Define your architecture here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "import tensorflow as tf\n",
    "from tensorflow.contrib.layers import flatten\n",
    "\n",
    "def LeNet(x,keep_prob):    \n",
    "    mu = 0\n",
    "    sigma = 0.1\n",
    "    \n",
    "    # Layer 1: Convolutional. Input = 32x32x3. Output = 28x28x12.\n",
    "    conv1_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 3, 12), mean = mu, stddev = sigma))\n",
    "    conv1_b = tf.Variable(tf.zeros(12))\n",
    "    conv1   = tf.nn.conv2d(x, conv1_W, strides=[1, 1, 1, 1], padding='VALID') + conv1_b\n",
    "\n",
    "    # Activation.\n",
    "    conv1 = tf.nn.relu(conv1)\n",
    "\n",
    "    # Pooling. Input = 28x28x12. Output = 14x14x12.\n",
    "    conv1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    # Layer 2: Convolutional. Output = 10x10x32.\n",
    "    conv2_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 12, 32), mean = mu, stddev = sigma))\n",
    "    conv2_b = tf.Variable(tf.zeros(32))\n",
    "    conv2   = tf.nn.conv2d(conv1, conv2_W, strides=[1, 1, 1, 1], padding='VALID') + conv2_b\n",
    "    \n",
    "    # Activation.\n",
    "    conv2 = tf.nn.relu(conv2)\n",
    "\n",
    "    # Pooling. Input = 10x10x32. Output = 5x5x32.\n",
    "    conv2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "    \n",
    "    # Flatten. Input = 5x5x32. Output = 800.\n",
    "    fc0   = flatten(conv2)\n",
    "\n",
    "    # Layer 3: Fully Connected. Input = 800. Output = 120.\n",
    "    fc2_W  = tf.Variable(tf.truncated_normal(shape=(800, 120), mean = mu, stddev = sigma))\n",
    "    fc2_b  = tf.Variable(tf.zeros(120))\n",
    "    fc2    = tf.matmul(fc0, fc2_W) + fc2_b\n",
    "    \n",
    "    # Activation.\n",
    "    fc2    = tf.nn.relu(fc2)\n",
    "    \n",
    "    # Dropout Layer \n",
    "    fc2 = tf.nn.dropout(fc2, keep_prob)\n",
    "    \n",
    "     # Layer 4: Fully Connected. Input = 120. Output = 84.\n",
    "    fc3_W  = tf.Variable(tf.truncated_normal(shape=(120, 84), mean = mu, stddev = sigma))\n",
    "    fc3_b  = tf.Variable(tf.zeros(84))\n",
    "    fc3    = tf.matmul(fc2, fc3_W) + fc3_b\n",
    "    \n",
    "    # Activation.\n",
    "    fc3    = tf.nn.relu(fc3)\n",
    "    \n",
    "    # Dropout Layer \n",
    "    #fc3 = tf.nn.dropout(fc3, keep_prob)\n",
    "\n",
    "    # Layer 5: Fully Connected. Input = 84. Output = n_classes (43).\n",
    "    fc4_W  = tf.Variable(tf.truncated_normal(shape=(84, n_classes), mean = mu, stddev = sigma))\n",
    "    fc4_b  = tf.Variable(tf.zeros(n_classes))\n",
    "    logits = tf.matmul(fc3, fc4_W) + fc4_b\n",
    "    \n",
    "    return logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 3\n",
    "\n",
    "_What does your final architecture look like? (Type of model, layers, sizes, connectivity, etc.)  For reference on how to build a deep neural network using TensorFlow, see [Deep Neural Network in TensorFlow\n",
    "](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/b516a270-8600-4f93-a0a3-20dfeabe5da6/concepts/83a3a2a2-a9bd-4b7b-95b0-eb924ab14432) from the classroom._\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** My final architecture is a slight modification of the LeNet architecture: <br>\n",
    "1) Convolutional layer with a 5X5 filter, strides of 1 and 'VALID' padding (input depth = 3 and output depth = 12)<br>\n",
    "2) A rectified linear activation unit <br>\n",
    "3) A pooling layer with a 2X2X1 filter and 'VALID' padding <br>\n",
    "4) Convolutional layer with a 5X5 filter, strides of 1 and 'VALID' padding (input depth = 12 and output depth = 32)<br>\n",
    "5) A rectified linear activation unit <br>\n",
    "6) A pooling layer with a 2X2X1 filter and 'VALID' padding <br>\n",
    "7) Flattening the layer in 6 to a 1-dimensional array with 5x5x32 = 800 entries <br>\n",
    "8) A fully connected layer with 120 neurons <br>\n",
    "9) A rectified linear activation unit <br>\n",
    "10) A dropout layer <br> \n",
    "11) A fully connected layer with 84 neurons <br>\n",
    "12) A rectified linear activation unit <br>\n",
    "13) A fully connected output layer with 43 logits, i.e. the number of classes <br>\n",
    "\n",
    "I initialized features' weights and biases using truncated normals (mean = 0 and standard deviation = 0.1) and zeros, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "### Train your model here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "from sklearn.utils import shuffle\n",
    "\n",
    "x = tf.placeholder(tf.float32, (None, 32, 32, 3))\n",
    "y = tf.placeholder(tf.int32, (None))\n",
    "one_hot_y = tf.one_hot(y, n_classes)\n",
    "keep_prob = tf.placeholder(tf.float32)\n",
    "\n",
    "EPOCHS = 50\n",
    "BATCH_SIZE = 512\n",
    "rate = 0.001\n",
    "\n",
    "logits = LeNet(x, keep_prob)\n",
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits, one_hot_y)\n",
    "loss_operation = tf.reduce_mean(cross_entropy)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate = rate)\n",
    "training_operation = optimizer.minimize(loss_operation)\n",
    "\n",
    "correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_y, 1))\n",
    "accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "def evaluate(X_data, y_data):\n",
    "    num_examples = len(X_data)\n",
    "    total_accuracy = 0\n",
    "    sess = tf.get_default_session()\n",
    "    for offset in range(0, num_examples, BATCH_SIZE):\n",
    "        batch_x, batch_y = X_data[offset:offset+BATCH_SIZE], y_data[offset:offset+BATCH_SIZE]\n",
    "        accuracy = sess.run(accuracy_operation, feed_dict={x: batch_x, y: batch_y, keep_prob: 1.0})\n",
    "        total_accuracy += (accuracy * len(batch_x))\n",
    "    return total_accuracy / num_examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "\n",
      "EPOCH 1 ...\n",
      "Validation Accuracy = 0.814\n",
      "\n",
      "EPOCH 2 ...\n",
      "Validation Accuracy = 0.927\n",
      "\n",
      "EPOCH 3 ...\n",
      "Validation Accuracy = 0.959\n",
      "\n",
      "EPOCH 4 ...\n",
      "Validation Accuracy = 0.965\n",
      "\n",
      "EPOCH 5 ...\n",
      "Validation Accuracy = 0.978\n",
      "\n",
      "EPOCH 6 ...\n",
      "Validation Accuracy = 0.982\n",
      "\n",
      "EPOCH 7 ...\n",
      "Validation Accuracy = 0.985\n",
      "\n",
      "EPOCH 8 ...\n",
      "Validation Accuracy = 0.988\n",
      "\n",
      "EPOCH 9 ...\n",
      "Validation Accuracy = 0.987\n",
      "\n",
      "EPOCH 10 ...\n",
      "Validation Accuracy = 0.990\n",
      "\n",
      "EPOCH 11 ...\n",
      "Validation Accuracy = 0.990\n",
      "\n",
      "EPOCH 12 ...\n",
      "Validation Accuracy = 0.992\n",
      "\n",
      "EPOCH 13 ...\n",
      "Validation Accuracy = 0.992\n",
      "\n",
      "EPOCH 14 ...\n",
      "Validation Accuracy = 0.992\n",
      "\n",
      "EPOCH 15 ...\n",
      "Validation Accuracy = 0.994\n",
      "\n",
      "EPOCH 16 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 17 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 18 ...\n",
      "Validation Accuracy = 0.994\n",
      "\n",
      "EPOCH 19 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 20 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 21 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 22 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 23 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 24 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 25 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 26 ...\n",
      "Validation Accuracy = 0.995\n",
      "\n",
      "EPOCH 27 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 28 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 29 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 30 ...\n",
      "Validation Accuracy = 0.996\n",
      "\n",
      "EPOCH 31 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 32 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 33 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 34 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 35 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 36 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 37 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 38 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 39 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 40 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 41 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 42 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 43 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 44 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "EPOCH 45 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 46 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 47 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 48 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 49 ...\n",
      "Validation Accuracy = 0.998\n",
      "\n",
      "EPOCH 50 ...\n",
      "Validation Accuracy = 0.997\n",
      "\n",
      "Model saved\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    num_examples = len(X_train)\n",
    "    \n",
    "    print(\"Training...\")\n",
    "    print()\n",
    "    for i in range(EPOCHS):\n",
    "        X_train, y_train = shuffle(X_train, y_train)\n",
    "        for offset in range(0, num_examples, BATCH_SIZE):\n",
    "            end = offset + BATCH_SIZE\n",
    "            batch_x, batch_y = X_train[offset:end], y_train[offset:end]\n",
    "            sess.run(training_operation, feed_dict={x: batch_x, y: batch_y, keep_prob:0.5})\n",
    "            \n",
    "        validation_accuracy = evaluate(X_validation, y_validation)\n",
    "        print(\"EPOCH {} ...\".format(i+1))\n",
    "        print(\"Validation Accuracy = {:.3f}\".format(validation_accuracy))\n",
    "        print()\n",
    "        \n",
    "    saver.save(sess, './lenet')\n",
    "    print(\"Model saved\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy = 0.961\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, tf.train.latest_checkpoint('.'))\n",
    "\n",
    "    test_accuracy = evaluate(X_test, y_test)\n",
    "    print(\"Test Accuracy = {:.3f}\".format(test_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 4\n",
    "\n",
    "_How did you train your model? (Type of optimizer, batch size, epochs, hyperparameters, etc.)_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** I trained my model using an AdamOptimizer based on backpropagation with a learning rate of 0.001. This optimizer minimized the \"loss\" function defined as the average cross entropy between the predicted logits and the actual one hot-encoded labels. During training I used 50 epochs and a batch size of 512. I also measured the validation accuracy for every epoch and printed that to screen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 5\n",
    "\n",
    "\n",
    "_What approach did you take in coming up with a solution to this problem? It may have been a process of trial and error, in which case, outline the steps you took to get to the final solution and why you chose those steps. Perhaps your solution involved an already well known implementation or architecture. In this case, discuss why you think this is suitable for the current problem._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** Being able to use the Amazon graphics cards, I was able to increase the number of epochs and batch size up to 50 and 512, respectively, without significantly increasing the training time. I started off my solution with a typical LeNet architecture which I ended up modifying as described in the previous answer. I went through a fairly long process of trial and error in order to find a good \"complexity\" for my model, trying not to fall into the risks of underfitting or overfitting. For example, adding too many neurons or layers to my model could easily result in overfitting. In my model's architecture I tried to correct this by adding a dropout layer. The LeNet architecture and convolutional neural networks are in general suitable solutions for the current problem, as our dataset is made up of images. In fact, these models can easily deal with spatial correlation, as in the case of images, and have very useful properties like translational invariance. Moreover, they present a hierarchical structure such that the very first layers capture very specific features of the images, whereas deeper layers compute more complicated \"concepts\".  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 3: Test a Model on New Images\n",
    "\n",
    "Take several pictures of traffic signs that you find on the web or around you (at least five), and run them through your classifier on your computer to produce example results. The classifier might not recognize some local signs but it could prove interesting nonetheless.\n",
    "\n",
    "You may find `signnames.csv` useful as it contains mappings from the class id (integer) to the actual sign name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementation\n",
    "\n",
    "Use the code cell (or multiple code cells, if necessary) to implement the first step of your project. Once you have completed your implementation and are satisfied with the results, be sure to thoroughly answer the questions that follow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "road_work.jpg\n",
      "speed_30.jpg\n",
      "bumpy_road.jpeg\n",
      "children.jpg\n",
      "stop.jpg\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEytJREFUeJztnHuQXFWdxz/n3tu339PznkkyIZnJCwKBBEKIuJYsgawF\nCgGjtURBV1Z8FCKLrgFcQHGFxAcKKcGCRRc3uKJi7bor6xawYiCIgDEmIWEmIe9kMo/M9Dy6e/p1\nz/7xO90mbCCdTPaSgv5VTd07997z6N/5nt/r/M5RWmuq5A9Zb3UH3klUZbaPVGW2j1Rlto9UZbaP\nVGW2j1Rlto80LmYrpd6nlOpUSm1TSt18ojr1diV1vE6NUsoGuoCLgb3AS8BVWuvNJ657by9yxlF2\nAbBNa70dQCn1E+By4A2ZnaiN65YJDQwlD6I8BUChKIOtPbnajkVBewBYSr5JpfPkc0UAotEgAAHb\nVGpZhIPyM6KpHABuwSMblQ9GQ/IuYHnlcmMFqVdZ8s7LS92e0mTG5L5UvafANn0NhQJSzggESyni\nsTr27TvAwOCQOhrDxsPsScCeQ/7fC5z3+o+UUtcB1wE0tzRw/w/u4D//44fYKen4QDIPQD4jjIo3\nREgWsgCEg/KjXlrXQ/fuEQDOO68dgKZaeafdMPOm1wMw/6VdALT1Ztm5MA7AszOb5FlsFICWhM3m\nPmnbjdYBkOqRutOBPJs2DwOQMDM+G9QkRmSAZ5zWAkCQEABRJ8B73vNhrlj66YoYNh5mH2kk/49M\n0lo/CDwIMGNmh86lXDLJIEGDlv4BYULAILxtQpBCxgWgb3gIgLBymD7tFACyRsuM5MIAhKwgZ/xk\nIwAL98qAjSo4t0sGLHipML3z3DEAXuwPcGBMcDuzZRCAxhph5sFUivZmM8CdUr4joahpl/tiswxK\no3cqAMm+PQyODlPwikfjFTA+Zu8FJh/yfxuw/80KKEtjh4oUaobJpAQd7XMFlYGMMCAYCFIXk+9f\n2dALQNYNMOf0ZgC6Ngp6J5zdCsCVIw4Ld2cAGDYiQ6dSeFEZvDN/tgWA12gEILdQs+hUGei9u6MA\n9A9LOTfcREuNDHB9i5R/dm+WKUYEzRmTcs1nyODuz4wyK6/wKlR747FGXgJmKKXalVIu8NfAL8dR\n39uejhvZWuuCUup64L8RffIDrfUrb1amUNQMJLPs2JcirwsANKZkioYSEQCyqVGGkzJdd+6VaR6r\njdLf1Q/AvIUNACzuTwBw4dOd5F1BnjaSTV1zNfrnj0ubEXl28XMii1+YFeN3CdETpPsAWNclMrzG\nskgfSAGwbZu0NxZ3GbZrAHBbBfUH8zIrx9IZcqlhdNGriGfjESNorZ8AnhhPHe8kGhezj5VsGxK1\nNvHWEIE+QShBQVn0FKNviy6J1wTln1sgiIrMj2NlRZGOtNYCMP3RHVLnWA+FrCg/e9kVAMTvWcmw\nMRvz//QwAAlENyz5cQ83TRXlGjSyuG22KOQdv+kjOSTyv6FevknnCpw+SRRoW6v0yxmQcjWtQeqb\nI9iByqRx1V33kfxFtqWoidmce1oNa7cKgqJhY/LFRA7WF/LoMwRV0V1iUqn6PAeDIqNbnu8BYMoL\nBwDI20X0pDb5/itflnaKReJflfuh378IQGprFwAX9ET4UFb0xeZlUwAIZqQvhajD7NkTAejflQTA\nw+EPa8QqGtgp+uX0xfJNz54eZuxKkssVKvr9vjI7V4A9fZquFx0ijcLIOuk3rpmK69ZnUO0yAN01\nIgqSa7O0hsSGXvlfojQDjpiK6ZERgt/9tjxrEgdG5/M4MbGvI/fdA0Dq/VcCMILHTRlRiJ8flDoL\nM8RZudSp51drdgIwYJiNgqw2PsEOUbJ1TSLesqk8+Vm5svd7NKqKER/JV2QrzyOQGaNr027a3i0e\nYVdKpmDnZpmiPTmHWS/Ks2JQEJOZ6LD0ZTG7OnrkXTIj/7tLLqfmQx+UBorGk3McCllBbeDsuQBY\ny28CoHDbHYRqRTl/5lciku6+RhTkxBkNXByUfg0l5Xpgf5LXegXRuf40APs3SduhiKLoHcFtfgOq\nIttH8ldBuhbxU0LMWtzAtmcHANgbFtNPh0WOTuktkPLEtNqWEgfjfUNxPrhdzLsR5HunSdx3ddst\nvLJZAo2zZ82SZ0rhBN3D2q7//PUADK59gdGnngTgTE/MyE+uPwjAPV6ROY1i3qX2yExLByyKEWGT\n3SqyetgWLO/Y28NQskCxWBm2fWX2WM6jc+cYvYNjpByZ8sN/7JaXafm/MxGhMSHBkUBBlOBV61Io\noxCzQzKFm+/7JgA3/Wg19915BwC333kXAH/3uc/yxJNPS/1JUXRtE0UTX/jtFai/Ekd3JCnK9rxn\nhFkTXx3j+fNEsYaGZMCznkXYNgoyLQNeExNraUpHA82tIZyqnX3yka/IpqCx+rPs6EzTNmUCAHZQ\nvLP9myRgeM70KUw6bSoAM5+TiN277RRDI4Jo56JFUm7pUgAWh3/F5kWXAPDPq38CwE9/8W+8umUr\nAI3NogzPmjkDgIc7prJ6xdcASF99LQCheunDLYM5Pr1bZlDLbIl1t9XFqQ3L/Y+ffAmAhWdIiHXa\nhEYcF9RRlw2Eqsj2kfw1/SxFIBIiEnMZM15bS60oxpmXTwfAtkIkukV5XtUr0b+MV0RHRHHFV94J\ngGdWUi699FJK+unDyz4m7QSDNLXKzNEFkbNf/JKYfnfd/Q22njUHgNk3fAaAwe+uAmBybT13edLO\n941cbqptZHJQnJ73TBWlPNAts/Dgzl5OmbwAvKrMPunIV2RrJbGMof4s8aiYUY2ni7ysS0hXfr++\nj6VrJZbc7Aq6BgcHcO+8HYDQqacBlJ0WAg6/fvIpeZYRp8MO2IykxQoZS4v5eOPyWwCYPa2DJx77\nGQBz75D4CcZyyezYyel/eg2AiybLTNtck6LNlaXWuXNltvT3i5COuDXU10Rw7Mow6yuzLa2IegHm\ntLZz/tmisJ5a8xwAT3eLyPhIJsAllgxAckSe6flnE/7UJ+W+YII+5gdalsX5C2Wd+XkTdPrKrTdz\n3Q03AlATk6WvXbt3A3DuvLl84qMfkbocEWGxVRJbSV+2tMy4xb8VUdHVGOLX3TKwI+mS6JMludTB\nXiZnz6SgKwtEVcWIj+Qrsh3bpqEmwRnt7WzbI85M537jKJiA/N9msuRNCkPR2FQtq74L4fBhdXm5\nXPl+MClm4fx58+Q6/xwaasQ5OWicmnBIPMotnZ3EaxOmEmkzcu4CAArLv0judjEL40r6cMlTO/jy\nqeJpNsWlD0ODIubW/nE7l7z3SnRlq2JVZPtJviI7m8+zvaebbGGUmRMFLZFFCwG48BmJb7QfGGCg\nX5BTf9utAKzasIGNq+4H4Kt3/AMAEyaI3NTA9x9+CIArLv0AALd/9U62bhNFd/VHrwIgFhWk3//g\nQ2zYtAmAdy0wiM5KHCR+w/UMrlkLQOq3vwVgfk8tl1myJPdYm8y0Sa6JlYx5JIfGTs7YSNgNcFrb\nJFJ7NrFxv4Q3zz8oHX3vHmHwUD5D7UIZgBfPF8W3ctk1jGbkB3d2dQKw7CPCxLpEDe+/+CIAPvdZ\nyUy6cfnNTJs+DYBv3PV1gLJd/8jq1dz/0A8BmDVDlHR9nXiIXrFI/DsrpR8XiVc6msrw8T4RAD2T\nZHFiZ62IpIVndhCLhLCsylzIqhjxkfwVI9k8u7bvo+fAfhxPzKX5a7YB4JpMyXwOcjf/PQBfe+B7\nACiKTJw0CYA9xoRbfstyAB5/7DFW3r3isHb27tnN9BliJzc1Nh327nv3focvfEls7os2/MnUITGV\n9qmn4E6ZCkB4xT8CkLn208QCYiIu+dM+AB64oAOArQf6UFofMQ/vSFRFto/ks+lnUZuI0dmbZHlW\n0DJby3gP90t2UsvyL3LvfvHYNv7PGgDqW5pI52TRIBSV2EVtUZTa5s2bufC9FwCgLKnLdSxcE//W\nZqmsFEu5etkyFs4/F4CbbxWEX7ZkCQD3f28Vf/Gud0k7H5QF4uxv1jD6o9UAzDcLFku2SOzmS/kU\nueQoXoUZUUdFtlJqslLqN0qpLUqpV5RSnzfP65VSTyqltpprXUUtvoOpEmQXgC9ordcppeLAH5RS\nTwIfB57WWq8wWzxuBpa/WUVjhSLbDg7jrd/F/KKYYumsWAmxc84G4PlFF/DNaz8BgBctpfKOos0M\nSJvc7VxekP7Iv/6UT3zyOgCSJsX4tX37uGzuOQAoWxBekqtFYIaJbf/Lo48CcKGxZkKxWHl2lBiT\nWPl1htf9EYDBrWJOLjbryhsmhImEguWk/aPRUZmtte4Gus39iFJqC5IIfzlwgfnsEeAZjsJsx7Zo\njEW5oRAgYlL7DxZMIs41Eq9wUqOsuEZCpa7J/Sh6Hk7JvDKxC22ZCrTHsFmDfG2rKNu7P/VZ5syR\nMGr+gCTz6BJDLIuiESm9+yX+8f07xWuc29FBvl/WI0ufq2AQ947bAMj8jcRn8MR7vaQ/QF+sEceu\nTBofk4JUSk0F5gG/B1rMQJQGpPkNylynlHpZKfXy6MjosTT3tqOKFaRSKgY8DtyotR5WFU6dw3Ye\nTOvQk2MxJqsAXkZiIsG4oDe34lsAnOXYLDBxEM9kGgkq5b40zTGxCwWM3PsAAAvMs4AbYMx4hcM5\nuSpXRBK2XRYp9Y78/OaAXPszGSxlZkwpbKoUyqzUOyGpI2tEWMdIigbbJlSh8VcRspVSAYTRj2qt\nf2Ee9yilJpj3E4Deilp8B9NRka0Ewg8DW7TW9xzy6pfAx4AV5vrvR63LAhW0eH7+VGJdYuqFRiS4\nT1HQki7m8UzA30JMKqUPQc7rthIqFI4rSZm5hpryc8sguTwDTQoEw6OY7Txosxcmb+q0Latcvyq1\no0GbsJ4y8W/LzKBtM9op1sTIn8DFg3cDVwMblVLrzbNbESb/VCl1LbAb+FBFLb6DqRJr5DmOvDMM\nYNGxNGZbDnXRWtbNnsXPZ0s6QHBE8uiUMnsS7QDKaHvbdE8X89i2INVSgrKcLShr6R/kL9fIhiI6\nxI222idjmT2OhR4JeLm/k1WcjfNmse08yf9zU2nTpshpC4+SRHeMPLeVhWVmVsBYQNrI8GRDnNmJ\nBJZd2jX55uRv3ohSqIBL2K3BDQvzPBPk94rSYTsQoiQ1iqWofDGPZxSXWxIBZmdYrH+AsMlsyj7z\njDTzdBHPMM0yU9w2YdBgewuZibJa7mUkPuM5pQ2weUIlZeiKp6osCBilnDfSwjamXtyDsONQqbFQ\njY34SP7mjSiFGwwSi0VwAoIg7RmTrCgos2yb0sZCzwqZXhaxTL6dbUw5x5byEw4MoksOj4mb4P05\nEldyZgomttK6t49GT2aJF/IOqyvvOQTMin4p2R5dJGgyAdygLB4rsxdHYREMR7CsyjBbRbaP5K/M\nNhRxXdywoLC0oqSMGeahyhG6EnmeonR6RI8ystRsiW7d10vRKLOSE6QOKV+6LxjHJTaQJHJAlPLA\n6SbFOCWLwvGIjW0UtTY6IuAG8Ezs3VLaXDF990DZvLH9cDj5LkacQIBgOIprEiq1KjHGKKFCjrKG\nNF5jMZ+nOykBq/0pedcYkut2T3G28ehCRqFqpcpVlM1lMxDW6DCt+7YD0D9Xgl9pxCpJuBFsjNIs\ndUtrwiFR4o5RpLaxtwuFQtnmroSqYsRH8hfZyFSMRKOUAVGOcUhXQqFQeatbwWQ/5XIFcEQ5xWKC\nrpLIePwDS3j21VcBmNktEb764SFCpqxtTL+8yRWxzjwV730XAzC1VpLuB3XetJOmvk6UYcEo7IDt\nlIVEKS6TM3EXNxzCdlQ1ZfhkJJ9Thi3C4Rh2wKGkVEomIGZxIDuWQlmC2rKMdF1SI+JVxiKCRkuL\nApuSqCVnFne7tIl1pDMoU18iLjOisUm2Uzc31hHIify3jEcYNZFHxryyw1KCa9ANYjl/jp0f8gq8\nAsdy7FMV2T6Sz6afplDMYVkWlnEknNLGpDET3w6HsAy6SqAZSvXQPyTvQ56Uq40KYrN5RS4lSC0e\nkk2qMRukRqRcPicrML37BmhuEblcGxGLqJQrWONqmk0sPWP6g2WRHZP6I1GZVfGQmJ+e5+Ecwz4P\n/z3IgEs4FCwfxFVSgrGITGVtKbJ5c7CWETHNzY10pEWJbd8jeSMHAmaBAU3BJFkqYw/nPV0+NKvE\niNKaQMANcXBAyrY3idIsBZ0S9W3lDaRBs1DgFTWuiZcUS+nKRgR6hTyeV6x412lVjPhIx32u33E1\nplQfkAL6fWv0+KmRyvs5RWvddLSPfGU2gFLqZa31fF8bPQ76/+hnVYz4SFVm+0hvBbMffAvaPB46\n4f30XWa/k6kqRnwk35h9Mp+1/SaZul9RSu1TSq03f5eMqx0/xMjJfta2yeiacGimLrAE+DAwqrX+\n1oloxy9kl8/a1lrngNJZ2ycFaa27tdbrzP0IUMrUPaHkF7OPdNb2Cf8xJ4Jel6kLcL1SaoNS6gfj\nTfj3i9kVnbX9VtPrM3WBB4BpwFwkR/3b46nfL2Yf81nbftORMnW11j1a66KWzMqHEHF43OQXs0/q\ns7bfKFO3lBJt6Apg03ja8SWefTxnbftMb5Spe5VSai4i8nYCnxpPI1UP0keqepA+UpXZPlKV2T5S\nldk+UpXZPlKV2T5Sldk+UpXZPtL/Ai5dyPAwCE2UAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a86cb3be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEkNJREFUeJztnHmQHNV9xz+vu6d7Zvac3ZWEhIQkxCEE4rbEYSnCRokL\nCsxhcxiIcVzgsouyXaaSEP+RglzlqmBX4qrEKWIoRICQFDdE3IjDBplDSCB039pLqz1md3bOPl7+\n+L1ZCSykkVY0KjS/qq2d6XlX//r7fvdrpbWmTvGQ9UUv4GiiOrNjpDqzY6Q6s2OkOrNjpDqzY6Q6\ns2OkcTFbKfUNpdR6pdQmpdQdh2tRX1ZSh+rUKKVsYAOwGOgE3gWu11qvOXzL+3KRM46+84BNWust\nAEqpR4BvAp/JbNdL6HSDh9aa9rZJALRl2gCoPnSl1DiW9Mf06XGHR0ZQlmzo5sbGT7ThgHObsZB2\n5UqJSrlEb28fw8MjB1z4eJh9LLBzr++dwPxPN1JK3QrcCpBKuyz4+lwCHXLDdT8B4MZv3QCA7/sA\n2LZdE8OrDNp7X6p9/W7+27YNwHMvv0LCSwGweOGFAJQCmduy9i9Vq2M5Zqzt2zeybes6fviDnx1w\nvTA+Zu+LI38kk7TW9wD3ADS3NuhyyadQrvDm716V36MIgEsWXwpAOpUi6SWrfWWifTD/QIypUkRk\nFitjOM6eawdPZj3mLkvlIqtXLqNYyNXUezzM7gSm7fV9KtC9vw6+H9KzaxQ3leS9le8D0NO3C4Dz\n530VgHYjVmAPk6NoD3Msc23Fmo0A7B4awU3IbVQfgLIUxZJBq3lO2jCqc6jCyTMnfnJhNYoupaxP\nrAFsRgoDhFFQU//xWCPvAicqpWYqpVzgOuDpcYz3padDRrbWOlBK3Qa8ANjAfVrrj/ffS4Fy8Mua\nYqkCQKm4A4DfPvAfAFx31bUcP+NEAPyKoLO1pQXLFlz09e0G4O67fwVANpsDg/yKkb3JZBIdCZKj\nUH6zHemfG81z43e+BcCFZ50u92LaYn8Gwg2SQzP++m2bAejp3UrSG8aywv3ftqHxiBG01kuBpeMZ\n42iicTH7YEkpsFREqVgmNGKuZOTdE089CsDzLz7H7JNPBeDOn98FQHtbG2XToaG5CYC/uf02ABKe\nix8Isv7y9tsBWPbci1TKgkIjZrn/wSXy3YmwE2LyRWEVkfv3Nap4L5dKALzx5hMAWE4j5UoTkbZr\nuv+6ux4jxYpsrSHUmrIfUgVVynPNb4KO4aES73/wLgCd3dsBOGnmCUSRdGhIio0891RB/7r3V7Du\ntTcAOFknANjpeYwaOd7empFry34PgDehldkLFgJgGXvZMfog9P2xa3vs9AhtjKHCaEHW1bUBgP7+\nbbQ0BkRRpab7j5nZmqAc0dKcIV8sAlAplQFQxnhNeikSYmazbNnLAFy8YDFJywPg9089C8Cz/34P\nAFs+/JDto1kAQl9ueoqvCAyzE32DADx2r7Rvacmw+tkXAOh/+3IALv3hLQBkJnRQqIiocBx5cDpS\nrN8gTrGbkAcxMigWrq0LDA0WCIPaFGRdjMRIsSIbrQmDgJZMM8m0IDU71AeAW/UawxBtkPLxmhUA\nPLP0SbqWiqh47onHAMbiG+nGNLpVUJgyY87qmI5ty60N5wT1hRExGXU5YmeuH4DH7v0tAK+/+goA\nt/zdncxbKM7VaCEPwLoNH7Bq1XsADA1sBaBYkB3kOM24blRzPKeO7BjpkEOsh0JeytPHzjoWP4hI\nGBc73ShorLrA2ld0TJoAgD80AMCMnjyJoph+hZa09Dv2WACS049ld1HaNbe0AnDqWefT1CYu+e6u\nTgDWf/SOjF+BKDsiC+qSXRUYvWG7Hn9283cAOG6+KOAXnluC0g0yV1J20O4+UdxNqSEqusxDS9bQ\n25v/XKN+B00aCMKQMNSghXmlvLEEtNjF55wxj/YOYZR+TaySlPYZSYtymnXR1wCYMv9cAEbzORJb\nxTrwkvIgHOUQFUTRpY310jFRHo6bTNJgLBQ1MArAzmWvy3zFAq//5l4ATh64BIBjph1Pf48oxGRa\nmO25otxHR4cItE0U1QbYuhiJkWJWkBLG0BpKZVEy1ZjHMRMkmUDCIrFuk/w2JGZbCTjxqisAmPon\nYiMXs8MAuLZLZsJkAJyEIM/1kmNRvlRaRECmQ9pEkUYZ77Vj9kkANE4UsbXx0cdpCEUadC0Vu3z0\nrCaSzbLGzs4d5j5EgadSNrlsljD8/KN+dTpIij024iQUYRhhOzJ1WBGUDA8Lilu3KqZ2iiwtmljE\ntMsuZc7llwFQGBRlmPAExa1uhnRaZHXZ7BbX9QiNi1o1y5JJUcSO4+K68tkzwe7W44+XBV5+GVsf\nfkTGKJoI4vYC0eki99Oe7BJlonztGUUpXxyLcx+I6siOkWJFdhRpSnlf5HRVgZvHPfuUuQBkNnUS\n5gXZjdPEgjjl6itY8dbbALzz1nIAstkhAJLJFLNOnAXAufPnAeC67lhOsyrHw0Dc95XvL2ewX3ZH\nS6tkhebOlblnn30OhY0Sq9697E0AvH5FfkhMw1JCdo5nC7LzRU05cKjVeo5ZjCicRIKE65D0hMtB\nIDfSaL53hA7ZkphWp10q5tfvlr/HPb/8FwAs4xlOmNgOiOhYtWIVADu3iwK78XvfJZWSrb9tq3h9\nD97/AAB9Pb0oS8zISkUU24tLJQZz7Z/fwMKLLgJg4F1J26m8T9ApzM1OknVNbRZvt6cnh7KjPUnJ\nA1BdjMRI8SLbAjdpo3WEHwgaQrOEze+tBCAzbJGe0AGAPX06AA//7d9TKEh48+Zb/wKAiy5eBMBA\n/yAP3CuoXfGuIPyCBZs54+yzAXj2CYkS9nZJYvnCRQs474LzAdi5vQuAJx+VZMD/PPAgs08/DYCO\nU04BoO+td0gMi0JtP0F2i23JbozCkFRC77PMYF9UR3aMFCuyLaVIJh0KhTKFUZGXgUnEdqTFhXai\nAt5kcUBS7SKXv/u9mxgdlSjcwovEqWlqkvTYxInH0NYh7datWQtAFCm6u3oA+Hi1xKJnzJRd8u3r\nriVpXPgzzj4HgOywxEoeuX8JH638EICvnCQOT89by0mFIuPzvijZwJK1lwt5cjmNH9RWhxIrs8NI\nMzpawnHUmOeojT3smfi7jkI8w+SkKQ+7YMECLGMT50alIOaZJ58BYNPGTXz4gYiPOafJ1j/ltDlj\n13KGkVOmmsBVKsXoiDy4ahh25owZgCjwnk4RLd6JJwBgOQ6OEXmVgjA5kRKx4qY9RgYL1BrMq4uR\nGCn25EEUafxSRLkoUDY7FLuqZhx7DDlVxOSGR7CMuEFLu6cfewqAnq5OmloltDrNKNS2tjZyObHV\nIxO3qJqCtp0YyzNW60G8pORBLWVTMEkDZdooy4JI2jW7ppvxfluaG9FoHKe28rM6smOkeJGNAhRh\npLFNyrqaK/WjauWSTVgRT60ac2jJZIgMykvGBPzZHVIjsnPHTp5+XFD+wv89D8BXFy6k1SQSMEWU\n1eoqtAk7snc9iDHldEAqadJzgeyIyPexTAVAxSSUW01KbrRSYmCwSFCjgjwgspVS05RSy5RSa5VS\nHyulfmKutymlXlJKbTT/MzXNeBRTLcgOgNu11iuUUk3A+0qpl4CbgVe01r8wRzzuAP56fwNprQl8\nH0spvLSgpWQyKtmiSaK6SYJhkYGjWUnWPv/0swwOSFTw5lu+D8Cc0ySeMe+C89m0UeLf2zZLXGPn\njh2ccNLJMp4j8r+7S6wMS9l4xvSrxrq7urrN+nwmTZkCgG/i5brs47eYukFPEDw0LGuu+CHFUlBz\npuaAzNZa9wA95nNOKbUWKYT/JrDINFsCvMYBmI0CpWxQmqTJPTqNwvR8Sbattl1KfZIbLJtw6ssv\nv0p/t+QSG0352cWLLwags7OLNaulntMzGfpMW4apx00F4Dhj1q0x9vbSZ55h/gVSBP/xR2JTL3vp\nZdO/gTlzJfc49Lak5CwsikqY2z0iIGg24d0oivA8B2XV5kMelMxWSs0AzgL+AEwyDwKtdY9SauJn\n9Bk7eWA5tdXEfVmp5uy6UqoReB34R63140qprNa6da/fh7TW+5XbXsrTk4+fgu8HeElhfMKYXbZJ\nInwtbMcaEbNtzk3XAbAt3civ/+GfACgWRUEm0yIKAj8gNNv4638qyeBbfvQjPJMgeGf5HwD4119K\nibFfqdBiTMWRYREVZVOddeX113DNlVdJPzOfypUZmCYKtLNNxFp+UHahm3RJuIp33tzFSLZyQHjX\nZPoppRLAY8BDWuvHzeVdSqnJ5vfJQF8tYx3NdEBkK8krLQEGtdY/3ev6PwMDeynINq31X+1vLDfl\n6WNmTqZU8EGJskmbKibf4OI81cwJgcjEoEFk8IV33cXGLVsAePsNqYyqKkzX85g9ZzYAC0wy2LJs\nAmO6pRpECa41cv3Vl15kV49EANMNkk475ytSFnHxFVew/qH/BqD7WTEj7dZWNkyTREUekdmWKeDM\nlUpkWlIsf7O3JmTXIrMvBG4CPlJKrTTXfg78AvhfpdT3gR3At2sY66imWCuiHDehmyZ2EPkhdkKA\n0NAk6FVGebYqm8VWtSJKENVx3rmc/eMfAxCaJHBkkItlYRn3pGgcHtBj2ZhqqbFn3HWlI0pmDMcE\nolIZUTU9K1ex+tf/BkDS+EDbmnw6J4sOKWXFlU+YatbmJpemFptXX+hmaLB8hFVEaYgqAcpSY3m7\nSlmYljSppd1UWKNEcZ3VLIqs9423WWNOkc28+moA/LIorbDij50Sq0YS1V7H9qqfS0YJRkE4djhM\nG+W8a916ANbdtwTbKNucicV0Nefo6xLF2JYRsTNxkgGITmBZtR+UrcdGYqR402KAshVRIBX9ALbJ\nVIeRPPeU57A2khh0mxbkTc1k2GqK4Csj8tuMq64EwGtpITKxlKpoicJwDG1jSXyDcDuVwDJRuwGj\nNDf818PSdjgHroib7smC5uZMRP+wjBJEIlt6+2SeiRNSNHouNfo0dWTHSbFXsepAY6GwXSNfjXIb\nOx4dhNhGWS7Lium+yG3nuCaR392vST3H7vVSuTpt0SImnCnnGZMmw0PCwTIRw+rp4NA4Q7lt2+h9\nSxydgRUfAOAZpJdtl3UpKZRPTZR+2Z6AMDBy36TyqgHE1iaXzdks5UpttX7xWiMJRze2t2I79hiT\nlTno6Zq6Ec+1iUomDVUVNVaC+Z7ERKaX5EH4gYiOcrFEorUFgPQkiRik2tuxjQcZVESRFs1h1Up/\nP6qaimuQtFt/WayT9V4JNVkUqTZJhFxejx39K5VkPfkRGTPZYJNMKrZvLlAqhofHg6zT4aF4kwdK\n4dgJ0HrsbQzVpGsUGIVmOVgGAwkT+E94Nm+bAP/miqD+ZEeU54SGRhxfkFfeIicCSuu2oIzpZldT\nXq78j/yQvImH9ZQkqhhMFzHQ1pLAtcSs2zIkCrIh5ZE3iB4ZEflRLst8btIiOogXPNSRHSPFqyAj\njV/ysV177FC+JWEGPCWICivR2MF7y5E2fiUgYZRUp0lN7UJQ1mEnyJhSg9aEoDftWXucJhODKfqi\nIKedeSrdpiRhYECcmSkNBvXlEoMmEW0bhV32A0aGZVf55U/D2KZ0EMmDOrJjpJgTvkiRerBHVjtU\nSwYEUZWwgjnLRMq0cS2P0Ljzp54kheuFijHptMNoXsy1vJHxU2ZOpjsrOyDX2wtAR4tE/+Z8YxH+\nKkG0iszrrMw2yJd8cnlZR1ubWD/ZbJ5kg6zDD6pv4xGMNjQphoeiI7NkWEiUijJbv1z9b04NRH5E\naIJH1UBRa1szyVYx02abWuqhQQl37urdTWNG6kWU6Tdj5nHYg7L1+0xMpM2k4ZQfUjElyTnzDq2E\nMTG3bs4xYZI8lO5u81YHJyQwCri5ScRN9VRDpVKhkK+LkSOSYnVqlFK7gTzQH9ukh04d1L7O6Vrr\nCQdqFCuzAZRS72mtz4110kOgz2OddTESI9WZHSN9Ecy+5wuY81DosK8zdpl9NFNdjMRIsTH7SH7X\n9n4qde9USnUppVaav0vGNU8cYuRIf9e2qeiavHelLnAFcA0wqrW++3DMExeyx961rbWuANV3bR8R\npLXu0VqvMJ9zQLVS97BSXMze17u2D/vNHA76VKUuwG1KqQ+VUveNt+A/LmbX9K7tL5pMpe5jwE+1\n1iPAb4BZwJlIjfovxzN+XMw+6Hdtx037qtTVWu/SWodailz+ExGHh0xxMfuIfte2qdS9F1irtf7V\nXtcn79XsSmD1eOaJJZ59aO/ajpU+q1L3eqXUmYjI2wb8YDyT1D3IGKnuQcZIdWbHSHVmx0h1ZsdI\ndWbHSHVmx0h1ZsdIdWbHSP8P/+RSpxSKe7UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a86da3a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEfhJREFUeJztnHmMXVd9xz/nLm+ZzbPZ4/E2XuKYpCQBEmIKKQSSAEmo\nKKBWSSSqSiBaWiihSICgqirUlqikRUitKpYGpWkpawiELCUkBJxAgu3ETbPZsfHMeOzx2LN53n63\n0z9+v/syBMd+8ZjrCL+vZN3nO+ee5Xd/57efa6y1tJENnDM9gbMJbWJniDaxM0Sb2BmiTewM0SZ2\nhmgTO0MsidjGmLcbY3YbY/YaYz55uib12wpzqk6NMcYF9gBXARPAduB6a+3Tp296v13wlvDspcBe\na+0vAYwxXwfeCbwosQcHB+3IyAgYgzlBxykDVKpVAKrVKsbIE44jm7Gzs1MW4Hm4ei9tY7GkPOSY\nE410ejA6Osr09PRJB1oKsVcDBxb9fwLY+sJGxpgPAB8AWLduHT9/9BEc4zSJZp5v13ymHgQA7Hh8\nJwCP73oCz5WpFgsFAF63VYZasWIFXZ1FAFzHBSC0CUmSSHs/B8gL+E3AYHjtJZe01HYpxD7em/y1\nFVlrvwR8CeCSSy6xOc8/bmf79u8H4NFHH2Hy0CQAv9R73V09rF6zBoD5eh2AH9x9NwDXXn01/edu\n/pW+vvsXf055XPjg+m/fDoDne80JNneAzvZ4zG8X/TCOecFdky6OlyKGl0LsCWDtov+vAQ6d6IGF\nUol7f3w/vuuyYWQ9AAODywH4yle/CsD4L/exauVK+duyHgByxS6KnR0AJLrghdl5AMrHFthzQAj7\nrZv/Udp/81uUpmcB2PWd7wKw9Ybrfn1CrUgYs5iDUjEVA+AYt7U+FEuxRrYDm40xG4wxOeA64PtL\n6O+3HqfM2dbayBjzIeB/ABe4xVr71ImeOTgxwac//gnCKGJgaAUAGzasB2B+Vjhx5UA/tbooxkOT\nslH6+voIogYA/YPyXL5DZPf+iXHu+spXAOi59y4AhoIGdRUb2z7/BQDizSJqjs7P47giylxPeK3o\ni6zfvGkjGPl9dGYGgGqjzqvPO0/WHEcA5FRvHDgwwej+MUqlcgsUW5oYwVp7N3D3Uvo4m7AkYr9U\n9PYP8J7r30uhkCesi8wd3fMcAKsHBwDoXz6AGhV4hTwA1XqNg1MHAShVRUFedtnvSaO+Zdhp4cJB\n5bCjQYSfF9Ow/KxYonf8i8jzxsY1DLhiveRywqHLlvUCcM/9d9Phy9+qDbFmZmshj27ZIPPv7tH2\nomfyJmHX9kcoV1rj7La7niEy5ezEJlQaNRzPY2hILI4u5cCaOjCUj2EPHAWg0AgBiMplrP69UhNz\n8PBTe6RP32Vk97PyqLYJjYMbi63eiKUPtu8CYNl5W1i7eh0AB8dHAZg+IrumMTtDOZT2nZ1dABQb\nIftu2wZArxFZHzpCttC1LJSO0Zifb2n9mRLb2AQ/bBCW6sw6NbmHLGD5pnMAqOzYyTP3/BCAfl1c\nLowwVrZ1lycyJqiLsoocB7chi61GYpIV1q6ho68fgIX/FSIn+8cBeEW+lxVrRCw4idhtoSq+zu5l\njI3Ki9u48ZXyfOUY8YPbZR5TIq7mcyIQEtenDwcvjFtaf1uMZIhMOVuc5oDYQupJRpFwbOrcOIP9\n+Opi51xRkDEBRj01T10MtyBTj5IA30pfZkE4dPlb3sLqV78WgJ03fhCAcl3Ew9j9D+CuXgXA/LE5\nAMJIRM7c7JFmGGHqqCrkcpmi8mTDl3GCDplX6Ll0RmaRh3litDk7Q2TK2UGjwcTYXvoHV7By5QgA\nrhFnZWpSXO5q6Ri5inBa3RVOxVrSiIqvgYxGXsy2qFYGDVwN9i0DYO3r30TnuecC0LXlFQAsPC3+\n1sRDD3POu98DwPCa9QBMT0ksJuoMyBe6ZZy8jDjQvYajAyL/TSKy2VNdEtiIUmWBOE5aWn+mxHYc\nh3yuk3o1pHRMlNqmTUIUvyD2bXLRa1n3mY3SPp/Tqw+6nkQtgdpRIdC+m/4eqvLC+t78ZgAK52wg\n3bTD1/4+AFNK7ODYPLVnRQlu/chfAlCZntMJeoRKOFcVcS5XYGL9agDmx54AYNeYiKRNI+vYse0B\nkvunW1t/S63aOC3IlLP9Qgdrzr8Yz3Hp7BQxUOiWLbpy5ZA0MuCeL9u0EYp4qFaqzM8K9+SXi6fZ\nePwXAETT05ic7IDeK94GgON4RLUFAIbe+BYAOr/2NWl/6ABP33EHAJe9//0AdPSIZ5gkFlfNwKbK\ncw1xJGaq58gO8LpE1AytP4dXWsv9j+5oaf1tzs4Q2XK2n2PlqrU4QDEv7zmMJdZRrUp8wQHCQD3H\nSLjM9VxWK+cbV3hu94MPoI0onHs+AF0Xi7mX1OtYlb35ATEpV195JQALt97C9N69AOzb9jAA5119\nNQC12Zlm7NrRzFDUaLAwNSpj10pyDUWee76LY8G0mD9oc3aGyNZdNwbf88BaInUkvKJkYNIUsDGG\nvMaLOzT85xjwVa5O/UziFCW1LmLHofvNlwPgd4ssbczNYtJ8pMbGV10l3Lv39m9TL4k8f/y/vw7A\n+W+XvxnHJdC0m+vLfJIoJmeEo4O6yOw46W/OuVKpNPOdJ8MZILZPvdHg6NHDAAz1STKgkFdvMU6a\nOcH0GgYhxoqNO/YDCZ9HFSFiYcVKBt4kIsIGYgI6rtsUB4kSr3u9xEOGLnsj1Xt+AMDoQ/Li9v5C\nlO3weVuwSrigJkrRzxcZOu8aAJ76qYi6XGdvcz2FfL7tQb4ckXFsRGpCHAfWrhUPMgiFG0MNbbqu\ni+vJtJJYo3hd3YRTRwA4+tBPpR/tr3vrpRTWSt45LMt2dxyHRGMpRmMdkXp/I9e8g4n7JKrYaMju\neOpbIk5yH/0Y85qKw8hz9XqNUQ3F3vXAkwBs2ChzvmSrQ6Xauhhpc3aGyFZmY3FMhMUSR8J5oatc\n4YpCM8YQqskXK2fXHJex70tJQn1SZL1XFBnff9XbSfm8Wd1hDImyvlXOjlUGD1xwIf0XXQTAocfE\nGTnysJiAuQ/fyPBq2SVBQ2R9oxE2EwlWO3V1rmEQEMcJrZaOZK4gc36ORlRvhjIdVS7pQqwDNc3p\nTemWrkURU/feK52oOChsuRCAzgtf0ySkUSJgLUZfgE1rPVTBWj/PyLXvAGBypyQFFsbHABi9+y6G\n3/NuAGL1Xo3jkOsQYl94kdjxrqO1K6UF4iiiVWq3xUiGyFhBGhzHIecYcmrqufq6feVKx/fwanKz\nX/OU808+SX3fPgA0k8XAFWLuOR0dBJoEcLy0ro/n7cZUUWo9SFCuMPy7kpnvHZHoYjgmec29d36P\n6CLZMVb9AN/zqNbl94Jm71dp8gEsketgW6yKanN2hsjc9EusZX5ujo58BYDeZRLwT000z/PoHZQ4\nyFC/RPh23PYfxGVp7w8Py3NvuFw6jCPy6oU2NKZiWFwVK5ydxjp81wFNEKx5q3iOe778rwBU9z7H\n+LYHAei66FUA5KIISqIsZ6dEOa/qlOdnjkxRHRsn0eTFyXBSzjbGrDXG/NgY84wx5iljzEf0fr8x\n5j5jzHN67WtpxLMYrXB2BHzMWvuYMaYb2GmMuQ/4E+B+a+1NesTjk8AnTtRRGAQcnjhAuVwiVtMt\n0NhFaV7kbnGgj/K9DwIQPyu1IQd2/ZyUd3ovlrrsLi19OLLvuWa9yMhGkcGNRqOZqvJzMk6tIg7P\nwQPjbNryOwCsf9tbARj9xm0yv1KVqrrujvY1t3sfxZ2SoVmmFs3en/wMgJlGjahexequOxlOSmxr\n7SQwqb9LxphnkEL4dwKXa7NbgQc5GbHDBlMTYxSLBQJkyztWinQqCxIcol4lfPIxuffD+wCYy+fw\nNON+7g1/DMBzz0og6rN/9WFKC6K43nXd9QDc8L4PUlM7uVISIn/m4x8FYO8zT3Px1jcA8NdfEPFx\n7tXvBGD8G7fh7ZHwa4fa+n35AgdH5V5Ni+5jrQioeNBTT3B+Ex6kMWY98GrgUWBIX0T6Qla8yDMf\nMMbsMMbsqGrw6GxFywrSGNMFfAe40Vq7YFo8q7L45MHw8LBtNGpARKUsnNPbK1yh2Sjy3Qlev4Qw\ng5xwkolCklWSdH1Ciyi/9kUpBZ6bmaFLC+W/cestAKxYuZYrrxHH5fM3fwqAPU9LXKOrq4vtj0i0\n74s3fxaA12+UcmJbzBPNiThbeGynTijHZFV2XawZ92lNYCSOQ91GRKfTqTHG+Aih/8tae7venjLG\nDOvfh4EjLY14FuOkR/OMsPCtwKy19sZF9z8HzCxSkP3W2o+fqK/1Gzfav/m7f2Bs/x6qGqHr15qM\nFcvFgekYGODIz0UB7b/pc3LPL3Kn7sEn1TX3NMFQLBZJ1NVP1JVPcOjrE+PosEbsXE0KYwyeOlCN\ntNRX+7q2kGdLTWR9osX6Kz74Z/hayhej6TCdy+M7thNg+dGPHmB2du60nBZ7A/Be4P+MMbv03qeA\nm4BvGmPeB4wDf9hCX2c1WrFGHuLFj+lc8VIGKxSKbN5yAWGSNBO8A8qBKwfFBe5ft4HKTyQKl2ZN\nai4UrEi8C9SBUR8IJwpo+ssagw7DgNqEVFid3yHWTloW4RrI6W7OdYlDZY1aEzEkaiomk1MArCkM\nsvzSSwEI1GIq5GS8+VoV6/pse/iRltafqQdpLcRxRFdPN0bPvOS06qnWEPFQmjvM+J13ygNpDhLD\nFVoybJWwGt8jjyEtl0oTBonjQoco17TC6VhafgwU0nxnnJ76kv/HjqWWZtXVKzxwz/dYpiHZuood\nG+lLDWLyHQVOfIT2ebRjIxkiY85OqDcaWAuJOg3WyrZNa0RIDN6gbO94tyq8ehk94vJ8XYdeF0cl\n7AuuAPELrsGi380jpM3YuoOT83+lvT/Qh2NSmZWmykSJRnFEf2dnMzZ/MrQ5O0NkfKYG6mEs0Tk1\n1wJN9FpNk0VRxGs+8WkAFp4TN9kmIUmgOyE171TemjgmSU/cKvcba4lTF7qpO4WvrHGJmykzld16\nja3F+LLTchrZ69l8DlWtuE1NzFh3YT0IqNdrWPsyrBuxSUxYK5M0Aha0UAYji4rTQ0HVKm6HiJH+\nS18HSAA/zbi7emLBppsytsRa/+V7qdo0JE2F+qt+RBiGmCbR9JCTioVGvUGkwSZHj+jFjYCorvUo\nqtSjMGr2VylXWq7PbouRDJEpZ5fnZ9n2/f8kchxGLpBQaVp0vn+3RPEOPv0IOV9s6ZHNcmIrV+xg\nbLfGNjpcvYrJGCcWmyjXm/SkgsHP6VcYlMNraVmZjQmaWlM4tqLad6G0wMr1Mma+VwoyZyf2MT/x\nDAAljVD2Da8HwPd8oijkOB+jOC7anJ0hMuXsKI6YOTZPLUjo6JdjGo4qvLF9wj3Liy7ligTjJyel\nxKCnu4s5jcYNdats7xEumzoWEGvIMJWcYWhwHb2XOjqqMAs5F1ejdlGgcWm1H2thyKGjMs6yTtk5\n5UoZL5EGo4dHpY+ZYwB09/RQLdea1VwnQ5uzM0S21giG2DjkfMuhvY8DMD0uplY+LdZxXCiKNRLl\npFp0oVLCWHGVQ0diKDN1sUrqNiGxwlm+q5E911AL08/kpFaJLLXD85oc1tmdniAWeW6qPvVY5lHX\n82mB20ElkHtFDQGUZg7rdYrYxkRhawnfbO3sxFKvhhTzHnn9xofVFFNOc5LlWkxPv2TQi5qGWpie\nJVLldGResj25vBCzEYa46ke6Gjfxc+B68gLqFVGCkR5MjQpQCbQkrS7xmE5fD5XW6lTVhu5Ypl+J\n6O1m3xE5XZYeu6ZZS+40K69aQVuMZIhT/q7fKQ1mzFGgArR2cPDMYpDW5zlirV1+skaZEhvAGLPD\nWtvat9nOIH4T82yLkQzRJnaGOBPE/tIZGPNUcNrnmbnMPpvRFiMZIjNiv5y/tX2CSt2/NcYcNMbs\n0n/XLGmcLMTIy/1b21rRNby4Uhf4A+CPgLK19ubTMU5WnN381ra1NgDSb22/LGCtnbTWPqa/S0Ba\nqXtakRWxj/et7dO+mNOBF1TqAnzIGPOEMeaWpRb8Z0Xslr61fabxwkpd4N+ATcCrkBr1f1pK/1kR\n+yV/aztrHK9S11o7Za2NreTWvoyIw1NGVsR+WX9rWyt1/x14xlr7z4vuDy9q9i7gyaWMk0k8+1S+\ntZ0xXqxS93pjzKsQkTcK/OlSBml7kBmi7UFmiDaxM0Sb2BmiTewM0SZ2hmgTO0O0iZ0h2sTOEP8P\nWzVHy7Vvu8cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a86d9fd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEqNJREFUeJztnHm0XVV9xz/7DPfc6b1335C8jJCBMIWGKAiFIsQiGBGL\nEwjVWltb7QKXtrpah66u6qrTamuXulqxuLQOi5ZCFaGOpYxqKwYQI/BISDCBhOTF5E13PtPuH7/f\nvaYK5JIXD1nmftd669x3zj577/M7v/2b9zHWWvrIBs7zPYFjCX1iZ4g+sTNEn9gZok/sDNEndobo\nEztDzIvYxpiNxpgtxphtxpj3HqlJ/brCHK5TY4xxga3ARcAuYBNwlbX2kSM3vV8vePO49yxgm7X2\ncQBjzA3AZcAzEtv1AusHBTzXxxhdVMbIRDwfgDBs4+g1o9fSNCUKWwAMDI0CUCiWurc7jrRztL0F\nHFf6sHEs4zhut73h/w1NktrueKn+Tg8612HI7nz0f8dAnKRM799NvTrd6fYZMR9iLwWePOj/XcDZ\nv9jIGPNW4K0AXq7AipM3UBkeJxcU5borRBgZWQrA7l3bKAVCSMcRgrXabfbu3ALAhkveBMCpp58F\nQM5PKBQDAPKBvDALFAek/2h6SiaiL8d1XVxSAAJf+p+rRwD4nkO9LS+n3kzknOsQxfrbl7m2Q2lf\nyLlMTdf55Adf1xPB5kPsp3uTvySTrLXXAdcBBPkBG1VnmQ0GGc7lAUhqDQCeau2UG5KQRlseqlgQ\ngiVpylBlIQBDlQUANJtyn2c8SIV4aSJEMYUi4b23ATB343UAFE47U45XvQNHiWZSaR8nVidvSZJU\n5y3XrOOQav9JKi8njuQagUuaxNCjKJ4PsXcByw/6fxnw1LPd4Hsui8ZKTFb3EykhlWlwkxoAqfEw\nkRCy7QnHplGEjUWMJKlwXhzJMYodUKLhSPv2rp20r/+U3Dv9MwDq/y0v01uzDm/DpXKtWQXAdl6W\nY7qiyOnQL4UOD3VFi9X2YYKNk6dhsafHfKyRTcAaY8xKY0wOuBK4dR79/drjsDnbWhsbY94OfAdw\ngc9bax9+9ntS4rhFwQ1IjSxlJ6kDkAYD0iip0aoLZ+c84f58uUx7ZjcAcX1W+hoRsZLEEXEsj+F7\nOenzgTtJpiYBKIyMA9CeOyDt778de95G+a2SsKswbYprhE0TFUm5nIcnUyVWEeN5wqNhbMnl3K6i\nPRTmI0aw1n4T+OZ8+jiWMC9iP1ekqaVabWNtSMGRoU1ZTDkbCoe32wnFglgOHZOrMjqOrQtnxmFT\nrimXmRTomJFzYnn4P76b1BfLxO/I25zI89bjW3Cf2AFAcekynVcobT2PUPtNYjk6gKO83xkz5wjX\nR9Uncev7MEmrp+fvu+sZIlPONgYCH8LIUlm0EoD6z3YBkKrTERSHMI5wpVGO8vwBjHJ7szkHHMxt\nCSjXpj++W67t2YlVbg8LQwDETVk5VGcJH/guALnlYrMbXUGua0iTjgOjNn4YdS2UZkN0STj9IABD\nBRebtFGT5ZDIlNg2tbSaEbnSIPmC2NmhOiKJTth3TNccDHIdr7LG8Se9AIB4er+0T2Tpp6nT9RLN\npjv0WoRTEIU7cvVfAbD/0x8AoL1zC/nNQmzzstfKUR2rJE67okJNcaJ2TJCXl9mcFVG299F7AVi2\n9DhKxUGs7Y3YfTGSITLl7CSJmZs5wPDAEqoz4sRMTYoYcf0CAH5+AF9FipevAFCdnWbmgDgnJ6w6\nGYBIPch2voK/4yEAzE83AxJfKZ9zMQDlsySCUP+uuPfNHRNEu7dJu+0Tct/q0wFIW7WuydcJFaRp\nDKag85BVNTMjx3p9Dt8Lut7sodDn7AyRKWc7jkuhXMHJD7H1XnE2cwMjAIypGRYEBRyNAHrq6Hj5\nCtN7dwCw74Bw1diwOCtxkmIe/J4MEItZaIIigxe/WsZstwEYOu8iAGZv/yppQ5Rl/Yd3AlA8cb3O\n0NLSQFTgC2kcAI2TBHkNMaiIjlsN2m7cjZ0cCtkqSCBOLe2wSaD2dW5QAkthLJG0xIIxYrc227I8\nC0GBkfElAMRtUYwzdSHYEhORe1zER6ILtXj62ZRPVdEwOy3nTlwLQGnNOtoT98uEtsrRmZMXGHmF\nbsylgyiMCSOJoQyPSyho5emXA7DlhzeTxgdE1PSAvhjJEJlyNsbg+h4YQ2lEOLrRlKUfK2e7bh7f\nlWkFah5aP8AgSjOvZtisioe1UxM4c6I8Y71v+KJX4aoHmTZEETt56WvwvIvZ++gDAORm9wGw9ev/\nCsDCjW/CWrXPNYxqU0urrZ6jDM3IAvERXvSya5i471u43o96evw+Z2eIbD1Ix8Mpj1MoDhJZ8cpK\nA2LexZHIYi8X4LvCQrlAzUE/j2a5qE7tBaA9I8fKzARxW1ZHbs1vSJ/rzyZRJWiV2xPl8NIZ5+J9\ndZHMpyqxFDPxQwAeHl3CkkWrAHDK0sZaS6oOlLGyWpr1GQAKgyMsXnM+fv7fenr+PmdniIxjIx5e\nroIXFPBciXV0zCo/18kRergaqbORcGOtPslMQyyCRiKy+0WB3je1k5baYsMvfjkATqlMMjuj/UqM\nu6Ayu10eorROHJzqXWJ+jjXEYtky8T327XgcgCUnnQvAouWru5mguN3RK8Kj1VoVz7QwR2VsxCak\nYY2w6RG1JLhvjJhNaUsCTFGzSaqZdFdDn9bGXaK1EFv3xEHVVnGMu0iSxQPn/rb01WzgBXJ9ZloI\necPNXwFg4xVXMrrhErl2zzcAyLVkvKXtNtsG5aXkfBmnNnMAq4mOjtca6IsLG00C3+tm3Q+FvhjJ\nEJlytp8rMH78qcxMbmN654/lpHp9uUA4N4picmpj+aogPcenrY7D8Z5w4WI1/dqNJsOXvkzajUmq\nLJo5QHlkDICv/Lsor09/5sMAFAYHueINEloNlp8AQPJTiZGM7t9L+4xXAXS5uV4P8TUNlnRWnKbf\n2s0G7aldJFG7p+fvc3aGyJSz2805djx4O63GDH4gQ7eaqlycTmWU14281WqiIH3HEKqTcpIqp0DN\nseZghYENohhtW9NTxiHSGPfxK8UBGSiKE1UZqmDKsmIKZ10AwMxj4u6X67PkNQxQdWRsE1tSTxMV\noSjpuicyu1HbyyBTGHpz1zNOHiSEzSppu0GzpTlEp5MZCbWV282SFAvqBaaGESXyWn3w1qw8eOXl\nGymuPklurcu51Pn5gu1k3EdHRaw8sWM7tf2iNIfOFmJP3/plALzmHIXt9wFwYO2F0kEYEbfVOko0\n/5mKpVN2m5QLpW449lDoi5EMkbEH6ZIrlkmilkY6wKp59/MSLhdHzcEkFO6PPJ/TtBppSHNmkdbu\nRevO4c/++A8BuPTSVwLw2te/jq/9x9cA+OD73gnA6NgwAF/87LUUS2Vp95arAfBWi+eZ3H8H3mPi\nTT46J6bo6PJ1FDQ9F+Tlvrwvc140OoS1EY7TeZpnR5+zM0S2nG3BpJbUWjTHSi4QZeMgHJskMd1A\niMJJU9bn1TTsxCXOPB+A8XM2sOtDHwFgar+kwKYOzHDDlz4HwMWvuAyA+38gSd7KaIWbrv8CAOe+\nXBIM4xfJceemO6loTWHuKVGam7Y/zPHHiYm4bJXExJcvPBGAUqlIo9ng6WtMfxmH5GxjzHJjzJ3G\nmAljzMPGmHfq+RFjzG3GmMf0ONzTiMcweuHsGHi3tfYBY8wAcL8x5jbgzcDt1tqP6RaP9wLvebaO\nUpsQtmoYkzKo5leSdgrepY0xtlvcHhs5uSwxLDUqq7Wv0fNfAcBdd97BgNZijy2USF29WmPpMkmz\nbXipmIV33SZVcgsWLmTJMjEH/1bl+duuebcQY9kKzD4pOT+lKCm5PcMlqlpnuHubWCNnrz9F5x4R\nxy3skYqNWGv3AHv0d9UYM4EUwl8GbNBmXwTu4hDENkihjuv4dJaer95iFKmCdA2uqs8oEUW5Pu+B\nFucMnfJCAO7ZJbGVf/zURxlVb/ETH/0gAP/85RtZsuw4AH7wXckzdupUpqb2cf6F8qIGB4SgN91y\nEwBXrn8x9f/8AgBLcjL2YN5hNpJXfOpaGdtVU6/ZrJLGya+mPtsYswJ4AXAvMK4vAmvtHmPMwme4\np7vzwNVKp2MVPRPbGFMGvgL8qbV2rtdI18E7D3J+wZKmYDwSOtlr4Yp8Trk5jEgRZbhMRc3qqEak\naTNHFeNDjzwGwO9c9nq+f8/tAKxcswaAv37fu1iljk6rJR5hqAkGYxy+fesNAPzdtRI32XiFJHJn\nN29i7r9uBGBYw6rl6WlCFU8nrhHF2NZQq5Agotdq+J5MP2OMjxD6emvtV/X0pDFmsV5fDOzracRj\nGIfkbCMs/Dlgwlr7DwdduhX4feBjerzlUH25nsPI2ADTMw0KBeVsFS2NmtbuxSkm0LiGas1S2MSO\nCXc5LzwPgDUt4ZNKZZibb7kegKsu+BMAVp90KgsWSenDTV+SPTWdcoOT176Aaa09adQl/jHckji1\nv+oUcqtOBSB5dBMAa4cWMHWa1Bla7cPzRc8Yp4jv+zhObwKil1a/Bfwe8BNjzIN67v0IkW80xrwF\neAK4vKcRj2H0Yo18j2e22i98LoM5xpDPGTzX0m4JJwcax+54vMXCEIEGoE6OJU4cTjUov0Qcj9IK\nkcWr9ojUuvO2WylrSu3+H3wfgMt/9w/YNyl7qTY/IO73H10jhtJrrnwzf371mwHY+5SYdMuPWyFz\nyOcpnSuVU1OPSKXqcaS4naSxmqTlvOiSZrPVLX3oBdnuPLDQTjyiyO1O0ssJ0YcrgwAYL6BSE2VW\n0SheUhig9GLZ4RWHspSf2LkDgInND/H+j10LwJc/80kAtkw8ynduFUW3dt0ZALziNW8AoNFus3zF\nagC2TkhB5oWXiJcZ7p9k6OwNAMzd8gUZe3qS/L49cu9x6jnmhAlcF1Kb44h5kH0cOWRbEYUhSR2G\nRkrkOia3FflRCoQ7qlHKKZrcTSY1Q77uPAZPexEAN//LPwFwz+3fBuCt7/pLzjjzHAD2bJTM+LUf\n/xCeVrO/78OfAKCuq8QYl1e+Trj8b94jUb/lK6VW5IKXXESyUJLH+fW/CUDt2zdSnBRxU2uKQm1r\n4sOmljCukXb2YR4Cfc7OENlytrXYJCLwUpp1kdVDgxIjnmmImbd8bBFje2U3bqh72IcveSMtV5TS\nqvXCxadvEJd75QlrOKDF6S+9/I0AbN35OOOLJDYSq5nm6WZGxzGMlWXMN73jLwAYXyxmolsoYrX9\n4AaR49W7vk6g85nbK4X7rbLol5xxevXUZQ69N50/PN9jwfhCouYsueKoTkCXJiJXnKlpXM0DJvpQ\nc3d/g8Z9sjlpWPfKOFpguft/Azy1DkLd5HTl6evJaSwk+tH/yDVfRJPvu6CxjTOXLAYgVjFQ3fKT\n7rW0s4F18QrSp7bLAzyxVa6dcJr02W5h3N6CUNAXI5ki402nKc16jdrcLOWyclpJP4WhsY99YUhV\nPcdhLbZM7/0WsRZidmxdT3doGX6e4E3UHk4xNBItFVNli16zSQydvjoyQOtCHOMSdz4CoIZ/6Ht4\nmuCwuqm1VpOE8YAvxfO9fiCnz9kZInMFmaYhlbGljFQksVOtikk2NiRKq1As8tCs7JcZPCBeYCGX\nZ0iTrVY5Nqd+RBGwWqkUtcTZKAY+nhZsOpoo7nw/xA+KxHFnK7amIjpeYBJ1ZXbU2boRwpR+Z+Qx\n3ZC62so4rRB4Dkqyz9kZInOZ3a61qDUtq09YB8DgoFgeszPC4c1qk8STae0blp1khVyBBaNiOdSb\n4uiMDkt2pjE1CY5waBBI+ziq4vkiZ/NWXP+WbtvIOXncnKySJ5+UGr/hAbF6HMdj/5TIZVc/BpAz\nLnVdFbOp3Bf8VJynZcuW4/seR+V2ajDgBhRLZQ7skZ0DqRVi794lXlphYIySJw83UpSHi8Imba3P\nLmpItuQIgWbiJymribj2NElb7dj+CLVE2k035PskQ4Oi8OLYoanlbXFJTMVgqaTQqvWIyrCMOTYk\nOyI2P7ydJeNiplYiTUSo9Nnx+KMUSmXCfmHl0YfD/q7fYQ1mzM+AOrA/s0EPH2P0Ps/jrbULDtUo\nU2IDGGPus9aememgh4FfxTz7YiRD9ImdIZ4PYl/3PIx5ODji88xcZh/L6IuRDJEZsY/mb20/S6Xu\nB4wxu40xD+rfJfMaJwsxcrR/a1sruhYfXKkLvAq4AqhZa//+SIyTFWd3v7VtrQ2Bzre2jwpYa/dY\nax/Q31WgU6l7RJEVsZ/uW9tH/GGOBH6hUhfg7caYzcaYz8+34D8rYvf0re3nG79YqQtcC6wG1iM1\n6h+fT/9ZEfs5f2s7azxdpa61dtJam1j5SuJnEXF42MiK2Ef1t7afqVK3UxKteDXw0HzGySSefTjf\n2s4Yz1Spe5UxZj0i8nYAb5vPIH0PMkP0PcgM0Sd2hugTO0P0iZ0h+sTOEH1iZ4g+sTNEn9gZ4v8A\nOqc0IklIjFsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a04095080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAElBJREFUeJztnHuQJXV1xz+nH7fvY+68dnZ2l90FFN24QhAi0SRWSokv\nCi0RC8RnRAnEEoyoWBqiIopGQ1YT0aQiAlqKDyoqEtSoiUpZGikQEXFXHu6uu8DusDM7j/u+t7t/\n+eOcvosIzLAzNltyT9VU9/Tv17/Hud/fOed3zvm1OOcYUD7kPdYDeDzRgNk50oDZOdKA2TnSgNk5\n0oDZOdKA2TnSspgtIieJyB0icreIvGulBvWHSnKwmxoR8YE7gecD9wA3Aa90zm1dueH9YVGwjHef\nAdztnNsOICJfAk4BHpbZnuc5z/cAByLZM736VskJzqV6azjwA4+suNtNtH6g74tIv57ny4G2svZF\n209TreSHQppq+0miz8SztnAkPS0LCsYal+IkEwBaLwy0rBAVSNOE2myTdqMjj8gtlsfs9cDuB/x/\nD/DMB1cSkXOAc0AZO7xqmDRNCEJlX6laAqBYsbH2AtKkrbfG2JFVZUZspDt31bT+aEGvxZBuW+tV\nxvRZNOwRRNp+FGr77UYMwOjaAguNFgCNWgcAP9LGQ3HUphoArFo/DkDS65AWijaXEIA1E5MAbHji\neprNBa79xPeWxLDlMPuhfsnfkUnOuU8BnwLwfd91Wh2SXo8g1K5dTxnVnDXkiYChtjKkkwziBPGU\nkeMTFQA6nS4AUcEnCvTd2owyb34/pMpbIn8BgOr6YQD23temvkeZPbRqAgAv1Pcac7NIQfupzynT\nC+USaVcbK/q2IuImAD+94SbqzXkatcZivNJ5LKnWQ9M9wMYH/L8BuO+RXnCkxHGX0lBE4Cvyei1D\n3IQythT59MpaVrblWglD6m1Fe1jWZ16oS7vT6LBQUybUDb1RIaDb6QHQzmYY6Pt7d88S20o47Aj9\nUf2K/h+NlymVtO/6/crQ6miFVlPv99w3D8DuHXt1DAWfKBSWqveWY43cBDxZRJ4gIgXgFcB1y2jv\nD54OGtnOuVhEzgO+DfjAlc65Xz7SO57vUR0rU65EmA5kaGJIr1WTmy6hHCi6OjVF5+7ZNsWKlif2\nbNQQuNCKSb1MPmtZ3O0Q9xTloa/vzdwzq202O0QVlb3NpoqTksnksXJE1+R5ZULFiUtq1Bfadq9t\nlaqGUSf0uglLNeiWI0Zwzn0T+OZy2ng80bKY/WjJ8z3K1bKabQYOSRSB7f0qN3seiKf3Zq3RacdE\nQWYi6pB3bFerJCoIY2tVaSZDanl0Ogm1OVV6GegSX98PyyEja7Re3NbStlklU3eDcyrH6wsqp4PQ\nZ/3Ro9qWr88684r0QiHEdd2BgS42/yXVGtCKUK7IFgQRj7Do4Zt554UqG9vTipY0TahW9VlosnF1\ndZiiWZqz+/Xa6hSsTdffBM2Z6desdQjMhPPMXPM9RV8QeoiKdvzA+rHNUKvdpljSZ75tZJJWwrxZ\nJn6oq7DX1vrdRooX5CSzHy0lccL8zAIjXpk4sZ1aZm/brg5JiY158byKEykFFH1lwsJsHQDjD2El\nYtevVfkVvExxQdsU3ehopG3ZVKNi1BddPWebGl8VZnW0iGcbF3Ha1uxUjentyuxhEz+9lnJ34sgK\nfgGmdjaXNP+BGMmRckV2UAiYOGIVvghh0TYUBlHndG0vTLepTZlJZqhs/CamulHNu6ICj107VEGO\nbSyx9vARAOb26rPqUEBY1pVQMR/H2KQqOc8LqdvmZLauoqsyaqbgbIdaTfv2bKWJc5SqWh6YLyUt\n6LVY8ojbMUuVIwNk50gH7WI9GIoqBXfY5kniXoxnMjEMFdnZpsUTodtUWdqeUf9H6Af0PEXaERtV\njheCtQA0mxFBRdtqdbX+4Rs2UhpSc3B8XOXsfbt3ArB7514qpoAnzF+y6449ANy7az9JquNZvUFX\nS7PVIzCzMzBlnpjM77YTup2Yvdv302n1fq9ev0dNaZxQ37dAe7ZNOKQiYnyyCsDM1BwA5aGAyrgy\nqmh+kKCUEhV1l3fSSWcA8ILTXwuA1Lp95erMrRoWor7i7RljGgvqLErTmBStnzotu39ele43rv48\nN/74OwBEo+aNHCrSbPasDQVBZUiZ3ml0mJ9uEJtbdjEaiJEcKVcxEhZ8NzpZZPWGIYYmzLZVQNMy\ndHiJIzD/sldUBXbC8Sdw6qsvAODYY/4UgPiTnwQgufpL/V1lakGAnnP9rWOGdjEPohMhNhs6Ns+g\n/+IXaNnZr+Pb118PwBcv/wQA7V6LxqyO7fBN6uOem1FTb++OWYgd9XqbJE4XFSMDZOdIuSK7UArc\n5JEjVFYX+uEq11VABJGOY2GhRoiaWm84640AnPKG86nefScAzbe9A4Dk+/+njZaCAyi2uSRwwCmS\nZhcrE0eclZkfJO6amXfiMxnZ8iEAthd0DP924Xv40Q9+CMDqjRqhycJo9f1Nmo0erVqbJE4GyD6U\nKFdkFysFt/GYCdLY9ZHWa2XxRtX0x/7xCfzdRR8A4PinHgtAe8sWOpd+DADP6jOqVoxL3YFIiSE1\n5YD37sHzSxzEYuEtq0O2yuYX6I1pu6WL3qlFrziNay7/LABXXPpxrV8yPZAI7UaXdrNLkiwus3Nl\ntud5LioG4EF5XE25alXNvDecdQ4AZ7z5LVR+9jMA6m9VpZjedCsyNmyN2Jxi8yaJT8pvM9Y5h0uy\ne+vcXKwpjjhRWz3hwI+jdQLSWMtiiyt6Jz+X0S3vB+C2fdMA/MsF7wXg9ttupxAWaNTbS2L2QIzk\nSLki2/c9VyyF+GFAEqjYOPetbwfgnReq6Ji5+EK6H/mo1jdzTcKA1HwWFCyDpKSbDlptXMdC6Yb6\nNHE4WzEZOUNq6gtxpaz35vTP8kjSdqcfrqOkdZJGk964+lVG/lXH2H7WnwFw/qtez44dd7Aw3SPu\nDZB9SFGu23XnFE3V0QJzFiwYHV8NQByrJ6571efwzezKBK47bB3+058OQPorTbhKt94FgBz9FGRS\n23AN3WxIIYKf3Kz1UpXBPO/ZWlarI7fcChwIXBCp60COOhJX0RURb9+lZb0u/oz6yxc+/mkd84tO\nAmBoeJTa/i6mAhalXJntBx7D42Vc7MDif5mTJ1uD3sgI3KuKiFN1UtHF7yG56kvaxktfDED3Ax/R\n99asJbjgfL0/UtNYeld+nvj2bdr+Jy4FwM3OW/1JZKuWdd+his874Tite/llJNd9S++f9EStc9UX\n6F6rMe1gnTq/0izwUUgplAPSWryk+Q/ESI6UK7IR8AoC3YTQlFOhYEs4C4t1O2SqxjNPnesmOItZ\n9t6viHbbtwOQ3rIVd9gaAIKXn6Z1Lr4U74yXABBu3gxA5ymahpgeu4ng69dovUu2aEcmtry5eZpv\ne4++d+lFAERvOove176h5SZunKV5hmWfNUcNc++2+SVNf4DsHClfmR0KI5MR4nk4zxIjLc6VWoJl\n2u0hJXu2VZVh50Wn4D/3eQCEV18FQO+DivDki9cSWIKkM2XofL9vGrq2moyZRefEw89WUbaDNEUs\nE6spXnmZPop0DJ2/f38fkZ49i+39yhg8cdMI0zvqS5r/osgWkY0i8n0R2SYivxSRt9jzcRH5rojc\nZdexJfX4OKalIDsG3u6cu0VEqsBPReS7wJnA/zrnPmxHPN4FvPORmxLwfMJKiD+kci+19IPUIioO\ngaaiPrjoXAD8TZuIv/MDALxYyzI0JmlKL7Jcj9jaSBLc927Qds8+U9/7J91iy+ZNJF/7L21jQREp\ntjLSuXnid1+i7e6fsX7oJ9ZLpC6GLDLk+5DEsnJ5I865PcAeu6+JyDY0Ef4U4DlW7bPAD1iE2b4v\nVEeL+EVHq6IMGhnXWF9iKcHJvmnCosX6LlGzLXn6cfgbzax7zVn6bJu6XMPhIvLlrwKQXmcm2lCR\ndErNx+7LXqPPXnCi1vnO90l+qO5Zr2TMM5s8PvdtpPv2ASAWhiOKcNNqZ7tE/TFZsCKJE3oS/25S\n+sPQo1KQInIkcDxwI7DGfojsB5l8mHfOEZGbReTmzG/8eKUlK0gRGQK+ApzvnFsQWdQVAPz2yYPh\nVZGrjkNAQHtMlc2ee+4GIHzOyQAUzn49nY/9hw4uUrEgP/wxccfwE9kmyPwb4hxu+07rzH7MqIhn\nyJSmmo/x5/5TqwBSLffvdRCa3J7+ZjeelWW5kun0LOlaFTNjrzgdgPtrauotLDSICuV+IGQxWlIt\n0cMkXwGuds591R5Picg6K18H3L+kHh/HtCiyRSF8BbDNOffRBxRdB7wO+LBdv754d6L+50LI2ITK\n6muvV1NudmY/AK/+4CUMP18DsLXz3gqA9+tdeKvUqU9m3pnf2YkgJuP7iipxiGTpDYan0SF7P8Ul\nmTgzP7h5FykXcXX1r8R2eCr669OZ/ND7ALjRQnPXXHweAPVmjaQnJPHSpPZSxMizgNcCvxCRW+3Z\nhSiTrxGRs4BdwOlL6vFxTLn6s6tjkTvuxHWMjBb7loBnh4mmptQKeOrmp3Ha2e8GYPPkkQDULngX\n8RVfAMAvmj+7bP7sNEX6UzgQHsumlW39+/uX9IA87kfFrHKv3iJ9wgYAxrb8IwCzf/5MPvNxtYp+\n8iOVoCU7sVaQiOk9C+z8+Qzt+uIZUfkyezRyxz97HaUooDxm51hsxjKiiqnVaBN3dJn/yTNeBsCL\nzzyX4n//DwALb9Zgg7dblZoUvAMZUVlHD5jSgwLpkEKa2c02965VKr7xTIYu+gcAbvz1zwH49Eff\ny9atvwLgSZvXAZB0VLn3uimUHHfcsJfmXHcQPDiUKFdkj01E7q9esgE/KCCWyps2FJWR7Sib011m\nplVJpZ5udI7afDSn/c37ADh68jCtd6H+n95yG2I7yMw3osA15Wd4SrIggjuAaDekq6lyoeai7Hva\nMVxxmeaNfOvLVwMwsbFKq6tI7lpy/thaVdYj6yPqszV+dcM+GgNkH1qUK7JXrx1yp77mGBpxl/r9\nmrhem9JrlrxeKBaYqaunLjPJXNohXtANzsmnvwmAv3zhCwHwOm0QXRViifVxu9dPSciokJmH4ojb\nuu0Oyorsu7ZriO0zl32Endt/o2WiOqVYEOpNO4W83nJKylrW6XRpzrXYc9c8nWZ8aCnIVWsq7kWv\nfirNboNiURnZtbihb/naSSFgwc6GF23CqQi9VBnUtMNE5aI68oMoYt+MihvxdS5D3uHEdUuedPrD\nxRXN4Kw12nQs9Xe4om3cf5/6UVp1oWI704XZ7Dh1iGc55AXrs2yHo/bP1pHQZ++dc3Sai1sjAzGS\nI+V7NC8UwnUBpbhIaEemsy+JZLCICkXKsaKrY+5Ul6b4TlG19knqNu/nenRjChqHpWDB4zXRZub2\n67vdWJE9zy8AqHbKBL59FsPcumOb1LZO20JUsm+JFA+Ma37mwCFTgEg3vxyWVmnNp+y/d4WCBwNa\nOcoV2XGaMFVboPuAw/WB6bEspQGv3d/i+YGlDHge7VTlbMfZYdNsY+LFBKP9vGAA3EiPE854CgDb\nfnQ7AO07FX2VSb+/g6xQ6I8L9Es8gdhq8iw052DDevvuiWVcxaYLkyRmfE2Z26IV9PoNaGUo3w8F\npEKp4VMWr+95SzIfRkfRFYY+xVDlc9jR4fmeT9EsgcgzK6SkV4+EclFlvG+ngOebVRoqovmj8GkA\nPPlwlfVJkNCzz2OUChaSi9Xn3e31cLbSWhYodiJ4qclvc8L0PLWMUj9lau+9SLJyXr8Vo/HKKK/6\ni5fRS1IKloPheTqE2JRVEncomd3ctU8TFcsVqsN6niW1AEG1rM6g0AsIQl3mlSG1g6+/6U5236Wh\nrKE16lp95UvOBKDdSWnZWZpe1z5NZIfZx8cncCaLROyHDgJaTRVBQaCKtd7WTyH1Om3arTo/uW5p\nX9kbiJEcKddNjYjsAxrAdG6dHjxNsPRxHuGcW71YpVyZDSAiNzvnTsi104Og38c4B2IkRxowO0d6\nLJj9qcegz4OhFR9n7jL78UwDMZIj5cbsQ/lb24+Qqfs+EblXRG61v5OX1U8eYuRQ/9a2ZXSte2Cm\nLvBS4OVA3Tn3zyvRT17I7n9r2znXBbJvbR8S5Jzb45y7xe5rQJapu6KUF7Mf6lvbKz6ZlaAHZeoC\nnCcit4nIlctN+M+L2Uv61vZjTQ/O1AX+HTgKOA7NUd+ynPbzYvaj/tZ23vRQmbrOuSnnXOL0k5iX\no+LwoCkvZh/S39p+uEzdLCXa6FTg9uX0k4s/+2C+tZ0zPVym7itF5DhU5O0E/nY5nQx2kDnSYAeZ\nIw2YnSMNmJ0jDZidIw2YnSMNmJ0jDZidIw2YnSP9Pzm7YXtQ0I0DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a870d5ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Load the images and plot them here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "from PIL import Image\n",
    "import os\n",
    "\n",
    "desired_width = 32\n",
    "desired_height = 32\n",
    "\n",
    "im_new_list = []\n",
    "\n",
    "for filename in os.listdir('img_internet'):\n",
    "    print(filename)\n",
    "    im = Image.open('img_internet/' + filename)\n",
    "    im_new = im.resize((desired_width, desired_height), Image.ANTIALIAS)\n",
    "    plt.figure(figsize=(1,1))\n",
    "    plt.imshow(im_new)\n",
    "    #print(im_new.shape)\n",
    "    im_new_list.append(im_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 6\n",
    "\n",
    "_Choose five candidate images of traffic signs and provide them in the report. Are there any particular qualities of the image(s) that might make classification difficult? It could be helpful to plot the images in the notebook._\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** I displayed the five test images above. The five test pictures I chose show quite different backgrounds and light conditions. In particular, the road work and bumpy road traffic signs show a poor contrast with their ground support and background, respectively. In fact they have something red right next to them. Moreover, the 30km/h speed limit traffic sign looks pretty dirty near the top."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Actual: 25,Road work\n",
      "Prediction: General caution\n",
      " (18) \n",
      "\n",
      "Actual: 1,Speed limit (30km/h)\n",
      "Prediction: Speed limit (30km/h)\n",
      " (1) \n",
      "\n",
      "Actual: 22,Bumpy road\n",
      "Prediction: Bumpy road\n",
      " (22) \n",
      "\n",
      "Actual: 28,Children crossing\n",
      "Prediction: Children crossing\n",
      " (28) \n",
      "\n",
      "Actual: 14,Stop\n",
      "Prediction: Stop\n",
      " (14) \n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEytJREFUeJztnHuQXFWdxz/n3tu339PznkkyIZnJCwKBBEKIuJYsgawF\nCgGjtURBV1Z8FCKLrgFcQHGFxAcKKcGCRRc3uKJi7bor6xawYiCIgDEmIWEmIe9kMo/M9Dy6e/p1\nz/7xO90mbCCdTPaSgv5VTd07997z6N/5nt/r/M5RWmuq5A9Zb3UH3klUZbaPVGW2j1Rlto9UZbaP\nVGW2j1Rlto80LmYrpd6nlOpUSm1TSt18ojr1diV1vE6NUsoGuoCLgb3AS8BVWuvNJ657by9yxlF2\nAbBNa70dQCn1E+By4A2ZnaiN65YJDQwlD6I8BUChKIOtPbnajkVBewBYSr5JpfPkc0UAotEgAAHb\nVGpZhIPyM6KpHABuwSMblQ9GQ/IuYHnlcmMFqVdZ8s7LS92e0mTG5L5UvafANn0NhQJSzggESyni\nsTr27TvAwOCQOhrDxsPsScCeQ/7fC5z3+o+UUtcB1wE0tzRw/w/u4D//44fYKen4QDIPQD4jjIo3\nREgWsgCEg/KjXlrXQ/fuEQDOO68dgKZaeafdMPOm1wMw/6VdALT1Ztm5MA7AszOb5FlsFICWhM3m\nPmnbjdYBkOqRutOBPJs2DwOQMDM+G9QkRmSAZ5zWAkCQEABRJ8B73vNhrlj66YoYNh5mH2kk/49M\n0lo/CDwIMGNmh86lXDLJIEGDlv4BYULAILxtQpBCxgWgb3gIgLBymD7tFACyRsuM5MIAhKwgZ/xk\nIwAL98qAjSo4t0sGLHipML3z3DEAXuwPcGBMcDuzZRCAxhph5sFUivZmM8CdUr4joahpl/tiswxK\no3cqAMm+PQyODlPwikfjFTA+Zu8FJh/yfxuw/80KKEtjh4oUaobJpAQd7XMFlYGMMCAYCFIXk+9f\n2dALQNYNMOf0ZgC6Ngp6J5zdCsCVIw4Ld2cAGDYiQ6dSeFEZvDN/tgWA12gEILdQs+hUGei9u6MA\n9A9LOTfcREuNDHB9i5R/dm+WKUYEzRmTcs1nyODuz4wyK6/wKlR747FGXgJmKKXalVIu8NfAL8dR\n39uejhvZWuuCUup64L8RffIDrfUrb1amUNQMJLPs2JcirwsANKZkioYSEQCyqVGGkzJdd+6VaR6r\njdLf1Q/AvIUNACzuTwBw4dOd5F1BnjaSTV1zNfrnj0ubEXl28XMii1+YFeN3CdETpPsAWNclMrzG\nskgfSAGwbZu0NxZ3GbZrAHBbBfUH8zIrx9IZcqlhdNGriGfjESNorZ8AnhhPHe8kGhezj5VsGxK1\nNvHWEIE+QShBQVn0FKNviy6J1wTln1sgiIrMj2NlRZGOtNYCMP3RHVLnWA+FrCg/e9kVAMTvWcmw\nMRvz//QwAAlENyz5cQ83TRXlGjSyuG22KOQdv+kjOSTyv6FevknnCpw+SRRoW6v0yxmQcjWtQeqb\nI9iByqRx1V33kfxFtqWoidmce1oNa7cKgqJhY/LFRA7WF/LoMwRV0V1iUqn6PAeDIqNbnu8BYMoL\nBwDI20X0pDb5/itflnaKReJflfuh378IQGprFwAX9ET4UFb0xeZlUwAIZqQvhajD7NkTAejflQTA\nw+EPa8QqGtgp+uX0xfJNz54eZuxKkssVKvr9vjI7V4A9fZquFx0ijcLIOuk3rpmK69ZnUO0yAN01\nIgqSa7O0hsSGXvlfojQDjpiK6ZERgt/9tjxrEgdG5/M4MbGvI/fdA0Dq/VcCMILHTRlRiJ8flDoL\nM8RZudSp51drdgIwYJiNgqw2PsEOUbJ1TSLesqk8+Vm5svd7NKqKER/JV2QrzyOQGaNr027a3i0e\nYVdKpmDnZpmiPTmHWS/Ks2JQEJOZ6LD0ZTG7OnrkXTIj/7tLLqfmQx+UBorGk3McCllBbeDsuQBY\ny28CoHDbHYRqRTl/5lciku6+RhTkxBkNXByUfg0l5Xpgf5LXegXRuf40APs3SduhiKLoHcFtfgOq\nIttH8ldBuhbxU0LMWtzAtmcHANgbFtNPh0WOTuktkPLEtNqWEgfjfUNxPrhdzLsR5HunSdx3ddst\nvLJZAo2zZ82SZ0rhBN3D2q7//PUADK59gdGnngTgTE/MyE+uPwjAPV6ROY1i3qX2yExLByyKEWGT\n3SqyetgWLO/Y28NQskCxWBm2fWX2WM6jc+cYvYNjpByZ8sN/7JaXafm/MxGhMSHBkUBBlOBV61Io\noxCzQzKFm+/7JgA3/Wg19915BwC333kXAH/3uc/yxJNPS/1JUXRtE0UTX/jtFai/Ekd3JCnK9rxn\nhFkTXx3j+fNEsYaGZMCznkXYNgoyLQNeExNraUpHA82tIZyqnX3yka/IpqCx+rPs6EzTNmUCAHZQ\nvLP9myRgeM70KUw6bSoAM5+TiN277RRDI4Jo56JFUm7pUgAWh3/F5kWXAPDPq38CwE9/8W+8umUr\nAI3NogzPmjkDgIc7prJ6xdcASF99LQCheunDLYM5Pr1bZlDLbIl1t9XFqQ3L/Y+ffAmAhWdIiHXa\nhEYcF9RRlw2Eqsj2kfw1/SxFIBIiEnMZM15bS60oxpmXTwfAtkIkukV5XtUr0b+MV0RHRHHFV94J\ngGdWUi699FJK+unDyz4m7QSDNLXKzNEFkbNf/JKYfnfd/Q22njUHgNk3fAaAwe+uAmBybT13edLO\n941cbqptZHJQnJ73TBWlPNAts/Dgzl5OmbwAvKrMPunIV2RrJbGMof4s8aiYUY2ni7ysS0hXfr++\nj6VrJZbc7Aq6BgcHcO+8HYDQqacBlJ0WAg6/fvIpeZYRp8MO2IykxQoZS4v5eOPyWwCYPa2DJx77\nGQBz75D4CcZyyezYyel/eg2AiybLTNtck6LNlaXWuXNltvT3i5COuDXU10Rw7Mow6yuzLa2IegHm\ntLZz/tmisJ5a8xwAT3eLyPhIJsAllgxAckSe6flnE/7UJ+W+YII+5gdalsX5C2Wd+XkTdPrKrTdz\n3Q03AlATk6WvXbt3A3DuvLl84qMfkbocEWGxVRJbSV+2tMy4xb8VUdHVGOLX3TKwI+mS6JMludTB\nXiZnz6SgKwtEVcWIj+Qrsh3bpqEmwRnt7WzbI85M537jKJiA/N9msuRNCkPR2FQtq74L4fBhdXm5\nXPl+MClm4fx58+Q6/xwaasQ5OWicmnBIPMotnZ3EaxOmEmkzcu4CAArLv0judjEL40r6cMlTO/jy\nqeJpNsWlD0ODIubW/nE7l7z3SnRlq2JVZPtJviI7m8+zvaebbGGUmRMFLZFFCwG48BmJb7QfGGCg\nX5BTf9utAKzasIGNq+4H4Kt3/AMAEyaI3NTA9x9+CIArLv0AALd/9U62bhNFd/VHrwIgFhWk3//g\nQ2zYtAmAdy0wiM5KHCR+w/UMrlkLQOq3vwVgfk8tl1myJPdYm8y0Sa6JlYx5JIfGTs7YSNgNcFrb\nJFJ7NrFxv4Q3zz8oHX3vHmHwUD5D7UIZgBfPF8W3ctk1jGbkB3d2dQKw7CPCxLpEDe+/+CIAPvdZ\nyUy6cfnNTJs+DYBv3PV1gLJd/8jq1dz/0A8BmDVDlHR9nXiIXrFI/DsrpR8XiVc6msrw8T4RAD2T\nZHFiZ62IpIVndhCLhLCsylzIqhjxkfwVI9k8u7bvo+fAfhxPzKX5a7YB4JpMyXwOcjf/PQBfe+B7\nACiKTJw0CYA9xoRbfstyAB5/7DFW3r3isHb27tnN9BliJzc1Nh327nv3focvfEls7os2/MnUITGV\n9qmn4E6ZCkB4xT8CkLn208QCYiIu+dM+AB64oAOArQf6UFofMQ/vSFRFto/ks+lnUZuI0dmbZHlW\n0DJby3gP90t2UsvyL3LvfvHYNv7PGgDqW5pI52TRIBSV2EVtUZTa5s2bufC9FwCgLKnLdSxcE//W\nZqmsFEu5etkyFs4/F4CbbxWEX7ZkCQD3f28Vf/Gud0k7H5QF4uxv1jD6o9UAzDcLFku2SOzmS/kU\nueQoXoUZUUdFtlJqslLqN0qpLUqpV5RSnzfP65VSTyqltpprXUUtvoOpEmQXgC9ordcppeLAH5RS\nTwIfB57WWq8wWzxuBpa/WUVjhSLbDg7jrd/F/KKYYumsWAmxc84G4PlFF/DNaz8BgBctpfKOos0M\nSJvc7VxekP7Iv/6UT3zyOgCSJsX4tX37uGzuOQAoWxBekqtFYIaJbf/Lo48CcKGxZkKxWHl2lBiT\nWPl1htf9EYDBrWJOLjbryhsmhImEguWk/aPRUZmtte4Gus39iFJqC5IIfzlwgfnsEeAZjsJsx7Zo\njEW5oRAgYlL7DxZMIs41Eq9wUqOsuEZCpa7J/Sh6Hk7JvDKxC22ZCrTHsFmDfG2rKNu7P/VZ5syR\nMGr+gCTz6BJDLIuiESm9+yX+8f07xWuc29FBvl/WI0ufq2AQ947bAMj8jcRn8MR7vaQ/QF+sEceu\nTBofk4JUSk0F5gG/B1rMQJQGpPkNylynlHpZKfXy6MjosTT3tqOKFaRSKgY8DtyotR5WFU6dw3Ye\nTOvQk2MxJqsAXkZiIsG4oDe34lsAnOXYLDBxEM9kGgkq5b40zTGxCwWM3PsAAAvMs4AbYMx4hcM5\nuSpXRBK2XRYp9Y78/OaAXPszGSxlZkwpbKoUyqzUOyGpI2tEWMdIigbbJlSh8VcRspVSAYTRj2qt\nf2Ee9yilJpj3E4Deilp8B9NRka0Ewg8DW7TW9xzy6pfAx4AV5vrvR63LAhW0eH7+VGJdYuqFRiS4\nT1HQki7m8UzA30JMKqUPQc7rthIqFI4rSZm5hpryc8sguTwDTQoEw6OY7Txosxcmb+q0Latcvyq1\no0GbsJ4y8W/LzKBtM9op1sTIn8DFg3cDVwMblVLrzbNbESb/VCl1LbAb+FBFLb6DqRJr5DmOvDMM\nYNGxNGZbDnXRWtbNnsXPZ0s6QHBE8uiUMnsS7QDKaHvbdE8X89i2INVSgrKcLShr6R/kL9fIhiI6\nxI222idjmT2OhR4JeLm/k1WcjfNmse08yf9zU2nTpshpC4+SRHeMPLeVhWVmVsBYQNrI8GRDnNmJ\nBJZd2jX55uRv3ohSqIBL2K3BDQvzPBPk94rSYTsQoiQ1iqWofDGPZxSXWxIBZmdYrH+AsMlsyj7z\njDTzdBHPMM0yU9w2YdBgewuZibJa7mUkPuM5pQ2weUIlZeiKp6osCBilnDfSwjamXtyDsONQqbFQ\njY34SP7mjSiFGwwSi0VwAoIg7RmTrCgos2yb0sZCzwqZXhaxTL6dbUw5x5byEw4MoksOj4mb4P05\nEldyZgomttK6t49GT2aJF/IOqyvvOQTMin4p2R5dJGgyAdygLB4rsxdHYREMR7CsyjBbRbaP5K/M\nNhRxXdywoLC0oqSMGeahyhG6EnmeonR6RI8ystRsiW7d10vRKLOSE6QOKV+6LxjHJTaQJHJAlPLA\n6SbFOCWLwvGIjW0UtTY6IuAG8Ezs3VLaXDF990DZvLH9cDj5LkacQIBgOIprEiq1KjHGKKFCjrKG\nNF5jMZ+nOykBq/0pedcYkut2T3G28ehCRqFqpcpVlM1lMxDW6DCt+7YD0D9Xgl9pxCpJuBFsjNIs\ndUtrwiFR4o5RpLaxtwuFQtnmroSqYsRH8hfZyFSMRKOUAVGOcUhXQqFQeatbwWQ/5XIFcEQ5xWKC\nrpLIePwDS3j21VcBmNktEb764SFCpqxtTL+8yRWxzjwV730XAzC1VpLuB3XetJOmvk6UYcEo7IDt\nlIVEKS6TM3EXNxzCdlQ1ZfhkJJ9Thi3C4Rh2wKGkVEomIGZxIDuWQlmC2rKMdF1SI+JVxiKCRkuL\nApuSqCVnFne7tIl1pDMoU18iLjOisUm2Uzc31hHIify3jEcYNZFHxryyw1KCa9ANYjl/jp0f8gq8\nAsdy7FMV2T6Sz6afplDMYVkWlnEknNLGpDET3w6HsAy6SqAZSvXQPyTvQ56Uq40KYrN5RS4lSC0e\nkk2qMRukRqRcPicrML37BmhuEblcGxGLqJQrWONqmk0sPWP6g2WRHZP6I1GZVfGQmJ+e5+Ecwz4P\n/z3IgEs4FCwfxFVSgrGITGVtKbJ5c7CWETHNzY10pEWJbd8jeSMHAmaBAU3BJFkqYw/nPV0+NKvE\niNKaQMANcXBAyrY3idIsBZ0S9W3lDaRBs1DgFTWuiZcUS+nKRgR6hTyeV6x412lVjPhIx32u33E1\nplQfkAL6fWv0+KmRyvs5RWvddLSPfGU2gFLqZa31fF8bPQ76/+hnVYz4SFVm+0hvBbMffAvaPB46\n4f30XWa/k6kqRnwk35h9Mp+1/SaZul9RSu1TSq03f5eMqx0/xMjJfta2yeiacGimLrAE+DAwqrX+\n1oloxy9kl8/a1lrngNJZ2ycFaa27tdbrzP0IUMrUPaHkF7OPdNb2Cf8xJ4Jel6kLcL1SaoNS6gfj\nTfj3i9kVnbX9VtPrM3WBB4BpwFwkR/3b46nfL2Yf81nbftORMnW11j1a66KWzMqHEHF43OQXs0/q\ns7bfKFO3lBJt6Apg03ja8SWefTxnbftMb5Spe5VSai4i8nYCnxpPI1UP0keqepA+UpXZPlKV2T5S\nldk+UpXZPlKV2T5Sldk+UpXZPtL/Ai5dyPAwCE2UAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a86d9f358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEkNJREFUeJztnHmQHNV9xz+vu6d7Zvac3ZWEhIQkxCEE4rbEYSnCRokL\nCsxhcxiIcVzgsouyXaaSEP+RglzlqmBX4qrEKWIoRICQFDdE3IjDBplDSCB039pLqz1md3bOPl7+\n+L1ZCSykkVY0KjS/qq2d6XlX//r7fvdrpbWmTvGQ9UUv4GiiOrNjpDqzY6Q6s2OkOrNjpDqzY6Q6\ns2OkcTFbKfUNpdR6pdQmpdQdh2tRX1ZSh+rUKKVsYAOwGOgE3gWu11qvOXzL+3KRM46+84BNWust\nAEqpR4BvAp/JbNdL6HSDh9aa9rZJALRl2gCoPnSl1DiW9Mf06XGHR0ZQlmzo5sbGT7ThgHObsZB2\n5UqJSrlEb28fw8MjB1z4eJh9LLBzr++dwPxPN1JK3QrcCpBKuyz4+lwCHXLDdT8B4MZv3QCA7/sA\n2LZdE8OrDNp7X6p9/W7+27YNwHMvv0LCSwGweOGFAJQCmduy9i9Vq2M5Zqzt2zeybes6fviDnx1w\nvTA+Zu+LI38kk7TW9wD3ADS3NuhyyadQrvDm716V36MIgEsWXwpAOpUi6SWrfWWifTD/QIypUkRk\nFitjOM6eawdPZj3mLkvlIqtXLqNYyNXUezzM7gSm7fV9KtC9vw6+H9KzaxQ3leS9le8D0NO3C4Dz\n530VgHYjVmAPk6NoD3Msc23Fmo0A7B4awU3IbVQfgLIUxZJBq3lO2jCqc6jCyTMnfnJhNYoupaxP\nrAFsRgoDhFFQU//xWCPvAicqpWYqpVzgOuDpcYz3padDRrbWOlBK3Qa8ANjAfVrrj/ffS4Fy8Mua\nYqkCQKm4A4DfPvAfAFx31bUcP+NEAPyKoLO1pQXLFlz09e0G4O67fwVANpsDg/yKkb3JZBIdCZKj\nUH6zHemfG81z43e+BcCFZ50u92LaYn8Gwg2SQzP++m2bAejp3UrSG8aywv3ftqHxiBG01kuBpeMZ\n42iicTH7YEkpsFREqVgmNGKuZOTdE089CsDzLz7H7JNPBeDOn98FQHtbG2XToaG5CYC/uf02ABKe\nix8Isv7y9tsBWPbci1TKgkIjZrn/wSXy3YmwE2LyRWEVkfv3Nap4L5dKALzx5hMAWE4j5UoTkbZr\nuv+6ux4jxYpsrSHUmrIfUgVVynPNb4KO4aES73/wLgCd3dsBOGnmCUSRdGhIio0891RB/7r3V7Du\ntTcAOFknANjpeYwaOd7empFry34PgDehldkLFgJgGXvZMfog9P2xa3vs9AhtjKHCaEHW1bUBgP7+\nbbQ0BkRRpab7j5nZmqAc0dKcIV8sAlAplQFQxnhNeikSYmazbNnLAFy8YDFJywPg9089C8Cz/34P\nAFs+/JDto1kAQl9ueoqvCAyzE32DADx2r7Rvacmw+tkXAOh/+3IALv3hLQBkJnRQqIiocBx5cDpS\nrN8gTrGbkAcxMigWrq0LDA0WCIPaFGRdjMRIsSIbrQmDgJZMM8m0IDU71AeAW/UawxBtkPLxmhUA\nPLP0SbqWiqh47onHAMbiG+nGNLpVUJgyY87qmI5ty60N5wT1hRExGXU5YmeuH4DH7v0tAK+/+goA\nt/zdncxbKM7VaCEPwLoNH7Bq1XsADA1sBaBYkB3kOM24blRzPKeO7BjpkEOsh0JeytPHzjoWP4hI\nGBc73ShorLrA2ld0TJoAgD80AMCMnjyJoph+hZa09Dv2WACS049ld1HaNbe0AnDqWefT1CYu+e6u\nTgDWf/SOjF+BKDsiC+qSXRUYvWG7Hn9283cAOG6+KOAXnluC0g0yV1J20O4+UdxNqSEqusxDS9bQ\n25v/XKN+B00aCMKQMNSghXmlvLEEtNjF55wxj/YOYZR+TaySlPYZSYtymnXR1wCYMv9cAEbzORJb\nxTrwkvIgHOUQFUTRpY310jFRHo6bTNJgLBQ1MArAzmWvy3zFAq//5l4ATh64BIBjph1Pf48oxGRa\nmO25otxHR4cItE0U1QbYuhiJkWJWkBLG0BpKZVEy1ZjHMRMkmUDCIrFuk/w2JGZbCTjxqisAmPon\nYiMXs8MAuLZLZsJkAJyEIM/1kmNRvlRaRECmQ9pEkUYZ77Vj9kkANE4UsbXx0cdpCEUadC0Vu3z0\nrCaSzbLGzs4d5j5EgadSNrlsljD8/KN+dTpIij024iQUYRhhOzJ1WBGUDA8Lilu3KqZ2iiwtmljE\ntMsuZc7llwFQGBRlmPAExa1uhnRaZHXZ7BbX9QiNi1o1y5JJUcSO4+K68tkzwe7W44+XBV5+GVsf\nfkTGKJoI4vYC0eki99Oe7BJlonztGUUpXxyLcx+I6siOkWJFdhRpSnlf5HRVgZvHPfuUuQBkNnUS\n5gXZjdPEgjjl6itY8dbbALzz1nIAstkhAJLJFLNOnAXAufPnAeC67lhOsyrHw0Dc95XvL2ewX3ZH\nS6tkhebOlblnn30OhY0Sq9697E0AvH5FfkhMw1JCdo5nC7LzRU05cKjVeo5ZjCicRIKE65D0hMtB\nIDfSaL53hA7ZkphWp10q5tfvlr/HPb/8FwAs4xlOmNgOiOhYtWIVADu3iwK78XvfJZWSrb9tq3h9\nD97/AAB9Pb0oS8zISkUU24tLJQZz7Z/fwMKLLgJg4F1J26m8T9ApzM1OknVNbRZvt6cnh7KjPUnJ\nA1BdjMRI8SLbAjdpo3WEHwgaQrOEze+tBCAzbJGe0AGAPX06AA//7d9TKEh48+Zb/wKAiy5eBMBA\n/yAP3CuoXfGuIPyCBZs54+yzAXj2CYkS9nZJYvnCRQs474LzAdi5vQuAJx+VZMD/PPAgs08/DYCO\nU04BoO+td0gMi0JtP0F2i23JbozCkFRC77PMYF9UR3aMFCuyLaVIJh0KhTKFUZGXgUnEdqTFhXai\nAt5kcUBS7SKXv/u9mxgdlSjcwovEqWlqkvTYxInH0NYh7datWQtAFCm6u3oA+Hi1xKJnzJRd8u3r\nriVpXPgzzj4HgOywxEoeuX8JH638EICvnCQOT89by0mFIuPzvijZwJK1lwt5cjmNH9RWhxIrs8NI\nMzpawnHUmOeojT3smfi7jkI8w+SkKQ+7YMECLGMT50alIOaZJ58BYNPGTXz4gYiPOafJ1j/ltDlj\n13KGkVOmmsBVKsXoiDy4ahh25owZgCjwnk4RLd6JJwBgOQ6OEXmVgjA5kRKx4qY9RgYL1BrMq4uR\nGCn25EEUafxSRLkoUDY7FLuqZhx7DDlVxOSGR7CMuEFLu6cfewqAnq5OmloltDrNKNS2tjZyObHV\nIxO3qJqCtp0YyzNW60G8pORBLWVTMEkDZdooy4JI2jW7ppvxfluaG9FoHKe28rM6smOkeJGNAhRh\npLFNyrqaK/WjauWSTVgRT60ac2jJZIgMykvGBPzZHVIjsnPHTp5+XFD+wv89D8BXFy6k1SQSMEWU\n1eoqtAk7snc9iDHldEAqadJzgeyIyPexTAVAxSSUW01KbrRSYmCwSFCjgjwgspVS05RSy5RSa5VS\nHyulfmKutymlXlJKbTT/MzXNeBRTLcgOgNu11iuUUk3A+0qpl4CbgVe01r8wRzzuAP56fwNprQl8\nH0spvLSgpWQyKtmiSaK6SYJhkYGjWUnWPv/0swwOSFTw5lu+D8Cc0ySeMe+C89m0UeLf2zZLXGPn\njh2ccNLJMp4j8r+7S6wMS9l4xvSrxrq7urrN+nwmTZkCgG/i5brs47eYukFPEDw0LGuu+CHFUlBz\npuaAzNZa9wA95nNOKbUWKYT/JrDINFsCvMYBmI0CpWxQmqTJPTqNwvR8Sbattl1KfZIbLJtw6ssv\nv0p/t+QSG0352cWLLwags7OLNaulntMzGfpMW4apx00F4Dhj1q0x9vbSZ55h/gVSBP/xR2JTL3vp\nZdO/gTlzJfc49Lak5CwsikqY2z0iIGg24d0oivA8B2XV5kMelMxWSs0AzgL+AEwyDwKtdY9SauJn\n9Bk7eWA5tdXEfVmp5uy6UqoReB34R63140qprNa6da/fh7TW+5XbXsrTk4+fgu8HeElhfMKYXbZJ\nInwtbMcaEbNtzk3XAbAt3civ/+GfACgWRUEm0yIKAj8gNNv4638qyeBbfvQjPJMgeGf5HwD4119K\nibFfqdBiTMWRYREVZVOddeX113DNlVdJPzOfypUZmCYKtLNNxFp+UHahm3RJuIp33tzFSLZyQHjX\nZPoppRLAY8BDWuvHzeVdSqnJ5vfJQF8tYx3NdEBkK8krLQEGtdY/3ev6PwMDeynINq31X+1vLDfl\n6WNmTqZU8EGJskmbKibf4OI81cwJgcjEoEFk8IV33cXGLVsAePsNqYyqKkzX85g9ZzYAC0wy2LJs\nAmO6pRpECa41cv3Vl15kV49EANMNkk475ytSFnHxFVew/qH/BqD7WTEj7dZWNkyTREUekdmWKeDM\nlUpkWlIsf7O3JmTXIrMvBG4CPlJKrTTXfg78AvhfpdT3gR3At2sY66imWCuiHDehmyZ2EPkhdkKA\n0NAk6FVGebYqm8VWtSJKENVx3rmc/eMfAxCaJHBkkItlYRn3pGgcHtBj2ZhqqbFn3HWlI0pmDMcE\nolIZUTU9K1ex+tf/BkDS+EDbmnw6J4sOKWXFlU+YatbmJpemFptXX+hmaLB8hFVEaYgqAcpSY3m7\nSlmYljSppd1UWKNEcZ3VLIqs9423WWNOkc28+moA/LIorbDij50Sq0YS1V7H9qqfS0YJRkE4djhM\nG+W8a916ANbdtwTbKNucicV0Nefo6xLF2JYRsTNxkgGITmBZtR+UrcdGYqR402KAshVRIBX9ALbJ\nVIeRPPeU57A2khh0mxbkTc1k2GqK4Csj8tuMq64EwGtpITKxlKpoicJwDG1jSXyDcDuVwDJRuwGj\nNDf818PSdjgHroib7smC5uZMRP+wjBJEIlt6+2SeiRNSNHouNfo0dWTHSbFXsepAY6GwXSNfjXIb\nOx4dhNhGWS7Lium+yG3nuCaR392vST3H7vVSuTpt0SImnCnnGZMmw0PCwTIRw+rp4NA4Q7lt2+h9\nSxydgRUfAOAZpJdtl3UpKZRPTZR+2Z6AMDBy36TyqgHE1iaXzdks5UpttX7xWiMJRze2t2I79hiT\nlTno6Zq6Ec+1iUomDVUVNVaC+Z7ERKaX5EH4gYiOcrFEorUFgPQkiRik2tuxjQcZVESRFs1h1Up/\nP6qaimuQtFt/WayT9V4JNVkUqTZJhFxejx39K5VkPfkRGTPZYJNMKrZvLlAqhofHg6zT4aF4kwdK\n4dgJ0HrsbQzVpGsUGIVmOVgGAwkT+E94Nm+bAP/miqD+ZEeU54SGRhxfkFfeIicCSuu2oIzpZldT\nXq78j/yQvImH9ZQkqhhMFzHQ1pLAtcSs2zIkCrIh5ZE3iB4ZEflRLst8btIiOogXPNSRHSPFqyAj\njV/ysV177FC+JWEGPCWICivR2MF7y5E2fiUgYZRUp0lN7UJQ1mEnyJhSg9aEoDftWXucJhODKfqi\nIKedeSrdpiRhYECcmSkNBvXlEoMmEW0bhV32A0aGZVf55U/D2KZ0EMmDOrJjpJgTvkiRerBHVjtU\nSwYEUZWwgjnLRMq0cS2P0Ljzp54kheuFijHptMNoXsy1vJHxU2ZOpjsrOyDX2wtAR4tE/+Z8YxH+\nKkG0iszrrMw2yJd8cnlZR1ubWD/ZbJ5kg6zDD6pv4xGMNjQphoeiI7NkWEiUijJbv1z9b04NRH5E\naIJH1UBRa1szyVYx02abWuqhQQl37urdTWNG6kWU6Tdj5nHYg7L1+0xMpM2k4ZQfUjElyTnzDq2E\nMTG3bs4xYZI8lO5u81YHJyQwCri5ScRN9VRDpVKhkK+LkSOSYnVqlFK7gTzQH9ukh04d1L7O6Vrr\nCQdqFCuzAZRS72mtz4110kOgz2OddTESI9WZHSN9Ecy+5wuY81DosK8zdpl9NFNdjMRIsTH7SH7X\n9n4qde9USnUppVaav0vGNU8cYuRIf9e2qeiavHelLnAFcA0wqrW++3DMExeyx961rbWuANV3bR8R\npLXu0VqvMJ9zQLVS97BSXMze17u2D/vNHA76VKUuwG1KqQ+VUveNt+A/LmbX9K7tL5pMpe5jwE+1\n1iPAb4BZwJlIjfovxzN+XMw+6Hdtx037qtTVWu/SWodailz+ExGHh0xxMfuIfte2qdS9F1irtf7V\nXtcn79XsSmD1eOaJJZ59aO/ajpU+q1L3eqXUmYjI2wb8YDyT1D3IGKnuQcZIdWbHSHVmx0h1ZsdI\ndWbHSHVmx0h1ZsdIdWbHSP8P/+RSpxSKe7UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0b2d57f390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEfhJREFUeJztnHmMXVd9xz/nLm+ZzbPZ4/E2XuKYpCQBEmIKKQSSAEmo\nKKBWSSSqSiBaWiihSICgqirUlqikRUitKpYGpWkpawiELCUkBJxAgu3ETbPZsfHMeOzx2LN53n63\n0z9+v/syBMd+8ZjrCL+vZN3nO+ee5Xd/57efa6y1tJENnDM9gbMJbWJniDaxM0Sb2BmiTewM0SZ2\nhmgTO0MsidjGmLcbY3YbY/YaYz55uib12wpzqk6NMcYF9gBXARPAduB6a+3Tp296v13wlvDspcBe\na+0vAYwxXwfeCbwosQcHB+3IyAgYgzlBxykDVKpVAKrVKsbIE44jm7Gzs1MW4Hm4ei9tY7GkPOSY\nE410ejA6Osr09PRJB1oKsVcDBxb9fwLY+sJGxpgPAB8AWLduHT9/9BEc4zSJZp5v13ymHgQA7Hh8\nJwCP73oCz5WpFgsFAF63VYZasWIFXZ1FAFzHBSC0CUmSSHs/B8gL+E3AYHjtJZe01HYpxD7em/y1\nFVlrvwR8CeCSSy6xOc8/bmf79u8H4NFHH2Hy0CQAv9R73V09rF6zBoD5eh2AH9x9NwDXXn01/edu\n/pW+vvsXf055XPjg+m/fDoDne80JNneAzvZ4zG8X/TCOecFdky6OlyKGl0LsCWDtov+vAQ6d6IGF\nUol7f3w/vuuyYWQ9AAODywH4yle/CsD4L/exauVK+duyHgByxS6KnR0AJLrghdl5AMrHFthzQAj7\nrZv/Udp/81uUpmcB2PWd7wKw9Ybrfn1CrUgYs5iDUjEVA+AYt7U+FEuxRrYDm40xG4wxOeA64PtL\n6O+3HqfM2dbayBjzIeB/ABe4xVr71ImeOTgxwac//gnCKGJgaAUAGzasB2B+Vjhx5UA/tbooxkOT\nslH6+voIogYA/YPyXL5DZPf+iXHu+spXAOi59y4AhoIGdRUb2z7/BQDizSJqjs7P47giylxPeK3o\ni6zfvGkjGPl9dGYGgGqjzqvPO0/WHEcA5FRvHDgwwej+MUqlcgsUW5oYwVp7N3D3Uvo4m7AkYr9U\n9PYP8J7r30uhkCesi8wd3fMcAKsHBwDoXz6AGhV4hTwA1XqNg1MHAShVRUFedtnvSaO+Zdhp4cJB\n5bCjQYSfF9Ow/KxYonf8i8jzxsY1DLhiveRywqHLlvUCcM/9d9Phy9+qDbFmZmshj27ZIPPv7tH2\nomfyJmHX9kcoV1rj7La7niEy5ezEJlQaNRzPY2hILI4u5cCaOjCUj2EPHAWg0AgBiMplrP69UhNz\n8PBTe6RP32Vk97PyqLYJjYMbi63eiKUPtu8CYNl5W1i7eh0AB8dHAZg+IrumMTtDOZT2nZ1dABQb\nIftu2wZArxFZHzpCttC1LJSO0Zifb2n9mRLb2AQ/bBCW6sw6NbmHLGD5pnMAqOzYyTP3/BCAfl1c\nLowwVrZ1lycyJqiLsoocB7chi61GYpIV1q6ho68fgIX/FSIn+8cBeEW+lxVrRCw4idhtoSq+zu5l\njI3Ki9u48ZXyfOUY8YPbZR5TIq7mcyIQEtenDwcvjFtaf1uMZIhMOVuc5oDYQupJRpFwbOrcOIP9\n+Opi51xRkDEBRj01T10MtyBTj5IA30pfZkE4dPlb3sLqV78WgJ03fhCAcl3Ew9j9D+CuXgXA/LE5\nAMJIRM7c7JFmGGHqqCrkcpmi8mTDl3GCDplX6Ll0RmaRh3litDk7Q2TK2UGjwcTYXvoHV7By5QgA\nrhFnZWpSXO5q6Ri5inBa3RVOxVrSiIqvgYxGXsy2qFYGDVwN9i0DYO3r30TnuecC0LXlFQAsPC3+\n1sRDD3POu98DwPCa9QBMT0ksJuoMyBe6ZZy8jDjQvYajAyL/TSKy2VNdEtiIUmWBOE5aWn+mxHYc\nh3yuk3o1pHRMlNqmTUIUvyD2bXLRa1n3mY3SPp/Tqw+6nkQtgdpRIdC+m/4eqvLC+t78ZgAK52wg\n3bTD1/4+AFNK7ODYPLVnRQlu/chfAlCZntMJeoRKOFcVcS5XYGL9agDmx54AYNeYiKRNI+vYse0B\nkvunW1t/S63aOC3IlLP9Qgdrzr8Yz3Hp7BQxUOiWLbpy5ZA0MuCeL9u0EYp4qFaqzM8K9+SXi6fZ\nePwXAETT05ic7IDeK94GgON4RLUFAIbe+BYAOr/2NWl/6ABP33EHAJe9//0AdPSIZ5gkFlfNwKbK\ncw1xJGaq58gO8LpE1AytP4dXWsv9j+5oaf1tzs4Q2XK2n2PlqrU4QDEv7zmMJdZRrUp8wQHCQD3H\nSLjM9VxWK+cbV3hu94MPoI0onHs+AF0Xi7mX1OtYlb35ATEpV195JQALt97C9N69AOzb9jAA5119\nNQC12Zlm7NrRzFDUaLAwNSpj10pyDUWee76LY8G0mD9oc3aGyNZdNwbf88BaInUkvKJkYNIUsDGG\nvMaLOzT85xjwVa5O/UziFCW1LmLHofvNlwPgd4ssbczNYtJ8pMbGV10l3Lv39m9TL4k8f/y/vw7A\n+W+XvxnHJdC0m+vLfJIoJmeEo4O6yOw46W/OuVKpNPOdJ8MZILZPvdHg6NHDAAz1STKgkFdvMU6a\nOcH0GgYhxoqNO/YDCZ9HFSFiYcVKBt4kIsIGYgI6rtsUB4kSr3u9xEOGLnsj1Xt+AMDoQ/Li9v5C\nlO3weVuwSrigJkrRzxcZOu8aAJ76qYi6XGdvcz2FfL7tQb4ckXFsRGpCHAfWrhUPMgiFG0MNbbqu\ni+vJtJJYo3hd3YRTRwA4+tBPpR/tr3vrpRTWSt45LMt2dxyHRGMpRmMdkXp/I9e8g4n7JKrYaMju\neOpbIk5yH/0Y85qKw8hz9XqNUQ3F3vXAkwBs2ChzvmSrQ6Xauhhpc3aGyFZmY3FMhMUSR8J5oatc\n4YpCM8YQqskXK2fXHJex70tJQn1SZL1XFBnff9XbSfm8Wd1hDImyvlXOjlUGD1xwIf0XXQTAocfE\nGTnysJiAuQ/fyPBq2SVBQ2R9oxE2EwlWO3V1rmEQEMcJrZaOZK4gc36ORlRvhjIdVS7pQqwDNc3p\nTemWrkURU/feK52oOChsuRCAzgtf0ySkUSJgLUZfgE1rPVTBWj/PyLXvAGBypyQFFsbHABi9+y6G\n3/NuAGL1Xo3jkOsQYl94kdjxrqO1K6UF4iiiVWq3xUiGyFhBGhzHIecYcmrqufq6feVKx/fwanKz\nX/OU808+SX3fPgA0k8XAFWLuOR0dBJoEcLy0ro/n7cZUUWo9SFCuMPy7kpnvHZHoYjgmec29d36P\n6CLZMVb9AN/zqNbl94Jm71dp8gEsketgW6yKanN2hsjc9EusZX5ujo58BYDeZRLwT000z/PoHZQ4\nyFC/RPh23PYfxGVp7w8Py3NvuFw6jCPy6oU2NKZiWFwVK5ydxjp81wFNEKx5q3iOe778rwBU9z7H\n+LYHAei66FUA5KIISqIsZ6dEOa/qlOdnjkxRHRsn0eTFyXBSzjbGrDXG/NgY84wx5iljzEf0fr8x\n5j5jzHN67WtpxLMYrXB2BHzMWvuYMaYb2GmMuQ/4E+B+a+1NesTjk8AnTtRRGAQcnjhAuVwiVtMt\n0NhFaV7kbnGgj/K9DwIQPyu1IQd2/ZyUd3ovlrrsLi19OLLvuWa9yMhGkcGNRqOZqvJzMk6tIg7P\nwQPjbNryOwCsf9tbARj9xm0yv1KVqrrujvY1t3sfxZ2SoVmmFs3en/wMgJlGjahexequOxlOSmxr\n7SQwqb9LxphnkEL4dwKXa7NbgQc5GbHDBlMTYxSLBQJkyztWinQqCxIcol4lfPIxuffD+wCYy+fw\nNON+7g1/DMBzz0og6rN/9WFKC6K43nXd9QDc8L4PUlM7uVISIn/m4x8FYO8zT3Px1jcA8NdfEPFx\n7tXvBGD8G7fh7ZHwa4fa+n35AgdH5V5Ni+5jrQioeNBTT3B+Ex6kMWY98GrgUWBIX0T6Qla8yDMf\nMMbsMMbsqGrw6GxFywrSGNMFfAe40Vq7YFo8q7L45MHw8LBtNGpARKUsnNPbK1yh2Sjy3Qlev4Qw\ng5xwkolCklWSdH1Ciyi/9kUpBZ6bmaFLC+W/cestAKxYuZYrrxHH5fM3fwqAPU9LXKOrq4vtj0i0\n74s3fxaA12+UcmJbzBPNiThbeGynTijHZFV2XawZ92lNYCSOQ91GRKfTqTHG+Aih/8tae7venjLG\nDOvfh4EjLY14FuOkR/OMsPCtwKy19sZF9z8HzCxSkP3W2o+fqK/1Gzfav/m7f2Bs/x6qGqHr15qM\nFcvFgekYGODIz0UB7b/pc3LPL3Kn7sEn1TX3NMFQLBZJ1NVP1JVPcOjrE+PosEbsXE0KYwyeOlCN\ntNRX+7q2kGdLTWR9osX6Kz74Z/hayhej6TCdy+M7thNg+dGPHmB2du60nBZ7A/Be4P+MMbv03qeA\nm4BvGmPeB4wDf9hCX2c1WrFGHuLFj+lc8VIGKxSKbN5yAWGSNBO8A8qBKwfFBe5ft4HKTyQKl2ZN\nai4UrEi8C9SBUR8IJwpo+ssagw7DgNqEVFid3yHWTloW4RrI6W7OdYlDZY1aEzEkaiomk1MArCkM\nsvzSSwEI1GIq5GS8+VoV6/pse/iRltafqQdpLcRxRFdPN0bPvOS06qnWEPFQmjvM+J13ygNpDhLD\nFVoybJWwGt8jjyEtl0oTBonjQoco17TC6VhafgwU0nxnnJ76kv/HjqWWZtXVKzxwz/dYpiHZuood\nG+lLDWLyHQVOfIT2ebRjIxkiY85OqDcaWAuJOg3WyrZNa0RIDN6gbO94tyq8ehk94vJ8XYdeF0cl\n7AuuAPELrsGi380jpM3YuoOT83+lvT/Qh2NSmZWmykSJRnFEf2dnMzZ/MrQ5O0NkfKYG6mEs0Tk1\n1wJN9FpNk0VRxGs+8WkAFp4TN9kmIUmgOyE171TemjgmSU/cKvcba4lTF7qpO4WvrHGJmykzld16\nja3F+LLTchrZ69l8DlWtuE1NzFh3YT0IqNdrWPsyrBuxSUxYK5M0Aha0UAYji4rTQ0HVKm6HiJH+\nS18HSAA/zbi7emLBppsytsRa/+V7qdo0JE2F+qt+RBiGmCbR9JCTioVGvUGkwSZHj+jFjYCorvUo\nqtSjMGr2VylXWq7PbouRDJEpZ5fnZ9n2/f8kchxGLpBQaVp0vn+3RPEOPv0IOV9s6ZHNcmIrV+xg\nbLfGNjpcvYrJGCcWmyjXm/SkgsHP6VcYlMNraVmZjQmaWlM4tqLad6G0wMr1Mma+VwoyZyf2MT/x\nDAAljVD2Da8HwPd8oijkOB+jOC7anJ0hMuXsKI6YOTZPLUjo6JdjGo4qvLF9wj3Liy7ligTjJyel\nxKCnu4s5jcYNdats7xEumzoWEGvIMJWcYWhwHb2XOjqqMAs5F1ejdlGgcWm1H2thyKGjMs6yTtk5\n5UoZL5EGo4dHpY+ZYwB09/RQLdea1VwnQ5uzM0S21giG2DjkfMuhvY8DMD0uplY+LdZxXCiKNRLl\npFp0oVLCWHGVQ0diKDN1sUrqNiGxwlm+q5E911AL08/kpFaJLLXD85oc1tmdniAWeW6qPvVY5lHX\n82mB20ElkHtFDQGUZg7rdYrYxkRhawnfbO3sxFKvhhTzHnn9xofVFFNOc5LlWkxPv2TQi5qGWpie\nJVLldGResj25vBCzEYa46ke6Gjfxc+B68gLqFVGCkR5MjQpQCbQkrS7xmE5fD5XW6lTVhu5Ypl+J\n6O1m3xE5XZYeu6ZZS+40K69aQVuMZIhT/q7fKQ1mzFGgArR2cPDMYpDW5zlirV1+skaZEhvAGLPD\nWtvat9nOIH4T82yLkQzRJnaGOBPE/tIZGPNUcNrnmbnMPpvRFiMZIjNiv5y/tX2CSt2/NcYcNMbs\n0n/XLGmcLMTIy/1b21rRNby4Uhf4A+CPgLK19ubTMU5WnN381ra1NgDSb22/LGCtnbTWPqa/S0Ba\nqXtakRWxj/et7dO+mNOBF1TqAnzIGPOEMeaWpRb8Z0Xslr61fabxwkpd4N+ATcCrkBr1f1pK/1kR\n+yV/aztrHK9S11o7Za2NreTWvoyIw1NGVsR+WX9rWyt1/x14xlr7z4vuDy9q9i7gyaWMk0k8+1S+\ntZ0xXqxS93pjzKsQkTcK/OlSBml7kBmi7UFmiDaxM0Sb2BmiTewM0SZ2hmgTO0O0iZ0h2sTOEP8P\nWzVHy7Vvu8cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0ba749d438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEqNJREFUeJztnHm0XVV9xz/7DPfc6b1335C8jJCBMIWGKAiFIsQiGBGL\nEwjVWltb7QKXtrpah66u6qrTamuXulqxuLQOi5ZCFaGOpYxqKwYQI/BISDCBhOTF5E13PtPuH7/f\nvaYK5JIXD1nmftd669x3zj577/M7v/2b9zHWWvrIBs7zPYFjCX1iZ4g+sTNEn9gZok/sDNEndobo\nEztDzIvYxpiNxpgtxphtxpj3HqlJ/brCHK5TY4xxga3ARcAuYBNwlbX2kSM3vV8vePO49yxgm7X2\ncQBjzA3AZcAzEtv1AusHBTzXxxhdVMbIRDwfgDBs4+g1o9fSNCUKWwAMDI0CUCiWurc7jrRztL0F\nHFf6sHEs4zhut73h/w1NktrueKn+Tg8612HI7nz0f8dAnKRM799NvTrd6fYZMR9iLwWePOj/XcDZ\nv9jIGPNW4K0AXq7AipM3UBkeJxcU5borRBgZWQrA7l3bKAVCSMcRgrXabfbu3ALAhkveBMCpp58F\nQM5PKBQDAPKBvDALFAek/2h6SiaiL8d1XVxSAAJf+p+rRwD4nkO9LS+n3kzknOsQxfrbl7m2Q2lf\nyLlMTdf55Adf1xPB5kPsp3uTvySTrLXXAdcBBPkBG1VnmQ0GGc7lAUhqDQCeau2UG5KQRlseqlgQ\ngiVpylBlIQBDlQUANJtyn2c8SIV4aSJEMYUi4b23ATB343UAFE47U45XvQNHiWZSaR8nVidvSZJU\n5y3XrOOQav9JKi8njuQagUuaxNCjKJ4PsXcByw/6fxnw1LPd4Hsui8ZKTFb3EykhlWlwkxoAqfEw\nkRCy7QnHplGEjUWMJKlwXhzJMYodUKLhSPv2rp20r/+U3Dv9MwDq/y0v01uzDm/DpXKtWQXAdl6W\nY7qiyOnQL4UOD3VFi9X2YYKNk6dhsafHfKyRTcAaY8xKY0wOuBK4dR79/drjsDnbWhsbY94OfAdw\ngc9bax9+9ntS4rhFwQ1IjSxlJ6kDkAYD0iip0aoLZ+c84f58uUx7ZjcAcX1W+hoRsZLEEXEsj+F7\nOenzgTtJpiYBKIyMA9CeOyDt778de95G+a2SsKswbYprhE0TFUm5nIcnUyVWEeN5wqNhbMnl3K6i\nPRTmI0aw1n4T+OZ8+jiWMC9iP1ekqaVabWNtSMGRoU1ZTDkbCoe32wnFglgOHZOrMjqOrQtnxmFT\nrimXmRTomJFzYnn4P76b1BfLxO/I25zI89bjW3Cf2AFAcekynVcobT2PUPtNYjk6gKO83xkz5wjX\nR9Uncev7MEmrp+fvu+sZIlPONgYCH8LIUlm0EoD6z3YBkKrTERSHMI5wpVGO8vwBjHJ7szkHHMxt\nCSjXpj++W67t2YlVbg8LQwDETVk5VGcJH/guALnlYrMbXUGua0iTjgOjNn4YdS2UZkN0STj9IABD\nBRebtFGT5ZDIlNg2tbSaEbnSIPmC2NmhOiKJTth3TNccDHIdr7LG8Se9AIB4er+0T2Tpp6nT9RLN\npjv0WoRTEIU7cvVfAbD/0x8AoL1zC/nNQmzzstfKUR2rJE67okJNcaJ2TJCXl9mcFVG299F7AVi2\n9DhKxUGs7Y3YfTGSITLl7CSJmZs5wPDAEqoz4sRMTYoYcf0CAH5+AF9FipevAFCdnWbmgDgnJ6w6\nGYBIPch2voK/4yEAzE83AxJfKZ9zMQDlsySCUP+uuPfNHRNEu7dJu+0Tct/q0wFIW7WuydcJFaRp\nDKag85BVNTMjx3p9Dt8Lut7sodDn7AyRKWc7jkuhXMHJD7H1XnE2cwMjAIypGRYEBRyNAHrq6Hj5\nCtN7dwCw74Bw1diwOCtxkmIe/J4MEItZaIIigxe/WsZstwEYOu8iAGZv/yppQ5Rl/Yd3AlA8cb3O\n0NLSQFTgC2kcAI2TBHkNMaiIjlsN2m7cjZ0cCtkqSCBOLe2wSaD2dW5QAkthLJG0xIIxYrc227I8\nC0GBkfElAMRtUYwzdSHYEhORe1zER6ILtXj62ZRPVdEwOy3nTlwLQGnNOtoT98uEtsrRmZMXGHmF\nbsylgyiMCSOJoQyPSyho5emXA7DlhzeTxgdE1PSAvhjJEJlyNsbg+h4YQ2lEOLrRlKUfK2e7bh7f\nlWkFah5aP8AgSjOvZtisioe1UxM4c6I8Y71v+KJX4aoHmTZEETt56WvwvIvZ++gDAORm9wGw9ev/\nCsDCjW/CWrXPNYxqU0urrZ6jDM3IAvERXvSya5i471u43o96evw+Z2eIbD1Ix8Mpj1MoDhJZ8cpK\nA2LexZHIYi8X4LvCQrlAzUE/j2a5qE7tBaA9I8fKzARxW1ZHbs1vSJ/rzyZRJWiV2xPl8NIZ5+J9\ndZHMpyqxFDPxQwAeHl3CkkWrAHDK0sZaS6oOlLGyWpr1GQAKgyMsXnM+fv7fenr+PmdniIxjIx5e\nroIXFPBciXV0zCo/18kRergaqbORcGOtPslMQyyCRiKy+0WB3je1k5baYsMvfjkATqlMMjuj/UqM\nu6Ayu10eorROHJzqXWJ+jjXEYtky8T327XgcgCUnnQvAouWru5mguN3RK8Kj1VoVz7QwR2VsxCak\nYY2w6RG1JLhvjJhNaUsCTFGzSaqZdFdDn9bGXaK1EFv3xEHVVnGMu0iSxQPn/rb01WzgBXJ9ZloI\necPNXwFg4xVXMrrhErl2zzcAyLVkvKXtNtsG5aXkfBmnNnMAq4mOjtca6IsLG00C3+tm3Q+FvhjJ\nEJlytp8rMH78qcxMbmN654/lpHp9uUA4N4picmpj+aogPcenrY7D8Z5w4WI1/dqNJsOXvkzajUmq\nLJo5QHlkDICv/Lsor09/5sMAFAYHueINEloNlp8AQPJTiZGM7t9L+4xXAXS5uV4P8TUNlnRWnKbf\n2s0G7aldJFG7p+fvc3aGyJSz2805djx4O63GDH4gQ7eaqlycTmWU14281WqiIH3HEKqTcpIqp0DN\nseZghYENohhtW9NTxiHSGPfxK8UBGSiKE1UZqmDKsmIKZ10AwMxj4u6X67PkNQxQdWRsE1tSTxMV\noSjpuicyu1HbyyBTGHpz1zNOHiSEzSppu0GzpTlEp5MZCbWV282SFAvqBaaGESXyWn3w1qw8eOXl\nGymuPklurcu51Pn5gu1k3EdHRaw8sWM7tf2iNIfOFmJP3/plALzmHIXt9wFwYO2F0kEYEbfVOko0\n/5mKpVN2m5QLpW449lDoi5EMkbEH6ZIrlkmilkY6wKp59/MSLhdHzcEkFO6PPJ/TtBppSHNmkdbu\nRevO4c/++A8BuPTSVwLw2te/jq/9x9cA+OD73gnA6NgwAF/87LUUS2Vp95arAfBWi+eZ3H8H3mPi\nTT46J6bo6PJ1FDQ9F+Tlvrwvc140OoS1EY7TeZpnR5+zM0S2nG3BpJbUWjTHSi4QZeMgHJskMd1A\niMJJU9bn1TTsxCXOPB+A8XM2sOtDHwFgar+kwKYOzHDDlz4HwMWvuAyA+38gSd7KaIWbrv8CAOe+\nXBIM4xfJceemO6loTWHuKVGam7Y/zPHHiYm4bJXExJcvPBGAUqlIo9ng6WtMfxmH5GxjzHJjzJ3G\nmAljzMPGmHfq+RFjzG3GmMf0ONzTiMcweuHsGHi3tfYBY8wAcL8x5jbgzcDt1tqP6RaP9wLvebaO\nUpsQtmoYkzKo5leSdgrepY0xtlvcHhs5uSwxLDUqq7Wv0fNfAcBdd97BgNZijy2USF29WmPpMkmz\nbXipmIV33SZVcgsWLmTJMjEH/1bl+duuebcQY9kKzD4pOT+lKCm5PcMlqlpnuHubWCNnrz9F5x4R\nxy3skYqNWGv3AHv0d9UYM4EUwl8GbNBmXwTu4hDENkihjuv4dJaer95iFKmCdA2uqs8oEUW5Pu+B\nFucMnfJCAO7ZJbGVf/zURxlVb/ETH/0gAP/85RtZsuw4AH7wXckzdupUpqb2cf6F8qIGB4SgN91y\nEwBXrn8x9f/8AgBLcjL2YN5hNpJXfOpaGdtVU6/ZrJLGya+mPtsYswJ4AXAvMK4vAmvtHmPMwme4\np7vzwNVKp2MVPRPbGFMGvgL8qbV2rtdI18E7D3J+wZKmYDwSOtlr4Yp8Trk5jEgRZbhMRc3qqEak\naTNHFeNDjzwGwO9c9nq+f8/tAKxcswaAv37fu1iljk6rJR5hqAkGYxy+fesNAPzdtRI32XiFJHJn\nN29i7r9uBGBYw6rl6WlCFU8nrhHF2NZQq5Agotdq+J5MP2OMjxD6emvtV/X0pDFmsV5fDOzracRj\nGIfkbCMs/Dlgwlr7DwdduhX4feBjerzlUH25nsPI2ADTMw0KBeVsFS2NmtbuxSkm0LiGas1S2MSO\nCXc5LzwPgDUt4ZNKZZibb7kegKsu+BMAVp90KgsWSenDTV+SPTWdcoOT176Aaa09adQl/jHckji1\nv+oUcqtOBSB5dBMAa4cWMHWa1Bla7cPzRc8Yp4jv+zhObwKil1a/Bfwe8BNjzIN67v0IkW80xrwF\neAK4vKcRj2H0Yo18j2e22i98LoM5xpDPGTzX0m4JJwcax+54vMXCEIEGoE6OJU4cTjUov0Qcj9IK\nkcWr9ojUuvO2WylrSu3+H3wfgMt/9w/YNyl7qTY/IO73H10jhtJrrnwzf371mwHY+5SYdMuPWyFz\nyOcpnSuVU1OPSKXqcaS4naSxmqTlvOiSZrPVLX3oBdnuPLDQTjyiyO1O0ssJ0YcrgwAYL6BSE2VW\n0SheUhig9GLZ4RWHspSf2LkDgInND/H+j10LwJc/80kAtkw8ynduFUW3dt0ZALziNW8AoNFus3zF\nagC2TkhB5oWXiJcZ7p9k6OwNAMzd8gUZe3qS/L49cu9x6jnmhAlcF1Kb44h5kH0cOWRbEYUhSR2G\nRkrkOia3FflRCoQ7qlHKKZrcTSY1Q77uPAZPexEAN//LPwFwz+3fBuCt7/pLzjjzHAD2bJTM+LUf\n/xCeVrO/78OfAKCuq8QYl1e+Trj8b94jUb/lK6VW5IKXXESyUJLH+fW/CUDt2zdSnBRxU2uKQm1r\n4sOmljCukXb2YR4Cfc7OENlytrXYJCLwUpp1kdVDgxIjnmmImbd8bBFje2U3bqh72IcveSMtV5TS\nqvXCxadvEJd75QlrOKDF6S+9/I0AbN35OOOLJDYSq5nm6WZGxzGMlWXMN73jLwAYXyxmolsoYrX9\n4AaR49W7vk6g85nbK4X7rbLol5xxevXUZQ69N50/PN9jwfhCouYsueKoTkCXJiJXnKlpXM0DJvpQ\nc3d/g8Z9sjlpWPfKOFpguft/Azy1DkLd5HTl6evJaSwk+tH/yDVfRJPvu6CxjTOXLAYgVjFQ3fKT\n7rW0s4F18QrSp7bLAzyxVa6dcJr02W5h3N6CUNAXI5ki402nKc16jdrcLOWyclpJP4WhsY99YUhV\nPcdhLbZM7/0WsRZidmxdT3doGX6e4E3UHk4xNBItFVNli16zSQydvjoyQOtCHOMSdz4CoIZ/6Ht4\nmuCwuqm1VpOE8YAvxfO9fiCnz9kZInMFmaYhlbGljFQksVOtikk2NiRKq1As8tCs7JcZPCBeYCGX\nZ0iTrVY5Nqd+RBGwWqkUtcTZKAY+nhZsOpoo7nw/xA+KxHFnK7amIjpeYBJ1ZXbU2boRwpR+Z+Qx\n3ZC62so4rRB4Dkqyz9kZInOZ3a61qDUtq09YB8DgoFgeszPC4c1qk8STae0blp1khVyBBaNiOdSb\n4uiMDkt2pjE1CY5waBBI+ziq4vkiZ/NWXP+WbtvIOXncnKySJ5+UGr/hAbF6HMdj/5TIZVc/BpAz\nLnVdFbOp3Bf8VJynZcuW4/seR+V2ajDgBhRLZQ7skZ0DqRVi794lXlphYIySJw83UpSHi8Imba3P\nLmpItuQIgWbiJymribj2NElb7dj+CLVE2k035PskQ4Oi8OLYoanlbXFJTMVgqaTQqvWIyrCMOTYk\nOyI2P7ydJeNiplYiTUSo9Nnx+KMUSmXCfmHl0YfD/q7fYQ1mzM+AOrA/s0EPH2P0Ps/jrbULDtUo\nU2IDGGPus9aememgh4FfxTz7YiRD9ImdIZ4PYl/3PIx5ODji88xcZh/L6IuRDJEZsY/mb20/S6Xu\nB4wxu40xD+rfJfMaJwsxcrR/a1sruhYfXKkLvAq4AqhZa//+SIyTFWd3v7VtrQ2Bzre2jwpYa/dY\nax/Q31WgU6l7RJEVsZ/uW9tH/GGOBH6hUhfg7caYzcaYz8+34D8rYvf0re3nG79YqQtcC6wG1iM1\n6h+fT/9ZEfs5f2s7azxdpa61dtJam1j5SuJnEXF42MiK2Ef1t7afqVK3UxKteDXw0HzGySSefTjf\n2s4Yz1Spe5UxZj0i8nYAb5vPIH0PMkP0PcgM0Sd2hugTO0P0iZ0h+sTOEH1iZ4g+sTNEn9gZ4v8A\nOqc0IklIjFsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0ba749a208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAElBJREFUeJztnHuQJXV1xz+nH7fvY+68dnZ2l90FFN24QhAi0SRWSokv\nCi0RC8RnRAnEEoyoWBqiIopGQ1YT0aQiAlqKDyoqEtSoiUpZGikQEXFXHu6uu8DusDM7j/u+t7t/\n+eOcvosIzLAzNltyT9VU9/Tv17/Hud/fOed3zvm1OOcYUD7kPdYDeDzRgNk50oDZOdKA2TnSgNk5\n0oDZOdKA2TnSspgtIieJyB0icreIvGulBvWHSnKwmxoR8YE7gecD9wA3Aa90zm1dueH9YVGwjHef\nAdztnNsOICJfAk4BHpbZnuc5z/cAByLZM736VskJzqV6azjwA4+suNtNtH6g74tIv57ny4G2svZF\n209TreSHQppq+0miz8SztnAkPS0LCsYal+IkEwBaLwy0rBAVSNOE2myTdqMjj8gtlsfs9cDuB/x/\nD/DMB1cSkXOAc0AZO7xqmDRNCEJlX6laAqBYsbH2AtKkrbfG2JFVZUZspDt31bT+aEGvxZBuW+tV\nxvRZNOwRRNp+FGr77UYMwOjaAguNFgCNWgcAP9LGQ3HUphoArFo/DkDS65AWijaXEIA1E5MAbHji\neprNBa79xPeWxLDlMPuhfsnfkUnOuU8BnwLwfd91Wh2SXo8g1K5dTxnVnDXkiYChtjKkkwziBPGU\nkeMTFQA6nS4AUcEnCvTd2owyb34/pMpbIn8BgOr6YQD23temvkeZPbRqAgAv1Pcac7NIQfupzynT\nC+USaVcbK/q2IuImAD+94SbqzXkatcZivNJ5LKnWQ9M9wMYH/L8BuO+RXnCkxHGX0lBE4Cvyei1D\n3IQythT59MpaVrblWglD6m1Fe1jWZ16oS7vT6LBQUybUDb1RIaDb6QHQzmYY6Pt7d88S20o47Aj9\nUf2K/h+NlymVtO/6/crQ6miFVlPv99w3D8DuHXt1DAWfKBSWqveWY43cBDxZRJ4gIgXgFcB1y2jv\nD54OGtnOuVhEzgO+DfjAlc65Xz7SO57vUR0rU65EmA5kaGJIr1WTmy6hHCi6OjVF5+7ZNsWKlif2\nbNQQuNCKSb1MPmtZ3O0Q9xTloa/vzdwzq202O0QVlb3NpoqTksnksXJE1+R5ZULFiUtq1Bfadq9t\nlaqGUSf0uglLNeiWI0Zwzn0T+OZy2ng80bKY/WjJ8z3K1bKabQYOSRSB7f0qN3seiKf3Zq3RacdE\nQWYi6pB3bFerJCoIY2tVaSZDanl0Ogm1OVV6GegSX98PyyEja7Re3NbStlklU3eDcyrH6wsqp4PQ\nZ/3Ro9qWr88684r0QiHEdd2BgS42/yXVGtCKUK7IFgQRj7Do4Zt554UqG9vTipY0TahW9VlosnF1\ndZiiWZqz+/Xa6hSsTdffBM2Z6desdQjMhPPMXPM9RV8QeoiKdvzA+rHNUKvdpljSZ75tZJJWwrxZ\nJn6oq7DX1vrdRooX5CSzHy0lccL8zAIjXpk4sZ1aZm/brg5JiY158byKEykFFH1lwsJsHQDjD2El\nYtevVfkVvExxQdsU3ehopG3ZVKNi1BddPWebGl8VZnW0iGcbF3Ha1uxUjentyuxhEz+9lnJ34sgK\nfgGmdjaXNP+BGMmRckV2UAiYOGIVvghh0TYUBlHndG0vTLepTZlJZqhs/CamulHNu6ICj107VEGO\nbSyx9vARAOb26rPqUEBY1pVQMR/H2KQqOc8LqdvmZLauoqsyaqbgbIdaTfv2bKWJc5SqWh6YLyUt\n6LVY8ojbMUuVIwNk50gH7WI9GIoqBXfY5kniXoxnMjEMFdnZpsUTodtUWdqeUf9H6Af0PEXaERtV\njheCtQA0mxFBRdtqdbX+4Rs2UhpSc3B8XOXsfbt3ArB7514qpoAnzF+y6449ANy7az9JquNZvUFX\nS7PVIzCzMzBlnpjM77YTup2Yvdv302n1fq9ev0dNaZxQ37dAe7ZNOKQiYnyyCsDM1BwA5aGAyrgy\nqmh+kKCUEhV1l3fSSWcA8ILTXwuA1Lp95erMrRoWor7i7RljGgvqLErTmBStnzotu39ele43rv48\nN/74OwBEo+aNHCrSbPasDQVBZUiZ3ml0mJ9uEJtbdjEaiJEcKVcxEhZ8NzpZZPWGIYYmzLZVQNMy\ndHiJIzD/sldUBXbC8Sdw6qsvAODYY/4UgPiTnwQgufpL/V1lakGAnnP9rWOGdjEPohMhNhs6Ns+g\n/+IXaNnZr+Pb118PwBcv/wQA7V6LxqyO7fBN6uOem1FTb++OWYgd9XqbJE4XFSMDZOdIuSK7UArc\n5JEjVFYX+uEq11VABJGOY2GhRoiaWm84640AnPKG86nefScAzbe9A4Dk+/+njZaCAyi2uSRwwCmS\nZhcrE0eclZkfJO6amXfiMxnZ8iEAthd0DP924Xv40Q9+CMDqjRqhycJo9f1Nmo0erVqbJE4GyD6U\nKFdkFysFt/GYCdLY9ZHWa2XxRtX0x/7xCfzdRR8A4PinHgtAe8sWOpd+DADP6jOqVoxL3YFIiSE1\n5YD37sHzSxzEYuEtq0O2yuYX6I1pu6WL3qlFrziNay7/LABXXPpxrV8yPZAI7UaXdrNLkiwus3Nl\ntud5LioG4EF5XE25alXNvDecdQ4AZ7z5LVR+9jMA6m9VpZjedCsyNmyN2Jxi8yaJT8pvM9Y5h0uy\ne+vcXKwpjjhRWz3hwI+jdQLSWMtiiyt6Jz+X0S3vB+C2fdMA/MsF7wXg9ttupxAWaNTbS2L2QIzk\nSLki2/c9VyyF+GFAEqjYOPetbwfgnReq6Ji5+EK6H/mo1jdzTcKA1HwWFCyDpKSbDlptXMdC6Yb6\nNHE4WzEZOUNq6gtxpaz35vTP8kjSdqcfrqOkdZJGk964+lVG/lXH2H7WnwFw/qtez44dd7Aw3SPu\nDZB9SFGu23XnFE3V0QJzFiwYHV8NQByrJ6571efwzezKBK47bB3+058OQPorTbhKt94FgBz9FGRS\n23AN3WxIIYKf3Kz1UpXBPO/ZWlarI7fcChwIXBCp60COOhJX0RURb9+lZb0u/oz6yxc+/mkd84tO\nAmBoeJTa/i6mAhalXJntBx7D42Vc7MDif5mTJ1uD3sgI3KuKiFN1UtHF7yG56kvaxktfDED3Ax/R\n99asJbjgfL0/UtNYeld+nvj2bdr+Jy4FwM3OW/1JZKuWdd+his874Tite/llJNd9S++f9EStc9UX\n6F6rMe1gnTq/0izwUUgplAPSWryk+Q/ESI6UK7IR8AoC3YTQlFOhYEs4C4t1O2SqxjNPnesmOItZ\n9t6viHbbtwOQ3rIVd9gaAIKXn6Z1Lr4U74yXABBu3gxA5ymahpgeu4ng69dovUu2aEcmtry5eZpv\ne4++d+lFAERvOove176h5SZunKV5hmWfNUcNc++2+SVNf4DsHClfmR0KI5MR4nk4zxIjLc6VWoJl\n2u0hJXu2VZVh50Wn4D/3eQCEV18FQO+DivDki9cSWIKkM2XofL9vGrq2moyZRefEw89WUbaDNEUs\nE6spXnmZPop0DJ2/f38fkZ49i+39yhg8cdMI0zvqS5r/osgWkY0i8n0R2SYivxSRt9jzcRH5rojc\nZdexJfX4OKalIDsG3u6cu0VEqsBPReS7wJnA/zrnPmxHPN4FvPORmxLwfMJKiD+kci+19IPUIioO\ngaaiPrjoXAD8TZuIv/MDALxYyzI0JmlKL7Jcj9jaSBLc927Qds8+U9/7J91iy+ZNJF/7L21jQREp\ntjLSuXnid1+i7e6fsX7oJ9ZLpC6GLDLk+5DEsnJ5I865PcAeu6+JyDY0Ef4U4DlW7bPAD1iE2b4v\nVEeL+EVHq6IMGhnXWF9iKcHJvmnCosX6LlGzLXn6cfgbzax7zVn6bJu6XMPhIvLlrwKQXmcm2lCR\ndErNx+7LXqPPXnCi1vnO90l+qO5Zr2TMM5s8PvdtpPv2ASAWhiOKcNNqZ7tE/TFZsCKJE3oS/25S\n+sPQo1KQInIkcDxwI7DGfojsB5l8mHfOEZGbReTmzG/8eKUlK0gRGQK+ApzvnFsQWdQVAPz2yYPh\nVZGrjkNAQHtMlc2ee+4GIHzOyQAUzn49nY/9hw4uUrEgP/wxccfwE9kmyPwb4hxu+07rzH7MqIhn\nyJSmmo/x5/5TqwBSLffvdRCa3J7+ZjeelWW5kun0LOlaFTNjrzgdgPtrauotLDSICuV+IGQxWlIt\n0cMkXwGuds591R5Picg6K18H3L+kHh/HtCiyRSF8BbDNOffRBxRdB7wO+LBdv754d6L+50LI2ITK\n6muvV1NudmY/AK/+4CUMP18DsLXz3gqA9+tdeKvUqU9m3pnf2YkgJuP7iipxiGTpDYan0SF7P8Ul\nmTgzP7h5FykXcXX1r8R2eCr669OZ/ND7ALjRQnPXXHweAPVmjaQnJPHSpPZSxMizgNcCvxCRW+3Z\nhSiTrxGRs4BdwOlL6vFxTLn6s6tjkTvuxHWMjBb7loBnh4mmptQKeOrmp3Ha2e8GYPPkkQDULngX\n8RVfAMAvmj+7bP7sNEX6UzgQHsumlW39+/uX9IA87kfFrHKv3iJ9wgYAxrb8IwCzf/5MPvNxtYp+\n8iOVoCU7sVaQiOk9C+z8+Qzt+uIZUfkyezRyxz97HaUooDxm51hsxjKiiqnVaBN3dJn/yTNeBsCL\nzzyX4n//DwALb9Zgg7dblZoUvAMZUVlHD5jSgwLpkEKa2c02965VKr7xTIYu+gcAbvz1zwH49Eff\ny9atvwLgSZvXAZB0VLn3uimUHHfcsJfmXHcQPDiUKFdkj01E7q9esgE/KCCWyps2FJWR7Sib011m\nplVJpZ5udI7afDSn/c37ADh68jCtd6H+n95yG2I7yMw3osA15Wd4SrIggjuAaDekq6lyoeai7Hva\nMVxxmeaNfOvLVwMwsbFKq6tI7lpy/thaVdYj6yPqszV+dcM+GgNkH1qUK7JXrx1yp77mGBpxl/r9\nmrhem9JrlrxeKBaYqaunLjPJXNohXtANzsmnvwmAv3zhCwHwOm0QXRViifVxu9dPSciokJmH4ojb\nuu0Oyorsu7ZriO0zl32Endt/o2WiOqVYEOpNO4W83nJKylrW6XRpzrXYc9c8nWZ8aCnIVWsq7kWv\nfirNboNiURnZtbihb/naSSFgwc6GF23CqQi9VBnUtMNE5aI68oMoYt+MihvxdS5D3uHEdUuedPrD\nxRXN4Kw12nQs9Xe4om3cf5/6UVp1oWI704XZ7Dh1iGc55AXrs2yHo/bP1pHQZ++dc3Sai1sjAzGS\nI+V7NC8UwnUBpbhIaEemsy+JZLCICkXKsaKrY+5Ul6b4TlG19knqNu/nenRjChqHpWDB4zXRZub2\n67vdWJE9zy8AqHbKBL59FsPcumOb1LZO20JUsm+JFA+Ma37mwCFTgEg3vxyWVmnNp+y/d4WCBwNa\nOcoV2XGaMFVboPuAw/WB6bEspQGv3d/i+YGlDHge7VTlbMfZYdNsY+LFBKP9vGAA3EiPE854CgDb\nfnQ7AO07FX2VSb+/g6xQ6I8L9Es8gdhq8iw052DDevvuiWVcxaYLkyRmfE2Z26IV9PoNaGUo3w8F\npEKp4VMWr+95SzIfRkfRFYY+xVDlc9jR4fmeT9EsgcgzK6SkV4+EclFlvG+ngOebVRoqovmj8GkA\nPPlwlfVJkNCzz2OUChaSi9Xn3e31cLbSWhYodiJ4qclvc8L0PLWMUj9lau+9SLJyXr8Vo/HKKK/6\ni5fRS1IKloPheTqE2JRVEncomd3ctU8TFcsVqsN6niW1AEG1rM6g0AsIQl3mlSG1g6+/6U5236Wh\nrKE16lp95UvOBKDdSWnZWZpe1z5NZIfZx8cncCaLROyHDgJaTRVBQaCKtd7WTyH1Om3arTo/uW5p\nX9kbiJEcKddNjYjsAxrAdG6dHjxNsPRxHuGcW71YpVyZDSAiNzvnTsi104Og38c4B2IkRxowO0d6\nLJj9qcegz4OhFR9n7jL78UwDMZIj5cbsQ/lb24+Qqfs+EblXRG61v5OX1U8eYuRQ/9a2ZXSte2Cm\nLvBS4OVA3Tn3zyvRT17I7n9r2znXBbJvbR8S5Jzb45y7xe5rQJapu6KUF7Mf6lvbKz6ZlaAHZeoC\nnCcit4nIlctN+M+L2Uv61vZjTQ/O1AX+HTgKOA7NUd+ynPbzYvaj/tZ23vRQmbrOuSnnXOL0k5iX\no+LwoCkvZh/S39p+uEzdLCXa6FTg9uX0k4s/+2C+tZ0zPVym7itF5DhU5O0E/nY5nQx2kDnSYAeZ\nIw2YnSMNmJ0jDZidIw2YnSMNmJ0jDZidIw2YnSP9Pzm7YXtQ0I0DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0a86ec1fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Run the predictions here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "\n",
    "actuals = []\n",
    "actuals.append('14,Stop')\n",
    "actuals.append('28,Children crossing')\n",
    "actuals.append('22,Bumpy road')\n",
    "actuals.append('1,Speed limit (30km/h)')\n",
    "actuals.append('25,Road work')\n",
    "logits_list = []\n",
    "\n",
    "for img in im_new_list:\n",
    "    im_new = np.asarray(img)\n",
    "    #im_cl = np.zeros([32,32,3])\n",
    "    #im_cl[:][:][0] = clahe.apply(im_new[:][:][0])\n",
    "    #im_cl[:][:][1] = clahe.apply(im_new[:][:][1])\n",
    "    #im_cl[:][:][2] = clahe.apply(im_new[:][:][2])\n",
    "    im_new = im_new[np.newaxis, ...]\n",
    "    im_new = -1 + 2*(im_new/255)   \n",
    "    with tf.Session() as sess:\n",
    "        saver.restore(sess, tf.train.latest_checkpoint('.'))\n",
    "        logits_new = sess.run(logits, feed_dict = {x: im_new, keep_prob: 1.0})\n",
    "        logits_list.append(logits_new)\n",
    "        prediction = np.argmax(logits_new)\n",
    "        #print(prediction)\n",
    "        print('Actual: ' + actuals.pop())\n",
    "        print('Prediction: ' + str(dict_signs[prediction]) + \" (\" + str(prediction)+ \") \\n\")\n",
    "        plt.figure(figsize=(1,1))\n",
    "        plt.imshow(img)\n",
    "        \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 7\n",
    "\n",
    "_Is your model able to perform equally well on captured pictures when compared to testing on the dataset? The simplest way to do this check the accuracy of the predictions. For example, if the model predicted 1 out of 5 signs correctly, it's 20% accurate._\n",
    "\n",
    "_**NOTE:** You could check the accuracy manually by using `signnames.csv` (same directory). This file has a mapping from the class id (0-42) to the corresponding sign name. So, you could take the class id the model outputs, lookup the name in `signnames.csv` and see if it matches the sign from the image._\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** The five images shown above were randomly taken from the Internet. My model has an 80% accuracy on these 5 test images (i.e. 4 out of 5 traffic sign correctly predicted). This result is less than the accuracy I got on the provided test data set, i.e. 96.1%. This could be due to the small size of this test set and to the fact that in different countries traffic signs might be slightly different."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 5: TopKV2(values=array([  1.00000000e+00,   3.33784432e-13,   1.13291385e-17,\n",
      "         6.04670010e-19,   5.52352697e-19], dtype=float32), indices=array([18, 14, 26, 25,  4], dtype=int32))\n",
      "Top 5: TopKV2(values=array([  9.99994159e-01,   5.75650529e-06,   1.48018671e-07,\n",
      "         2.58121873e-14,   5.18540812e-16], dtype=float32), indices=array([ 1,  5,  2, 42,  3], dtype=int32))\n",
      "Top 5: TopKV2(values=array([  1.00000000e+00,   1.32992107e-27,   9.80624012e-32,\n",
      "         6.32752093e-32,   2.42538877e-34], dtype=float32), indices=array([22, 29, 20, 25, 24], dtype=int32))\n",
      "Top 5: TopKV2(values=array([  1.00000000e+00,   5.10338929e-24,   8.78536049e-25,\n",
      "         1.31620701e-29,   9.82939574e-30], dtype=float32), indices=array([28, 24, 29,  3, 30], dtype=int32))\n",
      "Top 5: TopKV2(values=array([  1.00000000e+00,   2.26571966e-18,   1.57794256e-18,\n",
      "         1.39096188e-20,   1.63548388e-24], dtype=float32), indices=array([14,  5, 17,  1, 10], dtype=int32))\n"
     ]
    }
   ],
   "source": [
    "### Visualize the softmax probabilities here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "logits_soft = tf.placeholder(tf.float32, (n_classes))\n",
    "probabilities = tf.nn.softmax(logits_soft, name=None)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    for logits_i in logits_list:\n",
    "        logits_i = np.squeeze(logits_i)\n",
    "        prob = sess.run(probabilities, feed_dict = {logits_soft: logits_i})\n",
    "        #print(prob)\n",
    "        top5 = sess.run(tf.nn.top_k(tf.constant(prob), k=5))\n",
    "        print(\"Top 5: \" + str(top5)) \n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 8\n",
    "\n",
    "*Use the model's softmax probabilities to visualize the **certainty** of its predictions, [`tf.nn.top_k`](https://www.tensorflow.org/versions/r0.12/api_docs/python/nn.html#top_k) could prove helpful here. Which predictions is the model certain of? Uncertain? If the model was incorrect in its initial prediction, does the correct prediction appear in the top k? (k should be 5 at most)*\n",
    "\n",
    "`tf.nn.top_k` will return the values and indices (class ids) of the top k predictions. So if k=3, for each sign, it'll return the 3 largest probabilities (out of a possible 43) and the correspoding class ids.\n",
    "\n",
    "Take this numpy array as an example:\n",
    "\n",
    "```\n",
    "# (5, 6) array\n",
    "a = np.array([[ 0.24879643,  0.07032244,  0.12641572,  0.34763842,  0.07893497,\n",
    "         0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.08594638,  0.0178669 ,  0.18063401,\n",
    "         0.15899337],\n",
    "       [ 0.26076848,  0.23664738,  0.08020603,  0.07001922,  0.1134371 ,\n",
    "         0.23892179],\n",
    "       [ 0.11943333,  0.29198961,  0.02605103,  0.26234032,  0.1351348 ,\n",
    "         0.16505091],\n",
    "       [ 0.09561176,  0.34396535,  0.0643941 ,  0.16240774,  0.24206137,\n",
    "         0.09155967]])\n",
    "```\n",
    "\n",
    "Running it through `sess.run(tf.nn.top_k(tf.constant(a), k=3))` produces:\n",
    "\n",
    "```\n",
    "TopKV2(values=array([[ 0.34763842,  0.24879643,  0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.18063401],\n",
    "       [ 0.26076848,  0.23892179,  0.23664738],\n",
    "       [ 0.29198961,  0.26234032,  0.16505091],\n",
    "       [ 0.34396535,  0.24206137,  0.16240774]]), indices=array([[3, 0, 5],\n",
    "       [0, 1, 4],\n",
    "       [0, 5, 1],\n",
    "       [1, 3, 5],\n",
    "       [1, 4, 3]], dtype=int32))\n",
    "```\n",
    "\n",
    "Looking just at the first row we get `[ 0.34763842,  0.24879643,  0.12789202]`, you can confirm these are the 3 largest probabilities in `a`. You'll also notice `[3, 0, 5]` are the corresponding indices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** As we can observe from the softmax probabilities shown above, my model is very confident of the predictions it makes for the correctly classified traffic signs: <br>\n",
    "\n",
    "1) Speed limit (30km/h) --> 99.9% <br>\n",
    "2) Bumpy road           --> 99.9% <br>\n",
    "3) Children crossing    --> 99.9% <br>\n",
    "4) Stop                 --> 99.9% <br>\n",
    "\n",
    "My model is also very sure about its incorrect prediction for the road work traffic sign. In fact, its top 5 predictions are <br>\n",
    "\n",
    "i) General caution (18)       --> 99.9% <br>\n",
    "ii) Stop (14)                 --> 3.33784432e-11% <br>\n",
    "iii) Traffic signals (26)     --> 1.13291385e-15% <br>\n",
    "iv) Road work (25)            --> 6.04670010e-17%   (actual traffic sign) <br>\n",
    "v) Speed limit (70km/h) (4)   --> 5.52352697e-17% <br>\n",
    "\n",
    "Even though the prediction for the road work traffic sign is incorrect, the correct prediction appears in the top 5 predictions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Note**: Once you have completed all of the code implementations and successfully answered each question above, you may finalize your work by exporting the iPython Notebook as an HTML document. You can do this by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}