01_Simple_Linear_Model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #01\n",
    "# Simple Linear Model\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This tutorial demonstrates the basic workflow of using TensorFlow with a simple linear model. After loading the so-called MNIST data-set with images of hand-written digits, we define and optimize a simple mathematical model in TensorFlow. The results are then plotted and discussed.\n",
    "\n",
    "You should be familiar with basic linear algebra, Python and the Jupyter Notebook editor. It also helps if you have a basic understanding of Machine Learning and classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-gpu/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This was developed using Python 3.6.1 (Anaconda) and TensorFlow version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.4.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/MNIST/train-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/train-labels-idx1-ubyte.gz\n",
      "Extracting data/MNIST/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "data = input_data.read_data_sets(\"data/MNIST/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set has now been loaded and consists of 70.000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Test-set:\t\t10000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n",
    "print(\"- Test-set:\\t\\t{}\".format(len(data.test.labels)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(data.validation.labels)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One-Hot Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data-set has been loaded as so-called One-Hot encoding. This means the labels have been converted from a single number to a vector whose length equals the number of possible classes. All elements of the vector are zero except for the $i$'th element which is one and means the class is $i$. For example, the One-Hot encoded labels for the first 5 images in the test-set are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.],\n",
       "       [ 0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.test.labels[0:5, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need the classes as single numbers for various comparisons and performance measures, so we convert the One-Hot encoded vectors to a single number by taking the index of the highest element. Note that the word 'class' is a keyword used in Python so we need to use the name 'cls' instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.test.cls = np.array([label.argmax() for label in data.test.labels])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now see the class for the first five images in the test-set. Compare these to the One-Hot encoded vectors above. For example, the class for the first image is 7, which corresponds to a One-Hot encoded vector where all elements are zero except for the element with index 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.test.cls[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data dimensions are used in several places in the source-code below. In computer programming it is generally best to use variables and constants rather than having to hard-code specific numbers every time that number is used. This means the numbers only have to be changed in one single place. Ideally these would be inferred from the data that has been read, but here we just write the numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We know that MNIST images are 28 pixels in each dimension.\n",
    "img_size = 28\n",
    "\n",
    "# Images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = img_size * img_size\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = (img_size, img_size)\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function for plotting images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot a few images to see if data is correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4\nEGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7\n+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgi\nAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6q\neqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfe\neAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMyd\nOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABO\nOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy\n411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgw\nILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4j\nnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cC\nJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb\n4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVA\nMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy\n47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir\n2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBj\nw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDu\nvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUm\niYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6S\nH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK0\n9/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+v\neeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRe\nkpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXU\nUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8\nE4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+\newD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bs\ntNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2\nQNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32e\nxx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv\n3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrW\nfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23\nfoXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S\n3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJx\nU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDm\nlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2\nOOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c\n6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pm\nR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8V\neQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLye\nwIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQX\nN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBna\nPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfM\nrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4n\nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/\nVMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoM\nRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAAN\nGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++\nfQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwI\nwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXr\ngS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmA\nDRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohS\nSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYi\nIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVq\ne+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23\n334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCL\nFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9\nNbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2\neOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+\nM/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4\nf+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBY\nvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6\nUSzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDK\nUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQ\ndNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VH\nHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9V\nIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdf\nfHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7Z\njtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu\n3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFB\nlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23K\nDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQ\nTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp\n0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltu\nSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9Xwv\ngMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpD\nERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGI\nCBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90\nCX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq\n49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZ\nLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CR\nzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1z\nblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAm\noW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38wa\nAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlR\nGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZS\nktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr\n+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogA\nqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcY\nlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxT\nTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI\n3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0a\nAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp3\n3XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9\njooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9Fgju\nEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD8\n4he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt\n27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp\n37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneA\nBjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW\n91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBD\nZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjk\nkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBE\nDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcf\nfzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2p\nfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfE\nY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUX\nAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQW\nu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0b\nAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPH\njBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDw\nusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkD\nBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwag\nVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+e\nOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ\n4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQR\noR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5\nEgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5\nfgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJ\nOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKO\nZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4B\noLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5\nnh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0b\nN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcA\nsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjs\nM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/m\nnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz\n24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2yd\nszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY\n7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhR\nebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cya\nOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543sze\ncs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiw\nug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoEL\ngOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/\nSuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5\nlRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f025779c550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.test.images[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.test.cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow Graph\n",
    "\n",
    "The entire purpose of TensorFlow is to have a so-called computational graph that can be executed much more efficiently than if the same calculations were to be performed directly in Python. TensorFlow can be more efficient than NumPy because TensorFlow knows the entire computation graph that must be executed, while NumPy only knows the computation of a single mathematical operation at a time.\n",
    "\n",
    "TensorFlow can also automatically calculate the gradients that are needed to optimize the variables of the graph so as to make the model perform better. This is because the graph is a combination of simple mathematical expressions so the gradient of the entire graph can be calculated using the chain-rule for derivatives.\n",
    "\n",
    "TensorFlow can also take advantage of multi-core CPUs as well as GPUs - and Google has even built special chips just for TensorFlow which are called TPUs (Tensor Processing Units) and are even faster than GPUs.\n",
    "\n",
    "A TensorFlow graph consists of the following parts which will be detailed below:\n",
    "\n",
    "* Placeholder variables used to change the input to the graph.\n",
    "* Model variables that are going to be optimized so as to make the model perform better.\n",
    "* The model which is essentially just a mathematical function that calculates some output given the input in the placeholder variables and the model variables.\n",
    "* A cost measure that can be used to guide the optimization of the variables.\n",
    "* An optimization method which updates the variables of the model.\n",
    "\n",
    "In addition, the TensorFlow graph may also contain various debugging statements e.g. for logging data to be displayed using TensorBoard, which is not covered in this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Placeholder variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Placeholder variables serve as the input to the graph that we may change each time we execute the graph. We call this feeding the placeholder variables and it is demonstrated further below.\n",
    "\n",
    "First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just means that it is a multi-dimensional vector or matrix. The data-type is set to `float32` and the shape is set to `[None, img_size_flat]`, where `None` means that the tensor may hold an arbitrary number of images with each image being a vector of length `img_size_flat`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, [None, img_size_flat])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we have the placeholder variable for the true labels associated with the images that were input in the placeholder variable `x`. The shape of this placeholder variable is `[None, num_classes]` which means it may hold an arbitrary number of labels and each label is a vector of length `num_classes` which is 10 in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true = tf.placeholder(tf.float32, [None, num_classes])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we have the placeholder variable for the true class of each image in the placeholder variable `x`. These are integers and the dimensionality of this placeholder variable is set to `[None]` which means the placeholder variable is a one-dimensional vector of arbitrary length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true_cls = tf.placeholder(tf.int64, [None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variables to be optimized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from the placeholder variables that were defined above and which serve as feeding input data into the model, there are also some model variables that must be changed by TensorFlow so as to make the model perform better on the training data.\n",
    "\n",
    "The first variable that must be optimized is called `weights` and is defined here as a TensorFlow variable that must be initialized with zeros and whose shape is `[img_size_flat, num_classes]`, so it is a 2-dimensional tensor (or matrix) with `img_size_flat` rows and `num_classes` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights = tf.Variable(tf.zeros([img_size_flat, num_classes]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second variable that must be optimized is called `biases` and is defined as a 1-dimensional tensor (or vector) of length `num_classes`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "biases = tf.Variable(tf.zeros([num_classes]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This simple mathematical model multiplies the images in the placeholder variable `x` with the `weights` and then adds the `biases`.\n",
    "\n",
    "The result is a matrix of shape `[num_images, num_classes]` because `x` has shape `[num_images, img_size_flat]` and `weights` has shape `[img_size_flat, num_classes]`, so the multiplication of those two matrices is a matrix with shape `[num_images, num_classes]` and then the `biases` vector is added to each row of that matrix.\n",
    "\n",
    "Note that the name `logits` is typical TensorFlow terminology, but other people may call the variable something else."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "logits = tf.matmul(x, weights) + biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now `logits` is a matrix with `num_images` rows and `num_classes` columns, where the element of the $i$'th row and $j$'th column is an estimate of how likely the $i$'th input image is to be of the $j$'th class.\n",
    "\n",
    "However, these estimates are a bit rough and difficult to interpret because the numbers may be very small or large, so we want to normalize them so that each row of the `logits` matrix sums to one, and each element is limited between zero and one. This is calculated using the so-called softmax function and the result is stored in `y_pred`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = tf.nn.softmax(logits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predicted class can be calculated from the `y_pred` matrix by taking the index of the largest element in each row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-18-1b869074280b>:1: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use the `axis` argument instead\n"
     ]
    }
   ],
   "source": [
    "y_pred_cls = tf.argmax(y_pred, dimension=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cost-function to be optimized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the model better at classifying the input images, we must somehow change the variables for `weights` and `biases`. To do this we first need to know how well the model currently performs by comparing the predicted output of the model `y_pred` to the desired output `y_true`.\n",
    "\n",
    "The cross-entropy is a performance measure used in classification. The cross-entropy is a continuous function that is always positive and if the predicted output of the model exactly matches the desired output then the cross-entropy equals zero. The goal of optimization is therefore to minimize the cross-entropy so it gets as close to zero as possible by changing the `weights` and `biases` of the model.\n",
    "\n",
    "TensorFlow has a built-in function for calculating the cross-entropy. Note that it uses the values of the `logits` because it also calculates the softmax internally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits,\n",
    "                                                        labels=y_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now calculated the cross-entropy for each of the image classifications so we have a measure of how well the model performs on each image individually. But in order to use the cross-entropy to guide the optimization of the model's variables we need a single scalar value, so we simply take the average of the cross-entropy for all the image classifications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "cost = tf.reduce_mean(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimization method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a cost measure that must be minimized, we can then create an optimizer. In this case it is the basic form of Gradient Descent where the step-size is set to 0.5.\n",
    "\n",
    "Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.5).minimize(cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Performance measures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need a few more performance measures to display the progress to the user.\n",
    "\n",
    "This is a vector of booleans whether the predicted class equals the true class of each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "correct_prediction = tf.equal(y_pred_cls, y_true_cls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This calculates the classification accuracy by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then calculating the average of these numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create TensorFlow session\n",
    "\n",
    "Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize variables\n",
    "\n",
    "The variables for `weights` and `biases` must be initialized before we start optimizing them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function to perform optimization iterations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 50.000 images in the training-set. It takes a long time to calculate the gradient of the model using all these images. We therefore use Stochastic Gradient Descent which only uses a small batch of images in each iteration of the optimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for performing a number of optimization iterations so as to gradually improve the `weights` and `biases` of the model. In each iteration, a new batch of data is selected from the training-set and then TensorFlow executes the optimizer using those training samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize(num_iterations):\n",
    "    for i in range(num_iterations):\n",
    "        # Get a batch of training examples.\n",
    "        # x_batch now holds a batch of images and\n",
    "        # y_true_batch are the true labels for those images.\n",
    "        x_batch, y_true_batch = data.train.next_batch(batch_size)\n",
    "        \n",
    "        # Put the batch into a dict with the proper names\n",
    "        # for placeholder variables in the TensorFlow graph.\n",
    "        # Note that the placeholder for y_true_cls is not set\n",
    "        # because it is not used during training.\n",
    "        feed_dict_train = {x: x_batch,\n",
    "                           y_true: y_true_batch}\n",
    "\n",
    "        # Run the optimizer using this batch of training data.\n",
    "        # TensorFlow assigns the variables in feed_dict_train\n",
    "        # to the placeholder variables and then runs the optimizer.\n",
    "        session.run(optimizer, feed_dict=feed_dict_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-functions to show performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dict with the test-set data to be used as input to the TensorFlow graph. Note that we must use the correct names for the placeholder variables in the TensorFlow graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "feed_dict_test = {x: data.test.images,\n",
    "                  y_true: data.test.labels,\n",
    "                  y_true_cls: data.test.cls}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for printing the classification accuracy on the test-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_accuracy():\n",
    "    # Use TensorFlow to compute the accuracy.\n",
    "    acc = session.run(accuracy, feed_dict=feed_dict_test)\n",
    "    \n",
    "    # Print the accuracy.\n",
    "    print(\"Accuracy on test-set: {0:.1%}\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for printing and plotting the confusion matrix using scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_confusion_matrix():\n",
    "    # Get the true classifications for the test-set.\n",
    "    cls_true = data.test.cls\n",
    "    \n",
    "    # Get the predicted classifications for the test-set.\n",
    "    cls_pred = session.run(y_pred_cls, feed_dict=feed_dict_test)\n",
    "\n",
    "    # Get the confusion matrix using sklearn.\n",
    "    cm = confusion_matrix(y_true=cls_true,\n",
    "                          y_pred=cls_pred)\n",
    "\n",
    "    # Print the confusion matrix as text.\n",
    "    print(cm)\n",
    "\n",
    "    # Plot the confusion matrix as an image.\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)\n",
    "\n",
    "    # Make various adjustments to the plot.\n",
    "    plt.tight_layout()\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(num_classes)\n",
    "    plt.xticks(tick_marks, range(num_classes))\n",
    "    plt.yticks(tick_marks, range(num_classes))\n",
    "    plt.xlabel('Predicted')\n",
    "    plt.ylabel('True')\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for plotting examples of images from the test-set that have been mis-classified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_example_errors():\n",
    "    # Use TensorFlow to get a list of boolean values\n",
    "    # whether each test-image has been correctly classified,\n",
    "    # and a list for the predicted class of each image.\n",
    "    correct, cls_pred = session.run([correct_prediction, y_pred_cls],\n",
    "                                    feed_dict=feed_dict_test)\n",
    "\n",
    "    # Negate the boolean array.\n",
    "    incorrect = (correct == False)\n",
    "    \n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    images = data.test.images[incorrect]\n",
    "    \n",
    "    # Get the predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "\n",
    "    # Get the true classes for those images.\n",
    "    cls_true = data.test.cls[incorrect]\n",
    "    \n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function to plot the model weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for plotting the `weights` of the model. 10 images are plotted, one for each digit that the model is trained to recognize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_weights():\n",
    "    # Get the values for the weights from the TensorFlow variable.\n",
    "    w = session.run(weights)\n",
    "    \n",
    "    # Get the lowest and highest values for the weights.\n",
    "    # This is used to correct the colour intensity across\n",
    "    # the images so they can be compared with each other.\n",
    "    w_min = np.min(w)\n",
    "    w_max = np.max(w)\n",
    "\n",
    "    # Create figure with 3x4 sub-plots,\n",
    "    # where the last 2 sub-plots are unused.\n",
    "    fig, axes = plt.subplots(3, 4)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Only use the weights for the first 10 sub-plots.\n",
    "        if i<10:\n",
    "            # Get the weights for the i'th digit and reshape it.\n",
    "            # Note that w.shape == (img_size_flat, 10)\n",
    "            image = w[:, i].reshape(img_shape)\n",
    "\n",
    "            # Set the label for the sub-plot.\n",
    "            ax.set_xlabel(\"Weights: {0}\".format(i))\n",
    "\n",
    "            # Plot the image.\n",
    "            ax.imshow(image, vmin=w_min, vmax=w_max, cmap='seismic')\n",
    "\n",
    "        # Remove ticks from each sub-plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance before any optimization\n",
    "\n",
    "The accuracy on the test-set is 9.8%. This is because the model has only been initialized and not optimized at all, so it always predicts that the image shows a zero digit, as demonstrated in the plot below, and it turns out that 9.8% of the images in the test-set happens to be zero digits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 9.8%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8VXP+x/HXp0IpRUqS6swoKSEjt4RcalzLvYw7MdTP\n/TYMjUsMSRmXEXoIk2uJLi5R5FahppTKpShqJl0Zkojv74+9vnuvfc6ps9fZ9937+Xj0OGuvtfZa\nH75nfc9nfdd3fb/mnENERFJTI98BiIgUE1WaIiIRqNIUEYlAlaaISASqNEVEIlClKSISgSpNEZEI\nVGmKiESgSlNEJIJa6Xy5UaNGrqysLEOhFIfp06evcM41znccuaIyLn0q42jSqjTLysqYNm1aOoco\nOma2KN8x5JLKuPSpjKPR7bmISASqNEVEIlClKSISgSpNEZEIVGmKiESQ1tNzkeoaOHAgAGvXrgVg\n1qxZAIwcObLCvhdddBEA+++/PwBnnHFGLkIUqZQyTRGRCJRpSk717NkTgBEjRlS63cwqrBsyZAgA\nEyZMAODggw8GoEWLFtkIUfLos88+A6BNmzYA3HvvvQBcfPHFeYupPGWaIiIRKNOUrPPZJWw4w9xl\nl10AOOKIIwD44osv4tvGjBkDwPz58wEYPnw4ANdff33mg5W8mjFjBgA1asTyuWbNmuUznEop0xQR\niUCZpmSNf5/5hRdeqLCtffv2QCKLbNSoEQD16tUD4Oeff47vu++++wLw0UcfAbBy5cosRSz5NnPm\nTCDxe3DCCSfkM5xKKdMUEYkg55mm74f3yCOPALDDDjvEt9WuXRuA0047DYDtt98egFatWuUyRMmQ\n//73vwA45+LrfIY5fvx4AJo2bVrpd30/ToB58+YlbTvmmGMyGqfk3+zZswG47777ADjzzDPzGc5G\nKdMUEYkg55nm1VdfDcDChQs3uI/vl1e/fn0A2rVrl5FzN2/eHIBrrrkGgI4dO2bkuFK5Y489Fkg8\n9QbYaqutAGjYsOFGv/vss8/Gl8Ptm1KaPv30UwDWrFkDJPe4KDTKNEVEIlClKSISQc5vz4cOHQok\nuo+Eb73nzp0LJDq4Tpo0CYCpU6cCidfmvvrqqw0ef7PNNgMSXVj8w4jwcfxtum7Pc6Nly5Yp73vX\nXXcBidfpwnzXI/9TSseAAQOA2NQbUNjXpjJNEZEIcp5pHnbYYUk/w/wrdN7q1auBRObp//p8+OGH\nGzz+FltsASRe+Pev5wGsWrUKgJ122qlasUv2jBs3DoB+/foBsG7duvi2Jk2aAHDHHXcAsOWWW+Y4\nOsmG8MNgf03767Zu3br5CCklyjRFRCIo6Ncot9lmGwAOPfTQpPWVZanlPf/880AiWwXYfffdAejV\nq1emQpQM8a9chjNMz3c/8UPCSWl46623Kqxr3Ljwp5tXpikiEkFBZ5rVsWzZMgD69OkDJL/C59vL\nqupYLblz3HHHAYnXKr2zzjorvty/f/+cxiS54ac4CfMvnhQyZZoiIhGUXKb5wAMPAImMc+utt45v\n80/mJP98/9nJkycDibZM36Z1ww03xPf1w4RJaZgyZQoAw4YNi6/bc889AejatWteYopCmaaISAQl\nk2m+++67QKIvnzd69Oj4sh+WTPLPDy67YsWKpPV+WED1pS1dEydOBJJ7tvg+2n54yEKmTFNEJAJV\nmiIiEZTM7fnLL78MJMZePPzwwwHYf//98xaTVOTnBPKvxnpdunQB4JZbbsl1SJJjfrCesJNPPjkP\nkVSPMk0RkQiKPtNcu3YtAK+++iqQGLDj5ptvBhJDxUn+hGePvP3224GKo7F36NABUPeiUrZ06VIA\n3nnnHSB5MJ3jjz8+LzFVhzJNEZEIij7T9IPW+jayI488EoBOnTrlLSZJdvfdd8eXP/jgg6Rt/jVK\ntWWWvsceewyAb775Bkhcq8VGmaaISARFmWn6AWsBbr31VgAaNGgAwI033piXmGTDBg0atMFt/rVX\ntWWWvkWLFiV99kM/FhtlmiIiERRVpumfwl5yySXxdevXrwfgqKOOAtQvs9j4Mk2ll4O/m/D7/vLL\nLwB89913Ffb1r+gNHjy40mPVrFkzvnznnXcCmkYj28aOHZv0+ZhjjslTJOlRpikiEoEqTRGRCIri\n9vzXX38FEiOhfPnll/FtrVq1AhIPhKS4+HmbUnHKKacA0LRpUyDRdeWZZ55JKwY/22V4DE/JHN+Z\n3ZdXsVOmKSISQVFkmgsWLAASMxaG+e4sGn+xcPmHdAAvvvhitY/z3HPPVbmPf0hUo0ZyPtC9e3cA\nOnbsWOE7nTt3rnZMUrUXXngBSDy09aO0F+vsoso0RUQiKOhM03eG7datW9L6gQMHxpeLtdvCpmTU\nqFHx5QEDBgAVB+zw5s6dC2y8nfK8884DoGXLlhW2nXjiiQC0bdu2esFKxvz4448AvPLKK0nr/TBw\n4W5fxUSZpohIBAWdaT700ENAxdevwm0hZpbTmCQ9qc5r/dRTT2U5Esk2377sZ4Tt0aMHAJdeemne\nYsoEZZoiIhEUZKbp+3Xdf//9eY5ERKrLZ5p+nvNSoUxTRCSCgsw0/Rzm33//fdJ6//aPhhETkXxR\npikiEoEqTRGRCAry9rw8P1PhxIkTAWjYsGE+wxGRTZgyTRGRCAoy07zuuuuSfoqIFAplmiIiEZhz\nrvpfNlsOLKpyx9LS0jnXON9B5IrKuPSpjKNJq9IUEdnU6PZcRCQCVZoiIhFstNI0s23NbGbwb6mZ\nLQl93jwbAZlZu9A5ZprZ92b2f1V8p7eZLQ/2n2dm56YZw3AzO66KfczM/mlm881slpl1SOec+ZKn\nMm5pZpPMbK6ZzamqfIPvqIyrKR9lHJz3cV9mKe5fHGXsnEvpH3ATcFUl6w2okepxovwDNgOWATtW\nsV9v4J5geXtgBdCo3D61Ipx3OHBcFft0B8YGy52B97Lx/yCX/3JVxsAOQIdguT6wANhZZVw6ZRwc\n82BgH2BmivsXRRlX6/bczFoFWcKTwByguZl9G9rey8yGBstNzGyUmU0zsw/MbL8Ip+oKzHPOLU71\nC865pcBCoIWZ9TezJ8zsPeAxM6tlZoOCOGaZWe8gxhrBX5tPzOx1oFEKp+oBPBGc811gezMrmSeu\n2Sxj59x/nHMzg+X/AZ8AzVKNTWWcGdm+jp1zbwGrqhNbIZdxOp3bdwHOdM5NM7ONHedeYIBzbqqZ\nlQHjgPZmti9wjnPuwo18txfwdJSgzKwV0BL4IhTnQc65n8ysD7DMObePmW0BTDWz14D9gN8B7Yhl\nQXOBIcHxbiP21+flcqdqBnwd+rw4WLc8SrwFLutlbGa/B9oDH6YalMo4o3JxHUdWyGWcTqW5wDlX\ncU7dig4H2lhiWoptzKyOc+594P0NfcnMagNHA1ekGM9pZtYFWAf0ds59G5xztHPup2CfbkBbM+sV\nfG4AtAYOAp52zv0GLDazSf6gzrm/pnj+UpTtMq4PPA9c7Jz7IYXzqIwzL6tlXA0FX8bpVJprQsu/\nEWsT8WqHlg3YxzlX+fSDG3Y08L5zbkWK+z/pnLuskvXhOA3o45ybGN7BzI6PGBvAEqA5MDX4vGOw\nrpRkrYwt9gBiFDDMOTcmxa+pjDMv29dxVAVfxhnpchTU7KvNrLWZ1QDCwU8A+voPKT2dijmVcrfm\nZnapmaVzGzAe6ONvQ8ysjZnVAd4GegZtIs2INWBXZQxwZnCczsA3zrlSum1Lkskytljq8BixBwT3\nltumMs6TLF3HFRR7GWeyn+a1xP5jJhNrF/D6AgcEDbZzgfODAPc1syGVHcjMtgIOAV4st6ktsDKN\nGB8CPgdmmtnHwIPEsu2RwFfE2kCGAfFJTczsNjM7qpJjjQWWmNmC4Dh9K9mn1GSqjA8m9kexqyW6\nvvwx2KYyzq9MXscjgHeAdma22MzODjYVdRkX1WuUZvYS0MM5tz7fsUh2qIxLX7GXcVFVmiIi+abX\nKEVEIlClKSISgSpNEZEIVGmKiESQ1hxBjRo1cmVlZRkKpThMnz59hduERvVWGZc+lXE0aVWaZWVl\nTJuWyhtYpcPMNqlpAVTGpU9lHI1uz0VEIlClKSISgSpNEZEIVGmKiESgSlNEJAJVmiIiEajSFBGJ\nQJWmiEgEaXVuz5Z///vfAJxwwgkALFy4sNrHeu211+LLbdu2BaB58+bVD07yZuzYsQB0794dgPvu\nuw+Aiy66KL5PzZo1cx+YJFm2bBkAp5xyCgCdOnUC4IILLgBinekz4bvvvgPg7bffBuCII44AYLPN\nNsvI8TdEmaaISAQFmWmOHz8egHXr1qV9rDFjEnN2PfroowA888wzaR9XcmflytjMCOGMEuDiiy8G\n4Lzzzouvq1OnTu4Ck7jVq1fHl3fddVcgkQk2adIEyHyG+Yc//AGAFSticy/6V0Fbt26dkfNsiDJN\nEZEICirTXL8+NmXIyy+Xn8+9+jp27BhfHjRoEABr1sRmA61bt27GziPZ49uslixJnln11FNPBaB2\n7doVviO54bM8334JiTuDvn1jc5T5tudM6d+/PwBffvklAA8//DCQ/QzTU6YpIhJBQWWab775JgCT\nJ08G4Nprr037mKtWrYovz5kzB4Aff/wRUKZZyMLt2T6zKO+MM84AIDaNuuSD7+kyadKkCtv69euX\nsfN8/PHH8eWBAwcCcPzxsWnZe/bsmbHzpEKZpohIBKo0RUQiyPvt+ezZs+PLvXr1AqBVq1YAXH/9\n9WkfP9zlSIrHrFmz4sv+FtCrVSv2a3vkkUfmNCZJ8B3Yn3/++QrbfNe+xo3TnzHE35Z37dq1wjb/\n8stWW22V9nmiUKYpIhJB3jPN2267Lb7sH9AMHz4cgHr16lX7uP4B0FtvvRVfpwcGxWPUqFEb3FZZ\n1iG5deWVVwKJa9V3NAc4+eSTM3aed999F4ClS5fG151zzjkAnH766Rk7TxTKNEVEIshbpjly5Egg\nuSO7b8vce++90z6+76YSzi67dOkCwNZbb5328SW7wncI3uabbw7A7bffnutwpBx/XfmfzZo1i2/z\n5VQda9euBRJl/MADDySdBxJtpvmiTFNEJIK8ZZojRowAEq80QsUBGarDDyP31FNPAYknrQA33HAD\nkP2ho6T6/IsNU6ZMqbBtyy23BKBDhw45jUmqNm7cuPhyt27dgMQdXSrXte8c739OnTo1aXsm20nT\npUxTRCSCnGeaflin8n9JAPr06ZP28f3L+8uXLwegXbt28W2HHnpo2seX7Prwww83uC0TdyKSGZde\neikAb7zxBgD/+c9/4tt8e7RzDoDRo0dXeTy/b/keLjvttBNQWO3YyjRFRCLIeabpB2JYvHgxkBje\nK1MWLFiQ9Ll9+/YZPb5kV2WZpm8by8SdiGTGXnvtBSTe6Js5c2Z826uvvgrAgAEDANhuu+0AOOus\nszZ4PD/4yu6775603k+V4TPOQqBMU0QkAlWaIiIR5Pz23L9c77uNhAfs8K8+NmzYMPJx/QACviuT\nd8ABB1QrTskt/7qc7yoW1qBBAwB23HHHnMYkVdtmm20AOOSQQ+Lr/PKdd96Z8nG++OILIPFAyNcP\nfuzMQqJMU0Qkgpxnmn62QP/KpH+dEuDoo48G4IorrtjoMcKjOPsHP4sWLQIqdlmoUUN/F4qBn1fG\nZxphGqCj9N1yyy1A4vr1D5EyMbxcpqlGERGJIG+vUd50001AcmbhX8XygxFvSPivj//L5GfFK88P\nIyWFrXxbdHhQlQsuuCDX4UgOhMv88ccfB6B+/foAbLvttnmJKRXKNEVEIshbptm2bVsAnnvuufi6\nGTNmABU7qJd30kknVVjnO876QVE934Yqhcm/5FD+qXn4SXkmhgqUwvPKK69UWOefa4QHNS40yjRF\nRCLI+3QXYXvuuWfSzyh+//vfV7o+3A90t912q15gkjV+KLjyT8179OiRj3Akh8KZZt26dQG46qqr\n8hVOypRpiohEoEpTRCSCgro9T4e/vSt/m6db8sLmO7V7jRo1AuCyyy7LRziSA0OGDAGSZ5hs0qQJ\nUNgPgDxlmiIiEZRMpll+djwpDuPHj0/63Lx5cyAxSIeUHp9phq/Vo446Kmmf77//HoDVq1cD0KJF\nixxFVzVlmiIiEZRMpvnTTz8lfVan9sL2yy+/ADB//vyk9bVr1wY0Y+imxs8a619OGTx4MJCYecG/\nZlkIlGmKiERQMpnmsGHDgMRAD/369ctnOFIFP2Sff0Vyzpw5ALRu3TpvMUn+PPLIIwAMHToUgN69\newNw44035i2mDVGmKSISQclkmj5jufzyywHNcV7oatasCcBtt90GJJ6kFkM/PUnPfffdB8Df/va3\n+LqDDjoISMxt76fR2HzzzXMcXdWUaYqIRFAymebYsWPzHYJUww477ADAo48+mudIJFcOPPBAAN54\n4408R1I9yjRFRCJQpSkiEoEqTRGRCFRpiohEoEpTRCQCVZoiIhFY+UF7I33ZbDmwKHPhFIWWzrnG\nVe9WGlTGpU9lHE1alaaIyKZGt+ciIhGo0hQRiUCVpohIBButNM1sWzObGfxbamZLQp+zOvyImdUy\ns1lm9mIK+/YPxTbbzI5O89zvmlmHKvapbWYjzWy+mU0xs8KZxCSCfJWxmT1uZsvNbGaK+/f2+5vZ\nPDM7N83zDzez46rYx8zsn0EZz6rqd6JQ6Tre6D6Rr+ONVprOuZXOuQ7OuQ7AEGCw/+yc+zk4qZlZ\nNjLWK4CPI+x/VxDnqcBjVm6GNTPL9OAkFwBLnXOtgAeAv2f4+DmRxzJ+FIh6UTwZxHkIMMDMGoU3\nZqGMjwWaB2Xch1g5Fx1dxxsV+Tqu1v8kM2tlZnPN7ElgDtDczL4Nbe9lZkOD5SZmNsrMppnZB2a2\nXwrHbwl0BYZFjc059zFgwDZBNvGgmX0A3G5m9czssSCOGWZ2bHC+Lc1sRJDBPA/UTuFUPQA/cclz\nwB+jxlrIsl3Gzrm3gFXVic05txRYCLQIspMnzOw9YhdZLTMbFMQxy8x6BzHWCLLGT8zsdaDRRk7h\n9QCeCM75LrC9mZVMVyRdx0A1ruN0au1dgDOdc9OqqP3vBQY456aaWRkwDmhvZvsC5zjnLqzkO/cA\nV5PaL3YSM+sE/OScWxX8kWoK7Oec+83MBgCvOufONrNtgPeDC+j/gNXOubZmticwLXS8YcA/nHPl\nbyObAV8DOOd+NrM1Zra1c+5bSkc2y7jazKwV0BL4IhTnQc65n8ysD7DMObePmW0BTDWz14D9gN8B\n7YAdgLnEsi7M7DbgPefcy+VOFS/jwOJg3fJM/vfkma7jiNdxOpXmAufctKp343CgTSjL3sbM6jjn\n3gfeL7+zxdqZvnbOzTSzwyPEc7WZnQ18D/QMrR/hnPstWO4GHGlmfwk+1wZaAAcBAwCcczPMbI7/\nsnPunAgxlJqslHEaTjOzLsA6oLdz7tvgnKOdc3460m5AWzPrFXxuALQmVsZPB78Li81skj+oc+6v\nGYyx2Og6jiidSnNNaPk3Yqm0F06LDdjHt52koBNwgpl1D45T38wed86dVcX37nLO3VNFnAYc55xb\nEN6hXLNJqpYAzYGlFmtMr1tiWSZkr4yr60nn3GWVrC9fxn2ccxPDO5jZ8dU4ny/jqcHnHYN1pUTX\nccTrOCMNv8FfgNVm1tpijcnhX9AJQF//wap4muWcu8Y5t6Nzrgw4HXjN/482swG+/aKaxgMXh2LZ\nM1h8G/hTsG4PYNcUjjUG8L8ApwCvpRFXwctkGW+MmV1qZunczo8H+vhbTTNrY2Z1iJVxz6Btsxlw\ncArHGgOcGRynM/CNc66Ubs2T6DpO7TrO5NOya4n9x0wm1vbj9QUOCBrl5wLnA5jZvmY2JOI5dgeW\nphHjzUBdi3VnmAPcFKy/H9jWzOYBNwIz/BfMbNgGfkEeBpqa2XxibSnXpxFXschYGZvZCOAdoJ2Z\nLQ5uyQDaAivTiPEh4HNgppl9DDxI7I5qJPAVsbbMYcCUUCy3mdlRlRxrLLDEzBYEx+lbyT6lRtdx\nFYrm3XOL5d6vOOeOyHcskj1m9hLQwzm3Pt+xSOaVwnVcNJWmiEgh0GuUIiIRqNIUEYlAlaaISARp\nvcfZqFEjV1ZWlqFQisP06dNXbEqjequMS5/KOJq0Ks2ysjKmTUvlZYLSYWab1LQAKuPSpzKORrfn\nIiIRqNIUEYlAlaaISASqNEVEIlClKSISgSpNEZEIVGmKiESQ6UmKRLJu9erVAHz11Vcb3Kdly5YA\nDB48GID27dsDsPPOOwOwxx57ZDNEKWHKNEVEIlCmKQVv3LhxAIwdOxaASZMmAfD5559v8Dtt2rQB\nYOHChQCsW7cuaftvv/1W/isiKVGmKSISQUFnmv/73/8A+MtfYpPOzZkTm1xuwoQJ8X0222yz3Acm\nGbdgQWyOrAceeACAhx9+OL5t7dq1AEQZMPvTTz/NYHQiCco0RUQiKMhMc/jw4QDccMMNQMWnpD4D\nBdh2221zF5hkzeLFsTm87rmnstlbU7fLLrsAiaflUnjmz58PwIoVK+LrXnjhBSDRXl2jRiyfu/DC\n2MSknTp1iu/bunXrXIS5Qco0RUQiKKhM02cbl19+OZD4S1R+EviLL45Pecz9998PQMOGDXMRolRD\nOKPwmWTnzp0BOOKI2KSEm2++OQANGjQAoF69evHv/PDDDwD88Y9/BBJZ5L777gvAnnvuGd+3Tp06\nANStWzfD/xVSXbNnzwYS7dWjRo0CYPnyqqeQnzp1KpD87ML3jPC/Q//4xz+AxO9QtinTFBGJQJWm\niEgEBXV7PnDgQABWrly50f2eeeaZ+PIrr7wCJB4a+Vv3XKXqsmFr1qwBoGvXrvF1H330EQAvvvhi\n0r77778/ADNmzABiUzB4/kHgjjvuCCQeEkhhmjVrFpC4HX/22WcB+O6775L28+UJcOCBBwKJcr/r\nrrsA2GuvvQB4//334/v6+uHll18GEq/E+odG2abfPhGRCPKeaS5alJjfaNiwYUnb/F+QJk2aAPD6\n669X+L7/6+Wz1NNOOw2A7bffPvPBSkp+/vlnAP70pz8BiewS4Prrrwfg8MMPr/S7lc2K2KJFiwxH\nKJn25z//Ob7suw+Vf9Djy3y33XYD4Pbbb49vq127dtK+U6ZMAeDBBx8E4JxzzolvmzlzJpC4xvv0\n6QPAiSeeCEDjxtmdSFSZpohIBHnPNP1fDUh0Wj/ooIMAeOuttwD46aefAHjqqacA+Pvf/x7/ju8o\nu3TpUgB69OgBJNo61RUpd3zXIJ9B+AE2wn/5r776agC23HLLHEcnmeSvyQEDBgDwyCOPxLf51123\n2247AC666CIgUfapdAfz7Zbr168H4Oabb45v813P/GAsuaZMU0QkgrxnmuEhu3wndt+53fPtHeee\ney4AI0eOjG/zAz34v24+g9HT89zzT8TvuOMOIDEQ8DvvvBPfx3del+LmX3f0T7nDg6k0a9YMSHRi\n32effao83q+//grA119/DcCZZ54JwNFHHw0kBp6uzBlnnAHA1ltvnXL86VCmKSISQd4zzaeffrrC\nupdeegmA4447rtLvTJs2bYPH22+//YDk1/AkNyZPnpz02b/eGO6PJ6XBtzXWrFmzwjb/yqPvW+nv\nDD/55JOk/fwrrwDz5s1L+tmoUSMg8ayiMr5Xje+jnathIpVpiohEkPdM89RTT40vjx49GoAPP/wQ\nSPxl8i/8+/5f4fYN347h1/nBa307R7t27bIWuyQLtzVDogdD+Mln9+7dgeRBNqT4HHbYYQAccsgh\nQHIfat/3+pJLLqn0u7Vqxaodn61WpnyGGX4L7IQTTgDg3nvvBaBp06aRYk+XMk0RkQhUaYqIRGBR\n5l0pr2PHjm5jD2VSsWrVqvjyTjvtBCRejfSxlR9PMzwAhB8U4JhjjgHgs88+A+CCCy4AYMiQIWnF\nV56ZTXfOdczoQQtYlDL25VS+vML8gwM/uIIfE9N3NWnVqhUAu+66a4Xv+jmi/OAe2XrApDKO7ttv\nv40v+y5n7733HpCYXcG/Duu7GYZfrw0PyFEZ30EeEi9PpNPFKJ0yVqYpIhJB3h8EhV9zHDFiBAAn\nnXQSUDHj9A3Ld955Z/w7vuO7bxz2r1iOHz8eSHR+h0QmK9lx1VVXAXD33XdvcB/fidnfIfifUfjX\n87p06QIkDxUo+RHO+nymWRXfgR0qZpr169cHYNCgQQCcffbZ8W2VdXPKJWWaIiIR5D3TDPNDR/mu\nK36ADv9X7JZbbgEqDiMFcOONNwKJzrG++5L/DsDjjz+ejbAl4DOMU045BUgM0/fLL7/E9/HzQPmM\nszqWLVsGJO5MwjNP+o7OUrj8IB8bu0PwQ8L54QULiTJNEZEICirT9HzGuaGBaivjX8nq2bMnkMg0\n33zzzfg+/km9hovLDt/WtPfeewOJngxhEydOBBLZ50033QTABx98EPl8vq17+vTpkb8ruTd06FAA\n+vfvDyTfgXj+rsEPKFyIlGmKiERQkJlmOnx72pgxY4DkdhM/R3q/fv1yH5gAidfvPD8Itc80/aAL\n4ekNzj//fAAGDx4MJNq6pTj4sr3yyisB+P777yvss9VWWwGJtswtttgiR9FFp0xTRCQCVZoiIhGU\n3O25Hw3lmmuuAZLn1/YPHXr16gXAzjvvnNvgpIJu3boBiVkq/cMBP1oVwOeffw4kRgsvz48ULoXJ\nzxXl5wDzwnMF+ea0zp075y6walKmKSISQcllml6HDh0AuPXWW+Pr/Gt+1113HQDDhw8HkkeQltxq\n27YtkOgq9uyzz1bYJ9xtDBLjMfr5Y8Kv1Urh8A98fGf28k4//fT4sn8lthgo0xQRiaBkM00vPCjA\nQw89BCRmyfNtZbvvvnvuAxMgkeXfc889QCI7CXdY/+abbwAoKysDEmXq26ilsPzwww9A4i7i559/\nTtq+xx57AIkyLzbKNEVEIij5TLNx48bx5QkTJgCJ+bj9ABPqLJ1/fmbBcePGAfCvf/0rvm3KlClA\nIrP0Q8NJYXrjjTcAWLJkSaXb/XBvlQ28UwyUaYqIRFDymWaYH27fT5fh+4bNnTsX0MyVhcTPJlp+\nWQqfH6ZAjD8/AAAEVklEQVSxPN93+tBDD81lOBmnTFNEJIJNKtP0/CDH/ine/PnzAWWaIpkQniwR\nEm3Ql112WT7CyThlmiIiEajSFBGJYJO8Pfcz3X355Zd5jkSk9FxxxRVJP/2DoaZNm+YtpkxSpiki\nEsEmmWmKSPZcfvnlST9LjTJNEZEIzM/oV60vmy0HFmUunKLQ0jnXuOrdSoPKuPSpjKNJq9IUEdnU\n6PZcRCQCVZoiIhFstNI0s23NbGbwb6mZLQl93jxbQZnZFWY2J/h3cQr79zaz5UFc88zs3DTPP9zM\njqtiHzOzf5rZfDObZWYd0jlnvuSxjBeb2ezgPO+nsL/KuJp0HW90n8hlvNEuR865lUCH4OA3AT84\n5waWPymxttHfqjpZKoKgzwI6AuuB18xsnHOuqp7oTzrnLjOz7YGPzWyMc25F6Li1nHPrMxFj4Fig\nuXOulZl1Bh4ADsjg8XMiH2UccqBz7tsI+6uMq0HX8UZFLuNq3Z6bWSszm2tmTwJzgOZm9m1oey8z\nGxosNzGzUWY2zcw+MLP9qjh8W2Cqc26tc+4X4G3g+FRjc84tBRYCLcysv5k9YWbvAY+ZWS0zGxTE\nMcvMegcx1gj+2nxiZq8DjVI4VQ/gieCc7wLbm1nJPHHNchmnRWWcGbqOgWqUcTptmrsAg51z7YDK\nh2iOuRcY4JzrCJwC+ELY18yGVLL/bOBgM2toZnWBI4HmqQZlZq2AlsAXoTgPc86dDlwALHPO7QPs\nDfQ1sxbAScDvgHbAOUCn0PFuM7OjKjlVM+Dr0OfFwbpSkq0yBnDAG2Y23czOixKUyjijdB1HLON0\n3gha4JyblsJ+hwNtYtk/ANuYWR3n3PtAhbYs59zHZjYImAD8AMwAfk3hPKeZWRdgHdDbOfdtcM7R\nzrmfgn26AW3NrFfwuQHQGjgIeDq4NVlsZpNC8fw1hXOXqqyUcWA/59yS4DbsdTOb55ybXMV5VMaZ\np+s4onQqzTWh5d8AC30OT/5hwD7OueQp6TbCOfcw8DCAmQ0A5qfwtSedc5UN2BeO04A+zrmJ4R3M\nLOXbhpAlxP5yTg0+78jG/1IXo2yW8ZLg51IzGw3sA1RVaaqMM0/XccQyzkiXo6BmX21mrc2sBslt\nFxOAvv6DpfB0ysy2C36WAd2BZ4LPl5rZhWmEOh7oY2a1guO1MbM6xNpbegZtIs2Ag1M41hjgzOA4\nnYFvnHPL04itoGWyjM2snpnVC5brAl2Bj4PPKuM80XWcWhlnsp/mtcT+YyYTaxfw+gIHBA22c4Hz\ngwA31t71YrDvi8CFzrn/BevbAivTiPEh4HNgppl9DDxILNseCXwFzAWGAVP8FzbSFjIWWGJmC4Lj\n9K1kn1KTqTJuCrxnZh8BHwAvOOcmBNtUxvml67gKRfUapZm9BPTIcJcDKSAq49JX7GVcVJWmiEi+\n6TVKEZEIVGmKiESgSlNEJAJVmiIiEajSFBGJQJWmiEgEqjRFRCL4fy63uy42kCxvAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024ecaacc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 1 optimization iteration\n",
    "\n",
    "Already after a single optimization iteration, the model has increased its accuracy on the test-set to 40.7% up from 9.8%. This means that it mis-classifies the images about 6 out of 10 times, as demonstrated on a few examples below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize(num_iterations=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 43.1%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8TWX+wPHP1yAiipOS4jRdRDdK6KqJpIsSksl0Ubqh\nqWlK/WoyXWiKSWWaUboX3YhSTYwSuoiIXFNRRCkaXZhS8fz+WOu719rnsvdeZ+999t7H9/16nZe1\n937WWo/znPXs73rWcxHnHMYYY1JTLdcZMMaYQmKVpjHGRGCVpjHGRGCVpjHGRGCVpjHGRGCVpjHG\nRGCVpjHGRGCVpjHGRGCVpjHGRFA9nZ2LiopccXFxhrJSGObNm7fBObdrrvNRWayMqz4r42jSqjSL\ni4uZO3duOocoOCKyKtd5qExWxlWflXE0dntujDERWKVpjDERWKVpjDERWKVpjDERWKVpjDERpPX0\nPFv+/ve/A/Djjz8CsHDhQgDGjx9fKu3ll18OwJFHHgnAueeeWxlZNMZspyzSNMaYCPIq0jz77LMB\nGDduXJmfi0ip9+6//34AXnvtNQA6dOgAQNOmTbORRZMlmzdvBuDaa68FgnIFaNOmDRD8XTRr1qyS\nc2cq28aNGwFYvXp1uWn07+Duu+8G4KCDDgJg//33B+DQQw/NSt4s0jTGmAhyHmlqdAnlR5gHHHAA\nAF26dAFg5cqVsc8mTZoEwCeffALAmDFjALjhhhsyn1mTNV988QUADz74IAC/+c1vYp/paJWXXnoJ\ngIEDB1Zy7ky2vfzyy0BQxtOnTwfg448/Lnef5s2bA/DZZ58BsGXLlrjPt23bluFceizSNMaYCHIW\naWr0MHHixFKfaduERpFFRUUA1K1bF4Cff/45lrZdu3YAfPDBBwB88803WcqxyYb169cDcP755+c4\nJybbVqxYAcA///lPAEaPHh37THvKRFlSfPny5RnMXeos0jTGmAhyFml++eWXQPw3i0aYU6ZMAaBx\n48Zl7qv9OAGWLVsW99lpp52W0Xya7Bg5ciQAL7zwAgDvvfde0n3efPNNIPib0aejxx13XDayaDJs\nzZo1ANxzzz1pHUefcWh9Udks0jTGmAhyFml27doVCJ56A+y0004ANGjQIOG+zz77bGw73L5pCsdV\nV10FxD8lT2bChAlx/2pf3Oeeey6W5vDDD89UFk0EGzZsiG1rJHnMMccAQa+XmjVrAlC/fn0geEYB\nsGnTJgBOOukkIIgi9ZlF69atY2lr164NQJ06dTL8v0iNRZrGGBOBVZrGGBNBzju3RxkSN3z4cAA+\n+uijUp9pGK//mvx0yimnAMHDnK1btybdR7uc6e3YqlXeSgWffvopAEcccUQsbbY6NJuy6fDXE088\nMfaedv/Th3xKJ9WZP38+4C2zoXS45J577glAtWr5G8/lb86MMSYP5TzSTIUOsRo8eDAQP1xqt912\nA+COO+4AYMcdd6zk3JlkZsyYEdv+8MMPgWDylfIeBF122WWx7c6dOwPBA4Rp06YBMHTo0FL7jRo1\nCgimDDTZoQ9gzznnHCCILiEYwtypU6cy9y1r5ctCmmDHIk1jjImgICJNHXJZckA+BBN+6JRwJn/o\nRAq9e/eOvRfumhKmkUbPnj0B+Otf/xr7rOTdg7aDP/DAA6WOOWjQIAB++uknIJjco0aNGhX7T5g4\n2jXo9ttvB4IJNnbdNVhCXKf3q6p3fRZpGmNMBHkdaXbr1g0IhlWq8OQOQ4YMqdQ8mdT98ssvQPnR\nJQRDIHXAgj4pT0QjTW07u/rqq2Of6dNcjThPP/10APbZZ59IeTdl0yfi+gxBy0KHuELQ9lxVWaRp\njDER5GWkqZN5vPPOO0DQlqntJn/5y19iacNDsUzh0L6Vjz76KJBahFmSRpFjx46NvTdnzpwM5M6U\nR69JpcMbtX/l9sAiTWOMiSAvI83u3bsDpdvC+vTpA1j7VKEpa9TP7Nmz0z6ujioKjwIqOdJIn8Lr\nMigmPSWX0X711VcBuOWWW2Lv6R1AeJKNqsQiTWOMicAqTWOMiSCvbs91TSAd0K+OP/54AG699dbK\nzpJJg65dHmXOzCi0Y3X476Xk8MzwbaNJn67ppL9nfUgb/j1rN0AdCquT6Hz++ecA7LvvvgAceOCB\npY6/ZMkSIJjcIx8fMFmkaYwxEeQ80gyvHqlDs0rOxt6qVSvAuhcVGp1oJVM0ylm6dCkQ/L2URbsw\n2fDJzLrmmmsAuOuuu8pNow/hdNVJ/TeKRo0aAcFd5jPPPBP5GNlikaYxxkSQ80gz/I1VsmOyDqO0\ntkwDwVRwiSIXnXbs8ccfBwpryrFCoMMne/XqBQTdAHXILASrTqYywXR5vv76awDGjRsHxK88GR7c\nkgsWaRpjTAQ5jzRHjBhR7mcaUVhb5vZNl8jQCYwTadmyJQDHHntsVvO0vdJeCToMtqylZ15//XUg\niD5vvvlmoGJDXHWwwrx58yLvmy0WaRpjTAQ5jzQT0SfrqTwB1emoNK1+y3333Xel0m7cuBGAu+++\nu8xjhfsV3nnnnUDVnVA1mxItnqbD79TFF18MwBdffFHucbRvYCKZfmJvouvYsWPc6wULFgBBpKnX\naN++fWNptPz1mnzqqaeyns+KskjTGGMisErTGGMiyOvb80MOOSTltNoFonHjxgB89dVXQPqdYnW1\ny1x3cyhEuiKkzqIeduqppwKlh1iWNeRSb+9TWbnS5B9dTVRn2tems9GjR8fSfPzxxwBMnz69zGM0\nadIkizmMxiJNY4yJIOeRpnYngWD9kYp47rnnkqbRBuhq1eK/K3T+vzZt2pTa55hjjqlwnrZ3Oi/q\nsGHDYu8lWi8oGR0a2aJFCwAefPBBILi7MPlJy0tXjtX1oMLeeOONuNfVq3tVk96R6APZfGCRpjHG\nRJDzSHPChAmxbY1ISk7YoXSihkTtlBdddBEQrJIX1qNHDyD45jPZpWUQjiz0buKee+6JfLwbb7wR\nCNYyN4Whdu3aQFDmP/zwAxDfYV2fQegw2PPOOw8IOsbnE4s0jTEmgpxHmmFlPWUtSz53fDWl6drm\n4W19oqpPUHVC4a5duwJw6aWXxvbRzu06RNIUJu2JogMQnnzyydhns2bNAoLIUqeGy0cWaRpjTAR5\nFWma7UeXLl3i/jXbn3PPPbfM7XxnkaYxxkRglaYxxkRglaYxxkRglaYxxkRglaYxxkRglaYxxkRg\nlaYxxkRglaYxxkQgOkStQjuLrAdWZS47BaGZc27XXGeislgZV31WxtGkVWkaY8z2xm7PjTEmAqs0\njTEmgoSVpog0FJEF/s86EVkbel0zGxkSkWYiMl1ElorIEhFJOuOsiPQTkfV+vpaJyIVp5mGMiHRL\nkqa7iCz0z/meiByVzjlzJRdl7J/3FBFZLiKfiMi1KaQfEsrbIhE5Nc3zvyUirZKkGRn6XXwsIhVf\nqyOHcljGa/yyWiAis1NIn4vruIGITPKv5dkiknz+QedcSj/AzcA1ZbwvQLVUj5PCefYAWvnb9YAV\nwP5J9ukH3ONv7w5sAIpKpKkeIQ9jgG5J0tQlaBM+DFicqd9Brn4qsYxrACuBZsAOwKIUyngIcJW/\nfRCwXn//FSzjt/TvLMX0fwJG57qMCqWM/WOuAXaOkD4X1/HdwI3+9oHA1GTHrdDtuYjs60eCY4El\nwF4i8m3o894i8pC/vZuITBCRuSIyR0TaJzq2c+4L59wCf/t74EMg5fU7nXPrgM+Apn508oSIvA08\nJiLVRWSEn4+FItLPz2M1EfmXiHwoIlOBohTOs8n5v2mgDlClnqhls4yB9sAy59wq59wW4DngjFTz\n5pxbjHeR7+JHE6NEZA5wu4jUFZHH/HzMF5Gufh53FJFxfgTzPFAryu8D+D3wdMR98lqWyzgtlXUd\nAy2Baf45lwD7i0jDRDuk06Z5AHC3c64lsDZBupHAMOdcG6AXoIXQTkTuT3QCEfktXlTxXqqZEpF9\n8SKYlaF8dnTO/QG4BPjaOdcWOAIYICJNgZ7A3ni/wL7AUaHjDRWRUyiDiPQUkeXAC3jfklVNtsq4\nCfB56PUaInwxitcU8pNz7r/+W42B9s65QcBgYLJfxicAd4lILWAgsNE51wIvam0dOt6jiW7VRWQf\nP38zUs1jAcnmdeyAaSIyT0QuipKpSryOPwC6+2mOBPb0f8qVziTEK5xzc1NI1wloLiL6ehcRqe2c\nmw2U284hIvWA54ErnHObUjhPHxE5HtgC9HPOfeuf80Xn3E9+ms5ACxHp7b+uD+wHHAc87ZzbBqwR\nkel6UOfcjeWd0Dk3HhgvIr8DbvOPX5VktYwr4FoRuQD4ATg79P44v+zAK4OTReR6/3UtoCleGQ8D\ncM7NF5ElurNzrm+S8/YGngudoyrJZhm3d86tFZHdgakissw5906S81T2dTwUGCkiC/Aq0A+ArYky\nmE6luTm0vQ3vdkmFb30EaOucK3uJyTKI1zg9AXjUOTcpxd3GOueuSpJPAfo7514vcb4zU81bWZxz\nb4jI4yKys3Pu2+R7FIxslfFaYK/Q6z1JHOWo4c65spaxLFnG3ZxzK8IJQhd7RfQGIkVKBSRr17Fz\nbq3/7zoReRFoCySrNCv1OnbOfQec7+9fDa9J4NNE+2Sky5Ffs28Ukf38E4cz/xowQF8kug3yPxfg\nMWCBc25kic+uFJHL0sjqFKC/iFT3j9dcRGoDM4Gz/TaRJkCHZAfy24PE326D91CiKlWYcTJZxsC7\nQEvxekrsgHe7N8nfd5i2Q1bQFOCKUF70NnwmcI7/3qF4jf5JichBQG3n3Jw08lQQMnwd1xWRuv52\nHeBEYLH/Op+u451FpIb/8lLgNefc5kT7ZLKf5nV4/5l38Nqo1ADgaL/BdilwsZ/Z8tpCOuA1up8o\nQbeIk/zPWgDfpJHHB4CPgQUishgYhRdtjwdWA0uBR4FZukOCtpBewGI/rB9J/O1iVZWRMnbO/QL8\nEZiK9zsf45xb7n98CLAujTzeAtQRr6vLErynxQD3AQ1FZBlwEzBfd0jSptkbeCaN/BSaTF3HjYG3\nReQDYA4w0Tn3mv9ZPl3HBwNL/WcTHYGrk528oIZRisgrwBnOuV9znReTeX7k/qpzzlZbq8IK/Tou\nqErTGGNyzYZRGmNMBFZpGmNMBFZpGmNMBFZpGmNMBOl0bqeoqMgVFxdnKCuFYd68eRvcdjSrt5Vx\n1WdlHE1alWZxcTFz56YyAqvqEJHtalkAK+Oqz8o4Grs9N8aYCKzSNMaYCKzSNMaYCKzSNMaYCNJ6\nEGRMRW3ZsgWAo47y5omdP9+bP+P0008H4IUXXshNxoxJwiJNY4yJIK8jzTfffBMIopHly73Zw15+\n+eVYmldeeQWAU0+NX5zwyCOPBODYY4/Nej5N6jTC/NOf/gTAggULgGCS4MMPPzw3GTMmRRZpGmNM\nBHkVaX7//fcA9OnTB4DXX/dms69duzYAv/zyCwA//PBDqX1nzpwZ91r3qVOnTuy9UaNGAdCzZ89M\nZttEMHKkNxn/Aw88AEDHjh0BuPXWWwFo3z6rixyaHNq4cSMQtF9Pnjw59tnw4cOB4I7jrLPOAqBZ\ns2YA/PnPf46l3W233bKf2QQs0jTGmAjyKtK87rrrgPg2S4Aff/wRgBYtWgDQqFGj2Gf16tWLS7tt\nm7dgoLZ16r4AF13krY21//77A3DIIYdkLO8mNV9++WXc606dOgEWYVZFemd41113AXDfffcBpf8G\nIIgw9d/x48fHfb5hw4bY9iOPPJL5zEZgkaYxxkSQ80hz8eLFse2S3y577eWt8vrEE08AsO+++wKw\n8847x9LUrVs3bh+NNLWN7Lbbbot9pm2mN998MwAPP/wwALvsskt6/wmTsk2bvCXsa9asCQSRpql6\ntN36xhvLW3I8cPzxxwMwY8aMMj9//PHHY9sWaRpjTAHJeaSpkQcE7RbarjFo0CAg+BZKRbVq3veA\nRpM//xysbf/3v/8dgIkTJwJw4YUXAnDaaadVIOcmVV988UVs+6GHHgKCvreHHXZYTvJkskfvHsN3\neWW58847Y9tXXnklAIMHDwZg2LBhWcpd+izSNMaYCKzSNMaYCHJ+e67D6sIuuOACAAYOHJj28W+/\n/fbY9jPPPAPAp59+CsCECRMAuz3PtiFDhmTluLNmzQJgzZo1pT479NBDgaB7mcmu8APdG264AYD1\n69cDQXObdlSfNGkSAC1btozto81q+gD3zDPPBIIJXPRYEHQVXLhwYYb/F6mxSNMYYyLIeaR50003\nlXqvXbt2WTlXly5dgGA45bvvvpuV85h4OtAgrF+/fpGPc/nll8cdT4fl/e9//yuVVgc9XH311UDZ\nf2cmc3RoJASDU5xzANSoUQOAAQMGAHDQQQeVexxN27ZtWyC469QO8gCLFi0C4JJLLgFg9OjRaec/\nCos0jTEmgpxFmitXrgRg7dq1sfe00/rBBx+clXOecMIJQBBpmuzSCFCH0wHsueeeQBBBlPTrr78C\n8P7778fe69atGwDr1q0Dgghm1129FVjDHeR1v9WrVwNBB+vzzjsPCNrVTGa9+uqrsW1tw1TaZTA8\n6Uaq7rjjjlLH10jzvffei3y8TLBI0xhjIshZpDlmzBggiDghmLJNOz6bwqYd2b/66qvYe5deemmZ\nabUDvLZPldUxukmTJgCce+65APTv3x8Iotcwfeqq7Z86SYRFmpn1zTffADB79uxy02h5pSN8DB30\nkisWaRpjTAQ5izSffvppIH7yDR1KZaqG8BNVtd9++5WZVvty3n///UB8u5hOVDxixAgg8dNXpZO7\nmOyaN28eAJ999lmpz4477jig9FI0mfLtt98CwV1E48aNs3KekizSNMaYCHLeT/OAAw6IbR9zzDE5\nzInJtPBEHeX56KOPgGC0ltI+eAD33nsvEEwnF4Uu1GYTg2TH3Llzy/3slltuAbI39aL2kNDRSBZp\nGmNMHrJK0xhjIqj02/PNmzcDQSdmU3XpTPnaGb3kNsA//vEPIGjU15VI0x2AoPO0Vq/u/YlX5Nbe\nJKcDGEqWK0CHDh2ycs6yzlWZLNI0xpgIKj3SfPbZZwH45JNPACgqKqq0c+uUVEonBzDZUXKFwZLb\nEDws0vdTeXiUiO6vHet79OiR1vFMYvogqGS5ZlNZf1eVySJNY4yJIOddjrJNO98CvPTSS3GfDR06\ntLKzY0rQYZPvvPNO3L/hyaN16GXDhg2THq979+4A7LjjjkDFJokwhWGnnXYCUvu7yCSLNI0xJoIq\nG2lqhBmevFSf0Gonep2U2GSWtivq8LZENErQKd10oo3wpMFTpkwBgsltNcLQ1+HlNHTo5l/+8hcA\n2rdvX8H/hcknTzzxRKn3dMXZyh64YJGmMcZEUOmRZnFxMRAsR5BpW7duBYI1zsPD83QKMf1M+/CZ\nzNpjjz2AYFGzVatWxT6bNm0aELRTatujDoHTiWU1igRo0aIFENwpaDulPiHXY0AQYdryFpVDJwle\nsGBB7D1dBO3CCy8E4JFHHkn7POGF1Ro1agTAZZddlvZxK8IiTWOMicAqTWOMiaDS7091nR69hfvu\nu+9in23YsAGI1uFd1z7+17/+BQQPFMpaP0Rni8/Wapcm3sMPPwzEz6eoM6l37twZCFaLLDlDTXgm\ncO1+pO/pMLrmzZvHfQ7BetmmcrRq1QqA4cOHx947//zzAXjuuecAGDhwIFCxBzYXX3wxED/7f69e\nvQCoVatWBXKcPos0jTEmgpw/CVm2bFls+6STTgKizYun0YdGqUpXKuzatWvsvSOOOKLC+TTR6YO3\nyZMnx9773e9+B8CsWbMAOOuss+L20Sgy0RC5vn37AjBs2DCg8js3m9KOPvro2PY555wDwFNPPQXA\njBkzgGiRpj4wnDBhAgC77bZb7LPBgwenl9k0WaRpjDER5CzS1Hao8KqD4bWuo6pWzav/NerQtrLr\nr7++wsc0mRG+c3j33XeB0hO3PPjggwBcdNFFQFCeYfpZeLZ/kx9++9vfxrZ1sMHbb78NBDO4a7eh\ncBu00hn858yZAwTXr3Yzu+aaa2JpW7ZsmdG8R2WRpjHGRCDpTOjZpk0bl2iNkFSEpwLTYY2LFi1K\neX9dS6Z169ZA9ju8isg851ybrJ4kj2SijAuNlXFm6DBavSa1bXPvvfeOex+CdsqSzyb0mUR4OPQ+\n++yTdt7SKWOLNI0xJoKcPz3X/poQ9Lk0xhQ+bcvWyTaWL18OBM8x+vfvH0sbbrOEYPJofeKeT0Oe\nLdI0xpgI8qf6NsZUSfXr1wegbdu2QOnJwAuNRZrGGBOBVZrGGBOBVZrGGBOBVZrGGBOBVZrGGBOB\nVZrGGBNBWsMoRWQ9sCppwqqlmXNu11xnorJYGVd9VsbRpFVpGmPM9sZuz40xJgKrNI0xJgKrNI0x\nJoKElaaINBSRBf7POhFZG3pdM1uZEpFTRGS5iHwiItemkH5IKG+LROTUZPskOd5bItIqSZqBIrLQ\nP+ebIlKQ04nnsIwbiMgEEflQRJaJSNsk6fuJyHo/X8tE5MI0zz9GRLqlmPZIEdmaavp8Y9dxwjTF\nIjLNv5bfEJE9EqUHvIWsUvkBbgauKeN9AaqlepwUzlMDWAk0A3YAFgH7J9lnCHCVv30QsB7/IVco\nTfUIeXgLaJUkTb3Qdnfg5Uz9DnL1U1ll7B9zLHCBv10TqJ8kfT/gHn97d2ADUJRGGY8BuqWQrjrw\nBjA5lfT5/mPXcak0E4E+/nZn4NFkx63Q7bmI7CsiS0VkLLAE2EtEvg193ltEHvK3d/MjirkiMkdE\n2ic5fHtgmXNulXNuC/AccEaqeXPOLcb7A9jFjyZGicgc4HYRqSsij/n5mC8iXf087igi4/wI5nkg\n6YLKzrnvQy/rAFWqG0I2y1hEGgDtnHOPATjnfnbOfZdq3pxz64DPgKZ+dPKEiLwNPCYi1UVkhJ+P\nhSLSzz9nNRH5lx/ZTgWKUjzdVcAzeJV0lWLXMQAtgWn+9ut4AVBC6bRpHgDc7ZxrCaxNkG4kMMx5\nU8v3ArQQ2onI/WWkbwJ8Hnq9xn8vJSJyFPCTc+6//luNgfbOuUHAYGCyc64tcAJwl4jUAgYCG51z\nLfC+7VqHjvdoeSG+iPxRRFYAQ/EurqomW2X8W2C9X9nNF5HRIrJjqpkSkX3xIpiVoXx2dM79AbgE\n+Nov4yOAASLSFOgJ7I13kfQFjgodb6iInFLGeZoCpwIPppq3ArS9X8cfEFSUPYB6IlI/Ud7SmU9z\nhXMulYVFOgHNJVjHehcRqe2cmw3MTuP8JV0rIhcAPwBnh94f55zb5m93Bk4WEV2ishbQFDgOGAbg\nnJsvIkt0Z+dc3/JO6JwbCYwUkfOAG4CLMvR/yRfZKuPqQBvgCmAe8A/gWuCWJOfpIyLHA1uAfs65\nb/1zvuic+8lP0xloISK9/df1gf3wyvhp/29hjYhM14M6524s53z3AIOcc9skwTrsBW57v47/BNwn\nIhcBM4B1wNZEGUyn0twc2t6GF0qrcFgsQFvn3M8pHnctsFfo9Z4k/gZUw51z9yTJp+C1S60IJ8jA\nBfEUcC9Vr9LMVhmvAVbrxerfSqUSqY91zpWVrmQZ93fOvR5OICJnppi3sDbAOP/vowjoLCJbnXOF\nPYtuvO36OnbOrQXO9PevB/Rwzm1KtE9Guhz53wAbRWQ/EammmfC9BgzQF+Xd6oa8C7QUkWYisgPe\nrcAkf99h2n5RQVPwohvNi4bvM4Fz/PcOBQ5MdiAR2S/0siuwPI185b1MlrFzbg3wlX+bDdARWOrv\ne6WIpLOk6BSgv4hU94/XXERq45Xx2X7bZhOgQ7IDOeeaOueKnXPFwAvAJVWswoyznV7HRRLUtjfg\nNzskksl+mtfh/WfewYsk1ADgaL9RfilwsZ/ZMttCnHO/AH8EpuJdSGOcc1ohHYIXPlfULUAd8boz\nLMF7kghwH9BQRJYBNwHzdYcEbSFXicgSEVmA15ZS7m18FZKRMvZdATwrIgvx/rjv8N9vAXyTRh4f\nAD4GFojIYmAU3h3VeGA13t/Uo8As3aG8Ns3t1PZ2HXcElovIR0ADgr/DchXM2HP/2+BV51yXXOfF\nZI+IvAKc4Zz7Ndd5MZlXFa7jgqk0jTEmH9gwSmOMicAqTWOMicAqTWOMiSCdfpoUFRW54uLiDGWl\nMMybN2+D245m9bYyrvqsjKNJq9IsLi5m7txUBhNUHSKyXS0LYGVc9VkZR2O358YYE4FVmsYYE4FV\nmsYYE4FVmsYYE4FVmsYYE0FaT8+NMSZV8+d782fcdNNNAPz73/+Ofbbjjt4c1DNnzgTgsMMOq+Tc\npc4iTWOMicAiTZNxmzZ5c7h+/rm32sGoUaNKpbnwQm8xyVatkk3LaKqK//u//wNg6tSpQPykwXXr\n1gVgxIgRAIwZM6aSc5c6izSNMSYCizRNxmiEOXz4cABuu+22ctPef783b+3ZZ3vLwNx7770ANGjQ\nIJtZNDkwbZq32OP7778f9/611wZLoeudx3//+1/ynUWaxhgTQV5HmtOnTwdgwoQJAIwfPx6AL7/8\nMpamdWtveZBevXoBcP3112Ny4/bbbwfgjjuSrhjAr796E7OPHTsWgNdf99ZBe+yxxwDo3LlzFnJo\nKtM333irlpx11lkAfPutt6R6167e8kBDhgyJpa1ePa+rojgWaRpjTAR5Vb2vW+ettXTmmd4ieHPm\nzAFAl+TYay9vRdDmzZvH9tEntDfe6C1d3axZMwB+//vfV0KOTdjee+8d91qfjg4cODD23oEHegsE\n/vyztxLs4MGDgaDszzjjDACuu+662D6DBg0Cgr58pjDMmuWtXacRptK7wUKKLsMs0jTGmAhyXtVv\n2LAhtn3KKd4qqgsWLACCqPGBBx4AoF27dgDUr18/to9GmqeffjoA48aNA4KnsvoagvbP/fbzliyv\nyOLypnwTJ06Me63tzPpkvCyHHnooAN27dweCdrBbb701lmbFihUAPPLIIwDUqFEjQzk22TRjxgwg\nuFPs1q0bAO3bt89ZnjLBIk1jjInAKk1jjIkg57fn2hEagtvyJk2aALB8+XIAatasWe7++nBIuyPt\nsMMOQDDeVW5wAAAMj0lEQVQZQFkPhDZv3gxA7dq108q7iffqq68CQbOHPpxL5NhjjwXgxRdfBIKh\ndm+++WYsjXZL0ts87ZZUqA8Sqrqvv/4aKP33cPnll+csT5lkkaYxxkSQs6/qZ555BggG6AM0bNgQ\ngGXLlgGJI8yS9tlnHwCWLl0KwLnnnlsqjTZE16pVqwI5Nsl06tQJCDqq6yQMqTjqqKMAGDZsGBA8\nFATYuHEjAE899RQQPPTTB00mvzzxxBNAcC3Wq1cPCK7vQmeRpjHGRJCzSHPhwoUAbN26NfaednyO\nEqGUtOeee5b72U477QRYV6NsadGiBRBEmmV56KGHgCBqvPTSS8tMd84558S2//nPf8Z99tFHH6WV\nT5NdGmEqHfSQzxMLR2GRpjHGRJCzSFM7LIfpcLl0TJkyBYCffvqp1Gc6cYDJjjZt2sS91ruJcFno\nkEodRqmTskTx8MMPA3DAAQcAcOKJJ8Y+Cw98MLmhT81VVXlqrizSNMaYCCo90vzf//4HlB5yB0H/\nzIrQyOWGG24AYMuWLUDQjglw8MEHV/j4JjntnaBPT0844QQAvvrqq1ga7bmg5VURq1atAoKn5+GJ\nPB588EEgmPjDJvnIHe1XO2nSJAA++eQTIGjzDC+spmn1eYMOodYJXc4777xY2t/85jfZzHZSFmka\nY0wEOWvT1Elo0/XLL78AwZT6JdtKdRp9CL69THZof7w//OEPce+He0Poglk6kYoub/DKK69U+Lx6\n9wLQp08fILir0NFEBx10UIWPbypGo0Zt4yzZ1hnuxaI9ZzQKXb16NQD9+vUD4if2CS+TkQsWaRpj\nTARWaRpjTASVfnuukywUFxcD8Nlnn8U++89//gMEcyyWJ7xG0JNPPgmUvzbQBRdcUMGcmmw47bTT\n4v7VwQ0//PBDXLrwwyO9jWvUqFFcmr/+9a9AMM8mBJOxLFq0CIA///nPANx5552ArbOeC/ow9sgj\njwSChzpFRUWxNMcddxwAM2fOBGD06NFAsD5Y+PrWuiNXXQgt0jTGmAgqPdLUSTj0G6Vly5axz7Rz\nu0acPXr0AILGYY1GdF8IIhLt1KzrkehDH506zuQHbdDXoZA6UcfOO+8cl67k67LojPA6Sz8EHak1\n0pw6dSoA1ap58UHJhxEm8/TuTqN7LR9dgSERHaigs7svXrwYiB86q13OcsUiTWOMiSBnXY50Yg3t\nggIwdOhQIJjwQf/V6FQH/h9//PGxfXRiB20j0/Yv7VjdoEGDrOTfpO6ll16KbV955ZVA0C6tUwRq\nZ/SK0GgV4K233gKCySG0C5qujDh58mQAunTpUuHzmcRKTgGnq8pGoe2gxxxzDJBfk7RYpGmMMRHk\nfL0AnVAW4OSTTwZg3rx5cWk00ixrain9BtJhk6pnz54ZzaepuPCTcY0wtbx0FUqNEPUJa0VphKJT\nz2kU+v333wNBO5tFmtmjQ1d1aKQOZNEy1yVpEpk/fz4QDMHUY+UDizSNMSaCnEeaYbqedZR1kdes\nWVPm+4W+tnJVEp5Q+IsvvgCCnhIaQYQno84EnZZu27Ztce8fcsghGT2PKa1///4AvPfee0AwgcsV\nV1wBBL0eylrYUIdPDhgwAAh6W4SHXO66667ZyHbKLNI0xpgIrNI0xpgI8ur2vCJ0vXNTGC655BIg\n6GT+xhtvAMHQOu1OFh42t//++yc8pt7uQbAGkc7dmE8PELY3utLsa6+9BgQz7qvww1od/qq38PrA\ncI899gCgb9++sbTnn39+lnKcGos0jTEmAknnm7hNmzZu7ty5GcxOarSxGILJPb777jsAOnToAATD\n53SCkEwRkXnOuTbJU1YN2SrjTZs2AcGDmZJdkcKzc+sQyPLonKqJtG3bFgjm7Uy0BreVcWa9//77\nQDCAQR8GhpWcub1Tp04A/O1vfwMyv5JlOmVskaYxxkRQkG2a2l4FQYSp9Nss0xGmySydzX3lypUA\nPP7440AwrFIn3ICyI5Nkjj76aABOOukkAC6++GIgcYRpskOjRB1Oe9NNNwHxawRpW7YOcNHhtjqw\nJZ9YpGmMMREUZDi2fv36Uu/p0C19+mYKiz4R1X/XrVsX+0yHYepKkxqVaDtc+On64YcfDkDTpk2B\n1IbsmcqhE0CHJ3ApRBZpGmNMBAUZaT7//POl3tPVB3O9JrLJjN13373U9rBhw+LSnHLKKZWaJ2PA\nIk1jjImkICNNXTMbgn5drVu3zlV2jDHbEYs0jTEmAqs0jTEmgoK8PbdJGIwxuWKRpjHGRGCVpjHG\nRGCVpjHGRJDW1HAish5YlbnsFIRmzrncLlJSiayMqz4r42jSqjSNMWZ7Y7fnxhgTgVWaxhgTQcJK\nU0QaisgC/2ediKwNvc7a7KAicrWILPF/ks71JiL9RGS9n69lInJhmucfIyLdkqS5PvS7WCIiv4pI\n/XTOmwu5KGMRqSMic/xzLBWRwSnsMySUt0UicmqaeXhLRFolSVMsItNEZKGIvCEie6RzzlzJ4XXc\nQEQmiMiH/nXZNkn6XFzHDURkkl/Gs0WkZdIDO+dS+gFuBq4p430BqqV6nBTO0wr4AKgN1ADeAPZO\nsk8/4B5/e3dgA1BUIk31CHkYA3SLkP5M4D+Z+h3k6qcSy7gaUMffrgHMBdok2WcIcJW/fRCwHr9N\nvoJl/BbQKkmaiUAff7sz8Giuy6hQytg/5ljgAn+7JlA/SfpKv46Bu4Eb/e0DganJjluh23MR2deP\nEMYCS4C9ROTb0Oe9ReQhf3s3/9tmrh9dtE9y+BbAu865H51zvwAz8SqllDjn1gGfAU396OQJEXkb\neExEqovICD8fC0Wkn5/HaiLyL/8bcSpQFOHXAfB74OmI++S1bJaxc26bc26z/7ImXsWZ8hNJ59xi\nvIt8Fz+aGCUic4DbRaSuiDzm52O+iHT187ijiIzzI5jngVopnKolMM3ffh3onmoeC0E2y1hEGgDt\nnHOPATjnfnbOfZdon7BKvI5jZeycWwLsLyIJ10RJp03zAOBu51xLYG2CdCOBYc5b+a0XoIXQTkTu\nLyP9IqCDHzbXAU4G9ko1UyKyL9AMWBnKZ0fn3B+AS4CvnXNtgSOAASLSFOgJ7I33C+wLHBU63lAR\nKXfiRhGpC3QCJqSaxwKSrTJGRGqKyALgK+Bl59y8VDMlIkcBPznn/uu/1Rho75wbBAwGJvtlfAJw\nl4jUAgYCG51zLfCi1tah4z1azq36BwQVZQ+gnhRgE0wS2Srj3wLr/cpuvoiMFpEdU81UJV7HsTIW\nkSOBPf2fcqUz9nyFcy6VdT87Ac3Fn8INLzqo7ZybDcwumdg5t1hERgCvAZuA+cDWFM7TR0SOB7YA\n/Zxz3/rnfNE595OfpjPQQkR6+6/rA/sBxwFPO+e2AWtEZHooPzcmOe8ZwIwo36IFJCtlDF7kAbQS\nkV2AiSLSwjm3LMl5rhWRC4AfgLND74/zyw68Mj5ZRK73X9cCmuKV8TD/3PNFZEkoL33LOd+fgPtE\n5CJgBrCO1P4WC0m2yrg60Aa4ApgH/AO4FrglyXkq+zoeCoz0v8A/8H8SlnE6lebm0PY2vNslFb71\nEaCtf5GkxDk3GhgNICLDgE8S7wHAWOfcVUnyKUB/59zr4QQikvLtfxl6A0+msX8+y1oZK+fcRhGZ\nCZwEJKs0hzvn7kmST8Frx1oRThC62KPkbS1+05CI1AN6OOc2RT5QfstWGa8BVmuF7DeJlHV9llSp\n17Ef7Jzv718Nr0ng00T7ZKTLkV+zbxSR/fwThzP/GjBAX5RzGxRHRBr5/xYDpwPP+K+vFJHL0sjq\nFKC/iFT3j9dcRGrjtZue7beJNAE6pHIwP0o6CijslaJSkMkyFpFGepvr37J1Aj70Xw/TdsgKmoIX\n3ei59DZ8JnCO/96heI3+CYlIkQS17Q34t6RVVSbL2Dm3BvjKv80G6Ags9ffNm+tYRHYWkRr+y0uB\n10Lt7WXKZD/N6/D+M+/gfcuoAcDRfoPtUuBiP7PltncBL/hpXwAuc85977/fAvgmjTw+AHwMLBCR\nxcAovGh7PLAar1AfBWbpDknaNHsArzrnfkwjT4UkU2W8BzBDRD4A5gCvOOcm+58dgncbXFG3AHXE\n65a0BO9pMcB9QEMRWQbchNfsg5/P8to0OwLLReQjoAFwRxr5KhSZvI6vAJ4VkYV4X1L6+8un6/hg\nYKmILMcr76uTnbyghlGKyCvAGc65X3OdF5N5flT3qnOuS67zYrKn0K/jgqo0jTEm12wYpTHGRGCV\npjHGRGCVpjHGRGCVpjHGRGCVpjHGRGCVpjHGRGCVpjHGRPD/2VR0AzzFcB8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024ecaa3c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The weights can also be plotted as shown below. Positive weights are red and negative weights are blue. These weights can be intuitively understood as image-filters.\n",
    "\n",
    "For example, the weights used to determine if an image shows a zero-digit have a positive reaction (red) to an image of a circle, and  have a negative reaction (blue) to images with content in the centre of the circle.\n",
    "\n",
    "Similarly, the weights used to determine if an image shows a one-digit react positively (red) to a vertical line in the centre of the image, and react negatively (blue) to images with content surrounding that line.\n",
    "\n",
    "Note that the weights mostly look like the digits they're supposed to recognize. This is because only one optimization iteration has been performed so the weights are only trained on 100 images. After training on several thousand images, the weights become more difficult to interpret because they have to recognize many variations of how digits can be written."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXuUVdWV7r9ZFiVigYDFq0RSkYelgkFEMQg08UmUThM1\nieloD2+CGXZiunMTY9Lddoa2duJN9MZkmDRJHB06TSfaEjVqjIppub6JoChEy4BaCchDSkFALHmt\n+8dc39lrrzqnKKhzap9dzN8YNdZ57Oc8q/b61lxzzSXOORiGYRg9T03WF2AYhnGwYg9gwzCMjLAH\nsGEYRkbYA9gwDCMj7AFsGIaREfYANgzDyAh7ABuGYWSEPYANwzAywh7AhmEYGVG7PxsPGNDghg5t\nqtClVB9vvtmKrVvbpCfPaTauLAebfQHg1VeXtTnnhvTU+czGXWe/HsBDhzbhO99Zur/nyC1XXz25\nx89pNq4sB5t9AeDCC+VPPXk+s3HX2a8HcDlob+/4Wd++PX0VvRuzcWUx+1aeg8XG5gM2DMPIiB5X\nwIcckrzes0fLXbuKb8vvQ0q1gjxGsX14zj59unaNecdsXFnMvpXnYLGxKWDDMIyMsAewYRhGRvS4\nCyKU/nS0x92F3bu1rC1ydXE3hMcr5rSP94+7Hb3RqQ+YjSvNYYclr5ubtaRtWlu13LJFS7PvgXGw\n1GFTwIZhGBlRMQVcX6/l9u1askXi+/g1kKiJk0/W8tVXS+8zfHj6PCeO3wsA2NGetCm3367l+vVa\nUrlwH5JXFUGbctCgMxvHtp42TUvaMVQGk/q+pC9efx0AsO6k8wEAK1fqx6G9mprSx21p0XLt2vTn\nebRxKfuG9zxu5A598cwzAIATWbemTdRy9WotA5m1dcwkAMDLL+v7t97SkjY6/PDk+FRjtCt/x95e\nh8P6OGaMljQhexe0ycCBWoZ2Izwu//fb2tLHAICRI9Pn4XcbNqSPVQkbmwI2DMPIiIop4Fj5xp+H\nr9mCscX5wx+0ZIvE74FENcR+miXPalvy/vsdr2XJEi2psKn6SBjykqcwn1g18N7DlptKNG75Dz1U\ny1NO0fKMpteSnW6Zp6WXvI3zJ+hbjEodAwCeeCJ9fKqE2NeWRxvH9i0KKzGVLg1Aw9NRGVTiAd6u\nU448EgCwdbr2MJYv1+/ffTc5PFXfZD9hMFbNJI/2BUrX4cnBBEnagKalYj3rLC3r2rfqC3bRgKQC\nvvde+oRXnA0AuOPuusJH7GnHPuW411gJG5sCNgzDyIiKKWC2aEN8egq2XmHLwZbGu88KooEKdfZs\nLamEw/3feUfLxYu1jEehAWCid8MNG6YlWzS2dNynmO8oD8SqgfdO31j4muIgbuVpzzPGBFXh2GPT\nJ/KG4/FDhc3jU5XwuA0NWubZxrF9WS8nNe9INuqrN7p37ucBJL2Dwa3P6QtWUMpbIKmQngGP3AUA\naJp8AQDgkUeS73g81uWPflRL9jz4m+TRvkBHG/N/fnD9zmQj3iwrcSknLbu4QPLA2bRJS8rcH/8Y\nAPCpW24pbPrwoGMAJM+heJypknXYFLBhGEZGVEwBU1lRCbGlDv0q9BuyhaHSpW+H+z7+eLIPlS9H\ngbkvVQOVWHiuo4/WkmqNDSiPH/rTvFsuF360OLaR0R6hK4z3vGuX/xJaLlr0AQDA8OF6w0+dNqqw\nz9Tp0/UFh/u9oSgqlgaJrmhL/r7xiHWebUz70gysWzvQr7DNime1vOMOLalUJ0/WSIfjrxwPANi4\nOfE5vvFG+jyTJmoED3wPMFRaFM60axwFwR5NHu0LJDZmPRlc6/25i4J/eg4KUQmzcn3uc1qyvgaK\n+LXacanjDnjwv/UF/1mCf5LmiWkFHEdbVLIOmwI2DMPIiG4p4GKzSgiVQOyuCRUw1RlbGCoNlmyk\nHnoo2YeK4IortOTg8syZWobxkXwdj55SYfDzML6PLXK1qIfObExb0rZ0c7FnAQC7CkP4DMzdnDrG\nyJHalE+dHPjcfvp4apuXPqij9Py9lhZJ9UrVkDcbd2Zf1kve2z33dNyHqum++9TO/fvrTV18sX7e\n3KzKN/TL/+d/8jga7nD22Wqsyy5LnxdIxkUYZDFeBXWHul2t9gU6tzG/K0QmsXINGpRsRN/uJZcA\nAB5e2QggeT6M9/e5e/eAwi58LtAFPOWii0pew3Dfi2BkEOH7StrYFLBhGEZG2APYMAwjI8o2CMfB\nALJtW/pzuhXuvz/Zht0OlqedpiW71Ox2LVwYHvdNAMC11w4FkEyp5SDFmWcm23LwjW4LdiV5TUcc\nsY+bqjJiG9PVwMG3YvfF7wC6GBjlr921v/or/3bOnGQnjmiOHg0AaP/wFwEAt92mH2/atDq4Cu0+\nDxo0InUN7EHmycaxffmeA4ysR2FE2apVfKX2bWjok9qnWGggbbN5s9qOPWz+j4Rdds4j4GfsUjO8\nM0/2BTramPdTmDDU9wwAwIagivHeF1+Vfs9j0bbhRAnahy6xd9+tSe0TTsaiu6J/fy050M9tKmlj\nU8CGYRgZ0S0FHLZm8eQGTqfkYAyVQDjAwNdUsWytGG1y001abtv2enBWDVNZv14Txtx5p3rDBw1S\n6RtOYZxy2IsAgMbx2sQ9N1xDrThwQqplsKIYndmYg3C8fg7MhFNZAY56Moci4/R0ssWUV34OAPj9\nb39b2INa+VQvLXieTZsop9MDeSEcqIiVTrXauDP70p7siXFAOZyKzV7W6tVa0ama2DsJJwYR1nNu\ne+WVWlJpLVuWbMvQJ6o0vg/HqIDqtS/QuY2pNtnpikNTgcTeHHSj/eIeQzhIdr6OG+Okk9Ln5XnC\nwXr2Evlb0dY9UYdNARuGYWREtxRwGLYR+1fZijAKiq19OFEiTinJCBT6fBP/ZRJeksBVoNV5tHmz\n+oTXrp2QbDLLO4a9kps0XC9yaa2GsXD6Y5jYp9rS+nVmYyomKgB+nw774RvaUJ2Kzc3+ve8OtBTZ\nY5qXGDfcwG/YdfhAsLVKsTBhEpCoFv621WrjzuxbKrl6aN81a7SCNzf3SX0Xh+WF+3CM46tfTV8L\n/ZbhDFumZuVncbKj3laHWYY24LYXXqgle3iMLGMPevDAvclOdAwv1udD7VnnAEh+y2KhcXE+pTjk\nrxI2NgVsGIaREd1SwJ35RNhCsMVj6x5C1cTWiMo3DlSnalOofF/xJX2c2kylEoTHmWj8UOvMmR8D\nUHxCQbXRmY2ZZDpWFZs2hbk62dTzQDpUzDSUeP55AECQjBIFMfu73wEAFo1e4D/g75Ak6+nTRydy\nxMl4OlsupprozL689s7tq3Vq48YPpralPfh/EEZBcNIQR9153KiaAkh8pCM0yKTwv8LjxQnxq5ED\nqcNhlAK/oy2ofD82eZ2+aI26B+Fr/5CpeeIxAMCxx84AkP49+Nyh/5k27Yk6bArYMAwjIyr2bGfr\ncWKzH1P3zcve4Y2FbejnYWvEGOH29ni0/YXgyAzC5HxbOpVVCYcxmh9r8HKBUts7gMY16TW1ttaF\nl5Y74lHaxG8WhkFQtVK1qe+3kLmvVeVJXbCHd33hm/OP8a/u9SUjKRKffJSvp4NvrNSCinlgf+y7\neTOjcbSHQf/x5s1qgD59EgNQ3dEm9Esy6VTYMzvqKC1ZhamOY19wHu0LlLZxMTHLshDB4CvdxkP0\nmfKb3yT7cCp4vyh0ZcppOo/gB0uGFrZlz4NzCXqyDpsCNgzDyIiyKWD6eegvoQI4caT30fpmpiaQ\nm204EQBw4436/vXX/WKQYN43vg/H6OkgYzO4159fox/CZD9/HjkVADCqXuOBC/LYb9S37+B931gV\nEds4HpVP1EToo6RaO9wfQyMYCrGsfipSkPMex/vy7Otpfyb0oe830cucfURVEi8WGSucaqZ79tXy\nvfcGpI7FbUOf5gknaMkY+XnpFaBSdZg9jHiFo1Lx1tVOV2387LPJPmPHaskY/w99yH/h/59f3X0q\ngHTMNWdtzpqlaUHHjVGDbXSqfEPfOWfgxvHfpJI2NgVsGIaREfYANgzDyIiyuSDiAPRCF7dURhMA\nq8eoCyJJTs+QKZ8VHz7MBG+GZ/Kl75dA46nYPQkDrDm4N+q0pvS1+I0YKpRM+KhuYhuzq8TJLknX\nNexDcZCyPbUPQ6Hav6tGCudRNJ6tK8diEQ3Dbzn/Ncl6wqD4uAvJ4Htea5gopVrZl33JiBHJPOD6\neq1/7NIyeJ/lCSeon2HKcVsL+yx5Wd0UP/yhvl+wIH1enwMJQDItnwNEdGXk0b5A1+twuLo5vZbM\nMc4VRRa/rq4HLpgRTgaiy4ZpB1qHq2PtnIH6TJk1KwkGiNfX68k6bArYMAwjI8oehtZhuibjkyhz\ng1iOC67SAbSLL2Y7wMWwGObEKa9h00MlQU+8Zt2gYz4MFeE17Oyrx6uLouNrczaAQeKgdUJTb9mS\nRL5v2aIR/Lx1KrPzmnXqBYc3U4fiqNBo2p2/C8PSEiNzIInXQiVTzclh9kUceD/Bz27nZCKmUASA\n447TkrZnT2zccF9Pacs/NRX22TPyk6njnHuulhs3ahkOJnEB5d5kX6BjHaYdqW7DhEfxMm7cl8qY\nU4hZAkmYJdduK6hjn42nLpDLzc06MYuquSdtbArYMAwjI8qmgEv5R3a06zO+H507RZq2XbuYrYPK\n993ofZh7j6FQ6hCaPl1PzCmN4eF5yro29ftwEkiND13b7VvQvKiJUjamMotTIQJJj4ChNvQlYv58\nAEkLPDE43sbDj0nt297OCRhqqD59kmV795UWkQonDzamfTlFmLYKpwYD6ZSn9GFSARfse4/Pe0il\nFSS8f8SHXbLOMpkMk7sXm4RAYjtSDebBvkDpOkwfOldED1m0SMuf/jQ9qYUqt1hI3uLFWlIVX3qp\n/+Jna7TkihEAmo5Ln68n67ApYMMwjIwouw84HhWnehhHeRaGKfim5ROf6AcAuPNOZmvnNsOiEkj8\nj6rCYh9kePi/vshPg25XmVKzW9//eUNdap+8Eds4nqIZjgbzOyq0L37RfzHrPv3ev22k7APwovdF\n0veZBMUfkjpPZ+eu9iQ8nVFQsR7eS7HEN/TjcoS+5tYf6As6H329v+v+ZPJKYfkd31NhRMqsWVqG\nvTgmiuHvR19+HBURTsGvZljN2COjbcP0k0AylTjcZvFirWysY9wnSY6UpKMUUblKmzY2+GcBc0qG\nuSU9TI3Zk3XYFLBhGEZGlO0Zz7i9eKokE4uMY5O3Zk2yk5cAM2eqr3fDBm21Hn+ciTLoA96Z7FNI\nnKij+2xRqUA4zTO8mK3+OANqtYUME0TniVI2pl+NPu8inYxCqz7sLT+92EumQn78yy4r7BMv/fLs\ns0zoow7fUAHHvYg4cUqelDDtG0+TjTtvoUJlwvTBN16tLxjCQMeuZ9bHk9dUvvyXmDTe1+9bbulw\ngqlzvITzjufXNvRLHbdYYvFqhj7eug1/BgBsPFSXCdvs825NmaJlGAlCXy97DrznZBvOE0iSJM2c\nqV2EguudtmXQb+DIr/XPDvp4e7IOmwI2DMPICHsAG4ZhZETZxHWpAa1CSNRZvr8VjBDtHHlM6iN2\nT5qbdWSttVXLcIChvV1dD+wxsxvHCQbhAAq7axyoa2vT9oZdi7y5IkrZmOMJcdcJSKJtCtm46D/w\nH9R85CMAgK03/KCwz8ob05sy+1nfvtpHOzyJQiu8Zjctzy6I2L6j+mrXdu9IdYnVtPmubt/AB3Gz\nT7vFOa/sL7NSf1BXygjrMEOkCiFXnDMfJ8gGkn8gPwp3jD/+c8trUpvmJR9wXesf9YWvXC/4VN+8\n/lf8Qjeha4WTrGg3hus5x9XS1X8xffqkwj7XXqvljIE+E+KtfmdWSG4A4OnvaUkXnrkgDMMwDgLK\n9myPpxayNWH5L/NURVx1VZKJvt/uHQCA+npVqsf6ORZUplRgs2cn5+FnHCBi6zjsXT84tyWRAk1N\nOvEiXkGWhI7+PFDKxoRhUuF9caJEoePBnS+/XMurrgKQXoUhXpfrlFO010E7hrltOQjKc8chPHlJ\nEgMUUTreEDWtvm69/HLHnahaWTE5+OYXLrvjPq3bN99c+nwnnKBJqaZQlYVy2R/3tb6aTGaL/4rK\nN0/2BZBUHj/QeM40fQbsgNqJgj/sBHBwjD0GlmPGaO9i4kQtTxweJO2iDW/ymY5YqX19/8n8JCyQ\nYZ1Z1GFTwIZhGBlRNgUc+1OpztjwMOUjV78AgDlztNVjS0Nf5tFHa8kA9XC6K0Ok2JAO2+h9PN5h\ns7U+STO31reiceo7RsIx8UdeKGVj3hdb6pEjk23YqlNZ0O9ed8UVAIC99RqiF/rcGNLHaCj+DlTW\nYRhavCZcrBryMkUWSOzLOtY0209dX+nrGJcmDrsec+dqSeXrK/wvFqrCuuYa/ThMeUqb0eYUvpdd\npr/F2LEzCtvSD7l2dXqfPNoXAB5+Ru/xnGm+onjfeb/3NNRx1PTpWo4J5tP7nscF0/zNUt2yt/u4\nN8ZxwZziOJbyU58CANy1/RwASdgbkNTZLOqwKWDDMIyMKPuacIR+FKZ2o//1zjuTbejnYeQCS7by\nPl9MatYgpxZSWb9Yq/6zNgZpBynpwim5QJKYI2/B66SUjTkdlkl5CiseI5mYwtHmu+/W8uMf9/5x\nL+bChDNUABQPVL78fUIFHE9BzqsyA5Jr9mIM99yj5eTJWsdGDdQUk0xvCiR16RG/7a23qvJ99FFK\nLErfDxT2efVVDR2hYGP5zjvpY4ZQdOc9HSXvY8kK7f1OYWXlPycrG6NKgGSAgl0yVjZ2gzmpgl03\nAOuazwCQ2LIwSckLYyZCCl9nUYdNARuGYWRE2SPc4ul8VGVUsaF6opuGSjjOWEklxrwmQNJQMmKC\n/jS658JYTsbA5lUtlKKUjTkgH47mv/qqllQCy5ZpSbUVT+8EEiHBQX/av5iNGQfcm2zMeymoW59Z\ncsMGTplPtqUIi5PJDPIDF7W1WoZ1mHZksnWW/D3DCIDYvr3Fzhx/uXej9sQaGrSc7AVxXVghfRjU\nTr8Nbc1xDdbh9xYlu5RK0cq6Gz6HsqzDpoANwzAyouwKmD6q2I/Fligcoaf7J0xuEu5LRUyfHNDR\nX8MIiTzNuOou+7JxaIs4VphQNRSzMV1usavtYLHx/tTh2L7chkERVoeLQxvzfz+O/62tPbWw7fbI\nBUzi50Q45sP4dO7DbarNxqaADcMwMsIewIZhGBlRMUEeJwfpbDofHeJxd67YIFqpYOlicOCvtwxc\nxJiNK4vZt/JUysZ0adAFweMWO36WNjYFbBiGkREVC0PrSmsSb8MWjg76MHSn1D77+31vwGxcWcy+\nlcdsrJgCNgzDyAhxznV9Y5FNAP5UucupOj7gnBvSkyc0G1eWg9C+gNm4JzggG+/XA9gwDMMoH+aC\nMAzDyAh7ABuGYWTEAT+AReR7IvLl4P1DInJb8P5mEfnKPo7xVBfO0yoiDUU+nykiU/f3uoP9TxaR\nFSKyWkR+ICJyoMeqFL3Axv8qImtEpMSE6OzJs41FpJ+I/EZEWkTkDyJy47736nnybGO//4Mi8oK3\n8TwRKdsiRd1RwE8CmAoAIlIDoAHACcH3UwF0ajTn3AEbBcBMnv8A+TcAlwMY6/9mdeNYlSLvNr4P\nwKn73Cpb8m7jm5xzzQBOAnC6iHy0G8eqFHm38Sedcx8CMB7AEACf6Max0jjnDugPQCOANf71BAD/\nAeBhAIMAHApgC4A6//3XADwL4EUA1wXH2O7LGgA/AtACYBGABwBc5L9rBXAdgOcArADQDKAJwAYA\nbwBYDmC6N8pKAC8AeGwf1z4CQEvw/tMAfnygtqjUX55tHN3H9qxt2dtt7M/xfQCXZ23T3mpjAH2g\nouJT5bLNAU/EcM6tE5HdIjIK2ro8DeAoAB8G8A6AFc65nSJyDlRhngpAANwrIjOcc48Fh7vAG+p4\nAEMBvAzg34Pv25xzk0TkCwCucs7NFZF5/ke5CQBEZAWAc51zb4jIQP9ZI4DbnHPnRZd/FIBgDQis\n9Z9VFTm3cS7oLTb22/4l9CFcVfQGG4vIQ/66fgtgYRnMAqD7g3BPQQ1Koz4dvH/Sb3OO/3se2jI1\nQ40cMg3Anc65vc65DQAejb6/y5fLoMYvxpMA5ovI5QAOAfSHz+uDIcBsXHlybWMRqQXwSwA/cM69\n1umdZkeubeycOxfacz4UwBmd3ej+0N2pyPTtTIBK+jUAvgpgK4Cf+W0EwLedcz/uxnn8inLYgxLX\n7Jy7QkSmADgfwDIROdk591aJ470BIMjqipH+s2okrzbOE3m38U8ArHLO3dKNa6s0ebcxnHPtIvJr\nAH8FdX90m3Io4NkA3nbO7XHOvQ1gILRrQaf6QwA+KyL1ACAiR4nI0Og4TwK4UERqRGQY1Gm+L7YB\n6M83IjLaObfEOfdNAJsAHF1qR+fcegBbReQ0H/3wNwB+3YVzZkEubZwzcmtjEbkBwBEAvtzZdlVA\nLm0sIvUiMsK/roU+tFtKbb+/dPcBvAI6ovlM9Nk7zrk2AHDOPQzgFwCe9r6XhQiM4fkV1A/7EoAF\n0O7HO/s4930APi4iy0VkOoDvioaVrYT+oC+ISKOIPFBi/y8AuA3AagCvQn071UhubSwi3xGRtQD6\nichaEbm2y3fds+TSxiIyEsA/Qf2hz/ljzN2fG+9BcmljAIdDfdEvQgfx3gQwr6s3vS+qZiqyiNQ7\n57aLyJEAfg/gdO/jMcqE2bjymI0rT2+ycTWtkHS/H5GsA3B9Xg1a5ZiNK4/ZuPL0GhtXjQI2DMM4\n2LBcEIZhGBlhD2DDMIyMsAewYRhGRuzXINyAAQ1u6NCmCl1K9fHmm63YurWtR7OkmY0ry8FmXwB4\n9dVlba4HV8QwG3ed/XoADx3ahO98Z+n+niNFe3vHz+KlqauFq6+e3OPnNBtXlnLY9/TTk9fv+3lX\nS7t3yIpy4YXSo8sDHWx1GDhwG5sLwjAMIyN6PA74kCCV8Z49WnJ56Rh+H1KqFeQxiu3Dc1bLUtSV\nxmxcWd4IsoYc5XPoDR+uZWy7gQM7fv7MMygKj9XU1PG7JUu03L17vy41txwsddgUsGEYRkbYA9gw\nDCMjetwFEUp/Otrj7gK7WbVFri7uhvB4PFZDsCLUxIlatrVp2RLlMKpmp353qLSNQ+L9465db7Ux\n4SAcoRth2KCdAICNm+s67MNt3n1Xy8l+HHLA2pf0xWFJptQXWwek9uW22/0qe3Gd7i0cLHXYFLBh\nGEZGVEwBswWiQ5vvtwfr4/I1S7ZobKXYMhVr4bgN92ErxcEQANiyRUsqYH43Zkz6+9Wr930/1Uhs\nY9ogHMThPXIwaMOG9Oe0fX19ss+IEVquXKllrBqOChZvis9NeD6ep5jyqHZi+xLeG5DcH++fqnZH\nf1W+r7yi78PBM6rW007TcsCGP+oL331b8nKiejngx9+Uv9e2bVqWUod5oaefE9wmtBd/T577sMO0\nDP8n4n3KhSlgwzCMjKiYAo5bNPrKNgSJ49b6ZTHZsm3erCVbfe4bKrrWVi1bWtb7T1Q1fOQjekKq\nWyBp2djqsRXk+eKWL7zuPBDbmITvz5imvshCN2CMb9YpDbwRHnsm8VVS+dJeZ52l5YlNWwEAr7Ul\nCu2Y2j/rCxqZssHLBfpAX3ghuaZQ3VQzsX0HDdIyrMOxAua93XOPlqtWafmHPyT7sN5RxbadNA5A\nooxDtczj8ZzsxfHnpNnDsK081mHahP/r4VgOYfgenxN33KHlyy9r+U6Qlp320sVvAD4nxo7t0+H4\ntCE/a27WMuxNA5WxsSlgwzCMjKi4D5jKN1ZTAPD441rS5/j001omKneXL8N24jlf0nE7HQBw2WU6\ncvw3u4MVqsfMBgC8tl2XlRrpB5djRZwnxRAS23jjRi2pygDgjTdUgZ51ViOARKDWRI60pqZEAY8M\nlytFok7WbVflm1IGW7wxKdUozbwkHOZPWF/fWNglLwo4ti9vjT03IFG+tDnL++9PbxvuM3Omlk/6\ntYB/7Jeg5G8T9mCoCPv7hXloe37O8x9+eBduqAphj5XqM7YnAIyqXQcAmDZN6xB7ClOmaHn33TTY\nquDINDjX2tQDr1o1IlUCQJ8+HwCQRE2x/se+5krY2BSwYRhGRlRMAcdxd2yxR49OtmFLxpaffrKW\nljf9Fq/5ckVw5K2+pB9SW7Jp0/zbhW3JprfeCgA45sor9f12vd29AwcDSKIfQn/Qeu8yyoMqjm1M\nv1exWOjCSDudkmefDQD4xUJVvmvWJPu85UUDfzMmmjn2WC2p4ADgpJNUldRteSl9EdHQ9dRAVv/P\nwDMApFVONRLbl4qIPTQAOPlkLVf4KkpVy7q9axdztBxW2OfRR1l3WZcZeEon487Cti0t+mNOmKAK\nLlaI/K3fChZVP/JILfNQh3k/cZRMqhe2VNVsP98FmfSyGnvSxmUAgK9/2tc5VnYAO668GgCwYIG+\nZ++Fxw3rHsc8SqnwStrYFLBhGEZGdEsBdxbbGY/eHnGEloclQqCgsDZt0pIt0aBB6rPdvJnNSzg1\nhSpBVcTXvqbvj/ntD9MnBJJmik5MSl6vgHn9oaKhf7ha1MP+2Jjv//7vk22GvfKYvmjzB/LOxKeW\nqvL93Od4ns2FfU45RYf7aS7ahL9PmIyG0Q1jxhwPAJg4WcvBeDt9UUFQ5UivRqpBAXfFvps3pz8P\n401pC/be2MFIIhm0no4dO6jD8VevVqnlHLPzUGId22FbKrjYH0m1FsaoUrlXSx0ePz55zecAYdw0\nnwU1C/9bX4TduNtv1/KnPwUA+L4c2Nft58v6X/6ysEvN178OAPg8jTBbx4NwyS16jPZRhW15Kl4n\n/5+ef17LStrYFLBhGEZG2APYMAwjI8o2CBfnKWWXiYNaHLgZUBh4APr21e4Zndt0kH/mM9xCu22/\n+12yBAFD18jcuf7FDT5hKrOVAIWYt63t2t0e4GN42JWm6+HQQ0vdVXVRysbsEjOkZ1j/HclGHB06\n7jgAwGOYAQBYMF8/bm/XH2j69CQsh6GCHHxbtEjLZ5/d5fdJ+l10I7DbRts2Nambp75+cFdurSoo\nZd94Guu+lnQeAAAgAElEQVSHPpRsQ9cDXQJ0sbW3q62uuELrMMPIAGDxYi2TSQLkWL9tMpuI3eN4\nIJCTQuIufTUSehMG9/V1kze03YeLPRPNYAlnXzGGz/f3m3ycXhLYqLwUvOZ6HEP9tmfdfTcAYKB/\nBtRe9aMO13dig4a7NTc3pi7xvfdK31t3MQVsGIaREd1SwKFiiIOWOYAxxC9TV2jQgkGyyZNVAVOJ\nfuMbWv71SD9w5L3ep5xyaofzMMxnXL22WrjvPi2DUJSdfmDoZe9Mb21VNUYHeh5WceiKjZnUhVMo\nU0sucJTsE58AACy4Rt+yt/HQQ6p8w4GScFAyPM+jj2p7vWLF64XvVqzQgc4hQ/S3pHqmWqz2kKiu\n2DceQC52L/yM1e/ii9NT48OfhL2GPn3U9occMiJ1/DDMj78LVdqwYelriadAVyNh8iKs9mqWF87n\nASsMZ5qEI7Q0gt+mzg+o1YW9XQCnLl+evPbxZy2PPgoAKPzM/nzhWH0BXwHqdqtK375dh/cq+Zww\nBWwYhpER3VLAoe80nt5L1cBpxqNG7tUXy5Omum7RbwAAc+acDwDo9+Bd+sV8ncf59k06rXjlgo7n\nLii2K67Qki1q0NzyWjhFl+tqUU1QBYYJmKstrV9XbMxrHtfkA/jnP5Ls5G923W4N7WPP4bLLtJy6\nW3sbOwbOKOzSFoWJ8bxjx6oUWLXqyOAK1ccWp+7jvvy8Wm3cFfuOHatlsYkuVLjsNdCuVMIUZaEC\n5v1TwPF8FH+zZiXbMnqqsWFn+kS+8q7boiotFIzVlqT9iSeS18OHa+KhDa36vrZew8For37F8sry\nh/EVc8muSQCAxX5s4jf6GEF7+6TCLjfd9FkAwIy+56X2xcUXA0jX18Ijw5/nsaVqU05O4jOsEnXY\nFLBhGEZGdEsBd+YT4ahvQamyBeIoJ1AYMu/HoWQ/Ukl/JVvyMJEJFcdPvuzHPC/zsuSkkwAA62Z9\ntrDtllYtH/GCkElOOlvKpNrozMaxH7AgNcKsIf6HoKCgn33qRD8avbAVAFB7WqKAOUrP+Hfaa84c\nlkk6yoULtSyV6KjabdwV+0YZNlOBNlS4VK30gVPxxtPtgSTKh9OWqaipxMKJHwUhuHxl+iL8/87A\n8To+EqZKrDb4rw8kUVFMHUl/O6Nn5s6dCiDt06aNV69Wtcz/ZyY82rbNz+ZAYoQFC9ROM/gAYviU\nr6jFFOzbfTX6gUnBmISpknXYFLBhGEZGVFyfFJavoSS4/vrCdz6VN0axKfMZdbbO/QoA4P4b9eOw\nBaUKKzjbOI/Zy4dwVJv7UbHQ38fpo3lfzoX3yhZ7xse9DL300mQj77isadXERue1eZU890Et/ZB7\nOCWX/sT16/16OlCFsX27+th8jiMAibKjSoltmWcb077sgcUJuoEkRtS7Fjv02niMMLkMfYm0Sbxo\nLKs2ANTs9r5fhhFFF9WvTf+Lth2aTK3NE7wd2uDZZ7UME9/QllS+fL9tG5N2scvwbmGf++/3/uBv\neMNzzrPvXow7bW9h26eeUR3KHh+Pz9+sknXYFLBhGEZGlE0B05fGVoNqYVgfn5TF+ydXBvtwvtZu\n75w9xjeDA27/CQDgWxd56RrE9+F2bQZ3sqkkfnR41G3fLHxU/+V/AQDMmKatHVs6+vaKxgJWMbRx\nvAghB8bveF5HmMPR2qYmVUZTB/rfwXch/jzzbwAkfrSlX072oblHjNCZWevXqySgb7JudTLn6OKL\nj09dQxwNEc8uq2ZK2Zf3wPGHfn0T9fSlL2mdYkeMbnjOImRsdhhtESYzAhJ7M547nN22EzqLs609\nPaOwMVLE75e+raqilI1Zb1iPQhvFyXCefprdNfbQNkclAHgF7Ac01vkZoY0PPQQA+POE8wtb0v9c\nbIFaoLJ12BSwYRhGRtgD2DAMIyPK5oLgQFeHkA16172Hey86wjG2et9NGPq97+kHp5yi5apgradl\ny1L7cO2ApgkT9EWwLPLg1b/XF36AburkJgDA/721Lvy4qkN4QjgYEHfbGF7H8DFOOAGSLnDLWekc\nyA/6MTh2f9esiZLeAvjIRzTjy0jvV7r55uhCADQ0aUlbsrvGLnexVXurlVL2Zd0uhIcxUTKAfn7E\nZkOTrvIRT4Jg9Q9DyzgexPPR9cDJCJw4BAB1LS8CABp9vd7B7Lf16uPbW++n8weTHaqZUjbm57RN\nOB2+4zpx/K/nhCCOcCa+N7oVdhypoa30Njb6E43q+2Zh25kzh6bOw9+M/1eVrMOmgA3DMDKibAo4\nTuk4bIjXur/0g2V+8OfEUCJw5IKeeQ4scHox319zTbKPV8C88Ebu62cE/BHjCpuydS041duj9zkl\nHhwiTFFYSMqDZECHA44M5eEg0bZtTKzzWrITdGbA6tV6QCqzwpTNpcnMmMHemM3NaneuIlCtyXe6\nQqnJJJySivm3Jxt7Y6/228aTNajkwum4jz9OpaapWblaBidzfHb875ONW/WH2zv+RADANv8vs2WP\nKt/dXhXmJaUqiW3MTgVVbjjuzm35OJg1S+/9pJN0AJj1PUwoNfg0rY8MdeWqGXyWLHl9aGFb1tWL\nLtJy2zYt/+u/9uOGDhBTwIZhGBlRNgXMrIdUAH36+Gf7R3Xe5cBPaxlOMaQSrVnqW3wqYvpxiy3G\n5JvBofzMS4u/u2ZAh+MzFI6hcbG/KbnWfd9fNVDKB8WUn1S5oRLgPgy9o39r27bn/BZUvuHBVZFR\njXAyTeNAHzgYzirwEm/kSFUcK8M4QyQKJw82LmVf2qEwvHDkkR2+nOknCJ0zTW30XEs6SU6YjAeg\n7XXK+OTJau/PTvSf33RLsqmf4VHTpj7Lt95S5Ua7sk6HfuNqppSN4zC0sHfHFJzchh3kM3Y/nD7I\nyiSu9I9+3IihrhOnT08dJKyPVNic1NLP99JnzToHQPJ/VYk1DE0BG4ZhZETZFDBHeTmKSfVJRcpW\nJmxFuIjpiBGnprYZ+CctvzDtRf9BkNGZc5F9Vplv3arKl8vmvB9EpNNnRMHG92E6wTwSj6LHHYVQ\nZbClpypmsu8VK3zUCIqtt6IHiJN83/uIqrqPzU6S3nMmx3mzdGS6pUUjTKo9CU9nlLIvew+NVFNA\nwVlZtzzw2wJoGqN1et48fT96dPJd//4a3cOq/I9z/Yj8JT5TUlhB/fT8lzYMTl0DN+H/U54mvACl\nbRwnPgKSiUX8rtAT2e7/selA5gAHuBY1UJig7Y290y+RxYihkGGf0a3fb9Zygz8spyZXYuzIFLBh\nGEZGlE2nUHmyJeYoO1syZqEMZxDHSwOdfbaWhYQ7d3dcZmjjuTqF9ld+IJqN37tJHo4CsSqIlWLe\nVFpsY94HVS2VfjjiTt9jfM+nnKLS+NlnT/Zb7kx28knW2WOg/5i2fm550m5P4oG9TJg8+RgASTxs\nnmxcyr7xsj87m08s7FNHA3MnXw4eqFFAl1yitirUaSSj7AxzLwRlcxieJYAHnhmc2oc9mjhRUF4i\ne0rZmM8A2jhMwEUY7cDvtvfVKIi1vjxnTvIPP5xhFL6r8OJZmuBr4Q368YJgkQf+3zDu169f22G1\npEpgCtgwDCMj7AFsGIaREWUT13F3n851hpVQzodZjjhQR8l/1VVajmrwwSN/+ZcAgCXvJV2+X39f\nS3ZxOT7H84URQlwYgl2IvLsgSg20TB2/NfV+9OhkxQoO0vB3oL04/XX8eDXGE08kox7clgM99ADR\nJTFp+LrkZEtbUzsNPO3zAPJp41L2jbPmhdNkz7nkEn1xg+/bslJ7f80MP7tiZ0Njh+PRbbB1jrrV\naLP585Pj05VH2A3nb5Mn+wKlbUx3JOvr668n091bW9UwrLM8BgfSaINz/mJ9ckAa86tfBZBk/aOL\nIxzn5HMidmP2RB02BWwYhpERZXu2x1ML4xAmOrrDSQIcQCgkjPGqtq1Bw5369lXl61N4AugYPsLj\nMjQlHIzgunRUblR/eUoQE1LKxm/vVsU7eLeGMzU1JQqYyuJkP9bGATUGt3NtrlCZMJqHtr1gjp9W\nTjn2s0XJxly6YO5cAEBbNHCRJxuXsi8HjuMBJN1H62rT3G8BAI4ZqYOZTy3VcLzF83W7cCIG6/mZ\nZ2rJHiAHl8JtWWf5f9Nb6zBJJu4kPTLeOwfJWA3vvlt7fmef7es7DQkUDLV1uub9Xf4f+jHrNP8f\ngKQKR4svFw3rLDemgA3DMDKiYsl42LKx9ThWF1dI+c/ibRmNw7AqKuQg+1/B10tVS/Uc+9XCbWJ/\nGa8pD9NjQ0rZmHYbPlynqfK+AWDqZB9e9tOfasmMMrVquGGxBAEw/BKdRDB4i5+mfFuUwSecGDN7\nNgBgb7OGAm3SjKK5tHEp+1L5cu29sBd2/fUqn4YM0cGHTZuY3IiVlhpnV3BkVWyrVmm3rblZZRkz\nqoYzvQ+WOkzlS38s19gDknvnc4D1nekn6RsulocgVtysukHW2g6TW3rSxqaADcMwMqLsa8IRtjRs\nlKhQwwDrNWtUFdx/v+5MX0+c0i9srdhCjh2rZTwyWizZT5x4PW+qgZSy8Zo1WtK27DkAwOHnqi9y\nGGe50MHIkl0SOoWRpJgsDEnzRMyXeNZZhW03vqdq7oVkFmjRa80DpexL3yM7D0z7CQDr12uF3ETp\nDxqfU7ypfEckO/lE4v37q9TlhIw4aVT4urfXYapOrgg9IjAXlW+8wvS0afpDfPGLfsMfBzks/T8B\n94kT+gRzuwoJpPhdT9rYFLBhGEZGlD3Cja1GmEwDSFoX+ncBoLlZN2aLxlYpEFipfYFE3VGMNTao\nj3PLFlV6oY+ZajmvaqEUsY2ZjpLZPEMfMNMUvluv6SIb5mgZ90zCVZ/2eCHBbeLfcvfC5HVs495g\n61L2ZURJMFMYK1fqRm1t6sDdvl1Ldh6o7ML4dKo7jovw9+LnYZRFb6/D9Luyl8teL6deA0lodbwN\n9x228If64mc/S3by0npAX30+TJhQlzpW6GdnzzGLhEamgA3DMDKi7AqY/rJg3UYAxVsebsOWjS1a\n7IMJZ6LwOBwJHTNGW7aeSJxRLZSyMZVamPKTiXniZNLclyr3vSArJWOF6Z+jzfOS8KW77E8dZhRO\nd+xLn/LBUHcJ61K4fBaQqNFwBuDpp2tJu7PnMOxPPgXog690PMGXvgQA2FtblzofCW3N504847En\nMAVsGIaREfYANgzDyIiKdXrigZvOpvNxYC1cMBnomAMVSLoLPP769aWPz0GT3jaAQfbHxuyCxfMu\nOrNxV6a79mYbm30rB23LiRmc7MKQsF3BvBWuNxlPxy4Ykg+QQoJlFGZycNIMz1Psd+LxzAVhGIZx\nEFGxMLSutNjxNmyd2PqFoTul9tnf73sDZuPKYvatPBxsK7byBZC2AdVqemVpoLZWV18Z9vW7ACRJ\ndQCg/cH0tjwejxWv3p0VpoANwzAyQpxzXd9YZBOAP1XucqqODzjnhvTkCc3GleUgtC9gNu4JDsjG\n+/UANgzDMMqHuSAMwzAywh7AhmEYGWEPYMMwjIw44AewiHxPRL4cvH9IRG4L3t8sIl/ZxzGe6sJ5\nWkWkocjnM0Vk6v5ed5Hj3CsiVRKUkibvNhaRxSLyiogs939DD/RYlaIX2LhORH4iIn8UkRYRufBA\nj1Up8mxjEekf1N/lItImIrccyLGK0R0F/CSAqQAgIjUAGgCcEHw/FUCnRnPOdecBOpPnP1BE5AIA\nHdfkqR5yb2MAn3HOTfR/b3bzWJUg7zb+JwBvOufGATgewP/rxrEqRW5t7JzbFtTfidDojru6cS0d\nTnBAfwAaAazxrycA+A8ADwMYBOBQAFsA1PnvvwbgWQAvArguOMZ2X9YA+BGAFgCLADwA4CL/XSuA\n6wA8B2AFgGYATQA2AHgDwHIA0wF8AsBKAC8AeKwL118P4AlopV15oHao5F8vsPFiAJOztmMvt/Ea\nAIdnbcfebOPgGsZ5e0u5bHPAM+Gcc+tEZLeIjIK2Lk8DOArAhwG8A2CFc26niJwDYCyAUwEIgHtF\nZIZz7rHgcBd4Qx0PYCiAlwH8e/B9m3Nukoh8AcBVzrm5IjLP/yg3AYCIrABwrnPuDREZ6D9rBHCb\nc+68IrdwPYCbAew4UBtUml5gYwD4mYjsAfArADc4X5OrhTzbmN8DuF5EZgJ4FcCVzrmN5bFOeciz\njSMuBnBHOetwdwfhnoIalEZ9Onj/pN/mHP/3PLRlaoYaOWQagDudc3udcxsAPBp9T8m/DGr8YjwJ\nYL6IXA7gEEB/+GIGFZGJAEY75+7u2m1mSi5t7PmMc24CVHVMB3Bpp3eaHXm1cS2AkQCecs5N8td9\n075uNiPyauOQiwH8ch/b7BfdzQVB384EqKRfA+CrALYC4PogAuDbzrkfd+M8PlcS9qDENTvnrhCR\nKQDOB7BMRE52zr1VbFtoyztZRFr98YaKyGLn3MxuXGOlyKuN4Zx7w5fbROQXUGXz825cY6XIq43f\ngvbg+NC5E8DnunF9lSSvNtYLE/kQgFrn3LJuXFsHyqGAZwN42zm3xzn3NoCB0AccneoPAfisiNQD\ngIgcVWQ0/EkAF4pIjYgMgzrN98U2AP35RkRGO+eWOOe+CWATgKNL7eic+zfnXKNzrgnaov6xSh++\nQE5tLCK1HJEWkT7+Hqoy2gQ5tbHvCt8XnOdMAC914ZxZkEsbB3waZVa/QPcfwCugI5rPRJ+945xr\nAwDn3MMAfgHgae97WYjAGJ5fQdfzfgnAAmj34519nPs+AB/3oSHTAXxXRFaIhpQ9BeAFEWkUkQe6\ndYfZk1cbHwrgIRF5ETr48QaAn3b1pnuYvNoYAL4O4Fpv50uhqrIaybONAeCTqMADuGpyQYhIvXNu\nu4gcCeD3AE73Ph6jTJiNK4/ZuPL0JhtX0zKA9/sRyToA1+fVoFWO2bjymI0rT6+xcdUoYMMwjIMN\nywVhGIaREfYANgzDyIj98gEPGNDghg5tqtClVB9vvtmKrVvbpCfPaTYuLw0NDa6pqalSh88ly5Yt\na3NlXCHDbNyRrtp4vx7AQ4c24TvfWXrgV5Uzrr56co+f02xcXpqamrB06cFjz64gImVdLshs3JGu\n2rjHoyDa2zt+1rdvT19F78ZsbBj5wHzAhmEYGdHjCviQQ5LXe/ZouWtX8W35fUgpJcdjFNuH5+zT\np2vXmHfMxoaRD0wBG4ZhZIQ9gA3DMDKix10QYfeVg0Vxl3f3bi1ri1xd3JXm8YoNPMX7x13n3jow\nZTY2jHxgCtgwDCMjKqaAqaI4KMP324MlMPmaJVUZlRbVVTGVxm2OPFJLxoG/9VbHbRr8OqljxmjZ\n0pI+Vl5VWk/ZOFbLob0GDkyf+7DDtKyvTx8rrzY2jEpiCtgwDCMjKqaAY1X2vl8oZEOQOG7tWi23\nbNFy4kQtm5vTx+L3IVSz06Zp2W/1i/oinJHTkJZhOyd/EkCixpYv1zIM28pTGNWB2JhqllAZr17d\n8TNuSzU7fLiWxdQsexn87bgtyauNDaOSmAI2DMPIiIr7gKnKWlu1DP2vfE01S/W0IUqvHKqzYcO0\npCrjcdu2nAgAmDE+GKpftEjLjbpK9+7ZqoCfeEI/pk/z8MO7cENVSCkbh/aiWmVJG9Nnzs/b2jru\nc+ihWh53nJa0+Z13JtvS58vfjKo59jXn1caGUUlMARuGYWRExRRwHDu6fr2WK4N1camAqcrovn3+\neS3feENLjrQDwEUXafn44+ltqP7GzDu1sG3j/xqpL7wDlCqPvmaeP4ycYFRFHvyUsY1H+tsN7RUr\n0dGjtfzUcd5nvnixluwWAMA992hJiX3ppVpO1sxlhx32d4VN2XuhoubvS58zfcN5tbFhVBJTwIZh\nGBnRLQVcbGYU4Ug6fYOvvqpl6GskVFEzZ2pJHyaVHFUUADzyiJb0U4ZqDwDWrEleNw5pT13E7qbj\nU8cvFtdKVVkt6qwzGxMvTAvbFotomDVLy+Nb7tIXt/vuBg26aVOyEzdetUpLdi+OPRYAcMnfJpvG\nccQvv6wlfb6xDxqoPhsbRlaYAjYMw8gIewAbhmFkRNkG4eIAf7oaOPjG7484ItmG3WN2V8+YuRcA\nMGaMtgsLFujnzzyT7MNBnosv1pLd79tv15JjSgAw5WtN+sL3f7e3lb6WPBBPCT7zTC2HvfsaAODP\ntccASAbjAGDUFj/YtsAb6P77taTRb7oJAPBw+4zCPnRhnHatlpOad+iLu+8GAAy+5ZvJCegf8j/I\n9OlDAaRdQYZhFMcUsGEYRkZ0SwGHqjcOd6K65UALg/jffTfZZ9u2zQCAPXsG6QdevjacdgaAJFws\nHFRiGBrHib79bS05IeDyy4ML9LFQ/9PSCCAZbwoVYniN1UgxG597rpbD3nopte1wP5iZmshC43n1\nigkTtLztNgDA1FkDUpsBSUjZvHla3nhjPwDAedyXI6pAMtfZD9RNHajdjB0TdcDzvvv062q2sWFk\nhSlgwzCMjOiWAuZUVSDxS8b+1TjtYTqsSt/Qb/uPw1sBAP1qH/P7qF+SCXfC4y9cqCUV9T//s5Zh\nWNoDDw4GkChf+qWpxlmGScSrLW1iaGMmuBn2p9/rC4aH+ZksvL9Uakk64WmYK64AAMyYrcqXPZWT\nT052GeQ7JLQFextY6GeunHZasjEPwJN6RdzP9z76958KIJ1QqdpsbBhZYQrYMAwjI7qlgDvz69En\nGyvjTZvCNWtUARcmWlxyiZbz5wMAjviQKuBQNfM4VFRnnaUloyPCiR6cXcu0k/T9drYcT7UR2rgQ\ntcEboqr1KnRpq759771kn0+98oq+mD4dALDxBPWvc/ILfb/FVjq+9VYta+7xkzc4dzt0GFOW06iM\nrhg/HgCw5f5Sd2YYhilgwzCMjKiYBozjgpOR+SAMAirVmITn3gfrAAAf8ypqyis/1y8CJ/ADLRrr\nSgUcp2Bk5ASQDNBTKMa+x1ILVlY9vGDKfv+eqpbJbgAAd3gjzJkDABi2Zx0A4MorG1OHDO1W88xT\n+uISjRHG009rSaNfd12yMRUwE/ccfTQAYMnL6mPOrY0NowcwBWwYhpERZVPA9FXGyVnihR2B0Nmo\napjqlQoOPh1lQeFxuhuAFSvSX73+uiqvD35QL4CKGABOOCF9LfFCkbFKr3aYepNqNrX8EpK0njVP\nPJZ8yNyejIbw0wsn0fHOY60PMqZ7H3O7jx32c+lwKtNSzp2bbEsj8kf0jvY9LemvDcPoiClgwzCM\njLAHsGEYRkaUzQURT7hgd59jM4zXB0I/gHZXOQB0fMOb+uIPf9DyqqsAAHsbhhb26L9YS7oegGX+\nvXapDz10TGFbvxRch1A4Tm7g5+GKvdUMbfzYch3gGj9RQ8oG992R3nBMYgOcfbaWvFnOM/bb7Jx9\nAYDCzGQAwBX/cD4AoOaaawAAhUWqr70WALCufXBhW3oeZkzUUbbX2vTaOOiWNxsbRk9iCtgwDCMj\nyh6GFqtNwjGfLVuSmQXTpo0AANw1zytfr3hx5ZUAgHufUeXLcDLdX8uTTtLjPP+8DuQNGaLTYzkB\nJNyWYWhU43lPDMPJJrTL4tWaLIf3e8IJSYjZKfN+AgCo2fI2AGDjLlWvy7TjgNv9eFq4anFNiyb5\nqRsyREuGAXr1/OD8ZFv+rhvfG5C6hjxMcjGMrDEFbBiGkRFl0ymlfHxeRBXi9cMk6H7GMTDHZ1f3\ni8L9vEVXNmbCnTD3CxXXlClarl59Zur4YahZvJ5arHyp0vOiiGljKvloHkZBdb7wQrIP3ekjR6ry\npWqOV6RmgnsAwIKF6RN5f/GP5ml7zZnQAPC5z2n5zjvpa6ASzpuNDaMnMQVsGIaREWX31DE9ZDwF\nlSseU7kCwICLz9MXXtY+MFmXulnrFRZVbZhgnIP4H/6wllR4jLIIVW+cdpK+4Lz7J2MbU21S1YZL\nOBGmlKRN+bvQNsc07U02pmw95RQtb7gBAHDPbH3LpZCAjgn341nShmGUxhSwYRhGRpRNC77/vpYU\nT1RC9FtSIU155F+TnShJ/fTYm85KH6PITGR85jNacsoxQ16ZdD2EapnwmmKfaV7Yl43ZCyjmj2cv\ngnbjtpxVvHV70hYP4O/infRMklTMnxtPNefvkKeUn4aRFaaADcMwMsIewIZhGBlRtg5iqaxX7K5y\nEA5fXpJ86ae6Mrzp0UeZK1gzpg0apMH9o0cnu3C9srfe0pJd4E2btAzdDpxcwG5w3l0QpWzM1Szi\nlaiBxH3DxGm0F90Uzc3oyGWXAQDufWJwal+6kQormASfcSCQ19BbBjwNo5KYAjYMw8iIsumTeApy\nknxHKSjTcIljL79afdx/nz4qWbl2G5eIC1XU5s1acjCJ6W7HjtWSYVYA0L9/+tyxKstbgpiu2jhU\nteE0biBRwJx4wbkWYeja7t2DU+ejTTkhxi/3BiBZf449EirivNrYMHoSU8CGYRgZUTYFzBSPhOqM\nZU2bT7hTZIlj73IspKXkJlSsq1YluzDFJFfGoE+YqjmcikxFSJ9lrMryNj22lI2piGm/cGVobkMf\nLRfA8O73QnjaiBHJPpxqTPux5O9C1RzC4+TdxobRk5gCNgzDyIiyrwlH4hSQj7VoaskZ4VC+zwJ+\nvJdsx//FX+jns3SeMRN/h6qWPuBXXtGSEzGKKeB4CnLeVVl83VT2jHSgv5dqN/zsoou0/PrXtYx7\nKKHvfNgwLU8cr9OT123QdjqOOAGSCIx4UkhebWwYPYkpYMMwjIwoe5QmlQ8VEdNRMlFM09x/KWw7\n6pF/T+/85JNa/vrXAIBGnwRmaXuyJNHvfqflscf64zVpSR9mKLAZB9zb1BjvJ773GdNUsf5xddKu\nMmKByXjq1r6W2rmxr5fLbYlsnjLIG7FFy0bfvViHAanzAcmSRPy9DcPoOqaADcMwMqLsCpg+3zgZ\nOkfHFy9OPtve/lkAwBaqVu+zbffbbr9RS8aaAh39uoyCOJhmXNHGvHfy57Xano4bmSzSOa6vD4m4\npxQK1uoAAASYSURBVEiOSiAxZChh2V0hXi3Tv0vVCyT+4N7WyzCMnsAUsGEYRkbYA9gwDCMjKtZx\njwdlOpuSSrdCPLV22zYtw+5tqUkVxWCIVG/tHjNJDqEt2sb2K3w2fPgoAMCW8VoeP3AdAGBrva6c\nTHdCOBV54ED9rjCxw59n/XotadeQ3mpjw6gkpoANwzAyomJhaF1RRPE2VMIcZDryyH3vs7/f9wb2\nZWMmKopfA0ALVN3G04lDW/O48Vjc/vy2hmHsG1PAhmEYGSHOua5vLLIJwJ8qdzlVxwecc0N68oRm\n4/JyENqzK5TV5mbjonTJxvv1ADYMwzDKh7kgDMMwMsIewIZhGBlxwA9gEfmeiHw5eP+QiNwWvL9Z\nRL6yj2M81YXztIpIQ5HPZ4rI1P297mD/T4vIChF5UUQeLHaOrOkFNv6Ut+8fROT/HOhxDKO30h0F\n/CSAqQAgIjUAGgCcEHw/FUCn//zOuQP+5wYwk+ffX0SkFsD3AXzEOXcigBcBXNmNa6kUebbxkQC+\nC+BM59wJAIaLyJnduBbD6HV05wH8FIAP+9cnAFgJYJuIDBKRQwEcB+A5ABCRr4nIs14NXccDiMh2\nX9aIyI9EpEVEFonIAyJyUXCuL4nIc16xNotIE4ArAPxvEVkuItNF5BMislJEXhCRx/Zx7eL/DhcR\nATAAwLpu2KJS5NnGxwBY5Zxj+vZHAFzYLWsYRi/jgCdiOOfWichuERkFVUlPAzgK+sB4B8AK59xO\nETkHwFgAp0IfeveKyAznXPgPfAGAJgDHAxgK4GUAYbLgNufcJBH5AoCrnHNzRWQegO3OuZsAQERW\nADjXOfeGiAz0nzUCuM05d1507btE5G8BrADwLoBVAL54oLaoFHm2MYDVAI71D/K1AOYAqCuLYQyj\nl9DdQbinoA8GPhyeDt777Oo4x/89D1VrzdCHRcg0AHc65/Y65zYAeDT6/i5fLoM+RIrxJID5InI5\ngEMAfYAVeTBARPoA+FsAJwFohLog/mHft5sJubSxc24z1MZ3AHgcQCuAIlkkDOPgpbtTkemjnADt\nHq8B8FUAWwH8zG8jAL7tnPtxN87zvi/3oMQ1O+euEJEpAM4HsExETnbOvVXieBP9Pq8CgIj8N4Bv\ndOP6KklebQzn3H0A7gMAEfk87AFsGCnKoYBnA3jbObfHOfc2gIHQLjIHhx4C8FkRqQcAETlKRIZG\nx3kSwIXeTzkMOvizL7YB6M83IjLaObfEOfdNAJsAHN3Jvm8AOF5EOFPlbGiXvBrJq43BaxCRQQC+\nAOC2zrY3jION7j6AV0BH5p+JPnvHOdcGAM65hwH8AsDT3oe4EME/tedXUD/hSwAWQLvR7+zj3PcB\n+DgHiAB81w8grYQ+mF4QkUYReSDe0Tm3DsB1AB4TkRehivhb+3HfPUkubez5voi8BH343+ic+2PX\nbtkwDg6qZiqyiNQ757b78KXfAzjd+yqNMmE2NozqoppWUrvfj6zXAbjeHgwVwWxsGFVE1ShgwzCM\ngw3LBWEYhpER9gA2DMPICHsAG4ZhZIQ9gA3DMDLCHsCGYRgZYQ9gwzCMjPj/TSs2lPgY2EMAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024e9ba358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 10 optimization iterations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have already performed 1 iteration.\n",
    "optimize(num_iterations=9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 73.8%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xn8lWP+x/HXJ4kWUrKnYkpK9jZkmaQsQzFGoZ+1LGWG\nmbHM2KYZZamhhkGylTRmFGPJkkqJESklLYwyllAKoSTU9fvj3Ne57/Ptu5z7e/Zv7+fj0eN7n/vc\ny/Xt+p7rfK7rvhZzziEiIumpVegEiIiUEhWaIiIxqNAUEYlBhaaISAwqNEVEYlChKSISgwpNEZEY\nVGiKiMSgQlNEJIbamZzcpEkT16JFiywlpTTMmTNnlXNuh0KnI1+UxzWf8jiejArNFi1aMHv27Ewu\nUXLM7MNCpyGflMc1n/I4HlXPRURiUKEpIhKDCk0RkRhUaIqIxKBCU0QkBhWaIiIxqNAUEYkho36a\nxeirr74C4KOPPqrwmObNmwMwfPhwANq1awfAXnvtBcD++++fyyTWeGvWrAHg448/BuDuu+/e5Jjz\nzjsPgAMOOCB/CZO8Wr16NQDbbrstALVq1YwYrWb8FiIieVLykebEiRMBePrppwGYPn06AO+9916F\n57Ru3RqADz74AID169envL9x48Ysp3Lz4CPMYcOGAXDDDTdUeOzIkSMB6N27NwB/+9vfAGjcuHEu\nkyh59Ktf/QqA+vXrA9CvXz8AfvGLX+Tkfp9//jkQ/g3Vrp2b4k2RpohIDCURaS5duhSAO++8E4BR\no0Yl31u3bh0AcZYifvfdd7OYOvFuvPFGAG6++eYqj/3pp58AGDduHABTp04FYPTo0QB07949BymU\nfDrooIMAGDp0KABHHnlkTu83YsQIAH788UcgrPFkmyJNEZEYSiLSXLZsGRB+k1TX3nvvDYRPyyW7\n9thjj5TXZgbAJZdckty3zz77APDDDz8AcP311wOwfPlyAHr27AnAVVddlTznyiuvBKBevXq5SLbk\nyO67756X+0yePBmA2267DQifUSjSFBEpAgWPNFetWpXc9pFkly5dADj22GMBqFOnDgANGzYEoEGD\nBslz/BPbHj16AGEU2alTJwAOPPDA5LF169YFwqd5kl3//ve/U16fdtppQPhkvDy+T+wpp5wCwBdf\nfAHAX/7yl+Qxvk37gQceAGDLLbfMUooll+6666683GfatGlAGGH6ttRcUaQpIhKDCk0RkRgKVj1f\nu3YtAMccc0xy31tvvQXAE088kXLsIYccAsDcuXOBxPT8nh8u2bRpU6DmDNUqRc899xwQPgC65ppr\nqjzn8MMPB+DJJ58E4I9//CMAL7/8cvIY3y3Jdyvz3ZJy1XlZMrNgwQIAPv3007zcb8qUKSmv//Sn\nP+X0fiphRERiyPtXte9qcsYZZwBhdAlw9dVXA9CtW7dyzy1vxbxmzZplOYVSXT7ffEf16AO7qhx6\n6KFA2BH6+OOPT77nJ2H5xz/+AcBJJ50EhA+apLjMnDkTgK+//jplf7YfwPoHP74zu3/Qe8QRR2T1\nPmUp0hQRiSFvkabvGuSH2vkJNnbYIVx6+IorrgDUiblUtWnTBggjzfLcd999QBg1XnjhheUe52si\nEA6f9f773/9mlE7JPv/5Brj11ltT3jv55JMBuOCCC7J6T98OPm/evJTrb7fddlm9T1mKNEVEYshb\npOmfiPvJHPxEwNGnpL7zupSm9u3bp7yeP38+AN9//31ynx9S6du2/VR+cdx///1AOCw22gNDf0OF\n8dvf/ja5XXZCnFw9zfaDHfJNkaaISAx5izRfffXVlNd+eKPvXymlr1evXgA89NBDAHTt2hWAFStW\nJI/ZeuutgTDSrI4PP/wQCJ+eR9vA7733XiCc+EPt47n11FNPATB+/PhN3vO9Xfyk39nin8pH/67y\nSZGmiEgMeYs0J0yYkPLajx7585//nNzn+99FJ9mQ0uEX0Orbt2/K/mh/zYcffhgII5Mvv/wSgGee\neaba9/3uu++S22eeeSYA++67LxCOJtJ0gNn1zTffAOGSJmX7ZEI4gYuvXWTL//73PyB8au6df/75\nWb1PRRRpiojEoEJTRCSGvFXPV65cCYSTOfghUNHq+eDBgwG46KKLgHBOTL9+dsuWLYFw9u+ohQsX\nAuHkHnrAVJz8SoT+54YNGwD49ttvU46LNvL7v5kdd9wx5RjflSXa9cRPBPP2228D8Pvf/x6AW265\nBdA669niV36cPXv2Ju/5zuy+iSRftt9++7zcR5GmiEgMeYs0L7/8cmDTIVZRPurww+bKDp9Lh49G\njjrqKAD++c9/xr6G5I6fqd8PhfQTdZQd+pbOUDg/I7xfOx3g4osvBsJI068f46cM9A8gpXomTZoE\nhJPreK1atUpu+8/tFltsAYRT+vlaQHn8bPx+8g3PT/Lhaxvl8ZHtnnvuWfUvkAWKNEVEYshbpOmH\nT/oOyb5rSPSbxa866SPO6vBtLb5LS7SrybXXXlvt60r1+clZAC699FIAPvvsMyCsCfjO6NXho1WA\nV155BQjXifHrC/npyp5//nkgXH9K4vGTZLz55psp+/0zCoAhQ4akvOc/zyNHjqzwur6boZ9o3PNd\nxk488cTkvhdeeCHlGF8rqSwazSZFmiIiMeQt0vTtGx06dADKn97LTynmo89BgwYBMGvWrNj38+0o\nc+bMiX2uZFf0ybiPMH1k4leh9BGi7/1QXdtssw0QTj3no1DfGds/RVekWT2PPfZYufv9sjNQvWcR\nfmo5P6m4Hyjha6TRyWB8TxzPt2PniyJNEZEYimplqqOPPjrltR8m5SNN/4Tt3HPPTR7Tv39/AIYP\nHw6EEYYUj+iEwn6xrSuvvBIIawSZtGOXx09Lt3HjxpT9++23X1bvs7m57rrrAPj1r3+dst9P9Qiw\n0047AZsub+EncPG1zaiOHTsC4ZDYJk2aADBjxgwA7rjjjuSxvm+o73O71157VedXqTZFmiIiMajQ\nFBGJoaiq52V1794dCDvS+gdEo0aNSh7z3nvvARXPAL7bbrvlMIUSl1/HxXcynzZtGgBnnXUWEA5K\n+MMf/pA8p6rql+/kDuEaREuWLAHC6r9kR9khzt4uu+yS3G7cuDFQvblMGzVqlPLaz8rvuzpF+YeG\n+Z6tX5GmiEgMlsk3cfv27V15A/azZd26dQCcd955APzrX/+q8pzatRPB8wknnACE8zdCdtZdNrM5\nzrn2VR9ZM+Qqj30XE/9gpmxXJN9FDcIhkBUpO/SuPP5Bg5+3s7LJHZTHxSfazch3H/PRbXUi2kzy\nWJGmiEgMRd2mWbduXQBGjBgBhJ2kox3W/RRifj0S3zbmO8ZLcfKzub///vsAjBkzBgiHVfoJNyDs\nphTHYYcdBkCPHj2AsGtavqYPk+zaYYcdyt0uBEWaIiIxFHWk6fnOshMnTgRg7Nixyff8RAw+siw7\nUa2UhrPPPjvl5/Lly5Pv+RqGX2nSP2H37XDRp+sHH3wwEA7H22qrrXKYatkcKdIUEYmhqJ+eFyM9\nWa35lMc1n56ei4jkiQpNEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJQYWmiEgMKjRFRGLIqHO7ma0E\nPsxeckpCc+dcYWcMyCPlcc2nPI4no0JTRGRzo+q5iEgMKjRFRGKotNA0s+3NbF7wb7mZfRJ5XSdX\niTKzZWb2dnCf19M4vp+ZrQyOX2xm52V4/4fNrFcaxx1tZm+Z2UIzezGTexZKIfLYzOqb2azgHovM\n7Po0zhkcSdvbZnZChml4xcwOqOKY6N/VPDM7N5N7FkqhPsfBvWub2XwzeyKNYwuRx1ub2QQzW2Jm\nM82sWVXXrXQ+TefcF8ABwcUHAWucc38tc1Mj0Ta6saqbxXS4c251jOPHOecuM7OdgQVm9pRzblUk\nnbWdcz9lK3Fm1hi4A+junFtmZiU5kWeB8ngd8HPn3Foz2xKYaWbPOueqmmpnmHNuhJm1A6aZ2Y4u\n0iif7TwOjHPOXZbla+ZVgT/HvwMWAOku5JPvPL4AWO6ca2lmfYGbgDMrO6Fa1XMzaxlECOOAhcDu\nZrY68n4fM7sv2N7JzB43s9lBdNG5OvdMl3NuOfAB0Cz45nrIzP4DjA6+9W4L0jHfzPoFaaxlZneZ\n2TtmNhloksat+gKPOueWBff9PEe/UkHkMo+dcxudc2uDl3WALYG0n0g65xYABjQKagV3m9ks4EYz\na2Bmo4N0zDWzE4M01jOz8UFN5DFg6xj/HTVSrj/HZtYcOAZ4MG7a8pjHPYExwfajQI+qTsikTXNv\nYLhzri3wSSXH3Q4MDeauOw3wmdDJzEZWcI4DXjSzOWZ2fpxEmVlLoDnwfiSdRzvn+pL4VvncOdcR\n6AAMDMLxU4E9gLbAucChkesNMbPjy7nVXsD2ZvZS8IfUN046S0TO8tjM6pjZPGAFMNE5N6e84yo4\n91Dge+fcl8GuXYDOzrkrgeuB54M87grcamZbA5cAXznn2gCDgQMj13uwkmrcacEX7KNmtlu6aSwh\nufwcjwCuIMYXopfHPN4N+BjAOfcDsNbMtqssbZksd7E0jeoUQDegdSL6BxLfHHWdc68DFbVXdnbO\nfRJUtSeb2WLn3KtV3OdMMzsKWA/0c86tDu75pHPu++CY7kAbM+sTvG4ItAKOAB4JqibLzGy6v6hz\n7poK7lcb2JfEN2l9ElXMmc65pVWks5TkLI+DP9ADzKwR8G8za+OcW1zFfa4ws3OAb4Hekf3jI9XK\n7sBxZvaH4PXWQDMSeTw0uPdcM1sYSUtFbZVPAGOdc+vNbCCJiKl7FWksNTnJY0s8E/jYOTfPzLrF\nSE++8zi2TArNtZHtjSRCaS8aFhvQMfiQpMU590nwc7mZPQl0BKoqNCtqe4qm04ABzrmp0QPM7OR0\n0xaxDPjEOfcd8F3QBLAfUJMKzZzlseec+8rMZpCoFlVVaA5zzo2oIp0G9Cr75RX5sMdJ26rIy1Ek\nopeaJld5fChwipmdFFxnWzMb45w7u4rz8prHJKLr3YHllngoVr+qZylZ6XIUfAN8ZWatzKwWEC2E\npgAD/YtKqkH+/QZm1iDYrk8iklsQvL7UzC7KIKmTgAFmVju4XmszqwvMAHoHbZu7AUemca0ngMPN\nbIsgnR2BdzJIW1HLch7vaGYNg+16JKKYd4LXQ30bVTVNAn4duZevos0Azgj27Q/sU9WFzGyXyMte\nJNr9aqxs5rFz7krnXFPnXAsS7f8v+AKzmPIYeArwBflpwAtVnZDNfppXkfhlXiURhXkDgcOCdqFF\nQH+otC1kF+A/ZvYWMAv4t3NuSvBeG+CLDNJ4D/AeMM/MFgB3k4i2JwAfAYtIVMFm+hMqatMMGqpf\nBN4mUT25K43qZanLVh7vCrwUyeNnnHPPB+/tBywv55x0/Rmob4kuKwuBQcH+v5Nog14MXAfM9SdU\n0t71OzNbEKTzIiBW+3qJylYeV6aY8ngUsIuZLSHRJnp1VTcvqWGUZvYM0DMH3UqkCFiifvWcc+7Y\nQqdFcqMm5HFJFZoiIoWmYZQiIjGo0BQRiUGFpohIDCo0RURiyKRzO02aNHEtWrTIUlJKw5w5c1Zt\nTrN6K49rPuVxPBkVmi1atGD27HRGYNUcZrZZLQugPK75lMfxqHouIhKDCk0RkRhUaIqIxKBCU0Qk\nBhWaIiIxqNAUEYkhoy5HItn25ptvAnDzzTcn940fPx6Al19+GYAuXbrkP2GSdYsXhzMp+rz1Lrjg\ngnwnJ22KNEVEYlCkKQW1ZMkSAPr37w/A668nlptZt27dJsfeeuutgCLNmiIaaQ4fPhyAd999Fwgj\nz7Fjx+Y/YVVQpCkiEkPBIs2mTZsC0KhRo+S+q69OzDR/+umnx77enDmJFWBfeCF1iY+77747ub1s\nWWL2/h133BGAKVMSq2i0a9cu9v2kejZs2ADA1KmJte1OPfVUAL799lsAmjRJLDm/zTbbJM/5/PPE\nkvLr16/PWzol90455ZTk9sEHHwxAp06dAHjllVcAWLUqsbad/7soBoo0RURiKFik+cgjjwCp3zbn\nnHMOABdffHHa1/HLdfz4449A5dGIX+Jz5cqVAMydm1h3SZFmbq1YsSK5ffbZiYX/Jk2aBECDBg0A\nuO+++wA49tjE0jGPPfZY8pxLL700L+mUwmnevDkAu+++OxC2dyrSFBEpcQWLNA8//HAg7IMHcMst\ntwCbtktWxkea1VwoXnLIRwnHHXdccp+PIO6//34gjCx33XXXKq+3uc35uDnxfxfvvPMOENZA9957\n74KlqSKKNEVEYlChKSISQ8E7tx911FHJ7Q4dOgBh1yDvm2++AWDRokUAdO7cucrr/uIXvwDg/fff\n3+Q9362hWbNm8RMsafPV8+iQON/FqDoN+1dccUV2EiZF58MPExOpr127FoA//vGPhUxOpRRpiojE\nUPBIM6p+/foAtG7dutz3fSRamYkTJwLw2WefbfKe71r0zDPPAKkd6yX7fCN+po35O+yQWP9qjz32\nyDhNUjyiwyh9V7S2bdsC0KZNm4KkKR2KNEVEYiiqSDMb5s+fD5Q/4UO9evUARZjF7IMPPgBSh7/6\ndlCpGXy75TXXXJPcV7duXQCmT59eiCTFokhTRCSGGhNpjhkzBoAbb7yxwmP8RB1SvEaNGgVAw4YN\nk/sqy1MpPTfddBMATz75ZHLfmWeeCRTXcMmKKNIUEYmh5CNNP6XYb37zG2DTtsw//elPye1inkJ/\nc+f7dD744IMA9O7dO/nedtttV5A0SXb5iXKGDBkCwJFHHpl876GHHipImqpDkaaISAwlGWl+//33\nye0ePXoAsGbNmpRjttxySyAcGQSw88475yF1Uh1/+ctfgLDm4CfykNLnI8zjjz8eCJ8t3HbbbQVL\nUyYUaYqIxKBCU0QkhpKqnvtqebdu3ZL7XnvtNWDT+TRvv/12AA466KA8pU6qY/Xq1UC4CuVll10G\nqHpek/ztb38DwnW8Ro4cCaR+Nv2EHf6BYFkzZsxIbvvPup9L169g6bsr+bXG/GCWbFOkKSISQ0lE\nmv7hgH/o46NLCL9tPD/wX92LSsMll1wChBOs+PXPM+WH6vkoZ8KECUDYpWn//ffPyn2kYo8//jgQ\nDk7wEaLv3H7vvfcmj/3oo4+AMNIsuyJD9HNedl/Z136CmL59+2b19/EUaYqIxFDUkeZXX30FwJ13\n3gmE7V7lrQfk96UzQbEUnh9CN27cOCAchOBXJYzj66+/Tm5PnjwZgGuvvRYIJ6EeMGAAAD/72c+q\nmWIpj4/o/do+EHZef+KJJ4BNa4N+UhY/SQfADTfckHJMdWqK//d//weEtQtFmiIiRaCoI83nnnsO\nSB0KWRH/TbfnnnvmNE2SGb8u/aBBgwDYbbfdgHDChnT4dq+//vWvANxzzz3J9/zTeH9dH3lGh+xJ\n5gYPHgzAP/7xDyB8gg2btjUeccQRAJx88slAuBJtdHLqbDzpHjt2LJAa9eaCIk0RkRiKMtL0bZl3\n3HFHue/7ZSsAzj//fCBsq6pVS98DxcxHmPPmzQNg6tSpALRq1arCc9544w0ArrrqKgCmTZuW8v4h\nhxyS3PbRjBZhyw2/HnnZ9srocwa/YOHzzz8P5H/t8lzfTyWMiEgMKjRFRGIoqur5l19+CcB5550H\nwKxZs1Le9zMXRddE7tOnT55SJ9XlZ7kBGD16NADHHXccAF27dgXCbij+wc1jjz2WPMdXx/3DAj9z\n1S9/+UsgtWtJ7dpF9Sdd4/hqedluf76LF4Rz25bCLOzVoUhTRCSGgn8t+4c+EHZfePrpp8s9duDA\ngYCiy1LjBycALF++HAijRP9gyE/isGLFCiA1YvTR6HXXXQdAly5dcptgqZDPJ893HyrmdcqzTZGm\niEgMBY80o8Ol/AD/snzbiJ/cQUrDTz/9BJS//ku/fv3KPcd3QvfTewF07949B6mT6tBEOIo0RURi\nKVik+d133wEwf/78Ko/1T0f32GOPnKZJsstPOvu///1vk/d8O+Wpp54KhHnr28jq16+fjySKxKZI\nU0QkhoJFmn5qsPKiEM9PE6Z2lNLUqVMnYNOpwURKmSJNEZEYChZpnn766UDq5KMbNmxIOWbo0KEA\ntG7dOn8JExGphCJNEZEYVGiKiMRQ8M7tixYtKnQSRETSpkhTRCQGFZoiIjGo0BQRicEy6XhsZiuB\nD7OXnJLQ3Dm3Q6ETkS/K45pPeRxPRoWmiMjmRtVzEZEYVGiKiMSgQlNEJIZKC00z297M5gX/lpvZ\nJ5HXdXKRIDNrG7nHPDP71swqnbLdzPqZ2crg+MVmdl6GaXjYzHpVcUxjM3vKzOab2etm1jaTexZK\ngfK4uZlNN7NFZrawqvwNzilEHp8S5O88M3vDzA7N5J6Fojyu9Jh9zGymma03s8vSurBzLq1/wCDg\n8nL2G1Ar3evE+QdsCXwONK3iuH7AiGB7Z2AV0KTMMbVj3PdhoFcVxwwHrgm29wEm5+L/IJ//8pXH\nwK7AAcH2tsBSYK8izOMGhA9LDwIWFDqPlMdZz+OdgPbAzcBl6Vy3WtVzM2sZfIOMAxYCu5vZ6sj7\nfczsvmB7JzN73Mxmm9ksM+sc41bHAIudc8vSPcE5txz4AGhmZoPN7CEz+w8w2sxqm9ltQTrmm1m/\nII21zOwuM3vHzCYD6SzY3BZ4MbjnQmAvM9s+xu9W1HKZx865T51z84Ltb4B3gN3STVu+8tg5t8YF\nnyygPlCjupooj8E5t8I5Nxv4Kd20ZTL2fG/gLOfcbDOr7Dq3A0Odc6+ZWQtgItDOzDoB5zrnLqrk\n3D7AI3ESZWYtgebA+5F0HuGc+97MBgCfO+c6mtlWwGtm9gLQGdiDREG4K7AIGBlcbwjwH+fcs2Vu\n9RZwCjDTzA4Bmgb/voiT3iKX8zw2sz2BdsAb6SYqj3mMmZ0KDCHxATw+3TSWkM0+j+PKpNBcGpTQ\nVekGtDYz/7qRmdV1zr0OvF7RSWa2NXAC8Ls003OmmR0FrAf6OedWB/d80jn3fXBMd6CNmfmF0xsC\nrYAjgEeccxuBZWY23V/UOXdNBfcbAtxuZvNIFKBvARsqOLZU5TqPtwUeA37tnFuTxn3yncc45yYA\nE8zs58ANwfVrks0+j+PKpNBcG9neSKJNxNs6sm1AR+fcDzGvfwLwunNuVZrHj3POldeQG02nAQOc\nc1OjB5jZyTHThnPua+Ds4PxaJKoSFa/dUZpylseWeADxOPCgc+6pNE/Lax5HOeemmdkYM9vOObe6\n6jNKhvI4pqx0OQpK9q/MrFVQgEQTPwUY6F+Y2QFpXvZ0ylTNzexSM6usOl+VScAAXw0xs9ZmVheY\nAfQO2kR2A46s6kJmtp2ZbRm8vBCY4pxbW9k5pSybeWyJ0GE0MM85d3uZ94opj1sGacXM2pN4KFST\nCswUm2MeV0c2+2leReKXeRWIPrgZCBwWNNguAvoDmFknMxtZ3oXMbBvg58ATZd5qQ2ZthvcA7wHz\nzGwBcDeJaHsC8BGJNpAHgZmRtAwxs/LasvYFFpnZu8DRpN+MUMqylcdHkvhSPMbCri89gveKKY9P\nAxYETTC3A70zSFep2Kzy2Myamtky4DfAIDNbZmb1Krt5SY09N7NngJ7OubSfdElpUR7XfKWexyVV\naIqIFJqGUYqIxKBCU0QkBhWaIiIxZLQaZZMmTVyLFi2ylJTSMGfOnFVuM5rVW3lc8ymP48mo0GzR\nogWzZ6czmKDmMLPNalkA5XHNpzyOR9VzEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJQYWmiEgMKjRF\nRGLIqJ9mtq1YsQKAQYMGATByZOqMU3379gXghhtuSO7b3Drl1hQXX3wxAPfccw8AvXsnZl174IEH\nAKhbt25hEiZSBUWaIiIxFDzS9NElQLdu3QBYuHAhAJH1SAAYN24cAGeffXZynyLN0vLZZ58B8Pzz\nzwNhHj/66KMA9O/fH4CuXbsWIHUiVVOkKSISQ8EjzWuvvTa5/dFHHwFwwQUXANCoUSMAhg8fDsAP\nPyTWdBo2bFjyHB+dSmnYZZddANh5552BMM+9m266CYAOHTok922zzTZ5Sp1UxNcQli5dmrJ/0qRJ\nADz1VLrrpoU2btyY3G7VqhUAv/tdYtWYpk2bAsVZk1SkKSISQ8EjzW233Ta5ff/99wNw6qmnphzz\nySefADB+/HgA1q9fn3zPR5916tTJaToluw499FAAZs2albL/xRdfBODpp59O7jvjjDPylzApV8+e\nPQGyOhtSdKkd/xzjiScSaynus88+ADz33HNAGHkWA0WaIiIxFDzSvPXWW6s85t577wXggw8+AGDG\njBnJ9+bOnQtAp06dsp84yZkePRKrud55550A/PjjjynvRyMaRZqF98YbbwCb9mjJFR95DhgwAKhe\nm2muKNIUEYlBhaaISAwFr56nw1fBX3nllQKnRLKle/fuQNisUjZv//nPfya3L7zwQgBat26dp9RJ\nWWPHjgXCQQjp8A+PfF6X9dJLLyW3L7nkEgC++eab6iYxbxRpiojEUBKR5ueff57yer/99ktut2zZ\nMt/JkSzygxuOPfbYlP3R4bW+C9rbb7+dv4RJCj9Zjv+ZDdEuRxVp3Lhx1u6XLYo0RURiKIlIc9So\nUSmvd9ppp+T29ttvn+/kSBYddthhQBhRfPnll5sc8/XXXwNhe1d0QISUjuXLlwNw++23A+GQWai4\nK1N0GshioUhTRCSGoo40lyxZAmw6dMtHHgCvvfYaAPvuuy8A9evXz1PqJBvq1asHhBM1RCdw8ZYt\nWwaET1tPPPHEPKVO0uWHOkPqU3EI8/a7774DYM2aNWlf1w+r3H333ZP7evXqVe10ZoMiTRGRGIo6\n0vRtWGWfnkcnefATP/iJia+88koA2rRpk48kSpb8/ve/B+DZZ58F4NVXX93kmMGDBwNhnqs9u/B8\n/81bbrkluW/RokVZu/6ll14KwFZbbZXc59s5ff/Pn/3sZ0D+apmKNEVEYijqSNNPPnvMMccA8Oab\nbwLwxRdfbHLsmDFjAPjwww8BmDhxIhC2mUlx81P7+Z/RPnx+208a4SfEVaRZeL4tM5vRZXmi00H6\n2qT/OXAgxdfzAAAId0lEQVTgQADuuOOOnKbBU6QpIhKDCk0RkRiKunru1w3x65AsXrwYgHnz5iWP\nGTFiBBBW3aZPnw6Enab9zM8Qrksjxat58+ZA+Z2d/T4/q3u7du3ylzApl286qWxIZL9+/YAwb8vr\nVlaWX63Uz6PpHxBC2ATn/f3vfwegbdu2AFx88cVppb26FGmKiMRQ1JFmWb4bUbQ70fHHHw9A586d\nAXj33XcBeOutt4BwDSEpDb7jsn+wVx5f87jiiisAqF27pP6MaxQfRVY24MB/XrfYYou0r+sncPE/\n/fSQEP6NfPzxxynnvP/++2lfPxOKNEVEYij5r+iGDRsCULdu3QKnRLLB1xyitYmy3VlefvllIGzP\nvvzyy/OUOilrhx12SPmZKwceeGBy+8knnwSgS5cuQDg80683NmzYsJymRZGmiEgMJRlpfvrpp8nt\ne+65BwifrHv5Hlol2eHbJ/v375/c99vf/rbcY/0ABkWaxcU/+S47sXS2rF69GoANGzak7M/V/cpS\npCkiEkNJRZq+r9agQYOS+8pOG+cjTH+shtqVpnQWUfM9JKL99nxfQMmvqVOnJrd79+4NwDnnnAOE\nbY2Z9HLw0StAnz59gNShlQAnnXRSta8fhyJNEZEYVGiKiMRQ1NXzBx98EICrr74aCBuAy4blAKed\ndhoQzrXnh2BKaYo26nfs2BEIh8p6fr5Vv+YMhFVBya9169Ylt7/99lsgnHXov//9L7Bp9dx3GQLo\n0KEDAMOHDy/3+tG89/neoEEDIFyd9rjjjqv+LxCDIk0RkRiKMtIcPXo0EA68r2wo5HXXXZfyU0Pq\nap4TTjgB2DTS9GbOnJnP5Eg59t577+S2f4jnhzT7Ya9l+S5jUPFqlJXp2bMnEM4eny+KNEVEYijK\nsMx/M1UUYUanlrr++uuBeJMBSGk588wzgdSuZlGnnHJKHlMj5WnZsmVye86cOQBMmDABgMmTJwPh\nM4mVK1cC8Prrr1d53aZNmwJw5JFHbvJetC07nxRpiojEUJSRpp9A+F//+hcQfttMmTIFSP1Wq1VL\n5X5N59e89k9WhwwZAoQTNXTt2rUwCZNy+XW5zjrrrJSfa9euBcKn60uWLEmes3TpUiCMRg8++GAA\nGjduDIQTDBcDlTgiIjFYZdPUV6V9+/au7DDGms7M5jjn2hc6HfmiPK75lMfxKNIUEYlBhaaISAwq\nNEVEYlChKSISgwpNEZEYVGiKiMSgQlNEJAYVmiIiMWTUud3MVgIfVnlgzdLcOZfbRZ6LiPK45lMe\nx5NRoSkisrlR9VxEJAYVmiIiMVRaaJrZ9mY2L/i33Mw+ibyuk4sEmVl9M5sV3GORmV2fxjmDI2l7\n28xOyDANr5jZAVUc08/MVkb+P87N5J6FUog8Du47xv//pXl89P97sZmdl+H9HzazXlUc083Mvo78\nf1yTyT0LRZ/jSo+J/TmudD5N59wXwAHBxQcBa5xzfy1zUyPRNrqxqpulaR3wc+fcWjPbEphpZs86\n56qahmWYc26EmbUDppnZji7SYGtmtZ1zP2Upjd4459xlWb5mXhUojwEeAO4ERsU4Z5xz7jIz2xlY\nYGZPOedWRdKZizye5pyrtHAtdvocVynW57ha1XMzaxl8e4wDFgK7m9nqyPt9zOy+YHsnM3vczGYH\n3zydK7u2c26jc25t8LIOsCWQ9tMq59wCwIBGQTRxt5nNAm40swZmNjpIx1wzOzFIYz0zGx9EMI8B\nW8f476iRcpnHAM65l4Avq5M259xy4AOgWRCdPGRm/wFGm1ltM7stSMd8M+sXpLGWmd1lZu+Y2WSg\nSXXuXZPoc1w9mbRp7g0Md861BT6p5LjbgaHB3HWnAT4TOpnZyPJOMLM6lqi2rQAmOufmpJsoMzsU\n+N455z+QuwCdnXNXAtcDzzvnOgJdgVvNbGvgEuAr51wbYDBwYOR6D1YS4p8WfDAfNbPd0k1jCclZ\nHmfCzFoCzYH3I+k82jnXF7gA+DzI4w7AQDNrBpwK7AG0Bc4FDo1cb4iZHV/B7bqY2Vtm9qyZFc/0\n4dmjz3HMz3Emy10sTSPUBugGtLZwic5GZlbXOfc6UO7KSs65H4ADzKwR8G8za+OcW1zFfa4ws3OA\nb4Hekf3jI1WO7sBxZvaH4PXWQDPgCGBocO+5ZrYwkpaK2jieAMY659ab2UDgweD6NUnO8riazjSz\no4D1QD/n3Orgnk86574PjukOtDGzPsHrhkArEnn8SPC3sMzMpvuLOucqaqt8A2jhnFsTRDOPkyhk\nahJ9jmN+jjMpNNdGtjeSCKW9aFhsQMfgPzAW59xXZjYD6AFU9Z89zDk3oop0GtDLObc0eoBVY83l\naFsaiXa5wbEvUvxynscxVdT2VDaPBzjnpkYPMLOT497MOfd1ZPvpoIq4nXNudWXnlRh9jkNpfY6z\n0uUo+Ab4ysxamVktIPoHOgUY6F9UEiL793c0s4bBdj0S33DvBK+H+vaLapoE/DpyLx++zwDOCPbt\nD+xT1YXMbJfIy14k2oRqrGzmcWXM7FIzu6j6KWUSMMDMagfXa21mdUnkce+gbXM3YNM1YTdNy86R\n7c7ATzWswEyhz3F6n+Ns9tO8isQv8yqwLLJ/IHBY0GawCOgPlbaF7Aq8ZGZvAbOAZ5xzzwfv7Qcs\nzyCNfwbqW6I7w0JgULD/78D2ZrYYuA6Y60+opC3kd2a2IEjnRcD5GaSrVGQrjzGz8cDLQFszWxZU\nyQDaAF9kkMZ7gPeAeWa2ALibRI1qAvARsIhEFWxmJC0VtWn2MbOFQbvccFKrizWVPsdVKJlhlJaI\nvZ9zzh1b6LRI7pjZM0DPHHQrkSJQEz7HJVNoiogUAw2jFBGJQYWmiEgMKjRFRGJQoSkiEoMKTRGR\nGFRoiojEoEJTRCSG/wftSC5FFuLBNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024e9b3e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXuQXcV17r81SKNBjKSRNHqNHh4LCYSQsBDCgHhEtgGD\nIQnPgCv2LQfjFHF8C1/HrvhWHBe+dhIS29cm105worJJivgRcEKMbMAoQcECCYNAIAHCyDBGbzR6\nD2LQq+8fq7+9+/TsM89zZp89s35VU31mn/1cu8/eX69evVqcczAMwzAGn7q8T8AwDGO4Yg9gwzCM\nnLAHsGEYRk7YA9gwDCMn7AFsGIaRE/YANgzDyAl7ABuGYeSEPYANwzBywh7AhmEYOTGiLyuPG9fs\nJk9urdKp1B5vvtmGAwfaZTCPaTauLsPNvgCwefO6dufcpME6ntm49/TpATx5civuuuuZvh6jsNx+\n+5JBP6bZuLoMN/sCwFVXyW8G83hm497TpwdwJTh2LOMk+nEW3E9/th3qZNm4J8yOvSe0L+0Wl/G6\nWfckXsZts/Y/3KjUc6LWMR+wYRhGTgzaO6U7VRarhM7O0v+7e/OddFLp/yNHdl0n3n4ovknLUc7u\nR4+W/j9+fNd14vuQRU+2HEq2zqqP5RRvueVZ35H9+7UM63Rcn60upyXrcNZvvid6Y7fBsK0pYMMw\njJywB7BhGEZO5NoJFzeD33pLy44OLdkEmDgxXef48dJ12EzurrnQ2KjlqFFannxyz9sUkSxXQWzj\nd97RkvajbcJt2RTurgMphraMbTyU4DU2NKTL+JllU5OWs6ce1g+bNmmZ5YOYMQMAcLBhMgBg40Zd\nTPuHHDqkJe/nULQv0Ls6xzpL2tu7bsN6SOimiJ8tIXT99OT2qSSmgA3DMHKi6hqwXAcbkL6NWCYn\nNaJ0Xao2ADj/fC35Fty8WUu+8aiQAaCtrXS/3A/X4T6GmhIO7cnP+/ZpSRtQqU2dquWSIBx3QoOq\nt72dowEAr75auv+5c9PPVH4Uejt3aknFNhRaG3GIWaiAW1u1bOn4lX5Y72Us5exDD2lJAwHAueeW\nbDzWG3/p1VcDAF5ob0lW3bpVS9oxVsdDwb5A+nyIWxRAauNZTQf1AysZK3er/pBPzDkt2Wb9ei3D\n5w2Q2o+qOVwWPw8Go8VsCtgwDCMnKv5Mj303sfLlywtIX2DxW51vvGXLtFw873D6JV9t/g3W0qp+\nNPrTtu9M3ylUd1TCVMt8u/K4VINAbSuJOFA/LmnP8O1Oe3MZr3nRIi1powlbX0g38itP8F/OnDkf\nQGq3CU0nklVP+Hd4HLJGpd2bsMBagece+xh5vqwnzc3pdy2NXpWt9Yp37VotafgzztAyjPOLY9W4\nri+bmlMFzPt2yimlm1AZk9AnXKv2zYLnOmeOlrTthLZn05VW+t/86tVa8ofMjf0Nq+NOACz2z4Pk\nZvrvDi97f8kuAGDFCi1pU/Y5xaGZ1bCxKWDDMIycqNq7Mla+fJOHPhmu8/bbWk7yqSyogFmWOHPp\nzPz1r7Xka+uiiwAAjY2peqBiiX3KVIFZAw1qWT30VvmG5qL/lsv27Cldl4rjrKuDZoA3/PaOsQBS\ne1GozZiRvre5H7o44+gK9trXsl1J3MLgNdBG/L5latoCwFZvHFbWuDOCy7O69VlBeSBvvKx+knJD\nntnSKEpUBM973Dgtx4zR8rQZvpVLlfv00+lG69ZpuWOHljNnarlggZahw5jwoRL9P7r9DQDAvHmz\nkq9Yr1et0pLRWGy9cffVsLEpYMMwjJyomi6h8uHbnC+ksIeeYnbaNC0Z4cAe+QnY688yOE0qC76W\n+PpauRIAMJY7ARKFMWtGsz8Xfd884xM1xQoyPFQRFBttHCvUUAFv26blK69o6ZxK4N271ak4Z47a\n8citqSLgPaNAo51279Yy7NDnMdkQ4TZ0wWXZsVZ77ssNI2ZV4zWVSFR+SaXrVdlrW+sBpLYK95mI\nYr+bqd5mE5p1XyMCHz53z3OjveMWZhj3Wqv2Bboq98TPGnYOAanvHEgVL41x6aUAgJd2TgCQ/p5P\nPz3dZOFCLUd37i3Z9o12jexpCm7hvHlaPveclvzN0O8et9SBVLkPFFPAhmEYOVGxd2T8lohHrNH3\nGLpr+GLjG4h+oaQXmso07Hb2vt7kjcnv2K0ZyjP6iDynTdWDdy5Q3yYDKkKlUGtpLrNch7RxPKqN\n/thQ0VMhOcchcUe4FwBAY6PKuvr9bybb1D/8MPyXAICmS68DABw4oItDhU31QQUWj0AiYXx2Ldm4\nuzSRsS+YKn/HjtHJuu+8o59Z/RgEwW1pj9BmVH3veY+Wl12mZdMirZcNGXHccaRQX0Yp5k14jrFP\nm/Vz74zZAIAJl7YCAA53ptqQP3VWy5W3+lIbvckz5Zxz0uPccYeWZ589oWQfPBf+9oH0EUJVG6t0\nEkb0VKoOmwI2DMPICXsAG4Zh5MSABHTYtOBnNjXLNeMY+A90jSLhd11kfRAZv2ufdm686B3wuzfo\n8ssu04QmJQHcbHewt8/3TpyVdNRp8zEeslwU2HxjEpEoLh1AOMiE71r2PqjRr7jC//uFL6Qb3XOP\nlt7d03C1uiDYzGZTEEjdHnQjEZ5DPKghPM9aIx4Oy6Yp7czRxWz6AmlTlrahq4AdpOxgDl0IcQcl\nBxzVQcPbWoIeoqYmraO0M8s4+U9/cuLmQeyqfPnl0nLPHq2n4UAJ2nvDBv87xuu+VHfaoUPqexg/\nPjUC3Zv17dsBALOa1VBHRqg9wz4//v55j2hbuo/4vw1FNgzDGEJU7Jkepz2kaojVRKiAqY6oCJKw\ns83tJV+8tLk+2WaDV7w/+pGW8eCDP/1Ea3oABnWzp4gH9HFAU6bMLtkH0DUxUBGg+uHlhUOr0zd9\ndIOgPUEfulRVRPtV/5h8wzbE5d5utA/NSSUIpPeVx+RgGt73rPSJtaqASZx0hzZkNWLAPpDWu337\n/CAB+Ch+6E15/XWtuyLTkm2mTNGSrYazzy7d2eGps5N1ad94SHk1VdlgwJYynxPs4GTJ4cEAsGMH\n57uk8mWPrtr0xhvV1nffnW4z9v/9hX5g779v6nWefzmA0hYJj8n++6xnVYwNRTYMwyg4FXt/UoUx\ndCMOXufbJAxD47LR8MMQ2/xr3ifOeHytqof770+3ofrYsGFPyfE7OjSDxrXXTkiWncaciISOJb52\nT+n+mmqVeIhkHBKTlVydYWdUD2PG+I03qt5dHex/uy8v975yuoQZYhX6danimAeFwetZAwRILYWh\nhcTnQ3VLZbRmjZZhIpx9+/gP1Rn9lBxhoKFlzqWV7eSTddmpp3IfWk7xOx4dGLixcXLmubJqZ43C\nLRJxOBptvWPHW8FabL1pUqjx49Um7Lb4zEW/1A+fXZ5u8uijWl58sZZe1tLXHPqY48dC1u8oxsLQ\nDMMwCk7FNEj8JqC6pUJKUhnSzwsA7d7h6t/8R5YsBZAqXiqvRx89GOz5JV9SeWgIwHPPfQAA0OET\nyABIHU2ULHSk+dfs8fP1jRqOLK1lVRa/kcvNkBv6sVMFTBWhNknGtvgmRRCXjiSgwUezf8GPfXFO\n/ZyNjak/M84nE59LraejzLIv6wNVJm3IJC27d4f+dLbEaHQqt1IFDKROcP4WnnpKS/bYT8kw5s5I\nncU99dPSW5F5TbVG7F+PE2Ol9fVIsJXa8NxzVfl+5Su69HL8XD/c6Z2/oaylY/288wAAh+ctBgDs\n8hEVVMIA8OKLWnY3fL7cdQwUU8CGYRg5MaDneJZ6oM+PCUeohOs2+ylbwu5HOtl8QDCV7wMPaPno\no+xZXhUcdZcvqUI4Y6cqjNA/t5iKgq/XMnMgFSUdZU90VRGpiksVhfrVkx5eb+zwTexzteO7m7RF\n4hxTA6pszpqUMk7gQ7LSUdaqD7gcXatNqM7YI88mhcakNviJNjs7+X2qmvft02YBY2Fpw8NzzgJQ\nWod5L2M/ZdF9vyRO1kVGjkyzobOuMgvB5Rf5PqM7V5du9JGPpJ8544B/ELHxS+UbJtNh/wWfXYM5\nvNsUsGEYRk5UXIPwzUIfY90m77Nl8Gg47Mx3me+aqL7YX/xCF993H1WDD/rFa8ER6A+mBNA0inPn\nqqpgjCUAoD0aIkZZ7n1sYfxvESinGLtLzBInFKE/jWriyFfV6EG6I8zzdjrj41RtNJSqkrABEY90\n7I0yq3Xly/OLR3XyGidNSiMaDh1S3yI72zm6k/V/40atgzt3npRsQ8WVxP964olNgTTyhD+bODk8\nFWTWiMNaIav1E48biNcNW1L0zSb9FtzJDTcAAE7c8X8AlLYc+Nte3Kqtldm+1XLxxfUl3wOp+qYv\neDAxBWwYhpET9gA2DMPIiYo3BpNws0bfUcE0888/r2Wo/a+5BkA6vRunfkpDzTgEMYgTS2B4j6a+\np8997txglc2+B4Ne/GgochFyqYbE58skPD016xRteo0fr52Wn/+8Ln3tq1qGLgj8+Z9r+fFfRvvQ\nA4ZNSjbfipSftifiZjBDKbMGE/F66d1icqP6Y9pRtH1/ab5goOvw7WSi7yjhDpDOQsJt4uHmzKFd\ntE45hifG4YrJrCMBtAdt+KOH9Le/Z492Wq6+U5dzQAsAnHuuls3NWu9nHVM35tJG7z9blM6g/MJm\nvUcMM6QrYzBcZaaADcMwcqJqCjjpUWDvQeghJ17CMclJmqWes84yeD2MNucBNNxn0iTtwKMCnjAi\nGLTBHihOPRBJmo5nur+WWidOaMLLDVVomlJRw6I+9jH9f8LqnwAAmFcnneMhWOnjnF2E92Nkl/3H\n84+VC0crEqzD7CzzU5Ch3nfknBiRJodiCsnEKKzv/v8WlnOCNoY/wOGGCSXHiQd+AGlyI6rw2L7x\nzN9AMRJKsbVGs8WpPzmbN5AxI4YfTMFr57qdnWkT8MABratsmdxys2++MNZ1y5Zk3bOuvBIAsHWr\n6lG2OkwBG4ZhDGEq/oxPQkXoto2HA4fxMn4cZbsfknn0KMPPOLiCfprQualTn86cqYrOR6IkSa1L\nHGjRnHCUES+1qd4rqr8yHjIbJxEJp9CjOZj45aab/Bd/fU/JPucEn9/Yyvfyu3xJxae+t6yQJ6qu\nIivf2Pc7q8HPk8f0qN7QdWEdi52v8aAf3qxTgsxP3uE52t+oYw0aysbWxJzgZvBcuIwNyzjNazxn\nXK0Sp9Gk3zUeBszwOwA4dOgtX9Lu2urYsiV2fKf/79unIZNh6lQAXfN6AkE8YAuAdM4+U8CGYRhD\nmIo949kbnvifmqPEIvFcLcHK7F2eNk19wu3tM/yqOqqioSHN6MKXFRUv/XNLl/ioi2NBFuUo88eu\nt8eWnEpnVnBFAYiHHMfXEZqYCunaa7U8b6ZPNumlATudZwfZp5/yI8CnTVPVtmMHlUU6mCCGQo9+\nzHgwQxGgOps/z/t11/tWGw3Ncayh3GT3PW8CfcDx6IGw2RBnZPebUOWGrYiWBp+8yiu2+df4lbzB\nOUVXUQYVMWqDvm3OZMx6QhdtaUQPh36zf6fFlwx7GBmtB/z61ypjE7PHTYSMWRh4C+OBN9XEFLBh\nGEZOVEwBxz3yiazl65xKIOyi9Q6tZcsuCf8Npl3RN1voi6Gw4FDaM8/0X2Q4wQ426TDleG5OdoCG\nCTmKQDnfL9/UtFuYApIthGQeUvrivSFnU7JSIiONpkgVAGVsQ7S8axxwuckhi5DwKDmvOOcj//+N\nj0sPxwrHwbuUbqzvtG84OUA0gyl/Kkns/DNBeA7ngeJ9Y7e+/11N8Qlo2k5JIzNquWVH5Tt7qk+o\nQ7t5ey1bpv0zoQk2bWLMfyxJ6VeP0xOkUVFLlqD0OFkhIv7Gxy3KwUh8ZArYMAwjJ+wBbBiGkRMV\nbwxStm9v1yZRC30GLMO2qF95/lTtaLj5Zg1MZ2uBTcKw9cbZA9gM5qSnJ8/QsLSw6VKuJXlSlCSt\nKMTB63EYXZyLGcgI1OeX11+v5S6fX9kPCweA9b7Ve1LS56YdGmPG6ILQBcEmZZwNjWWt2zrTNcKL\niUOWWIE43xiADr9OI/1ZHItM3w/bwoEB9kLr+X6/+6T/Mx5zC3QdF8vS99gd7NTfWdesd7VDeO9n\nt/oOzs3+uiI/2pw56oLwk7H4ZVqJ7r333QBSL0Laj6bhrIm7AcDNN2vJ24DV3ti8h7wvAA426yzU\njBQsN+S/GpgCNgzDyImK6ZJ4ht7E1x33mrGnDUjf9H5WhqU+tmzXqfpGoroNneDx8EN2/u2MXnDh\nOVA58k3GgPeiDcSIE5jE/TxMQBJ2wkz0Y1pol4O+pTCWnW7eCIebZyXbvO0FXjrfmHaCUOSFAwXi\naCvenzjgvlYVcAjry94OVZUTOJAnvpigE66RhqX88sO4H9/vE8X4iXrDlhnhJuyTu+6aqOMaSI3N\nc/C9qScW6P7bNpaeey0S/n6PHFPNV09bRq2L0ewcDirxF29VeyxbpuFn8RSPvPZQAX9ome/k47hl\njuxgCGAwe8bLPl/YgQNa8rnA846H21cSU8CGYRg5UfFZkeN49Kl+5uGxXgCXhOD4GXmTt6DPxjOl\nSTeeQn9lkKPuV231JfunKqOwDv2TcSB1/CYr2txkPE/6qmJhlpXKj9dItcD709ysaiIRecE2VL5s\ntMQRfqFAi6MNi6Z8w/PjZ9ato9O1tTBlnv+CdTdQT0zMw0Qx61doScXLkglegK5+cw4TP+H1UF0y\nrh6pvPMrb/ezfu+MUljWMqE6Z3ba8872lZVydoOf/YbGDzt+vIP7ErYGZnB8tq9srPhhRb3fG54G\nYrzqZZcBAJ7dmIbtMR0ufytxXa4mpoANwzByouIKOO6hZzKM5mZVwlPTzkeMpfyiLzjKA/jaTu0R\npaoAunYKv/KKlnxrdTd8kIMG6EstWhJrXnOs5HkdVKNZc3BRiVEBc90yE0UD6JogJUsZ0N4sa13x\ndgdtw9YC61qDTxvJgTzhtIbbtmnJmby3bPEJfJLBKyzTZDydnemMv0CakIbuysbGscl3DQ36mf0Y\njHYoqp3Zr0MFupjNLF4QnwU0LNB1jDCNwR9yMIw+gZWXI5B8J8WzbXovwyQ93C37S/j7ItW0tSlg\nwzCMnKjasz2OhqB7hj4gADhwQN9GjY3vLVl330NadhnejNSXxrdUVq4TUk7hFlU9xMSCgB29WUNR\nqerilgltkQzpRtdMinS9ZcVD8j7Q/kNhaqL4GsKpboDS4da0Fd3DTU2TS9bNSphOwcYuDv42spLA\n0OZxBEzR6zCv+cm1qgGnTl0KAGheoOXYzjfTlWlk/qCj4ct7G7Q/IwywSn4DvhWz2z93+EwJbRwL\namLpKA3DMIYwFX/Gx77gOFl4VrIWqrNyk0qGSpaTbsYjr2J/aLhsqBHbmDbN8sPGowE5Wo6+RIqL\nF19Mt2GUBRNTx8o39pGF+xkKNi93DWkESbqMA6pOPz17m1i5hp9ZxnW5N+dSdOK89fEz4PjxtCXR\n0aGf0ygFVby0G/t2GMcLdLVx1vOBZNXnwcIUsGEYRk7YA9gwDCMnBq2Bk9Upw6Ycs+TTQd5dBw6b\nEnHnRPx9T8uGEt01ZdkJlDbtet5fuY6f3pzDUKS75ittPhD7xsfp7hyGKrGrIHwG0O7lXJTxeuF+\n+lKX87CxKWDDMIycqNozv9zbpDcKNest2N/jDWWqbePhaNOQcjbry7q9qcPD2c6VqMPsROuuM61W\nbWwK2DAMIyfEOdf7lUV2A/hN9U6n5niXc27SYB7QbFxdhqF9AbPxYNAvG/fpAWwYhmFUDnNBGIZh\n5IQ9gA3DMHKi3w9gEfmGiHw6+P8REVke/P91EflMD/t4shfHaROR5ozly0RkaV/PO9j+HBHZICKb\nReRvRUT6u69qMQRs/BciskVEanbCnCLbWERGi8hPRWSTiLwoInf2Zz/Vpsg29ts/LCLPexvfLSIV\nm6ZzIAr4CQBLAUBE6gA0AwhyamEpgG6N5pzrt1EALOPx+8nfA/gEgLn+74oB7KtaFN3GDwJ47wC2\nHwyKbuOvOefmATgbwIUicuUA9lUtim7j33POvQfAAgCTANw4gH2V4pzr1x+AFgBb/OeFAP4JwM+h\nc5iPArAfQL3//nMAngbwAoAvBfvo8GUdgL8DsAnAowB+BuAG/10bgC8BeBbABgDzALQC2AlgG4D1\nAC72RtkI4HkAj/dw7tMAbAr+/zCA7/TXFtX6K7KNo+voyNuWQ93G/hh3AfhE3jYdqjYGMBIqKm6q\nlG36HZ7snNsuIsdEZBb07bIGwHQAFwA4AGCDc+6IiFwOVZjvBSAAfiIilzjnHg92d5031HwAkwG8\nDOC7wfftzrnFIvJJAJ91zt0qInf7m/I1ABCRDQA+6JzbJiJNflkLgOXOuQ9Fpz8dSaZQwH+e3l9b\nVIuC27gQDBUb+3V/G/oQrimGgo1F5BF/Xg8BuL8CZgEw8E64J6EGpVHXBP8/4de53P89B30zzYMa\nOeQiAPc5504453YCeCz6/t98uQ5q/CyeAHCPiHwCwEmA3viiPhgCzMbVp9A2FpERAH4A4G+dc691\ne6X5UWgbO+c+CG05jwLw/u4utC8MdIAefTsLoZJ+C4A/AXAQwPf8OgLgr5xz3xnAcXz2WhxHmXN2\nzt0mIucBuArAOhE5xzm3p8z+tgEI5xCe4ZfVIkW1cZEouo3/AcCrzrlvDuDcqk3RbQznXKeI/AeA\n34W6PwZMJRTw1QD2OueOO+f2AmiCNi3oVH8EwC0i0ggAIjJdRCZH+3kCwPUiUiciU6BO8544BGAM\n/xGRU51zTznnvghgN4CZ5TZ0zu0AcFBEzvfRD/8DwH/04ph5UEgbF4zC2lhEvgJgHIBPd7deDVBI\nG4tIo4hM859HQB/am8qt31cG+gDeAO3RXBstO+CcawcA59zPAXwfwBrve7kfgTE8P4b6YV8CcC+0\n+XEA3fMggGtFZL2IXAzgq6JhZRuhN/R5EWkRkZ+V2f6TAJYD2Azg11DfTi1SWBuLyN+IyFYAo0Vk\nq4jc0eurHlwKaWMRmQHgz6D+0Gf9Pm7ty4UPIoW0MXQ665+IyAvQTrw3Adzd24vuiZoZiiwijc65\nDhGZCOCXAC70Ph6jQpiNq4/ZuPoMJRvXUpK2Fb5Hsh7Al4tq0BrHbFx9zMbVZ8jYuGYUsGEYxnDD\nckEYhmHkhD2ADcMwcsIewIZhGDnRp064ceOa3eTJrVU6ldrjzTfbcOBA+6BmSTMbV5fhZl8A2Lx5\nXbsbxBkxzMa9p08P4MmTW3HXXc/09RgldHYGB4+mou4LnOywmpPt3X77kurtvAyVsHHWRJC1Oinh\nYNu4EvadODH9/I4fd7V/f/fbdDfJZHy/wnX7MjFtOa66SgZ1eqBaqsOD8ZwA+m9jc0EYhmHkRK66\niG8nlkePaklVkfXWKvdGO8mnSB45svzxBqK4i06sKOL/afssaFOuM1xtnHVNTU1aNjZqOXWqli1N\nh0s2OjGiPtmGtq/HkdId++bhrkOjk3WPHy/5Crt3a3ngQOm+hgPlnhfd1cd4W9Kb+jkYddgUsGEY\nRk7YA9gwDCMnBq2hGDcbgLR5RdjM6vAziGV1bIwapeXJJ2vZ0KAlmwv8P/4cngMpWjO5p/PPao6W\nczXETduwc5Tb0H7NfpYt2ry7DhJuU8SmcblOMUI3w/jx6TK6HN5+W8spY7zrYfPmkp3VtbYm29S3\nt+uHuIJv1TkCpoTL6OOYodlTpy48DQCwfn32LopO7GYAurrH6KIM6yzQu98z6+dJ3czqFtfzaj4n\nTAEbhmHkRNU1IN8iVAh8uwDp2ztWvBQIO3eWfh9CVbZgQen/oeqlOok76Cgq2KGxpyApxXt6E2d1\ngMUqgTbmtVNNhDY+dEjLadO0pE2pAGm/8Dse55RTSo/31lvZ51GLlFO+sd1phxDa8eDJ2oE21ive\ngxgLIKrDjRMAAE1+SoDRHW+WHmjHjnRdGjhqzsVKsWituZj4ekJ7xS1jKmJ2RGa1mLkN7TJunJZs\nvYTmjOs1W4dxC7oaNjYFbBiGkRMVf6bHKoLKlwohhP7cWPlyG77FngliunfsUEk1fvwp/n9dvnCh\nllTEQPpG45uMb0qqtKL6z8qFlPENTTsCXX3mZNKk0m3DbeL9sVy0SMsZwWRO9fvfLN2IzRZv/CNn\nzwYArA3ScGe1aGqJcvZlPQrrDS+XdTWty6p8N24s3TbcH93CDQ066cOcOVp2Bgqbtupo03KUr++s\n91khWEVQwz31ETRm2MC7yNHWpiWfC1u2sHm1L9iDD/HDCV/q6Jnx4/W+hHWY92HOHJR8x1Z1Voun\nUjY2BWwYhpETVXtX8g1HHyCVQdAZnCjUMX7SEb5VqCp++lMtd+z4QbDn7QCAfftaAACrVn0YAHDB\nBfrt1Vena8a92vyfPp4sn2kReu+zIkqArj4rILU//bl8u9P2WT3/cYuE28xu2qsfHl2TrvzEE6U7\n4EZePtTffLM/3uJkk1pXwISXFPuvw/95LayzVEuMUoj3BaT3iYESL79cum4cHQSk94CtEN43Hi9U\naUVQwCSuf/FgrHAdPjtYP1ObUvluDvZMI1IJ6zRu+/ZN8eWcZM2dO8eWnFPsdo9tXUlMARuGYeRE\n1d+V5RRC1jpUFlQPW7Z4dYXHgrXZQ3w+gK6RDeH+ub9YGdKHxONmRQ3UooqI1Xk8RJjnHvq3eO3z\n5mnJ6xrb6H1j3gjbO1IVQIXBKJLEFjTypCDp0/TpWlJq07h02PmNz7qtOdmkvX1WyfnWGrHypT0Y\nTRP6y+Nokg0btKSKpcloFiC9PytWaPn668zj4m2G8EfybgDAwoV6f+jT5y3oco+Q3bKrNab4YGdG\nJcSRCElkSIhf6USDRpqwim3dqs27TZumJatu3ly6Ke3P30MYyUPK+Xz5nAhjh9mnMlAbmwI2DMPI\niQE9v0NFFqszvjVCtQBkx9/xLU7FwZ5j4KAvTw/2wGFI5wIAbvWTcN9wg5b1nQeTNTuOjS05B76t\nYsWd5ZOsFfWQ5ZPmefONTPHJcw7f7rQtVfIEeD/uA6u09E7Fqa2pAuYxYyXwkxX6vl6w4L3JstnX\nejlHycG0O70AAAAgAElEQVST4E3MCNKcM2dWySp50p1949jUrEgewsuj8o1bHPwfSFt4r7/OJsDz\nvqQSTpUcoM2cV1+dDyC9n3HLL+u3WCt1OKxHSX9Cs/+d8sf5jK8Mq1drGTaPoiGZdX6bWX6dWd74\nS8OKzxtCWXuNtphxvi+Ddd/YXxqxQmUd14Pw2UX7mwI2DMMoKPYANgzDyImKN1LY1I2Ts7Al0Jz2\nxSTNKbYG2BpZ4idJaGu7EgDg3MLgCOqcHz9ew5o+8hFd2rL58S4H2AlttkX9QYlDvprhJdUgbi7T\n9UDYQRMye6pPDkMjPPCAltGAibpg57N8RPqu3fp+fuopXc7BFGGIVWOjhgNeeKGW5336Iv0i6gU5\nOPW05HN71EFSK5Rzo8WEoZTchq4HNrFZhzkwiOYG0s63tEOZMGdwGBalnxmqyd9K/Huq5TC08NyS\nnyddBBxNwUrFMrwIGjMeVxy7t1atSjY5smVLyTnUs9eSD4xrrkm+m+VvUuP5E0pOgfcsa8AWn3NZ\noZ99wRSwYRhGTlTtXcmXUxzMHHZGzG5UNbt3BIdi6vKPfUxLqomVK9O4qrY2/bxsmf6/uPFX+uGe\n+7X0gf8AsNM70amseU7lOpnC8ywCvA7aiS2KxYtOpCut9grjfm8fKmHemHjcN4DX2vS9/MMf6v8r\nV/pd+f6Ro0fTjs5Jk1Sh8X6cc45ue8EFp5Wc085ANdf6QIw4VSHrRFYLg2bkPTjd9xefd/IL+mGj\nyqf1xy5JtkkbBxxHzEEBVL5pJ5yI7pi/m3PO0ZIqvLvOoFoZVBSqxOQ8+ePjlwxnZIUJJ96bO7f0\nu7jXnk3a4ILr2aPGyhtHAwTrnmjyyZFQuvt41fD+0+4DtbEpYMMwjJyouN6LU7nR58M39uytj6cr\n+9fHBO/YmuBXmjpV30gPPqirhf4zvp0uvdQv+Na3tKSsCmTt/siHw5dgVlrFIhEreIoFiogSJy1l\nLJVvPMrC/3/iyquSTdY/ULqbxx6j4mW4VCppdu9WhbZhg6o23ncqBJ5TmDSmVgcK8HzifgyaKk7q\nBKSKlz7a+Z3P6odnvPH82Pi1y9NtKNze/W61HW/N0aOq9BYuTI3FlgXdoDweBzBw21CJZQ0wyhOm\nNwXS32CTV51jGRbGMdb+QrY3zE62oTv3RS9mWS+bm+v9vs4CUPqceM97tI/opk/5pgP7PpiJK+gr\nor3ixmA8KKsaddgUsGEYRk5U7B0ZTeyavGD45m459oZ+WLEq3YjONspjL+nWb9W3I3vfwzco1cgV\nV/gFK70EoAM5yEfZHvXaU8HQl9Pd1Dq1SPzWpQJmq2OK+OGbdNYCwL//e8k+DnuZMDqS/3WXXZZ8\nZnpE8u53q2/y9dfnc+3ku5EjVYpRkcUJtRmpEUa/1LKNs6CdY0UMpENqp7zj6/fDq7T0iu5X+9WW\nYU86uyliF+bEiSqxgluRdNbXHdOkMoePpbMrA2mdphIO91srdg4HsDCNJs/x7XFqnz1RQMOmTek2\njL6hOzeOYjp6lOFAqfP+D/5ADXPTHf5m8fnAeh+EssT7i2eMquZzwhSwYRhGTlT8HRmrtJYmH4d6\np3eC3XdfujIdZ5Sz5+rw4jhxRjgJIpXU7P3e1xaPFw2kRnu7KmkKbSqZ7nowa20YZ3d0iU9clBHW\n4Y3Y7o1K9+UMv/EIOs4Cv/HEiZf7Uv9PGxdBz7Qnw/VesrxWE+70hrhqxREPADChU9OjJj5GNq98\n02/TKv03VM1spDGcNU4/uXRBGmWCex8oOeho+ko5DLdJDb9/fxo7HHf4501vhnvzN58Va84G3b59\nDB9RZ+3Ro/wBvObLVAH/4Aca7/utb+mw99FRHtbDI7oOvY/TJ8TPgqyoKYuCMAzDKCgD0nlZKfCo\nUJMwPjpz/Ktta+DcoTdrMnP6+dfJLZ/6FABg0SIdXRX6t35nmVcHX/G9+3xtMe4vmL9o0SJVcnFc\nX3ejV4qgfKnoKV5pn8ZGTdP3/ttuS1f2b/5mr9Ca4xydVFSBk3aM/4rx2Lt2aRnHdgOpueNER2Xm\nkqw5ukvGHyufrJSnSZw1g3sv0pGAr/kpidizHsa/8zOPx00TZR1KWHaA8P7w/kUybUSg6GqtDnfX\n0uRlMI0nlW84hdW+fUxSRKXLBOyv+/IVX6Z9F5yAYDR8C5w/Fm+crAk8k+mfojI2fbCbAWMK2DAM\nIyfsAWwYhpETFWusxB0KyaQJ7aU9GGGL1GemRZP3wNcvX16y7mI/7nLx/j3pRp/ysSivvqol45wY\n0B20D5f51jWbeLHDvwgdRN01kXn+9OrwulauTEOVGhquAwB88I+1ZIcmt0n22db12Awh5IzToxt0\niPOTa9P3NqN52FyLR4kOJegNqB8RDPVmW5kG9QMvZs/QsLH9+/VehPeRzV+2ivnbSSKjngl8bpx6\nIez5C3fofyvHaniG77DexvWC3sc9/idOl0RpRyKn9PYxf8l8b35QBTjf4OXJFl/4gv/A+8Ob53tA\nNwdhbvG57fMejngm8fA6zAVhGIZRcCruro9DOpIYpjvvBAA0B2ng2DGUvHrYO8G3PSO4wwm16K1n\nLBnHavpy16lLk1XXrdKSLz9uwnPLSq5Sy8QdWnGHFzuJwkyQ7KBj5wY7zbZsYajTPr+PdyXbXKlZ\nQJO3PE3MjWfMSGfEiNVcd7P1FoV41mxWv0SEhgam8mUle/ppLf2FL/Yts589nGodijJWew6rH71/\ne+kKQHpT43HQfv+HG7XjqVYS7/SVOAQvngEZAHbs0Gvs7GQ8KjvbtIk7Zoz2uH3+8+k2t9zg6/dy\nX/H9c+i/Nuq2YZhbPNR85kwtB+M5YQrYMAwjJyqmgOnDoRCgD6e5WQdD7Icv581Ptpn6zVsApG+e\nOFysruMguhBPguXHdb60X0PW7v9Oumo8UIGb8jjxENOiwNAmqgTaL2vuKn6O57tK5x/TBZ2dYXb3\n+SXbjkXpfQhHMbOBw/vORkscslXrKShD4lZcnL+oJC7Sf9nhVWsjFbF3IB5coC0y5kQCUoHLBl+S\nQZXDxkN5xkFKbGJEOUgpxrOShtcyvIx4kBRDvkJFz5bHzp3ahO3sVMXLPoqbbvLlBW+kG3HOQ/64\nvR2f/ob+G05mwGPzXOJGR5ycqZKYAjYMw8iJij3T6c86cKD0f+YB59sk7AWNJ9KN82RceqkGlzdf\nfV2yDWc9PuiTV3OYIve1Zk26/3jm2FgpVvPNVg14/hzBzWBzKgT6KtmzDHQdRMB1t2xp8WtQCqQz\nT9NN38V/6+Xu2IYjyaKORu3lj5PxFGlIdwzVZCh0gaDuNned/4cCv8NvPNXPOzTWj2a56KIJySYU\ntbTz6OV/qx84V1HYhOGN40l52fzsJh10U2vDjnsLIw3Yco5TGITPCYr/ONEXWxA33uhXvGdluhFX\n8g72796r9fQ//1MXh3WbLZ14uieuYz5gwzCMIUjF9AnVJN9SfGHzf/ogw6TJsX+S6ozJSpi6Lokp\nBjBnTqnyJTxOmPqOfp5YAReVeKLTOBk43Y9hyChtGyfHWb1ax4pv3arlDTek23B0ctITHUnhE8F7\nm0qXvmCq77glVAToW48Tc8eTui64LY0CmfW7vwsAmPDoo7quX37isccAAHVf+QoA4A/D4eGMYb/t\nR1ryxlHS0bkZLvM3479W12eeU5bfvxZhfWH9oILn9fD5wHsQrhPXNfrk69p9GtYgFe12H6nDeTrv\nuUdLthKz0qPymPyfx2PK0WpgCtgwDCMn7AFsGIaRE1XrImHzjU0KuiTCcJl4MlM2C7icLbOgZZF0\ntnEdDuOku+H0tC8pWRaHncV5gWu5oygMx6Ht4rnsRu/ULFGLR6ifYcEVZyXf+ZGxXVxAcTMutDG/\nq9voZ/bl6A2/Ul3g45jVrCez65B2CkWJ7WpudoaY0L7xoAC61ei+yZpg95vf/GMAwOh16wAAk7/3\nPV3Hfz+ZcxaGPjPumJWYI1/oemAaOgDPblaX2zP3dD1fIK0LtWrfGF56XD/iAT2vvhreDO14F9Ge\nXmZa5PPijU4dXDFjSZoNbeW9WtLscRLA0K1AG9LVGYdQhnPBVRpTwIZhGDlRsfcm3xJ8w8Rqk0or\nDJGKg7D59qPi4FsyzKXKdbgNE8Vk5Z8NO/zCcyji8FggHewQq4YrrtAZZOvu/1cAQH2QE5lGpCY+\ny3fmXD4j2snGwChP+96IF18sPSANF8owr4rZqcKwojgMrQgKLQ7zY2gd6yk7isL5ylhXb7/9uwCA\n87x6bWTvTzwVN5D2bvphygdb9e489JAufuqOdFV2DPFc2Pjg7njORbAv0LXVGT8nTkomtTgabKVf\n0mwMv2TVpcoNf/txlY3DMTncOOs7ljyXatZhU8CGYRg5UfFZkemTJfEbOgxu37OndN3Yx0m/JDNN\nhvvhWyv2ZYY+Zi5LZ4wo3UcRCM+V6fHiYb2M3/8dZs259970y9ihTiPE0zKEN27bttLvaGTPkSVp\nwiP65MsNCKh1W2edH32MvHyqJTYIDh1Kh2b/4AedvlSJKnKJ36+WWSFhvAVx2kMqvLDFx2Xx4IAi\n1uWQuDHF6+P1TpqUGu6UU7hMS/bzxM+L8HdBG3K/vA+0OVV0SOaMJ1XGFLBhGEZOVFwB820eqzX2\nfoaDBOij5YALKlYKLirfsIc+jgDgNnGOHqB81EBRiYPu44DxZ7dqL/C82z6TbDN64y/1Q5jiEEgN\nx52ETYczztDSz29GAz71nA4C2PVwumo8vDvupS/SkOS4DlM9hfUPANauTedf27hRPzu31Zda0Y8e\n5cy97M1Pk+QfOqT3ib36Z5+ty1nvwwZH7JfksNgi+dZD4rpLPyuX89rDesTvqGrp36Uy5v0KW2Ec\nTBRHYfH/cCAGj8X9ZaVNqBamgA3DMHKi4u/P+I0c9yyHb3e+seK3PMMhqZbDNxF9RHGCDro44wQq\nQNqTHMfzFU09kHI25rWHYpbJ08fdpOWUkXtLV/JS4HAwqy7dxmu9bzmO5Q39muV8kUW1bQivk2qJ\nqipsxXHZ/v26MJ5RlzCCBUh9mbF/l/sN1RkVbzVjUfOA9YPRHfTJ8trD+nNas6+zrJhT/YMins54\nTpDPdrM+XMZyhxfNApBGTISpb2OlW2527GpgCtgwDCMnqvZsj98i7FnOmmSS6oC+HK6bFXbqc54k\nUFlT0IXrFmnEW1+Ifb+Eii2MtaY4SH2GE/y2WrIVQj98CO0W98AfDUI0h6LyLdfCYL0M+xTKTUIa\nj/TKmtCR/k/W+6z0h/yuiHHVvSEeNxC3hgEAa32CejZlqYTj8KZwtGGU23ZEqyrgLIUd+3wHs9/C\nFLBhGEZO2APYMAwjJ6ousmMZHwY5M7wn9qVzmziHbfgdS7ov4nCWrHWHKrQPXQNhUpkwr2r4Xey+\nCAPTy3X8dGfPoWzj+LqzQqRCtwzQNbFPFqyzvekcruUcv/2hXH2hSyzsSG6a934AQMsIn/eXDwbG\npXGjsKeTFdo/ZHYGo/OBbJdQHs8LU8CGYRg5UbVnfbnOmaw3ebmwj3h5uGy4qNvuiG3QnUrqi417\nOt5wodz19qYF0Bf7Dje7Al1nnu7NrNltTUw3Oblkm2Rwx/h03UZv0/bVpcfLsnWeg4VMARuGYeSE\nOOd6v7LIbgC/qd7p1Bzvcs5N6nm1ymE2ri7D0L6A2Xgw6JeN+/QANgzDMCqHuSAMwzBywh7AhmEY\nOWEPYMMwjJzo9wNYRL4hIp8O/n9ERJYH/39dRD6TvXWyzpO9OE6biDRnLF8mIkuztukLIvITEdk4\n0P1Ug6LbWERWicgrIrLe/03ueavBZQjYuF5E/kFEfiUim0Tk+v7uq1oU2cYiMiaov+tFpF1Evtmf\nfWUxEAX8BIClACAidQCaAZwZfL8UQLdGc84N5AG6jMfvLyJyHYBeRCDmRuFtDOD3nXOL/N+bA9xX\nNSi6jf8MwJvOudMAzAfw3wPYV7UorI2dc4eC+rsIGt3xbwM4ly4H6NcfgBYAW/znhQD+CcDPAYwH\nMArAfgD1/vvPAXgawAsAvhTso8OXdQD+DsAmAI8C+BmAG/x3bQC+BOBZABsAzAPQCmAngG0A1gO4\nGMCNADYCeB7A4704/0YAq6GVdmN/7VDNvyFg41UAluRtxyFu4y0ATsnbjkPZxsE5nObtLZWyTb/H\nfjjntovIMRGZBX27rAEwHcAFAA4A2OCcOyIilwOYC+C9AATAT0TkEufc48HurvOGmg8d5vIygO8G\n37c75xaLyCcBfNY5d6uI3O1vytcAQEQ2APigc26biDT5ZS0AljvnPpRxCV8G8HUAh/trg2ozBGwM\nAN8TkeMAfgzgK87X5FqhyDbm9wC+LCLLAPwawKecc7sqY53KUGQbR9wM4EeVrMMD7YR7EmpQGnVN\n8P8Tfp3L/d9z0DfTPKiRQy4CcJ9z7oRzbieAx6LvKfnXQY2fxRMA7hGRTwA4CdAbn2VQEVkE4FTn\n3L/37jJzpZA29vy+c24hVHVcDOCj3V5pfhTVxiMAzADwpHNusT/vr/V0sTlRVBuH3AzgBz2s0ycG\nOvqZvp2FUEm/BcCfADgI4Ht+HQHwV8657wzgOO/48jjKnLNz7jYROQ/AVQDWicg5zrk9WetC37xL\nRKTN72+yiKxyzi0bwDlWi6LaGM65bb48JCLfhyqbfx7AOVaLotp4D7QFx4fOfQA+PoDzqyZFtbGe\nmMh7AIxwzq0bwLl1oRIK+GoAe51zx51zewE0QR9wdKo/AuAWEWkEABGZntEb/gSA60WkTkSmQJ3m\nPXEIwBj+IyKnOueecs59EcBuADPLbeic+3vnXItzrhX6Rv1VjT58gYLaWERGsEdaREb6a6jJaBMU\n1Ma+KfxgcJwPAHipF8fMg0LaOODDqLD6BQb+AN4A7dFcGy074JxrBwDn3M8BfB/AGu97uR+BMTw/\nBrAVWnnuhTY/DvRw7AcBXOtDQy4G8FUR2SAaUvYkgOdFpEVEfjagK8yfotp4FIBHROQFaOfHNgD/\n2NuLHmSKamMA+FMAd3g7fxSqKmuRItsYAH4PVXgA10wuCBFpdM51iMhEAL8EcKH38RgVwmxcfczG\n1Wco2biWMpGu8D2S9QC+XFSD1jhm4+pjNq4+Q8bGNaOADcMwhhuWC8IwDCMn7AFsGIaRE33yAY8b\n1+wmT26t0qnUHm++2YYDB9plMI9pNq4szc3NrrW1tVq7LyTr1q1rdxWcIcNs3JXe2rhPD+DJk1tx\n113P9LziEOH225cM+jHNxpWltbUVzzwzfOzZG0SkotMFmY270lsbD3oURNYssZWcjTTc/3CcbRYw\nGxtGUTAfsGEYRk4Mmn7JUmXxdyyPHu15f8ePa3nSSaXLR44sf8xYrQ0n9daTjUO79WT/LBsPZ9sa\nRn8xBWwYhpET9gA2DMPIiVw74eKm7ltvadnRUVp2dqbr8DObuOPGaclmcdj05eeGhtKysbF/517r\nxG4GILUx7RPbtDvXEKHdSJZ7YdSo0uN0t65hGIopYMMwjJyouj6JVVlHMAVme3tpudOn1Ni8uXSb\nUIGdfrqWVMubNmn59ttavvNOuu6MGVrOmaMlY8W5LtVarPCKRnc2ptJlye/27y8tdwbpTLgO1WtT\nU2kZ2outiebm0v/j0pSwYXTFFLBhGEZOVEyXlAv54vIspRX7IalQzz9fS/p3qXaBVM3GaoxKOFRa\n/I7rvvpq6TnGSi/+XKvEto7V/44d6XdtbVoe8Cmr2dqgvbZs8fF8CGdkocEP+pJzP2qTYtKkNPaP\ntp03T8sFC/yaM0rPMfS7F8HGhjEYmAI2DMPIiYprkVidUfmScOAE1RP9h0t8WoCpU7WkQt26tes2\nEydquWdP6XHDdanGqJqpBqn+eJyiUE75soXA1gWvM/zM1gbt95GPaHn++XpDFi1Kp95iRMPJJ2vJ\n+7DWTyazalW6f7YqYmVNlRtHooTLDGO4YwrYMAwjJyruA2bcKUsqH34/JphiL+5Bp4qiaqZafuCB\ndBsmXaIq436nTNFy4cJ0XR6bkRNUwjw3+kVDlU6FWMsqjcqX50+VSwUc+oBjP/j112t508Xb9cPd\nd2t5w53JNge9gcbOnaull8vX3XwzAODo0dOSdadP1/LFF7WMoyx43CwfcC3b2DAGA1PAhmEYOTEg\nDZI1ioqxtbECotoJR7VxGf25jIJ4/nktly/X8v77022c46zW7KFXeTtv3rsAADNndj0/qu7Yp3kg\nYzLrcsll8iK0MZUv/d60cdwamBSkgWbr4mMf0/KsES/ph+XeqI8+quU55yTbjD37bP2waJGWdM57\nOR22Yn77t7XkvXvkES17k1DJMIY7poANwzBywh7AhmEYOVG1hnY8LJYhXxxcERK6DYC0M279ei2d\nez74ljN9sFdHfRqbNmmbd9myNBvMDTdoOfqYd1f4nqDm5jpf6uIwRIqfQ1dJnoQuCOZAjkO72Pxn\nsz+cnuuma4/oB/ZksseO5b/8CwDg/z4wO9mG9l/gTezHWGDxVN3XOaPS/U8ZcxgAcOjQaABphyfJ\nSnxUa24ew8gLU8CGYRg5UXENQhUWd7rx/zBdIQdNMMD/wgu15AAKDkl+9dVwXKu+MxoaNN6MqpAD\nC77+9XTN0ft9qJXvCTzYodsyXKu7gRjHjgEyqPMh9wzD8qjc2ZpgpxuH/54250S60b0/9OW9Wv7W\nbwEA9n7tuwDSVsJjj+0LjjQWAPDhD59Usl9K1ynvBOPJ/1tv3txLfw8AsGuXLmYHYVZCpXCYunNl\nLtYwhgGmgA3DMHKiYgq4nD+PajMr8c1DD2l57rlaUvGe1fQGAODWW2f5bScm22zcqJ+pyuhzvPHG\nrsd+rbMFAHBsK7fVkj7SOGFQSJjSsVZg64EKOE4TOWuEV/zrA4W6enXpTj76UQBpa4O+8zFjxier\nMLHOpZdqecmCvfrh/hVaTkzvBw1Fhcv7QtseOtTzdRnGcMUUsGEYRk4MSAGHajbL1wekne3097KH\nHUgVKf2GZ56p5U0zdeVFi1QBcxwAkCosqliOFWDERCj44mGwp5yiJf2TWREP4eda8E9mpcrktdPm\niS97Y9TcCL/0jvWfb5pVsv/3vU9l9aggsoGDK265+k398Pf/qCXHG4dhFv6msaVD21L50kfPQTDx\nNdWan90wBhNTwIZhGDlRtUjMeAqcNAF4mPhbZetzz+kw4m9/W7PlXLtyKQDgmN/2mmu67p8pEe+5\np/T/UIFTqF19dem29KUyYqOcOqtVaFu2Buo7fZxzVpYbZiDyCrjVtwY+9zkt2Uqg/x0AWp77qX64\nw5ePPaYlxyBfdVW68vveBwDY7Fsz+3wwBVNkZk33ZHHAhqGYAjYMw8iJimuQOO6XpJEGYbzpmyXl\nL37xbgDApk0af0rxFiZZZzrKhx+G3+aJaF+LknWbm3V/9EMzeiBOTl4UeN60R+L7pXHp7A4DnPmd\nz1F52rRntVwWhVBwJlQgnduIhqNj99prAQBHbvz9ZFXeB/qAy/UFhJjyNQzFFLBhGEZO2APYMAwj\nJ6rWGGTnG5vN+xLPw9vBWvN9qc3g971PXQ8cisx9hDMpx4l6AN9RBE42l06JkRX6BqRDeovggshq\nrsczTDfOmQAAqPPGPtGczu9Wt8DvgMaIEjUfbtBtN7j3JtvMPF8/t9Dfw95Mn1T4hz9Mz2XlSi3p\n9eAYjXiQSzjYxVwQhqGYAjYMw8iJqnXCxWFoDMhPkxsCY8ZoPNhtt+n/f3OzdhBhuSqvsV5xHTtW\nn2zDWSHY33TokB/HnMyQkSbu6UlpZQ1BLgIMn+NAFirhJUt06PXojoPJunuhCnfCHN8r5m/Iax2q\nklf7iTFCW5wnv9QPbCL4kTCP79S54Kh6gVRYs4OTIX2mcg2jZ0wBG4Zh5ETFdQqVFBUwlREHDYwa\nleaj/Pa3tXz/iMf1w70+afiyZQCAFzap8g2HF9MtyVSMO3Z8wH/zii/fStYdMUJVX5y4nOq56CqN\n6pNKmL7u6dPHJuswkmzkyNF+Gy3jRPnh2Ar8h3ewc2z4lVcCAB7+gv4bRqzx/saJ1+MxIUW3tWFU\nA1PAhmEYOVExXULly052llTCjGy49dZ0m/e/7CXwtm1afvrTAIBn2zVhzLe+qYtXrOh6PA7SEFG1\n59y7/DdHknWoCHkO8TgFnmNRoI05mzN9v7w+RieE1xWn1WTLgaOKmcwonCF6ir9Jhzv1/bzS25+J\n80O1G896TXhOWVNQGYahmAI2DMPIiYop4DjqgdD/SgV83nnBl8emablQY3f/eZUq32965fvcc5yM\nMw3kHTnyAyXHoW95y5ZgriMP/ZMxWbGpRYDRD7Gfnb7geL5NANi9mxERjJPWG3LxxSeV7CPcZsUK\nfS9TSVPdMmVlOK0UWxP8jlEq3Q1FNgxDMQVsGIaRE/YANgzDyIkBuSDCJjxdAhzmy44alnQHvPJK\nus26husAAFP9Opy497nnmP7sJV+mbd54+HA6TFnb0iLTku84txmjqeIOo3AWiCLAwSzhrB1AOvsE\n3Qip2wFIw/K4TO3T0aGdl7QFbQ+kYWa8v7TjNG/a0LVDFxDvM8+NLggbmGEY5TEFbBiGkRMV0yVU\nPOP95LpUa1RLcWpZIE2oQxVF5TVypC5oavpwyb4B4KKLtKTSYqhZQ4PKs0VpOuBkVt9YpfEc2JlU\ny+osbGWwdUF7UNEzpCy1UzDFR6J8j/h1SpUvB7aECY+oqOfO1TKehy8MQ+M5UemyE25k1z5RwzAi\nTAEbhmHkRMVmRY6H96bKtLQMFSrVMUufeydRXNwXh9oCqeqjeuaADCpFhruF33E/ocoLz6mWCW1M\nlUkb8JqZAjK1RSo/d+zQlsGYMWpU+nOpZmmTeWmOpKQlwmWclTprwo04VI19AUUL8TOMPDAFbBiG\nkRMV835SAcX+yXiOuBlptshEsZ0247B+4IiCTV4+e8m1dE4guRqYYGdsyXHiEkhVGHcbz4JctNl5\neV/OA2AAAAV8SURBVJ5Uojz/M87QktcVRimMGFGa5J725yzV/D9rNmnOlMxtqXz3BdP60d5xFMye\ncPJrwzAyMQVsGIaRExVXwIRqKU5LGa5H1fTkek2R2NCgCb+bZ2hJdVaHE8k2e/fXleyHCixN0J7u\nnwlmeA5DJSaV5x+ngqTPNkyAc95C37qgY9ffmF0f1ITszIM0fXq6DdUrWyj1W1/TD74lMX7G7GRd\nquF4eDTPKR6abhhGiilgwzCMnKi4FozVJX2LccIYIO2Bp0ritlRw6civ9D0Rj2KjqqXiCn3AQz0Z\nOG0bX/tZC9IWA9Z7Z++aNVr6xEdTLlL5POUdf2N2pQHaUxL5Gg0d9DI3jHCggua5xJOxGoZRHlPA\nhmEYOWEPYMMwjJyoeuM87jAKm69cFgfvx0H8oQuBTV2GO/VmOPFQdUHEHVy020ub0vdqJxbrugu1\npM0713MfmoO53c9CEu6XLo05c3Rm5WSwxab0mPHMJ4Zh9B5TwIZhGDlRNW1YTnVmLY+XRX0+/TrO\ncKDcPGxhRyehLTlEvJzNga4zHDOCbTjb2jCqgSlgwzCMnBDnXO9XFtkN4DfVO52a413OuUmDeUCz\ncWUZhvbsDRW1udk4k17ZuE8PYMMwDKNymAvCMAwjJ+wBbBiGkRP9fgCLyDdE5NPB/4+IyPLg/6+L\nyGd62MeTvThOm4g0ZyxfJiJL+3rewfYfFpENIvKCiDycdYy8GQI2vsnb90UR+ev+7scwhioDUcBP\nAFgKACJSB6AZwJnB90sBdPvjd871+8cNYBmP31dEZASAuwC8zzl3FoAXAHxqAOdSLYps44kAvgrg\nA865MwFMFZEPDOBcDGPIMZAH8JMALvCfzwSwEcAhERkvIqMAnAHgWQAQkc+JyNNeDX2JOxCRDl/W\nicjficgmEXlURH4mIjcEx/qfIvKsV6zzRKQVwG0A/peIrBeRi0XkRhHZKCLPi8jjPZy7+L9TREQA\njAWwfQC2qBZFtvFsAK8653b7/1cCuH5A1jCMIUa/Q+udc9tF5JiIzIKqpDUApkMfGAcAbHDOHRGR\nywHMBfBe6EPvJyJyiXMu/AFfB6AVwHwAkwG8DOC7wfftzrnFIvJJAJ91zt0qIncD6HDOfQ0ARGQD\ngA8657aJSJNf1gJguXPuQ9G5HxWRPwKwAcBbAF4F8Mf9tUW1KLKNAWwGcLp/kG8FcA2A+ooYxjCG\nCAPthHsS+mDgw2FN8P8Tfp3L/d9zULU2D/qwCLkIwH3OuRPOuZ0AHou+/zdfroM+RLJ4AsA9IvIJ\nACcB+gDLeDBAREYC+CMAZwNogbog/nfPl5sLhbSxc24f1MY/AvALAG0Ajvd4tYYxjBjo4FL6KBdC\nm8dbAPwJgIMAvufXEQB/5Zz7zgCO844vj6PMOTvnbhOR8wBcBWCdiJzjnCs3M9kiv82vAUBE/hXA\n5wdwftWkqDaGc+5BAA8CgIj8IewBbBglVEIBXw1gr3PuuHNuL4AmaBOZnUOPALhFRBoBQESmi8jk\naD9PALje+ymnQDt/euIQgDH8R0ROdc495Zz7IoDdAGZ2s+02APNFhCNVLoM2yWuRotoYPAcRGQ/g\nkwCWd7e+YQw3BvoA3gDtmV8bLTvgnGsHAOfczwF8H8Aa70O8H8GP2vNjqJ/wJQD3QpvRB3o49oMA\nrmUHEYCv+g6kjdAH0/Mi0iIiP4s3dM5tB/AlAI+LyAtQRfyXfbjuwaSQNvbcJSIvQR/+dzrnftW7\nSzaM4UHNDEUWkUbnXIcPX/olgAu9r9KoEGZjw6gtainB4Arfs14P4Mv2YKgKZmPDqCFqRgEbhmEM\nNywXhGEYRk7YA9gwDCMn7AFsGIaRE/YANgzDyAl7ABuGYeSEPYANwzBy4v8DndVjb6GoKXYAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024e845cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 1000 optimization iterations\n",
    "\n",
    "After 1000 optimization iterations, the model only mis-classifies about one in ten images. As demonstrated below, some of the mis-classifications are justified because the images are very hard to determine with certainty even for humans, while others are quite obvious and should have been classified correctly by a good model. But this simple model cannot reach much better performance and more complex models are therefore needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have already performed 10 iterations.\n",
    "optimize(num_iterations=990)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 92.0%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4FNX9x/H3F1GqKMVCqBqkiYqKWOJPLIAoFlQi2DVi\nxYLGbjQawYJR0SS2mGABS1RAxYINxIIoBFCaIlYwKCio2JXz+2Pn7My93Ht3525fPq/n4WF298zM\nuffcOfudM6eYcw4REUlPnUJnQESklKjSFBGJQZWmiEgMqjRFRGJQpSkiEoMqTRGRGFRpiojEoEpT\nRCQGVZoiIjHUzWTnFi1auPbt22cpK6Vh5syZK5xzmxQ6H/miMi5/KuN4Mqo027dvz4wZMzI5RMkx\ns48KnYd8UhmXP5VxPLo9FxGJQZWmiEgMqjRFRGJQpSkiEoMqTRGRGFRpiojEoEpTRCSGjPppFqOV\nK1cC8PHHH1ebpl27dgDcdNNNAHTr1g2Ajh07ArDddtvlMosCrFq1CoAmTZoAUKeOvr+lNOgvVUQk\nhpKPNCdOnAjAE088AcCUKVMAWLRoUbX7dOrUCYAPP/wQgB9//LHC52vWrMlyLqWy3//+9wA0atQI\ngCFDhgBwwAEH5OR8n3/+OQDNmjUDoG7dkv/TlwJRpCkiEkNJfN0uXrwYgH/84x8A3HnnncnPvv/+\newDiLEX8zjvvZDF3Uhs77LADACNHjgSgV69eOT3fqFGjAPj5558BuP7663N6PilfijRFRGIoiUhz\nyZIlQBgt1Fbnzp2B8Gm5FE6bNm3ycp7nnnsOgBtvvBEI268VaebOL7/8AsAHH3wAwNixYwH45ptv\nqt3nsMMOA6Br164AbLzxxrnMYkYUaYqIxFDwSHPFihXJbR9J7r777gD069cPgA022ACAjTbaCIDG\njRsn91m9ejUA++67LxBGkTvvvDMA22+/fTJtgwYNgPCJrRTOrbfempfzTJ48GQgjTN+WKtkxZ84c\nACZNmpR8z/doeeWVV9I+ju8z7Xu2XHPNNQAMGDAgK/nMJkWaIiIxqNIUEYmhYLfn3377LQB9+vRJ\nvudD/QkTJlRIu+uuuwIwa9YsIDE9v+eHS7Zu3RrQcLxiN3fuXAA+/fTTvJzv+eefr/D6z3/+c17O\nW+58t7/Ro0cDMH369ORnm2++OQAnnngiAJdccgkAG264YYVj+AEHAOPGjQPgyiuvBOCII44AwkEQ\n9957b3Z/gAyohhERiSHvkeZPP/0EwJFHHgmE0SWE30i9e/euct+qVsxr27ZtlnMouTRt2jQAvvrq\nqwrvZ/vhnH/w4zuz+4eAe+yxR1bPs6556aWXALj44ouB8Hr2A08gjDD9A9zqtGjRIrntuxo1b94c\ngKFDhwJhBHr22Wcn0+644461/wGyQJGmiEgMeYs0fdegq6++Gggn2Nhkk3Dp4fPPPx+Ahg0b5itb\nkge+7AFuuOGGCp8dcsghAJx88slZPedjjz0GwOzZsyscv5g7TZeCAw88EAjL9IorrgDgtNNOy8rx\n/cQtvg3Tt5UuX748K8fPBkWaIiIx5C3S9E/Er732WiCcCPjll19OpvGd16W8nHPOOcntypOl5Opp\n9r///e+cHHdd54dCmhlQcaBJNqy//voA1KtXL6vHzSZFmiIiMeQt0nzttdcqvPbDG33/Sik/jz/+\nOAAPP/zwWp/5nhB+2Fy2+Kfyn332WVaPKwm33347EJbbNttsk9Xj+2kgZ8yYAUCrVq2AcGh1MVCk\nKSISQ94izUceeaTC66effhoIRwAAHHTQQUDFSTak9Hz99dcAXHXVVcDafTIBxo8fD0D9+vWzem4/\nHZl/au75voOSmVNOOSWnxx8zZgwA3333HRD218x222kmFGmKiMSgSlNEJIa83Z77zqm+q4If5ha9\nPR8+fDgAp556KhDOifnJJ58A0KFDBwC23nrrtY4/b948IJzcQw+YCsdPxOAb86N8Z/ZsP0BIxQ/P\nk+LkuyL6+mDYsGFAOBimmCjSFBGJIW+R5nnnnQesPYwu6tdffwXCwf/RSQDStemmmwKw5557AvDg\ngw/GPobUjp+920+84m211VbJbV+m6623HhCuIuqnCqyK7/DsJ9/w/CQf/u6lKj6y3XLLLVP/AJJX\nfv0mCP9m/Dpef/zjH4Hw76SYKNIUEYkhb5Gmb7M4/PDDATjqqKOAitGDX3XSR5y14dvTfIfq6MqT\nf/rTn2p9XEnNT5Lx3//+t8L7vv0aYMSIERU+82XtO01XxXdB85NQe36VQz+JBMCzzz5bIY2foKOm\naFTyy9cFN998c/I9v8bXP//5TyDs1F6MFGmKiMSQt0jTt03stNNOALz77rtrpXnhhReAMPr00069\n8cYbsc/n28pmzpwZe1+pnUcffbTK9/2SJFC7dmo/DZmfcLpJkyZAeLfSo0ePZNrKU4hla8oyqT1/\n93fBBRcAcP/99wPQt2/fZBp/l1KMbZiVKdIUEYmh4OueR+2zzz4VXvuhcD7S9E9RTzjhhGSak046\nCQjXTfbfYpJ/l112GQBnnnlmhff9NIAAm222GbD28hZ77703EN6JRPXs2RMIh9b5ZRKmTp0KwN/+\n9rdkWt83tHv37gB07NixNj+KZNHAgQOBcB307bbbDgivXQinnFu1ahVQ9dI2qXz55ZdA2K/bnyfb\nFGmKiMSgSlNEJIaiuj2vzDcU+46v/gGRX3MZYNGiRQBMmTKlymMUc9eFclN5+KvXsmXL5HazZs2A\n2q0D1bRp0wqv+/TpA4QPEaL8cFqtBpB//lbbd1B//fXXK3zuV6D1Aw8gXCtszZo1QDirUeUBDVH+\n2u7Xrx8At912GwDff/99hXxkmyJNEZEYijrS7NKlCwCDBg0C4KGHHlorzeTJkyu8rls38SP1798f\ngOuuuy6XWZQI/7uv6mFOLkXXGfLrEUWjW8kvH1neddddQDioJJ0HM747mZ9vt6phtn7tej9Qws/f\n6v/+ahqSmw2KNEVEYijqSLNBgwYAjBo1CgjbKKId1v1aML6LwrHHHguEHeOl/Pn2sMrbUhh77bUX\nEA5g8W3Rcabn8+3VNfHDZ3279UUXXQTkri3TU6QpIhJDUUeanu8QPXHiRADuu+++5GfTpk0DwsjS\nTw0nIoXh2xb9pOG5UrlnhK8n/P+5okhTRCSGkog0KzvmmGOq3BYRyTVFmiIiMajSFBGJQZWmiEgM\nqjRFRGJQpSkiEoMqTRGRGFRpiojEoEpTRCQG81Mv1Wpns+XAR9nLTklo55xbZ2aFUBmXP5VxPBlV\nmiIi6xrdnouIxKBKU0QkhhorTTNrbmazg3/LzGxp5PUGucqUmS0xs7eD80xPI/0QM1sepF9gZn/I\n8PxjzGxAGun2MbM5ZjbPzF7M5JyFUqgyDs5d18zeMrMJaaQdHsnb22bWP8Nzv2Jm3dNId4SZzQ/K\n+N5MzlkohShjM2tnZlMiv7sz0tgn79exmV0U+V3MM7NfzKzG1fhqnOXIOfcF0D04+BXAaufcXyud\n1Ei0ja5J5weJ4f+cc6tipB/rnBtmZpsDc83scefcikg+6zrnfslW5sysGfA3oK9zbomZleREngUu\n43OBuUC6S1Ne75wbZWbdgMlmtqmLNMrnoIw7A+cBuznnVqmMY/kZGOacm21mTYBZZvasc+7dFPvl\n9Tp2zl0LXBsc+xDgNOfcVzXtU6vbczPrEHyDjAXmAW3MbFXk88FmdlewvZmZjTOzGWb2hpntUptz\npss5twz4EGgbRCf3mtmrwN1BZHNjkI+3zGxIkMc6ZnarmS00s+eAFmmc6mjgP865JcF5P8/Rj1QQ\nuS5jM2sH9AFGx82bc24uYEDTIJq4zczeAK42s8ZmdneQj1lmdmBwvoZm9nAQwTwK1E/jVCcDf/Nf\n3irj9MvYOfepc252sP01sBBIez3tPF7HUUcAD6RKlEmbZmfgJudcV2BpDeluAUY653oAhwO+EHY2\ns9ur2ccBL5rZTDM7MU6mzKwD0A54P5LPfZxzR5O4CD53zvUEdgKGmllbYCCwBdAVOAHYLXK8EWa2\nfxWn6gg0N7OXgj+ko+Pks0TksoxHAeeTKOtYzGw34Afn3JfBWy2BXZxzFwCXA88EZbw3cIOZ1QfO\nAFY657oAw4HtI8cbbVXfqncEupjZq2Y2zcz6xs1rCchlGROk2RLoBryZbqbyeB37zxsDvYFxqfKW\nySTEi51zM9JI1xvolIj+gUR00MA5Nx2orr1yF+fc0iBEf87MFjjnXktxnqPMbE/gR2BIcDsF8Jhz\n7ocgTV8SF8Hg4PVGwFbAHsADwa3JEjOb4g/qnLu0mvPVBbYhES01AqaZ2TTn3OIU+SwlOSljS7Qz\nfRLcuvWOkZ/zzex44BtgUOT9hyO3lX2B/czsouB1faAtiTIeCeCcm2Vm8/zOzrkTqjlfXWBLoBeJ\nC/glM+saRE7lIpfXMcGt+aPAmc651WmcJ9/XsXcw8FKqW3PIrNKMLi68hsTtkhe99TGgp3Pup3QP\n7JxbGvy/zMweA3oCqSrNsc65YSnyacDpzrkXogmCtoy4lgBLnXPfAd8Ftw7bAuVUaeaqjHcDDjWz\ng4LjNDGze5xzx6XY73rn3KgU+TRgQOUvr8jFHscSEhfSL8BiM1sM/BaYVZuDFamcXceWeMg0Dhjt\nnHs8zd3yfR17g4H7UqYiS12Ogpp9pZltZWZ1gGjmnweG+hfV3AYR+bxxECpjZo1IRHJzg9dnm9mp\nGWR1EnC6mdUNjtfJzBoAU4FBQZtIKxKRRSoTgP8zs/WCfPYk0W5TlrJZxs65C5xzrZ1z7Um0DT/r\nK0wzG+nbIWtpEnBmJC/+NnwqcGTw3nbA1mkcawKwZ7DPpiQqzA8yyFtRy/J1bMDdwGzn3C2VPium\n6xgza0rii/yJdNJns5/mhSR+mNdIfEN7Q4HfBQ2284GTgoxW1xbSEnjVzOYAbwDjnXPPB591Ab7I\nII93AIuA2WY2F7iNRLT9CPAxMJ/Eg4lpfofq2kKChxEvAm+TuD251Tm3IIO8lYJslXFNtgWWZZDH\nK4FGluiWNA+4Inj/7yTaoBcAlxGJFmto03wSWB38TM8D58Ts0VGKslXGvUg8WOljYZeefYPPiuY6\nDhwGPO2c+z6dk5fUMEozexI4OJtdDqR4BNHJ0865foXOi+ROqV/HJVVpiogUmoZRiojEoEpTRCQG\nVZoiIjGo0hQRiSGTzu20aNHCtW/fPktZKQ0zZ85csS7N6q0yLn8q43gyqjTbt2/PjBnpjMAqH2a2\nTi0LoDIufyrjeHR7LiISgypNEZEYVGmKiMSgSlNEJAZVmiIiMajSFBGJIaMuRyLZcuuttwIwcOBA\nADbdtCTXMJN1gCJNEZEYFGlKXv30U2K1hDFjxgDwyiuvAHDPPfcAcOmliaVc2rZtm9znkksuAWDQ\noOiyQFJqfvklMX2mX3pkvfXWK2R2ak2RpohIDIo0JecmTZqU3D777LMBeOedd6pMu2rVqgr/Axxz\nzDFAGKX611IaJk6cCITl1qJFYjlyfwcBcNxxiTX16tQp/jiu+HMoIlJEchZpnnbaaQDsvPPOABx/\n/PG5OpUUqW+++QaAa6+9Nvne4sWJlXV32WUXAHbffXcADjrooAr73nTTTcnt8ePHAzBrVmIttD33\n3BOANm3a5CDXkm3dunUD4A9/+AMAjzzyCAAnnnhiMs3o0aMBuOuuuwDo2LFjPrMYiyJNEZEYMlpY\nrUePHq66KaX8EzLf3+65554DYNttt631+TI1d+5cIHxye+GFFwLQtGnTtI9hZjOdcz2yn7viVFMZ\nV8e3YfoIc8qUKcnPmjRpAsBXX31V4zGibZonnHACABMmTACgc+fOQPg31bp161j5S0VlnB/PPvts\nctv3jPBP2BcuXAhAq1atcnLuTMpYkaaISAyqNEVEYsjZg6CNNtoIgOXLlwPw4IMPAtChQwcAGjZs\nmKtTA/Dll18CcP/99yff+8tf/gLAihUrAFi2bBkAd999d07zsq744YcfALjooosAmD17NlDxgc2V\nV16Z1rE23njj5PZ9990HhA+LJk+eDEDfvn2BsBsTwCmnnFKrvEv++fKD8AHQ4YcfDsDbb78N5O72\nPBOKNEVEYshZpOkftvjowD8UeO+994AwGtl8882T+/zmN7+JfZ6PP/4YgDfeeAOAp556CoCXXnoJ\ngA8++KDafV999dXY55Pq9e/fH4BOnToBYaQ5atSoZJpDDz009nEbN24MwHnnnQeEkeaCBQvWOr4f\nmjdkyJDY55HCOeyww4Cwq5HvXtavX7+C5ak6ijRFRGLIWaTpvyH8/74biu/Y6odWbbDBBsl9/LaP\nOH/88UcAfv/731c49sqVK5Pb9957LxB2pI5jwIABsfeR6r311ltA2Gbs7yJ++9vfZuX4++23HwBP\nPvkkAIcccggQdk8BOPnkkwF4//33Abj66quzcm5J39KlSwF46KGHgPCuz3ft89P/+YENAEuWLAHg\n22+/BcKyLUaKNEVEYshZpFm3buLQvo3xsssuA+COO+4AwmjEP3GN8p95w4cPT3k+/83kpxrzT+2r\nytM111wDwEknnZTyuJKan0DY91jwTzzHjRsHwHbbbZeV8/gBE/vvvz8QDsWcOnVqMo0frOE7wivS\nzC1/h/fwww8n3zvnnHOAsCyaN28OwK+//gqE0wBGn2f4qQD9sGs/gKEYKdIUEYkhb1PDXXXVVUAY\nJfj2Dv+tA1C/fn0g/GaaP39+IpN1K2bTD6sD2GeffYBwwH/lts31118/ue2fuu62226Z/ChSyerV\nqwFYs2YNADvssAMAPXv2zOl5H330USDs2wdhGfteE48//jiw9oQgkh3+9x2d/m/YsGFAOCGHjyL9\n1H4+bbRMfJ9pP8z2o48+AqBdu3Y5y3ttKdIUEYkh75MQ77rrrhX+j/ax81577TUg7IPZrFmzCp/3\n7t07ue2nEIt+00X5UUCgCDNX/NNsL199JP1ktqeffnryPR/5+LZy3z9YkWZ2+WcHfmLhsWPHJj87\n4IADqtzH947xdyZRW2yxBRC2T/u2zccee6zC62KgSFNEJAZVmiIiMRTlGkH+Nrq62+noHKC+ob8y\nf+vmZ5CX3PHDJ/2t1VZbbZXX8/sHiBDeAvqHDn4tIv+AcMMNN8xr3sqVH9K65ZZbAtCrV6+U+zz9\n9NMA/PGPfwSge/fuyc+eeeYZAKZPnw6Ek7D4Wfr9sEoofHckRZoiIjEUZaSZip+oAeDll1+u8Jmf\nUsx3rPZdGCR/fON9ly5d8nK+6IOHHXfcEYBp06YBYYf7v//97wBcfPHFeclTufMDGPbaay+g5gje\nP9D1g0n8nYGPPAE222wzIHxg17VrVwD69OkDVBzy7Gd8912Z8k2RpohIDCUZaVY1rNJ/0/kuRtHJ\nACS/fJexQvBDZH1bmPfiiy8CijSzxU/t6CfTiXZCP/DAA4FwUh7f/umjST+wJTqMsjI/WblfB8pH\nnBBOXjxz5kwAGjVqlMmPEpsiTRGRGEoq0vRTwvnp5aL8N98ZZ5yR1zxJOGzuuuuuK3BO1ubbzy64\n4IIC56S87LTTTgAce+yxQMXBA36iaT/ooWXLlkAY7ceZbLxyxAnhMF2/gqWfLKRBgwYxf4raUaQp\nIhJDSUWaI0eOBMK1kaPq1FH9Xyi+b6Qvg//9739AOMWf7zObK34CWwjXsvfq1asHFOfED+XAR/DR\n6fkWLVoEwO233w6EkWcmPVl8xAlhZOmXyPBTBPolb3yZ54pqGhGRGFRpiojEUBK35//9738B+Otf\n/1ptGg2PKzzfFcx3JH/ggQcA2HbbbVPu64fj+bV9/Ovoe5X51QD8XIwQ3hp6frZ3/X3khl8p1A+D\nhLCZJrr+Vzbtu+++ALz55ptA+GDID3LwM57lqslOkaaISAwlEWn61Qx99FE5moCKg/+lME455RQg\nHPLmh7JGu6N8/fXXVe7rJ2HwK0tGJ2WIrjaZiu/ofMUVVwDh+kS+24vkRnTSlHzxw3T/9a9/AeHc\nnv5O56yzzsrJeRVpiojEUBKRZsOGDSv8H+W7F3Tr1i2veZLq+eFyflq+aNToZ9p//fXXK+zjJ1/w\n5fnpp58mP9tkk00qpPUz9/vhdFWd26+RLuXvyCOPBMIVaP1qmG3atEmmyeY66oo0RURiKIlIc/bs\n2QDMmTNnrc/8N4imgCtefvqwytsi2eSHUPvO7354LyjSFBEpmJKINGvi2zNEZN223nrrAeGQzjVr\n1uTkPIo0RURiKIlI0/fP9Gsfv/3228nPWrduXZA8iUhx8iOBNCJIRKQIqNIUEYmhJG7PmzdvDoQz\nQftOrACdOnUqSJ5EZN2kSFNEJIaSiDS9Zs2aVfhfRCTfFGmKiMRgzrna72y2HPgoe9kpCe2cc5uk\nTlYeVMblT2UcT0aVpojIuka35yIiMajSFBGJQZWmiEgMNVaaZtbczGYH/5aZ2dLI65wsNWdmXSPn\nmG1m35jZGSn2GWJmy4P0C8zsDxnmYYyZDUgz7a5m9mu66YtNIco4OO/+ZvaOmb1nZuenkX54JG9v\nm1n/DM//ipnVuLCUmZ1hZm8F53zZzDrXlL5YFeg6bmdmU8xsvpnNS3UNB/vk/To2s2Zm9nhQztPN\nrGvKAzvn0voHXAGcV8X7BtRJ9zhx/gHrA58DrVOkGwKMCrY3B1YALSqlqRvjvGOAAWmkqwtMBp5J\nJ32x/8tXGQfl+j7QDqgHvA10TLHPcGBYsN0NWE7wILOWZfwK0D1FmiaR7UOBiYUuoxIq49/43y/Q\nBFicRhnn/ToGbgIuDba3Bp5Lddxa3Z6bWYfgG2QsMA9oY2arIp8PNrO7gu3NzGycmc0wszfMbJcY\np+oDLHDOLUl3B+fcMuBDoG0QndxrZq8Cd5tZXTO7McjHW2Y2JMhjHTO71cwWmtlzQIs0TzcMeJBE\n4ZaVHJfxLiTK9SPn3I/Af4CD082bc24uiYu8aRBN3GZmbwBXm1ljM7s7yMcsMzswyGNDM3s4iGAe\nBVIun+iciy6d2Qgoq64muSxj59ynzrnZwfbXwEKgVbp5y+N13BV4MTjnPKCjmTWvaYdMRgR1Bo51\nzs0ws5qOcwsw0jn3upm1ByYC3cxsZ+AE59ypNew7GHggTqbMrAOJCOb9SD73cM79YGanA58753qa\nWT3gdTN7lsRFvAWJX+BvgPnA7cHxRgCvOueeqnSetkB/YB/g/+LksYTkqoxbAZ9EXi8Btks3U2a2\nG/CDc+5LMwNoCezinFtjZiOBZ5xzx5tZU2B6cAGdAax0znUxs+2BGZHjjQZu9hd5pXOdBZxNIjou\nx7U6cn4dm9mWJO4O3kw3U/m6joE5JO4ippnZrkDr4N8X1eUtk0pzsXNuRupk9AY6BX/ckIgOGjjn\npgPTq9vJzOqTqJTOTTM/R5nZnsCPwBDn3KrgnI85534I0vQFupjZ4OD1RsBWwB7AA865NcASM5vi\nD+qcu7Sa840CLggu1DSzWHJyWsa1cL6ZHQ98AwyKvP9wUHaQKOP9zOyi4HV9oC2JMh4J4JybZWbz\n/M7OuROqO6Fz7hbgFjM7FrgEODFLP0uxyPV13AR4FDjTObc6jfPk+zoeQaJ8Z5OoQOcAv9aUwUwq\nzW8j22tI3C550VsfA3o6536Kefz+wHTnXLq3vmOdc8OqeD+aTwNOd869EE1gZrVZdakH8HBQoC2A\nvmb2q3PuiVocq1jlqoyXAm0ir1sH76VyvXNuVIp8Gol2rMXRBFn4YrsfuJnyqzRzdh1b4iHTOGC0\nc+7xNHfL63XsnPsKOC7Yvw6JJoEPatonK12Ogpp9pZltFZw4mvnngaH+haV4YhlxBJVuzc3sbDOr\n6XY+lUnA6f42xMw6mVkDYCowKGgTaQX0SnUg51xb51x751x7YAJwcplVmBVkuYxfB7pa4glrPeBw\n4PFg35G+HbKWJgFnRvKyfbA5FTgyeG87Eo3+NTKzrSIvDwTeySBfRS+bZWyJb6m7gdlBtB79rGiu\nYzPb2MzWD16eAjzvnPu2pn2y2U/zQhI/zGsk2qi8ocDvggbb+cBJQWZ3NrPbqzqQmW1Iov1oQqWP\nulBDW0Ma7gAWAbPNbC5wG4lo+xHgYxJtIKOBaZG8jDCz/TM4ZznJShk7534GzgKeI/E7H+Oc8xXS\ntsCyDPJ4JdDIEt2S5pF4Wgzwd6C5mS0ALgNm+R3MbHQ1lcAwS3SXmU2iTbTa2/gykq3ruBeJwKeP\nhd2b9g0+K6breBtgvpm9Q+L5RMrmwJIae25mTwIHO+d+KXReJPuC6ORp51y/QudFcqfUr+OSqjRF\nRApNwyhFRGJQpSkiEoMqTRGRGDJaI6hFixauffv2WcpKaZg5c+YKtw7N6q0yLn8q43gyqjTbt2/P\njBnpDCYoH2a2Ti0LoDIufyrjeHR7LiISgypNEZEYVGmKiMSgSlNEJAZVmiIiMajSFBGJQZWmiEgM\nqjRFRGJQpSkiEoMqTRGRGFRpiojEkNHY80ysXLkSgLFjxybfu/baawFYurTqNbYGDBgAwHHHHbfW\ne1K8brjhhuT2Qw89BMCbb1ZczXXbbbcF4KabbgJg7733zlPupJh89dVXALz++usA7L//2itUbLjh\nhhXSdurUCYBXX30VgObNa1y2PGOKNEVEYsh7pPn9998DcMghiYXuXnrppbXS7LXXXkAYffhvkvHj\nxwNw9NFHJ9OOGTMGUMRZTH7++WcATjrpJAAmTZqU/OzEExMr4I4bNw4II4oLL7wQgPvuuw9QpLku\niN5t/POf/wTgkUceAcAvw9OlSxcA/vSnPyXTfvjhhwBccsklALRt2xaA9ddfn3xQpCkiEkPeI807\n77wTCCPMLbbYIvmZjzBvvz2xImjlb45TTjkFgCOPPDL53qBBgwB48MEHgTCClcLxkeY999wDwMKF\nC5Of+bsGb+DAgQC89957QNimKeXH/12MGDECCOsCgGbNmgFw3XXXAbDzzjsDsPXWieXpJ0+enEzr\n70q22WYbIGwnb9KkSc7yHqVIU0QkBlWaIiIx5P32/JZbbqnw+plnnklud+zYscZ969RJ1PGjR49O\nvvfTTz+64GV+AAALwklEQVQBYcj+u9/9DoBNN90088xKray33noAtGzZEoBNNkm9FMtRRx0FwMUX\nXwzAlClTkp/tueee2c2g5JV/EDh8+HAA5syZA8DgwYOTaa6//noAVq9eDYTX+FlnnQXA1KlTk2n7\n9OkDhLfyTZs2zVneq6JIU0QkhoJ1bvemT5+e3E4VaXoNGjRIbvtvr3322QeAgw8+GIBp06ZlK4sS\nU7169QB44YUXAGjYsGHsY3z00Tq1tllZuuKKK4DwwU/37t2BMIps0aJFMq2/A/3Xv/4FwMcffwyE\n3Q7//e9/J9P67oUbb7xxrrJeI0WaIiIx5D3SvPnmmwE4/PDDAbjggguSn/kuBP4bKR1du3YF4Lbb\nbgPg1FNPBcJO9NGoVPLLd0xOx48//pjDnEg++Qjz6quvBqBHjx4APPvss0A4DNKnA/jLX/4ChN0J\nfbul70KYr+5E6VCkKSISQ94jzQMOOACAK6+8EoDLL788+dl+++0HwB133AHAQQcdVOUx5s6dm9y+\n6KKLgLBt0w+/8m0jZ5xxRtbyLrnjByd47du3L0xGpFYWL16c3PbXrx8y6+8uN9hggwr7XHbZZcnt\n888/HwjvDH1PmWJUvDkTESlCBXt67r9Zou1evt+Wb8c44ogjgLC9Y8sttwRg1apVyX2efPJJIBye\nd/LJJwNw7rnnAhWHXPqhWlJ8vvjiCwB23XVXAHr16lXI7EhMixYtSm5/9tlnANStm6heKkeYnu/P\nC9CoUaMc5i67FGmKiMRQ8H6avo0TwmnCrrrqKiCcoPixxx4DYPvttwdg9913X+s48+bNA+Caa64B\nwjbONWvW5CLbkiXvvvsuEE7x50cESWnxPV8gnKrNTzTur8FibqeMozx+ChGRPFGlKSISQ8Fvz6O6\ndesGhN1P/vznPwPhmkB+aN3LL7+81r6+42zlBmW/bgiEQyylePguYStWrADC2/SJEycm0/jhcn4g\nxC677JLPLEoaWrVqldz2t+r3338/EE7CMWHChPxnLAcUaYqIxFBUkaZnZkA4RNKvJbJ8+XIAlixZ\nkkx76aWXAvD0008Daw/dW7BgQXJbkWbx8XcNvjO7n+bLD1IA+N///geE3ZHatGkDwPz58wFo3Lhx\nXvIq6fGTaxx66KEAPPHEE0B45zhkyBAgLMdSo0hTRCSGoow0q+Mns41OauunhPOR5jHHHAOEbWLR\nyUv9kEspHn5tIB8tbrTRRmul8WvLfPrpp0A4+azveuZXMgTYaaedcpdZSYu/Ph9//HEgHB7tuxIu\nW7YMqLjCZClFnYo0RURiKKlIMx1+wtvWrVsDFTvdSvGJPnWtjl+VtF27dgDceuutQDiAoXfv3sm0\nb731VoW0Uji+ffr5558Hwl4xfqjzf/7zn2Ra3/ulFO4UFGmKiMRQdpFmZVW1kUl58G1ijz76aPI9\nH4X6dk8pPN9e7aeK69+/PwD9+vVLpvELIvo7hc6dO+czi7Eo0hQRiUGVpohIDGV3e+7n8vMrIVY1\nI5KUF7/eFIQrAvj/69evX5A8ydr8oJWWLVsC8I9//CP52R577AGED4R0ey4iUibKLtJ8//33Afjh\nhx8A2HfffQuZHcmDgQMHJrcvueQSoOIwTCkufhi0X5EhyncVLGaKNEVEYii7SNOvteyV0vAsqR2t\n/ZR/lYe0+hUna+LXth8xYgQQdnoHGDRoEAB9+/bNaj5zQZGmiEgMZRdpzpkzBwgjzHr16hUyO5IH\nfmIIyZ/vvvsOgNGjRwNw7LHHJj/bcccdK6SdO3cuEE6m469RH11COOlKKUzzp0hTRCSGsos0/bDJ\nF198EYANN9ywkNmRHPrpp58AuOGGG5Lv+en/dIeRW35ClKFDhwIVJ/hu0KABEC5L8tRTTwFhm+b4\n8eOBihOtaN1zEZEyVfKRpp/6bYsttgDCfpkdOnQoWJ4ktz755BMALr/8cgAWL16c/Gzw4MFA+ayx\nXaz8dH1+YuFoLxU/qmfhwoVAOBWcnzA8GmGWIv1liYjEoEpTRCQGy2S4WY8ePdyMGTOymJ3iZ2Yz\nnXM9Cp2PfFEZlz+VcTyKNEVEYlClKSISgypNEZEYMmrTNLPlwEfZy05JaOec2yR1svKgMi5/KuN4\nMqo0RUTWNbo9FxGJQZWmiEgMNVaaZtbczGYH/5aZ2dLI6w1ylSkzO9fM5gX/zkwj/RAzWx7ka4GZ\n/SHD848xswEp0hxqZm8F53zTzHbL5JyFUogyNrNGZvZGcI75ZnZ5GvsMj+TtbTPrn2EeXjGz7inS\nRP+uZpvZCZmcs1AKeB0vCcpqtplNTyN9Ia7jiyK/i3lm9ouZbVTjgZ1zaf0DrgDOq+J9A+qke5w0\nztMdmAM0ANYHJgNbpNhnCDAq2N4cWAG0qJSmbow8jAEGpEjTmLBNeAdgbrZ+B4X6l8cyrgM0CrbX\nB2YAPVLsMxwYFmx3A5b7338ty/gVoHu6f1fl8i9fZRwccwmwcYz0eb+OK6U/BHg2Vbpa3Z6bWYcg\nQhgLzAPamNmqyOeDzeyuYHszMxtnZjOC6GKXFIfvArzunPveOfczMDX4YdLinFsGfAi0DaKTe83s\nVeBuM6trZjcG+XjLzIYEeaxjZrea2UIzew5okcZ5VrvgNw00AsrqiVouy9g5t8Y5923wcgMSFWfa\nvz/n3FwSF3nTIJq4zczeAK42s8ZmdneQj1lmdmCQx4Zm9nAQwTwKrPNr++b4Os5Ivq7jSo4AHkiV\nKJM2zc7ATc65rsDSGtLdAox0iSFLhwO+EHY2s9urSP820MvMmplZI2A/IO2FfsysA9AOeD+Sz32c\nc0cDJwOfO+d6AjsBQ82sLTAQ2ALoCpwA7BY53ggz27+acw00s3eACSS+JctNrsoYM9vAzGYDnwET\nnXMz082UJZpCfnDOfRm81RLYxTl3AXA58ExQxnsDN5hZfeAMYKVzrguJqHX7yPFG13CrfnhwYf7H\nzFqlm8cSkrMyJvFF+KKZzTSzE+NkKp/XcfB5Y6A3MC5V3jKZGm6xcy6dAau9gU4WLBRPIjpo4Jyb\nDqzVzuGcm2tmNwLPA6uBWcCvaZznKDPbE/gRGOKcWxWc8zHn3A9Bmr5AFzMbHLzeCNgK2AN4wDm3\nBlhiZlMi+bm0uhM65x4BHjGzvYCrguOXk5yUMYBz7iegu5k1BcabWRfn3IIU5znfzI4HvgEGRd5/\nOCg7SJTBfmZ2UfC6PtCWRBmPDM49y8zmRfJSXVvlBOA+59yPZjYUGI3K2L9OWcYkvsiWmtnmwHNm\ntsA591qK8+T9Og4cDLzknPsqRbqMKs1vI9trSNwuedFbHwN6BhdJWpxzdwJ3ApjZSOC9NHYb65wb\nliKfBpzunHshmsDM0r79r4pzbrKZ3WNmGzvnVqXeo2TkrIw959xKM5sK7AukqjSvd86NSpFPI9GO\ntTiaIHKxx8nbisjLO0lEqOUml9fx0uD/ZWb2GNATSFVpFuo6Hgzcl07CrHQ5Cmr2lWa2lZnVoWIb\n5PPAUP+ihtsgImk2Df5vDxwEPBi8PtvMTs0gq5OA082sbnC8TmbWgES76aCgTaQV0CuNPHaw4Eo0\nsx4kHkqUU4VZQTbL2Mw2teAJpZk1JBHFLAxej/TtkLU0CUj2uDAzfxs+FTgyeG87YOtUBzKzlpGX\nA0i0+5WtLJdx4+CWl6CZrQ8wN3hdNNdxsH9TErfyT6STPpv9NC8k8cO8RuKpmTcU+F3QLjQfOCnI\naE1tIROCtBOAU51zXwfvdwG+yCCPdwCLgNlmNhe4jUS0/QjwMTCfxC3YNL9DDW0hhwNzg3a5W6h4\nu1iuslXGvwFeMrM5wBvAk865Z4LPtgWWZZDHK4FGlujqMo/E02KAvwPNzWwBcBmJZh+CfFbXpnmu\nmc0N8nkqEKtdrkRlq4xbAq9Gyni8c84vdF5M1zHAYcDTzrnv0zl5SQ2jNLMngYOdc78UOi+SfUHk\n/rRzrl+h8yK5U+rXcUlVmiIihaZhlCIiMajSFBGJQZWmiEgMqjRFRGJQpSkiEoMqTRGRGFRpiojE\n8P+WVeJogJBJeAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024e83e0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model has now been trained for 1000 optimization iterations, with each iteration using 100 images from the training-set. Because of the great variety of the images, the weights have now become difficult to interpret and we may doubt whether the model truly understands how digits are composed from lines, or whether the model has just memorized many different variations of pixels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvWt0XdWVJvotIQshZCELGRlZOMI2xjHgGGPAOHaGQ4Dw\nSiCEBBhFelBVpJqkk9vk1Z3cTqeTG+4tqkIqVDe3iupiVKgKIyGFwyOQxDwq+Ab8oMBgbAMONqD4\nAXYshB/CGFv2uj/m/PZee2lL1uNI+xx7fmNoLJ191n7Nvc5e33wu572HwWAwGEYfVUVfgMFgMByp\nsBewwWAwFAR7ARsMBkNBsBewwWAwFAR7ARsMBkNBsBewwWAwFAR7ARsMBkNBsBewwWAwFAR7ARsM\nBkNBqB5M56amZj9xYvsIXUr5YcuWDnR1dbrRPKfJeGRxpMkXANauXdnpvR8/WuczGQ8cg3oBT5zY\njoceem6w56hYXHHFnFE/p8l4ZHGkyRcApkxxfxjN85mMB45BvYBHC9V6VT09xV5HpWIwcqOsq3NG\nwt69+X0NfWMgsqcca2ulpZwHs6+hN/LkV+7yMhuwwWAwFIQRnx/6m9Xj7/r6vH9/uu3997N9jj5a\n2jFjpM2b8fpieeU+Ow4WlFcoR/4f3yvZV2Nj7+M0NR4EAHTtkPm5s7PvYx2KtR1OMs67/2OPlfbA\nAWmPOir7meyW8gaA+vrscXbvlvbtt6XdsSPtG+4X7sPth5N8QwxGG8jrm/dbCD+HcutLhqMhW2PA\nBoPBUBDsBWwwGAwFoeQkO6b+/akS770nLc0HROz8CY8Rf/fWW9Ju2SJtaK448URpqWZT9ePnPDWu\nElW6WNahjLq7s9vYh7Lg9qwMZF6mLN95R1qqyKFazONxW6xe9+fkK1dZ92W2Oe44accHwUZVq56X\nf2g3iFsebOrUdCf+ryeobm4FAHR0ZHcN/6dp45hjpOUYbm6WNjZVVApowiHyTIl8T/DeG3ZslH82\nbMjuTAECqW3ypJOknTBBWxHYwcampCt/A2vWSEvzEcfySJp7jAEbDAZDQRgxDjKQEKZx46TtizVz\nEgsZcFubtJs3S0uWO3Zs7/OSwZEF8hroVOJnsgigspwbsROC957HoHhfbOfPl7ZqyW/lH9IMANgr\nrKFh+nTdUAMglfGM9j29T0CoMLu6ZZ9162Qz2UR4DeWKmNWTPFHeVZs3pp2fflpasrGtW7M7z5ol\nLQds+J22NfrgzjhjcuaQQCrecHcg/R2QTLe3p99VwtilCFrG7ZN/KLffd0gbsNkGPhCqu6+9Jm3s\nkQyFQBWBx6Gg9FghWeYYpYw5Vnna00/Pbg9PPVwYAzYYDIaCULK5sj87ZPg5ZLN5ExeQMtLWHS/L\nP+9uTb/cLAeYwZ03bZJWqfC+C85PunJmY1cyX5KVeAINtwFAua9XymulbDmDr12b9iH7J2MimyOz\naiLLjZksgFc7hMW+8IJ8PuUU/SIUGE+6YkXmRE1z5+pHsW+SvMS7lyPIdNjWQFnaOh04IUUlc+ND\niA2HlM/WYAw/8UR2nw99CABQd8klAID29tlJV/5e+NzI3DiWY1tweAnlyIR5TbwfrNCMOQ7aUE4E\nZUnj+wc/KC1ZLW84fJFwPD8nx9+ndnYefvv23l35vog1IMo2fk+F9zNUGAM2GAyGgjBiURBxQDq3\n06MefsfZMPakY2sObY5tOwx/0LYmoAIzeWCd9qad3g4A6O5uAJBMjhlMmSIt7Z1FI7z1eLYlC2JL\nkXAmB1K2TzsWTZKPP84ewgwOHGhN9lm1KrsvWfTFF0u7d286b8+briyEctfns6dWvMw7Ikd1uSFP\nI6OcmZCSMF+y2ZDCczwq48fChdkTxGEo4Qn44JRa7asXmb3zQto1Vkx4aj5jij2M/iln8HqrOl6X\nf2INgj/Ad99Nd6IM4xcEtzN8oT87+2Y53yQ9X3fjvKQrHxm1xEcekZYJMnwG4SMM7cHDgTFgg8Fg\nKAglZ8B0ppPdcpLiRMfPQMrGOJu0Nqp3PTYO59l2ONtxWiLlDk8Qu7F1n/nzZwBIGWO4C6+7HO1n\nsZ09FgUJVciAOYvH90pGnBcHTEJBEhIpEhkCuKtaWNvWHmm79Tzr10tLBSUMgw2jX4q2s+c950SJ\n4g1TFeAXIf25/HIAwJ72GZlj8Bk11EZGXACr1wrv6d47CQCw5E7ZTvIXDneeki2fF03ufOZxin65\nIpE31axXXpGWmgTtu3m/eVLU+KXCfULnBwVFesu+6vOY0fPHtK+eu7OzJnMaXiLdTOElGQM2GAyG\nCkfJeR5nYk5ScUJQOIvQAc/Jr2tHHQCgqVZsb8xWqdrwaroT6d2DD2YPGDCMBLGrWKli3Rtv6Pkv\nA5CdOEmkyxl9RZzEWW5AKoKYRJDFkglzOwD8x/8o7a9+JS2fE5lA6KimHZ3beD5+Pv54acPnHnqZ\n3aiWux8YanpUE4uFRsGGIQdK6eoW/Yt8plrHGyblC9SSmZqN9XLnCQCAj31Mtm/bJm0SbYJUjvEz\nP+207D6hFtdXJt9oo9/r4AXzt0l56WB7U98FAFCr74G9OrY2bWrSXaRtccpmw8y4eCASS5ZkzgMg\nUSdm6nNuvnp2piszQEeiPK4xYIPBYCgI9gI2GAyGgjDixXgIFt3oz3hNLW36dJkXmrZqIkagv3Y1\nT5PvqAYyKD7Py8e4qTi+SPeZ/93LMucNUV1dnuoxkMo2TrGmSEKHF/tSi6bJgZqf+pBQdfvfpDup\n7vVnt9wCAHi9fiaA1FSzdGnalQ5AauuxZslrKffkCyBHVeaYUqFt3CsmgzCNdY6uqFQ3NeuhPFgv\noY6UfxgeyQPMqFc1+cxscZ6X16W8iOal+FkTcYp57n2UEZJro63vjDOkVZPAtt1iegjNgpT34sXS\nPvCAeIdPOkleKj/4gTyXa8LfK21jfAA0DfE5hCfgO0QddhNmiQmCvxW+u1gaoZQwBmwwGAwFoWRz\nZRzOxBmZM3dcGjLchxNQ6AgCkMyKjz2RzhPsO2GCBFJ/8QZlD7SY56XJEowr0ZalBGtr09RPhqGV\nI/pyAsRp36Ec+V0c0ZcExC/6V+n3ta8l+9Bt0f7xj8t5LxQGTE0hTA6g34OaDX2hEydmP5cqbGck\nQVK2sVNY2CStWMTCQnfdJd+Hz4FB+8cfL+ORoXscp2Rv119/QrJPW5v8f8EF8nkyU50VeStikH3R\n6cZQPrZhWcdyY8ChvKr2qoPzox8FAGw7ShKAljwpm1eulPbhh9N91q1jUoYmb0Cc6Js2nQ0AOPNM\nfbk82JHuxB8BM1T4ICio3/8+7csfiaprHOdR1dCkHGYpYQzYYDAYCsKw5so8RsaiKyyszGgPziZh\nVAjNMJxhWCAjDWmS+eHGG9N9tm9/Vv8TarBu3Rna57MAgJltXWln0jNOafzMGU+DwMPiJ+Vmq8xb\nfy22+cahfnmgMkARXHGFfrFTdg7Ky4ABQO1a/IT78BmGZnbaQEk44sQBajyh/aycGFooM4YbUVY/\n+5kw31/8Qj4/+yyD90M1iexMqRt2absx0/f7378k2EeY21e+IoOtulrOw2xm2uWBVPZxBj6HMH9f\nrDseflcuCJ/3wVoZXd310j50r2y/5x5pn3qK8gsSJSAaw7hx8ls/91xpv/xl+XZa5zL5J6+uLIXK\nzBUKsqUl7avG3se2iqYXK84jWfTeGLDBYDAUhJJzETJf2v5mtysjrZbpY0dtGmAde+3j8orf/a60\n27cvCc5AF7zYfRYvltmQTKy2Nl1q5LgPnAMAaOEBSbmZUaB2oAlnp0cfiWDrUiFe3odp35R1XgF7\nTvj0IK9fL7KYM0en8/dE7QhSXXAp/1Fm0K4fY3svkBIJ2pZZxIjnpV21XIobxQjZGQtF8T5/+ENp\nt2//ufag3fCs4Ai035J21mg7SVsyreODfeTBPfigPIM4w/mqKw8mPRsbhSNRy2Etcg5l2pFbJ6T7\nbNsu+4T1bIpEOB7J4OMs7zSHQgaMc2kozze/Ke3VV0tLWZDc4jk9QVgmlCnIPDk/6wXs+tx/Srou\nWiTtvWrjZ0o4AzRsSSKDwWA4DFHyd3qcWZjYXzU1kJ5lAJhEJrVW+2isXsdWYcn/63/R1rYmOMMf\ntG0ID5vMigyGAFJG81+v04shVaSxWS82ZHShfbPcEC9BFG+PtQ8gDfxYv55yE9vkc88t0M8vAQAC\n6xmS3fVEu5VpU7ahvEKzW4i4gmDoQS4nG3AIRsDQBrh9O+2QDEtgeMnpyT5jx34AQMrGElamYMow\n7csAcPfd0nLs0o7L53r/gykviqNHKNdedaqCYPYWVSnfPFCDcgPvkfdO5TQdRxKlQI0WSMczqw9Q\nY67r1udD+y7j/gFsnCs+oUk9GjnBh6q/+fA9wefBa9i5M3vNeb87W5LIYDAYKhzDeo/neejJeFp7\n1AvMqY41DsNphLSVhkTtu24d7Wdcxyb0iB6baZmtkmenOfPM6DycVkkjKiE4NQfx7BsX5QlZfOrR\nJQWTOTdhGP8o1b/DEOxaUg6lZt9SYkETW8hOyBq4C1k4RTsS2UOlQJ6tn/Zq3su4cYzdlepENCOG\n988EK9pia7rV58EBSQPjjdcm++zdK8yUzzGqlJiJFGIxI9pOedhTT5W2DlHhIDlB5jyjDe9Fvv2x\nRP7kKevYDh5qcXFdnSTOnRso/OD3zH9f3yELnU6e3y4b9McxP+f5cx/a1+OlnfLGjC1JZDAYDBUK\newEbDAZDQSiZO4QZf4l6wGIYMUcPo5z5Px0IuirsPTezwxJtO4MDaO4lRPejqsI2r94wNqsuwTTE\nqPZouQWuHwpUhVh7mZoYtdBQhU1XIz4ZAHDeeSK/G27Qzd/VtfTCE3z72wCAXy6WrU8+yXgm0dG7\nu9OIdKqHsdONbTmndsdgMgMX36XJgepxfI9AOt75TFZvljBIjr0adg4G2dVX12SOS4cQHaah+s3U\nXJoieBim9icmjtAEoScvKqnIOfnZ56nnNEnx3uOVPOKcqfB/Ot90EeneHYIav5QTU8VPP1245ty5\n4rxvWrss6fvJTj1po5zg/GvlOF3VYoKKHYaAOeEMBoOh4lHyVOTE6RLW7APSMv9cLAzo5Tnr2iHz\nQRpPzSWUT0z30SD4k0+W9OFPf1q2xqXjAKBmr6Y1cuoitVGj/cH5HwEAdAeV6cKZrej1ymL0tSYc\nnQb83BkoDFwu6xOfEOZL8k/lo0k9Tw3hib7wBQDAohu4gVqLhFyFzIry7ssxmFeetFxWbIjR2qxJ\nFaSiHDdrNHHnwgsBAK93p4V1uFoLme7MbpXVLZr5wodBbx2A+maRdt0qYWGXThfK29YmDqPQicow\ntnNPVKc2hcYHSPocUkbGwrWd0/fNjgL4nMMCXE31KmO9/lNPlXvm64LieuKJdB+y5WgRabT/uaQO\nt+gJfrthUrLPE5rizOOSHNctvl/+4Vp0QKpOkH7rTk36UqmdLu+aNUE0bKm0C2PABoPBUBBKxkFo\ny0lsfpz2aISkkZiUDEinJ21JOLjr9u2Mzg752UUAgJvVTqxm44T9MQwKSItiV3HKpHFU44jiohtA\nNt233AuyU3xxmciQcTBk6sorpSVRmlmvAeqaIzwtmNJZEDwlVWLoPPFE6RPa2alUcPc4dIcI7Wfl\nFP2XuU5SLKpgHCBkrzp+JtcG9laOKdKjP2jCS1zTMLjppu6Nme9oazy9XTZXIU0rTgS3dmv2fFzu\nmteaY5gul7T6DFukuqbvhZZL2gEAU6fKmMtbO3L5cqaAy5hds0a06TvvFPm1tQnzZeIVkNrvdU0B\nTN7wWPbA4Y+fz4jPkD8sNbwzxWUkVp42BmwwGAwFoWQMmHYrkojWeO0bUo3QW8tqMrqNNrgJE8RL\nvH49Uz7T0nGf+5xQXNp0eLi4Df+f3Kbn1p26ehoy3+fZInt6ys8GTFDWJENx4HiQ7Z0wAU74yXJF\nZFZcAjmgzbS/8XjXXSeG3jjpAkgJBZMY+EgJHjZnIeFCwWvIpJ436w2T+r8gSSrJTbJmYlgvMjbI\nUzhkUXG2BZAyaj1P0+a12X1Dyhiv9s3z3HGHtCws9a1vJV121bdqX5QfOFjJ4DWKY54O1E2bRBsI\nx/CKFZJ14j3V633a92VtxTdxyimpA+i//3dpW+/VpbYoNw5qrhgApL8Fai8cFFqNp+O53ruUCsaA\nDQaDoSCUjItw0k4Y0EkaIBkzgdAAyBqFZAe33w4AuO++/wIA+Ku/OqPXeUisjztO2rz0W4Ksa0+P\nMOraCcIM1mtN99BeTJSzDbivlGPaeakVBA73JE6VIiYj3TdhBgCg5jz9IqAcs7S2CTM8431D8Bri\nhUJjm3BI6srFNglk2fi+WtGMaqgmkH3ygqlGPPVUuhPHMKvtUAXkvnl0n8fpy1AeGkAp0DjkhSrH\nZz4j7VlpiUzKupzknICDiTJgfU0dXNd8aiEAYMGCNDKdQR2LF8uYZe2dePxzvAJBeU4OQL6HKJS8\nKlJ8HpfJYr1d9WJbpjzz3hfDhTFgg8FgKAj2AjYYDIaCUDITRK+VQ7khdsKFuijVKm7ToPIWje7/\nm1s/DwB4tSNVR3gY+klizS8EtbT4dAwnoUpRDk6hvpCXwEBx0eRAnxC1XpppAKCqRxwWk+uzToht\n70jN5Z1t58uxgzh+am0zql/Vlg6k+l4n2LhVnk2a8izoLwytHOQdyxIAaqpVbeWYpT2Nnak+hyaC\nePFD9qH3U/VnmsFC1FVrUgLVYZo21gaZQTwuVXZeMMM5uXxDUvov/7dQJMJnX8fMHf4YOZaYoKV2\nhdZg4F+lMtj/MUm8iKuWcfyHvtF9PcIta2iPS2Nbs5+BVLa65EaXrhEZW4rCcWupyAaDwVDhKBkX\nYQoyJ7Z90yWcpIYzHKfBMKyG/3PKZgooHRs6G06YkK7zxq5xMkLejER2HAdQx8kCIzGzjQR4bXSs\nzWt/U/6JqwmtCuKPImfQTx8U5hsnb1yblqtNCW5nVAg1Fj7SkMHk1PoI85xv5YS8Z5+wprhuNO+b\nzrmwyLGuK5jEKHFfFeI+LXMUxv0zYeDMM+W7mnUqtC1SGCkTshaqMyEoYA2VOqgOZgDoyUkwKhKZ\n3xSZOjO24mpGOUsQv14vzJe/Y5JXrhEX51kBwIypql1w/PM8Tz4pbei1V21l2VZJi96hTj7+NnhJ\n/a1QPlQYAzYYDIaCUDK+R9svJ3HaZ2bNkpm5arOmX4ZFQzgLseII05U5ld16KwCgIVjrqUFtatXV\nwuQ4wfF8IRnkqTiD0VxG8lCWYTr9gLNtEkFzrxZ8ifM3QxslC5WsEHndpSu/Ll8uLeuQhEQr+Z/C\npJrBZQwCdtIZRRURcShUOWsWBIdjE5ku7z9m/m+8ke5EmcQ52QrmcoS1qbjmGEng9OlSNKeJ9kn+\nDoA0Kyb2qdDGrA8rZ0GMspF55ncWy5Z2dl6s/qD3tE1LdmnUroyUpJIRa3HZdHfRLo4+WkLJ+Hiq\nFizI7gRgV7MWBFLmy98XRZ2XOm82YIPBYKhwlGxNuHiCJmngLDV9usxEmTc+aQGLt4fsGEinorB0\nXJLbKp5KkmXuGpp22JUkhYyj3BhCfwivMSZkk+gh583rDXc1p+yhQ+VP2yxNYcxcJXMNC5kkiOsJ\najnGjZvTp8joB8qdXeMIlHKVdcjOOJY69oq2sGOHtN3d4oMguW07eUayDwulk9WS4PGRsKB6GJkQ\nR2DwuZ5/9tnyT1jFPu7EB6gnYvLI3jKLfAiRcVE06wDhfVFQ0aoCdVemvqK6vTK4miZkI6o6NDqK\nOSgvvZSehq8Wjs9PfEKPpcJ/tXZm2leZL8dsvFJzXwWmSgFjwAaDwVAQSh4HTJssZxHW3EgLLKdF\nk/fq/40XfBIA0Kye+BntezIHZawpkNoutyzJnofnD82fnLlon+YMF3vAy5Wd9QWar46eMg8AMGa6\ntAylfHFR2pcMl+mbZGIx8QhtlIweqVfv8wUXSNusfUJ5MWBlnK4UFT+PcpdtyIB57ZQvZUL/AmUX\nFv3n/ceBEhyH/D0EmcKJfGMF4+VOiRzaWv/JpG+yipYuw8NIgGpqHCrfUOMoN5mHMmY8dB0FxYt9\n/HFpqQ2zyBCQqhcczEpNL+XS0BtEY2hJ1kND6sjQQlz7qsXO+/NnpQ2VDMqfS1LFy2qNJIwBGwwG\nQ0EoOQMm4tmD39PRCwDbtklLFpZW9hPbG2f1sAhGHAlAlht7LsNriLP0KoWdxYjNgYw4iZlbWGeE\n3zEChGZjyi3P9svnwD5kd7E84//DPn19X24Iry+29ZF9kqyRgIWuCsqKZIygzPLsh7FXPS6lGvox\n4sUr4/hq9i1nOYcMOE3oE7bfqPkCDRyYsV8IAN59N3sgHoQt1Q0OciBRPWgjZ1cmEIbPgDJkQEY8\nzkdStsaADQaDoSDYC9hgMBgKQsnJdUzXY5UpNBHQ6B2DpodY8wiPH6u6AzGcV6rpIUasIsVmmFBe\nVJ/DFQaANNafMs47fn8p2zEqXaZAb3nGY4nyDestx4iTewaTiJKX8trXfpUqb153nN3eqes+Vjdq\neNgFaZhYz8L8Y9G8xmOFq7GMUTNRX++JvHcKHcmjKVtjwAaDwVAQRuxdH88ifTFXoO+UYM5IeX2H\nMkvF56lUFkEMhh31JbfBzPqVlFZcCvQ1hvOQV97yUH1LMYYPN/R3f339fvt7twwERRaMMgZsMBgM\nBcH5QSz965zbDuAPI3c5ZYcPeO/Hj+YJTcYjiyNQvoDJeDQwJBkP6gVsMBgMhtLBTBAGg8FQEOwF\nbDAYDAVhyC9g59yPnHM3B58fdc7dFXz+oXPuq4c4xrIBnKfDOdecs32hc27eYK872P8s59wa59wG\n59z/dM65oR5rpHAYyPj/ds5tcs51H7p3MahkGTvn6pxzv3LOrXPOveScu3UoxxlpVLKMdf/FzrkX\nVcZ3OueOOvReA8NwGPBSAPMAwDlXBaAZwGnB9/MA9Cs07/2QhQJgIc8/RPw9gM8DOEX/Lu6/eyGo\ndBk/DOCcYew/Gqh0Gd/mvZ8O4EwAH3bOXTKMY40UKl3Gn/XefwjA6QDGA/jMMI6Vhfd+SH8AWgFs\n0v/PAPDPAB4DMA7A0QB2AKjR778B4FkAqwF8LzhGt7ZVAP4OwDoAjwP4NYCr9bsOAN8D8DyANQCm\nA2gHsBXAFgCrACxQoawF8CKA3x3i2k8EsC74fB2AfxiqLEbqr5JlHN1Hd9GyPNxlrOf4WwCfL1qm\nh6uMAYyBkIprSiWbIYfUe+/fdM71OOcmQWaX5QAmAjgPwE4Aa7z3+5xzF0EY5jkAHIBfOuc+4r3/\nXXC4q1RQMwCcAOAVAP8UfN/pvZ/tnPsigK977290zt2pD+U2AHDOrQHwce/9Fudco25rBXCX9/7S\n6PInAghXMdus28oKFS7jisDhImPt+wnIS7iscDjI2Dn3qF7XbwAsyuszFAzXCbcMIlAKdXnwean2\nuUj/XoDMTNMhQg4xH8B93vuD3vutAJ6Mvr9f25UQ4edhKYC7nXOfB3AUIA++Ul8MAUzGI4+KlrFz\nrhrAzwD8T+/96/3eaXGoaBl77z8O0ZyPBnB+fzc6GAw3qZS2nTMglH4TgK8B2AXgx9rHAfhL7/0/\nDOM8Wg0VB9DHNXvvb3LOnQvgMgArnXNnee/f7uN4WwAE5fPRptvKEZUq40pCpcv4fwNY772/fRjX\nNtKodBnDe7/XOfcQgCsg5o9hoxQM+HIAXd77A977LgCNENWCRvVHAfyZc64eAJxzE51zJ0THWQrg\n0865KudcC8RofijsBjCWH5xzU7z3z3jvvwNgO4A+aq0B3vu3AOxyzs3V6If/AOChAZyzCFSkjCsM\nFStj59wtAI4DcHN//coAFSlj51y9c+5E/b8a8tJeN4BzDgjDfQGvgXg0V0TbdnrvOwHAe/8YgJ8C\nWK62l0UIhKH4BcQO+zKAeyDqx85DnPthAJ9yzq1yzi0A8AMnYWVrIQ/0Redcq3Pu133s/0UAdwHY\nAOA1iG2nHFGxMnbO/bVzbjOAOufcZufcdwd816OLipSxc64NwH+D2EOf12PcOJgbH0VUpIwBHAux\nRa+GOPH+CODOgd70oVA2qcjOuXrvfbdz7ngA/w7gw2rjMZQIJuORh8l45HE4ybicCgs+oh7JGgDf\nr1SBljlMxiMPk/HI47CRcdkwYIPBYDjSYLUgDAaDoSDYC9hgMBgKgr2ADQaDoSAMygnX1NTsJ05s\nH6FLSRGbpYuqU7ZlSwe6ujpH9eyjJeNywWjL+EiTLwCsXbuy04/iihgm44FjUC/giRPb8dBDzw3q\nBPHS0f0tRMi+bPOWmudx4kUQh7LA4aH2ueKKOYM/6DAxFBkPBeWyQOloy3go8qVsjj6693cHDmQ/\nczxyaXuO5e4hFuTsa5HKwSzOOWWKG9XlgUoxhvPuL17Svq/vi8BQZWwmCIPBYCgIozZnvPde721k\nD1s1io8sobNT2phNhOBs19zcd9942ep4++G6zHoeQ4g1B7b790sbMrmjtNz0WM1B6m/ZbjI8oi9Z\nVrKMee158qjq2Sf/dHRoq4N3sxTbq9GdG8KB2RzVDI/VuPAE3E+37auuAwC8845sfvddacNnXsmy\n7g/x2O3r+xAjoRmXEsaADQaDoSDYC9hgMBgKwoiT7VgteDso+kZTA1uaItaskXaLFogMNTb+36bF\nJFtapB2v/sfp09O+jY3SHsoEUamg+h/fT3hfNP2MGSPtjh3ZPjxGqCFPnSotnU4t41TNVrU6oyI3\n12cO0LWjKtOV4LOIr6+ckWcRAAKzA5AO3tgrzAFKQWzf3vsEFDrNF2+9Je377/fuo6hpbwcAtGjL\nh/V6x+HBpfLGMMcozWU0u/T1uw735zaa1fg7yDMzFGE+OzyemsFgMFQgSv5uj2cLEoI81kOn2zqt\nrkkysVOLy3Hy53YgnQ0nTJCWM1pMPABgUtvB7Mmj9mB9A4AsW4tZZTkhvo3YecmWhCoEGWgsU/b9\n8IfTvufP3SP/kC6vzdY6OThrdvL/qlXSzm7vAgA06cU1Ne7NnHjbew193k+5Ig4ho+wmtaWDY9tR\nrQCAMRNf8i+tAAAgAElEQVSkXb9ev1DGdeLc7LHC482ernJ+Uhd14EPati3tzAfEA/CZ6AA9OHVa\n+BFA+cs1RF/OYd5m+D/HLOXH7WxDR3/8XqBoqUEfd1zal85VPl9qfmPjQpgBSvV+MAZsMBgMBWHE\neB5nkfqsiTBjTyOL5axHJsrZijPbxcGC8bRP0gTG2ZC235rurrQzZ1Gyh+c0OHzTJgBA1ac+pR1a\nB3JLZYPYZk6SlJe4MlcZGDUDPgcyNcox1DL+ZVGd7ivttOn6QPRBVe3dk/SdXa8Pbas+RLI5srhT\nTwUAtHz0o8k+XfUi75DllANiMy7ly+27d0t79NEpb6GfgkPs7rulJdPi9jlBvgnH7ObNIt/58y8D\nADR1PC9fhAZz7siL4oGVguXZTCshvDKWNZHnk+A4odxeeUXa3/9e2rVrpQ3HMEEZ8HixzwhI30Oz\nZknL3wq3U+ThNZVKxsaADQaDoSCUbI6MZzTOMInHOJk9apJ9YnsrGTBnOjKQW25Jz3PssdLSxsNZ\ni7PUNdc0JX2POUbamc1vyj9PP5098IIFcv7mlAH3Z7MuGgy+p7M8TlihdsAWSGfxpupd8o/e2Cmn\niJxO0tWwwkiTRx6RdsMGHl+YGplB1drVaefYyEYX9cMPS/uTn0j7rW8luzRdfz0AYG9tHarKiAL0\nZY+kvE88UVoyLyCVEf0YcXQJn9GSJek+1Do43mlHb2ycnfkeAKr1t7Bbr2Fs9Mwb9XPIxPpKQCoH\n8PfLQA+OuzqoVkXBNaY/wNlTa7PfqZBf7Zbf7T33yOYw0IQyjJNnYpM6kDJb7hM/uzw5DqcEQogy\nGv4Gg8FwZGFY7+88lsiZgXbCg8qeyITDfWi74axEYvrkkzIFjR0rB9u9O6xz8VbmfFu3ipGTrC+0\nKZGxTb9RZsoaGopJC3XK25tjOyoXhPIiuXzpJWlpG+PMzXbGhMAOTjy3KvOxSY3Dy9bJ8wntW5z5\nL79c2tZGshPtkDft8+FNnCjta68BAF5Xw+nk225L++qBe3rqelW+G23kjeG4KBTlHheWAlJtjdvo\n+6BHPs9GS/ly/MdMO8S4cdLy+VDR4LO+4AJpw3HfmDXZF47w2sZH9cKqtqp2ShWCggtjoeMXhGLa\nBz4AAPi/br4OALB6c6r9Uj4Na5dljz9Vxy5VZyARWFf9JADp74oaSl5Bpf7S8wcDY8AGg8FQEIbE\ngPsr9sLZo3aCMKvOJMZWbL8MRAB6Eyky1gULstPLU0+F7nI1hELYLL38dBaHccC0N3Hym3nDDZmL\nfLNbYlPDmY33ETqii0SedzuO842LGGFCutPB5hMAAFWc8XVa37Zbno8GhGRslGRmnPlnzZK+ZHfn\nnh2kG9IIyoepVX02K/PlGuStSYAsUEsm01xe0ScxW43jTuOkN6D3GKbfgWDmZ/jMyKy4bf/+P+o3\nL2t7QtLXuRkA0nHNMc3fGcfp6af3vqZSsbTBwnuRUczaAaCq43X5h2OAjg0KiuMpdGRoJE1iMF68\nWFo+EHVkzAwNuw/reGM0ThxKwZAWADjjDABA/dxJmetlV/4eQj/JoQoDDRTGgA0Gg6Eg2AvYYDAY\nCkLJnHDUBqgFMCSEmkWckBH+T4P5WWdJy3CfBx+U9qmn1gVnPQUAcOutJwMAzjxTtlLdCoP7qYpx\n22+XyHyzdasY66lahGE/VOV6enovjVQE8vxdVJEo68TZSMsA1TgAVXHFo3vvBQC0qPfmvPOuAgA8\n80x6fMqFLdOUKaedO9N5e84cSYVtWv4r2aChflSqX9W2I7j+wIBRVohrJMeFiyjncGUMOkQZ4E8T\nRBzCFJoIGHa2f79WncLa6EpS24H3YtKprZXcZjoE+dtJkzrSvYs2nzkn980COHW1B9MveaG0ffFi\naUPUrKuu6tQM01Svoax8IHwQ/EynOrNggHTw8kXEHwsTgsJYwg9+EABQs+Qx+Tx/vl6qmN7yki5K\nZeYxBmwwGAwFoWQMODZK08HAGTtOIQaAacqP5l0c5Qs++ywA4PaOy7TnG8k+F198DQDgC1+Qzw31\nMrs+82xV5jxA1rEEpBPmU09JyySE0HHHWbu6urjFQA+FkE0BwJVXSpuksoaeTj6IODZJsy0mKZ37\n9rcvS3ahs4GHefRRafl4Vq5MD08H5+WXy/6TlU3MVprXqc+yLrxgvYZy0DLynCiMgCK7Of54aekL\nCh1t1PA4rnk8KiEkdkxuAYB33tEfRZKUNENb9bBhXNL3lFNkQPKZ89nQp0pfUuh/KrsEjLwF8bgE\nC4WtP0I6xp9YnHb9+MdFTi3jZAxvnHo+gPSep23+be/z6I98o36cxPi3xAEceNSWL5eW2V2qSu7Y\nIU65PLZbqpK2xoANBoOhIAwrDC2ccDgjcEbmZHLaadIqMUoDo4GUFpClcVbS9oEHFmjHk5NdaOZp\n+OYX5Z8bbwQAvP/+7MyhgN42PH5mcDuZb8jK+ytBN5rIC3OhjMnykzTjdcuyHUJKP2WKtHEYzgoN\nEPvKV+QY96QR8u3t5wBI2W1U/wX33ZdW1b/vPmG6//k/02Ynnb/xjX8HAPz1PWoFpkEfSOlbVLS9\nCOSxRdp4aVPn/be8r3xqfCrfSy+QB7TtHWFpZMcXzZFkmION4m9guqxADMTjxwuljn0hoRZH5rtw\nobQzT1d7qmoYr9bLuH/hhT5usEDwHZBJoOALgq3eNEvDTlBZUNsAgBanHoVVHQCASfpgVndoiVO1\n2WYyWXTwtqkGlvxYGPZ2Warx9TKa6wPhc+FvkNpxuG24MAZsMBgMBaFk1iLaY1hMnezh3JN19npQ\njTpPPJHuxJnngQcAAH/Ug5xwikQ6fOtb3wEA3HXXNckuLS+qp3JdGBmRb2PW+OpkJqNdjm1cdg7I\n2naKtk/GiFeATpjSb3RWp1E7SLPc1TwZANCw6J9kA73DZMR6kI0Tzkn2eU4CJfCTn4idbvx4mfop\nxzFjjk/67t9P5rtEW3neP/jBQgDA4sUfAwA88sh/SfapzzEJlhNoluTwnNaoY3itDpyc9XLGtosd\nt+6Rf5XtGkpRdckl2i1Nkz3vPEmh1ZpEyfOkwz5coZrjOSkodbs+HDXQT7v5ZgDA1hPT51cuKcgJ\nQuNpbPtV+fH9UbdZNKZzUzM4uqol0ua5HTLW7pVbxo9/TBVKns911/1Jss9Pb5PfQtXXvy4b4hcE\nnUhAGplBO/RW4aWx2yROoy4FjAEbDAZDQRgSA87zCnKWIPP9yFSdsW+7XVoy1pxKy/t0+tOCiThB\nZyuadsKFNpPZinRQjWutH1P7D1kgkMYaniw25AnzhaUwnZMRGuW4YGR/iwby1pNCJgyGjvNTATR0\nauonmS/lT9qvIRSh+YxVO1nRfupUYby0Q4Y2ys2bRc247TZpd7NuIg5kLiWMUw0L8RetZeTFdnIM\nJNrUU2pHZIB6GHKgP4K6VWqHp41dd361U5gvU+aBlPnyOdI8zvEYFm9PnPVU22i7Z6txs9VTezPg\nosdyIqZQxaRQeWNq5ObQ3dssbDccLysWSUvl+b77/k2/uT1zvp/97P7k/1tv/QgAYNL/+B+ygc6d\nvMo64fNEn7XvR2TZJ2PABoPBUBCGZQMOGTCJVWIvu0dtVWS+9ISGRlq1udRodspU0rBwDSJkQ/ZQ\nq/tz7frFizN9M1MTKZsegBNdvDR1XphifX35xQHzepmptbFHitnwlpvb5HND95vpTqQNvPmbbsoe\nVOlWx2/STWS4u3cL842LT4fshFoKIwe2bTsx04de/JBkhJlF5SRj3h9tsQnjYWgDN4Q3w3HNoGmq\nD9/+NoB0+FNOADBpwr7McdauFW9+XnRRov11RufmF/obCn+L3L+oYjwE72fX3nQRhga97n0aA82g\nBMopXnYISOP5Fy9m/HS2tCpwNgDgpJPGJFs4/iZdeKH8c+ed0t5xh7QhK+d7Qn0nTfqcm/Qzy1Tm\nFe2yguwGg8FQobAXsMFgMBSEYTnhQvqd+H1oAaAeywr3/6aG83DpUupI116bPQGXYlBN49KF6Sq8\nuEFUiV2qszQwNiRe9hdIlgvY9p6oeAdUhaDBP6/0aGghKTdQ7eFKvKmzLPv9lVemdXYn05xDW0AU\ndP7LJSKb73639/koCyauXH21tKF6yMQX+gFjcw5P15cDo2gnXAjmBlC+HKqTePFxIeZwJz4M1j3W\nQUbTQ9PmYB29DXK81T3iFI7TmJk7AwDT2tVcwVWnr9GQTIZx6XPdn2brl40TLk6AAoD3jhbTQ1z+\nNzZvhcWhWPAIYCYEwyz5W/8kgNSSAATp+jzwj34EAOjWZ1hPTzyQ2jh4coZx6sPrjt4bgK2IYTAY\nDBWPkhXjSar890iw9KzLZVaqorH7D7quW8gidKbZc/1fZL6a0Syz/qVTNY31+m8mu+zQpI0GbiDF\nIPMNvR16UeOmCtNYpOEscanA0KeSE8lVNogN/nG5Ta00mfgbAGDKFHEgLFggLdMpNdclCYF6O80u\nTsqCMpGFn/mMGY0F9JZT6sCTlqFVoeOuVA6MUiBvzT3KsxcDjuka0Gtp4x4NfaxWQSRMKaRP+l2H\n+u3IsLhgS1P3xqTrnh55bpgjYVW1Ory55mLXXilzxIUfgN4ptEWjvzX04j58XYSOd2oImzbRySbv\nmLFjJcmHxajCZd4aenRdxA6VuybE1OepuDypVl3ad6GkKTNycyRhDNhgMBgKQsk4SLyCKNnswoUS\nIN50tc5EYQpxOGWFIFtQKrdXWS8ANNLmS6bLsCplFc9sn5z0Zex1p8assxA2ywvSthnOxmQs5ZAk\nwOsgeG2csL94rc7yyVQtLClkwEuXSrt4saS5OCe6g/dMexGb3BlnpEatsyWqJ0ke4Mq7NR2ikexp\nm5b0ZbHtfT0ylzOsqGWsMLR944ShhQw49CGUUxgaZc0wNA7VWdfOAwBU8eZ0xWcA6UPRcLSE0Shb\nrotW4AaArlqx0VMOHMJ1K7SsYqBW9DRmSyJWQeTNNf1YGjRvnJSDhtEXYj8So/jiTHkgTZwaN46l\nOUU1o0+CbWv9rnSnei1d2S2JMG13/B2AVH5MNwZSeXGMdqo5n2ydxfbDQl2lkq0xYIPBYCgIJZ8j\n4xVw2M6dexEAoPn0i5K+jGG/97vSkixM/ZLYeGqUgtV+4xvpCdhJ3Zw/f0eOt0VtmTmmtoSNJ4xG\niTezlvMKLZcLO8uLHphUq8kuTyu1v+UWAMD/eeutAIArrzw/2YflO1esEOZLcjVxonw+7zz5HKa/\n8pmxb021sIaNtcJ8J3X/Me2sLKFGDY8t4xgxIHSiRp9XfX1itS/bgke8LibqMIeF5HWeprRnBhmh\nLvgqJgh96UvSakJGGOnT1CmJMpf2KO378abscf/0T5O+tOdWbVa7sEb7cEzHq/QA5em/iMExRtZJ\nBsz7CqNp+D+fD58Hf7etzUxsSfd5fpVwy8cfl898prW1sj0MluK1xGsXEMx5iRdCKAWMARsMBkNB\nGFZB9hCcNcLay0AaHkkPfRgEwf9pjzz5ZGFJ//iPsv0rX/ksAOCGWz7b63x0SG9RmxHDL+nJBlJ7\ndGxvok0pLrpR7ugl96ikJOnEjFf+36TLX18vRe1XXz8z07Vub1fmoAeb00UQeVjK9PUOmafJsg4G\nCyYmzCwOKYkKnISMgygXLSMGxzLlQLvk6ddKNE1DqDJxORuWVdQfQC13ZksqBqSBxjQ284QLFmT3\nAVAVDV4+C5qhGdWStzxYOdqAOSx4jfFivnx/5BVJ4hiK1x04WC1+jKqtHck+TO+mDyQelmGBL9Y1\n4rV86EPS0hdCpTtPG7VUZIPBYKhQ2AvYYDAYCsKwUpH7C/SmFkWVgtl+3v8hp7fEg9EUQFWDDqRQ\nm6XqQC1w4sRsn9A/QtWBGt5xx0nLSLY8Z0U5qm1Ecm3x0hjMwWQKNwUHJJkWM6/Vvk+oJ1IFeXCq\nONbChUp4Hh42qTvMdNhQcLwWelP4oHXnPdUNmUsFelknCkHe2OU10kTF6+R6a1R5Tz89DXWcNF5t\nYSqHWq3shyeflFZTYDO2Ofb58IeljXORw0pdelEvd0jYGZ1VrD7H0KjwkZRbGFrOAiKJOPgdt/Pa\nw/th9T8mAMUmieQYE9LwSJoo+T5IEsU6sucHgPXr5eTjx8vJ6bDjYwizlgkLQzMYDIYKx7De4+Es\nECcJcFbijMNVTo855gO9jsPZjrkV3Jdx7yF7IlngcfmZJVvDcJx4lqKRPZ7RyoUpHArJdSpdOHi1\nOCer1r0s2xlnF6ZbshgSET2gqh4J4ZkzJ63ZSnnzO554T2M2gQAAap7TlSDeeCN7fKUnZDZkbMGp\nC0WeFkdGRabFW6FYSe7D+59ENYFUi6qE0uY9OlDD2i1VHKTMB2dhHc39ZvIAALyljlD67fhb4bXy\nc9G1f/tDnoOQzkNef7wCdciA+bvldxzeLB/OxdXpNAbSccfwytgRn13URITH502tOl0DMbtvKWEM\n2GAwGApCyd7pnB04c3GGYSGX0KwVI7brkkxwpgsJXWwm43fnnqlsjUay8Eu9uFdrJXyKzDpe6KDc\nwXvfVy32wHUaHtXZKeFRp50mn1umd6U7aRT7nmZJaa2r1bAxyknpQ1NYy48MjdRDP9dt1fXlQoFF\na3v1EmYZ2Hv7Q54WF5tkOZbJhJnSDgCLF4vmcO65oo20/7m05+rKC3W688ub00QUjt24LOrWJb2v\nry+GGzO6vLCtcgQ1IWoZVALIVDmcQhttXAOJ2u+LL0qbZzqnNh0X1+Kwp105PF7MsOPQyZDJmw3Y\nYDAYKhwlmyvjGTpeUTRcrJiIZ3e2nPH4fciAGcnQMl7SYxN68rh6nQ8cSDvzO2Vncapp3urOlQAy\n+DgAgYxg/vzUhlin03bduudlQ5zPyYo7oVEsdi/rg/jdWi1sEtjZJ7RlryG2TRLhYylXxKtOk63x\nMxMyQiXrN7qW3t//vbSMSpg1S5hxW5u0eYkoTLGNmReLRYXXFHv+46SESgG1TjLfpmotoEP1Yp1Q\n3/o5n0z2oXzi6JmY3YYrT/M9sXOntByzPBYzxoH0fcDj8BmSJbMdifeEMWCDwWAoCCW3AfcVo0fk\nFb6J+8RLgISzPO08r70mc0dnp9hD58yRIsqZknR6sm3vpB7+/q6tUsDi6bx+3g9n+ZAp1JFZxNM3\nXch07YcUjW5gpsYqWMQojLWO4zl5Gl4D2UNYXKXcWRuvj0yIrC1PI9P660lB9DiVNk69BdIoHKaF\nx/HqeauNk43zWvqyCZcjwmvj/TTVq8/maVUnqPbqQGntuT/ZpzUKe5g9S9VpCjUJ8g2EolS3RYU6\nbe4E3Szvi1DLCBcjCL+jrMeMwYjBGLDBYDAUhJLPm33NxAxHDZf5iKv60btMrySZQZ7thQ56ToJk\nE1Onpt5mbiMzjGe2SrD99leUhKySn+P6LwBQPUuKtDdU68KmZBqkdzSchbX2eICoDuB77wl7yIvp\njRffjLWLcmZoMfpa+inO1AR6R0wQcQRFf+eJM9fCcXmorLZKkGveNe7ThQBqGP7AUCi+IFghBwAW\nL5aWP+hHH5U2Fno48Pkdx7fuO1PHefOCNJuRp8zLwgu3WxywwWAwHEawF7DBYDAUhBFTYOIwmTx1\nn44LarpTpkhLVY9aBFfYBdJCOnFVfKoNYYgU+8S1ayrB9JAHyiOuRdqX4wtIzTm1tWI+OK5F6gKP\nUTklZosN6T5cvSI5X2TuyVOR+ygDXNGIzSmxCSj+P+zb1+fwOAPdfqjvKgmx34zrm1dr0abGuZLu\n3hDGlMVFg2PvM+MCw4UHafOMPZt6jObgPcF3Rl+/p5GUvTFgg8FgKAgj7oSLDdihgfvMM6U91EwT\nMmA60Po6X953AymfWUnoy0k0EGYfrhgC9CYTecftT8blvPpCqTCYe+ur72CY8JGIeIWMTjQE38r/\nx6oTPU7qab5SHGqhPLn68cGIY/I8YVhkX2Gpo/F8jAEbDAZDQXB+EMvSOue2A8irqH644gPe+/Gj\neUKT8cjiCJQvYDIeDQxJxoN6ARsMBoOhdDAThMFgMBQEewEbDAZDQbAXsMFgMBSEIb+AnXM/cs7d\nHHx+1Dl3V/D5h865rx7iGMsGcJ4O51xzzvaFzrl5g73unOP80jm39tA9Rx+VLmPn3BLn3O+dc6v0\n74ShHmukcBjIuMY597+dc68659Y55z491GONFCpZxs65scH4XeWc63TO3T6UY+VhOAx4KYB5AOCc\nqwLQDOC04Pt5APoVmvd+OC/QhTz/UOGcuwpA9yE7FoeKlzGAP/Hez9K/Pw7zWCOBSpfxfwPwR+/9\nNAAzAPx/wzjWSKFiZey93x2M31mQ6I77D7XfYE4wpD8ArQA26f9nAPhnAI8BGAfgaAA7ANTo998A\n8CyA1QC+FxyjW9sqAH8HYB2AxwH8GsDV+l0HgO8BeB7AGgDTAbQD2ApgC4BVABYA+AyAtQBeBPC7\nAVx/PYCnIYN27VDlMJJ/h4GMlwCYU7QcD3MZbwJwbNFyPJxlHFzDNJW3K5Vshpzr4b1/0znX45yb\nBJldlgOYCOA8ADsBrPHe73POXQTgFADnAHAAfumc+4j3/nfB4a5SQc0AcAKAVwD8U/B9p/d+tnPu\niwC+7r2/0Tl3pz6U2wDAObcGwMe991ucc426rRXAXd77S3Nu4fsAfghgz1BlMNI4DGQMAD92zh0A\n8AsAt3gdyeWCSpYxvwfwfefcQgCvAfiS935baaRTGlSyjCNcC+DnpRzDw3XCLYMIlEJdHnxeqn0u\n0r8XIDPTdIiQQ8wHcJ/3/qD3fiuAJ6PvSflXQoSfh6UA7nbOfR7AUYA8+DyBOudmAZjivX9gYLdZ\nKCpSxoo/8d6fAWEdCwB8rt87LQ6VKuNqAG0AlnnvZ+t133aomy0IlSrjENcC+Nkh+gwKw812pm3n\nDAil3wTgawB2Afix9nEA/tJ7/w/DOM/72h5AH9fsvb/JOXcugMsArHTOneW9fzuvL2TmneOc69Dj\nneCcW+K9XziMaxwpVKqM4b3fou1u59xPIczmX4ZxjSOFSpXx2xANji+d+wD8+TCubyRRqTKWC3Pu\nQwCqvfcrh3FtvVAKBnw5gC7v/QHvfReARsgLjkb1RwH8mXOuHgCccxNzvOFLAXzaOVflnGuBGM0P\nhd0AxvKDc26K9/4Z7/13AGwHkLMOs8B7//fe+1bvfTtkRn21TF++QIXK2DlXTY+0c26M3kNZRpug\nQmWsqvDDwXk+BuDlAZyzCFSkjANchxKzX2D4L+A1EI/mimjbTu99JwB47x8D8FMAy9X2sgiBMBS/\nALAZMnjugagfOw9x7ocBfEpDQxYA+IFzbo2TkLJlAF50zrU65349rDssHpUq46MBPOqcWw1xfmwB\n8I8DvelRRqXKGAD+K4Dvqpw/B2GV5YhKljEAfBYj8AIum1oQzrl67323c+54AP8O4MNq4zGUCCbj\nkYfJeORxOMm4nCqSPqIeyRoA369UgZY5TMYjD5PxyOOwkXHZMGCDwWA40mC1IAwGg6Eg2AvYYDAY\nCsKgbMBNTc1+4sT2EbqU8sOWLR3o6up0o3lOk3Fp0dzc7Nu5PLYBALBy5cpOX8IVMkzGvTFQGQ/q\nBTxxYjseeui5oV9VheGKK+aM+jlNxqVFe3s7nnvuyJHnQOCcK+lyQSbj3hiojAuNghjoirrhaqV9\nrXBsK8ym6G8F3uGsDG0yNhhKC7MBGwwGQ0EYdU6Tx8D27s3vk9d3//7s5zFjpK2t7fucfTG3w43R\n9cduY5kOhAnHGkqevI4U2RoMIwFjwAaDwVAQ7AVsMBgMBWHEFMVYxe3P9ECzwrvvStvdnW07O3vv\nSxW3vl7aCRN6n4dmiWOPlXbcuOz24TikygkDkTW3vfeetO9r0T7KODQD8XnQvEMZH3dc7+PGJqDY\n2WemCIOhbxgDNhgMhoJQcn7Sl7Onv9AoMq62NmmntekqQaS+K4IKdqSz550HAHi+owkAsGOHbA5Z\n2oknSssQxZdekrZRF3KZOlVaMrxKQyxTsliyXCDVKigfinSrli8hA+7oSPfZvDl7/OnTpWWsPR8B\nkMqYMmVLmcbMOP7fYDiSYQzYYDAYCkLJuAjZUszCyMDyQLbEPuee9Kb884Au83TMMdKGMWY05K6V\nxRVmk9Jdfrm0IZXrkdvbMXUyAODpp2UzGV5zc7YNr7+cEcuaIiDLZQukTJf3OEcTz8h8yVTvvTfd\nh8d78sl3AADPPlsDABg3TqgvNQcgfYZkx/FnsudQxnms2GA4EmEM2GAwGArCsDhIaIMkG+M2etnJ\nxvLYZsyEdtW3AgAaLrwQAPBmjywHddRR6T4tv9IVqEmtZs0CAKxeJyxtZpiQoSeYrrbllhZpx+oi\nJ2R64fF5H9XVQDmUSs6znb8jxBSvvSYtSX9I/gnKm+JiW9e5Uf558EEAwOzaINTkSyJT3KwPhpRX\nD7ZsQ7pM17p10j77rLTUMngtlOecoOQDr8kYsOFIhzFgg8FgKAhD4iADiZ89+mhpaQs8/XRpa3r2\nJH129dQBSBnRkiV6UdXCsGifXLUqPe7/cf2VAICDjRL98Mgjsl2JMB5bNy3py3O27t0FAGh5V1ne\nr+REk7TDxgnn9Lr+6mrAjWohyiwGwnwZHEKbNtuzzkr3IeMlia2Dyn/RImnvuw8AsCOINKEJWRUH\nVH/5y/KP2tnnXXBB0retTeZwajO//720tD2TCYfVCmknNhiOdBgDNhgMhoJQMitcnJlGVtvQ+br8\ns0rZJykZgIa1y+QfZaILFzbI9o7VmYN0dLQm+9y/RJgvSRhjhye1HZS2OQ1j2LZbGHZiiGZAMI2/\nShnrp/ZmwEVnycWRDgCwbZu0ZJWMZCC75DO47rp0nzir8LcrRCbbJ34VAHDNPwurbfzmN5N9GklR\nFy4EAOy68j9krmnVkvT4ZL7UQMi0N23Kfp+Hnp7ysLMbDEXBGLDBYDAUBHsBGwwGQ0EYkgmCqm5e\n0k+o5uAAAA5aSURBVEJieujWpIpXXpGWSRVPPZV2pk6taKDqq3aF13eIuWH58rTPz38u7cUXS3vt\ntdJOnChzScu49JbGc0Wmdap/0zPEyj1qx9i6OT0+L6G7uzzU41DGTG6hRYVqP80xNBHMnL4v2WfX\nXgnPa6gW59seiAmirvagHkwf2PXX9zp318KrAABr1Ql66qnS0uwT4oknpKUJirK3pAuDoW8YAzYY\nDIaCUDJeQqbWAAn5Srw+zDNm7FSed4vR+3TQvfEGAGDyJZcAAC65JJ0nXnxRWjIuOqCYiXwQNUlf\nEt7WmH4pLevqacj0A1KmVltbbBgakScu3vOVEpGH1q3Pyz+kmw+uTfo2UKb6POo0hRtz5wIAfr15\nJgBg1tyrkn3o5GvTRzh/vrRVENY8blz6PGq6u+TUl4u2smFD9hhk6/054wyGIxXGgA0Gg6EgDIsB\nh8SSDOdgvbDKqh2ao0pqHJaUjHcifSVbW79e2htvBAB8MjCEtvzopwCAX/xCPtPmfPfd0oYpr0nR\nmBf0eKRnSnmbdOc5c85P9iHjDAvaFIG8MDSa0Wn/bn3hV/LPM89IS+NsuEQ4jdrMGd69GwDwfI8w\nX7L/MB173tQ/yj+PPy4tha3UO9UxAPzmNwCASX/6p9KqQXr1nNkAUhdAyIDNHmwwCIwBGwwGQ0Eo\nGRchw9m+XdqWuP5hvHYQkOTWPtN8GQDgXY1GOP99tVMyXVYTAgDg3FPE5jj+JrE5kiHS5jizfVd6\n/Acelnblyuw1sKK4UrGQnTFHo+gkgbyViElmJ+99Wf7hfZHG0jhMigwkZTuT3PDPfAYA8PgD8pFZ\nxnV3/k26D4VAjeGFF6RV23xGPeD/zBdXtaP9gtmZ0/e1onI52NkNhqJgDNhgMBgKQslswMSWLdJ2\n7Jci6CxL2Th1BoCsV7/5DGmXKtH96s0am3rzEmnJ6G6+Od1Jjb2T1T58/xNic2bhnX21DUnXmg9+\nEACw51N/AiCIfVXs6pb5Z0cQBVEuBdkp23D5H9qAE3s6aT9t53G9zfA71TxWb27KdKnrVntv+GB4\ncp6HF3HmmdJuzgmcVta952KJplikBd7jcqThLgbDkQ5jwAaDwVAQSh4HTPMhne50yJNMhSZgRikk\n1Q3Vo54cjAw4TIXT9K/nNwjTJbPi8cNgi7VrxQ6Z2lNlviEDi0s1higX+2TCehEsHhqvJsob5AqZ\nQcGjrr2S+RYvxkkZsOh9a6hlsMZnvNImo1WobgDJs9rYKFEVd90qm/n8eam2KKfB0BvGgA0Gg6Eg\n2AvYYDAYCkLJ1oSLEa/SwED/sJALNVqGgb0+XcLRJjO8imFQb7+d7qSOtb2qStM6wS7Mag7PTTMF\ni9fQwkENPrymUD0uhzC0EIm8eSNEYpsQ7OtJ51XmtNCUQesEZfOTn0j7/vtpesV3bpQ05cR+ENUH\nfnNvU9J3h17T3XdIS5MTo97yEI6bcih4ZDAUBWPABoPBUBBK7g7humVkOWSXLBv5FzcGoWBKw7rq\nJwEAmno0JIqeNFLkG25IdtlTK+yrUVksmR2ZV16EFOsC0RFFAkmWye1A+a3YG14HmTuLCMWrIdM3\nNg2vJvuMHStr5M2YIAksfDCrOsX5xizmDKmmSsKEDqXNG7tF9iyEFJ578WJpJ06U9sCB7KEsEcNg\n6A1jwAaDwVAQSsbzyHiZeEEwkiyp9830YiAx2DZpUgXuukta5q8qbX61M7U5khxrNcVeYVU8VHju\nt97KXlsQpQUgm1kbMuByYGehvZTXSdZJE/n+/dIm0WGBUXsqnzCNtSoo5myQ5GZC8WhYV2G+ukHm\n6SXKcqlRhGB44c6d0h53nLRkvqFNuFy0C4OhaBgDNhgMhoJQMi5COyrZJhmWBi2gbq/aIPMqc997\nb/Y7TQroulxW4110Z9p1yhRpWX6SdtGbbpKWzAsATjpJWq6CREbMNNzQ9kuUW+HwkC12Rysr0d7N\nPlQgJkyoS/ZhLaT580/I7PMrrWRJsnz+wtQ2fxCXZY5Huz7tumEeRnxNsUZCZjxmTN/3aDAcqTAG\nbDAYDAWhZAyYTIgsicvYJAXSt+ZUZeEqj5pC+/J0KeTCsNZFd0sbMlXafsncyGp5npDBknUvXCg2\n5JYx8nlfvXymTTUMo6XNtWg7ZV6MdczOed1knXwGYSz0HXdk+1KWlFsaF5zOxTwOzxdXuwzlRTlx\nG23J8fMpN83CYCgHGAM2GAyGgmAvYIPBYCgIJVO033tP2nBtMSBwvj34oLRXX518t23KPADAuHHy\nebGqy3OjTNhQfWXCAMPRqFIz5ZYqLwDUvS+6dMsO9Typnry3WkwQVLVDx13Rpof+wGuLk0UYjkbT\nQxgm9s47TOPepq14Ld96S8w+bW0nZvYFUrnzPHSoMfwtlDFNJfGCJ+xDU0T4DMul5rLBUDSMARsM\nBkNBKBnfI4tlIH7CckiRGJcWJAmMjdZzY3gTGRwda2GaLB1CZFpkZ0xJDuvngkkhPKfStFUrsucJ\naxSXC/KYOJMZoto7vRxeWRwPANi+Xdpt27L76CLJGacftQqG/HFlYyayhGyZz4jL9sUsndrFu+/m\nXZvBcGTDGLDBYDAUhJIxYDId2vq4kG5zsyQAzP7EJ3rtE9sUp3Uuk3+YLXDv5niXZPmMtqtnZjaT\neYXFeHbUSpGfSbV7AACr19Vk+pL5VspqDWT3DPWjNkBGHIeWAb01hXiNNrLZkD1zgWOy5fi8YVgg\nz0UtpaFeEjoO6tzOVbJDlLOMDYbRhDFgg8FgKAgl4yJkSbRTLl0q7b/9m7Rz50oJxW9/O92Hdsi6\n6n3yD+kzFxTjGnFnn53upGnLDbUaVZFU+RFMC++oXmhflxYQj5kvbcAhIyuXRIw8UDyMOImjIchi\nGUUCpBoBIxso2lgWoQ2Y//M8ZLdk0UlyDYB5c/TZLVmSuciqJF+5qdfxiXKUscEwmjAGbDAYDAVh\nWBwkj8HES98wRpUt6+4AKQM+7TSxzbbQmHnlldKytmR4IpaqpAGZNJAV2QMcXHg+AGCrsj5GZvTH\ncsuNleWx8/HjpeUiyNQ6Wo7ZJf8E9TXb54sdnPdOcVF8ZNWMhgBS8fMZTqvdKP9QZQjrd96xInuh\n0XJJjIoJGXC5ydhgKArGgA0Gg6EglIyLkEkx3Jd2SBIies7pYQfSut/cVl8/LdOXDDm0Oc7QgFN6\n2cmseuYK2w2Z3LsdyPTpa3mcSmVkk9q0hCQFtkENvqS3AGrqhf7X6M3PVqHObtQ+7VF4BJAKng+G\naYZc+TTPYHzaadIqbe5CWkQfqFwZGwwjCWPABoPBUBDsBWwwGAwFoWSKIVVMpiRz1Ym88COCPp14\njbE4NCrQqJO6tTQn0AGVSUGOrimuRVup6nB8P107RBZNcdWcMLea5gkKmd5QOjMpZC4sBwAf/ai0\n9MIxho0XEAqQjlM1W2zcnDUNVaqsDYbRgDFgg8FgKAgl5yd5JKkvkDz1x5KHev7wuIcbC2NIGdsd\nyoSrq2u0R2vSt6de/q9WcpzIZPpFuq987q9cZE/0T20gz1p+2TGoWzAYDDAGbDAYDIXBee8H3tm5\n7QD+MHKXU3b4gPd+/Gie0GRcWhyB8hwISipzk3EuBiTjQb2ADQaDwVA6mAnCYDAYCoK9gA0Gg6Eg\nDPkF7Jz7kXPu5uDzo865u4LPP3TOffUQx1g2gPN0OOeac7YvdM7NG+x1B/tf55xb45xb7ZxbnHeO\nonEYyPgale9Lzrm/GupxDIbDFcNhwEsBzAMA51wVgGYApwXfzwPQ74/fez/kHzeAhTz/YOGcqwbw\ntwA+6r2fCWA1gC8N41pGCpUs4+MB/ADAx7z3pwGY4Jz72DCuxWA47DCcF/AyAOfp/6cBWAtgt3Nu\nnHPuaAAfBPA8ADjnvuGce1bZ0Pd4AOdct7ZVzrm/c86tc8497pz7tXPu6uBcX3bOPa+Mdbpzrh3A\nTQC+4pxb5Zxb4Jz7jHNurXPuRefc7w5x7U7/jnXOOQANAN4chixGCpUs48kA1nvvuSjREwA+PSxp\nGAyHGYacouC9f9M51+OcmwRhScsBTIS8MHYCWOO93+ecuwjAKQDOgbz0fumc+4j3PvwBXwWgHcAM\nACcAeAXAPwXfd3rvZzvnvgjg6977G51zdwLo9t7fBgDOuTUAPu693+Kca9RtrQDu8t5fGl37fufc\nFwCsAfAugPUA/tNQZTFSqGQZA9gA4FR9kW8GcCWAGhgMhgTDdcItg7wY+HJYHnzWRYlwkf69AGFr\n0yEvixDzAdznvT/ovd8K4Mno+/u1XQl5ieRhKYC7nXOfB3AUIC+wnBcDnHNjAHwBwJmQtLHVAL51\n6NstBBUpY+/9OxAZ/xzAU5BcuQOHvFuD4QjCcJN0aaM8A6IebwLwNQC7APxY+zgAf+m9/4dhnOd9\nbQ+gj2v23t/knDsXwGUAVjrnzvLev93H8WbpPq8BgHPuXwF8cxjXN5KoVBnDe/8wgIcBwDn3F7AX\nsMGQQSkY8OUAurz3B7z3XQAaISoynUOPAvgz51w9ADjnJjrnToiOsxTAp9VO2QJx/hwKuwGM5Qfn\n3BTv/TPe++8A2A7gpH723QJghnOOmSoXQlTyckSlyhi8BufcOABfBHBXf/0NhiMNw30Br4F45ldE\n23Z67zsBwHv/GICfAliuNsRFCH7Uil9A7IQvA7gHokbvPMS5HwbwKTqIAPxAHUhrIS+mF51zrc65\nX8c7eu/fBPA9AL9zzq2GMOL/ZxD3PZqoSBkr/tY59zLk5X+r9/7Vgd2ywXBkoGxSkZ1z9d77bg1f\n+ncAH1ZbpaFEMBkbDOWFcirU+Ih61msAfN9eDCMCk7HBUEYoGwZsMBgMRxqsFoTBYDAUBHsBGwwG\nQ0GwF7DBYDAUBHsBGwwGQ0GwF7DBYDAUBHsBGwwGQ0H4/wGEpDljtLdWogAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024ec81ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also print and plot the so-called confusion matrix which lets us see more details about the mis-classifications. For example, it shows that images actually depicting a 5 have sometimes been mis-classified as all other possible digits, but mostly either 3, 6 or 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 959    0    2    4    0    4    9    1    1    0]\n",
      " [   0 1109    2    2    0    2    4    2   14    0]\n",
      " [   6    8  914   20   12    2   14   14   36    6]\n",
      " [   0    1   13  950    1   15    1   10   13    6]\n",
      " [   1    2    3    2  919    0   15    2    7   31]\n",
      " [  10    4    3   56   10  750   16    7   29    7]\n",
      " [   9    3    4    2   10   13  914    1    2    0]\n",
      " [   2   10   20   10    7    1    0  950    2   26]\n",
      " [   6    9    7   42    9   25    9   13  843   11]\n",
      " [  11    6    2   16   41    6    0   30    5  892]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAEmCAYAAABcYEo9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHVVJREFUeJzt3X+QXlWd5/H3pzv8CjAG6UhhEkwcGByKWgn2RhSlkCgL\nyACz5bgwowIyk10XHRB3GZzdWnZnt2pxtdTRmaI2EjSO/DRAyTqMgIgyWBIJIfIrKAGBJBNIWn6o\nIELS3/3jntam7aTv033Pc597n8+LutXPvc99zrmX7nz79LnnfI8iAjMz676Bui/AzKxfOQCbmdXE\nAdjMrCYOwGZmNXEANjOriQOwmVlNHIDNzGriAGxmVhMHYDOzmsyq+wLG0x77hmbvn7WOxW8cylq+\n9Y7cczyVufw2eOKJxxkZGan0f9Xg770hYvuvSp8fv9p2c0ScUOU1VKW3AvDs/dlz6X/LWsf3r/5w\n1vKtd4yO5g3BAwMOwVM5+q3DlZcZ219ijzedXvr8l+79Ys+2unoqAJuZTUmA2vHLzwHYzJpH7Xh8\n5QBsZs3jFrCZWR3kFrCZWW3cAjYzq4FwC9jMrB5qTQs4668RSSdI+rGkDZIuylmXmfURDZTfeli2\nq5M0CPw9cCJwGHCGpMNy1WdmfUQqv/WwnL8elgAbIuKxiHgZuBo4NWN9ZtYX5BZwCfOAjeP2N6Vj\nryJpmaQ1ktbw619mvBwza4WxmXAtaAHX/hAuIpYDywEG9luYO3+KmbVBj7dsy8oZgDcDC8btz0/H\nzMxmoD0TMXLexd3AIZIWSdodOB24MWN9ZtYvBlR+62HZWsARsV3SR4GbgUHg8oh4MFd9ZtYnPBGj\nnIi4CbgpZx1m1od6/OFaWbU/hDMz60x7+oAdgM2sedwCNjOrSUtawO24CzPrH51MwijRUpZ0uaSt\nkh4Yd+y1km6V9Ej6ul86LklfSPlt7pN05LjPnJnOf0TSmWVuxQHYzJpnYLD8NrWvABNXTb4IuC0i\nDgFuS/tQ5LY5JG3LgEuhCNjAxcBbKdIwXDwWtHd5G2Wuzsysd1SbCyIi7gCemXD4VGBler0SOG3c\n8a9G4S5gjqQDgX8D3BoRz0TEs8Ct/G5Q/x3uAzaz5unsIdyQpDXj9penFAi7ckBEbEmvnwIOSK93\nluOmVO6biXoqAC9+4xDfv/rDWevY719/NGv5z979d1nLt/IGenwWVBk7RvOmRxls4v+jzidijETE\n8HSri4iQlOUb4S4IM2uYrqSjfDp1LZC+bk3Hd5bjZlq5bxyAzax58qejvBEYG8lwJvCNccc/lEZD\nHAU8n7oqbgaOl7Rfevh2fDq2Sz3VBWFmVkqF44AlXQUcS9FXvIliNMMlwLWSzgGeAN6fTr8JOAnY\nALwInA0QEc9I+p8UScgA/iYiJj7Y+x0OwGbWPBXOhIuIM3by1tJJzg3g3J2UczlweSd1OwCbWbPI\nuSDMzOrjXBBmZvWQA7CZWfcVa3K2IwBn60iZLMGFmdmMqcOth+Xsyf4KJeZCm5l1Rkjlt16Wc024\nOyQtzFW+mfWvXg+sZdXeByxpGUVaNxYcdFDNV2NmTdCWAFz7YLqIWB4RwxExPHdobt2XY2YN4C4I\nM7M6NODhWlkOwGbWKKL3W7Zl5RyGdhXwA+BQSZtSUgszsxlzF8QUdpHgwsxsRno9sJblLggzaxwH\nYDOzOvghnJlZfdwCNjOrQZtGQTgAm1njqImrOU/CAdjMmkXugsgigGLJpXye+eEXs5Y/75yrspYP\n8MTyf5e1/IEu/HAPdKEFs2M0789SN2zfMZq1/IHMS/vk+g44AJuZ1cQB2MysBn4IZ2ZWp3bEXwdg\nM2sYP4QzM6uPA7CZWU0cgM3M6tKO+OsAbGbN05YWcM6E7Ask3S7pIUkPSjovV11m1j86Scbe64E6\nZwt4O/CJiFgraV/gHkm3RsRDGes0sz7Q64G1rJwrYmwBtqTXv5C0HpgHOACb2Yy0JQB3ZVl6SQuB\nxcDqSd5bJmmNpDUjI9u6cTlm1nTqYOth2QOwpH2A64DzI+LnE9+PiOURMRwRw0NDc3Nfjpm1gPuA\nS5C0G0XwvSIirs9Zl5n1Cc+Em5qK/0MrgPUR8dlc9ZhZfxHQkvibtQviaOCDwHGS1qXtpIz1mVlf\nqH4YmqSPp+GyD0i6StKekhZJWi1pg6RrJO2ezt0j7W9I7y+c7p1kC8ARcWdEKCL+VUQckbabctVn\nZv1DKr9NXZbmAX8JDEfE4cAgcDrwKeBzEXEw8CxwTvrIOcCz6fjn0nnT0pVREGZmVcrwEG4WsJek\nWcBsiiG0xwGr0vsrgdPS61PTPun9pZpmp7QDsJk1Swet3xQWh8aGuqZt2fjiImIz8BngSYrA+zxw\nD/BcRGxPp22imMdA+roxfXZ7On//6dyKc0GYWaOIjtcUHImI4Z2WJ+1H0apdBDwHfB04YSbXWJZb\nwGbWOFX2AQPvBn4aEdsi4hXgeopBBHNSlwTAfGBzer0ZWFBch2YBrwF+Np37cAA2s2ZR0QIuu5Xw\nJHCUpNmpL3cpRcqE24H3pXPOBL6RXt+Y9knvfyemuZy7uyDMrFGKccDVDQSOiNWSVgFrKZKI3Qss\nB/4RuFrS/0rHVqSPrAD+QdIG4BmKERPT4gBsZg1T/RTjiLgYuHjC4ceAJZOc+xLwJ1XU21MBuOrf\nbHV48kvT/mVY2gEfWDn1STMwcuVZWcsH2DE6rb/YOjLY2YOajo124R5mDbqXcDINDxO/0VMB2Mys\njKY31MY4AJtZs5Qf3dDzHIDNrFHa0FU5xgHYzBqnJfHXAdjMmsctYDOzmrQk/joAm1nDeEUMM7N6\ntGlFjJxLEu0J3AHskepZlWabmJnNQO8vtllWzhbwr4HjIuKXaXHOOyX9U0TclbFOM+sDLYm/+QJw\nyg70y7S7W9ryz900s9ZrSws460RzSYOS1gFbgVsjYvUk5ywby1S/bWRbzssxszbofEWMnpU1AEfE\njog4giKZ8RJJh09yzvKIGI6I4blDc3Nejpm1wNhMuIrXhKtFV1ItRcRzFMmNu7LMh5m1mwPwFCTN\nlTQnvd4LeA/wcK76zKx/tKULIucoiAOBlZIGKQL9tRHxzYz1mVmf6PWWbVk5R0HcByzOVb6Z9akG\ntGzL8kw4M2sUeSKGmVl9WhJ/HYDNrHkGWhKBHYDNrFEkGMi84Gq3OACbWeO0JP46AJtZ8/ghnE1q\nsAu/mkeuPCtr+QeefUXW8gG2fPnPstexYzRv7qdufK+LnFb5vLIjb/m5Sm9J/HUANrNmEcVQtDZw\nADazxnEfsJlZHRqQZKcsB2Aza5yWxF8HYDNrFuGJGGZmtWlJ/HUANrPmcR+wmVkNmpBovazsATgl\nZF8DbI6Ik3PXZ2bt15Y+4G6sCXcesL4L9ZhZn1AHWy/LvSz9fOC9wGU56zGz/uJFOcv5PHAhMJq5\nHjPrE8UwtPJbqTKlOZJWSXpY0npJb5P0Wkm3Snokfd0vnStJX5C0QdJ9ko6c7r3kXBX5ZGBrRNwz\nxXnLJK2RtGbbyLZcl2NmbdFB67eDFvDfAt+KiDcBb6boNr0IuC0iDgFuS/sAJwKHpG0ZcOl0byVn\nC/ho4BRJjwNXA8dJ+trEkyJieUQMR8Tw3KG5GS/HzNqiymXpJb0GOAZYARARL0fEc8CpwMp02krg\ntPT6VOCrUbgLmCPpwOncR+kALGmPTgqOiE9GxPyIWAicDnwnIj7Q4fWZmf2ODlvAQ2N/Zadt2YTi\nFgHbgC9LulfSZZL2Bg6IiC3pnKeAA9LrecDGcZ/flI51bMoALGmJpPuBR9L+myV9cTqVmZnN1DT6\ngEfG/spO2/IJRc4CjgQujYjFwAv8trsBgCgSM1ee3rhMC/gLwMnAz9KF/Ah4VyeVRMR3PQbYzKpS\ncR/wJmBTRKxO+6soAvLTY10L6evW9P5mYMG4z89PxzpWJgAPRMQTE47tmE5lZmZVqHIccEQ8BWyU\ndGg6tBR4CLgRODMdOxP4Rnp9I/ChNBriKOD5cV0VHSkzE26jpCVApFltHwN+Mp3KzMxmSsoyE+5j\nwBWSdgceA86maKBeK+kc4Ang/encm4CTgA3Ai+ncaSkTgD9C0Q1xEPA08O10zMysFlXH34hYBwxP\n8tbSSc4N4Nwq6p0yAEfEVopRDGZmPaHXZ7iVNWUAlvQlJnn6FxETh3KYmWUn1JUVqbuhTBfEt8e9\n3hP4Y149Bs7MrHv6KR1lRFwzfl/SPwB3ZrsiM7Mp9E0XxCQW8dsZIZUKYHS08rHOrzLQgj9dXvz1\n9qzlb17xp1nLB3jHJbdnr+N7Fx6btfziWUxe23fkrWO3wbz/HnKV3o08ut1Qpg/4WX7bBzwAPMOE\nWSJmZt0i+qQFrOIu38xvZ3mMRjd+7ZuZ7UIL/pAFpmjJp2B7U0TsSJuDr5nVrup8wHUp05WyTtLi\n7FdiZlZCkWayHSti7LQLQtKsiNgOLAbulvQoRZYgUTSOp50F3sxsJnq9ZVvWrvqAf0iREeiULl2L\nmVkpPd6wLW1XAVgAEfFol67FzGxKRT7gdkTgXQXguZIu2NmbEfHZDNdjZjalfhgHPAjswwzGUqf1\n4H5BkT94e0RMlm3IzKwjLWkA7zIAb4mIv6mgjndFxEgF5ZiZIakvuiDacYdm1jotib+77Er5nUTE\n0xDALZLumWQlUgAkLRtbrXRkZFsFVZpZ27VlIsZOW8AR8UwF5b8jIjZLeh1wq6SHI+KOCfUsB5YD\nHPmWYc+0M7NdatMoiKwPEyNic/q6FbgBWJKzPjPrD1L5rZdlC8CS9pa079hr4HjggVz1mVmf6KD7\nobFdEBU4ALghzcWeBVwZEd/KWJ+Z9Qm1ZIxAtgAcEY9RpLI0M6tM0Qdc91VUI2cL2MwsCwdgM7Oa\n9HqaybIcgM2sUdwFYWZWF8FgSyKwA7CZNYpbwGZmNWpJF7ADsJk1jRjwOGCry+w98n7bRkfzp+T4\n3oXHZq/joL+4Omv5m1eckbV8gN1mNTvQ5Lh64RawmVk9GjDFuCwHYDNrnLZkQ3MANrNGcReEmVmN\n3AI2M6tJS+Jva1Z3NrM+IYrAVXYrXa40KOleSd9M+4skrZa0QdI1knZPx/dI+xvS+wuney8OwGbW\nLCqS8ZTdOnAesH7c/qeAz0XEwcCzwDnp+DnAs+n459J50+IAbGaNow62UuVJ84H3ApelfQHHAavS\nKSuB09LrU9M+6f2lmmZ6tqwBWNIcSaskPSxpvaS35azPzNpvbFHOshswNLbyetomW6H988CFwGja\n3x94LiK2p/1NwLz0eh6wESC9/3w6v2O5H8L9LfCtiHhf6j+Znbk+M+sDHTY3RyJieKdlSScDWyPi\nHknHzuzKOpMtAEt6DXAMcBZARLwMvJyrPjPrHxWPgjgaOEXSScCewO9RNB7nSJqVWrnzgc3p/M3A\nAmCTpFnAa4CfTafinF0Qi4BtwJfTk8XL0urIryJp2difBiMj2zJejpm1Q/kHcGW6ZiPikxExPyIW\nAqcD34mIPwNuB96XTjsT+EZ6fWPaJ73/nYiYVgKVnAF4FnAkcGlELAZeAC6aeFJELI+I4YgYHhqa\nm/FyzKwNcg1Dm8RfARdI2kDRx7siHV8B7J+OX8Akca2snH3Am4BNEbE67a9iBhdqZjYm15pwEfFd\n4Lvp9WPAkknOeQn4kyrqy9YCjoingI2SDk2HlgIP5arPzPpH1cPQ6pJ7FMTHgCvSCIjHgLMz12dm\nbSevilxKRKwDdjr8w8ysU2N9wG3gZDxm1jhuAZuZ1aQd4dcB2MwaRsCgW8BmZvVoSfx1ADazphFq\nSSeEA7CZNY5bwBkIGMi83vTo6LSmbJcvf3pTwjuSez2sbvxwd2NNr80rzsha/tCffiVr+QAjV56V\ntfxXto9OfdIM5PjXUAxDa0cE7qkAbGY2JbkFbGZWGwdgM7Oa+CGcmVkNiiWJ6r6KajgAm1njuAVs\nZlYT9wGbmdXELWAzsxq0qQ84W1pNSYdKWjdu+7mk83PVZ2b9Qh3918uytYAj4sfAEQCSBimWcr4h\nV31m1ic8EaNjS4FHI+KJLtVnZi3WkvjbtQB8OnDVZG9IWgYsA1hw0EFduhwza6qiD7gdITj70kpp\nQc5TgK9P9n5ELI+I4YgYnjs0N/flmFkLeFXk8k4E1kbE012oy8z6Qa9H1pK6EYDPYCfdD2Zm09Hr\noxvKytoFIWlv4D3A9TnrMbP+IpXfelnWFnBEvADsn7MOM+s/PR5XS/NMODNrnpZEYAdgM2uUYnRD\nOyKwA7CZNUsD+nbLcgA2s8ZxADYzq0XvJ9kpywHYzBrHLWAzsxo0YYpxWT0VgAOIiLovY0YGu5Ap\n+tfbR7OWv/tg9hQh7BjNew+QP2HL0187M2v5AId+4v9lLX/9p0/OWn42LYnA+f+lmZlVrMqE7JIW\nSLpd0kOSHpR0Xjr+Wkm3Snokfd0vHZekL0jaIOk+SUdO9z4cgM2scSqeirwd+EREHAYcBZwr6TDg\nIuC2iDgEuC3tQ5Fg7JC0LQMune59OACbWeNUmY4yIrZExNr0+hfAemAecCqwMp22EjgtvT4V+GoU\n7gLmSDpwOvfhAGxmzdJJ9C0i8JCkNeO2ZTstWloILAZWAwdExJb01lPAAen1PGDjuI9tSsc61lMP\n4czMyuhwHPBIRAxPWaa0D3AdcH5E/Fzj+i8iIiRVPkLALWAzaxRRfTpKSbtRBN8rImIsfe7TY10L\n6evWdHwzsGDcx+enYx1zADazxqmyD1hFU3cFsD4iPjvurRuBsbGGZwLfGHf8Q2k0xFHA8+O6Kjri\nLggza55qxwEfDXwQuF/SunTsr4FLgGslnQM8Abw/vXcTcBKwAXgROHu6FWcNwJI+Dvw5xRyL+4Gz\nI+KlnHWaWftVmQsiIu5k5yF96STnB3BuFXVn64KQNA/4S2A4Ig4HBimWpzczmxEvSVS+/L0kvQLM\nBv4lc31m1gd6PK6Wlq0FHBGbgc8ATwJbKDqqb5l4nqRlY+PzRka25bocM2uTKp/C1ShnF8R+FDNG\nFgGvB/aW9IGJ50XE8ogYjojhoaG5uS7HzFpibEmiqnJB1CnnMLR3Az+NiG0R8QrF0vRvz1ifmfWD\nDvp/e70POGcAfhI4StLsNM5uKcUcazOzGWlJD0S+h3ARsVrSKmAtRbahe4Hlueozsz7S65G1pKyj\nICLiYuDinHWYWb/p/b7dsjwTzswap9f7dstyADazRmlC325ZDsBm1jhqSRPYAdjMGqcl8dcB2Mya\npyXx1wHYzBqmARMsyuq5AByVL/rxarm/cd3om9ptMG8e/Zd3jGYtH2CPWfnXAsj9vXhle/7/Tw9/\n5uSs5S/8yKqs5T/z5LOZSm5HBO65AGxmtitjSxK1gQOwmTVOS+KvA7CZNY9bwGZmNfFUZDOzurQj\n/joAm1nztCT+OgCbWbM0IdF6WQ7AZtY4bekDzjoaXtJ5kh6Q9KCk83PWZWZ9pCVLYuRclPNw4C+A\nJcCbgZMlHZyrPjPrHy2Jv1lbwH8IrI6IFyNiO/A94N9mrM/M+oQX5ZzaA8A7Je0vaTZwErAgY31m\n1hc6WZS+tyNwzkU510v6FHAL8AKwDtgx8TxJy4BlAAsOOijX5ZhZS7QpF0TWh3ARsSIi3hIRxwDP\nAj+Z5JzlETEcEcNDQ3NzXo6ZWU/JOgxN0usiYqukgyj6f4/KWZ+Z9Ye2tIBzjwO+TtL+wCvAuRHx\nXOb6zKwP9HrfbllZA3BEvDNn+WbWhxowuqEsz4Qzs0ZpwvjeshyAzax5WhKBHYDNrHEGWtIH4QBs\nZo3TjvCbeRywmVkWFSeDkHSCpB9L2iDpohyXPBkHYDNrnCqnIksaBP4eOBE4DDhD0mGZbwFwADaz\nhhmbilxhMp4lwIaIeCwiXgauBk7NeAu/0VN9wPeuvWdk7z0GnujgI0PASK7r6UL5banD99A/dXRa\n/huqvoC1a++5ea/dNNTBR/aUtGbc/vKIWD5ufx6wcdz+JuCtM7nGsnoqAEdER8kgJK2JiOFc15O7\n/LbU4Xvonzq6cQ9TiYgT6qy/Su6CMLN+t5lXp8qdn45l5wBsZv3ubuAQSYsk7Q6cDtzYjYp7qgti\nGpZPfUpPl9+WOnwP/VNHN+6hqyJiu6SPAjcDg8DlEfFgN+pWRHSjHjMzm8BdEGZmNXEANjOrSSMD\ncO5pg5Iul7RV0gNVlz2ujgWSbpf0kKQHJZ1Xcfl7SvqhpB+l8v9HleVPqGtQ0r2Svpmh7Mcl3S9p\n3YSxnFXWMUfSKkkPS1ov6W0Vl39ouv6x7eeSzq+4jo+n7/MDkq6StGeV5ac6zkvlP1j19fetiGjU\nRtFJ/ijwRmB34EfAYRXXcQxwJPBAxvs4EDgyvd6XYr28yu6DYsLQPun1bsBq4KhM93IBcCXwzQxl\nPw4MZf6ZWgn8eXq9OzAnY12DwFPAGyoscx7wU2CvtH8tcFbF1304xUrnsyke3n8bODjn96Uftia2\ngLNPG4yIO4Bnqixzkjq2RMTa9PoXwHqKf0hVlR8R8cu0u1vaKn/iKmk+8F7gsqrL7gZJr6H4hbsC\nICJejrxLZy0FHo2ITmZ8ljEL2EvSLIog+S8Vl/+HwOqIeDEitgPfo1jn0WagiQF4smmDlQWuOkha\nCCymaKVWWe6gpHXAVuDWiKi0/OTzwIXAaIayofilcYukeyQty1D+ImAb8OXUjXKZpL0z1DPmdOCq\nKguMiM3AZ4AngS3A8xFxS5V1ULR+3ylpf0mzgZN49eQFm4YmBuBWkbQPcB1wfkT8vMqyI2JHRBxB\nMbNniaTDqyxf0snA1oi4p8pyJ3hHRBxJkanqXEnHVFz+LIrupksjYjHwApAlHWEa5H8K8PWKy92P\n4q/ARcDrgb0lfaDKOiJiPfAp4BbgW8A6YEeVdfSjJgbg2qYNVk3SbhTB94qIuD5XPelP6tuBqufQ\nHw2cIulxiq6g4yR9rcoKUuuOiNgK3EDRBVWlTcCmcX8drKIIyDmcCKyNiKcrLvfdwE8jYltEvAJc\nD7y94jqIiBUR8ZaIOAZ4luK5hc1AEwNwbdMGqyRJFP2O6yPisxnKnytpTnq9F/Ae4OEq64iIT0bE\n/IhYSPF9+E5EVNbykrS3pH3HXgPHU/wpXJmIeArYKOnQdGgp8FCVdYxzBhV3PyRPAkdJmp1+rpZS\nPFOolKTXpa8HUfT/Xll1Hf2mcVORowvTBiVdBRwLDEnaBFwcESuqrIOi9fhB4P7UTwvw1xFxU0Xl\nHwisTMmmB4BrI6LyYWKZHQDcUMQUZgFXRsS3MtTzMeCK9Av9MeDsqitIv0DeA/z7qsuOiNWSVgFr\nge3AveSZMnydpP2BV4BzMz+s7AueimxmVpMmdkGYmbWCA7CZWU0cgM3MauIAbGZWEwdgM7OaOADb\nTknakbJ3PSDp62kK6nTLOnYsW5qkU3aVxS5lJ/uP06jjv0v6T9O9RrNucwC2XflVRBwREYcDLwP/\nYfybKnT8MxQRN0bEJbs4ZQ7QcQA2axoHYCvrn4GDJS1MuZi/SjErbYGk4yX9QNLa1FLeB36Tt/lh\nSWsZlzlL0lmS/i69PkDSDSlv8Y8kvR24BPj91Pr+dDrvP0u6W9J943MbS/ovkn4i6U7gUMwapHEz\n4az7UorDEymSsAAcApwZEXdJGgL+K/DuiHhB0l8BF0j6P8CXgOOADcA1Oyn+C8D3IuKP06y9fSiS\n4RyeEgkh6fhU5xKKPMc3pqQ8L1BMgT6C4md5LZAzMZBZpRyAbVf2GjdN+p8pcle8HngiIu5Kx48C\nDgO+n6YM7w78AHgTRYKYRwBSkp7J0kkeB3wIiuxtwPMpu9d4x6ft3rS/D0VA3he4ISJeTHU0LieI\n9TcHYNuVX421QsekIPvC+EMUuYbPmHDeqz43QwL+d0T83wl1eFkcazT3AdtM3QUcLelg+E0Gsz+g\nyLy2UNLvp/PO2MnnbwM+kj47mFao+AVF63bMzcCHx/Utz0uZue4ATpO0V8qa9kcV35tZVg7ANiMR\nsQ04C7hK0n2k7oeIeImiy+Ef00O4rTsp4jzgXZLup+i/PSwifkbRpfGApE+n1R2uBH6QzlsF7JuW\ndLqGYl3Af6JIVWrWGM6GZmZWE7eAzcxq4gBsZlYTB2Azs5o4AJuZ1cQB2MysJg7AZmY1cQA2M6vJ\n/wfJ8CJtZ+lFxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f024e942470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_confusion_matrix()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now done using TensorFlow, so we close the session to release its resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This has been commented out in case you want to modify and experiment\n",
    "# with the Notebook without having to restart it.\n",
    "# session.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "These are a few suggestions for exercises that may help improve your skills with TensorFlow. It is important to get hands-on experience with TensorFlow in order to learn how to use it properly.\n",
    "\n",
    "You may want to backup this Notebook before making any changes.\n",
    "\n",
    "* Change the learning-rate for the optimizer.\n",
    "* Change the optimizer to e.g. `AdagradOptimizer` or `AdamOptimizer`.\n",
    "* Change the batch-size to e.g. 1 or 1000.\n",
    "* How do these changes affect the performance?\n",
    "* Do you think these changes will have the same effect (if any) on other classification problems and mathematical models?\n",
    "* Do you get the exact same results if you run the Notebook multiple times without changing any parameters? Why or why not?\n",
    "* Change the function `plot_example_errors()` so it also prints the `logits` and `y_pred` values for the mis-classified examples.\n",
    "* Use `sparse_softmax_cross_entropy_with_logits` instead of `softmax_cross_entropy_with_logits`. This may require several changes to multiple places in the source-code. Discuss the advantages and disadvantages of using the two methods.\n",
    "* Remake the program yourself without looking too much at this source-code.\n",
    "* Explain to a friend how the program works."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}