assignment1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5hIbr52I7Z7U"
   },
   "source": [
    "Deep Learning\n",
    "=============\n",
    "\n",
    "Assignment 1\n",
    "------------\n",
    "\n",
    "The objective of this assignment is to learn about simple data curation practices, and familiarize you with some of the data we'll be reusing later.\n",
    "\n",
    "This notebook uses the [notMNIST](http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html) dataset to be used with python experiments. This dataset is designed to look like the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, while looking a little more like real data: it's a harder task, and the data is a lot less 'clean' than MNIST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": true,
    "id": "apJbCsBHl-2A"
   },
   "outputs": [],
   "source": [
    "# These are all the modules we'll be using later. Make sure you can import them\n",
    "# before proceeding further.\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import sys\n",
    "import tarfile\n",
    "from IPython.display import display, Image\n",
    "from scipy import ndimage\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from six.moves.urllib.request import urlretrieve\n",
    "from six.moves import cPickle as pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jNWGtZaXn-5j"
   },
   "source": [
    "First, we'll download the dataset to our local machine. The data consists of characters rendered in a variety of fonts on a 28x28 image. The labels are limited to 'A' through 'J' (10 classes). The training set has about 500k and the testset 19000 labelled examples. Given these sizes, it should be possible to train models quickly on any machine.\n",
    "\n",
    "--Assume already downloaded and processed data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 186058,
     "status": "ok",
     "timestamp": 1444485672507,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "EYRJ4ICW6-da",
    "outputId": "0d0f85df-155f-4a89-8e7e-ee32df36ec8d"
   },
   "outputs": [],
   "source": [
    "load = pickle.load(open(\"notMNIST.pickle\", \"rb\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cC3p0oEyF8QT"
   },
   "source": [
    "Extract the dataset from the compressed .tar.gz file.\n",
    "This should give you a set of directories, labelled A through J."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 186055,
     "status": "ok",
     "timestamp": 1444485672525,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "H8CBE-WZ8nmj",
    "outputId": "ef6c790c-2513-4b09-962e-27c79390c762"
   },
   "outputs": [],
   "source": [
    "num_classes = 10\n",
    "\n",
    "def extract(filename):\n",
    "  tar = tarfile.open(filename)\n",
    "  root = os.path.splitext(os.path.splitext(filename)[0])[0]  # remove .tar.gz\n",
    "  print('Extracting data for %s. This may take a while. Please wait.' % root)\n",
    "  sys.stdout.flush()\n",
    "  tar.extractall()\n",
    "  tar.close()\n",
    "  data_folders = [\n",
    "    os.path.join(root, d) for d in sorted(os.listdir(root)) if d != '.DS_Store']\n",
    "  if len(data_folders) != num_classes:\n",
    "    raise Exception(\n",
    "      'Expected %d folders, one per class. Found %d instead.' % (\n",
    "        num_classes, len(data_folders)))\n",
    "  print(data_folders)\n",
    "  return data_folders\n",
    "  \n",
    "train_folders = ['notMNIST_large/A', 'notMNIST_large/B', 'notMNIST_large/C', 'notMNIST_large/D', 'notMNIST_large/E', 'notMNIST_large/F', 'notMNIST_large/G', 'notMNIST_large/H', 'notMNIST_large/I', 'notMNIST_large/J']\n",
    "test_folders = ['notMNIST_small/A', 'notMNIST_small/B', 'notMNIST_small/C', 'notMNIST_small/D', 'notMNIST_small/E', 'notMNIST_small/F', 'notMNIST_small/G', 'notMNIST_small/H', 'notMNIST_small/I', 'notMNIST_small/J']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4riXK3IoHgx6"
   },
   "source": [
    "---\n",
    "Problem 1\n",
    "---------\n",
    "\n",
    "Let's take a peek at some of the data to make sure it looks sensible. Each exemplar should be an image of a character A through J rendered in a different font. Display a sample of the images that we just downloaded. Hint: you can use the package IPython.display.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PBdkjESPK8tw"
   },
   "source": [
    "Now let's load the data in a more manageable format.\n",
    "\n",
    "We'll convert the entire dataset into a 3D array (image index, x, y) of floating point values, normalized to have approximately zero mean and standard deviation ~0.5 to make training easier down the road. The labels will be stored into a separate array of integers 0 through 9.\n",
    "\n",
    "A few images might not be readable, we'll just skip them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 30
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 399874,
     "status": "ok",
     "timestamp": 1444485886378,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "h7q0XhG3MJdf",
    "outputId": "92c391bb-86ff-431d-9ada-315568a19e59",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "notMNIST_large/A\n",
      "('Could not read:', 'notMNIST_large/A/Um9tYW5hIEJvbGQucGZi.png', ':', IOError(\"cannot identify image file 'notMNIST_large/A/Um9tYW5hIEJvbGQucGZi.png'\",), \"- it's ok, skipping.\")\n",
      "('Could not read:', 'notMNIST_large/A/RnJlaWdodERpc3BCb29rSXRhbGljLnR0Zg==.png', ':', IOError(\"cannot identify image file 'notMNIST_large/A/RnJlaWdodERpc3BCb29rSXRhbGljLnR0Zg==.png'\",), \"- it's ok, skipping.\")\n",
      "('Could not read:', 'notMNIST_large/A/SG90IE11c3RhcmQgQlROIFBvc3Rlci50dGY=.png', ':', IOError(\"cannot identify image file 'notMNIST_large/A/SG90IE11c3RhcmQgQlROIFBvc3Rlci50dGY=.png'\",), \"- it's ok, skipping.\")\n",
      "notMNIST_large/B\n",
      "('Could not read:', 'notMNIST_large/B/TmlraXNFRi1TZW1pQm9sZEl0YWxpYy5vdGY=.png', ':', IOError(\"cannot identify image file 'notMNIST_large/B/TmlraXNFRi1TZW1pQm9sZEl0YWxpYy5vdGY=.png'\",), \"- it's ok, skipping.\")\n",
      "notMNIST_large/C\n",
      "notMNIST_large/D\n",
      "('Could not read:', 'notMNIST_large/D/VHJhbnNpdCBCb2xkLnR0Zg==.png', ':', IOError(\"cannot identify image file 'notMNIST_large/D/VHJhbnNpdCBCb2xkLnR0Zg==.png'\",), \"- it's ok, skipping.\")\n",
      "notMNIST_large/E\n",
      "notMNIST_large/F\n",
      "notMNIST_large/G\n",
      "notMNIST_large/H\n",
      "notMNIST_large/I\n",
      "notMNIST_large/J\n",
      "('Full dataset tensor:', (529114, 28, 28))\n",
      "('Mean:', -0.081659399)\n",
      "('Standard deviation:', 0.45423427)\n",
      "('Labels:', (529114,))\n",
      "notMNIST_small/A\n",
      "('Could not read:', 'notMNIST_small/A/RGVtb2NyYXRpY2FCb2xkT2xkc3R5bGUgQm9sZC50dGY=.png', ':', IOError(\"cannot identify image file 'notMNIST_small/A/RGVtb2NyYXRpY2FCb2xkT2xkc3R5bGUgQm9sZC50dGY=.png'\",), \"- it's ok, skipping.\")\n",
      "notMNIST_small/B\n",
      "notMNIST_small/C\n",
      "notMNIST_small/D\n",
      "notMNIST_small/E\n",
      "notMNIST_small/F\n",
      "('Could not read:', 'notMNIST_small/F/Q3Jvc3NvdmVyIEJvbGRPYmxpcXVlLnR0Zg==.png', ':', IOError(\"cannot identify image file 'notMNIST_small/F/Q3Jvc3NvdmVyIEJvbGRPYmxpcXVlLnR0Zg==.png'\",), \"- it's ok, skipping.\")\n",
      "notMNIST_small/G\n",
      "notMNIST_small/H\n",
      "notMNIST_small/I\n",
      "notMNIST_small/J\n",
      "('Full dataset tensor:', (18724, 28, 28))\n",
      "('Mean:', -0.074636228)\n",
      "('Standard deviation:', 0.45862204)\n",
      "('Labels:', (18724,))\n"
     ]
    }
   ],
   "source": [
    "image_size = 28  # Pixel width and height.\n",
    "pixel_depth = 255.0  # Number of levels per pixel.\n",
    "\n",
    "def load(data_folders, min_num_images, max_num_images):\n",
    "  dataset = np.ndarray(\n",
    "    shape=(max_num_images, image_size, image_size), dtype=np.float32)\n",
    "  labels = np.ndarray(shape=(max_num_images), dtype=np.int32)\n",
    "  label_index = 0\n",
    "  image_index = 0\n",
    "  for folder in data_folders:\n",
    "    print(folder)\n",
    "    for image in os.listdir(folder):\n",
    "      if image_index >= max_num_images:\n",
    "        raise Exception('More images than expected: %d >= %d' % (\n",
    "          image_index, max_num_images))\n",
    "      image_file = os.path.join(folder, image)\n",
    "      try:\n",
    "        image_data = (ndimage.imread(image_file).astype(float) -\n",
    "                      pixel_depth / 2) / pixel_depth\n",
    "        if image_data.shape != (image_size, image_size):\n",
    "          raise Exception('Unexpected image shape: %s' % str(image_data.shape))\n",
    "        dataset[image_index, :, :] = image_data\n",
    "        labels[image_index] = label_index\n",
    "        image_index += 1\n",
    "      except IOError as e:\n",
    "        print('Could not read:', image_file, ':', e, '- it\\'s ok, skipping.')\n",
    "    label_index += 1\n",
    "  num_images = image_index\n",
    "  dataset = dataset[0:num_images, :, :]\n",
    "  labels = labels[0:num_images]\n",
    "  if num_images < min_num_images:\n",
    "    raise Exception('Many fewer images than expected: %d < %d' % (\n",
    "        num_images, min_num_images))\n",
    "  print('Full dataset tensor:', dataset.shape)\n",
    "  print('Mean:', np.mean(dataset))\n",
    "  print('Standard deviation:', np.std(dataset))\n",
    "  print('Labels:', labels.shape)\n",
    "  return dataset, labels\n",
    "train_dataset, train_labels = load(train_folders, 450000, 550000)\n",
    "test_dataset, test_labels = load(test_folders, 18000, 20000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vUdbskYE2d87"
   },
   "source": [
    "---\n",
    "Problem 2\n",
    "---------\n",
    "\n",
    "Let's verify that the data still looks good. Displaying a sample of the labels and images from the ndarray. Hint: you can use matplotlib.pyplot.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f5059c8b710>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPwAAAD8CAYAAABTq8lnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnVuMZGdxx/81957umemZnWWxrHXMA1GiCMlWJCvSJvI8\nWMgoEoQXI0uRrAhFPBBAPGHzAOvkIQQJhJQHFIKNTIIgKAjHRCLBRAyGh3CJDDZgc5FsyTa7Yxvv\n7szs3Ge/PEzX2eqa+s6lr6f71E866sv09JztPf+u+uqrC4UQ4DhONZgY9gk4jjM4XPCOUyFc8I5T\nIVzwjlMhXPCOUyFc8I5TIToWPBHdS0TPE9GviegjvTwpx3H6A3WyD09EkwB+CeAeAK8A+BGA+0MI\nz4nX+Aa/4wyREALp5zq18HcB+E0I4cUQwiGArwB4Vzcn5zhO/+lU8LcCeEk8frn1nOM4JaZTwbu7\n7jgjSKeCfwXAefH4PE6svOM4JaZTwf8YwFuJ6HYimgHwHgBP9O60HMfpB1Od/FII4YiI/gbAfwOY\nBPCIjNA7jlNOOtqWy/XGvi3nOEOll9tyjuOMIC54x6kQLnjHqRAueMepEC54x6kQLnjHqRAueMep\nEC54x6kQLnjHqRAueMepEC54x6kQLnjHqRAueMepEB2VxzrFIDpVtDT2+JDScuKC7yNVFDoj/+0u\n/vLgLn2fqLLYNUTkn0dJcMH3Ab+4bVz4w8dd+gFRlQs9j/tORO7mDwm38D3GEnZVxA64FS87LnjH\nqRAu+B5juapVcl9DCJX6944avoYfEC6Cm/hnMTzcwvcBv6Dj+GczXNzC94kQggevWrjIy4MLvo/w\nhV5F4bvIy4kLfgD4xe+UBV/DO06FcAvfY6T7zver6NIDcc9GP+8e0OBwwfcQIsLExESSbaaPcSdN\nuHJ/nu/r55z+44LvEVLYExMTbQc/N+5I0VpWPISAGzdunLq1Xu/0Bxd8D5Fin5ycxOTkZHJ/XAWf\nllmovwBu3LiRHESUiN7FPji6EjwRvQhgE8AxgMMQwl29OKlRhQXPYp+amkruT05ODvv0ek4escvb\nGzdu4Pj4GMfHx22vd8EPjm4tfACwFkJ4oxcnM+pIl54Fz8e4CT4tIGeJPYSA4+PjU51wqhLfKAu9\ncOn9f6uFFjvfTk9PY2pqvFZPWdbdCtJxPEM+5jW8Mxh6YeG/TUTHAP4phPDPPTinkUQH7KTYZ2Zm\nxkrwWdY9Fo2XYmcXn78EnMHQ7VV4IYRwiYjOAniSiJ4PIXyvFydWZmJNLljgs7OzmJ2dxdzcXHJ/\nZmZmCGfaX7TwOSCnA3R8yMAlP8duvnfByYdcAslbHQuJeU5dCT6EcKl1+xoRfR3AXQDGVvDWB823\nk5OTmJmZQa1Ww/z8fHLU63XMz89jdnZ2aOc9KI6OjlIPac35S2FychJHR0dDPvPRIG3LV3tK165d\nM9+jY8ET0TyAyRDCFhHVAbwdwMOdvt+oEEuomZqawuzsLObn59FoNLCwsNB2zM/PD/nM+8/BwUFy\n7O/vn7qvxa6/BJx09C6Q3gnqq+ABnAPw9dYfmQLwpRDCt7p4v9KjhS6z6qSFbzQaWFpaQrPZTG4b\njcawT79rsoS5t7eHvb097O7utt3KXQop9qmpKRweHrrgc8LXnNz94YDw9PR0rlyPjgUfQngBwB2d\n/v6oId13KXZ2p9jCS8EvLy/jzJkzWFlZweLiYtv7jDrWv2NnZwfXr19PbmdmZtouRCn2o6MjHB4e\nuoUvCFv46enpJCDMn3Oerd/xCR0PgJjYWfAzMzOJS7+0tISVlRWsrq7iTW96E5rN5rBPv2u0MPXj\nra0tbG9vY2trK9mZ4ItQBulY7JyB6ILPh972nZmZSYLCs7OzLvh+obff+FZa+MXFRaysrODs2bM4\nd+4cVlZWTr3HKJIm+mvXrmFzcxOzs7OJxSEiHB8f4/DwMBH7wcFB8mXAn6OTjRQ8W3feDarVai74\nXmOt4fU3rnbpV1dXce7cOZw9e3bYp981WaW/tVoNs7OzmJqaSkTMYj84OEjEzu4oi35Uv/wGjV7D\ns+BrtRpqtRqmp6cz38MFXxBdHMMXLX/j8rctb8nJiD0zrhc4B+qkO8/Cl4UyVnmskw0HhznXY25u\nru06c8H3GG3RZaSU11FsueQFnydffBS+BGKC5WN/fz8RPQfueE2/tbWVBPT29vawv7+fuPmeXnsa\n63qYmJhIDEutVkO9XsfCwgIWFxexsLCQK7nLBZ8TqzCGXVMWPEdLef0aC0qNgrgtdB27PEIIidi1\n4Le3t7G5uZn8bHd3NxH80dGRC16Qdq3wdcduPAuet35d8D3GEjxvi0jBs4XXGVHjAFe9yVJXvm9Z\n+OvXrycWnvfp9/f3k2Qct/A2VryELTy781Lwy8vLmJuby3xfF3wBYhZeCl7uiWZtO43al4C07Bxx\nPzo6SoTPgt7Z2TEtvMzEOzg4cAufgQ6Myu1fbeGXl5dRq9Uy39MFXwCr1l3uhWoLX2QNPyro5Bne\nUz86OsL+/r7p0rOF59fJ3zk+PvbgnUJbd34sA3YcGF5cXESz2cTKykqu9G0XfE70Gl7vheo1vBb8\nOCDLWmUCDVtrmVqrBb+5udnm/uslgWMLXVr5LAtfr9cz/4YLvgCxWnctelnQME4WXrepYsGz6GNr\neBY9B/f4vXxr7jSx0msWvLTwUvArKysu+F4TW8PHLPy4Buw4gUYG4fb29pL1Om+/sXvPr3FhZ2OV\nwPIxNzeX5HnIMux6vZ7sxWfhgs9JrF+dtPDanR/lLDJLnJw1JwNzbMl3d3dx5coVbG5uYnt7Gzs7\nO2177U42nFgjczzksbS0hMXFRTQaDczPzyeZjfKay8IFX4A8UXpr/V5m0ee1umzd2bLv7Oy0Fcts\nb2/jypUruHbtGra3txPLfnh46Gv0nPAaXS4V5bG0tISFhYVE8NxRSRcqpeGCL0DeoJ3ehy+T4Iu6\n1fL1bOH39/exs7OD7e1tXLt2LSmaYcFvbW25he8A2VeB99r5mJ2dTQTPXZTcwvcZXf9ufRvHXPpR\nWr/GOtJql54Ff+XKlcSdt1x6Gaxz4shceVmTweKWFr5eryeCZwufp1GqCz4naR1pLQsvSz/LZOHT\nyBr+aLn0bNl/97vfJfnyLHh26d3C50MXx3AkngW+uLjYtoZnyy8TvbJwwReA/0NkYCXPGn4USOsz\nz/djLj0LnoN3vC0nLbyTjax15/6IvPXG22+WS+9r+D4RW8PHBD8Kos8SunysXfqtra1E8K+//npb\nnjwfbuHzQ6rVud5rjwXtfA3fB9Kq5XQevdyHHyWxx8peeQ2uU2d5S45TZznjjm9Z7L5+bydWESer\n4XgNzy693JLj9fvc3JzZeyANF3wBilTLjYKF1wFFK+1VHrz9JhNr9vb22ophWOT8Pp5JdxPdCFWn\nzsokLtlEhWveWfCW2PPigi+A1QBj1LblLIhuTi5ht10WuLCQOSinBc+uuyyOYcF7hL4d3d5cNkTV\nXZN0xySeb8DuvOwKnPcac8HnJMuljzXAKHvijYQtO7vlcojE/v5+1MLza1joUvBu4dvR/RDloZtS\nyvZV3NWG02q1hS+F4EflQs9LHpde7olKC1+GzyJNeESUlL1yJF4PlsgSvG6IwRbeOcFqfipvrb6I\nbOFZ8LItNV9rsu9/Fm7hc2Ll0Wftw+cNpAwK60snbetNBudkKq3uTcfegNX2yi18O5bo5a5P2hp+\nYWGhra2a9CZLYeHHjbxBOx2lL4N1jyEDd9rCs+C5Co4Fb1XCHRwcmJF9F3w7VgKXbHOu1/DSpW80\nGqd+p2hg2AVfgLxr+FGJ0mvYFZe17Zw5t7m5merSHx4enhK3rH132qPzutW53H+PufT1et2cGFvE\ni+yr4Hme2jgwOzvblsfMmU4cQNFr+DIL3hKgDtjJfPnNzc2kQIYFL7fkfL89H7JrDXuH8laOGedr\ni68vPvh9+NYaCJJGXwV//vz5fr79QJmZmcHq6irOnj2L1dVVLC8vY2lpKdkXlX3py5p4Iy2w1XnG\ncuVlNp0sjNnd3U0i8x6Yywdn0sWq4ZaXl0/lyuvAXFGBa1zwOZmensbKygqWl5eTW5kIMSoufVo2\nHbvyu7u7ba78tWvXcPXq1TaXnq370dGRW/ac6J500prPz88n48UtwetEHaAz8bvgczI9PY3FxcUk\np5nv83+OnjpTtqBdLF2Wb2V0Xrrz0sLLiL0UvFv4fMh6dx4T1Wg0kkMakXq9jrm5OdNbtESfl0zB\nE9GjAP4cwKshhLe1nlsB8G8Afg/AiwDuCyFc1b87ToKfmppq+8+RhxyiKKOnZduWk2LXE2Ri0Xkp\neN3DzgVfDLl+Z8HLtFleJnJGHVv4WAJX1mOLPBb+CwD+EcAXxXMPAngyhPBJIvpI6/GD+hfHSfCT\nk5OJ6yUDKrJMUSdSlCG1Nq1ARufNy+g8F8ZIl14PktCCH8VmH4NEruH15Jhms4lms5m5huf3sd47\nD5mCDyF8j4huV0+/E8DdrfuPAVjHmAt+YmKibfvNSrjRWyXWumuQxMQuk2N0y+k0Cy+nzchDW3jO\nzXfaibn03Gaal4gxwXcjdKbTNfy5EMJG6/4GgHPWi2677bYO37588Lez1VVU581b2yZlQK/h8whe\nWnhrKeDFMfmRQTtp4dmd5712XSAjg3bd0nXQLoQQiMj8H//c5z6X3L9w4QIuXLjQ7Z8bGlYOtHwc\ni5iWTfByTJSshjs4OEgCcjKllu/v7u5Gt/Wcm+jrQP7/yy05XhLKjjax4pg8y8L19XWsr69nnl+n\ngt8gojeHEC4T0S0AXrVe9NBDD3X49uVEF8PEtknKyo0bN9qmxMhKuL29vaTjLCfWcFKNrnpz0dtY\nJa/yOTlIQg6U0Ek2nbRKW1tbw9raWvL44YcfNl/XqeCfAPAAgH9o3T5uvajXEeq0f3Q/Ljr996y0\nxrS90GF/Aci1tLTu7Lbree5a8LJFVRGRV/ULIOYF8n2ZaKOFzsFfOdSkSCebvOTZlvsyTgJ0q0T0\nEoCPAfgEgK8S0XvR2pazfrdIJ46yo/OgY1a+bEjRW5Vwcv4bD5GQgpcTXpk896uKroCTMR4peEv4\n0rpLwffy+soTpb8/8qN7sn53nAQPnG5NlPYfUaYvARY9W3jOlZf96Dg4F7PwQLy5pfWzKiINgg7s\nysKYNAtvBYTZwIQQur6u+pppV6akk27RkfdYJL5MQtdol54Fz1F4KXhZBaenvjo2VhWctNh51vDS\nO7DyOLoVfV8FP44WXt7GnisrMmjHLr0ujuExUbE1vMa/ANrhvXY9e5C34rSF5/14fmyVvmrD2Y3o\nXfBdMgpCZ3i/Xda6S8HLcc8ySu+lr/nQQTvpyuvqOGv089zcXHRbT19nnYree9qNGbF1tZzrroN2\nvIaXo59lnzrPlc+HtO56v50TbWQvBdlHgb0BST++ZL3jzRgRK33lvHm5/85bcjJSH+tC69Y9H1bq\nrBR5s9lMRkXJya+dROJL6dI7g8Mqe5X3pXXX++8sejkqirPw3MLnx0qdlX3lZXFMN4IvbdDOGSy6\nCk62i9aCtyy87EPvFr44bOF1+avsnyCHQUrBF/kb3dBXwY/bhVL2mIRVFMMHJ9xIl17myl+/fj2Z\nHiOnyHhxTH6khefpr9xxlmvduUCG1/Bycoz+nPtxvbmFL0AvEh96jQ7SsejlBBhZIGO59Gzh9eQY\nd+mLoevdpeBl+6puXfpucMEXpEyitzLfYmWvcnSUdulZ9LpO3stfi8HbcVa9Owte9rHrxKXvFhd8\nB5RJ9Iy27lLsukIuZuF1RZwPkSgGW3i5hpcu/cLCQlumnVt4pys4YCebWbBll2t2uf0mX+Nkk5YY\nIweTWAMhG42GOVp8kG3QXPAdUDbrzshceTkEUpe/yrlw3oQyP1atu7zVWXQsfN0aLU+te7+uMRd8\nQcokdh3ZlZNjuDiGXXYuf9UNLlzw+eFCKd2s1Kp3l2KXopetzGWBjP47/cIFX4AyiZ2Roreq4bgR\nJZe/8uQYbzNdHKsaTt5aFt4a72yVvg6qCMtz6ccAXe8eK3/lajhd7+6Czw9bZFnnLgN10qpb1l6W\nv2qXfhB6cQs/RuhhkCz4q1evnip/dQtfHGnhrYGQadadLbxugDro2QUu+DEhZuHZlefyV9ngwgVf\nDLmG5xRaOaPAKn/VXwBWc8tBesIu+DGCG1xYLr2cDad71rng82FZeN26KsvKy/eybvuNC36EsGrd\ndYPKWL371tZWsk2n57q74PORVu/OSTay1t1qOT1sXPAjhDX9lQ+5/24dMtmGs+84I8+z6fJh1bvz\npNd6vZ4UxwwzVz4LF/yIYM2Fkznvh4eHbZNdpdD5kOWvsjDGBZ8PXe/OgudxUVzvbjW4KAsu+BFC\n58rLQxfGaLHv7u625dV7+Wtx9LhnTpldWlpKCmSsevdhTxCWuOBHCF36ysfx8XGb2GPC11Nf3cIX\nwyp/lcMgs1z6MhRdueBHAD0uSgqerbZubmFZeMszcMHnJ638VU5/TXPphy16F3zJ0eWq0qWX7rll\n1bXofdxzd8TW8Lx+X1hYMOvdez1Mohtc8CUm1uAiq9bditDzuGd+H58kU5yYS88WvtFonEqljQXt\nhiV6F/wIIcWug3S8584173I9z18MTjbWCDG+5cy6WL17vV5PMu94/z3vuOdB4YIvMbHyV51Yw7dy\n+qunzhYnNuaJ78uEGiubzpr8Ouh69yxc8COClSvPo6E4R142uPB69+Lo8lc9+llOkbHErrPqYgMh\nPWjnRJEXR1quvB737B1tiqNz5fUhp75K0cu2VbpGvkxiB4DMec5E9CgRbRDRs+K5i0T0MhE93Tru\n7e9pOoDd0YaF/sYbb3hHmy6Jlb/KnPmYWy8tvFy7y8q4YYsdyCF4AF8AoAUdAHw6hHBn6/iv3p+a\no0lrcHHlyhXvaNMlWvC67FWL3ap3t1pYlUHoTKZLH0L4HhHdbvyoPP+KiqCDdrrBhR4MyXv0Lvh8\nSMHL2e5yq022mLbW8brOvSyWnclj4WN8gIh+SkSPEFGzZ2fkmORpcKG70rqFL4bl0kuh5w3asUtv\n9awbNp0G7T4L4G9b9/8OwKcAvFe/6OLFi8n9tbU1rK2tdfjnqkGRenfee9/Z2Uki9TK91uvdiyOH\nQbJl5712We8uM+nKUu++vr6O9fX1zNdRnkyrlkv/jRDC2/L+jIiCZ3EVQ9a4yxTYEAIODw9x+fJl\nXLp06dTtxsYGNjY2koES1uGJN9nUarWk2o3LXnmARKPRwMrKCs6cOYPV1dXklo8zZ85gZmZm2P+E\nhFYOxym3oiMLT0S3hBAutR6+G8Czaa93stG58rrmXde7W7dy6qsXxhRHF8fUajXU6/W2kc+cL88W\nftCTY7olU/BE9GUAdwNYJaKXAHwcwBoR3YGTaP0LAN7X17OsCGn17jJX3rrlPXfOsXfRF0fXu8vi\nGK555xTaWq3Wlj47KuSJ0t9vPP1oH86lssj1uhwEKY+YyOVjFrmXv3aGbGHFe+963DO7/By0GzsL\n7/QXXetuVcNZ7rx13xr17ILPj1X+Wq/XT7WvYpdeW3j+nMssfhf8EMlT/qrnuludbVj0VrDPRz7n\nR457np2dbetZ12w2k0428/PzbeWvZap3z8IFXyLSBJ9m4flWb+V5zXsxrKCdXMM3Go1TCTfD3o4r\nigu+JGix63p3q9advwx4ne9kEytkkdY91nO+Xq+3TZthsZfVmlu44EsEZ9Kx2DmxhpNr9CDIg4MD\nT6wpgEx1tVJgZfMKeehad13v7kE7Jxe6wYUUPGfSyXp3FvzOzk6y7+6ps/mRIrcOKe7YrVXvrsVe\nZvG74IcMi17myrPgdUWctPB7e3tJ6qyv0fPD4raaXMQsu8yVT6t3L7PQGRd8CeALJWbhdXML6dK7\nhc+Pnv4qhcvbcVrg+jk9271sxTFZuOBLhAzYWYLf3NxsE7wXxxRHWnjZzYa34izrLp8f5mz3XuCC\nLwmWS68Fz+t5Xf7qLn0+5Bpej3yW0XfL0o9KvXsWLvgSwT3rpOB5Dc/Wnbfm3MIXRwtei95y5/Vj\nfh95q++XGRf8gLGy64D2nvO8/85BO47S83O63t0tfD7kfHfZwooPqzml3oYbdVzwA0SmuuopMDKr\nzpokI4Xu4547QwpeCpwTbLgwRja34OKYccEFPyCk0K18dzk2SqbTylFRspmFdOdd8PmwOtpwgYzM\npuMvArbsLninI/QQR3nI3Hld+sr77lLsbuGLIxtUck48d7fhIhk525234UYtEp+GC34AWBNgZRmr\n7Fenq+OklZf18VL0Lvh8SAvPLr1sciG72VgWvsxVcHlxwfcZ3ZhSil0e1iRYaeF9vnv3SAuvXXru\nX5e1hh910bvg+0isC60WvR79rC28nu+uZ7y74POh6911G6u0oN2o1Ltn4YIfIDJwZ7Wy4jV6LErv\n8927Q6/hpUuvLbwH7ZxCWEKMta86ODhIsuc4QCf7y3ub6c7QVthy6bkzLQfuuJuNnu+uGVUr74Lv\nE7r0VafO6q41nFFnZdK5Fc+Plf3Gt9aQSCl6OUZKC36USmDTcMH3kax6d16by1nvujjGq+HyowUu\n89w5pTYmeJ4oIwUvS2CtvzOKuOD7jBR9rN6drbucC+cWvjO02OWRNgZaDpfg5pSjWvOehgt+ALDo\nZYINt7Biy87VcFb5qwu+GLE2VjGXXs6Lk4Uy7NKPi9gBF/xA0fXucgLs5uamu/Q9INavTpfExtbw\n/PMyz3jvBhf8ALGaVMryVx204yaVbuGLYVn4LMHz+l32q+PX+7acU5hYzzoufY1ZeK93z4+O0OuG\nlXkEH2twOS644PuEVfduufSyBbXVzcYtfDFY2FajylgbanlYwb5R62qThgu+h8RSaQG0JdywS8+C\n50i9jNC74Itj9aqTPeu4Ei5tr31chc644HtMLP1VV8XJwhi28nJmnOxo4y59PuReO4tZRtw5MKdz\n5bXgxxkXfB/QnW2kO68FLy08P8di5wi9W/h86K03PQdOW3i5164t+riKPzUaQUTnieg7RPRzIvoZ\nEX2w9fwKET1JRL8iom8RUXMwp1tutMitJhcySs9Zdu7S9waro40sgbXSZy0Lr4U/TmSFHw8BfDiE\n8EcA/gTA+4noDwE8CODJEMLvA/if1mMH7U0urFJY2ZVWWngO2Ok9eBd8fuQaXmbScUWcTp+NWXgr\nRXdcSHXpQwiXAVxu3d8moucA3ArgnQDubr3sMQDrqLjodcDOEr0VtJOBO9nYwltYFUev4XXNe54C\nmXESt0XuNTwR3Q7gTgA/AHAuhLDR+tEGgHM9P7MRxFq7SwuftYa3mlvw4WRjreHZwrNLr8tftYXn\n9xlXcgmeiBoAvgbgQyGELdX9IxCRaYIuXryY3F9bW8Pa2lo351p6ZDReH7JtVeyw2le7dc+PtYbn\ntFnZoDLW0WaUhb6+vo719fXM11HWBUVE0wD+E8A3QwifaT33PIC1EMJlIroFwHdCCH+gfi9U6WKV\nraZlgwu+v7Ozg9/+9rfJ8corr7Tdf+2115L3ifWvd9JZXFxEs9nE8vIyms1m27G8vIylpSU0m00s\nLS213efbqanx2bRqFWyd+gbLitITgEcA/ILF3uIJAA+07j8A4PFeneioIq27dNnl5JhYYE72p3er\n3jnSpZdNKrmFlexKK4N241Qck0XWV9oFAH8J4Bkierr13EMAPgHgq0T0XgAvArivb2c4QujyV9lm\nmvPkZeqsHPesLbmVteekY/Wdj3Wl5XX8uBXHZJEVpf8+4l7APb0/ndHFqneXUXie7a4bXFjJNe7G\nd0ZsDS+bVGoLLzvajGqfuiKMz6KlBOjyV3bpuQR2e3s7+RKQ1XDWXruLvTixNtT1eh2Li4untuWs\nrrTjLnoXfI/Q6bOyOEb2q8tj4fX7OvnQiTd6Dc8jpPgYx77zWbjge4gO2uny17Q1PODi7hY9Skqv\n4Wu1WuLu8xjoWBvqccUF3yOsNbyO0lsW3rPpipHWhlqWwsrZ71whJ4dLyBZWljUfVyvvgu8hsaAd\ni5672XiDi86INafkQzeg5IO/BNiis1VPS7gZR7EDLvieoWveZeqs1XPeBV8c2bJKt6+amJgwO9hI\n0et+dTHRj6vYARd8T9EFMjJXXjaolDXvLvhiyA60uo2VZd21pZe/a1n3cRY74ILvCbJQRte7Swsv\nt+RkwM6LY/KhO9DKzrK6Z50ldha89BCqUjTDuOC7xBolpV16XsNzsM7X8J1h9ZjXgTrd1kqL3upX\nXwWhMy74LtDZcbLJhSV4OfvdXfrO4K03WQbLR5Z1Z8Hz+4xzs8oYLvgeobflDg4OTrn0ViWdCz4/\nlksvrXta0I6FL9/Luj/uuOALYBW4pLWh1t1pZScb72hTHO3O6/123clGj4uqUoJNDBd8AazWVXwr\nm0+yu64tOQvcy2E7QwteCl0m1+huNi70m7jgcyLbTcu2VXxfdq6RgudOtRyR18J38qNbWMmKuLRx\nz1Vao2fhgi+AbmElLbe27pbodZ86t+7FkE0qrRnv0sLHCmOqjgu+ALohJa/Zj46O2qy7JXa5Xrfc\neicb6dJbveet0ld36dtxwedErt9lgE5m1WWJXnezdbEXIxawky59rCMtML4FMUVwwRfAErzcW8+y\n8FZjShd8ftJc+tgaXkfnqy56F3wBYoLXAbtYpJ7fQ7+nkw/t0nOEXrv0MQvPVFn0LvicxMTOTSq5\nKEbWusuONh6R7x4rSm+59HpuXFXFbeGCL4CVKy+z6axhkJ5Y0zv0PrwcJ6XHQGuXvqqZdRoPX+ZE\n17vL1Fnd4ELXu7t17w16Da/HSXniTTZu4QsQs/DW9Fcvjuk9snBGuvS8hpez4K01fJUtO+NffQXQ\nFj427nlvby9XR1qnGFaUnoN2Wam1LvYTXPA5sRpcyBZWukGl17v3ntgaPs2ld7G344IvgNWzzmpS\n6YLvD7F9eLbwMZfe1/A38TV8Tqx6d3fpB0vWGt6K0nvxTDsu+JxYUXq5LRcbMuEWvjixttGy6QW7\n9FL0UuwZ7nYfAAAGoUlEQVRyyIRb+Ju44A0sgUoLb63j9frdLXxxdDRdN5iUYpf95q121LJ9tXMT\nF7wilvqqZ8fpbja6BbV3tOkMq9ccH5bYdT873eXG3fl2Ur/+iOg8EX2HiH5ORD8jog+2nr9IRC8T\n0dOt497BnG5/0U0pdZGLdun1SGhZMec96zpDdpPVveu0hbeaV1rDJlzwN8my8IcAPhxC+AkRNQD8\nHxE9CSAA+HQI4dN9P8MhYPWuixXNSJdeF864hS+GNUpKHrL7rOXOT09Ptw2a0KOonAzBhxAuA7jc\nur9NRM8BuLX147H+BLNEr1363d1ds0LOBV8cKXIpXmnldStqFr41isrFfpPcEQ0iuh3AnQD+t/XU\nB4jop0T0CBE1+3BuQ0G79XybtoaPBe3cwhfHsu56yoy79J2TS/Atd/7fAXwohLAN4LMA3gLgDgCX\nAHyqb2c4BKz1e541vE668TV8MSyXXgtdBu2k8HWk3oN2NplReiKaBvA1AP8aQngcAEIIr4qffx7A\nN6zfvXjxYnJ/bW0Na2tr3Z1tH7HaT8lb2ZxS3+dDN6j0NlbF0ME6vWaPDZeQVr2qrK+vY319PfN1\nqYKnk6/GRwD8IoTwGfH8LSGES62H7wbwrPX7UvCjgGxSye4432dLLhNr2G33fvO9wSp/lYfOpHOX\n/SbaoD788MPm67Is/AUAfwngGSJ6uvXcRwHcT0R34CRa/wKA93V5vqVArtP1IV127banWXUXfX6s\nrrScSScr47JaWDlxsqL034e9zv9mf05neFhReBlx56CcnO2uLby27i72YkgLzy68LIiJ5cp7Nl1+\nPNNOIINyekacdOl1+qzsauOi75y04hjdr87LXzvDBd/C2nrjbbf9/f1Ml95qQS3f28kmq/xVD5rw\nrbfiuOAFscQa3mvP49IzLvLiWGt42eDCLXz3uOBbpKXPsnWPWXirUaULvji6Ii5PV1oP2hXDBS+Q\nLn3MwuvCmFhHWr8Ii6PbV1nTYb0rbXe44AW8vaZr3q1gHYtduqFOd8QaW6R1pXWXvhgu+BZZ+fLs\nzltFMS743qBTZbU7711pu8cFL5CZdRyUk259rDklrz2d7rDGSKW59Byld7HnxwUvsBpVxiw8Cz7N\npffAXTEssWvBZ02HddJxwbfIanKRNjNOC96F3hlFLbxH6YvjghdYa3jZ4CLm0rPgXejdkbaGj+3D\nu+CL4YJvwRbeqnnXY6Dllpws6XS6Q2/JxYJ27tJ3zsA+qTy1usPku9/9bpvoeS0vLX6sE63Vfy2t\n+2onx+HhYc/fs2znl9W/Tne00Z91GmW//gZ1fi74Fk899dSwTyGVg4ODYZ9CKmU/v7Jff2MneMdx\nho8L3nEqBPUrskxEHrJ2nCESQjgV2Oib4B3HKR/u0jtOhXDBO06FGIjgieheInqeiH5NRB8ZxN8s\nAhG9SETP0MlgzB+W4HweJaINInpWPLdCRE8S0a+I6Fs0xGk/kfO7SCUYMErxAail+PxSzm8gn1/f\n1/BENAnglwDuAfAKgB8BuD+E8Fxf/3ABiOgFAH8cQnhj2OcCAET0ZwC2AXwxhPC21nOfBPB6COGT\nrS/N5RDCgyU6v48D2Br2gFEiejOAN8sBqAD+AsBfoQSfX8r53YcBfH6DsPB3AfhNCOHFEMIhgK8A\neNcA/m5RSpOQHUL4HoAr6ul3Anisdf8xnFwkQyFyfkAJPsMQwuUQwk9a97cB8ADUUnx+KecHDODz\nG4TgbwXwknj8Mm7+A8tCAPBtIvoxEf31sE8mwrkQwkbr/gaAc8M8mQilGjBKNweg/gAl/PxoCANa\nByH4Udj3uxBCuBPAOwC8v+WylpZwsg4r2+daqgGjLXf5azgZgLolf1aGz4+GNKB1EIJ/BcB58fg8\nTqx8aeA5eSGE1wB8HSfLkLKx0Vr/gYhuAfBqxusHSgjh1dACwOcxxM+Qbg5A/RcegIoSfX4UGdA6\niM9vEIL/MYC3EtHtRDQD4D0AnhjA380FEc0T0ULrfh3A2xEZjjlkngDwQOv+AwAeT3ntwGmJiIkO\nGB3AeZgDUFGSzy92foP6/AaSaUdE7wDwGQCTAB4JIfx93/9oTojoLTix6sBJf4AvDfv8iOjLAO4G\nsIqT9ebHAPwHgK8CuA3AiwDuCyFcLcn5fRzAGk7c0WTAqFgzD/Lc/hTAUwCewU23/SEAP0QJPr/I\n+X0UwP0YwOfnqbWOUyE8085xKoQL3nEqhAvecSqEC95xKoQL3nEqhAvecSqEC95xKoQL3nEqxP8D\n3N5O+dR77XYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5059da3c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "plt.imshow(load[\"train_dataset\"][5], cmap='binary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GPTCnjIcyuKN"
   },
   "source": [
    "Next, we'll randomize the data. It's important to have the labels well shuffled for the training and test distributions to match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": true,
    "id": "6WZ2l2tN2zOL"
   },
   "outputs": [],
   "source": [
    "np.random.seed(133)\n",
    "def randomize(dataset, labels):\n",
    "  permutation = np.random.permutation(labels.shape[0])\n",
    "  shuffled_dataset = dataset[permutation,:,:]\n",
    "  shuffled_labels = labels[permutation]\n",
    "  return shuffled_dataset, shuffled_labels\n",
    "train_dataset, train_labels = randomize(train_dataset, train_labels)\n",
    "test_dataset, test_labels = randomize(test_dataset, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "puDUTe6t6USl"
   },
   "source": [
    "---\n",
    "Problem 3\n",
    "---------\n",
    "Convince yourself that the data is still good after shuffling!\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "plt.imshow(load[\"train_dataset\"][5], cmap='binary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cYznx5jUwzoO"
   },
   "source": [
    "---\n",
    "Problem 4\n",
    "---------\n",
    "Another check: we expect the data to be balanced across classes. Verify that.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPwAAAD8CAYAAABTq8lnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvV2MbMt1HvZVz093z9+55/LQFCEwIR8SQA8UKASmIJHU\nPUAIQ0YAWX5RwMAwYciBH2LGECxAkh/MQ+shtgQKggPBiCPKkBxDjhBBjPwgR3SQcyI+0DIN/cWm\nbBMQBUriPfeee87Mmeme7p6eqTycWft+vXqtqto9/bN7pj9gY9fevXvv2rXrq/VTq6pCjBEbbLDB\n3UBr1RnYYIMNlocN4TfY4A5hQ/gNNrhD2BB+gw3uEDaE32CDO4QN4TfY4A5hZsKHEL4/hPCHIYT/\nFEL4sXlmaoMNNlgMwiz98CGELQD/AcAnAfwpgH8D4FMxxq/RNZsO/g02WCFijEGf257xXh8F8PUY\n4zcAIITwzwH8JQBf44u4MXn06BE++9nPmr/FGHFxcYHz83Ocn5+j3+9PpS8uLhBjdLerq6tq08dX\nV1cYj8cYDocYDAbm/qtf/So++tGPYmdnx9x2d3fR7XbR7XbR6XSqtGy7u7sI4VX5WvvLy0v0er2J\n7ezsbOJ4MBjg/Py82nP6j//4j/GRj3wE733ve/HgwQO8973vrbYHDx7g9ddfx3g8xuXlpbttb29j\ne3sbW1tbE3t9zktfXl5W5Slp2f/0T/80PvOZz+Di4gLj8RgXFxdTm1c2st/d3cXu7i7a7ba5l7qi\nv3uqXsj2+c9/Hj/6oz9avZO1hRCS+auLnDDlez569AiPHj2q/YySezNmVem/HcA36fhPrs9tsMEG\nDcasEn5mdd1rgcfjMUajEUajUSV5WcqPRqOp//D9LMnOxyLh9SbPvLy8rJ7B9xDpeHV1NfUOfE2J\nhO/3++j1euj3++j3+xgMBtWm8yOSUqS2vAOXkZRNr9fD7u5uUrqLhNfSPSXZ9XFKwo9GI5ydnU1J\neE57ZSN71gZ2d3er/Wg0ciV86Sb5S0n4VqtVSfkQwsRxq9Wayu+s0l+ulzo8i/YwK2Yl/J8C+AAd\nfwCvpPwEWEW5d+/elJrNFWg4HE6ouWdnZ9XW6/UwHA6nSO6p9taeyWJtR0dHOD09rVRc2XZ2dqp9\nu91Gu91Gp9Op0nK8s7Mz8eF0Zbi6uqoaMMtsYdJbW6fTwfn5eUXura2tqizEHOLGiYkuxykyl2zW\nd5PjD33oQ/izP/uziuDSWHHaKxvZi0ovJpQ+zn371Cb5Y3LXefft7e2qAWi1WmbaI66uFzHGiXeP\nMeLhw4dJwuXw+PFjPH78OHvdrE67bbxy2v3XAP4MwG/DcNrxvXUlZHtzPB5jMBjg9PS02l6+fDmR\ntgjP6ZRdJxVTJKfeRqMRrq6ukhJPV0KxK7lSXr/3xF7SV1dXExJdS/fBYJBskK6urnDv3j0cHR3h\n3r17E9vR0REODg5MycvkzFVqJoEmRKvVmmqwLT9JarPKh/faZ6KPve9tbda1OZLrBp43EQDWPeSc\nRXpdD7RmoM9ZYB6VagPXDcl8nHYxxnEI4W8C+L8AbAH4ApPdg1RASwL0+32cnZ3h9PQUJycnU9tg\nMJj4kHqf266uriaeq6WPRQj+uFYl4Eq5vb1dFTQXOpVZpbLzntOes+vi4qLSEKRiidYyHA7R7/ex\nv78/RXAtiT0i6zTvOe05RFmLkoZcpy8vL6fKRe+5LC3C5b557nyqQdva2so6DZn47PDc3t6u7p96\nvxTBLSJbwvimZsCsKj1ijL8B4DdqXF9VQqnYInFHo1FF+JcvX+Lk5AQvXrzA8fFxtT8/P6/u4+1z\nDYDWKrTaqyu6rvD8wbXKLyo2YJNe7EitWfAxN0ZMmvF4jBhjVQbskxA1v9PpZCWwR2r9zlYZtFqt\nCbJbaa9smfApMugy1enUd7a+uz6XkvCtVgvtdrvqheFNzmmtY2dnp3p/AFOqvUd0aRh0/dACIsen\nWUg/M+HrQgpdKgA7n0RK9Xq9SsIfHx/j+fPn1dbv96v78D11Wn9s/WxL+l1eXlYVwrLRZJ+zeRnW\nx8ipvNr+5rS8i5wbjUYYDAbo9XqVL8GTvJrwOWJ7m+cfkX3KYchOT4/0ujHVUtT6viXkl+OchO90\nOtjb25vapGHWZhyTPYRQaV9yrKW51CVJS75mldaz/HdphAferawi4cV2HQwGUyr9ixcv8Pz5czx7\n9gzPnj0zCS/INQKSTlVWKbzUliOJhm6xLTJrUnvSk/8/Go1Mlde6nt8vR+hUY8eE12XJz/J8CCUS\nXvtNdAOrv2uO/DrNDZ1l1uzt7eHg4AD7+/vVfjgc4uLiojJF2+129U5MdtGAUvVH8mDlLyfdb9Iw\nMJYq4S2VXkgvnnlR6UXCP3v2DG+//TZ6vd7U/Uqfax1be8/+9lptJoX+j5cXiyiWNOLrBaIZWbY3\nVyTv/prAKZJbaY/smvSW/6BEwuecibocve/opXMSfn9/H4eHhzg8PKycqbpbVPw98j6s/Vl1JOXB\n11phqt7IXt+nbkOwMgnPZJcuKumfFvVeR6StGl5F1WnrP0C+oubQarVweXlpViZx5KVIaUnuFMl1\nxdX3TxFeHzNBvL02NfSey7F0bxFe7yUt8RieliJaqXa8yl68+F6DqjWYGGNFektDtHBTSb8SCS9O\nKg6w4b7o0Wg00ZpaqPPSpdpAXXDh8zM89eym+cipfpqA3v81Ma6urqqKKWm9WQTXz7Skfu6dOU98\nnxDCxLe3yjF1LtWg6ryLf0P8IuyEjfFdv1O/35+IwdDxGDs7O1PdfHzMvTs6voDt/0ViZSq9FVHH\nhBe7iW0lwCZ6zv5ZZEF6albuPyXXpf6bumeKdJyWngk+BmzThUmpn8PHKRNF51d7qDXprefo+5WQ\n3kqLxGf/xNbWFgaDwYRDTVT50Wg05bTTcRlWPz6f44ZCNtaeloGlq/TcBz4ajSZUeSG8dFFZhGdo\nlTonAeeFkudYjcxN8lPSqFiqbC4PWqLLsyy/hUXiXEOTeucU6bnxSRFeH+caAOs5rHpznIOo8SL5\nuS8+NcjKixiUXgDRXMVRKf34y8BKuuW0wy4n4QWeI02fk8Kz1O15vk9KcyiV8oK6JopWzS1V3SKK\nVyb8m0f8nAStS3IvT0IGkcL8bCvvVl6s31goCKEt34XWRAeDwYTK7o00FEmvw64lLWSXPPD/biXh\ncyr9+fl51T+fkvAp55lVgXSFmec75TBvjUOT27q3R0jvfyXOyBTpclLfewfrXvo6ubb0+tQ51lYs\nXwULJSa7SG/dK6LTMoSag3V4L4OzpIGR8RkchbhoNKIffhaV3pJCulLr9E0xC2EX2dB4DVmqMfCO\n9b10ec3yLKvhsd7Buj7VCHn3zOVN7uPdWwsk7XjTPRj6WKS4bN1ud+J4PB5Pkb3b7U4EVi0aK/PS\nl6j0ov4wtPRJ2fGeVJsVpZW+5D7zQB1Jp8+XqMl8bcl1pfkouabE5Jn1u3rlL12eXoxCrhw6nQ72\n9/cnNgngGQwGlWNQHHjdbrcyXW8d4YF3Sa/jrnUsOUv2nE0o0PbZsnATqX8T4t/0HedN3HmhbiO2\niGeknJZyHddLSUugjjcmotVqVXa+SH8WctI7YJlUnNYmUR0slfAMT1LXfZGUw4r388AyK34KTcnH\nXYHlnPSu0yYre/13dnamBubwAJ3Ly8tkaLPA02RLsDLCC3Ked33sqewlXuR5Yx5Sdl7q/QbzhfVt\ncuacaLDiA2CiXl1dJQnf6XRweXmZHJiVIn0plk54z2mSIrmFHNn1tU1FnY/W5Pe4rSgRJnwtS3gm\n+3g8xvb29oRE15OixhinhgXLc3hwjnba1iH9yiU8wyJ+HZR6iOeB22gq3GV4TsFSoSLnhdza839x\ncVERnonODUCMsQrY0ePspRtR8jqrH2jlNnzKdvdexvsQ8puVnhc25Lx9KCFNaZekePrlWKS9dO9p\nsrOEB16NiGy32xOEtqLx1s5pB5TZ7AzPruIC5vP6ug020EhJdknzdSn/EEt4kex6whFNdia8BASx\nZBey63EPuXfwsBLCp7od6tizFtFLjjfYAJh9xGXOSy/qN0/cKZLaIrtsPIiGyS799J4a33gvvdWf\nqbcNNlglPBKl6mbOxudAM5mHkOfKExOAJ9lgku/u7rpTlJWSfumEF0Jb86/pKZl0AeYCEuRaTx1b\nNpracC2jPJr67owcSTx13rrGgiWJpctOSM/rGch5iczjoBwZnqsn9pStVDte+mg53jjiTk+JlJL2\nHtn5OKXyLwspX8Qi798EzBqOu2yUSMa6+U9998vLy2q24Z2dncoul/MW0SUSVdYw5E3yXzpd1kok\nvDcNEofUpsbBCyynn9UqL5MYJZWjJKCj9L83xbzv5zXQ605673856LrJEl5IKhOTMuF5ERKe1pyD\ndJjspeW79Fh6S6WvK+EBu1vPU+eXWdlmIfOyJfUin+dpVetAeqC8bHL10krrKDzgXbJbkl2vMain\nfQshVB78UqzMaZey41MOvFQorqfOr1LCN6mie+Uwr/LR/cTet2tKeVjI9XWX5t2qpyLhZdk0IbvY\n8+fn5xOEtwbiaMkui4iujYTXNjyr86WtqOW0W7WEtyrOovIxK2EXZR7kyr5JjWAK88ij1kCF5BJ9\nx5NmygQweuVdPYJUyM4TaDSO8FZXXIr0uW46TXQdebcKCZ8j+byJPy+iz1vCr7LBbQJSmqdedos3\nWRZdq/E8zbfMgS/DbOuOp18o4XnqHr1WmrVgoqW65Po910FN1LiLJLir0KRnZ7TuepYRdXo2XO2x\n9/rqS7BQwg+HwyrN68jxWujsrNATWHqk9zzxfJ2XXha8Z84zL3UdTd7/5l0+qy77JqGOUGJprxeu\n4Gmv9dx6jQmtHY1GVVoIzZteJ13bL3qJIqD5IbXLIPqs9/Z6EOZp8pSQ/S42AnW+EUfS6QAbb469\nUixNwus10Vmy675HVu09tV7b66waWdeuCk2q3B7BF5HHDdnTSPml9Lp3WsLrJa3qkP5GhA8hfAPA\nSwCXAC5ijB/l31nCM9k9Kc/2vLXMsMBS4a1rVoW6z55VQs+KXNfTvO5dcv424iaOYiY9S3km+yol\nfATwMMb43PpRS3hPlZf+R+6GsJwSnve9qcRPYZZ8pWISlpmPJj5jXVBix3sqvV67zlumPIV5qPRu\nrctJeE187p5jlZ6xbp75JtjuGsvoplyHb7No1JX0OacdS/hZ1HlgPhL+X4UQLgH8LzHG/5V/9CS8\nRXaZt9uLrZ946KYy3Qib8lsu6hDfk/Dahp+V9Dcl/MdijN8KIbwXwJdCCH8YY/wt+fHnfu7nqgu/\n4zu+A+9///snvPBWsA2/NPDu5H260Lw0Y1GSrMROtZyIdYiWujb3XrOUx6y/aXh9zKny2DRAk9AB\nOdaa80zyEAKePHmCL3/5y9l734jwMcZvXe/fDiH8GoCPAqgI/+lPf7q69uzsDG+//faEmq6dEtf3\nMvdWQVgvntrfFLl+fm64rHTpfbxnAumxBJIuKYdZrsmltWbmjZHQ0ZR1g0fuInJ+qjfeeAOf/OQn\nq3Of+9znzGtnJnwIYQ/AVozxNISwD+AvAJh4CtvwLNVZkvMKmnTvqTS3dnrvtXpWo+AhV+G8hkhX\nXMsk4XiClNQrkX4eoXPv7v2eOs6l9Z5XWrH2XtnkelzuGjwT1uuiroObSPj3Afi16w++DeCfxRh/\nky/QkXYXFxdTccEyCECm9tWqDB+z3aKn+rGutxqCHFLqukVylu7sdLS23D1yKjCQJnOu/PTxLJvO\nAx9b4dI8CITLIoRQhV5vyG4jZwrNgpkJH2P8IwAfSV1TR8LL7B+pjbsj9MoclsTX+xrvZp5LbTLW\n2duY8LlNP0+Oc6RNvbu3T6XrNhLSEyMBVJwWgvOc7fJedcZz30V4pJ+F+EsLreU136XySoUSlV4T\n2SI2Bx5IWkceWdtNCM8FnrLTU9JN4gos27W0AcgRtqQMvGOvUbCe5zUGOnKSo8IAmGSv+23uClJ1\n8CZYGuH1uF5gUqWPMU4Q2NvrpXh0N4VW+0sDFLwC5rQer897XgabRzZxWtv6JaTX85SnCKnf1zOB\n6my5RoHTsuy3zMLKZBdyc5lKucl9Nqp9WorPQ71f6mg5lnLApEoPwCW0pHmv055mwBU/B68gPccc\nH8skhKzKSszBzs4ORqNR8v8p4nMDmZPkuTLwGoQ6jYPX6PT7/YrsXObyDkJqJvtGwueR8unUxUIJ\nzzNpCjl3d3fR6XSm7HlZbE9vWpJb5LekvEf4kgLzbHhNUk34nZ2diuDD4XDqXTyi61GBlhMPQJaI\nKbKXEL2U9B7hpfG14r63trYwGAyqtJBffB8bTEM7RfmcvqYUCyX84eFhlR6NRhNdcO12G51OB3t7\ne9jf369ae0sieXa8rlCeWp/z0jPBrWulcrZarQkJxVKYVVNLzc4Rnv0anp1cQniPvHXVe8/eT+1Z\n4+Jv3O12sbe3h36/j36/j16vVwkDIXyr1XIHSt0GlKrhVk+Iblh1L0kdLJTwBwcHVXo8Hk9IeKkE\nYvdZ3XK6wteRTBbpNdi0EOiWlAs25UkXldWypzXhLbVeGhSvAZuHPZ4icimpU558TXT9nc/OzrC7\nu1tJeCH7cDicqsD8bdaZ9NKQe79ZSDlGdX2uS/ylSfjxeFxVhm63OzUeXtS6VEuYqmw5CagrjnwI\nS7pbm5UnTluEZ7JzDIKnzud8F3VJPcs+1eB6m0f4breL8/Pz6lsL2aW8hOx65RQmifWd1gEe0XPS\n3SvftZDwTHhxanU6nQlPtji3xuOxq+paoZeW2m1Jq5RKnyK9JdFS4G4zbX6UEl6vKqK3UrLnCFtC\nYi6z0utEe9NTmElaiM1kl+WTWq3W1DdeN5J7SEl5+d2CJ9lvQvqlEl6cdd7klTyghkMyU7PZsort\nqZ2tlu+hT5HdMgk0EWTvqeQsrXOE73Q6E+owp0U6WiRnp2Rdwurf9XvpvfU/SUvj3e12zR4LTXaZ\nj91aG+22qPSMOna810Br0sv1pViaDR9jnCCwFW/NgSpe4IrXreU5vXjztAUgPUJJbM7cvbUaL0T3\n8q8JL/auLCXM6U6nk/VVyHvwO3l77zedLrmX7Dn2QMcgSEyGJjvb9NZz15ns2jQpgf5GnrY6C9mB\nJUp4rytKNlmCxwrL1DPZWhvb0J5EknMpaMJbobtWawugula0le3t7cpZmSK8ePj39/fdrdvtThHc\nInzu3bzj0orplWUIYaqR1mleSLHX61WLI4pK7zXG64Sc+i7XpJASPjkzNYeFEr7dbldpL8pM0hcX\nF5WklC464F3nDhNeawccvZeSXl6XGpsEuWAV3eqyycD5stKpBk/eXe4n3Zfi/Nrf3zfz5BG+rkSx\n/peyLa1jqxHiyskReLpM5T7rTngNqwEoKT9tElpRpbr8SrDUpaa0VOO9rLEl/bTSZyvbcDjMjkQD\n0uO7cz6AnMqck/BWvvTCGp5J0mq1Jt5DQo6F9Lxaiew9gqScXymCl/4/5YG2tDXZ+v3+xIKJ1hiD\n2whdXl4dteIYuMdDfDk6dLkO6Ze+PryQXBNiMBig3+/j7OwMZ2dnOD09nUgL4a3GIkV4TntEtyLZ\nLFXKs99FurK2odNWrwPvt7a2qoqhyS7aDb+P9R78LqXHukH0ruUeDY/0op2JRiarooq3XmIuhPAl\nTtl1g0e8Eh+TJrwmvThy9aIUjZXwTHg9fFRsu7OzM7x8+RInJycT+/Pz86RKbEkxT5ppsmsvv6e+\nyz2tDcCU9pLqYuRNCC95YLJLlyVH8okJYHVlWe/L6boSvoT0smd/zMXFxcTIOSF7TsLnGrB1hxYU\nvNfz1+kgJiG8+D20f6kEK5Xw7NAZDoeVOv/y5UscHx/j+PgYL168wIsXLyrCW34AXUk8G8kjPF9n\nqesewbXjxPNR5LoUY3x3pKCsCCphx+ywZMLPQnad1mWV0xRS6rxcoyU8O+ksld6S7tbzm446UtbT\nHlMqPXfPapW+8RJepAATXqSASPjj42M8f/4c77zzDt555x30ej2TKCnPrmdn8p7TpeROOQV15fUq\ns07LBxTJvre3V63GIxJePNlWI6LfyXvfeUh4nebfLQkvfhhPwuvJQVJ5aCLqkp2FCvuJRMLrxSQl\nOlWku1bpGynhgUmVV8aPsyQQCX96eoqTkxO88847ePvtt/H2229PEB6oN2Sw1Lkl1+q95Q/Qab5f\niuD6OtnLRxSyHxwcVOQQSSgVxCK7lw/9nrNIeG4sdMPBxGdzTRx10pCLhBcbXs+P4DnubgPZdZlZ\nZBePfE7CW976Rkp4rdKzhGfCawn/7NkzvPXWWzg9PZ2637zyJfAKLleg2mSw7m0dM4Ts+/v7ODg4\nqMihJbxUDo/sGjeR8B7p+Zw+ZpWenXUs4XnhUE/CrwNy9cITKJaEt+x3y4a3Ro82VsILLLWcpb5I\nBlEHe70ezs7OVpHVpWF3d9clQ4kPQMMjTq6ByxGOn6f/Lw03k1y+3cuXL3F2dlbZ8WyuSAzFbSK7\nXKPLW4hu9a+Lyi57ttf1xCKzBuAsnfCWw0KPbfe85HUqdp1C8LCoCmjlLeUr4POl7+WZMYwSU0iu\nk/ul/Ci9Xq8i+Onp6YTz9fj4GKenpzg9PUWv15si/LqT3dPytBmkbXUms4RSiyqvA5W8EG/ruR6W\nRnidyRzZvdhhgWVX8rOs59etVPNoNGZ9nudD0NfnGgHtWZ+FWNop6cVCaLKfnJzg5OSk6m2R33u9\n3oQtLxJ+HWH1CKWEkNR5ttNlL4Tn7jcrwKaxobUaFtm9LaW2sMSxnuEd13GwLAspJ6En4Wf92HKf\n0nf0vPBWHMV4PJ4gPJNeCC9qvuWfWBekHLgpISRg55w1WYiOqLMcdF4dKfmuK1Hpc6Rn7yW3aIAd\n/OE9x0oLSv63TJVeny+R8CXPyZWTQDvgvDR3u/EmobMSHWlJeI66kzHybMOvE7xvBUwLJP4OWsIz\n0VMSXhNe58HqJrWwUglvqfWpwRcapaq8lxcLllNrXsTPqeZWOvdf73518sxeeU9Dknvqrjce1Sg2\nvEh3HUClh8/Kto6EF+jy0b/p+qRteOlj39vbw97enhlC63njPTM3hZUQnls6sQlz6nxK8loFa+01\nUiZBSXfdTVFifliNQarhSGkuN4lFkGul6427U0Via5VeJLxES1rzIejBT+sITfqUuWlJeIm9KJHw\n+pl1sVSnnexZVRfS54aipl6wbqXV11j3qSsl68Aiek7C5+6Ruq60v97TkLRKbxFegmtYpdc2fCoK\ncZ3gaWA5c5MlvIRQawnveek5/kLgpVNYuoTndGrj6xclYUvyu4rKqANfSvveS1HyXlajpANrJF5C\nD2UWpxynz8/P147YGjmTsaRcRZhxoI1Ieu5/t8a+WzZ73TJdeiw9p7mVT40mqxt0IvcvcVrVPT8P\n6MAVr29bDyVmE+im5Oey4XOejajtdx1JZw2O0UuL3UbU6fJkc5Y1XGuCC8tJNw8s3UufCtwoIb93\nT31cQvaSe+XO14FnQkgemejWNFhCei6bWd+H8yPlVOLz0Oq8nrTEGu9eR5tYdy0gB+3D0jPbWMFn\na0n4lHQvlfD8X88BZZGqjtNulmtKoSW79QxPwltj6+dl/3rOQg3LQ8/hzyzh9Vj33HP1uXUjviV0\nPAjhPQk/ryAbC9mZD0MIvxBCeBpC+AM693oI4UshhP8YQvjNEMJrJQ9Lqa5eA2A1Any/3LO855b8\nd1GVznsXT8pbE3bWHUPOUrzUd6L/b6n0LOH1jDZ6vDsjV4FX5bfxMI/8sHTPSfhZRsKVID/VKfBP\nAHy/OvfjAL4UY/wvAfzf18e1kFLrZxkswve1nuM9t869F4FUw1SH7DeBRXovr9YAJ8uG1yPh9PNK\n89UE1MlHifmSkvCLVumzhI8x/haAF+r0DwD4xev0LwL4wdIHWhW8zsww1j2sZ6SeWwclDUTdLfcc\na6osT62/CazuQIv48pw6Kn1dp11TyK1h9Vbc5F4pCa8D0OpOUFmCWW3498UYn16nnwJ4X8mfLEnm\nkdxqBLz7pZ7VRKRIb02PlZPwllaTeg5gxylYkpjP5VT6mzrtVtUN6mERjVCphNcj4+aFGzvtYowx\nhGB+pUePHlXpT3ziE/je7/1ek9CzOKWaVDHmAU+V54lCZJELXraqpBxK1f9cAyxBNbJJKK1sPIWV\nXm3ntsFzOHpdmyGEiYksvE1CatmBV0L6J0+e4MmTJ9l8z0r4pyGEb4sxvhlCeD+At6yLmPCy6ohH\neG/KYuD2kduDRXZRnYfD4ZREsBxintljaVQ67c35LwNmnj9/PrG9ePECJycn1Rh3T6UvaWiaJt2B\nsgEpTEZr2ireHx4e4rXXXsNrr72Ge/fu4d69ezg6OsLR0REODw9xcHAwEW3HUXap58cY8fDhQzx8\n+LA6/5M/+ZPm9bMS/tcBfBrAP7jefzH3B658JR7odQ27tFDaz8zE4xFpMjiFh1ZaDSPfxyJ8as58\nnmPQWuPv4uJiYjIL2WQmG5nUQhqnlNPOe/cmwiK99ntIWqYYt5b63t7erghvkf3w8BD7+/sThOfp\nqL3IR53OIUv4EMIvA3gDwIMQwjcB/F0Afx/Ar4QQfhjANwD8UMnDtJroSZPbQvi6/cyWhOcgF1l2\n2lpiy0PKTNBLYglZrZVfh8NhFR/PcfIvX76sJLxeC5C75XSeSuzSpnz7FOklHUKoGmQ9sYXsPQnP\nhLdU+3k67rKEjzF+yvnpk3Uf5qnzlsTJebe9vuKmoEQV1Ko4l42W8MPhsBouKUS1GsVU74C+rx7P\nLl1s3l6vCMQrA/V6PXMhSc/PkCN9k74l4JOeN4mR18tDyd6T8EL6vb29iUbCUuk9yV7aIKwstDbn\nhZ6l26kpduCsrTFrPkJqXk1XBlZY0zvL//W92FbXHna9Ui+v6Wel9Zp/+thaAjyl0luVtQnfb1aI\nSr+7u1uNgBM1fW9vD0dHRxXhZc8SvtvtTpgCPGmlV6fq1vmlT1PNZNcqpnbarTPpNUq6vbTqzavy\nsKpslRUXdEZoAAAgAElEQVTDc9aJhGeiS/DMYDAwJTcf8zTTWgMYDAZmb0uJl76J3ysHK25BJDwv\nJnJwcFA55I6Ojirpbtnx7XZ7qmuOu+hSeQHKynElK89YpE/Z8OtE+pTdnsuXdtppG77dbk/ZxiVk\n15qDHukmG89UY22Wbc97r2fgtsGy39mGF5Ve1hhg1V1Ibjntdnd3J2LoOa2fb5VriWa5snnpPUlQ\n4rArebEmS3pgWp21ot40uEHQ9v329vaU1103qkxuvfX7fZPk4ph7+fKla/tL+jbACkiSfa7bbX9/\nf4LYLMW1+n5wcGB65eVZufowa/1euoRPqfYlUWR1sArSW84d65xuta0VSPT6Ytvb2wjh1WKSsh6f\nzNfP/d7a+y57SyXnY63G6+mkxZzQa943sWGdBUJqb69Xh9Hb/v5+RWgmN5Nc+tplZlq20yUPpT6g\nWer3Spx2OuCjjhqfi21ueuXzJAjHVFvri1mEl1lkJG1JYL1gJ89Bx3ueuYaddBwq6wVI3Rawam5J\ncP4W0nXGabHZRXrrtDjvhPDS9cZhtJwX3qfy3EinHZAeAlqX+IBvL6+6EuakvKWupdYYk8qhCR9j\nxHg8rhw93A/Ojj4+p/vWOe11yQ0Gg8pv4MX13wbwd7AGtPB00jwHHR+nNume033t7JSbpYdnroE3\n80KqyyhH9pKXSfVPrqJCpvqZPQ+vVDRrQcF2u11dK4Qfj8cTjYclvTmtGwBr8wJvRqOROe5hXVT6\nEk822+lWtJx0s7F6zl54XkjC2us143Qknaf91Xm/HFY6xZVlw1veZYGl8nC6CdKdYfUzW5Jd1EjL\nhmc1MqURXV5eTnnd+ViktDUvfMoM4GMmeWmXWxOgyz9VR7Smxeu/dbvdyj63nHI84yxH2MnG67pz\nQ5Kbc35eUXZAw1X6EswrIGGREGnvkd0aKmktGSxEFQlvrfzibefn5xVxucuP01Y3KW9WY9x0CV/H\n7NOmlf4GYqPfu3cP9+/fx/379/H6669XaXbE8SKRTHTPP7AogmusfBLLlDrvVSZP9RFiNb0CalVe\n241cSVhCiM3OXno96wyv4KqHr3qRcNYa7d53sMysppZ3XeJYphU3uJrwDx48wIMHD/Ce97wHDx48\nQKfTmbD7eYw7h8haJp2X33mTf+XTVK+j1FgkUl2XOhhHq/C8EIQ1dt1yuvHxXUJKMHBDbDXGvCac\n2PUHBwfodDpTDr9UtFyuji9C0q8k8EbA0mKdJMessBo1sYH16Di9msv29vaU55wni2T1XI+E8+Ic\n1snptkx4PiberLIej8fVPViDm7V8U47fWbGySLu7DK5QrVZraoQch73KiqxbW1sTw1fZm26RPTf0\neJ5k50q57t9WN4alpL+4uMD29is6SU+KbKkJLErys7ZOuw2mbeCrqyuEECYGzPDkkNJf22q1zG40\nS7rryrjISEbv3DoSP+df0qaVJem1M5a/96zEnSfpV054r2KsY4XxYH0sITr3reuRbCLhd3d3EUJw\nZ6Lx1EuWROKUs0YizlLW844AaxJKVHpPrWf7f55alNXFOwtW4qXnvXfutqBE3Y3RH8kmXTsAXOmS\nmsXGinu/qYRfZKz3qlGq0nuNbKvVwuVl/SXB6uTvJqRfuYQX3EZnnfdhrHe1nHY864k0Cno0nOxT\nRGfCyzPn3ciuI7k9eCq91ph0z4mEyS56rMFNSN8IL33u3G2C17CFEKYkPI+k0lFulhTyVHlNeC8/\ns6KpkY51kbLhSyW8dMEtOvR4VtKvzEvvtaK3pT++5GNo8mspwlF07AGW/dbWVpW2hhlLl59UDi3Z\ndTp3XPf95LomfsOUeaVVebHNef4B7VSVmYR12DI3BvwNVxV8s9IprqyW02oZm1hhUigN59TnvCi8\n1KQLHJopTj4dUit7LnNLXc2ldf5Lv0vdgJN5oa4U5Pp5eXk5tQjEcDisnKgyclFMrYuLi6nRcDwq\nrtPpVBqbt3GILT9XjueBlU5xpdWjFOnXBfrD1Bm4IddYcfYyQaK1SVSXNZMNR+NZ3XUpx58VVGJ9\nk7rfadVdd6nnaunO56XnRJNdfC56hJy3qowVZ8/DZLmh0cS/KVY2AUaqu2OdCc8okfSe5LckvMTW\n8+QLPBKLY+uF6DzOPTdFlaR5L0EjnqmV+j7W++fefd7Q3VklZJe4CP3bcDisSKnJ3u/33eGx3ADw\n95O9PFPyGWOsvrs8e20lvGcjiZS5LWTX4MqdGq1lbUx4juHmfbvdnhr/rtO5CTD0JhVOvhN/O4H3\njbwKWtLgLQKlz7DeUd5fvodF9tPT04rc1iaDb3jCDBkIBaDyx1hTn+lGa9Z3AxoyTXUqBHRdiW/Z\nX6WV25LwotLzME2eRqnb7SZnlOWprPR8dnz+/Px8Yo41bpQ5VFQqap13yjV4qwbXT66v8i3kGnGo\nCtllCK0mNc+Q0+12q+/Fi3SI83VnZ8eMu89J9rrluBIJr/uRtYTnPswmVgwPdTyspVLeUum73e7U\n9Mf7+/sT69CxWi4bO/G0Q6/f70+M1wbeJfvFxcWE5JGKqnsZcu+/LqTn9+OISC6PwWAwNQRWCG9N\nfyWTZ8hko5rs7Xa7ct4BmNAmuOHRea2LpdvwltPOkvDr3C2Xst1TFd0iOqdZwsuUyLJW2dHRUXLW\nGllZxtrOzs6mZk/lyi0VmiHvUrd7rqlEB6YjPnVjJaHP7FTjbyTfRq84I9twOHTJLsuBy7PkmpsM\nvLGwkvHwXtii1Re/zkj1rZb+33LcsfNOVMX9/f2p0Vv6OOXlZ2+/1V0kIaMc0KODfBh11PwmNQJa\na/FgfVOe2VfHUcicgJbmJtvl5eWUB1+epRvcWbHSWPrcVnKvJlWWeUEHYqS6avR12tHH5Snr0kmj\nytfxVE56YofDw8NqsUjLqy+b9r3ohl1jXb9bSoNjzYi1AQATM9/IOYmQvLi4mFg9VjZpJHjGnJuU\nWyMi7eqSP9VNsa6VSOAROEV2/X+RHjHGCcmwu7s7EUDDUznJpIvcC3BwcDBh72uPvz72xuGXSM1V\n982XIuenYL/UaDSaIDY33MCkA3A0GuH8/LxqaFn1l7EU8yiblXjp9blFqPJNrzgppEjP563/sYTX\n2NnZmSC7qPIi1TudDgaDAfb29sxFI/XSVHzMa9fznhv1Ou/exO+ny9xyorGE11Kf/6M1ASnjo6Mj\njMfjSqjJzLnzKo8s4UMIvwDgvwHwVozxw9fnHgH46wDevr7sJ2KM/7L0oUxqS9rL+dJ7NbmS1IF2\nEnnqvL5G38Pr3mE1Xsh+cXGBTqdThYamJLjMjSez8PR6vSrMlCfokPzqys7IfaummWo5smsVXROb\nTSlW43koNDv1eCXabrdrmkSp/HgokfD/BMD/DOCX6FwE8DMxxp8pegr/sUB990ivyX1biM7IqfIW\n8a3/6kaCVUohu3bC5RapOD09rTYxAdi25HhwrtizkrcppM+RieshN3BMdl7xl1V+nuhEVuDVy06L\nxJ81f4ws4WOMvxVC+KD1nOKn2PdNqvR1Jb2+t5VeN6QI7H1kVue564y9wlzmvLFn2UqfnJxMLITI\nZBeppCVbbm1zybPkt2nI2ez6d5HEHJ0nZS+kF7KLZBcHnZgBQva9vb2q3144UYfcFm5iw38mhPBX\nAXwVwN+OMR7n/lDHQ19iy1sF0MRKUxeWva5JL7/p/wlYrddzq2nIuVSX23g8nlguSZNdKrccSyOh\nnVSpCtsUiS6oQ3bJuzSm1rfZ3t6eILteoeby8rJaX16Cq1jNB1Y3480/AvD3rtM/CeDzAH5YX/To\n0aMq/T3f8z34zu/8TgB+NFnKXl1HcLdhSirIe+sFDPUac9YKJrkliqw8WM6/EMKEymnNrGNpZayd\n8fzrTH5pAHSjowUA51m/wzqA88kElb2W/hwfMRwOsb29jYODg8pPIo5TnpnYc94+efIEjx8/zuZx\nJsLHGN+SdAjh5wH8C+s6JvxgMMDx8bE5uT9Xbj02OCcRrHOrriCe5qHPsSc9t8jB4eEh9vf3J9Yv\n09FxjNJy4AaB/ytaAd+bI/1EPWVvMo8A4+m5pIHyxtmz3dsk30zuO3p5tfLN18UYq/H27ODTA5n0\nGAc9toK3N954Aw8fPqye97nPfc58p5kIH0J4f4zxW9eHfxnAHxT+b0qa6QX2ONqrhPTeczyJsWxY\nlcayxfUCkhxFx+uMaxu6xEaeNY+6IeEuPO46ksAdVlG1NrK1tZWcYZeDgbwehlWghPSA74fQeWft\niM2gGOME4b1BTdY2V6ddCOGXAbwB4EEI4ZsAPgvgYQjhI3jlrf8jAH+j5GFMdmstNU32nLqq06uW\nCAxPElj2OHvOdbSbBMCwhOdFC3MfPEUWr+LKOSG8nBPCs2TnYbt6GWT9Pa0BPfJsa/z5qhtrnZfU\nuVIvOnvyueyvrq6mhifrOAjmiwTjhBCmHKYplHjpP2Wc/oXsnRW0vW6p81xJSqW7ZbMuG14eLdIz\n2VklS0l4kfIy6kpL+JIBFrrSpRxplvOPCW9J9r29vQlfg+VnkH59zrOQ3Wp4mkB2QcpBXEfCsg9E\ntCRpAHhNAi3h+/1+Vd4cr8ITZZRgqaG1WqUvlfCzqKzLqjC5vFmVQjd8mvDSTaNteJ41pVTCp/KV\nUkNlz04nidTj/Io2MhqNsk5F+U3O6a47T+NoCvE9D7n2f1jQ0pwdoFIfmOyWDb+zszNh+kijK12t\nJVg64UslvPb26vuknrGsClK3ZZf/cKOnGz9W6Tmm/eDgoCI6q82lEt7LkyXdLY8+8IqwrMa32+0J\nO5wdr1aDrbvyxEvN3zpnE68aOXLnemS42461mxBC0mEnhNeaoqj2pVga4bkCaQ89E7+OhNeVch0q\nBzvrLE1Hr0fOEl4PZb2JhM/lkfcCeR5LGva0a8nODZFoBlzRrfHlVr6aCk8zyplJ3EXHYJXek/By\nLyF73QUvViLhPUk/L3V+GShV5a3/5DQdHqoq5O90OhN99DktSJ4zi/PLIrzcQ2x6qy9dvPDWZCYA\npoJwRMrr0FPdkNSp0KuEp/Lz7zrNe2kA9XLhvV4Pp6enaLUmpxZj08obgqyxdMLLXjut6gTecCXW\nlVkX5KrgkUy/N0t4b1+3QbSeXdpwztKQMaE5Bnx/f78iq+6dsfqSU6vgllTmJqJOHRXNR4bK9nq9\nykErcQzcMAKTZV6CpY+HZzJ7pNeboLRPdBnvMOv/cja8ZerwxAl1NaA6ZVMqnbxzrVar0lBkhJd8\nHz2bjkX44XA40W13cXEBwJ5Dz8v3qht6gdcTkipH0Xh4Nlwhu0h3j+ztdruoXq7ES28R3SJ+CqlC\ntFrTpqBEwnvSXnv260j5eZpGXplKBdR99VIpLbJznRCblR2RPG7cekfv3Kq+eynRPeKPx+PKfu/1\nehM9GHqmIj2irmQarJXMeDOrhNdokkfXU6M16fR76yCklEqfMntSPoO65ZIypTzEGCd6Grj7Tipl\niuwhBLfbrpTs+vcmNPZ16ihLeAm0YSenJdk7nU6lGTWK8J6XOiXdLedRKgBCo0kqvtWQacddTtrX\naQzr5q8uLDuUu4lYssvQT0sz4U1Ldh5xNwtWRfpZ6yjb8IPBoHKQSqN3dXVldt/KFOXb23k6r0yl\n1x9aR9elKrUVzGL9vqh3SP2We7Yl3T2Cazuey0M3oItEqYkkhNddRrL3CM//F8nOM8Jwdx7Q3EFT\njFnqKHvqhey86IV0ffIAq/39/WoIrZ452MLKnHaeZC+14YFm2Od1yJZS6S2y699meW7dxuAmZaq7\n3fQ9PQ2OocnOwToe+H45s2rZqPNsPUOQSPbhcFgF3ejRlLK4hQyzzWElEl4Hm3CUliXZrCAOK70K\n5Ppe9bWclk33N3M/t/5fnecuWvJbz5PvpCV4CGFqemurC46nfbppJOG6QeoCDxWWxk/4cHh4ODWD\nMJdnDku14dlelTnVpJLzons8zJIHZDAZSkixCKS6BnNOLf6gPLMp8CrslD+gXjJKHFeeyZMyfTyU\n2pmpd5Pf+F5eQ2PVAZFYo9FoakllduClkCr/VQuEupA6IqZNCO+uZKtXE7JWAs5hqRJef2yOGpIP\nrgeI8Nhqb4GDJnzUksrGhNezl2xtbU2EVlqEZw+93s8rzx6Y2B7BSjQOjg7jaMJut4vz8/OJKbTY\nr6PzXdpYrRtYmOn16TkQSa9uw0IhhZVJeCa7Jjyve86k53HEPIXQsj92HTVe/88bCy2BJxbZZc/+\nDun2ApDtjqlLzBxmJX9Kwo/HY3fyjGU0aE0A93xwZKF0eVpE16TPYamEF/tdSMrnRqORq9JLBRCi\n6OmQcuq05Ui6KXKk9yS+ttE04S0pLx+TnXzyXnIPaQRyeeDzdYmUcoqlTAu+VhNeRtxdXl5OaHbe\nEFt+h7rlvw7QZJdzYs/rdQMbK+GBd1V6fSx9j1ql1/Ycf2AmSwratpw36fUzcna8brl5thIeHqnX\nbZN+bL5/qkvOI7+V17o9DTmp7pGfCS/SnWPlPcLn3rG0/NcBkn89Sy0PNNJE5+Mclq7SS1oku9jh\nKQkvUt4ifE6iWOfmXSlK7scNlNZwmPCphSB4eKRUAkuyp/I2K/Fz0r1Ew2IbniW8lAlP3VVHpV93\nkgv0e/BY+RCCab83VqUHUJFcWnl2UMiSR7JpsnMfI3syPSlXt/IuA0x09mEISrz08h8ZTFH6Hrpx\n9IhfUpZcfjdR6WW6JikLbuhLJPxtBNvxAnl/a/LPxqr0KdU6xjjVD6/XL5eQTZFqdQJ0mgL9Mbkc\nrDnhvf75nKMyZbvfpLwsopdAazNC+MvLyyruPoQwodJbDjx+pre/DfC0Mp5nwCN9DisZPANMVx6u\nEHoTR5XM5Z2L1iqp1KuS8la+vffm81xGJd1f1rtZkjp1rOGVV0lZyu/8TTm2IoQwMdmHbBKfIVNj\nW12yTemaXQb43VlACPFzWBnhLXjET42yWicw2TXxc2QvGWeQIt5NyV7n/QRao+BvKxobNwR6hh8m\nvayzplfF0evP5/K17g0D+4JY0jcutDYlcbQNyJVcTxKx7sS3COtJeC/IxmowUs/L9ZGXSPZZy9gi\nPftv+FxKune7XbRarQlVlp9hSfmU9reOxGeya+neCAlfWqha8lkSfhFkX6Varzct1T3yl6r2s6r1\nHupIUe+/8n4xxolYAnFAcsCVRf4QQvEClbOaPatCaUNkkZ7j7XNojEpvEUCr9NYQWus+3rlVf2At\nmT2iW1I9RfyS51rEWIVmJO/CaemezdnwwOSIPB21OEteVl0nJB+cLonlsCR8owbPlCAl3a150NZN\nnQd86Z7bcrZ7yXNLjxdFAt2A6d4HLd21Ss/55AovDYj3bnKuCeTWqJNXLiuW7mvjtLMcOjkpX5fs\ndVrPZUG/X4kNX1ell+fM6q2fRzl5z5bniVov12npronPFd4aceiBtSoveGgVmMUkAzDlpecFPXJY\nKeF1X3xp33ITSFsXFlkttd77TZNc7wG/b1oHceigGyvIY17v7N3bSrM25836c9N1C5rS6M8CVue5\n/13GXpSUxUoIbxW4tku0baIXN2B1MPesdVD7tW1vnbNIrmEF6GiPuPVMPpdqVHSeb+LM4zxbedd1\nwgpEqot1JbuAyc4LVshMwTksnfCe9LGCCazxv/rDz/LspiFF8FI1nknCe91llSJz6nk3IX0daLJb\npNfvV3LPVLmtC7TtzrMDyUCkHJZKeN2Sc5o/rEd6PSmill5yP/1xm/pRU9K0tBEApstVk6WE8JZ5\nwb/n3mNeNr8l4b33sb5/7v7rDCkTLeFl7MGNnXYhhA8A+CUAfw5ABPCPY4z/MITwOoD/HcB/DuAb\nAH4oxnhs/H8is7LXH4odMV53gyXhZ1XrdJ5WAc87K/s6zjmBRRQuL+sZ+nkxvjuxhvSZ82913qck\nv6n3YIl2U5V+3ckOYMpZyRJ+a2trLl76CwA/EmP83RDCAYB/G0L4EoC/BuBLMcafCiH8GIAfv95q\nZT5lr+m1xdiOtyTXOsFTo/lY/zaLSq/9Ht79JW11b2n7f5HlnWuwPJVe52ld/DZ1oSW8zOorjswb\nh9bGGN8E8OZ1+iyE8DUA3w7gBwC8cX3ZLwJ4jELCp9R5y4bXKj3HT99Uwq8aulKWEN/6H8Miu5Sd\n17hYEjyEdydR1HlcVJmnVHpPwq+rhjcrpCxEwvNEn3ONtAshfBDAdwH41wDeF2N8ev3TUwDvK7mH\npdbr0U+zqPSztPBN+thacpeQ3Xo/T1vSk2Z69+fFLlLaRB3Sl/pTUkIgN1z4rkBLeJ7CO8Y4P8Jf\nq/O/CuBvxRhPlW0eQwhmqT969KhKv/HGG/i+7/u+rFTXkjzVLVfXS980cCXncuAVVKXbpd/vu8E5\nQlRrNhQuv5ypkFvfjvM8j/fW6V6vh36/PzHvOk8AYpl2Hulzjf46NhQp8/fZs2c4OzvL3iNL+BDC\nDl6R/Z/GGL94ffppCOHbYoxvhhDeD+At679MeHE46AxzN4Mn0b0uuVQLn/rgq/7YWsMBUBFdlgk+\nPT3F8fFxNa/f1dXVVICODkDxylC2nEpvLVHNm/yn9B1LjyX97NkzPHv2DM+fP8fx8TFOT0/R6/Vw\nfn5ekd/T9Lw83CZbPtVg379/Hw8ePKiu/eY3v2neI+elDwC+AODfxxh/ln76dQCfBvAPrvdfNP4+\nAU+FtyS5R3q+vkSlsz74qshu2af8m0zkyYTvdDrY2dmpJv/wJLO8Y4rsEpSRIj3PNsREl2O5XqDT\nrLGk9vzejOfPn1eEPzk5wcuXL5OE53pQ91usKyzTj7W9HHIS/mMA/gqA3w8h/M71uZ8A8PcB/EoI\n4Ydx3S1XmmHtVPJIngu44Za9lPRN+NBcQdlzzhK+1+tVkl3U9dFoZNrvnM5pSDmHIE8txkTnKaY8\nXwJ79C1fDb+/Lg/B8fExXrx4gRcvXlQS/uzsDOfn55WJw+9T0lPDPQzW89cRXqN/Y8LHGL8MwLvL\nJ+tmNGWDeJWUN237l7buTfnIVgMlnnAt4Znsl5eXGAwGU/fR986ZQ0C6H17PI8jHMmtw6v9ai9Fb\nKv8A8PLlS5ycnFSblvCyZLL255TUg6bUgXnBInuJ+bL0SLsSwrPKxsdsEnB63SD55w+mJbx8wKur\nK1xcXOD8/NwkEB97DSZ76QE/yk6IzZNH8nHK/hfCaw96zrHG6PV6OD09rbazs7MJwlt98utaB2aF\n9Q10dGQKS53iqsSGTxGf7+NJj6ZDSzpulYXwItmF7MPhEGdnZ26DJ2lPsnsSXsCE56nB9Z4rV47w\n1j6Hfr+Pfr9feevZaz8cDt2gm3X6/vOAp843ivACy4Yv2UTC833WDTkpNxqNqo/GZO/3+9WCDanN\nI7vulgNsKa/XAuDj3d1dk/B8bMVU8HEOEiY6GAymtuFwWOwMvAtIOW9TWLlKb0XWlfS5r/NHtvLO\nffBMeulTPz8/T5LdiqzT8QsCj/ia5PrYIzqbHynS58Br6ulFOXSDv+4oIaf3P0uyN1LCW11THrlT\nKttt+vAMXSY8m4tHdMs8yo050D0XrI5re5/z5TmKLAlv7XPQ66XdVhu9lOzyXfT/LMLzIqMpLH2a\nai3lU17Xu0R2YHq8M5NSE9w6p7Wm0t4MIb38T88KyxqWp0qyl97zM+TAZNf14TZDNwCa5B7prajL\nHFYq4TXxvX52/v9thpQJ29t8LucF141oriwtKS7Ptswvz0svx55DtZTwXuzFbXbMWdJek9z6nTee\nFiyHlTntcmroOnviZwWrzmJzyzkZm57bLE3AKz9Neraz5f8S5cfzyMve8vh7363k+6Xqwm2B5T+R\ndMl7era7qPU5rEzCW6T3KutdAJeLHEvZ8JLJFpEk7TWUqXKU82yDcyMjKr6eAcciO9/Pyl8OnmZy\n2+uA1pJSaTm2bPjGqfQCS1201Pnb/qE1LBU9hGA60XL7uoST60STsJx0nnSSY35OHcluvf9trwOe\n4y4l6XMe+sZKeMvWTKn0dwVaWgN26+/9N3Vc9/n62R5SDqdZnm/tbzN0GYuZxXvrP7fCS+9J+bvw\n4QF/nPiq8rBB88BEt+bvz2GlXnov2KYO0S3ps6m0GzQRKU2ptK5riS6DnCRQKodGqPS5wBuNXOAC\nq0cbLA6zRoulcBu/meXvSCFVBizdeUSjREPmsJJuubqkZ3j9lnJvff42VqBVYhEkt+5/G75bSVnp\nrtHUvXS/u6jxHP6cw9JHy6XI7kWHpbyW1vFtqCyMUpIt8r0XTXTreev8HWctrxLp7kn4drudvf9K\n++EtspeGhFpdRCV9mOuGOhVnEQ3esomun73O386C53OqE3jj2fCNkvCCUpW+rjov6dtUQVZJtjrP\nn1c+b8u3y9VTC9a7W/V/rWx4JnpKyucCL6z+Yd13edvIXwfLevd5N0i39ZvNs5y0Ss82fKNUek+d\nl/HeFxcXyVFzKaxaEm6wwTxgRTbytre3N7F1u92JfafTyT5jJRJeE10mObgLY6GXgWWVWamHuc79\n1h0pG73kv9aaAHJ8cHCA1157Dffv38drr702sR0dHTVLwgOTKj2TXsgu847fpbHQKcybUIt4fuqa\nu/LtSrvfSu4jKnq73Z5Q1Xd3dyvC37t3b2rfOMJrdV7GPLOU18sJbaR8PdIvopy4tyN1zSJ6CJr+\n3efdGLdaLWxvb6PdbqPb7U5tBwcHuHfvXrUdHR1NHDfOaae98p5Kv5Hwk2hCGeQannnncd73m3ej\nuQjNSyS8EP7g4AD7+/s4ODjAwcEBDg8PcXR0hKOjo4m0bI0ifErCM+GtVUWaUOE3WI6JsYhvXTeW\nYVX1TST87u4u9vb2cHBwMEXqw8NDd2sU4YG0Dc+E30j45mIW0q/yG87SQOVIXye8W1DHhmcJf3R0\nhPv37+P+/fsV4UXi662xhE9JeCb9hvDNRCnpV/3tluXwzAV/1TETNOHv3btXrQx779497O/vV2q+\npOWmA8YAAA2DSURBVOW4caPlgPSqGXVmWNlgtVi2Tb9s3ES1T9VbXbetfnYhsKjq7Jjja7g/XhYf\nzWGphOcIIelu6HQ66Ha7E4sODAaDajFFns8NyI9/zw242eDu4CY+hzr1h5+TCpzR01Hp9OHhYdXH\nLl54UeH39/fR7XbR6XQqcksffelCksASCW/FAO/u7laEl9VGBoNBNZh/e3t7Qvpr6A+6KJLXqTR3\nqaHxCLVID/syyjf3DOu9Y4wTizrqtA6HtQJsNOFFwgvhJZpO+uWZ8KVIEj6E8AEAvwTgzwGIAP5x\njPEfhhAeAfjrAN6+vvQnYoz/MnOv6qV5OF+3263s+PPz8+pluAVjFajEPrIk/iyY1eFz0+euExY5\nKnEW55hGSRyBdf2s0ENYea9X4+UlvXZ2dnB4eIjXX399SsKLnS6qO/9H8yOHnIS/APAjMcbfDSEc\nAPi3IYQv4RX5fybG+DN1CsNS6bkv/vz8HP1+f6r1Sg2FZTRpWGgdAsyjkVj1wJNl95vXfd9FxAl4\nDZIevippqfOyibSW7eDgwFXpxV7nZby1hC95xyThY4xvAnjzOn0WQvgagG+Xd6tTQFIQrM632+3K\nIz8ajdDpdNDpdKYkvF76SO6Xs9fn4XRZNHIe3rr/B+6OdtGURo6/gae+6wg6MWU5rR102obvdDpT\nJsHCbPgQwgcBfBeArwD4GIDPhBD+KoCvAvjbMcbjzP+nbPh2u111v41Go0pl0U4JUVlKh72uKnjD\nGr88S3/uKqO9moCblsuywcTXKj2PWRfCW572vb09M3qOCd9ut01ToY4NX3TltTr/fwD4WzHGMwD/\nCMCHAHwEwLcAfL7oYY4NL10L7JDQTjspUA2W8vOIykvZjbP8d155WOb/mwb2gHu/NQVCemuCCtFg\nJYpOJPr9+/fxnve8B6+//vqUSs8OO80Ry8eVQ1bChxB2APwqgP8txvhFAIgxvkW//zyAf2H999Gj\nR1X6E5/4BL77u7+7UkN2dnYmpqTWZOduOSH9vJxxG7yLElt5VqS+0TJ6VGaJ40h18aa63EIIE4JM\n6jLvhejexnHzFsm5n103go8fP8bjx4/z5ZNROQOAXwTwTozxR+j8+2OM37pO/wiAPx9j/O/UfyPf\nezweV11v0ucu6eFwiJOTE7z55pt4+vQp3nzzzYnt6dOnePny5dQkGny8aIfdLCq9dy53z5uq9HWe\naZHCkqip/1nP1pqX97uV3zoqPZ+z+r35OAddn3R941VaPaeckFOkOTvnWH0XQmu1Xv/e7XartCw0\nkfpOgmsBOfVDTsJ/DMBfAfD7IYTfuT73dwB8KoTwEbzy1v8RgL+RK0z+CFxY8sF0F4Wo9KLWb21t\nTUx/1Wq1lhp6O8tzZunPrXv/Ov/PkVunrd/0ffhYN74WcbQvZp5+GSb5LGunc/3ijctAL/zAaSG5\ndsbpfcpxpyX6zs7OhFMuRfIS5Lz0X4Zt5//GLA/Tjjv+iDzgn8nOav11ngBgYrHDZXW5LAL8rFkb\nlbr/98jsSUZ9Dd+H95bmVaKJ6XOl5W/dS+xnFiySztUVvdah5EFIz6PZLEmup5zS01BxN5zukrM2\nz06fxVQRLD3STgpf20UpCc8htvIh6jgqZiHUPKTvIq696f89SW6RnaWlJ2F4ryUkS29vKevUO6XK\n3yO7Fiq85/tZ/5elsaWOjcfjiXzKfYXwliqu1XVW2TudTjLwhs/LXud9FjOLsdRYev4Y+lxdCS8V\nIveSuoBmIWId4q+LM9Eju6USyzH/T9J8TqvCooXpNe85D3LeglX+Of9Eqh9cf0d9LxnFqU0UeQ8t\n4dnJxk43vZe0ELjOxv3sNyU7sGQJL+oWH3Orya2dJrz8j+0+vrfX4lvn6pJyXUhcgpRUt+xeTXy+\nh74fT0vGmpiAfS/y3xIbvq4GYy3UsLOzkyX8eDye+I3NRrk3B9CI15270Dj+Xe93d3fdgTNe/7oO\nrPHqdClWIuE5Lc44LeE16aVbTre63sumCmERdv86QDeSnoRPVTrPHABQqcNC9svLy6k8yHfje8zr\nW3hkTy3SwM9mx55uuOR3LeElWEYmqODpp/SeGx2vLPWxpVHpd66DpRNeWnRp6aWQ+UNZ616LSpYi\neF1JkDtXtzAFlhMqd/8cmay8zGK7p9R2LWG008vKG+eRHV65ZcSYUOwNr/Nuukx4UUUWIHKs64h+\nhp5BmSdmGY1G2N/fn5pPTmaOlamiRcLrtJ6gYh6N3Cz1c+kz3sher0DDXWxS+XRUnsTdiySRvcyB\nZ/X9ctoiT0mrmmpovD5myznlkUWbN7nwyVSlZeg8W9JbS/UU6XPl45HcOtYecW8BUf1++htyXjTZ\n9V6us+4dY6xiRazJVZnwEgmnp4mWkW17e3sTXnYd674qsgMrXFvO6u9kYoj6xEvpbG1tYTweTxFe\ntIVcl1COcPJca58rYOuZ3KDxe1nPkPdlU0ZvHhm8CqTzrAntkTvVrZVSQ0skfGrTDaRuNHPfTpO8\nlPCy14TXU6jLABcd6y5pGcIqXXAcHs71xKo7uW9X+lsOK1k91pLuTHiWONxlIeT2iK/vGUKYSGsC\n63SJwyr1bpaXmhsAz0ZmsrMaqrtpLCLwsaW58LEVKZbadCOQsj8BmKTW53iSUp3WBLca7FSDrIef\n6mNR6XOE5/USeGO7XYfFHh4eVn3yOt5d8uiZel59mofNrtGY1WN1VBPb8KzSM8k14S0vMT/X6moq\nVWtLWmlLTeU8pcgl3l8djsmVJyX9UoSXtO6X1vvcJvfyiJ+S5ldXV9XEpUJ0fawXEtX7XMNcSngu\nP07rxVFkQlVJS3gs96/znp8rm5bwXh3y/DyMm5IdWNFikh7ptUrPNjx3aQjxW61WlebGwPIOa8J7\nUtaLk859tBhjUl29uroyiSbpnZ2dibBLic5iFdGSepafwNt7/bxWF6i1aYLLPuW042MmOJPLWovA\n0pJEEHg9CZrgerMIz8c6T3qvv5H+Trp3SQ/xTnEjh3mQHWiAhLfmoGcbngmvye6RXtvcHBqZ8kLn\nyJCyv6TCsKoqFUh8DLoi8PHOzs7EtMNainS7XddHUEp4qzLqvOQI791bO+0s4ntE0lOTW9vl5WXW\n18ATo1ozy+g6qDc9fbpumGRMu6eFcRcy508LCy0o9G/sa5gX0QUrseEt0nteekultzaJkNJqvEh2\nseNTFTpFCFFpPdJbEoLzdnV1NXFPvckUR17wxt7eXlLdTRFeNu/ZXqCT1nS8+8s+J+E9VZmXGNNm\nEJ9LmUNCeE1CTrMNnyO8tckzvE33ZOjy1/XHSkse5010wdII//jxY3z84x8H4LewAnak8YdNfSxu\nRNhOL/HKt1otDIdDtNttU9LL2HzJO0M7fvQ5yRMA997WJAmiLkp45te//nV8+MMfNgnPKi+Xn6Rl\nr8cq6M2y5/VYBn1P2T958gQf//jHiwlvecGtbjuL8JwnTqek71e+8hU8fPiwFuH1sQgFa+NeAF0P\nSvDkyRO88cYbxdfPivK5cW6IJ0+eLOtRM6Hf7686C0n83u/93qqzkETTv+8mf6+wNMJvsMEGq8eG\n8BtscIeQnOLqRjcO4e6NTtlggwYhGlNcLYzwG2ywQfOwUek32OAOYUP4DTa4Q1gK4UMI3x9C+MMQ\nwn8KIfzYMp5ZByGEb4QQfj+E8DshhN9uQH5+IYTwNITwB3Tu9RDCl0II/zGE8JshhNcalr9HIYQ/\nuS7D3wkhfP+K8vaBEML/E0L4dyGE/y+E8D9en29E+SXyt5TyW7gNH0LYAvAfAHwSwJ8C+DcAPhVj\n/NpCH1wDIYQ/AvBfxRifrzovABBC+ASAMwC/FGP88PW5nwLwLMb4U9eN5v0Y4483KH+fBXAaay4w\nuoC8fRuAb4u0ACqAHwTw19CA8kvk74ewhPJbhoT/KICvxxi/EWO8APDPAfylJTy3LhYTyzgDYoy/\nBeCFOv0DeLUoCK73P7jUTBGc/AENKMMY45sxxt+9Tp8BkAVQG1F+ifwBSyi/ZRD+2wF8k47/BO++\nYFMQAfyrEMJXQwj//aoz4+B9Mcan1+mnAN63ysw4+EwI4fdCCF9YpckhCCF8EK8WQP3XaGD5Uf6+\ncn1q4eW3DMKvQ7/fx2KM3wXgLwL4H65V1sYivrLDmlauMy0wuihcq8u/ilcLoJ7yb00ovzCnBVrr\nYhmE/1MAH6DjD+CVlG8M4vU6eTHGtwH8Gl6ZIU3D02v7DyGE9wN4K3P9UhFjfCteA8DPY4VlGN5d\nAPWfxusFUNGg8gvOAq3LKL9lEP6rAP6LEMIHQwi7AP5bAL++hOcWIYSwF0I4vE7vA/gLAP4g/a+V\n4NcBfPo6/WkAX0xcu3Rck0jwl7GiMgyvhqx9AcC/jzH+LP3UiPLz8res8ltKpF0I4S8C+FkAWwC+\nEGP8nxb+0EKEED6EV1IdeDVc+J+tOn8hhF8G8AaAB3hlb/5dAP8ngF8B8J8B+AaAH4oxHjckf58F\n8BCv1NGI6wVGyWZeZt4+DuD/BfD7eFdt/wkAv40GlJ+Tv78D4FNYQvltQms32OAOYRNpt8EGdwgb\nwm+wwR3ChvAbbHCHsCH8BhvcIWwIv8EGdwgbwm+wwR3ChvAbbHCHsCH8BhvcIfz/L13+/3KVW8wA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f4cbb1bfc50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "index = np.random.randint(load[\"train_labels\"].shape[0])\n",
    "plt.imshow(load[\"train_dataset\"][index], cmap=\"binary\")\n",
    "print load[\"train_labels\"][index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 20066.,  19720.,  20107.,  19904.,  20094.,  20044.,  20157.,\n",
       "         20036.,  19910.,  19962.]),\n",
       " array([ 0. ,  0.9,  1.8,  2.7,  3.6,  4.5,  5.4,  6.3,  7.2,  8.1,  9. ]),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEaBJREFUeJzt3H+MndV95/H3pzhBpFAjlMrhhxNQa1Zx5FUIBaNmVxmV\nFrlVC1StwNk2RbtWVdVpSPNHtTh/NPBPVSqlhKoCaTdOMLRxg6CloBCKw2a0qSrisiWJE4diqli1\nB2wqkzqbqqls9ds/7vHxjTN4xjP2PBfP+yVd+Tzf+5znnOdq5n7mPM+9TlUhSRLADw09AUnS5DAU\nJEmdoSBJ6gwFSVJnKEiSOkNBktSdNBSSrE7yxSTfSPL1JLe3+p1J9id5vj1+dqzPliR7kryQ5Iax\n+tVJdrXn7h2rn5vks63+bJJ3nIkTlSTNba6VwhHgI1X1LuA64INJ3gkU8IdVdVV7fB4gyVrgVmAt\nsAG4L0nase4HNlXVGmBNkg2tvgk41Or3AHefxvOTJJ2Ck4ZCVR2oqq+09neBbwKXtqczS5ebgO1V\ndaSq9gIvAeuTXAxcUFU7234PAje39o3AttZ+FLh+geciSVqked9TSHI5cBXwbCt9KMlXk2xNcmGr\nXQLsH+u2n1GInFif4Xi4XArsA6iqo8DhJBed2mlIkk6HeYVCkvOBR4APtxXD/cAVwLuBV4CPn7EZ\nSpKWzIq5dkjyJkaXdf6kqh4DqKpXx57/JPBE25wBVo91v4zRCmGmtU+sH+vzduDlJCuAlVX12izz\n8D9pkqRTVFWzXep/XXN9+ijAVmB3VX1irH7x2G6/COxq7ceBjUnenOQKYA2ws6oOAN9Jsr4d8wPA\nX471ua21fxl45vXmU1U+qvjYxz42+Bwm4eHr4Gvha3Hyx0LMtVJ4L/CrwNeSPN9qHwXen+TdjD6F\n9C3gN9qb9u4kDwO7gaPA5jo+s83AA8B5wJNV9VSrbwUeSrIHOARsXNCZSJIW7aShUFV/zeyric+f\npM/vAb83S/3/Aetmqf8bcMucM5UknXF+o/kNaGpqaugpTARfh+N8LY7ztVicLPS601JLUm+UuUrS\nJEhCnc4bzZKk5cVQkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQk\nSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqS\npM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHUnDYUkq5N8Mck3knw9ye2t\nflGSHUleTPJ0kgvH+mxJsifJC0luGKtfnWRXe+7esfq5ST7b6s8meceZOFFJ0tzmWikcAT5SVe8C\nrgM+mOSdwB3Ajqq6EnimbZNkLXArsBbYANyXJO1Y9wObqmoNsCbJhlbfBBxq9XuAu0/b2UmSTslJ\nQ6GqDlTVV1r7u8A3gUuBG4FtbbdtwM2tfROwvaqOVNVe4CVgfZKLgQuqamfb78GxPuPHehS4frEn\nJUlamBXz3THJ5cBVwJeBVVV1sD11EFjV2pcAz451288oRI609jEzrU77dx9AVR1NcjjJRVX12olz\nuOSS/zTf6Z4WK1bA5z73COvWrVvScQGOL7CWXlUt+Zie79IZ4nz1xjGvUEhyPqO/4j9cVf9//Ae6\nqirJkvyUvfLKz4xtXQusP6PjnX/+L/G9733vjI5xckP88g73ZjXU+Q73Br3cztdAOtOmp6eZnp5e\n1DHmDIUkb2IUCA9V1WOtfDDJ26rqQLs09GqrzwCrx7pfxmiFMNPaJ9aP9Xk78HKSFcDK2VYJI388\nn3M6bc4557wlHW9SDPmmMQzDd2kMG0hDWOoQnJqaYmpqqm/fddddp3yMuT59FGArsLuqPjH21OPA\nba19G/DYWH1jkjcnuQJYA+ysqgPAd5Ksb8f8APCXsxzrlxnduNagaoCHloehfrb8mZ6vnCzJkvwX\n4P8CX+P4WW4BdgIPM/oLfy9wS1X9c+vzUeB/AEcZXW76q1a/GngAOA94sqqOfbz1XOAhRvcrDgEb\n203qE+dSS/1Cr1x5DYcPP7ekY36/of6CdVzHPZvGHm7coS+XJaGqTml5dtJQmCTDhsLy+kF2XMc9\nu8Y2FE6lj99oliR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2h\nIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5Q\nkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSd2coZDkU0kOJtk1\nVrszyf4kz7fHz449tyXJniQvJLlhrH51kl3tuXvH6ucm+WyrP5vkHafzBCVJ8zeflcKngQ0n1Ar4\nw6q6qj0+D5BkLXArsLb1uS9JWp/7gU1VtQZYk+TYMTcBh1r9HuDuRZ2RJGnB5gyFqvoS8O1Znsos\ntZuA7VV1pKr2Ai8B65NcDFxQVTvbfg8CN7f2jcC21n4UuH7+05cknU6LuafwoSRfTbI1yYWtdgmw\nf2yf/cCls9RnWp327z6AqjoKHE5y0SLmJUlaoIWGwv3AFcC7gVeAj5+2GUmSBrNiIZ2q6tVj7SSf\nBJ5omzPA6rFdL2O0Qphp7RPrx/q8HXg5yQpgZVW9NvvId461p9pDkgQwPT3N9PT0oo6Rqpp7p+Ry\n4ImqWte2L66qV1r7I8A1VfXf2o3mzwDXMros9AXgx6uqknwZuB3YCXwO+KOqeirJZmBdVf1mko3A\nzVW1cZY51Oj+9tJZufIaDh9+jqUedySO67hn0bhDjj3cuPN5fz2jM0ioqtnu/76uOVcKSbYD7wPe\nmmQf8DFgKsm7Gb3S3wJ+A6Cqdid5GNgNHAU21/FXZTPwAHAe8GRVPdXqW4GHkuwBDgE/EAiSpKUx\nr5XCJHCl4LiO+0Yed8ixXSmcSh+/0SxJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2h\nIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5Q\nkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmco\nSJI6Q0GS1M0ZCkk+leRgkl1jtYuS7EjyYpKnk1w49tyWJHuSvJDkhrH61Ul2tefuHaufm+Szrf5s\nkneczhOUJM3ffFYKnwY2nFC7A9hRVVcCz7RtkqwFbgXWtj73JUnrcz+wqarWAGuSHDvmJuBQq98D\n3L2I85EkLcKcoVBVXwK+fUL5RmBba28Dbm7tm4DtVXWkqvYCLwHrk1wMXFBVO9t+D471GT/Wo8D1\nCzgPSdJpsNB7Cquq6mBrHwRWtfYlwP6x/fYDl85Sn2l12r/7AKrqKHA4yUULnJckaREWfaO5qgqo\n0zAXSdLAViyw38Ekb6uqA+3S0KutPgOsHtvvMkYrhJnWPrF+rM/bgZeTrABWVtVrsw9751h7qj0k\nSQDT09NMT08v6hgZ/aE/x07J5cATVbWubf8Bo5vDdye5A7iwqu5oN5o/A1zL6LLQF4Afr6pK8mXg\ndmAn8Dngj6rqqSSbgXVV9ZtJNgI3V9XGWeZQS70gWbnyGg4ffo5hFkJxXMc9i8Ydcuzhxp3P++sZ\nnUFCVWXuPY+bc6WQZDvwPuCtSfYBvwv8PvBwkk3AXuAWgKraneRhYDdwFNhcx1+VzcADwHnAk1X1\nVKtvBR5Ksgc4BPxAIEiSlsa8VgqTwJWC4zruG3ncIcd2pXAqffxGsySpMxQkSZ2hIEnqDAVJUmco\nSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMU\nJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkK\nkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSt6hQSLI3ydeSPJ9kZ6tdlGRHkheTPJ3kwrH9tyTZk+SF\nJDeM1a9Osqs9d+9i5iRJWrjFrhQKmKqqq6rq2la7A9hRVVcCz7RtkqwFbgXWAhuA+5Kk9bkf2FRV\na4A1STYscl6SpAU4HZePcsL2jcC21t4G3NzaNwHbq+pIVe0FXgLWJ7kYuKCqdrb9HhzrI0laQqdj\npfCFJM8l+fVWW1VVB1v7ILCqtS8B9o/13Q9cOkt9ptUlSUtsxSL7v7eqXknyo8COJC+MP1lVlaQW\nOcaYO8faU+0hSQKYnp5menp6UcdYVChU1Svt339K8hfAtcDBJG+rqgPt0tCrbfcZYPVY98sYrRBm\nWnu8PjP7iHcuZrqSdFabmppiamqqb991112nfIwFXz5K8pYkF7T2DwM3ALuAx4Hb2m63AY+19uPA\nxiRvTnIFsAbYWVUHgO8kWd9uPH9grI8kaQktZqWwCviL9gGiFcCfVtXTSZ4DHk6yCdgL3AJQVbuT\nPAzsBo4Cm6vq2KWlzcADwHnAk1X11CLmJUlaoBx/X55so3sTSzvXlSuv4fDh51jqcUfiuI57Fo07\n5NjDjTv0+2sSqurET4ielN9oliR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWG\ngiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpD\nQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSd3E\nhEKSDUleSLInyf8cej6StBxNRCgkOQf4Y2ADsBZ4f5J3DjurSTY99AQmxPTQE5gg00NPYIJMDz2B\nN7SJCAXgWuClqtpbVUeAPwNuGnhOE2x66AlMiOmhJzBBpoeewASZHnoCb2iTEgqXAvvGtve3miRp\nCa0YegJNzWenH/mRXzjT8/g+//qvLy7peJI0tFTN6/34zE4iuQ64s6o2tO0twL9X1d1j+ww/UUl6\ng6mqnMr+kxIKK4C/B64HXgZ2Au+vqm8OOjFJWmYm4vJRVR1N8lvAXwHnAFsNBElaehOxUpAkTYZJ\n+fTR6/JLbSNJVif5YpJvJPl6ktuHntPQkpyT5PkkTww9lyEluTDJI0m+mWR3u0e3LCXZ0n5HdiX5\nTJJzh57TUknyqSQHk+waq12UZEeSF5M8neTCuY4z0aHgl9q+zxHgI1X1LuA64IPL+LU45sPAbub5\n6bWz2L3Ak1X1TuA/A8vy0muSy4FfB95TVesYXYreOOScltinGb1XjrsD2FFVVwLPtO2TmuhQwC+1\ndVV1oKq+0trfZfSLf8mwsxpOksuAnwM+CZzSpyvOJklWAv+1qj4Fo/tzVXV44GkN5TuM/nh6S/vw\nyluAmWGntHSq6kvAt08o3whsa+1twM1zHWfSQ8Evtc2i/UV0FfDlYWcyqHuA3wH+feiJDOwK4J+S\nfDrJ3yX530neMvSkhlBVrwEfB/6R0acY/7mqvjDsrAa3qqoOtvZBYNVcHSY9FJb7ZYEfkOR84BHg\nw23FsOwk+Xng1ap6nmW8SmhWAO8B7quq9wD/wjwuEZyNkvwY8NvA5YxW0ecn+ZVBJzVBavSpojnf\nUyc9FGaA1WPbqxmtFpalJG8CHgX+pKoeG3o+A/pJ4MYk3wK2Az+V5MGB5zSU/cD+qvrbtv0Io5BY\njn4C+JuqOlRVR4E/Z/SzspwdTPI2gCQXA6/O1WHSQ+E5YE2Sy5O8GbgVeHzgOQ0iSYCtwO6q+sTQ\n8xlSVX20qlZX1RWMbiT+n6r6taHnNYSqOgDsS3JlK/008I0BpzSkF4DrkpzXfl9+mtEHEZazx4Hb\nWvs2YM4/Jifiy2uvxy+1fZ/3Ar8KfC3J8622paqeGnBOk2K5X2b8EPCn7Q+nfwD++8DzGURVfbWt\nGJ9jdK/p74D/Neyslk6S7cD7gLcm2Qf8LvD7wMNJNgF7gVvmPI5fXpMkHTPpl48kSUvIUJAkdYaC\nJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLU/QfQDDm0PIfGcwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5efafb74d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(load[\"train_labels\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LA7M7K22ynCt"
   },
   "source": [
    "Prune the training data as needed. Depending on your computer setup, you might not be able to fit it all in memory, and you can tune train_size as needed.\n",
    "\n",
    "Also create a validation dataset for hyperparameter tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 411281,
     "status": "ok",
     "timestamp": 1444485897869,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "s3mWgZLpyuzq",
    "outputId": "8af66da6-902d-4719-bedc-7c9fb7ae7948"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Training', (200000, 28, 28), (200000,))\n",
      "('Validation', (10000, 28, 28), (10000,))\n"
     ]
    }
   ],
   "source": [
    "train_size = 200000\n",
    "valid_size = 10000\n",
    "\n",
    "valid_dataset = train_dataset[:valid_size,:,:]\n",
    "valid_labels = train_labels[:valid_size]\n",
    "train_dataset = train_dataset[valid_size:valid_size+train_size,:,:]\n",
    "train_labels = train_labels[valid_size:valid_size+train_size]\n",
    "print('Training', train_dataset.shape, train_labels.shape)\n",
    "print('Validation', valid_dataset.shape, valid_labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tIQJaJuwg5Hw"
   },
   "source": [
    "Finally, let's save the data for later reuse:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": true,
    "id": "QiR_rETzem6C"
   },
   "outputs": [],
   "source": [
    "pickle_file = 'notMNIST.pickle'\n",
    "\n",
    "try:\n",
    "  f = open(pickle_file, 'wb')\n",
    "  save = {\n",
    "    'train_dataset': train_dataset,\n",
    "    'train_labels': train_labels,\n",
    "    'valid_dataset': valid_dataset,\n",
    "    'valid_labels': valid_labels,\n",
    "    'test_dataset': test_dataset,\n",
    "    'test_labels': test_labels,\n",
    "    }\n",
    "  pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)\n",
    "  f.close()\n",
    "except Exception as e:\n",
    "  print('Unable to save data to', pickle_file, ':', e)\n",
    "  raise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 413065,
     "status": "ok",
     "timestamp": 1444485899688,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "hQbLjrW_iT39",
    "outputId": "b440efc6-5ee1-4cbc-d02d-93db44ebd956"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Compressed pickle size:', 718193801)\n"
     ]
    }
   ],
   "source": [
    "statinfo = os.stat(pickle_file)\n",
    "print('Compressed pickle size:', statinfo.st_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gE_cRAQB33lk"
   },
   "source": [
    "---\n",
    "Problem 5\n",
    "---------\n",
    "\n",
    "By construction, this dataset might contain a lot of overlapping samples, including training data that's also contained in the validation and test set! Overlap between training and test can skew the results if you expect to use your model in an environment where there is never an overlap, but are actually ok if you expect to see training samples recur when you use it.\n",
    "Measure how much overlap there is between training, validation and test samples.\n",
    "\n",
    "Optional questions:\n",
    "- What about near duplicates between datasets? (images that are almost identical)\n",
    "- Create a sanitized validation and test set, and compare your accuracy on those in subsequent assignments.\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data shape: (200000, 28, 28)\n",
      "Test data shape: (18724, 28, 28)\n",
      "Cross Validation data shape: (10000, 28, 28)\n",
      "I have removed: 12365 images from the train data\n",
      "I have removed: 2428 images from the test data\n",
      "I have removed: 1100 images from the cross validation data\n",
      "Training data shape: (187635, 28, 28)\n",
      "Test data shape: (16296, 28, 28)\n",
      "Cross Validation data shape: (8900, 28, 28)\n"
     ]
    }
   ],
   "source": [
    "train_dataset = load[\"train_dataset\"]\n",
    "test_dataset = load[\"test_dataset\"]\n",
    "valid_dataset = load[\"valid_dataset\"]\n",
    "\n",
    "print 'Training data shape: ' + str(train_dataset.shape)\n",
    "print 'Test data shape: ' + str(test_dataset.shape)\n",
    "print 'Cross Validation data shape: ' + str(valid_dataset.shape)\n",
    "\n",
    "train_dataset.flags.writeable = False\n",
    "test_dataset.flags.writeable = False\n",
    "valid_dataset.flags.writeable = False\n",
    "\n",
    "dup_train_indices = []\n",
    "dup_test_indices = []\n",
    "dup_valid_indices = []\n",
    "\n",
    "dup_table={}\n",
    "\n",
    "dup_train = 0\n",
    "dup_test = 0\n",
    "dup_valid = 0\n",
    "\n",
    "for idx,img in enumerate(train_dataset):\n",
    "    h = hash(img.data)\n",
    "    if h in dup_table and (train_dataset[dup_table[h]].data == img.data):\n",
    "       dup_train_indices.append(idx)\n",
    "       \n",
    "    dup_table[h] = idx\n",
    "\n",
    "for idx,img in enumerate(test_dataset):\n",
    "    h = hash(img.data)\n",
    "    if h in dup_table and (train_dataset[dup_table[h]].data == img.data):\n",
    "        dup_test_indices.append(idx)\n",
    "        \n",
    "for idx,img in enumerate(valid_dataset):\n",
    "    h = hash(img.data)\n",
    "    if h in dup_table and (train_dataset[dup_table[h]].data == img.data):\n",
    "        dup_valid_indices.append(idx)\n",
    "\n",
    "train_dataset.flags.writeable = True\n",
    "test_dataset.flags.writeable = True\n",
    "valid_dataset.flags.writeable = True\n",
    "\n",
    "load[\"train_dataset\"] = np.delete(train_dataset, dup_train_indices, 0)\n",
    "load[\"train_labels\"] = np.delete(load[\"train_labels\"], dup_train_indices, 0)\n",
    "\n",
    "load[\"test_dataset\"] = np.delete(test_dataset, dup_test_indices, 0)\n",
    "load[\"test_labels\"] = np.delete(load[\"test_labels\"], dup_test_indices, 0)\n",
    "\n",
    "load[\"valid_dataset\"] = np.delete(valid_dataset, dup_valid_indices, 0)\n",
    "load[\"valid_labels\"] = np.delete([\"valid_labels\"], dup_valid_indices, 0)\n",
    "    \n",
    "del train_dataset\n",
    "del test_dataset\n",
    "del valid_dataset\n",
    "\n",
    "print 'I have removed: ' + str(len(dup_train_indices)) + ' images from the train data'\n",
    "print 'I have removed: ' + str(len(dup_test_indices)) + ' images from the test data'\n",
    "print 'I have removed: ' + str(len(dup_valid_indices)) + ' images from the cross validation data'\n",
    "\n",
    "\n",
    "print 'Training data shape: ' + str(load[\"train_dataset\"].shape)\n",
    "print 'Test data shape: ' + str(load[\"test_dataset\"].shape)\n",
    "print 'Cross Validation data shape: ' + str(load[\"valid_dataset\"].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "L8oww1s4JMQx"
   },
   "source": [
    "---\n",
    "Problem 6\n",
    "---------\n",
    "\n",
    "Let's get an idea of what an off-the-shelf classifier can give you on this data. It's always good to check that there is something to learn, and that it's a problem that is not so trivial that a canned solution solves it.\n",
    "\n",
    "Train a simple model on this data using 50, 100, 1000 and 5000 training samples. Hint: you can use the LogisticRegression model from sklearn.linear_model.\n",
    "\n",
    "Optional question: train an off-the-shelf model on all the data!\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=100.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=15, multi_class='ovr',\n",
       "          penalty='l2', random_state=None, solver='lbfgs', tol=0.0001,\n",
       "          verbose=0)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logistic_classifier = LogisticRegression(C=100.0, solver='lbfgs', max_iter=25)\n",
    "logistic_classifier.fit(load[\"train_dataset\"].reshape((load[\"train_dataset\"].shape[0], 784)), load[\"train_labels\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic regression using raw pixel features:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.90      0.88      0.89      1648\n",
      "          1       0.90      0.86      0.88      1658\n",
      "          2       0.88      0.92      0.90      1642\n",
      "          3       0.90      0.91      0.91      1647\n",
      "          4       0.89      0.84      0.86      1673\n",
      "          5       0.86      0.92      0.89      1664\n",
      "          6       0.90      0.87      0.88      1681\n",
      "          7       0.91      0.86      0.88      1629\n",
      "          8       0.82      0.81      0.81      1385\n",
      "          9       0.84      0.91      0.88      1669\n",
      "\n",
      "avg / total       0.88      0.88      0.88     16296\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print(\"Logistic regression using raw pixel features:\\n%s\\n\" % (\n",
    "    metrics.classification_report(\n",
    "        load[\"test_labels\"],\n",
    "        logistic_classifier.predict(load[\"test_dataset\"].reshape((load[\"test_dataset\"].shape[0], 784)))\n",
    "        ))\n",
    "      )"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "1_notmnist.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}