code/bonus/reading_mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Sebastian Raschka](http://sebastianraschka.com), 2015\n",
    "\n",
    "https://github.com/rasbt/python-machine-learning-book"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka 12/29/2015 \n",
      "\n",
      "CPython 3.5.1\n",
      "IPython 4.0.1\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -a 'Sebastian Raschka' -v -d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# to install watermark just uncomment the following line:\n",
    "#%install_ext https://raw.githubusercontent.com/rasbt/watermark/master/watermark.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bonus Material - Reading MNIST into NumPy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, I provide some instructions for reading in the MNIST dataset of handwritten digits into NumPy arrays.\n",
    "The dataset consists of the following files:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Training set images: train-images-idx3-ubyte.gz (9.9 MB, 47 MB unzipped, 60,000 samples)\n",
    "- Training set labels: train-labels-idx1-ubyte.gz (29 KB, 60 KB unzipped, 60,000 labels) \n",
    "- Test set images: t10k-images-idx3-ubyte.gz (1.6 MB, 7.8 MB, 10,000 samples)\n",
    "- Test set labels: t10k-labels-idx1-ubyte.gz (5 KB, 10 KB unzipped, 10,000 labels) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset source: [http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After downloading the files, I recommend to unzip the files using the Unix/Linux gzip tool from the terminal for efficiency, e.g., using the command  \n",
    "    `gzip *ubyte.gz -d`   \n",
    "in your local MNIST download directory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define a simple function to read in the training or test images and corresponding labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import struct\n",
    "import numpy as np\n",
    " \n",
    "def load_mnist(path, which='train'):\n",
    " \n",
    "    if which == 'train':\n",
    "        labels_path = os.path.join(path, 'train-labels-idx1-ubyte')\n",
    "        images_path = os.path.join(path, 'train-images-idx3-ubyte')\n",
    "    elif which == 'test':\n",
    "        labels_path = os.path.join(path, 't10k-labels-idx1-ubyte')\n",
    "        images_path = os.path.join(path, 't10k-images-idx3-ubyte')\n",
    "    else:\n",
    "        raise AttributeError('`which` must be \"train\" or \"test\"')\n",
    "        \n",
    "    with open(labels_path, 'rb') as lbpath:\n",
    "        magic, n = struct.unpack('>II', lbpath.read(8))\n",
    "        labels = np.fromfile(lbpath, dtype=np.uint8)\n",
    "\n",
    "    with open(images_path, 'rb') as imgpath:\n",
    "        magic, n, rows, cols = struct.unpack('>IIII', imgpath.read(16))\n",
    "        images = np.fromfile(imgpath, dtype=np.uint8).reshape(len(labels), 784)\n",
    " \n",
    "    return images, labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned `images` NumPy array will have the shape $n \\times m$, where $n$ is the number of samples, and $m$ is the number of features. The images in the MNIST dataset consist of $28 \\times 28$ pixels, and each pixel is represented by a grayscale intensity value. Here, we unroll the $28 \\times 28$ images into 1D row vectors, which represent the rows in our matrix; thus $m=784$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may wonder why we read in the labels in such a strange way:\n",
    "\n",
    "    magic, n = struct.unpack('>II', lbpath.read(8))\n",
    "    labels = np.fromfile(lbpath, dtype=np.int8)\n",
    "\n",
    "This is to accomodate the way the labels where stored, which is described in the excerpt from the MNIST website:\n",
    "\n",
    "<pre>[offset] [type]          [value]          [description] \n",
    "0000     32 bit integer  0x00000801(2049) magic number (MSB first) \n",
    "0004     32 bit integer  60000            number of items \n",
    "0008     unsigned byte   ??               label \n",
    "0009     unsigned byte   ??               label \n",
    "........ \n",
    "xxxx     unsigned byte   ??               label</pre>\n",
    "\n",
    "So, we first read in the \"magic number\" (describes a file format or protocol) and the \"number of items\" from the file buffer before we read the following bytes into a NumPy array using the `fromfile` method.\n",
    "\n",
    "The `fmt` parameter value `'>II'` that we passed as an argument to `struct.unpack` can be composed into:\n",
    "\n",
    "- '>': big-endian (defines the order in which a sequence of bytes is stored)\n",
    "- 'I': unsigned int"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If everything executed correctly, we should now have a label vector of $60,000$ instances, and a $60,000 \\times 784$ image feature matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labels: 60000\n",
      "Rows: 60000, columns: 784\n"
     ]
    }
   ],
   "source": [
    "X, y = load_mnist(path='/Users/Sebastian/Desktop/', which='train')\n",
    "print('Labels: %d' % y.shape[0])\n",
    "print('Rows: %d, columns: %d' % (X.shape[0], X.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check if the pixels were retrieved correctly, let us print a few images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEKCAYAAAAy4ujqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEYRJREFUeJzt3X2wXHV9x/H3J4SAEMSYkpsplKTagq0MTWGgw4TQNWBE\nagUNpUIdUVt1xEe0VgytuRc6DDKFEqrOFAwYrI4IJA3pOC1Q3bYZSWFaCGKI0akJiSSXUEIwGQgP\n+faPPUn3rvf+du/dp8P9fV4zOzl3v+fs+d7N/ew5Z8+TIgIzy8uUfjdgZr3n4JtlyME3y5CDb5Yh\nB98sQw6+WYYc/MxJ+n1JW1sc91JJ/zHB+Ux4Wus8B79kJP1M0sIez3Y8B3O0c+BHy9NK2i/pF8Xj\nOUk3tzFfazC13w3Y+Eg6JCJe6XcfPRDAyRHxs343Mhl5iV8ikm4HjgfWFEu5P5c0p1j6fVDSFuBf\nR1s9r19TUM0Vkn4qaaekb0t6XYs9fL6Y7jlJj0m6oGGUKZL+TtKzkjbUr51Ieq2kr0l6UtJWSVdL\n0kTfDvz32TV+Y0skIt4HPAG8IyJeGxF/U1c+C3gT8LYDoyde6pPAO4EFwK8Cu4CvttjGT4H5EfFa\nYAj4B0kDdfXfA34CzAQGgZV1HyorgBeBNwC/C7wV+LPRZiJpjaS/aNLLvxUfIndJmtNi/9YCB7+c\nGpeSASyNiOcjYl8L038EuDIitkfES8BVwIWSmv5/R8TdETFcDN9JLeSn140yHBE3RcQrEfEd4MfA\nH0iaBbwduDwiXoiIp4EbgYvHmM8fRsR1iVbOAuZS+7DbDvxTK/1ba7yN/+qxbRzjzgFWSdpf/Czg\nJWCAWojGJOl9wOXUQgdwJPArdaP8vGGSLdTWKuYAhwLbi7V7FY8nxtH3QRGxthh8TtKngN3AbwE/\nmsjr2UgOfvmMtQpf//xe4IgDP0g6BDimrv4E8MGIeGA8M5Z0PHAz8JYD00p6mJFrIMc2THY8sBrY\nCrwAzIzOn/Kphn+tTV51Kp8d1LaR6zX+wW8CDpf0dklTgb8EptXV/x64pggyko6R9M4W5n0ksB94\nWtIUSR8ATmoYZ0DSJyRNlfRH1FbFvxsRO4B7gb+VdFTxBeMbJJ3VwnxH/rLSb0v6naKH6cAN1NZ4\nHh/va9noHPzyuRb4K0nPSPpM8dyIJWhEPAdcBiynFohfMHJTYBm1pfC9knYDP2DkdvqoIuJx4Hpg\nHbUPoDcDaxtGWwf8JvA0cDWwOCJ2FbX3UfsA2gA8A9wJzB5tXpK+K+mKMVoZAO6gtnr/U+DXqH3h\nmcNuzJ6QL8Rhlh8v8c0y5OCbZcjBN8tQW8GXdK6kjZI2Sfp8p5oys+6a8Jd7xVFUm4CzgSeBh4D3\nRMTGhvH87aFZn0TEqMc+tLPEPx34SURsKQ4L/TZw/hgzP/hYunTpiJ/L9nB/k7e/MvfWjf5S2gn+\nsdSO1jpgG798VJeZlZC/3DPLUDvH6v+c2nHaBxzHL5/AAcDg4ODB4de9rqXTwvumUqn0u4Uk9zdx\nZe4N2u+vWq1SrVZbGredL/cOoXZK5tnUzvh6ELg4aod91o8XE52HmU2cJGKML/cmvMSPiFckfZza\niRlTgOWNoTezcur6sfpe4pv1R2qJ7y/3zDLk4JtlyME3y5CDb5YhB98sQw6+WYYcfLMMOfhmGXLw\nzTLk4JtlyME3y5CDb5YhB98sQw6+WYYcfLMMOfhmGXLwzTLk4JtlyME3y5CDb5YhB98sQw6+WYYc\nfLMMOfhmGXLwzTLk4JtlyME3y5CDb5YhB98sQw6+WYYcfLMMTW1nYkmbgd3AfuCliDi9E01Z5+zf\nvz9Z37dvX1fnv2LFimR97969yfqGDRuS9RtvvDFZX7JkSbL+5S9/OVl/zWtek6xff/31yfpHP/rR\nZL1f2go+tcBXImJXJ5oxs95od1VfHXgNM+uxdkMbwH2SHpL0oU40ZGbd1+6q/vyI2C7pGGofAI9H\nxNrGkQYHBw8OVyoVKpVKm7M1s0bVapVqtdrSuG0FPyK2F//ulLQKOB1IBt/MuqNxoTo0NDTmuBNe\n1Zd0hKTpxfCRwCLgsYm+npn1TjtL/AFglaQoXuebEXFvZ9oys26acPAj4mfAvA72Mint3r07WX/l\nlVeS9fXr1yfr996b/qx99tlnk/Wbb745We+3uXPnJuuf/exnk/Xly5cn60cffXSyvmDBgmR94cKF\nyXpZeVecWYYcfLMMOfhmGXLwzTLk4JtlyME3y5CDb5YhRUR3ZyBFt+fRT9u2bUvW581LH+qwa1fe\nZzRPmZJe9tx3333JerPz5ZuZNWtWsj59+vRk/Zhjjmlr/t0kiYjQaDUv8c0y5OCbZcjBN8uQg2+W\nIQffLEMOvlmGHHyzDLV7zb3szZw5M1kfGBhI1su+H3/RokXJerPff+XKlcn6YYcdlqz7+ozd4SW+\nWYYcfLMMOfhmGXLwzTLk4JtlyME3y5CDb5Yh78dvU7Pzwb/+9a8n63fddVeyfsYZZyTrixcvTtab\nOfPMM5P11atXJ+vTpk1L1nfs2JGsL1u2LFm37vAS3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfL\nUNPr6ktaDrwDGI6Ik4vnZgB3AHOAzcBFETHqjeAn+3X127Vv375kvdl+8iVLliTr1113XbL+/e9/\nP1k/66yzknUrr3avq38b8LaG564A7o+IE4HvAV9or0Uz66WmwY+ItUDjZWLOB1YUwyuACzrcl5l1\n0US38WdFxDBAROwA0vchMrNS6dSx+smN+MHBwYPDlUrF11Ez64JqtUq1Wm1p3IkGf1jSQEQMS5oN\nPJUauT74ZtYdjQvVoaGhMcdtdVVfxeOAe4D3F8OXAulTuMysVJoGX9K3gB8AJ0h6QtIHgGuBt0r6\nMXB28bOZvUo0XdWPiEvGKJ3T4V6y1Oy68s3MmDGjrelvuummZH3BggXJujTqbmIrOR+5Z5YhB98s\nQw6+WYYcfLMMOfhmGXLwzTLk4JtlqOn5+G3PwOfjd9WLL76YrF9yyViHYdSsWrUqWV+/fn2yftJJ\nJyXr1j/tno9vZpOMg2+WIQffLEMOvlmGHHyzDDn4Zhly8M0y5P34k9wzzzyTrL/xjW9M1l//+tcn\n6xdckL7A8vz585P1d73rXcm6z/efOO/HN7MRHHyzDDn4Zhly8M0y5OCbZcjBN8uQg2+WIe/Hz9yD\nDz6YrJ977rnJ+u7du9ua/6233pqsL168OFmfPn16W/OfzLwf38xGcPDNMuTgm2XIwTfLkINvliEH\n3yxDDr5Zhprux5e0HHgHMBwRJxfPLQU+BDxVjLYkIv55jOm9H/9VbPv27cn65Zdfnqzfeeedbc3/\nyiuvTNY/97nPJetHHXVUW/N/NWt3P/5twNtGef6GiDileIwaejMrp6bBj4i1wK5RSr40itmrVDvb\n+B+X9Iikr0k6umMdmVnXTZ3gdF8FroqIkPTXwA3An4418uDg4MHhSqVCpVKZ4GzNbCzVapVqtdrS\nuBMKfkTsrPvxFmBNavz64JtZdzQuVIeGhsYct9VVfVG3TS9pdl3t3cBj4+rQzPqq6RJf0reACjBT\n0hPAUuAtkuYB+4HNwEe62KOZdZjPx7e2vPDCC8n6unXrkvVzzjknWW/2t3PhhRcm63fccUeyPpn5\nfHwzG8HBN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhnyfnzrq8MOOyxZf/nll5P1qVPTx6A9+uijyfqJ\nJ56YrL+aeT++mY3g4JtlyME3y5CDb5YhB98sQw6+WYYcfLMMTfSae5aJJ598MllfuXJlsv7AAw8k\n68320zdz2mmnJesnnHBCW68/WXmJb5YhB98sQw6+WYYcfLMMOfhmGXLwzTLk4JtlyPvxJ7mdO3cm\n61/5yleS9dtuuy1Z37Zt27h7Go9DDjkkWZ87d26yLvmmzqPxEt8sQw6+WYYcfLMMOfhmGXLwzTLk\n4JtlyME3y1DT/fiSjgNuBwaA/cAtEXGTpBnAHcAcYDNwUUTs7mKvWdqzZ0+yvmbNmmT9qquuStY3\nbdo07p46aeHChcn6tddem6yfeuqpnWwnG60s8V8GPhMRbwbOAD4m6U3AFcD9EXEi8D3gC91r08w6\nqWnwI2JHRDxSDO8BHgeOA84HVhSjrQAu6FaTZtZZ49rGlzQXmAesAwYiYhhqHw7ArE43Z2bd0fKx\n+pKmA3cBn4qIPZIab4g35g3yBgcHDw5XKhUqlcr4ujSzpqrVKtVqtaVxWwq+pKnUQv+NiFhdPD0s\naSAihiXNBp4aa/r64JtZdzQuVIeGhsYct9VV/VuBDRGxrO65e4D3F8OXAqsbJzKzcmpld9584E+A\nH0p6mNoq/RLgS8B3JH0Q2AJc1M1Gzaxz1O1710uKbs+jzPbu3Zusb926NVl/73vfm6w//PDD4+6p\nkxYtWpSsp1Y3ofl18X0+/cRJIiJGfQN95J5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLkINvliFfV7+J\n559/Pln/9Kc/nayvXbs2Wd+4ceO4e+qk8847L1n/4he/mKzPmzcvWT/00EPH3ZN1n5f4Zhly8M0y\n5OCbZcjBN8uQg2+WIQffLEMOvlmGJv1+/M2bNyfr11xzTbJ+//33J+tbtmwZb0sddcQRRyTrV199\ndbJ+2WWXJevTpk0bd09Wfl7im2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZmvT78e++++5kffny\n5V2d/ymnnJKsX3zxxcn61Knp/6IPf/jDyfrhhx+erFuevMQ3y5CDb5YhB98sQw6+WYYcfLMMOfhm\nGWoafEnHSfqepB9J+qGkTxTPL5W0TdJ/F49zu9+umXWCmt27XtJsYHZEPCJpOvBfwPnAHwO/iIgb\nmkwfzeZhZp0niYjQaLWmB/BExA5gRzG8R9LjwLEHXrtjXZpZz4xrG1/SXGAe8J/FUx+X9Iikr0k6\nusO9mVmXtBz8YjX/LuBTEbEH+CrwhoiYR22NILnKb2bl0dKx+pKmUgv9NyJiNUBE7Kwb5RZgzVjT\nDw4OHhyuVCpUKpUJtGpmKdVqlWq12tK4Tb/cA5B0O/B0RHym7rnZxfY/ki4HTouIS0aZ1l/umfVB\n6su9Vr7Vnw/8O/BDIIrHEuASatv7+4HNwEciYniU6R18sz5oK/gdmLmDb9YHqeD7yD2zDDn4Zhly\n8M0y5OCbZcjBN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhly8M0y5OCbZajnwW/1fOF+cX/tKXN/Ze4N\netufg9/A/bWnzP2VuTeY5ME3s/5z8M0y1JMLcXR1BmY2pr5dgcfMyser+mYZcvDNMtSz4Es6V9JG\nSZskfb5X822VpM2S1kt6WNKDJehnuaRhSY/WPTdD0r2SfizpX/p596Ix+ivNjVRHudnrJ4vnS/Ee\n9vtmtD3Zxpc0BdgEnA08CTwEvCciNnZ95i2S9D/AqRGxq9+9AEg6E9gD3B4RJxfPfQn434i4rvjw\nnBERV5Sov6W0cCPVXkjc7PUDlOA9bPdmtO3q1RL/dOAnEbElIl4Cvk3tlywTUaJNn4hYCzR+CJ0P\nrCiGVwAX9LSpOmP0ByW5kWpE7IiIR4rhPcDjwHGU5D0co7+e3Yy2V3/oxwJb637exv//kmURwH2S\nHpL0oX43M4ZZB25aUtzFaFaf+xlN6W6kWnez13XAQNnew37cjLY0S7gSmB8RpwDnAR8rVmXLrmz7\nYkt3I9VRbvba+J719T3s181oexX8nwPH1/18XPFcaUTE9uLfncAqapsnZTMsaQAObiM+1ed+RoiI\nnXW3TboFOK2f/Yx2s1dK9B6OdTPaXryHvQr+Q8BvSJojaRrwHuCeHs27KUlHFJ+8SDoSWAQ81t+u\ngNq2Xv323j3A+4vhS4HVjRP02Ij+iiAd8G76/x7eCmyIiGV1z5XpPfyl/nr1HvbsyL1it8Qyah82\nyyPi2p7MuAWSfp3aUj6o3Tr8m/3uT9K3gAowExgGlgL/CNwJ/BqwBbgoIp4tUX9voYUbqfaov7Fu\n9vog8B36/B62ezPatufvQ3bN8uMv98wy5OCbZcjBN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhn6PyWk\n17mut7UnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1049b7f98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEKCAYAAAAy4ujqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEe9JREFUeJzt3X2wXHV9x/H3JyAgCYSQmmQIIaAUdewQapXaAWQto4Zi\nBeShURTElHHkcXAQEMrkJmJHHCFNVByI6CTiM0UDM05ByqzAYCotQSEBo5Y8Sm6SmhDCFJqYb//Y\nk7h3ufe3m7t3d0/y+7xm7mTv+Z6z55tNPnsefmf3KCIws7yM6nUDZtZ9Dr5Zhhx8sww5+GYZcvDN\nMuTgm2XIwc+cpFMlrWlx3oskPTrM9Qx7WRt5Dn7JSHpe0t92ebV7cjFHOxd+tLyspBMk/aeklyU9\nIWlaG+u1Bg7+XkbSfr3uodMkvQ74MbAIOKz4c7Gk/Xva2D7EwS8RSYuAo4D7JW2VdI2kqZJ2SvqE\npFXAvw+2e16/p6Ca6yX9VtJGSd+TdFiLPVxXLLdV0jOSzmqYZZSkL0vaIml5/d6JpEMlfV3S7yWt\nkfQ5SRrGS1EB9ouI+RGxPSK+DAjo9p7QPsvBL5GIuBBYDXwgIg6NiC/Vld8NvAV4/67ZE091JfBB\n4BTgCGAzcHuLbfwWOCkiDgVmA3dLmlhX/2vgN8B4oA+4t+5NZSHwf8Abgb8E3gv842ArkXS/pGuH\n6OFtwK8apv2ymG4jwMEvp8atZACzIuJ/I+LVFpb/JHBjRLwQEduBOcC5kpr+e0fEv0ZEf/H4h9RC\nfmLdLP3FlviPEfED4NfAGZImAKcDV0fEKxGxCfgX4MNDrOfvI+KLQ7QxBnixYdpW4JBm/VtrfMy0\n91i7B/NOBX4kaWfxu4DtwETghdSCki4ErgaOLiaNBv6sbpZ1DYusorZXMRV4HfBCsXev4mf1HvS9\nyzbg0IZpY4GXhvFcNghv8ctnqF34+ukvAwfv+qU44feGuvpq4PSIOLz4GRcRoyOiWeiPAu4ELi2W\nGQcsY+AeyOSGxY4Cfg+sAV4Bxtet87CIOD61ziEsAxqXO76YbiPAwS+f9dSOkes17vqvAA6SdHpx\npvufgAPq6ncA/1wEGUlvkPTBFtY9GtgJbJI0StLFwF80zDNR0hWS9pd0HrXzDj+JiPXAg8BcSYcU\nJxjfKOndLay3URX4Y7GeAyRdWfT18DCeywbh4JfPF4CbJP1B0qeLaQP2AiJiK3ApcBe1Q4CXGHgo\nMA9YDDwo6UXgcQYepw8qIp4FbgWWUHsDehvwWMNsS4A/BzYBnwPOiYjNRe1Cam9Ay4E/AD8EJg22\nLkk/kXT9EH1sB84CLqJ2YvJC4MyI2NHs72Ctkb+Iwyw/3uKbZcjBN8uQg2+WobaCL2m6pOckrZB0\n3Ug1ZWadNeyTe8VVYCuA06iN4z4BzIiI5xrm89lDsx6JiEE/K9HOFv9E4DcRsaoYfvkecOYQK9/9\nM2vWrAG/l+3H/e27/ZW5t070l9JO8CdTu1prl7W89qouMyshn9wzy1A7H9JZR+067V2O5LUf4ACg\nr69v9+PDDmvpY+E9U6lUet1CkvsbvjL3Bu33V61WqVarLc3bzsm9/ah9JPM0ap/4+gXw4ahd9lk/\nXwx3HWY2fJKIIU7uDXuLHxF/lHQ5tQ9mjALuagy9mZVTx6/V9xbfrDdSW3yf3DPLkINvliEH3yxD\nDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfL\nkINvliEH3yxDDr5Zhhx8sww5+GYZcvDNMtTOLbTMWLNmTbI+b968ZH3u3LnJ+tVXX52sX3XVVcn6\nlClTkvVceYtvliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2VI7dy7XtJK4EVgJ7A9Ik4cZJ5oZx3W\nW+vWrUvWp02blqxv2bJlJNt5jXHjxiXrGzdu7Oj6y0wSEaHBau1ewLMTqETE5jafx8y6qN1dfY3A\nc5hZl7Ub2gB+KukJSZeMRENm1nnt7uqfFBEvSHoDtTeAZyPiscaZ+vr6dj+uVCpUKpU2V2tmjarV\nKtVqtaV52zq5N+CJpFnASxFxW8N0n9zbi/nk3t4rdXJv2Lv6kg6WNKZ4PBp4H/DMcJ/PzLqnnV39\nicCPJEXxPN+OiAdHpi0z66QR29UfcgXe1S+1VatWJevNzsesXr06WZcG3dPcbezYscn6gQcemKxv\n2LAhWV+xYkWyPnXq1GR9v/32S9bLrCO7+ma293LwzTLk4JtlyME3y5CDb5YhB98sQw6+WYY8jr+X\n2759e7LebJx++vTpyfrKlSuT9Wb/ts3G8U899dRk/fOf/3yyfvLJJyfrzfq78847k/WZM2cm62Xm\ncXwzG8DBN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhlq9zv3rMc+85nPJOtf+cpXutTJ8PzsZz9L1l9+\n+eVk/eyzz07W77333mR96dKlyfq+ylt8sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDHscvuTVr\n1iTrd999d7Le7nchNBsnP+ecc5L1j370o8n6lClTkvW3vvWtyfp1112XrN9zzz3Jeq7fFeEtvlmG\nHHyzDDn4Zhly8M0y5OCbZcjBN8uQg2+Woabfqy/pLuADQH9EHF9MGwd8H5gKrATOj4gXh1je36uf\nsG7dumR92rRpyfqWLVvaWv8FF1yQrC9YsCBZX758ebL+5JNPJuszZsxI1g8++OBkvZlm97cfPXp0\nsr5s2bJkvdl1CL3U7vfqfxN4f8O064GHIuLNwMPAZ9tr0cy6qWnwI+IxYHPD5DOBhcXjhcBZI9yX\nmXXQcI/xJ0REP0BErAcmjFxLZtZpI3WtfvIgvq+vb/fjSqVCpVIZodWa2S7VapVqtdrSvMMNfr+k\niRHRL2kSsCE1c33wzawzGjeqs2fPHnLeVnf1Vfzsch/w8eLxRcDiPWnQzHqrafAlfQd4HDhO0mpJ\nFwNfAN4r6dfAacXvZraXaDqO3/YKMh/H37RpU7I+Z86cZP2rX/1qsj5x4sRk/ZhjjknWb7311mT9\nXe96V7Jeds3G8aVBh7l3u/TSS5P1+fPn73FP3dLuOL6Z7WMcfLMMOfhmGXLwzTLk4JtlyME3y5CD\nb5Yhf69+m3bs2JGsX3PNNcl6s+/FHzt2bLL+wAMPJOvHHntssr59+/ZkPXfPP/98r1voCG/xzTLk\n4JtlyME3y5CDb5YhB98sQw6+WYYcfLMMeRy/TatXr07Wm43TN7NkyZJk/bjjjmvr+V//+te3tbzt\nnbzFN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhly8M0y5HH8Nl122WXJerN7Cpx99tnJervj9LnbuXNn\nsj5qVHrbt6/eE8JbfLMMOfhmGXLwzTLk4JtlyME3y5CDb5YhB98sQ03H8SXdBXwA6I+I44tps4BL\ngA3FbDdExL91rMseWrp0abL+yCOPJOvN7r9+3nnn7XFP1rpm4/TN/n3e8Y53jGQ7pdHKFv+bwPsH\nmX5bRLy9+NknQ2+2r2oa/Ih4DNg8SCn9VmlmpdXOMf7lkp6S9HVJ6fs8mVmpDPda/duBORERkm4G\nbgNmDjVzX1/f7seVSoVKpTLM1ZrZUKrVKtVqtaV5hxX8iNhY9+sC4P7U/PXBN7POaNyozp49e8h5\nW93VF3XH9JIm1dU+BDyzRx2aWU+1Mpz3HaACjJe0GpgFvEfSCcBOYCXwyQ72aGYjrGnwI+Ijg0z+\nZgd6KaVXXnklWX/11VeT9SOOOCJZP+OMM/a4p5zs2LEjWZ8/f35bz3/uuecm6zfccENbz19WvnLP\nLEMOvlmGHHyzDDn4Zhly8M0y5OCbZcjBN8uQv1e/ww466KBkfcyYMV3qpJyajdN/7WtfS9avvfba\nZP3oo49O1m+88cZk/YADDkjW91be4ptlyME3y5CDb5YhB98sQw6+WYYcfLMMOfhmGfI4fod97GMf\n63ULPbVu3bpk/ZZbbknWb7/99mT94osvTtYXLFiQrOfKW3yzDDn4Zhly8M0y5OCbZcjBN8uQg2+W\nIQffLEOKiM6uQIpOr6OTHn/88WT9lFNOSdabfR78d7/73Z62VCrf/e53k/UrrrgiWd+8ebAbMf/J\nlVdemazPnTs3Wc+ZJCJi0Ltae4tvliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2Wo6efxJR0JLAIm\nAjuBBRExX9I44PvAVGAlcH5EvNjBXntCGnQYtOX62rVrk/U5c+Yk6zNnzkzWDznkkGR92bJlyfod\nd9yRrD/66KPJ+sqVK5P1N73pTcn6jBkzkvVm4/g2PK1s8XcAn46ItwF/A1wm6S3A9cBDEfFm4GHg\ns51r08xGUtPgR8T6iHiqeLwNeBY4EjgTWFjMthA4q1NNmtnI2qNjfElHAycAS4CJEdEPtTcHYMJI\nN2dmndHyd+5JGgPcA1wVEdskNV6AP+QF+X19fbsfVyoVKpXKnnVpZk1Vq1Wq1WpL87YUfEn7Uwv9\ntyJicTG5X9LEiOiXNAnYMNTy9cE3s85o3KjOnj17yHlb3dX/BrA8IubVTbsP+Hjx+CJgceNCZlZO\nrQznnQRcADwtaSm1XfobgFuAH0j6BLAKOL+TjZrZyPHn8Zv4+c9/nqw3+zx+uyZPnpysH3744cn6\n008/PZLtvMb06dPbql9++eUj2Y7V8efxzWwAB98sQw6+WYYcfLMMOfhmGXLwzTLk4JtlyOP4TWzd\nujVZP//89HVLDz30UFvrb/baNfs+gGYmTEh/tupTn/pUsn7TTTe1tX7rHI/jm9kADr5Zhhx8sww5\n+GYZcvDNMuTgm2XIwTfLkMfx27Rt27ZkfdGiRcl6s++Nb3cc/+abb07WL7nkkmR9/PjxybqVl8fx\nzWwAB98sQw6+WYYcfLMMOfhmGXLwzTLk4JtlyOP4Zvsoj+Ob2QAOvlmGHHyzDDn4Zhly8M0y5OCb\nZahp8CUdKelhScskPS3pimL6LElrJT1Z/KTvh2xmpdF0HF/SJGBSRDwlaQzwX8CZwD8AL0XEbU2W\n9zi+WQ+kxvH3b7ZwRKwH1hePt0l6Fpi867lHrEsz65o9OsaXdDRwAvAfxaTLJT0l6euSxo5wb2bW\nIS0Hv9jNvwe4KiK2AbcDb4yIE6jtESR3+c2sPJru6gNI2p9a6L8VEYsBImJj3SwLgPuHWr6vr2/3\n40qlQqVSGUarZpZSrVapVqstzdvSh3QkLQI2RcSn66ZNKo7/kXQ18M6I+Mggy/rknlkPpE7utXJW\n/yTgEeBpIIqfG4CPUDve3wmsBD4ZEf2DLO/gm/VAW8EfgZU7+GY94I/lmtkADr5Zhhx8sww5+GYZ\ncvDNMuTgm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZ6nrwW/28cK+4v/aUub8y9wbd7c/Bb+D+\n2lPm/srcG+zjwTez3nPwzTLUlS/i6OgKzGxIPfsGHjMrH+/qm2XIwTfLUNeCL2m6pOckrZB0XbfW\n2ypJKyX9UtJSSb8oQT93SeqX9Ku6aeMkPSjp15Ie6OXdi4borzQ3Uh3kZq9XFtNL8Rr2+ma0XTnG\nlzQKWAGcBvweeAKYERHPdXzlLZL038BfRcTmXvcCIOlkYBuwKCKOL6bdAvxPRHyxePMcFxHXl6i/\nWbRwI9VuSNzs9WJK8Bq2ezPadnVri38i8JuIWBUR24HvUftLloko0aFPRDwGNL4JnQksLB4vBM7q\nalN1hugPSnIj1YhYHxFPFY+3Ac8CR1KS13CI/rp2M9pu/UefDKyp+30tf/pLlkUAP5X0hKRLet3M\nECbsumlJcRejCT3uZzClu5Fq3c1elwATy/Ya9uJmtKXZwpXASRHxduDvgMuKXdmyK9tYbOlupDrI\nzV4bX7Oevoa9uhltt4K/Djiq7vcji2mlEREvFH9uBH5E7fCkbPolTYTdx4gbetzPABGxse62SQuA\nd/ayn8Fu9kqJXsOhbkbbjdewW8F/AjhW0lRJBwAzgPu6tO6mJB1cvPMiaTTwPuCZ3nYF1I716o/3\n7gM+Xjy+CFjcuECXDeivCNIuH6L3r+E3gOURMa9uWplew9f0163XsGtX7hXDEvOovdncFRFf6MqK\nWyDpGGpb+aB26/Bv97o/Sd8BKsB4oB+YBfwY+CEwBVgFnB8RW0rU33to4UaqXepvqJu9/gL4AT1+\nDdu9GW3b6/clu2b58ck9sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDDr5Zhv4f/WguQqhZ1JwA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b134358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEKCAYAAAAy4ujqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEG5JREFUeJzt3X2sXHVex/H3ByohbWXB2odIeWpQaNbUupstMQgOWyis\nW4GAIlsTClUCERYCIfKk6e2uEXajKCVLEOhuyrqEh1UsEKLdpRkUl4eKdG1tS7uRFmjpBZby0OJC\n7f36x5xbp8OdM3Pn8dz7+7ySCWfO95w53zvlM+dx5igiMLO0HNLvBsys9xx8swQ5+GYJcvDNEuTg\nmyXIwTdLkIOfOEm/Jen1JqddLOlfW1xOy/Na5zn4BSPpVUlf7PFiR3MxRzsXfox6XkmXSBqStKSN\n5VoNB3+MkXRov3voFUlHAjcDG/rdy3jj4BeIpAeAY4EnJH0g6QZJxw2v8SRtB54eafO8ektBFTdJ\n+omktyU9lIWomR5uzOb7QNIGSefXTHKIpLskvSdpY/XWiaQjJN0vaaek1yV9XZLaeEtuA+4EftrG\na9gIHPwCiYhLgNeAhRFxRET8ZVX5dOBk4OzhyXNe6hrgXOA04JeA3cDdTbbxE+DUiDgCWAb8naTp\nVfVTgK3AFGAA+IeqD5WVwCfALODXgbOAPxppIZKekPQn9ZqQNA/4fETc02TfNgoOfjHVriUDWBoR\n/xMRHzcx/xXArRHxZkTsA74G/K6khv/eEfH3ETGYDT9KJeTzqiYZjIjlEbE/Ih4BXgG+LGka8CXg\nuoj4WUS8A/wN8JU6y/mdiPjmSLWsz28BVzXxt1oLJvS7AWvaG6OY9jjgMUlD2XMB+4DpwJt5M0q6\nBLgOOD4bNQn4xapJdtTMsp3KVsVxwM8Bb2Zb98oer42i72FXAT+OiLUtzGtNcPCLp94mfPX4vcDE\n4SfZAb+pVfXXgCUR8dxoFizpWOBe4IzheSW9zMFbIEfXzHYssAp4HfgZMCXa/8rnF4HTJX05e/4L\nwFxJcyPimjZf2/CmfhHtorKPXK12038LcLikL0maAPwpcFhV/W+Bv8iCjKSpks5tYtmTgCHgHUmH\nSLoM+NWaaaZL+qqkCZJ+j8pxh6ciYhewGvhrST+fHWCcJen0JpZbazEwG/i17PHvVI433NrCa9kI\nHPziuR34M0nvSro+G3fQGjQiPgD+GFhBZRfgQw7eFbiTylp4taT3gR9x8H76iCJiE/BXwPNUPoA+\nCzxbM9nzwC8D7wBfBy6MiN1Z7RIqH0AbgXeBR4EZIy1L0lOSbqrTxwcR8dbwA/gY+CAiPmz0N1hz\n5B/iMEuP1/hmCXLwzRLk4JslqK3gSzpH0mZJWyTd2KmmzKy7Wj64l11dtQWYD+wE1gIXR8Tmmul8\n9NCsTyJixO9KtLPGnwdsjYjt2WWhDwHn1Vn4gcfSpUsPel60h/sbv/0Vubdu9JenneAfTeVqrWFv\n8OmrusysgHxwzyxB7Vyrv4PKddrDZvLpL3AAMDAwcGD4yCOb+lp435RKpX63kMv9ta7IvUH7/ZXL\nZcrlclPTtnNw71AqX8mcT+UbXy8CX4nKZZ/V00WryzCz1kki6hzca3mNHxH7JV1N5YsZhwArakNv\nZsXU9Wv1vcY364+8Nb4P7pklyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S5CD\nb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk\n4JslyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCZrQzsyStgHvA0PAvoiY14mmzDpl06ZNufUzzzwz\nt75u3brc+tSpU0fdUxG0FXwqgS9FxO5ONGNmvdHupr468Bpm1mPthjaAH0haK+nyTjRkZt3X7qb+\nqRHxpqSpVD4ANkXEs7UTDQwMHBgulUqUSqU2F2tmtcrlMuVyualpFREdWaikpcCHEXFHzfjo1DLM\nRivlg3uSiAiNVGt5U1/SREmTs+FJwAJgQ6uvZ2a9086m/nTgMUmRvc73ImJ1Z9oys25qOfgR8Sow\nt4O9dMXWrVtz67t355+JnDfPlyaMZS+88EJuff78+T3qpFh8Ks4sQQ6+WYIcfLMEOfhmCXLwzRLk\n4JslyME3S1C71+oX3tNPP51b37x5c27d5/GLrdHl4I2u49iyZUsn2xkzvMY3S5CDb5YgB98sQQ6+\nWYIcfLMEOfhmCXLwzRI07s/jL1++PLe+YMGCHnVi3bBnz57c+m233ZZbv/baa3PrRf5prXZ4jW+W\nIAffLEEOvlmCHHyzBDn4Zgly8M0S5OCbJWjcn8ffv39/v1uwLrryyivbmn/27Nkd6mRs8RrfLEEO\nvlmCHHyzBDn4Zgly8M0S5OCbJcjBN0tQw/P4klYAC4HBiJiTjTsKeBg4DtgGXBQR73exz7p27tyZ\nW9+xY0ePOrF+ePfdd9ua/6yzzupQJ2NLM2v87wBn14y7CfhhRJwErAFu7nRjZtY9DYMfEc8Cu2tG\nnweszIZXAud3uC8z66JW9/GnRcQgQETsAqZ1riUz67ZOXaufewOzgYGBA8OlUolSqdShxZrZsHK5\nTLlcbmraVoM/KGl6RAxKmgG8lTdxdfDNrDtqV6rLli2rO22zm/rKHsMeBy7NhhcDq0bToJn1V8Pg\nS3oQ+BHwK5Jek3QZcDtwlqRXgPnZczMbIxpu6kfEojqlMzvcS0tWr16dW//oo4961Il1w969e3Pr\n69evb+v1p0yZ0tb8Y5Wv3DNLkINvliAH3yxBDr5Zghx8swQ5+GYJcvDNEjTmf1d/w4YNbc0/d+7c\nDnVi3XDrrbfm1hv9HsOcOXNy64cddtioexoPvMY3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRI0\n5s/jt+uUU07pdwtj2scff5xbf+mll3Lr9957b2794YcfHnVP1ZYvX55bP/zww9t6/bHKa3yzBDn4\nZgly8M0S5OCbJcjBN0uQg2+WIAffLEHJn8d/7733+rr8Rt8nHxoayq0/88wzufVXX301t/7JJ5/k\n1u+6667c+v79+3PrkyZNyq0vWLAgt97oPPu+ffty67Nnz86tp8prfLMEOfhmCXLwzRLk4JslyME3\nS5CDb5YgB98sQQ3P40taASwEBiNiTjZuKXA58FY22S0R8U9d6zLHxIkTc+uScuvnnntubv2kk04a\ndU+j8dxzz+XWIyK3PmFC/j/h5MmTc+uNfo/ghhtuyK2fdtppufVG9y1odJ7/mGOOya3v3bs3tz51\n6tTceqqaWeN/Bzh7hPF3RMTnskdfQm9mrWkY/Ih4Ftg9Qil/VWpmhdXOPv7VktZJul/SZzrWkZl1\nXavX6t8NfC0iQtKfA3cAf1hv4oGBgQPDpVKJUqnU4mLNrJ5yuUy5XG5q2paCHxFvVz29D3gib/rq\n4JtZd9SuVJctW1Z32mY39UXVPr2kGVW1C4D2bllrZj3VzOm8B4ESMEXSa8BS4AxJc4EhYBtwRRd7\nNLMOU6PzxG0vQIpuLyPPypUrc+vN7hP1y6JFi3LrJ554Ym79hBNO6GQ7HffUU0/l1hcuXJhbP/nk\nk3PrGzduHHVP44UkImLEs2++cs8sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S9C4/139xYsXt1W3\n7nryySfbmn/JkiUd6iQtXuObJcjBN0uQg2+WIAffLEEOvlmCHHyzBDn4Zgka9+fxbXy74IIL+t3C\nmOQ1vlmCHHyzBDn4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEEOvlmCHHyzBDn4Zgly8M0S5OCbJcjB\nN0uQg2+WoIbfx5c0E3gAmA4MAfdFxHJJRwEPA8cB24CLIuL9LvZqCYqI3Pr27dtz67NmzepkO+NG\nM2v8/wWuj4jPAr8BXCXpZOAm4IcRcRKwBri5e22aWSc1DH5E7IqIddnwHmATMBM4D1iZTbYSOL9b\nTZpZZ41qH1/S8cBc4HlgekQMQuXDAZjW6ebMrDua/s09SZOB7wPXRsQeSbU7X3V3xgYGBg4Ml0ol\nSqXS6Lo0s4bK5TLlcrmpaZsKvqQJVEL/3YhYlY0elDQ9IgYlzQDeqjd/dfDNrDtqV6rLli2rO22z\nm/rfBjZGxJ1V4x4HLs2GFwOramcys2Jq5nTeqcAfAOslvUxlk/4W4BvAI5KWANuBi7rZqJl1TsPg\nR8S/AYfWKZ/Z2XbMDiYptz40NNSjTsYXX7lnliAH3yxBDr5Zghx8swQ5+GYJcvDNEuTgmyWo6Wv1\nzYpozZo1ufX58+f3qJOxxWt8swQ5+GYJcvDNEuTgmyXIwTdLkINvliAH3yxBPo9vhdbod/WtNV7j\nmyXIwTdLkINvliAH3yxBDr5Zghx8swQ5+GYJ8nl866sLL7wwt37PPff0qJO0eI1vliAH3yxBDr5Z\nghx8swQ5+GYJcvDNEtQw+JJmSloj6b8krZf01Wz8UklvSPqP7HFO99s1s05Qo+87S5oBzIiIdZIm\nAy8B5wG/D3wYEXc0mD/8nWqz3pNERGikWsMLeCJiF7ArG94jaRNw9PBrd6xLM+uZUe3jSzoemAu8\nkI26WtI6SfdL+kyHezOzLmk6+Nlm/veBayNiD3A3MCsi5lLZIsjd5Dez4mjqWn1JE6iE/rsRsQog\nIt6umuQ+4Il68w8MDBwYLpVKlEqlFlo1szzlcplyudzUtA0P7gFIegB4JyKurxo3I9v/R9J1wBci\nYtEI8/rgnlkf5B3ca+ao/qnAvwDrgcgetwCLqOzvDwHbgCsiYnCE+R18sz5oK/gdWLiDb9YHecH3\nlXtmCXLwzRLk4JslyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S1DPg9/s94X7\nxf21p8j9Fbk36G1/Dn4N99eeIvdX5N5gnAffzPrPwTdLUE9+iKOrCzCzuvr2CzxmVjze1DdLkINv\nlqCeBV/SOZI2S9oi6cZeLbdZkrZJ+rGklyW9WIB+VkgalPSfVeOOkrRa0iuS/rmfdy+q019hbqQ6\nws1er8nGF+I97PfNaHuyjy/pEGALMB/YCawFLo6IzV1feJMk/Tfw+YjY3e9eACT9JrAHeCAi5mTj\nvgH8NCK+mX14HhURNxWov6U0cSPVXsi52etlFOA9bPdmtO3q1Rp/HrA1IrZHxD7gISp/ZJGIAu36\nRMSzQO2H0HnAymx4JXB+T5uqUqc/KMiNVCNiV0Ssy4b3AJuAmRTkPazTX89uRtur/9GPBl6vev4G\n//9HFkUAP5C0VtLl/W6mjmnDNy3J7mI0rc/9jKRwN1Ktutnr88D0or2H/bgZbWHWcAVwakR8Dvht\n4KpsU7boinYutnA3Uh3hZq+171lf38N+3Yy2V8HfARxb9XxmNq4wIuLN7L9vA49R2T0pmkFJ0+HA\nPuJbfe7nIBHxdtVtk+4DvtDPfka62SsFeg/r3Yy2F+9hr4K/FjhR0nGSDgMuBh7v0bIbkjQx++RF\n0iRgAbChv10BlX296v29x4FLs+HFwKraGXrsoP6yIA27gP6/h98GNkbEnVXjivQefqq/Xr2HPbty\nLzstcSeVD5sVEXF7TxbcBEknUFnLB5Vbh3+v3/1JehAoAVOAQWAp8I/Ao8AxwHbgooh4r0D9nUET\nN1LtUX/1bvb6IvAIfX4P270ZbdvL9yW7ZunxwT2zBDn4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEEO\nvlmC/g9tiWfqoofyNwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b23ce80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEKCAYAAAAy4ujqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD/hJREFUeJzt3X+s3fVdx/Hn69KVhQsUUvsDWykSBpiRBjFDFmSeilsZ\nxZWgIJsJtCgBHdvShghDyb1sxnQLoMwEM9tuKQ7CAEGomdqN7kQQkWZSBVtaiGspLb1Q20Irosh9\n+8f53u7cwznfc3p+ftvP65HccO73/T3n876Hvs73x/n+UERgZmkZGnQDZtZ/Dr5Zghx8swQ5+GYJ\ncvDNEuTgmyXIwU+cpF+WtKPFea+V9FSb47T9XOs+B79gJP1Y0q/0edjDOZijkwM/Wn6upG9KeknS\n+5Ku6WBMq8PBP8JIOmbQPfTJRuB3gR8NupGjkYNfIJLuA04F1kp6W9LNkuZJGpd0naTtwJP1Vs+r\n1xRUcaukVyS9KelBSSe12MMt2fPelvSipMtrZhmS9GeS9kvaVL12IulESask7ZK0Q9JXJamd9yIi\n/jwifgj8TzvPt3wOfoFExDXAq8BlEXFiRNxZVf4EcDawcGL2nJf6IvAZ4CLgp4F9wL0ttvEKcGFE\nnAjcAXxH0qyq+i8CLwPTgVHg0aoPlTXA/wKnAz8PfBL4nXqDSFor6fdb7Mm6zMEvptqlZAAjEfHf\nEdHKEvAG4A8i4vWIeA/4CvAbkpr+/46Iv4qIsezxw1RCfn7VLGMR8Y2IeD8iHgK2AIskzQQ+DSyL\niHcjYg/wp8BnG4zzaxHx9Rb+FuuBKYNuwFr22mHMOw94TNJ49ruA94BZwOt5T8x2pC0DTssmDQM/\nVTXLzpqnbKeyVjEP+BDwerZ2r+zn1cPo2/rEwS+eRqvw1dP/Czhu4pdsh9+MqvqrwHUR8U+HM7Ck\nU4G/ABZMPFfS80xeA5lT87RTgceBHcC7wPTwKZ+F51X94tlNZRu5Wu2q/1bgw5I+LWkK8IfA1Kr6\nN4E/zoKMpBmSPtPC2MPAOLBH0pCkpcA5NfPMkvQFSVMkXUllv8P3ImI3sA74E0knZDsYT5f0iRbG\n/QBJH5L0YSp/+1RJx7a7o9A+yMEvnhXA7ZL2SlqeTZu0BI2It4HfA1ZT2QQ4wORNgXuoLIXXSXoL\neIbJ2+l1RcRm4C7gWSofQB8Fnq6Z7VngI8Ae4KvAr0fEvqx2DZUPoE3AXuBhYHa9sSR9T9KtOe2s\nA94BPk7lg+wdKjsrrQvktTKz9HiJb5YgB98sQQ6+WYI6Cr6kS7ITKbZKuqVbTZlZb7W9cy87Cmwr\ncDGwC9gAXB0RL9XM572HZgMSEXW/Au1kiX8+8HJEbM8OC30QWNxg8EM/IyMjk34v2o/7O3r7K3Jv\nvegvTyfBn0PlaK0Jr/HBo7rMrIC8c88sQZ0cq7+TynHaE+bywRM4ABgdHT30+KSTWjotfGBKpdKg\nW8jl/tpX5N6g8/7K5TLlcrmleTvZuXcMlVMyL6ZyxtdzwGejcthn9XzR7hhm1j5JRIOde20v8SPi\nfUk3UTmmeghYXRt6Myumnh+r7yW+2WDkLfG9c88sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S5CD\nb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk\n4JslyME3S5CDb5YgB98sQQ6+WYIcfLMEOfhmCXLwzRLk4JslyME3S9CUTp4saRvwFjAOvBcR53ej\nKbMJK1euzK3feOONufXx8fHc+pYtW3LrZ555Zm79SNVR8KkEvhQR+7rRjJn1R6er+urCa5hZn3Ua\n2gC+L2mDpOu70ZCZ9V6nq/oXRsTrkmZQ+QDYHBFP1840Ojp66HGpVKJUKnU4rJnVKpfLlMvlluZV\nRHRlUEkjwIGIuLtmenRrDEuPd+61TxIRoXq1tlf1JR0n6fjs8TDwKeDFdl/PzPqnk1X9WcBjkiJ7\nnfsjYl132jKzXmo7+BHxY+DcLvZiCXryySdz68uXL8+tDw11tn9aqrsmfNTzV3FmCXLwzRLk4Jsl\nyME3S5CDb5YgB98sQQ6+WYI6PVbfrCNbt27Nrb/77rt96iQtXuKbJcjBN0uQg2+WIAffLEEOvlmC\nHHyzBDn4Zgnq2qW3Gg7gS28lbdOmTbn1BQsW5Nb37t2bWz/vvPNy6+vW5V8bZnh4OLc+ZcqRe6hL\nTy69ZWZHLgffLEEOvlmCHHyzBDn4Zgly8M0S5OCbJejI/ZLSCuGVV17JrV966aW59Wbf0zezYsWK\n3Pq0adM6ev2jlZf4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEEOvlmCmn6PL2k1cBkwFhHzs2knA98F\n5gHbgKsi4q0e9mkFtWrVqtz6jh07Onr9K664Irfe7Hx+q6+VJf63gYU1024FfhARZwHrgS93uzEz\n652mwY+Ip4F9NZMXA2uyx2uAy7vcl5n1ULvb+DMjYgwgInYDM7vXkpn1WreO1c+9qN7o6Oihx6VS\niVKp1KVhzWxCuVymXC63NG+7wR+TNCsixiTNBt7Im7k6+GbWG7UL1TvuuKPhvK2u6iv7mfAEsCR7\nfC3w+OE0aGaD1TT4kh4AngHOlPSqpKXACuCTkrYAF2e/m9kRwtfVt1zvvPNObv2EE07IrQ8N5S9b\npk+fnltvts169tln59ZT5uvqm9kkDr5Zghx8swQ5+GYJcvDNEuTgmyXIwTdLkK+rn7j9+/fn1hcv\nXtzT8Zsdzu3v6XvDS3yzBDn4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEH+Hj9xTz31VG79mWee6ej1\nr7zyytz6kiVLOnp9a4+X+GYJcvDNEuTgmyXIwTdLkINvliAH3yxBDr5Zgnxd/aPchg0bcusLF9be\nAX2yAwcO5NYXLVqUW7///vtz68PDw7l1a5+vq29mkzj4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEFN\nz8eXtBq4DBiLiPnZtBHgeuCNbLbbIuLvetalNdTsuvgXXHBBT8c/44wzcuv+nr6YWlnifxuod5TH\n3RFxXvbj0JsdQZoGPyKeBvbVKdU9IsjMiq+TbfybJG2UtErStK51ZGY91+419+4FvhIRIemPgLuB\n3240c/X90UqlEqVSqc1hzayRcrlMuVxuad6WTtKRNA9YO7Fzr9VaVvdJOj3UbOfe9OnTezr+smXL\ncut33nlnT8e3xrpxko6o2qaXNLuqdgXwYvvtmVm/tfJ13gNACZgu6VVgBFgg6VxgHNgG3NDDHs2s\ny3w+/hHu9ttvz62vWLGip+Pv2rUrtz5jxoyejm+N+Xx8M5vEwTdLkINvliAH3yxBDr5Zghx8swQ5\n+GYJavdYfeuTnTt35tYfeeSRno6/dOnS3Lq/pz8yeYlvliAH3yxBDr5Zghx8swQ5+GYJcvDNEuTg\nmyXI5+MX3CmnnJJb37NnT0evv3BhvSun/8Sjjz6aW586dWpH41vv+Hx8M5vEwTdLkINvliAH3yxB\nDr5Zghx8swQ5+GYJ8vf4BXfMMcfk1oeGOvvsXr9+fW79oosu6uj1bXD8Pb6ZTeLgmyXIwTdLkINv\nliAH3yxBDr5Zghx8swQ1va6+pLnAfcAsYBxYGRHfkHQy8F1gHrANuCoi3uphr0elm2++Obc+Pj7e\n0/Hnz5/f09e3Ymplif9/wPKI+CjwceDzks4GbgV+EBFnAeuBL/euTTPrpqbBj4jdEbExe3wQ2AzM\nBRYDa7LZ1gCX96pJM+uuw9rGl3QacC7wLDArIsag8uEAzOx2c2bWGy3fO0/S8cAjwJci4qCk2gPw\nGx6QPzo6euhxqVSiVCodXpdm1lS5XKZcLrc0b0sn6UiaAvwN8LcRcU82bTNQiogxSbOBH0bEz9V5\nrk/SydFs595dd92VW+/0JJ29e/fm1qdNm9bR69vgdOMknW8BmyZCn3kCWJI9vhZ4vO0OzayvWvk6\n70Lgt4AXJD1PZZX+NuBrwEOSrgO2A1f1slEz656mwY+IfwQanRT+q91t5+jT6f3tm63KH3vssbn1\nkZGR3Prw8HBu3Y5OPnLPLEEOvlmCHHyzBDn4Zgly8M0S5OCbJcjBN0uQr6vfY1u2bMmtn3POObn1\nZufjn3XWWbn1TZs25dbt6OXr6pvZJA6+WYIcfLMEOfhmCXLwzRLk4JslyME3S5CDb5YgB98sQQ6+\nWYIcfLMEOfhmCXLwzRLk4JslyME3S1DL986z9syZMye3vmjRotz62rVru9mOGeAlvlmSHHyzBDn4\nZgly8M0S5OCbJcjBN0tQ0+BLmitpvaR/l/SCpC9k00ckvSbpX7KfS3rfrpl1Q9Pr6kuaDcyOiI2S\njgd+BCwGfhM4EBF3N3l+0tfVNxuUvOvqNz2AJyJ2A7uzxwclbQYmjkqp+6JmVmyHtY0v6TTgXOCf\ns0k3SdooaZWkaV3uzcx6pOXgZ6v5jwBfioiDwL3A6RFxLpU1gtxVfjMrjpaO1Zc0hUro/zIiHgeI\niDerZlkJNDyofHR09NDjUqlEqVRqo1Uzy1MulymXyy3N29JNMyXdB+yJiOVV02Zn2/9IWgZ8LCI+\nV+e53rlnNgB5O/da2at/IfAPwAtAZD+3AZ+jsr0/DmwDboiIsTrPd/DNBqCj4HdhcAffbAB8m2wz\nm8TBN0uQg2+WIAffLEEOvlmCHHyzBDn4Zgly8M0S5OCbJcjBN0uQg2+WIAffLEF9D36r5wsPivvr\nTJH7K3Jv0N/+HPwa7q8zRe6vyL3BUR58Mxs8B98sQX25EEdPBzCzhgZ2BR4zKx6v6pslyME3S1Df\ngi/pEkkvSdoq6ZZ+jdsqSdsk/auk5yU9V4B+Vksak/RvVdNOlrRO0hZJfz/Iuxc16K8wN1Ktc7PX\nL2bTC/EeDvpmtH3Zxpc0BGwFLgZ2ARuAqyPipZ4P3iJJ/wH8QkTsG3QvAJJ+CTgI3BcR87NpXwP+\nMyK+nn14nhwRtxaovxFauJFqP+Tc7HUpBXgPO70Zbaf6tcQ/H3g5IrZHxHvAg1T+yCIRBdr0iYin\ngdoPocXAmuzxGuDyvjZVpUF/UJAbqUbE7ojYmD0+CGwG5lKQ97BBf327GW2//qHPAXZU/f4aP/kj\niyKA70vaIOn6QTfTwMyJm5ZkdzGaOeB+6incjVSrbvb6LDCraO/hIG5GW5glXAFcGBHnAZcCn89W\nZYuuaN/FFu5GqnVu9lr7ng30PRzUzWj7FfydwKlVv8/NphVGRLye/fdN4DEqmydFMyZpFhzaRnxj\nwP1MEhFvVt02aSXwsUH2U+9mrxToPWx0M9p+vIf9Cv4G4AxJ8yRNBa4GnujT2E1JOi775EXSMPAp\n4MXBdgVUtvWqt/eeAJZkj68FHq99Qp9N6i8L0oQrGPx7+C1gU0TcUzWtSO/hB/rr13vYtyP3sq8l\n7qHyYbM6Ilb0ZeAWSPpZKkv5oHLr8PsH3Z+kB4ASMB0YA0aAvwYeBn4G2A5cFRH7C9TfAlq4kWqf\n+mt0s9fngIcY8HvY6c1oOx7fh+yapcc798wS5OCbJcjBN0uQg2+WIAffLEEOvlmCHHyzBDn4Zgn6\nf9GqffzoNTn8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b2e9a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def plot_digit(X, y, idx):\n",
    "    img = X[idx].reshape(28,28)\n",
    "    plt.imshow(img, cmap='Greys',  interpolation='nearest')\n",
    "    plt.title('true label: %d' % y[idx])\n",
    "    plt.show()\n",
    "\n",
    "for i in range(4):\n",
    "    plot_digit(X, y, i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, we can save the NumPy arrays as CSV files for more convenient retrievel, however, I wouldn't recommend storing the 60,000 samples as CSV files due to the enormous file size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.savetxt('train_img.csv', X[:3000, :], delimiter=',', fmt='%i')\n",
    "np.savetxt('train_labels.csv', y[:3000], delimiter=',', fmt='%i')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = np.genfromtxt('train_img.csv', delimiter=',', dtype=int)\n",
    "y = np.genfromtxt('train_labels.csv', delimiter=',', dtype=int)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}