1. Defining Neural Networks in PyTorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Defining Neural Networks in PyTorch\n",
    "\n",
    "The time for the main course has come: Neural networks!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preliminaries\n",
    "\n",
    "Let's import the libraries we'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import numpy as np\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch import optim\n",
    "import torch.nn.functional as F\n",
    "\n",
    "import utils\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from torchvision import datasets, transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "As usual, for demonstration purposes, we will rely on the MNIST dataset. \n",
    "\n",
    "To import it, we'll use the `torchvision` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "train_set = datasets.MNIST('MNIST_data/', download=True, train=True, transform=transform)\n",
    "train_loader = torch.utils.data.DataLoader(train_set, batch_size=64, shuffle=True)\n",
    "\n",
    "test_set = datasets.MNIST('MNIST_data/', download=True, train=False, transform=transform)\n",
    "test_loader = torch.utils.data.DataLoader(test_set, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_iter = iter(train_loader)\n",
    "images, labels = data_iter.next()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've downloaded the training and test data and put them into a `DataLoader` that will allow us to fetch batches of data using an iterator later on.\n",
    "\n",
    "Let's take a look at some sample image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f72a57e40f0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAH0CAYAAADVH+85AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAHNVJREFUeJzt3X/QbXVdL/D3J0lADESnYposQEMmRLnnWCh0EXDy6i0BBe44U8Y0ypQy1zB1/JF0sdKgHyDK9cdgxajTpcKJxhvhL0CQYzodxtBJATvnxHXCEDgCglDI9/6x18nT0/OcH3vt8+znfPfrNbNnPXut9VnrcxaL5/2svdePaq0FAOjT9827AQBgzxH0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANCxfebdwJ5QVZuTHJhky5xbAYBpHZrk/tbaYWMW0mXQZxLyTx5eALCw5vrRfVX9aFX9cVX9c1U9UlVbqupdVXXwyEVvmUV/ADBnW8YuYG5H9FX1tCQbkvxQkr9K8tUkP53k15K8qKqOb63dM6/+AKAH8zyif28mIf/a1tpprbU3t9ZOTnJxkmckecccewOALlRrbfVXWnV4kn/M5COJp7XWHttu2g8kuTNJJfmh1tqDUyx/Y5J1s+kWAObm5tba+jELmNcR/cnD8BPbh3yStNYeSHJTkickee5qNwYAPZnXd/TPGIa3rTD99iQvTHJEkk+vtJDhyH05R07fGgD0Y15H9AcNw/tWmL5t/JNWoRcA6NZavY6+huEOTyBY6XsL39EDwMS8jui3HbEftML0A5fMBwBMYV5Bf+swPGKF6T8xDFf6Dh8A2AXzCvrrhuELq+o/9DBcXnd8ku8k+dvVbgwAejKXoG+t/WOST2Ryw/5zlkx+e5IDknxommvoAYDvmefJeK/J5Ba4766qFyT5SpJjk5yUyUf2vzHH3gCgC3O7Be5wVP+cJJdnEvCvT/K0JO9O8jz3uQeA8eZ6eV1r7f8l+eV59gAAPZvrY2oBgD1L0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAx/aZdwPAYjrrrLNG1b/3ve+duvYXfuEXRq37qquuGlUPq8kRPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0zPPo+Q/Wr18/de3GjRtn2Am7Yv/99x9Vf9ppp01de/rpp49a98te9rJR9a21qWtPPfXUUet++OGHp6695pprRq0bdtfcjuiraktVtRVe35hXXwDQk3kf0d+X5F3LjP/2ajcCAD2ad9B/q7V2/px7AIBuORkPADo27yP6favqF5P8WJIHk9yS5IbW2nfn2xYA9GHeQX9Ikg8vGbe5qn65tfaZnRVX1UqneR85ujMA6MA8P7r/kyQvyCTsD0hydJIPJDk0yd9U1bPn1xoA9GFuR/SttbcvGfXlJL9aVd9O8vok5yd56U6WsexF38OR/roZtAkAe7W1eDLe+4fhCXPtAgA6sBaD/q5heMBcuwCADqzFoH/eMNw01y4AoANzCfqqOqqqnrzM+B9Pcunw9iOr2xUA9GdeJ+OdmeTNVXVdks1JHkjytCQ/l2S/JFcn+YM59QYA3ZhX0F+X5BlJ/ksmH9UfkORbST6byXX1H25jHk0FACRJqsc8dXnd9A4++OCpa7du3TrDThbHmMe9nnfeeaPWffTRR4+qH6OqRtXvrb+73vrWt46qv/DCC2fUCXuJm1e6lHxXrcWT8QCAGRH0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHfM8ehjpne9856j6N7/5zVPXjv3/96GHHpq69g//8A9Hrfumm24aVT/GunXjfj2M+W9+7733jlr3EUccMXXt1q1bR62bufA8egBgZYIeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADq2z7wbgL3dtddeO6p+/frpn0C5cePGUev+/d///alr9+ZHnn7pS18aVf+7v/u7U9dW1ah1w+5yRA8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHfM8ehjpU5/61FzrF9X+++8/de1FF100at2ttalrP/nJT45a99atW0fVs3gc0QNAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHRM0ANAxwQ9AHTMY2qBuTjyyCNH1b/zne+cuvbUU08dte6NGzdOXfua17xm1Lphd83kiL6qzqiq91TVjVV1f1W1qvrITmqOq6qrq+reqnqoqm6pqnOr6nGz6AkAmN0R/duSPDvJt5N8PckO/1SvqlOTfDTJw0n+LMm9SV6S5OIkxyc5c0Z9AcBCm9V39K9LckSSA5O8ekczVtWBSS5L8t0kJ7bWXtlae2OSY5J8LskZVfXyGfUFAAttJkHfWruutXZ7a63twuxnJPnBJFe01v5uu2U8nMknA8lO/lgAAHbNPM66P3kYXrPMtBuSPJTkuKrad/VaAoA+zSPonzEMb1s6obX2aJLNmZw7cPhqNgUAPZrH5XUHDcP7Vpi+bfyTdragqlrpGpdx1+0AQCfW4g1zahjuyvf9AMAOzOOIftsR+0ErTD9wyXwraq2tX278cKS/bvdbA4C+zOOI/tZheMTSCVW1T5LDkjyaZNNqNgUAPZpH0F87DF+0zLQTkjwhyYbW2iOr1xIA9GkeQX9lkruTvLyqnrNtZFXtl+R3hrfvm0NfANCdmXxHX1WnJTlteHvIMHxeVV0+/Hx3a+0NSdJau7+qzs4k8K+vqisyuQXuKZlcendlJrfFBQBGmtXJeMckOWvJuMPzvWvh/ynJG7ZNaK1dVVXPT/IbSU5Psl+SryX59STv3sU77AEAOzGToG+tnZ/k/N2suSnJf5/F+gGA5XkePczZm970pqlrTzrppBl2srqOPvroUfWHHHLIzmdawV/8xV+MWvc555wzde3WrVtHrRt211q8YQ4AMCOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA65jG1kGTfffeduvamm24ate5169aNql9UmzZtmrr27LPPHrXuBx54YFQ9rCZH9ADQMUEPAB0T9ADQMUEPAB0T9ADQMUEPAB0T9ADQMUEPAB0T9ADQMUEPAB0T9ADQMUEPAB0T9ADQMUEPAB0T9ADQMc+jhyRPfOITp659+tOfPsNOds+DDz44qv7222+funbz5s2j1n3iiSeOqj/ssMOmrv3TP/3TUet+yUteMqoeVpMjegDomKAHgI4JegDomKAHgI4JegDomKAHgI4JegDomKAHgI4JegDomKAHgI4JegDomKAHgI4JegDomKAHgI5Va23ePcxcVW1Msm7efbAYDj/88FH1Yx6R+/DDD49a92233Taqfoxjjz12VP3HPvaxqWsf//jHj1r3McccM3Xtli1bRq2bhXNza239mAU4ogeAjgl6AOiYoAeAjgl6AOiYoAeAjgl6AOiYoAeAjgl6AOiYoAeAjgl6AOiYoAeAjgl6AOiYoAeAjgl6AOiYoAeAjnkePbBXevWrXz117aWXXjpq3b/3e783de1b3vKWUetm4ayN59FX1RlV9Z6qurGq7q+qVlUfWWHeQ4fpK72umEVPAECyz4yW87Ykz07y7SRfT3LkLtT8fZKrlhn/5Rn1BAALb1ZB/7pMAv5rSZ6f5LpdqPlia+38Ga0fAFjGTIK+tfbvwV5Vs1gkADADszqin8aPVNWvJHlKknuSfK61dssc+wGA7swz6H92eP27qro+yVmttTt2ZQHD2fXL2ZVzBACge/O4jv6hJL+dZH2Sg4fXtu/1T0zy6ao6YA59AUB3Vv2IvrV2V5LfXDL6hqp6YZLPJjk2yauSXLILy1r22kLX0QPAxJq5M15r7dEkHxzenjDPXgCgF2sm6AffHIY+ugeAGVhrQf/cYbhprl0AQCdWPeir6tiqevwy40/O5MY7SbLs7XMBgN0zk5Pxquq0JKcNbw8Zhs+rqsuHn+9urb1h+PnCJEcNl9J9fRj3rCQnDz+f11rbMIu+AGDRzeqs+2OSnLVk3OHDK0n+Kcm2oP9wkpcm+akkL07y/Un+JcmfJ7m0tXbjjHoCgIU3q1vgnp/k/F2c94+S/NEs1gsA7Ng874wHMLUNG6b/hm/sMznWrx/1eHBYVWvtrHsAYIYEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0zGNqgb3SwQcfPHVta23UutetWzeqHlaTI3oA6JigB4COCXoA6JigB4COCXoA6JigB4COCXoA6JigB4COCXoA6JigB4COCXoA6JigB4COCXoA6JigB4COCXoA6Jjn0QNzceihh46qv/zyy6eufeyxx0at+x3veMeoelhNjugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA65jG1rBmveMUrpq49/PDDR61706ZNU9d+6lOfGrXuO++8c1T9vIzd5hdccMGo+qc+9alT127YsGHUui+++OJR9bCaHNEDQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMc8j56Z+Zmf+ZlR9R/60Iemrm2tjVr3PL3lLW+ZuvbCCy8cte4jjzxy6trLLrts1LqPP/74UfWf//znp6495ZRTRq0b9iajj+ir6ilV9aqq+suq+lpVfaeq7quqz1bVK6tq2XVU1XFVdXVV3VtVD1XVLVV1blU9bmxPAMDELI7oz0zyviR3JrkuyR1JfjjJy5J8MMmLq+rMtt0hV1WdmuSjSR5O8mdJ7k3ykiQXJzl+WCYAMNIsgv62JKck+evW2mPbRlbVW5N8IcnpmYT+R4fxBya5LMl3k5zYWvu7Yfx5Sa5NckZVvby1dsUMegOAhTb6o/vW2rWttY9tH/LD+G8kef/w9sTtJp2R5AeTXLEt5If5H07ytuHtq8f2BQDs+bPu/20YPrrduJOH4TXLzH9DkoeSHFdV++7JxgBgEeyxs+6rap8kvzS83T7UnzEMb1ta01p7tKo2JzkqyeFJvrKTdWxcYdL0pxIDQEf25BH9BUmemeTq1trHtxt/0DC8b4W6beOftKcaA4BFsUeO6KvqtUlen+SrSV6xu+XDcKcXRrfW1q+w/o1J1u3megGgOzM/oq+qc5JckuQfkpzUWrt3ySzbjtgPyvIOXDIfADClmQZ9VZ2b5NIkX84k5L+xzGy3DsMjlqnfJ8lhmZy8t2mWvQHAIppZ0FfVmzK54c0XMwn5u1aY9dph+KJlpp2Q5AlJNrTWHplVbwCwqGYS9MPNbi5IsjHJC1prd+9g9iuT3J3k5VX1nO2WsV+S3xnevm8WfQHAoht9Ml5VnZXktzK5092NSV5bVUtn29JauzxJWmv3V9XZmQT+9VV1RSa3wD0lk0vvrszktrgAwEizOOv+sGH4uCTnrjDPZ5Jcvu1Na+2qqnp+kt/I5Ba5+yX5WpJfT/Lutjc/igwA1pDqMVNdXjcfF1100aj6c89d6e/EnRvzqNckufLKK6euveKKcY9lOProo6eu3bhxpXtG7ZpnPvOZU9c+8YlPHLXuL3zhC6Pqf/7nf37q2nvuuWfUumEV3bzSpeS7ak/fAhcAmCNBDwAdE/QA0DFBDwAdE/QA0DFBDwAdE/QA0DFBDwAdE/QA0DFBDwAdE/QA0DFBDwAdE/QA0DFBDwAdE/QA0DHPo2dmzj777FH1H/jAB6auHbsfb968eeraww47bNS6q2rq2rH/7gcffHDq2osuumjUui+55JJR9Vu3bh1VD3sJz6MHAFYm6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY4IeADom6AGgY/vMuwH6cdlll42qf+SRR6aufeMb3zhq3T/5kz85de2mTZtGrXvDhg1T11555ZWj1n3rrbdOXXvbbbeNWjewOhzRA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHqrU27x5mrqo2Jlk37z4AYKSbW2vrxyzAET0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdEzQA0DHBD0AdGx00FfVU6rqVVX1l1X1tar6TlXdV1WfrapXVtX3LZn/0KpqO3hdMbYnAGBinxks48wk70tyZ5LrktyR5IeTvCzJB5O8uKrObK21JXV/n+SqZZb35Rn0BABkNkF/W5JTkvx1a+2xbSOr6q1JvpDk9ExC/6NL6r7YWjt/BusHAFYw+qP71tq1rbWPbR/yw/hvJHn/8PbEsesBAHbfLI7od+TfhuGjy0z7kar6lSRPSXJPks+11m7Zw/0AwELZY0FfVfsk+aXh7TXLzPKzw2v7muuTnNVau2NP9QUAi2RPHtFfkOSZSa5urX18u/EPJfntTE7E2zSMe1aS85OclOTTVXVMa+3Bna2gqjauMOnIaZsGgJ7Ufz4ZfgYLrXptkkuSfDXJ8a21e3ehZp8kn01ybJJzW2uX7ELNjoL+CbveMQCsSTe31taPWcDMj+ir6pxMQv4fkrxgV0I+SVprj1bVBzMJ+hOGZeysZtl//PAHwLpdbhoAOjXTO+NV1blJLs3kWviThjPvd8c3h+EBs+wLABbVzIK+qt6U5OIkX8wk5O+aYjHPHYabdjgXALBLZhL0VXVeJiffbczk4/q7dzDvsVX1+GXGn5zkdcPbj8yiLwBYdKO/o6+qs5L8VpLvJrkxyWuraulsW1prlw8/X5jkqOFSuq8P456V5OTh5/NaaxvG9gUAzOZkvMOG4eOSnLvCPJ9Jcvnw84eTvDTJTyV5cZLvT/IvSf48yaWttRtn0BMAkD10ed28OesegE6MvrzO8+gBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA6JugBoGOCHgA61mvQHzrvBgBgBg4du4B9ZtDEWnT/MNyywvQjh+FX93wr3bDNpmO7Tcd223222XTW8nY7NN/Ls6lVa218K3uZqtqYJK219fPuZW9hm03HdpuO7bb7bLPpLMJ26/WjewAggh4AuiboAaBjgh4AOiboAaBjC3nWPQAsCkf0ANAxQQ8AHRP0ANAxQQ8AHRP0ANAxQQ8AHRP0ANCxhQr6qvrRqvrjqvrnqnqkqrZU1buq6uB597ZWDduorfD6xrz7m5eqOqOq3lNVN1bV/cP2+MhOao6rqqur6t6qeqiqbqmqc6vqcavV97ztznarqkN3sO+1qrpitfufh6p6SlW9qqr+sqq+VlXfqar7quqzVfXKqlr29/ii72+7u9163t96fR79f1JVT0uyIckPJfmrTJ49/NNJfi3Ji6rq+NbaPXNscS27L8m7lhn/7dVuZA15W5JnZ7INvp7vPdN6WVV1apKPJnk4yZ8luTfJS5JcnOT4JGfuyWbXkN3aboO/T3LVMuO/PMO+1rIzk7wvyZ1JrktyR5IfTvKyJB9M8uKqOrNtd/cz+1uSKbbboL/9rbW2EK8kH0/SkvzPJeMvGsa/f949rsVXki1Jtsy7j7X2SnJSkp9IUklOHPahj6ww74FJ7krySJLnbDd+v0z++GxJXj7vf9Ma3G6HDtMvn3ffc95mJ2cS0t+3ZPwhmYRXS3L6duPtb9Ntt273t4X46L6qDk/ywkxC638vmfy/kjyY5BVVdcAqt8ZeqrV2XWvt9jb8htiJM5L8YJIrWmt/t90yHs7kCDdJXr0H2lxzdnO7kaS1dm1r7WOttceWjP9GkvcPb0/cbpL9LVNtt24tykf3Jw/DTyzzH/2Bqropkz8Enpvk06vd3F5g36r6xSQ/lskfRbckuaG19t35trXX2Lb/XbPMtBuSPJTkuKrat7X2yOq1tdf4kar6lSRPSXJPks+11m6Zc09rxb8Nw0e3G2d/27nltts23e1vixL0zxiGt60w/fZMgv6ICPrlHJLkw0vGba6qX26tfWYeDe1lVtz/WmuPVtXmJEclOTzJV1azsb3Ezw6vf1dV1yc5q7V2x1w6WgOqap8kvzS83T7U7W87sIPttk13+9tCfHSf5KBheN8K07eNf9Iq9LK3+ZMkL8gk7A9IcnSSD2TyfdbfVNWz59faXsP+N52Hkvx2kvVJDh5ez8/kxKoTk3x6wb9uuyDJM5Nc3Vr7+Hbj7W87ttJ263Z/W5Sg35kahr43XKK19vbhu65/aa091Fr7cmvtVzM5iXH/JOfPt8Mu2P+W0Vq7q7X2m621m1tr3xpeN2Ty6dvnkzw9yavm2+V8VNVrk7w+k6uHXrG75cNw4fa3HW23nve3RQn6bX/BHrTC9AOXzMfObTuZ5YS5drF3sP/NUGvt0Uwuj0oWcP+rqnOSXJLkH5Kc1Fq7d8ks9rdl7MJ2W1YP+9uiBP2tw/CIFab/xDBc6Tt8/rO7huFe+VHWKltx/xu+Lzwsk5OCNq1mU3u5bw7Dhdr/qurcJJdmck33ScMZ5EvZ35bYxe22I3v1/rYoQX/dMHzhMndD+oFMbiDxnSR/u9qN7cWeNwwX5pfFCNcOwxctM+2EJE9IsmGBz4CexnOH4cLsf1X1pkxuePPFTMLqrhVmtb9tZze2247s1fvbQgR9a+0fk3wikxPIzlky+e2Z/JX2odbag6vc2ppWVUdV1ZOXGf/jmfx1nCQ7vO0rSZIrk9yd5OVV9ZxtI6tqvyS/M7x93zwaW8uq6tiqevwy409O8rrh7ULsf1V1XiYnkW1M8oLW2t07mN3+Ntid7dbz/laLct+KZW6B+5Ukx2Zyp67bkhzX3AL3P6iq85O8OZNPRDYneSDJ05L8XCZ32bo6yUtba/86rx7npapOS3La8PaQJP8tk7/2bxzG3d1ae8OS+a/M5JakV2RyS9JTMrkU6sok/2MRbiKzO9ttuKTpqCTXZ3K73CR5Vr53nfh5rbVtwdWtqjoryeVJvpvkPVn+u/UtrbXLt6tZ+P1td7db1/vbvG/Nt5qvJE/N5HKxO5P8a5J/yuTkjCfPu7e1+Mrk0pL/k8kZqt/K5CYT30zyyUyuQ6159zjHbXN+Jmctr/TaskzN8Zn8cbQ1k6+KvpTJkcLj5v3vWYvbLckrk/zfTO5o+e1Mbul6Ryb3bv+v8/63rKFt1pJcb38bt9163t8W5ogeABbRQnxHDwCLStADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB0TNADQMcEPQB07P8DQOjqpsUogdAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f7350152940>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 250,
       "width": 253
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(images[1].numpy().squeeze(), cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Networks with PyTorch\n",
    "\n",
    "There are many ways to build a neural network in PyTorch.\n",
    "\n",
    "In the cell below we are creating a subclass of `nn.Module` to model our network's architecture.\n",
    "\n",
    "Notice we are defining the architecture in the constructor, while the `forward` method is used to forward propagate the information through the network. \n",
    "\n",
    "PyTorch supports functional equivalents for things like ReLUs and activation functions such as softmax. Here we are using them under the `F` alias we gabe to the `torch.nn.functional` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Network(\n",
       "  (fully_connected_1): Linear(in_features=784, out_features=128, bias=True)\n",
       "  (fully_connected_2): Linear(in_features=128, out_features=64, bias=True)\n",
       "  (fully_connected_3): Linear(in_features=64, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class Network(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.fully_connected_1 = nn.Linear(784, 128)\n",
    "        self.fully_connected_2 = nn.Linear(128, 64)\n",
    "        self.fully_connected_3 = nn.Linear(64, 10)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.fully_connected_1(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.fully_connected_2(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.fully_connected_3(x)\n",
    "        x = F.softmax(x, dim=1)\n",
    "        \n",
    "        return x\n",
    "    \n",
    "model = Network()\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initializing weights and biases\n",
    "\n",
    "Both are automatically initialized, but we can override their values. \n",
    "\n",
    "The weights and biases are tensors attached to each layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[ 0.0326,  0.0228, -0.0310,  ...,  0.0218,  0.0339, -0.0046],\n",
      "        [ 0.0137, -0.0112, -0.0206,  ...,  0.0184, -0.0125, -0.0350],\n",
      "        [ 0.0149, -0.0148, -0.0295,  ...,  0.0258,  0.0101, -0.0170],\n",
      "        ...,\n",
      "        [ 0.0012,  0.0308, -0.0005,  ..., -0.0200,  0.0170,  0.0072],\n",
      "        [-0.0129,  0.0275,  0.0207,  ...,  0.0219, -0.0276,  0.0338],\n",
      "        [ 0.0245, -0.0036,  0.0288,  ...,  0.0060, -0.0067,  0.0277]],\n",
      "       requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([-0.0063,  0.0080,  0.0192, -0.0143, -0.0038,  0.0241, -0.0184, -0.0194,\n",
      "        -0.0048, -0.0245,  0.0209, -0.0109,  0.0292, -0.0073, -0.0121, -0.0193,\n",
      "         0.0087, -0.0044, -0.0040,  0.0299, -0.0214,  0.0170,  0.0114, -0.0145,\n",
      "        -0.0186, -0.0011, -0.0102,  0.0035, -0.0081, -0.0027,  0.0016, -0.0118,\n",
      "         0.0128,  0.0106, -0.0012,  0.0169,  0.0093,  0.0209, -0.0248, -0.0159,\n",
      "         0.0292,  0.0153,  0.0221, -0.0093, -0.0045, -0.0264,  0.0044,  0.0177,\n",
      "         0.0115,  0.0091,  0.0169, -0.0259,  0.0155,  0.0114,  0.0119,  0.0055,\n",
      "        -0.0015,  0.0322,  0.0311,  0.0139,  0.0022, -0.0207, -0.0061,  0.0020,\n",
      "        -0.0354,  0.0087, -0.0288,  0.0168,  0.0008, -0.0270, -0.0111,  0.0211,\n",
      "        -0.0047, -0.0171,  0.0055,  0.0312,  0.0307,  0.0311,  0.0068,  0.0139,\n",
      "         0.0067,  0.0188,  0.0309, -0.0061, -0.0182,  0.0161, -0.0143,  0.0106,\n",
      "        -0.0286, -0.0266,  0.0237, -0.0214,  0.0221, -0.0075,  0.0305,  0.0206,\n",
      "        -0.0061, -0.0181, -0.0043,  0.0174, -0.0137,  0.0048, -0.0085,  0.0020,\n",
      "        -0.0181, -0.0028, -0.0152,  0.0147, -0.0344,  0.0064, -0.0222, -0.0296,\n",
      "         0.0144,  0.0250, -0.0067,  0.0151,  0.0199, -0.0231, -0.0180,  0.0344,\n",
      "        -0.0092, -0.0097, -0.0302, -0.0015, -0.0354,  0.0190, -0.0018,  0.0201],\n",
      "       requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "print(model.fully_connected_1.weight)\n",
    "print(model.fully_connected_1.bias)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To modify these tensors, we can use in-place operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "        0., 0.])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set biases to all 0s\n",
    "model.fully_connected_1.bias.data.fill_(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0135, -0.0052,  0.0008,  ...,  0.0111, -0.0221, -0.0054],\n",
       "        [-0.0075,  0.0216, -0.0119,  ...,  0.0218,  0.0084, -0.0003],\n",
       "        [ 0.0055, -0.0016,  0.0005,  ..., -0.0074,  0.0210, -0.0143],\n",
       "        ...,\n",
       "        [ 0.0142,  0.0058,  0.0168,  ..., -0.0169, -0.0060, -0.0081],\n",
       "        [-0.0038,  0.0151,  0.0096,  ...,  0.0194, -0.0001,  0.0157],\n",
       "        [ 0.0064, -0.0116,  0.0134,  ..., -0.0092, -0.0070,  0.0032]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sample from random normal\n",
    "model.fully_connected_1.weight.data.normal_(std=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward pass\n",
    "\n",
    "Cool, we have our network. Let's see what happens we pass in an image. For this we must convert the image into a tensor and then pass it to the `forward` method of the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoEAAAQTCAYAAAAbLfPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xu0Zndd3/HPNxkuIcBAQiESkCEUmEAsmimXEEQBpep4QZRKLahYUGtqVGDZ1KpcqnbQWoOota0ICq5VBYuiw1XNEhEw7RDKCgRQwigEEEhwTGAIufz6x3miwziT7EnmnH32+b5ea521c579O/t8OSxW3vyey64xRgAA6OWEuQcAAGDjiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADS0be4BAIDNo6o+lOSuSfbPPApHtiPJ340x7n9bLyQCAYBD3fWkk0465cwzzzxl7kH4xy677LIcPHjwuFxLBAIAh9p/5plnnrJv37655+AIdu3alXe+8537j8e1vCYQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoaNvcAwAAm8ulVxzIjgv2zj1GS/v37N6w32UnEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQCwMLXmu6vqHVV1dVV9tqouqarzq+rEuedjGUQgACzPryd5aZL7J/mtJP8zye2TvDjJb1VVzTgbC7Ft7gEAgOmq6klJnp7kQ0keMcb41Orx2yX57STfkuQ7k7x8rhlZBjuBALAsT14df+6mAEySMcZ1SX589e0PbPhULI4IBIBlOW11vPwI52567OyqutsGzcNCeToYAJblpt2/+x/h3BmH/PPOJO842kWqat9RTu28lXOxMHYCAWBZ/mB1fHZVnXLTg1W1LckLDll39w2disWxEwgAy/K/kjwtydcmeW9VvTbJZ5N8VZIHJPmLJA9McsPNXWSMsetIj692CM8+ngOzOdkJBIAFGWPcmOQbkzw3ycez9k7h707ykSSPSXLlauknZhmQxbATCAALM8a4PsnPrb7+XlWdlORLkxxM8p4ZRmNB7AQCwNbx9CR3TPLbq4+MgaMSgQCwMFV11yM89vAke5Jck+SFGz4Ui+PpYABYnjdX1cEklya5OslDk3xdkmuTPHmMcaTPEIQvIAIBYHleneSpWXuX8ElJPprkV5PsGWPsn3EuFkQEAsDCjDF+NsnPzj0Hy+Y1gQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEPb5h4AANhczjp9e/bt2T33GKwzO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA1tm3sAAGBzufSKA9lxwd65x2Bl/57d63JdO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBALAAlXV7qp6U1V9pKoOVtXlVfWqqjpn7tlYBhEIAAtTVS9K8gdJzk7yhiQvTvLOJN+U5M+q6mkzjsdCbJt7AABguqo6Lclzk/xNkn82xvjEIecel+SPk7wwySvnmZClsBMIAMtyv6z9+/vPDw3AJBljXJTk6iT/ZI7BWBYRCADL8hdJPp/kEVV1j0NPVNVjk9wlyR/OMRjL4ulgAFiQMcZVVfXvk/zXJO+tqt9NcmWSByT5xiRvTvK9t3Sdqtp3lFM7j9esbG4iEAAWZoxxYVXtT/JrSZ51yKm/TPLyw58mhiPxdDAALExV/UiSVyd5edZ2AE9OsivJ5Ul+s6p+5pauMcbYdaSvJO9bx9HZREQgACxIVX1lkhclee0Y49ljjMvHGJ8dY7wzyTcnuSLJc6rqjDnnZPMTgQCwLF+/Ol50+IkxxmeTXJy1f79/2UYOxfKIQABYljusjkf7GJibHv/8BszCgolAAFiWP10dv6eqTj/0RFV9bZJzk3wuyds2ejCWxbuDAWBZXp21zwH8qiSXVdVrknw8yZlZe6q4klwwxrhyvhFZAhEIAAsyxrixqr4uyXlJnpq1N4PcKclVSV6X5BfGGG+acUQWQgQCwMKMMa5LcuHqC24VrwkEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAa2jb3AADA5nLW6duzb8/uucdgndkJBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoaNvcAwAAm8ulVxzIjgv2zj3GlrV/z+65R0hiJxAAoCURCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQABYkKr6rqoat/B1w9xzsvltm3sAAOCYvCvJC45y7suTPD7J6zduHJZKBALAgowx3pW1EPxHqurtq3/8Hxs3EUvl6WAA2AKq6qwkj0pyRZK9M4/DAtgJBICt4XtXx5eOMW7xNYFVte8op3Yev5HYzOwEAsDCVdVJSZ6W5MYkvzrzOCyEnUAAWL5/meRuSfaOMT485QfGGLuO9Phqh/Ds4zgbm5SdQABYvu9ZHf/7rFOwKCIQABasqh6S5NFJPpLkdTOPw4KIQABYtmN6QwjcRAQCwEJV1R2TPD1rbwh56czjsDAiEACW6ylJ7p7kdVPfEAI3EYEAsFw3vSHEHUI4ZiIQABaoqs5M8ph4Qwi3ks8JBIAFGmNclqTmnoPlshMIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0tG3uAQCAzeWs07dn357dc4/BOrMTCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQ0La5BwAANpdLrziQHRfsnXuMLW3/nt1zj2AnEACgIxEIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaGjb3AOsl68+4Slj7hkAbq033/iqmnsGYGuzEwgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBICFqqovr6rfqaqPVdW1q+Obqurr5p6NzW/L3jYOALayqvqxJP8pyaeS/EGSjyW5R5IvS/KVSV4323AsgggEgIWpqqdkLQD/MMmTxxhXH3b+drMMxqJ4OhgAFqSqTkjyoiSfTfLthwdgkowxrtvwwVgcO4EAsCyPTnL/JK9O8umq2p3krCSfS3LxGOPtcw7HcohAAFiWh6+Of5PknUm+5NCTVfWWJN86xvjkzV2kqvYd5dTO2zwhi+DpYABYlnuujt+X5KQkX5XkLlnbDXxjkscmedU8o7EkdgIBYFlOXB0razt+/2/1/Xuq6puTfCDJV1TVOTf31PAYY9eRHl/tEJ59PAdmc7ITCADL8unV8fJDAjBJMsY4mLXdwCR5xIZOxeKIQABYlvevjn97lPM3ReJJGzALCyYCAWBZ3pLk+iQPrKrbH+H8Wavj/g2biEUSgQCwIGOMTyX5rSTbk/zEoeeq6quT/IskB5K8YeOnY0m8MQQAlufZSR6Z5D9W1WOTXJzkfkm+OckNSZ41xjja08WQRAQCwOKMMT5RVY9M8mNZC79HJbk6yd4k/3mM8Y4552MZRCAALNAY46qs7Qg+e+5ZWCavCQQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADW2bewAAYHM56/Tt2bdn99xjsM5EIEmSE+5yl+mLb7hh8tLx+c9PX3v99dNnAABuE08HAwA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABpy27hN4sR7nDpp3Yef8eDJ17zmIddOXvuic189ee3/ueaMyWtf88ePnLx2nDh5aXa+5OOT115/+f7pFwaAJuwEAgA0JAIBABoSgQAADYlAAICGRCAAQEPeHQwAfIFLrziQHRfsnXuMNvbv2T3L77UTCADQkAgEAGhIBAIANCQCAQAa8saQdbTtjB2T1z7hte+etO68u7/hVk5z/HzTyZ+avPYn/9XF6zLDL3319NvnvezXv2by2nv/7NtuzTgAsDh2AgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANuW3cOrrsh+81ee3/vvv713GSree8Y/h7nfdD09c+4tzvmLTuPj9w9eRrXv/hj0xeCwAbxU4gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCG3jVtHj97lVnBL838f/spJ6x72nT8w+Zr3/Um3jQNg87ETCADQkAgEAGhIBALAwlTV/qoaR/n6+NzzsQxeEwgAy3QgyYVHePyajR6EZRKBALBMfzvGeP7cQ7Bcng4GAGjITiAALNMdquppSb44yWeSvDvJW8YYN8w7FkshAgFgmU5L8orDHvtQVT1jjPEnt/TDVbXvKKd23ubJWARPBwPA8rwsyROyFoInJ/mSJP89yY4kr6+qh803GkthJxAAFmaM8YLDHro0yfdV1TVJnpPk+Um++RausetIj692CM8+DmOyyYnAdXTVs+41ee1DnjntNmQPfNiHJ1/z2V/8pslrP3zdqZPXXnLN/Sav/cur7zF57bF44j0vm7z2vLu7fR/Qxq9kLQIfO/cgbH6eDgaAreMTq+PJs07BIohAANg6zlkdL591ChZBBALAglTVQ6vqlCM8fr8kv7j69pUbOxVL5DWBALAsT0lyQVVdlORDSa5O8oAku5PcMcnrkvyX+cZjKUQgACzLRUkenOTLsvb078lJ/jbJW7P2uYGvGGOM+cZjKUQgACzI6oOgb/HDoOGWeE0gAEBDIhAAoCERCADQkAgEAGjIG0PW0Q3vmX67sn/6w9PWHcvbvS586JMnr62D105eOz594BjWXjF57bH4o/s+ZPLaT732zpPXvuCel0xa95SnTH9N9jt+8naT1wLARrETCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGjIbeO2sGO5bd3SXP/hj0xe+4g7X37cf/+zT/2/k9e+cN+5k9e++TcfNXntZ0+/cfLaE07/7OS193/quyevBWC57AQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGnLbOBbpo8999OS1Tzzp4mO48omTVt2xpv9P56dP+/Ppa58zfe2xuOqGayevfe8Ht09e+72/+6zJa+/9lum3uTvp947lvzMAbg0RCAB8gbNO3559e3bPPQbrzNPBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGvJh0SzSDSdNX3ti1foNshCnnHiHyWsfc+LnJq99z1NfMnnt+V/+2Mlr//qNd5y89sbPTZ8XgH9gJxAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDPiwaAPgCl15xIDsu2Dv3GFvC/j275x7hqOwEAgA0ZCeQTeP6x++avHbvM39m8toTMv0ec7erEyetu25MvuS6uebGayevvfMJ028bt15+8fS3Tl77oAv/7fS133fxrRkHoD07gQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICG3DaOTeOTXzr91mb33jZ97Y25cfLaqbeD+8B1n598zSe9bfot0G733jtNXnvPfddNXvtX3zL9PnevecIvTV575u3X5/9Hvu8bps/wjd/38HWZAWCrsxMIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaMht49g07n3RgclrX/29p01eu/P2H5u89qlv/zeT1p1x4Q2Tr3nGxe+avHa9POj109f+6EO/Y/LaH3ntqyavPeeO104fAoB1ZycQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAA2AKq6ulVNVZfz5x7HjY/EQgAC1dV903ykiTXzD0LyyECAWDBqqqSvCzJlUl+ZeZxWBARCADLdn6Sxyd5RpLPzDwLC+K2cWwa45L3TF77ym/7F5PX3nCn201ee8bb5r/F29xueM/7J6/9/ku+ffLaS8552a0ZB7gZVXVmkj1JXjzGeEtVPX7umVgOEQgAC1RV25K8IslfJ/nRW/Hz+45yaudtmYvlEIEAsEw/keTLkjxmjHFw7mFYHhEIAAtTVY/I2u7fz40x3n5rrjHG2HWUa+9LcvZtGI+F8MYQAFiQQ54G/kCSH595HBZMBALAstw5yYOSnJnkc4d8QPRI8rzVmv+5euzC2aZk0/N0MAAsy7VJXnqUc2dn7XWCb03y/iS36qliehCBALAgqzeBHPG2cFX1/KxF4K+PMX51I+dieTwdDADQkAgEAGhIBALAFjHGeP4YozwVzBReE8gi3fiu905eW+s4x1Js+6LTJq/966efMXnt3of/zDFMcYdjWAvAerMTCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGjIbeNgEznxHqdOXrv/V75o8tpn7Hz75LXn3/33J691KziA5bITCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGjIbeNgnV39bY+avPZfP2/v5LXP3P6GWzPOIjzspedPXnu/TL8lHgD/wE4gAEBDIhAAoCERCADQkAgEAGjIG0MAgC9w1unbs2/P7rnHYJ3ZCQQAaEgEAgA0JAIBABoSgQAADYlAAICGvDsY1tkdP3395LXP3H75Ok5y/L3swI7Ja1/yim+avPZ+L/rzWzENAMfCTiAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIbeNg3V2+4vePXnt1z/1eyavve4u0//n+7FHT1/7xW84OHnt7d5/xeS19/nk2yavBWD9iUAA4AtcesWB7Lhg79xjtLV/z+4N+T2eDgYAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQD4uGdTau+/zktSf86SWT197hGGbY8bpjWHwMblifywKwAewEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEAAWpqpeVFV/VFUfrqqDVXVVVV1SVc+rqlPnno9lEIEAsDw/nOTkJG9O8uIkv5nk+iTPT/LuqrrvfKOxFNvmHgAAOGZ3HWN87vAHq+qnkvxokv+Q5Ps3fCoWxU4gACzMkQJw5bdXxwdu1CwslwgEgK3jG1bHd886BYvg6WAAWKiqem6SOyfZnuSfJ3lM1gJwz4Sf3XeUUzuP24BsaiIQAJbruUnudcj3b0jyXWOMT840DwsiAgFgocYYpyVJVd0ryaOztgN4SVV9/Rjjnbfws7uO9Phqh/Ds4z0rm4/XBALAwo0x/maM8ZokT0xyapLfmHkkFkAEAsAWMcb4qyTvTfLQqrrH3POwuYlAANha7r063jDrFGx6IhAAFqSqdlbVaUd4/ITVh0XfM8nbxhif3vjpWBJvDAGAZfmaJD9bVW9J8sEkV2btHcJfkeSMJB9P8qz5xmMpRCAALMsfJvkfSc5N8rAkd0vymSQfSPKKJL8wxrhqvvFYChEIAAsyxrg0yXlzz8HyeU0gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA1t2c8JfPONr6q5ZwAA2KzsBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA1tm3sAAGBzOev07dm3Z/fcY7DO7AQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADS0be4BAIDN5dIrDmTHBXvnHmPL279n96y/304gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAsCBVdWpVPbOqXlNVf1lVB6vqQFW9tar+TVX5dzuTbJt7AADgmDwlyX9L8rEkFyX56yT3SvLkJL+a5Gur6iljjDHfiCyBCASAZflAkm9MsneMceNND1bVjya5OMm3ZC0If2ee8VgKW8YAsCBjjD8eY/z+oQG4evzjSX5l9e1XbvhgLI6dQADYOq5bHa+/pYVVte8op3Yev3HYzOwEAsAWUFXbknzH6ts3zDkLy2AnEAC2hj1JzkryujHGG29p8Rhj15EeX+0Qnn2cZ2MTshMIAAtXVecneU6S9yV5+szjsBAiEAAWrKrOS/LiJO9N8rgxxlUzj8RCiEAAWKiq+qEkv5jk0qwF4MdnHokFEYEAsEBV9e+T/HySd2UtAD8x80gsjAgEgIWpqh/P2htB9iV5whjjUzOPxAJ5dzAALEhVfWeSFya5IcmfJjm/qg5ftn+M8fINHo2FEYEAsCz3Xx1PTPJDR1nzJ0leviHTsFieDgaABRljPH+MUbfw9ZVzz8nmJwIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGts09AACwuZx1+vbs27N77jFYZ3YCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAa2jb3AADA5nLpFQey44K9c4+x5e3fs3vW328nEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQCwMFX1rVX1kqr606r6u6oaVfXKuediWbbNPQAAcMx+LMnDklyT5CNJds47DktkJxAAlueHkzwoyV2T/NuZZ2Gh7AQCwMKMMS666Z+ras5RWDA7gQAADdkJBICGqmrfUU55fWETdgIBABqyEwgADY0xdh3p8dUO4dkbPA4zsBMIANCQCAQAaEgEAgA0JAIBABryxhAAWJiqelKSJ62+PW11PKeqXr7650+NMZ674YOxKCIQAJbnS5N852GPnbH6SpK/SiLLayvCAAAIh0lEQVQCuVmeDgaAhRljPH+MUTfztWPuGdn8RCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANDQtrkHAAA2l7NO3559e3bPPQbrzE4gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoKFtcw8AAGwul15xIDsu2Dv3GFva/j275x7BTiAAQEciEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBYIGq6j5V9WtV9dGquraq9lfVhVV197lnYxm2zT0AAHBsquoBSd6W5J5Jfi/J+5I8IskPJvmaqjp3jHHljCOyAHYCAWB5fjlrAXj+GONJY4wLxhiPT/LzSR6c5KdmnY5FEIEAsCBVdUaSJybZn+SXDjv9vCSfSfL0qjp5g0djYUQgACzL41fHN40xbjz0xBjj6iR/luROSR610YOxLF4TCADL8uDV8QNHOf8XWdspfFCSPzraRapq31FO7bz1o7EkdgIBYFm2r44HjnL+psfvtgGzsGB2AgFga6nVcdzcojHGriP+8NoO4dnHeyg2HzuBALAsN+30bT/K+bsetg6OSAQCwLK8f3V80FHOP3B1PNprBiGJCASApblodXxiVX3Bv8er6i5Jzk1yMMk7NnowlkUEAsCCjDE+mORNSXYkOe+w0y9IcnKS3xhjfGaDR2NhvDEEAJbn+7N227hfqKonJLksySOTPC5rTwP/xxlnYyHsBALAwqx2A/95kpdnLf6ek+QBSX4hyTnuG8wUdgIBYIHGGB9O8oy552C57AQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANbZt7AABgcznr9O3Zt2f33GOwzuwEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADdUYY+4ZAIBNoqquPOmkk04588wz5x6FI7jsssty8ODBq8YYp97Wa4lAAODvVdW1SU5M8v/mnqWBnavj+47hZ3Yk+bsxxv1v6y/fdlsvAABsKZcmyRhj19yDbHVVtS+Z72/tNYEAAA2JQACAhkQgAEBDIhAAoCERCADQkI+IAQBoyE4gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQADY4qrqPlX1a1X10aq6tqr2V9WFVXX3Y7zOKauf27+6zkdX173Pes2+NMfjb11VX11VP1dVf1RVV1XVqKq3HvdZfVg0AGxdVfWAJG9Lcs8kv5fkfUkekeRxSd6f5NwxxpUTrnPq6joPSvLHSf5Pkp1JvinJJ5KcM8a4fD3+MyzFcfxb/27W/q6fS/KXSc5K8mdjjMccz3ntBALA1vbLWYuS88cYTxpjXDDGeHySn0/y4CQ/NfE6P521APz5McYTVtd5UpIfXF3/l9dh9qU5Xn/rF2Ut/O6c5BvWZdLYCQSALauqzkjywST7kzxgjHHjIefukuRjSSrJPccYn7mZ65yc5JNJbkzyRWOMqw85d8Lqd+xY/Y6Wu4HH6299hOvuSPKh2AkEAI7B41fHNx0aJUmyCrk/S3KnJI+6heuck+SkrIXI1YeeWF33TatvH3ebJ16u4/W33jAiEAC2rgevjh84yvm/WB0ftEHX2coW9zcSgQCwdW1fHQ8c5fxNj99tg66zlS3ubyQCAaCvWh1v6xsEjtd1trJN9zcSgQCwdd20+7T9KOfveti69b7OVra4v5EIBICt6/2r49Feh/bA1fFor2M73tfZyhb3NxKBALB1XbQ6PnH1US5/b/WxJecmOZjkHbdwnXes1p27+rlDr3NCkice9vs6Ol5/6w0jAgFgixpjfDBrH9+yI8l5h51+QZKTk/zGoZ9bV1U7q2rnYde5JskrVuuff9h1/t3q+m/s+hmByfH7W28kHxYNAFvYEW5ldlmSR2btM/0+kOTRh97KrKpGkowx6rDrHH7buIuTnJl/uG3co1ch1NZx/Fs/JskzV9/eOcm3ZO1v/Pqb1owxvus2zysCAWBrq6r7Jnlhkq9JcmrW7l7xu0leMMa46rC1RwyT1blTkjwvyZOSfFGSK7MWJj8xxvjIev5nWIrj8beuqu9K8rKb+z1H+u/nmGcVgQAA/XhNIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQ/8fSjj26hHxepoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f7350152d30>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 521,
       "width": 320
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_iter = iter(train_loader)\n",
    "images, labels = data_iter.next()\n",
    "\n",
    "images.resize_(64, 1, 784)\n",
    "\n",
    "image_index = 0\n",
    "ps = model.forward(images[image_index, :])\n",
    "\n",
    "image = images[image_index]\n",
    "utils.view_classify(image.view(1, 28, 28), ps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well, our networks is not that smart... This shouldn't be a surprise as we haven't trained it yet."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Another way of building networks\n",
    "\n",
    "If you're a fan of functional programming or really dig Keras `Sequential` API, then the following cell will look familiar to you.\n",
    "\n",
    "Basically, we're defining our network as a list of layers or operations the input tensor will be passe through sequentially."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential(\n",
      "  (0): Linear(in_features=784, out_features=128, bias=True)\n",
      "  (1): ReLU()\n",
      "  (2): Linear(in_features=128, out_features=64, bias=True)\n",
      "  (3): ReLU()\n",
      "  (4): Linear(in_features=64, out_features=10, bias=True)\n",
      "  (5): Softmax()\n",
      ")\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApMAAAQTCAYAAAA4dqIfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X20pndd3/vPN5kQYoCBwIFgtAyJgUwJohkFAhQJKEWHUh57XC44QhWfsFSR1hRREq2eQYoCejy2GEhFVxVwga1BTJBU5CFiJ6gnEB4EppXwJImOAYYQMr/+se/Y6ZyZzLW/e+/72nvm9Vprr2v2ff3mmu/cZDHv/bufaowRAADoOGnuAQAA2LrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaNs29wAAwNpV1ceT3C3JvplHYevYkeTvxhj3X8tFxCQAHB/udtppp52xc+fOM+YehK3h+uuvz4EDB9Z8HTEJAMeHfTt37jxj7969c8/BFrFr165ce+21+9Z6Hc+ZBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaNs29wAAwPq47ob92XHxFXOPsWXt27N77hG2JDuTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBIAlqRX/vKquqaqbq+qLVfW+qnp+VZ0893zQISYBYHn+Y5LLktw/yW8neXWSOyV5ZZLfrqqacTZo2Tb3AABwIqiqJyd5VpKPJ3noGONzi9tPSfL6JE9L8t1JLp9rRuiwMwkAy/HUxfHlt4dkkowxbk3yk4tv/8XSp4I1EpMAsBxnLo4fO8K522+7oKruvqR5YF14mBsAluP23cj7H+Hc2Yf8+rwk1xztIlW19yinzmvOBWtiZxIAluP3FscXVNUZt99YVduSXHrIunssdSpYIzuTALAcv5XkmUm+PckHquo/J/likm9Nck6SjyQ5N8ltd3SRMcauI92+2LG8YD0HhinsTALAEowxDiZ5UpIXJvl0Vl7Z/c+TfCLJo5LcuFj62VkGhCY7kwCwJGOMryR5+eLr71XVaUm+IcmBJO+fYTRoszMJAPN7VpI7J3n94q2CYMsQkwCwJFV1tyPc9s1J9iT5fJKfXvpQsEYe5gaA5bmqqg4kuS7JzUkelOQ7ktyS5KljjCO9ByVsamISAJbnjUm+Myuv6j4tySeT/FqSPWOMfTPOBW1iEgCWZIzxsiQvm3sOWE+eMwkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANC2be4BAID1cf5Z27N3z+65x+AEY2cSAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIC2bXMPAACsj+tu2J8dF18x9xjHlX17ds89wqZnZxIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQBYoqraXVVXVtUnqupAVX2sqt5QVRfOPRt0iEkAWJKqemmS30tyQZK3JnllkmuT/NMk76qqZ844HrRsm3sAADgRVNWZSV6Y5DNJvn6M8dlDzl2U5O1JfjrJb8wzIfTYmQSA5bhfVv7d/ZNDQzJJxhhXJ7k5yf8xx2CwFmISAJbjI0m+nOShVXWvQ09U1aOT3DXJ2+YYDNbCw9wAsARjjJuq6seT/EKSD1TVm5PcmOScJE9KclWS7z/Wdapq71FOnbdes8JqiEkAWJIxxiuqal+S1yR57iGn/jLJ5Yc//A1bgYe5AWBJqupfJ3ljksuzsiN5epJdST6W5Der6uePdY0xxq4jfSX54AaODkclJgFgCarqMUlemuQ/jzFeMMb42Bjji2OMa5M8JckNSX6sqs6ec05YLTEJAMvxxMXx6sNPjDG+mOS9Wfl3+RuXORSslZgEgOU4dXE82tv/3H77l5cwC6wbMQkAy/HHi+P3VdVZh56oqm9P8sgkX0ry7mUPBmvh1dwAsBxvzMr7SH5rkuur6k1JPp1kZ1YeAq8kF48xbpxvRFg9MQkASzDGOFhV35HkeUm+MysvuvmqJDcleUuSV40xrpxxRGgRkwCwJGOMW5O8YvEFxwXPmQQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGjbNvcAAMD6OP+s7dm7Z/fcY3CCsTMJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAEDbtrkHAADWx3U37M+Oi6+Yeww2mX17dm/o9e1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAEtQVc+uqnGMr9vmnhNWa9vcAwDACeLPklx6lHP/KMljk/z+8saB9SEmAWAJxhh/lpWg/P+pqvcsfvkfljcRrA8PcwPAjKrq/CQPT3JDkitmHgdWzc4kAMzr+xfHy8YYx3zOZFXtPcqp89ZvJJjOziQAzKSqTkvyzCQHk/zazONAi51JAJjPP0ty9yRXjDH+aspvGGPsOtLtix3LC9ZxNpjEziQAzOf7Fsd/P+sUsAZiEgBmUFX/MMkjknwiyVtmHgfaxCQAzGNVL7yBzUpMAsCSVdWdkzwrKy+8uWzmcWBNxCQALN8zktwjyVumvvAGNisxCQDLd/sLb3ziDVuemASAJaqqnUkeFS+84TjhfSYBYInGGNcnqbnngPViZxIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBt29wDAADr4/yztmfvnt1zj8EJxs4kAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABt2+YeAABYH9fdsD87Lr5i7jG2vH17ds89wpZiZxIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBt29wDbJRvO+kZY+4ZALquOviGmnsGgCnsTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAWDJquofVdXvVNWnquqWxfHKqvqOuWeD1TpuP04RADajqnpxkp9J8rkkv5fkU0nuleQbkzwmyVtmGw4axCQALElVPSMrIfm2JE8dY9x82PlTZhkM1sDD3ACwBFV1UpKXJvliku86PCSTZIxx69IHgzWyMwkAy/GIJPdP8sYkf1NVu5Ocn+RLSd47xnjPnMNBl5gEgOX45sXxM0muTfLgQ09W1TuSPH2M8dd3dJGq2nuUU+eteUJo8DA3ACzHvRfHH0hyWpJvTXLXrOxO/kGSRyd5wzyjQZ+dSQBYjpMXx8rKDuSfL75/f1U9JcmHk3xLVV14Rw95jzF2Hen2xY7lBes5MExhZxIAluNvFsePHRKSSZIxxoGs7E4myUOXOhWskZgEgOX40OL4t0c5f3tsnraEWWDdiEkAWI53JPlKknOr6k5HOH/+4rhvaRPBOhCTALAEY4zPJfntJNuT/NSh56rq25L84yT7k7x1+dNBnxfgAMDyvCDJw5L8RFU9Osl7k9wvyVOS3JbkuWOMoz0MDpuSmASAJRljfLaqHpbkxVkJyIcnuTnJFUn+7zHGNXPOBx1iEgCWaIxxU1Z2KF8w9yywHjxnEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQtm3uAQCA9XH+Wduzd8/uucfgBCMmOe6dfPftk9d+8NKdk9Z95Om/MvmaBzMmr90MTkpNXnvRdU+bvPbUn7v79Bn+6H2T1wIwLw9zAwDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGjzcYpsSV953K7Jax/xC++ZvPZN93rbpHUHV/Fz2MEcnLx2c5j+d7vq/NdPXnvzb3x58tqLXvmvJq/96n/37slrAVh/diYBAGgTkwAAtIlJAADaxCQAAG1iEgCANq/mBoDjxHU37M+Oi6+Ye4wtZ9+e3XOPsKXZmQQAoE1MAgDQJiYBAGgTkwAAtHkBDpvGaj4i8b4/89HJay++1593xmGd3fWkO01e+xPP/U+T177iU//npHXbf/OaydcEYDo7kwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADafJwiG6p2PWjy2kf/wnsmr/URice3p9zls5PX/vKzJq79zeYwANwhO5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2nycIhtqzxtfM3ntzjv52eYff98PTV5761dNv78+9cRbJ6/90Le+evLazeAPH/zbk9Y9Kd+8wZMAnJj86w0AQJuYBACgTUwCwJJU1b6qGkf5+vTc80GH50wCwHLtT/KKI9z++WUPAutBTALAcv3tGOOSuYeA9eJhbgAA2uxMAsBynVpVz0zyD5J8IclfJHnHGOO2eceCHjEJAMt1ZpLXHXbbx6vqOWOMPzrWb66qvUc5dd6aJ4MGD3MDwPK8NsnjshKUpyd5cJJ/n2RHkt+vqofMNxr02JkEgCUZY1x62E3XJfmBqvp8kh9LckmSpxzjGruOdPtix/KCdRgTVkVMkiQ5+YFfN3ntF3/pK5PXPvhO105eezBj8tq5/dAnHj157Q3PvM/ktad+5E+nr528Mjn3DdPXPuI5Pzx57e9c+rLJa+978mnTh4ATz69mJSan/58LbBIe5gaA+X12cTx91imgQUwCwPwuXBw/NusU0CAmAWAJqupBVXXGEW6/X5JfXnz7G8udCtbOcyYBYDmekeTiqro6yceT3JzknCS7k9w5yVuS/Lv5xoMeMQkAy3F1kgcm+casPKx9epK/TfLOrLzv5OvGGFvnlYiwICYBYAkWb0h+zDclh63GcyYBAGgTkwAAtIlJAADaxCQAAG1egHMc+8rjjvjxrUf0U69+9eS133TqbZPXHlzFzysHc3Dy2tX4uc99w+S1V7/kkZPWnfbm965igq31HsRnvPY9k9e+6LlPnLz2svtd1RkHgE3OziQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANh+nuMWcdPrpk9ee+uJPTV67mo9I3Azed8v0n4Pe/Npvmbz2zDe/uzMOAJyw7EwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaPNxilvMRy87Z/La/+8Bl23gJOvvT245ZfLaS7/3eyavPfPtPiLxePa2A3edewSAE5qYBIDjxPlnbc/ePbvnHoMTjIe5AQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0OZNyzeJbfc9c9K6Fz7kbRs8yfp6/5e/MnntJT/w/ZPXnvL2/9YZh+PQv3jXd01ad26u3eBJAE5MdiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQ5k3LAeA4cd0N+7Pj4ivmHuOEtm/P7rlHWDo7kwAAtNmZ3CT2PfvsSeuec7fV/MS5MT8rrOYjEn/8/g+bvPaU+IjEzeCk00+fvPa+d94//bob9N9j/c2dNuS6AExjZxIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm49T3CRe/j2XTVp3MAc3eJJje+o7fnDy2nNz7QZOwka48elfP3ntv73Pqyav3aj/cr/ut764QVcGYAo7kwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADafJziJnHV/gdNWnfRaX+ywZMc29mXzz0BG+l3f+Zlq1h96obM8MOfeMzktbX3g5PWjeYsANwxO5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTALAjKrqWVU1Fl/fO/c8sFpiEgBmUlVfm+SXknx+7lmgS0wCwAyqqpK8NsmNSX515nGgTUwCwDyen+SxSZ6T5AszzwJtPk5xk3jT+79h0rqfO3P+j1M8cK9TJq+9ywbOwXQ3f+fDJ6+918l7J689mIOdcY7pmk/eb/Lar771AxsyA2ykqtqZZE+SV44x3lFVj517JugSkwCwRFW1LcnrkvyPJC9q/P6j/cR33lrmgi4xCQDL9VNJvjHJo8YYB+YeBtZKTALAklTVQ7OyG/nyMcZ7OtcYY+w6yrX3JrlgDeNBixfgAMASHPLw9oeT/OTM48C6EZMAsBx3SfKAJDuTfOmQNyofSV6yWPPqxW2vmG1KWCUPcwPActyS5LKjnLsgK8+jfGeSDyVpPQQOcxCTALAEixfbHPHjEqvqkqzE5H8cY/zaMueCtfIwNwAAbWISAIA2MQkAMxtjXDLGKA9xsxV5ziSrds4Lrp+89jNv2MBBTnAnn3v25LVPf/GVGzjJND/+6Qsnr73PK++8gZMAsJ7sTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBo83GKm8Td333qpHUnPW7+/r/sH1w9ee1JN9TktRdd97TJa0956RmT1576mc9PXnvb+z80ad248CGTr7lRrn/2nSav/d17fGTy2lPq5Mlrbx2Tl+bKNz908tqv/a/vnn5hAGY1f5kAALBliUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANp8nOImceZvfWDSun/7g18/+ZovutefdcdZR9N/Xrnq/NdPv+zrpi/90K23TV778598wqR1r73fayZf82AOTl67UVYzwWo+IvEf/tH3TF57zs/vnbx2FSMAMDM7kwAAtIlJAADaxCQAAG1iEgCANi/AAYDjxPlnbc/ePbvnHoMTjJ1JAADaxCQAAG1iEgCANjEJAECbmAQAoM2ruTeJ2/52/6R17/zRh0++5pN+4v6T115+7m9NXnvGyadOXrsZPPCUkyevvex+V01cefz+HLZzFR+R+HXPuX7y2nHLLZ1xANjkjt9/EQEA2HBiEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANh+nuMVse/ve6YvfPn3pdz3hRyev/fKP3Dh57X998BumD0FuPvjlyWsv+m/Pnbz21Ldsn7z2nFe/Z/LaMXklAMcrMQkAx4nrbtifHRdfMfcYJ6R9e3bPPcJsPMwNAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANm9aTpLkTm/901WsnX7dJz78OZPXfuzJp09ee9t9b5k+xMxO/fidJ6+933/ZP3ntV+99f2ccAFhXdiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtPk4RTbWNX8xeenZ12zgHFvEmHsAAFglO5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkACxJVb20qv6wqv6qqg5U1U1V9b6qeklV3XPu+aBDTALA8vxoktOTXJXklUl+M8lXklyS5C+q6mvnGw16ts09AACcQO42xvjS4TdW1c8meVGSf5Pkh5Y+FayBnUkAWJIjheTC6xfHc5c1C6wXMQkA8/sni+NfzDoFNHiYGwCWrKpemOQuSbYn+aYkj8pKSO6Z8Hv3HuXUees2IKyCmASA5Xthkvsc8v1bkzx7jPHXM80DbWISAJZsjHFmklTVfZI8Iis7ku+rqieOMa49xu/ddaTbFzuWF6z3rHAsnjMJADMZY3xmjPGmJI9Pcs8kvz7zSLBqYhIAZjbG+O9JPpDkQVV1r7nngdUQkwCwOXz14njbrFPAKolJAFiCqjqvqs48wu0nLd60/N5J3j3G+JvlTwd9XoADAMvxhCQvq6p3JPlokhuz8orub0lydpJPJ3nufONBj5gEgOV4W5L/kOSRSR6S5O5JvpDkw0lel+RVY4yb5hsPesQkACzBGOO6JM+bew5Yb54zCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtx+37TF518A019wwAAMc7O5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABt2+YeAABYH+eftT179+yeewxOMHYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBo2zb3AADA+rjuhv3ZcfEVc49xXNq3Z/fcI2xadiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmASAJaiqe1bV91bVm6rqL6vqQFXtr6p3VtX3VJV/k9mSts09AACcIJ6R5P9N8qkkVyf5H0nuk+SpSX4tybdX1TPGGGO+EWH1xCQALMeHkzwpyRVjjIO331hVL0ry3iRPy0pY/s4840GPLXUAWIIxxtvHGP/l0JBc3P7pJL+6+PYxSx8M1sjOJADM79bF8SvHWlhVe49y6rz1GwemszMJADOqqm1J/q/Ft2+dcxbosDMJAPPak+T8JG8ZY/zBsRaPMXYd6fbFjuUF6zwbHJOdSQCYSVU9P8mPJflgkmfNPA60iEkAmEFVPS/JK5N8IMlFY4ybZh4JWsQkACxZVf1Ikl9Ocl1WQvLTM48EbWISAJaoqn48yS8m+bOshORnZx4J1kRMAsCSVNVPZuUFN3uTPG6M8bmZR4I182puAFiCqvruJD+d5LYkf5zk+VV1+LJ9Y4zLlzwarImYBIDluP/ieHKSHznKmj9KcvlSpoF14mFuAFiCMcYlY4w6xtdj5p4TVktMAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtG2bewAAYH2cf9b27N2ze+4xOMHYmQQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoG3b3AMAAOvjuhv2Z8fFV8w9xpa3b8/uuUfYUuxMAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCWpKqeXlW/VFUaNqE7AAAJTUlEQVR/XFV/V1Wjqn5j7rlgLbbNPQAAnEBenOQhST6f5BNJzpt3HFg7O5MAsDw/muQBSe6W5AdnngXWhZ1JAFiSMcbVt/+6quYcBdaNnUkAANrsTALAFlJVe49yyvMvmYWdSQAA2uxMAsAWMsbYdaTbFzuWFyx5HLAzCQBAn5gEAKBNTAIA0CYmAQBo8wIcAFiSqnpykicvvj1zcbywqi5f/PpzY4wXLn0wWAMxCQDL8w1Jvvuw285efCXJf08iJtlSPMwNAEsyxrhkjFF38LVj7hlhtcQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBA27a5BwAA1sf5Z23P3j275x6DE4ydSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAALSJSQAA2sQkAABt2+YeAABYH9fdsD87Lr5i7jFOGPv27J57hE3BziQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBYImq6muq6jVV9cmquqWq9lXVK6rqHnPPBh3b5h4AAE4UVXVOkncnuXeS303ywSQPTfIvkzyhqh45xrhxxhFh1exMAsDy/EpWQvL5Y4wnjzEuHmM8NskvJnlgkp+ddTpoEJMAsARVdXaSxyfZl+T/Oez0S5J8Icmzqur0JY8GayImAWA5Hrs4XjnGOHjoiTHGzUneleSrkjx82YPBWnjOJAAsxwMXxw8f5fxHsrJz+YAkf3i0i1TV3qOcOq8/GvTZmQSA5di+OO4/yvnbb7/7EmaBdWNnEgA2h1ocxx0tGmPsOuJvXtmxvGC9h4JjsTMJAMtx+87j9qOcv9th62BLEJMAsBwfWhwfcJTz5y6OR3tOJWxKYhIAluPqxfHxVfW//ftbVXdN8sgkB5Jcs+zBYC3EJAAswRjjo0muTLIjyfMOO31pktOT/PoY4wtLHg3WxAtwAGB5figrH6f4qqp6XJLrkzwsyUVZeXj7J2acDVrsTALAkix2J78pyeVZicgfS3JOklcludDncrMV2ZkEgCUaY/xVkufMPQesFzuTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbdvmHgAAWB/nn7U9e/fsnnsMTjB2JgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANrEJAAAbWISAIA2MQkAQJuYBACgTUwCANAmJgEAaBOTAAC0iUkAANpqjDH3DADAGlXVjaeddtoZO3funHsUtojrr78+Bw4cuGmMcc+1XEdMAsBxoKpuSXJykj+fe5YTwHmL4wdnnWLtdiT5uzHG/ddykW3rMwsAMLPrkmSMsWvuQY53VbU3cV/fznMmAQBoE5MAALSJSQAA2sQkAABtYhIAgDZvDQQAQJudSQAA2sQkAABtYhIAgDYxCQBAm5gEAKBNTAIA0CYmAQBoE5MAsElV1ddU1Wuq6pNVdUtV7auqV1TVPVZ5nTMWv2/f4jqfXFz3azZq9q1mPe7rqvq2qnp5Vf1hVd1UVaOq3rmRc28G3rQcADahqjonybuT3DvJ7yb5YJKHJrkoyYeSPHKMceOE69xzcZ0HJHl7kj9Ncl6Sf5rks0kuHGN8bCP+DlvFOt7Xb87K/fqlJH+Z5Pwk7xpjPGqDRt8U7EwCwOb0K1mJm+ePMZ48xrh4jPHYJL+Y5IFJfnbidX4uKyH5i2OMxy2u8+Qk/3Jx/V/ZgNm3mvW6r1+alYC8S5J/siGTbkJ2JgFgk6mqs5N8NMm+JOeMMQ4ecu6uST6VpJLce4zxhTu4zulJ/jrJwST3HWPcfMi5kxZ/xo7Fn3FC7k6u1319hOvuSPLx2JkEAGbw2MXxykPjJkkWQfiuJF+V5OHHuM6FSU7LStDcfOiJxXWvXHx70Zon3rrW674+YYlJANh8Hrg4fvgo5z+yOD5gSdc5nrmP1khMAsDms31x3H+U87fffvclXed45j5aIzEJAFtPLY5rfeHDel3neOY+OgYxCQCbz+27YduPcv5uh63b6Oscz9xHayQmAWDz+dDieLTn6Z27OB7teX7rfZ3jmftojcQkAGw+Vy+Oj1+8hc/fW7xdzSOTHEhyzTGuc81i3SMXv+/Q65yU5PGH/XknovW6r09YYhIANpkxxkez8rY9O5I877DTlyY5PcmvH/q+h1V1XlWdd9h1Pp/kdYv1lxx2nR9eXP8PTtT3mEzW774+kXnTcgDYhI7wEX/XJ3lYVt4T8sNJHnHoR/xV1UiSMUYddp3DP07xvUl25n99nOIjFkF1wlrH+/pRSb538e1dkjwtK/fx79++Zozx7I36e8xFTALAJlVVX5vkp5M8Ick9s/JpLG9OcukY46bD1h4xcBbnzkjykiRPTnLfJDdmJXB+aozxiY38O2wV63FfV9Wzk7z2jv6cI/3vs9WJSQAA2jxnEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCANjEJAECbmAQAoE1MAgDQJiYBAGgTkwAAtIlJAADaxCQAAG1iEgCAtv8JYFwMCuT3Ab4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72a1f1cc18>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 521,
       "width": 329
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "input_size = 784\n",
    "hidden_sizes = [128, 64]\n",
    "output_size = 10\n",
    "\n",
    "model = nn.Sequential(nn.Linear(input_size, hidden_sizes[0]),\n",
    "                      nn.ReLU(),\n",
    "                      nn.Linear(hidden_sizes[0], hidden_sizes[1]),\n",
    "                      nn.ReLU(),\n",
    "                      nn.Linear(hidden_sizes[1], output_size),\n",
    "                      nn.Softmax(dim=1))\n",
    "print(model)\n",
    "\n",
    "images, labels = next(iter(train_loader))\n",
    "images.resize_(images.shape[0], 1, 784)\n",
    "ps = model.forward(images[0, :])\n",
    "utils.view_classify(images[0].view(1, 28, 28), ps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem with this way of creating a network is that we end up with obscure indices instead of proper names for the layers. We can solve this passing an `OrderedDict` to the `nn.Sequential` constructor. Each element in an ordered dict will be a pair of the form (_operation name_, _operation_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (fully_connected_1): Linear(in_features=784, out_features=128, bias=True)\n",
       "  (relu_1): ReLU()\n",
       "  (fully_connected_2): Linear(in_features=128, out_features=64, bias=True)\n",
       "  (relu_2): ReLU()\n",
       "  (output): Linear(in_features=64, out_features=10, bias=True)\n",
       "  (softmax): Softmax()\n",
       ")"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import OrderedDict\n",
    "\n",
    "model = nn.Sequential(OrderedDict(\n",
    "            [('fully_connected_1', nn.Linear(input_size, hidden_sizes[0])),\n",
    "             ('relu_1', nn.ReLU()),\n",
    "             ('fully_connected_2', nn.Linear(hidden_sizes[0], hidden_sizes[1])),\n",
    "             ('relu_2', nn.ReLU()),\n",
    "             ('output', nn.Linear(hidden_sizes[1], output_size)),\n",
    "             ('softmax', nn.Softmax(dim=1))]))\n",
    "\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just for fun, let's create a more complex network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (fully_connected_1): Linear(in_features=784, out_features=400, bias=True)\n",
       "  (relu_1): ReLU()\n",
       "  (fully_connected_2): Linear(in_features=400, out_features=200, bias=True)\n",
       "  (relu_2): ReLU()\n",
       "  (fully_connected_3): Linear(in_features=200, out_features=100, bias=True)\n",
       "  (relu_3): ReLU()\n",
       "  (output): Linear(in_features=100, out_features=10, bias=True)\n",
       "  (softmax): Softmax()\n",
       ")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_size = 784\n",
    "layer_sizes = [400, 200, 100]\n",
    "output_size = 10\n",
    "\n",
    "model = model = nn.Sequential(OrderedDict(\n",
    "            [('fully_connected_1', nn.Linear(input_size, layer_sizes[0])),\n",
    "             ('relu_1', nn.ReLU()),\n",
    "             ('fully_connected_2', nn.Linear(layer_sizes[0], layer_sizes[1])),\n",
    "             ('relu_2', nn.ReLU()),\n",
    "             ('fully_connected_3', nn.Linear(layer_sizes[1], layer_sizes[2])),\n",
    "             ('relu_3', nn.ReLU()),\n",
    "             ('output', nn.Linear(layer_sizes[2], output_size)),\n",
    "             ('softmax', nn.Softmax(dim=1))]))\n",
    "\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoEAAAQTCAYAAAAbLfPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xu8Z3dd3/v3JxkuIUAgoVwagkMQmJRRMNMiEBQBpeIUCiKnl4ccsUe0igcVOTW1IoEePUM9VMCKaEFRbB9F6JFagxrEHBAw5XRQOSOBcBuF4Z5ASGCIJPmeP/YvPeMwk1lJZu/1W/vzfD4e+7Gyf+s7a3/YXObF93dZNcYIAAC9nDL3AAAAbD0RCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhnbMPQAAsF6q6iNJ7prk4Myj8NV2JvnCGOP+t/VCIhAAONpdTzvttDPPO++8M+cehL/p8ssvz+HDh0/KtUQgAHC0g+edd96Z+/fvn3sOjrJnz568+93vPngyruU1gQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEM75h4AAFg/Bw5dnZ0XXjz3GO0d3Ld3065tJxAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAsDC14Z9V1WVVdU1Vfamq/rSqnlNVp849H8sgAgFgeX49yauT3D/J65L8+yS3T/KyJK+rqppxNhZix9wDAADTVdVTkjwjyUeSPHyM8dnV47dL8ltJnpbke5K8Zq4ZWQY7gQCwLN+5Or7kpgBMkjHGV5I8f/Xt/7rlU7E4IhAAluXeq+OHj3HupsfOr6q7bdE8LJSngwFgWW7a/bv/Mc6de8Q/70py2c1dqKr2H+fUrlsxFwtjJxAAluV3V8fnVtWZNz1YVTuSvPCIdXff0qlYHDuBALAs/ynJdyd5YpL3VtXvJPlSkm9N8oAkH0jywCQ3nOhCY4w9x3p8tUN4/skamPVkJxAAFmSMcWOSJyd5XpJPZuOdwv8syceSPDrJlauln55lQBbDTiAALMwY4/okL1l9/Q9VdVqShyU5nOQvZhiNBbETCADbxzOS3DHJb60+MgaOSwQCwMJU1V2P8djfS7IvybVJXrTlQ7E4ng4GgOV5c1UdTnIgyTVJHpLkO5Jcl+Q7xxjH+gxB+BtEIAAszxuS/ONsvEv4tCQfT/KqJPvGGAdnnIsFEYEAsDBjjJ9L8nNzz8GyeU0gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQ0I65BwAA1s/us8/I/n175x6DTWQnEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgoR1zDwAArJ8Dh67OzgsvnnuMbevgvr1zj2AnEACgIxEIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAFioqtpbVZdU1ceq6nBVfbiqXl9Vj5x7NtafCASABaqqFyf53STnJ/n9JC9L8u4k/zDJO6rqu2ccjwXYMfcAAMAtU1X3TvK8JJ9K8vVjjE8fce6xSf4oyYuS/OY8E7IEdgIBYHm+Jht/h/+3IwMwScYYlya5JsnfmmMwlkMEAsDyfCDJXyd5eFXd48gTVfXNSe6S5A/nGIzl8HQwACzMGOOqqvqJJP82yXur6o1JrkzygCRPTvLmJD9woutU1f7jnNp1smZlfYlAAFigMcZLq+pgkl9N8qwjTn0wyWuOfpoYjubpYABYoKr6F0nekOQ12dgBPD3JniQfTvIfqurfnOgaY4w9x/pK8r5NHJ01IQIBYGGq6luSvDjJ74wxnjvG+PAY40tjjHcneWqSQ0l+vKrOnXNO1psIBIDl+Qer46VHnxhjfCnJu7Lxd/w3bOVQLIsIBIDlucPqeLyPgbnp8b/egllYKBEIAMvzx6vj91fV2UeeqKonJrkgyZeTvHOrB2M5vDsYAJbnDdn4HMBvTXJ5Vf12kk8mOS8bTxVXkgvHGFfONyLrTgQCwMKMMW6squ9I8uwk/zgbbwa5U5KrkrwpycvHGJfMOCILIAIBYIHGGF9J8tLVF9xiXhMIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0tGPuAQCA9bP77DOyf9/eucdgE9kJBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoaMfcAwAA6+fAoauz88KL5x5jWzq4b+/cIySxEwgA0JIIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIAAsTFU9s6rGCb5umHtO1tuOuQcAAG6xP0vywuOc+6Ykj0vye1s3DkskAgFgYcYYf5aNEPwqVfUnq3/8la2biCXydDAAbBNVtTvJI5IcSnLxzOOw5uwEAsD28QOr46vHGCd8TWBV7T/OqV0nbyTWlZ1AANgGquq0JN+d5MYkr5p5HBbATiAAbA//U5K7Jbl4jPHRKX9gjLHnWI+vdgjPP4mzsYbsBALA9vD9q+MvzzoFiyECAWDhqurvJHlUko8ledPM47AQIhAAlu8WvSEEEhEIAItWVXdM8oxsvCHk1TOPw4KIQABYtqcnuXuSN019QwgkIhAAlu6mN4S4Qwi3iAgEgIWqqvOSPDreEMKt4HMCAWChxhiXJ6m552CZ7AQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAAN7Zh7AABg/ew++4zs37d37jHYRHYCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAa2jH3AADA+jlw6OrsvPDiucdo6eC+vVvyc+wEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADe2Ye4DN8m2nPH3MPQPArfXmG19fc88AbG92AgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBALBgVfVNVfWfq+oTVXXd6nhJVX3H3LOx3rbtbeMAYLurqp9K8q+TfDbJ7yb5RJJ7JPmGJN+S5E2zDcfaE4EAsEBV9fRsBOAfJvnOMcY1R52/3SyDsRieDgaAhamqU5K8OMmXkvzTowMwScYYX9nywVgUO4EAsDyPSnL/JG9I8rmq2ptkd5IvJ3nXGONP5hyOZRCBALA8f291/FSSdyf5uiNPVtXbknzXGOMzN3eRqtp/nFO7bvOErD1PBwPA8txzdfznSU5L8q1J7pKN3cA/SPLNSV4/z2gshZ1AAFieU1fHysaO35+vvv+LqnpqkiuSPKaqHnlzTw2PMfYc6/HVDuH5J3Ng1o+dQABYns+tjh8+IgCTJGOMw9nYDUySh2/pVCyKCASA5Xn/6vj545y/KRJP24JZWCgRCADL87Yk1yd5YFXd/hjnd6+OB7dsIhZHBALAwowxPpvkdUnOSPLTR56rqm9L8veTXJ3k97d+OpbCG0MAYJmem+Qbk/yrqvrmJO9K8jVJnprkhiTPGmMc7+liEIEAsERjjE9X1Tcm+alshN8jklyT5OIk/8cY47I552P9iUAAWKgxxlXZ2BF87tyzsDxeEwgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhHxYNt8Kpd7/75LXXPuaBk9ceesz0/192zkM+OXntHU69fvLag+88Z/Lanc//k8lrAVgvdgIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADbltHBzh+sfvmbTu+3/p9ZOv+ZTT33Jrx7lZbzl8h8lrf/TXnzV57df+5qHJa6ffjA6AdWMnEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEM+LBoA+Cq7zz4j+/ftnXsMNpEIZNu74bHnT177b171S5PWPez2m/NfnR/5+CMnr/3go6df95zr3jl5rbuAAPTg6WAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQ24bxyKdetaZk9eedtGhyWs343ZwX3/ZMyavPecfvX/y2nG9G7wBcOvZCQQAaEgEAgA0JAIBABoSgQAADYlAAICGvDsYAPgqBw5dnZ0XXjz3GNvewX17Z/vZdgIBABoSgQAADYlAAICGRCAAQEPeGMIiXf6vv3by2g9+7StP+s9/wWceOnnt/Z7xkclrb3QrOAC2iJ1AAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQENuG8cinXKXr2zKdX/o0AWT1n30medMvuaNX7ri1o4DAJvGTiAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIbeNY5HOOvPaTbnugavuM2ndnd/rVnAALJudQACAhkQgAEBDIhAAFqiqDlbVOM7XJ+eej/XnNYEAsFxXJ3npMR7fnBdOs62IQABYrs+PMS6aewiWydPBAAAN2QkEgOW6Q1V9d5L7JflikvckedsY44Z5x2IJRCAALNe9k7z2qMc+UlXfO8Z464n+cFXtP86pXbd5Mtaep4MBYJl+LcnjsxGCpyf5uiS/nGRnkt+rqofONxpLYCcQABZojPHCox46kOSfV9W1SX48yUVJnnqCa+w51uOrHcLzT8KYrDERyNr4wj99xOS173jYL05ee+MtmOHwG+81ad2d8+FbcFWALfXKbETgN889COvN08EAsL18enU8fdYpWHsiEAC2l0eujp6y4GaJQABYmKp6SFWdeYzHvybJv1t9+5tbOxVL4zWBALA8T09yYVVdmuQjSa5J8oAke5PcMcmbkvyf843HEohAAFieS5M8OMk3ZOPp39OTfD7J27PxuYGvHWOM+cZjCUQgACzM6oOgT/hh0HBzvCYQAKAhEQgA0JAIBABoSAQCADTkjSGsjSufdHjy2lNSk9e+6LNfN3ntfS75xKR110++IgCsJzuBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIbcNo61setvf2pTrvvT9/h/J6995e9+YdK6V73iSZOvedaBL09ee7urvjR57Y3ved/ktQBwNDuBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIbcNo7Ndcqpk5fe8dSvbMoIV3xl+m3b3viD3zpp3bVPHJOv+Yb/7eWT196Qmrz2f//4Eyev/e8X75689n4/t3/y2nHddZPXArBeRCAA8FV2n31G9u/bO/cYbCJPBwMANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGjIh0WzqU457Y6T1z7kLp/YlBluXzdOX/tXV01ad+6Ffzn5ms9+weMnr/3LC/dMXnvZs14yee2df/D/nrz2MY/+rslr7/K0T01ee+MXvzh5LQCbz04gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhnxYNADwVQ4cujo7L7x47jFaOrhv75b8HDuBAAAN2QlkU92SW4Xt//z9pl/4HgcmL925406T177vR+4zad3X/uj028aN666bvPZ+L3zn5LVPe9uzJ6/9j7/xC5PXvvXr3jB57ZPv+dTJa2/8iNvGAawTO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhtw2jrVx3fXz/8fx3rs+PfcIk5166bsnr33JZy+YvPZn7zn9ugAsl51AAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEPz36cLVk794dMmr732zddNXnvnusPktQ896+OT1n34dreffM3xlb+evBYAtoqdQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAWAbqKpnVNVYfX3f3POw/kQgACxcVZ2T5BeSXDv3LCyHCASABauqSvJrSa5M8sqZx2FBRCAALNtzkjwuyfcm+eLMs7AgbhvH2rjh8g9MXvsvPv64yWt/+b5/MnntK86+bNK6b3rSD0y+5p3+r/82ee0tccqd7jR57T1u96nJa1/9hftOXjs+9/nJa4GTr6rOS7IvycvGGG+rqun/40h7IhAAFqiqdiR5bZK/SvKTt/Ia+49zatetnYvlEIEAsEw/neQbkjx6jHF47mFYHhEIAAtTVQ/Pxu7fS8YY01/zcpQxxp7jXH9/kvNv7XVZBm8MAYAFOeJp4CuSPH/mcVgwEQgAy3LnJA9Kcl6SLx/xAdEjyQtWa/796rGXzjYla8/TwQCwLNclefVxzp2fjdcJvj3J+5Pc6qeK2f5EIAAsyOpNIMe8LVxVXZSNCPz1McartnIulsfTwQAADYlAAICGRCAAbBNjjIvGGOWpYKbwmkAW6QM/9ZDJa//iV/5o8tpdt7vDpHX3+rEPTb7mNb9dk9dmjMlLP/NPHjp57XPv/ouT1+59/5Mmrx2fPzR5LQDrxU4gAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCG3jWORbnfJf5+89slv/6HJa6947KsnrXvduZdMvubu5//w5LX3e9E7J6+94cmfm7z2lvjM6+43ee094rZxAEtlJxAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkNvGse3t+pefmbz2+b/zsEnrfvZe75l8zV9+5ismr33xrz9p8tr/+NBfnbz21LrT5LUA9GAnEACgIREIANCQCAQAaEgEAgA05I0hAMBX2X32Gdm/b+/cY7CJ7AQCADQkAgEAGhKBAAANiUAAgIZEIABAQ94dzLZ3/Uc/Nnntnz/pnEnrfvZN102+5k+cdfn0tZf+18lrH3S7O05e+2+vOnfy2nu/6aOT114/eSUA68ZOIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAht42DI1z/sUOT1r3160+bfM235vxbO85Mpt9mD4DlEoEAwFc5cOjq7Lzw4rnHaOvgvr2b/jM8HQwA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIAAsUFW9uKreUlUfrarDVXVVVf1pVb2gqs6aez7WnwgEgGX6sSSnJ3lzkpcl+Q9Jrk9yUZL3VNU5843GEuyYewAA4Fa56xjjy0c/WFU/k+Qnk/zLJD+05VOxGHYCAWCBjhWAK7+1Oj5wq2ZhmUQgAGwvT1od3zPrFKw9TwcDwIJV1fOS3DnJGUn+bpJHZyMA9034s/uPc2rXSRuQtSUCAWDZnpfkXkd8//tJnjnG+MxM87AQIhAAFmyMce8kqap7JXlUNnYA/7Sq/sEY490n+LN7jvX4aofw/JM9K+vFawIBYBsYY3xqjPHbSZ6Q5KwkvzHzSKw5EQgA28gY4y+TvDfJQ6rqHnPPw/oSgQCw/fzt1fGGWadgrYlAAFiYqtpVVfc+xuOnrD4s+p5J3jnG+NzWT8dSeGMIACzPtyf5uap6W5IPJbkyG+8QfkySc5N8Msmz5huPJRCBALA8f5jkV5JckOShSe6W5ItJrkjy2iQvH2NcNd94LIEIBICFGWMcSPLsuedg2bwmEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGtu3nBL75xtfX3DMAAKwrO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDO+YeAABYP7vPPiP79+2deww2kZ1AAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGdsw9AACwfg4cujo7L7x47jG2pYP79s49QhI7gQAALYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAsDCVNVZVfV9VfXbVfXBqjpcVVdX1dur6n+pKn+/c0I75h4AALjFnp7kl5J8IsmlSf4qyb2SfGeSVyV5YlU9fYwx5huRdScCAWB5rkjy5CQXjzFuvOnBqvrJJO9K8rRsBOF/nmc8lsB2MQAszBjjj8YY//XIAFw9/skkr1x9+y1bPhiLYicQALaXr6yO159oYVXtP86pXSdvHNaVnUAA2CaqakeS/3n17e/POQvrz04gAGwf+5LsTvKmMcYfnGjxGGPPsR5f7RCef5JnY83YCQSAbaCqnpPkx5O8L8kzZh6HBRCBALBwVfXsJC9L8t4kjx1jXDXzSCyACASABauqH03y75IcyEYAfnLmkVgIEQgAC1VVP5Hk55P8WTYC8NMzj8SCiEAAWKCqen423giyP8njxxifnXkkFsa7gwFgYarqe5K8KMkNSf44yXOq6uhlB8cYr9ni0VgQEQgAy3P/1fHUJD96nDVvTfKaLZmGRfJ0MAAszBjjojFGneDrW+aek/UmAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZ2zD0AALB+dp99Rvbv2zv3GGwiO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA3tmHsAAGD9HDh0dXZeePHcYyzWwX175x7hhOwEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEAAWpqq+q6p+oar+uKq+UFWjqn5z7rlYlh1zDwAA3GI/leShSa5N8rEku+YdhyWyEwgAy/NjSR6U5K5JfnDmWVgoO4EAsDBjjEtv+ueqmnMUFsxOIABAQ3YCAaCpqtp/nFNeY9iAnUAAgIbsBAJAU2OMPcd6fLVDeP4Wj8MWsxMIANCQCAQAaEgEAgA0JAIBABryxhAAWJiqekqSp6y+vffq+Miqes3qnz87xnjelg/GoohAAFiehyX5nqMeO3f1lSR/mUQEcrM8HQwpjJ6/AAAIf0lEQVQACzPGuGiMUTfztXPuGVl/IhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhox9wDAADrZ/fZZ2T/vr1zj8EmshMIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaGjH3AMAAOvnwKGrs/PCi+ceo5WD+/Zu6c+zEwgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCADQkAgEAGhKBAAANiUAAWKCqum9V/WpVfbyqrquqg1X10qq6+9yzsQw75h4AALhlquoBSd6Z5J5J/kuS9yV5eJIfSfLtVXXBGOPKGUdkAewEAsDyvCIbAficMcZTxhgXjjEel+Tnkzw4yc/MOh2LIAIBYEGq6twkT0hyMMkvHnX6BUm+mOQZVXX6Fo/GwohAAFiWx62Ol4wxbjzyxBjjmiTvSHKnJI/Y6sFYFq8JBIBlefDqeMVxzn8gGzuFD0rylpu7UFXtP86pXbduNJbETiAALMsZq+PVxzl/0+N324JZWDA7gQCwvdTqOE60cIyx55gX2NghPP9kDsX6sRMIAMty007fGcc5f9ej1sExiUAAWJb3r44POs75B66Ox3vNICQRgQCwNJeujk+oqr/x93hV3SXJBUkOJ7lsqwdjWUQgACzIGONDSS5JsjPJs486/cIkpyf5jTHGF7d4NBbGG0MAYHl+KBu3jXt5VT0+yeVJvjHJY7PxNPC/mnE2FsJOIAAszGo38O8meU024u/HkzwgycuTPNJ9g5nCTiAALNAY46NJvnfuOVguO4EAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDIhAAoCERCADQkAgEAGhIBAIANCQCAQAaEoEAAA2JQACAhkQgAEBDO+YeAABYP7vPPiP79+2deww2kZ1AAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgoRpjzD0DALBGqurK00477czzzjtv7lE4yuWXX57Dhw9fNcY467ZeSwQCAH9DVV2X5NQkfz73LA3sWh3fN3H9ziRfGGPc/7b+4B239QIAwLZzIEnGGHvmHmS7q6r9yTy/a68JBABoSAQCADQkAgEAGhKBAAANiUAAgIZ8RAwAQEN2AgEAGhKBAAANiUAAgIZEIABAQyIQAKAhEQgA0JAIBABoSAQCwDZXVfetql+tqo9X1XVVdbCqXlpVd7+F1zlz9ecOrq7z8dV177tZsy/NyfhdV9W3VdVLquotVXVVVY2qevtJn9WHRQPA9lVVD0jyziT3TPJfkrwvycOTPDbJ+5NcMMa4csJ1zlpd50FJ/ijJ/5NkV5J/mOTTSR45xvjwZvxrWIqT+Lt+YzZ+r19O8sEku5O8Y4zx6JM5r51AANjeXpGNKHnOGOMpY4wLxxiPS/LzSR6c5GcmXudnsxGAPz/GePzqOk9J8iOr679iE2ZfmpP1u35xNsLvzkmetCmTxk4gAGxbVXVukg8lOZjkAWOMG484d5ckn0hSSe45xvjizVzn9CSfSXJjkvuMMa454twpq5+xc/UzWu4Gnqzf9TGuuzPJR2InEAC4BR63Ol5yZJQkySrk3pHkTkkecYLrPDLJadkIkWuOPLG67iWrbx97myderpP1u94yIhAAtq8Hr45XHOf8B1bHB23Rdbazxf2ORCAAbF9nrI5XH+f8TY/fbYuus50t7nckAgGgr1odb+sbBE7WdbaztfsdiUAA2L5u2n064zjn73rUus2+zna2uN+RCASA7ev9q+PxXof2wNXxeK9jO9nX2c4W9zsSgQCwfV26Oj5h9VEu/8PqY0suSHI4yWUnuM5lq3UXrP7ckdc5JckTjvp5HZ2s3/WWEYEAsE2NMT6UjY9v2Znk2UedfmGS05P8xpGfW1dVu6pq11HXuTbJa1frLzrqOj+8uv4fdP2MwOTk/a63kg+LBoBt7Bi3Mrs8yTdm4zP9rkjyqCNvZVZVI0nGGHXUdY6+bdy7kpyX//+2cY9ahVBbJ/F3/egk37f69s5JnpaN3/Hv3bRmjPHM2zyvCASA7a2qzknyoiTfnuSsbNy94o1JXjjGuOqotccMk9W5M5O8IMlTktwnyZXZCJOfHmN8bDP/NSzFyfhdV9Uzk/zazf2cY/37c4tnFYEAAP14TSAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEMiEACgIREIANCQCAQAaEgEAgA0JAIBABoSgQAADYlAAICGRCAAQEP/Hwzo/FkesRGJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72a1f46978>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 521,
       "width": 320
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "images, labels = next(iter(train_loader))\n",
    "images.resize_(images.shape[0], 1, 784)\n",
    "ps = model.forward(images[0, :])\n",
    "utils.view_classify(images[0].view(1, 28, 28), ps)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}