autoencoder/linear-autoencoder/Simple_Autoencoder_Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Simple Autoencoder\n",
    "\n",
    "We'll start off by building a simple autoencoder to compress the MNIST dataset. With autoencoders, we pass input data through an encoder that makes a compressed representation of the input. Then, this representation is passed through a decoder to reconstruct the input data. Generally the encoder and decoder will be built with neural networks, then trained on example data.\n",
    "\n",
    "<img src='notebook_ims/autoencoder_1.png' />\n",
    "\n",
    "### Compressed Representation\n",
    "\n",
    "A compressed representation can be great for saving and sharing any kind of data in a way that is more efficient than storing raw data. In practice, the compressed representation often holds key information about an input image and we can use it for denoising images or oher kinds of reconstruction and transformation!\n",
    "\n",
    "<img src='notebook_ims/denoising.png' width=60%/>\n",
    "\n",
    "In this notebook, we'll be build a simple network architecture for the encoder and decoder. Let's get started by importing our libraries and getting the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The MNIST datasets are hosted on yann.lecun.com that has moved under CloudFlare protection\n",
    "# Run this script to enable the datasets download\n",
    "# Reference: https://github.com/pytorch/vision/issues/1938\n",
    "from six.moves import urllib\n",
    "opener = urllib.request.build_opener()\n",
    "opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
    "urllib.request.install_opener(opener)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np\n",
    "from torchvision import datasets\n",
    "import torchvision.transforms as transforms\n",
    "\n",
    "# convert data to torch.FloatTensor\n",
    "transform = transforms.ToTensor()\n",
    "\n",
    "# load the training and test datasets\n",
    "train_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=True,\n",
    "                                   download=True, transform=transform)\n",
    "test_data = datasets.MNIST(root='~/.pytorch/MNIST_data/', train=False,\n",
    "                                  download=True, transform=transform)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create training and test dataloaders\n",
    "\n",
    "# number of subprocesses to use for data loading\n",
    "num_workers = 0\n",
    "# how many samples per batch to load\n",
    "batch_size = 20\n",
    "\n",
    "# prepare data loaders\n",
    "train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, num_workers=num_workers)\n",
    "test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, num_workers=num_workers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x11bedad30>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAEyCAYAAACbGke8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAD65JREFUeJzt3X+o1XWex/HXa63+yCyV2TVxap0i\nDIv2tpgtjWxF6/SDom5FjNDgUmR/JBgMsuE/U38YspWzSBE6ZGMx4zTQzGaxbEVaLrRIV7My3bYI\na5SbUmaa/ULve/+43+Daev1+POfce8553+cDLvec73nfz/f97Vuvvr+PI0IAkMVftbsBAGglQg1A\nKoQagFQINQCpEGoAUiHUAKRCqAFIhVADkAqhBiCVk0ZzZra5fQFAoz6NiL+uK2JLDUC3+KikqKlQ\ns32N7fdsf2D7vmbGAoBWaDjUbI+T9JikayXNlDTP9sxWNQYAjWhmS222pA8i4sOI+E7SHyTd2Jq2\nAKAxzYTaNEl/GfJ+VzXtKLYX2O6z3dfEvACgyIif/YyIVZJWSZz9BDDymtlS2y3prCHvf1xNA4C2\naSbU3pB0nu2f2D5F0s8lrWtNWwDQmIZ3PyPisO2Fkl6UNE7S6oh4t2WdAUADPJrfUcAxNQBN2BwR\ns+qKuKMAQCqEGoBUCDUAqRBqAFIh1ACkQqgBSIVQA5AKoQYgFUINQCqEGoBUCDUAqRBqAFIh1ACk\nQqgBSIVQA5AKoQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgBSIVQA5AKoQYgFUINQCqEGoBUCDUA\nqRBqAFIh1ACkQqgBSIVQA5AKoQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgBSOWkdjeA7jZu3Lja\nmjPOOGMUOjnawoULi+pOPfXUoroZM2YU1d1zzz21NQ8//HDRWPPmzSuq++abb2prli1bVjTWAw88\nUFTXyZoKNds7JR2UdETS4YiY1YqmAKBRrdhSuzIiPm3BOADQNI6pAUil2VALSS/Z3mx7wbEKbC+w\n3We7r8l5AUCtZnc/50TEbtt/I+ll2/8TERuHFkTEKkmrJMl2NDk/ADiuprbUImJ39XuvpD9Lmt2K\npgCgUQ2Hmu3xtid8/1rSzyRta1VjANCIZnY/p0j6s+3vx/l9RPxnS7oCgAY1HGoR8aGkv2thLxjG\n2WefXVtzyimnFI112WWXFdXNmTOnqG7ixIm1NbfcckvRWJ1s165dRXUrVqyorent7S0a6+DBg0V1\nb731Vm3Na6+9VjRWBlzSASAVQg1AKoQagFQINQCpEGoAUiHUAKRCqAFIhVADkAqhBiAVR4zegzN4\nSsfRenp6iurWr19fW9OOR2ZnMDAwUFR3xx13FNV9+eWXzbRzlP7+/qK6zz//vLbmvffea7adTrC5\n5OnabKkBSIVQA5AKoQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgBSKXZ7/1EEz7++OOius8++6y2\nJsMdBZs2bSqq279/f23NlVdeWTTWd999V1T39NNPF9Wh/dhSA5AKoQYgFUINQCqEGoBUCDUAqRBq\nAFIh1ACkQqgBSIWLb9to3759RXWLFy+urbn++uuLxnrzzTeL6lasWFFUV2Lr1q1FdXPnzi2qO3To\nUG3NBRdcUDTWokWLiurQPdhSA5AKoQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgBSIVQA5CKI2L0\nZmaP3szGmNNPP72o7uDBg0V1K1euLKq78847a2tuv/32orHWrl1bVIcxa3NEzKorYksNQCq1oWZ7\nte29trcNmTbZ9su2369+TxrZNgGgTMmW2m8lXfODafdJeiUizpP0SvUeANquNtQiYqOkHz5O4kZJ\na6rXayTd1OK+AKAhjT56aEpE9FevP5E0ZbhC2wskLWhwPgBwQpp+nlpExPHOakbEKkmrJM5+Ahh5\njZ793GN7qiRVv/e2riUAaFyjobZO0vzq9XxJz7WmHQBoTsklHWsl/bekGbZ32b5T0jJJc22/L+mf\nqvcA0Ha1x9QiYt4wH13V4l7QhAMHDrR0vC+++KJlY911111Fdc8880xR3cDAQDPtIDnuKACQCqEG\nIBVCDUAqhBqAVAg1AKkQagBSIdQApEKoAUiFUAOQCt9RgGMaP358Ud3zzz9fW3P55ZcXjXXttdcW\n1b300ktFdUiH7ygAMPYQagBSIdQApEKoAUiFUAOQCqEGIBVCDUAqhBqAVLj4Fk0599xza2u2bNlS\nNNb+/fuL6jZs2FBb09fXVzTWY489VlQ3mv+dYFhcfAtg7CHUAKRCqAFIhVADkAqhBiAVQg1AKoQa\ngFQINQCpEGoAUuGOAoy43t7eoronn3yyqG7ChAnNtHOUJUuWFNU99dRTRXX9/f3NtIPj444CAGMP\noQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgBSIVQA5AKdxSgY1x44YVFdcuXL6+tueqqq5pt5ygr\nV64sqlu6dGltze7du5ttZ6xqzR0Ftlfb3mt725Bp99vebXtr9XNds90CQCuU7H7+VtI1x5j+64jo\nqX7+o7VtAUBjakMtIjZK2jcKvQBA05o5UbDQ9tvV7umk4YpsL7DdZ7vsixgBoAmNhtrjks6V1COp\nX9IjwxVGxKqImFVygA8AmtVQqEXEnog4EhEDkn4jaXZr2wKAxjQUaranDnnbK2nbcLUAMJpOqiuw\nvVbSFZJ+ZHuXpF9JusJ2j6SQtFPS3SPYIwAU4+JbdJ2JEyfW1txwww1FY5U+Qtx2Ud369etra+bO\nnVs0Fv4fHucNYOwh1ACkQqgBSIVQA5AKoQYgFUINQCqEGoBUCDUAqRBqAFLhjgKMad9++21R3Ukn\n1d5RKEk6fPhwbc3VV19dNNarr75aVDeGcEcBgLGHUAOQCqEGIBVCDUAqhBqAVAg1AKkQagBSIdQA\npEKoAUil7DJpYBRcdNFFRXW33nprbc0ll1xSNFbpnQKltm/fXluzcePGls4TR2NLDUAqhBqAVAg1\nAKkQagBSIdQApEKoAUiFUAOQCqEGIBVCDUAq3FGApsyYMaO2ZuHChUVj3XzzzUV1Z555ZlFdKx05\ncqSorr+/v7ZmYGCg2XZwHGypAUiFUAOQCqEGIBVCDUAqhBqAVAg1AKkQagBSIdQApMLFt2NM6YWr\n8+bNK6orubB2+vTpRWO1Q19fX1Hd0qVLi+rWrVvXTDtoAbbUAKRSG2q2z7K9wfZ22+/aXlRNn2z7\nZdvvV78njXy7AHB8JVtqhyX9MiJmSvoHSffYninpPkmvRMR5kl6p3gNAW9WGWkT0R8SW6vVBSTsk\nTZN0o6Q1VdkaSTeNVJMAUOqEThTYni7pYkmbJE2JiO8fSfCJpCnD/M0CSQsabxEAyhWfKLB9mqRn\nJd0bEQeGfhYRISmO9XcRsSoiZkXErKY6BYACRaFm+2QNBtrvIuJP1eQ9tqdWn0+VtHdkWgSAciVn\nPy3pCUk7ImL5kI/WSZpfvZ4v6bnWtwcAJ6bkmNpPJf1C0ju2t1bTlkhaJumPtu+U9JGk20amRQAo\n58HDYaM0M3v0ZpbIlCnHPAdzlJkzZxaN9eijjxbVnX/++UV17bBp06bamoceeqhorOeeK9vB4BHc\nHWFzybF57igAkAqhBiAVQg1AKoQagFQINQCpEGoAUiHUAKRCqAFIhVADkArfUTACJk+eXFS3cuXK\norqenp7amnPOOadorHZ4/fXXi+oeeeSRoroXX3yxtubrr78uGgv5sKUGIBVCDUAqhBqAVAg1AKkQ\nagBSIdQApEKoAUiFUAOQChffVi699NKiusWLF9fWzJ49u2isadOmFdW1w1dffVVUt2LFitqaBx98\nsGisQ4cOFdUBx8OWGoBUCDUAqRBqAFIh1ACkQqgBSIVQA5AKoQYgFUINQCqEGoBUuKOg0tvb29K6\nVtq+fXttzQsvvFA01uHDh4vqSh+tvX///qI6YLSwpQYgFUINQCqEGoBUCDUAqRBqAFIh1ACkQqgB\nSIVQA5AKoQYgFUfE6M3MHr2ZAchmc0TMqiuq3VKzfZbtDba3237X9qJq+v22d9veWv1c14quAaAZ\nJfd+Hpb0y4jYYnuCpM22X64++3VEPDxy7QHAiakNtYjol9RfvT5oe4ekzv1uNwBj2gmdKLA9XdLF\nkjZVkxbaftv2atuTWtwbAJyw4lCzfZqkZyXdGxEHJD0u6VxJPRrckjvms2psL7DdZ7uvBf0CwHEV\nnf20fbKkFyS9GBHLj/H5dEkvRMSFNeNw9hNAo1p29tOSnpC0Y2ig2Z46pKxX0rZGugSAVio5+/lT\nSb+Q9I7trdW0JZLm2e6RFJJ2Srp7RDoEgBPAxbcAukVrdj8BoJsQagBSIdQApEKoAUiFUAOQCqEG\nIBVCDUAqhBqAVAg1AKkQagBSIdQApEKoAUiFUAOQCqEGIBVCDUAqhBqAVAg1AKkQagBSIdQApFLy\nxSut9Kmkj34w7UfV9G7V7f1L3b8M3d6/1P3LMBr9/21J0ah+8coxG7D7Sr5MoVN1e/9S9y9Dt/cv\ndf8ydFL/7H4CSIVQA5BKJ4TaqnY30KRu71/q/mXo9v6l7l+Gjum/7cfUAKCVOmFLDQBahlADkErb\nQs32Nbbfs/2B7fva1UczbO+0/Y7trbb72t1PCdurbe+1vW3ItMm2X7b9fvV7Ujt7PJ5h+r/f9u5q\nPWy1fV07ezwe22fZ3mB7u+13bS+qpnfTOhhuGTpiPbTlmJrtcZL+V9JcSbskvSFpXkRsH/VmmmB7\np6RZEdE1F03a/kdJX0p6KiIurKb9q6R9EbGs+h/MpIj4l3b2OZxh+r9f0pcR8XA7eythe6qkqRGx\nxfYESZsl3STpn9U962C4ZbhNHbAe2rWlNlvSBxHxYUR8J+kPkm5sUy9jSkRslLTvB5NvlLSmer1G\ng/+CdqRh+u8aEdEfEVuq1wcl7ZA0Td21DoZbho7QrlCbJukvQ97vUgf9QzkBIekl25ttL2h3M02Y\nEhH91etPJE1pZzMNWmj77Wr3tGN33YayPV3SxZI2qUvXwQ+WQeqA9cCJgubMiYi/l3StpHuqXaOu\nFoPHI7rtOp/HJZ0rqUdSv6RH2ttOPdunSXpW0r0RcWDoZ92yDo6xDB2xHtoVarslnTXk/Y+raV0l\nInZXv/dK+rMGd6u70Z7qOMn3x0v2trmfExIReyLiSEQMSPqNOnw92D5Zg2Hwu4j4UzW5q9bBsZah\nU9ZDu0LtDUnn2f6J7VMk/VzSujb10hDb46uDpLI9XtLPJG07/l91rHWS5lev50t6ro29nLDvw6DS\nqw5eD7Yt6QlJOyJi+ZCPumYdDLcMnbIe2nZHQXW6998kjZO0OiKWtqWRBtk+R4NbZ9LgI5x+3w3L\nYHutpCs0+KiYPZJ+JenfJf1R0tkafDTUbRHRkQfjh+n/Cg3u8oSknZLuHnJ8qqPYniPpvyS9I2mg\nmrxEg8ekumUdDLcM89QB64HbpACkwokCAKkQagBSIdQApEKoAUiFUAOQCqEGIBVCDUAq/weIvwgZ\nKAa3AAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "    \n",
    "# obtain one batch of training images\n",
    "dataiter = iter(train_loader)\n",
    "images, labels = dataiter.next()\n",
    "images = images.numpy()\n",
    "\n",
    "# get one image from the batch\n",
    "img = np.squeeze(images[0])\n",
    "\n",
    "fig = plt.figure(figsize = (5,5)) \n",
    "ax = fig.add_subplot(111)\n",
    "ax.imshow(img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Linear Autoencoder\n",
    "\n",
    "We'll train an autoencoder with these images by flattening them into 784 length vectors. The images from this dataset are already normalized such that the values are between 0 and 1. Let's start by building a simple autoencoder. The encoder and decoder should be made of **one linear layer**. The units that connect the encoder and decoder will be the _compressed representation_.\n",
    "\n",
    "Since the images are normalized between 0 and 1, we need to use a **sigmoid activation on the output layer** to get values that match this input value range.\n",
    "\n",
    "<img src='notebook_ims/simple_autoencoder.png' width=50% />\n",
    "\n",
    "\n",
    "#### TODO: Build the graph for the autoencoder in the cell below. \n",
    "> The input images will be flattened into 784 length vectors. The targets are the same as the inputs. \n",
    "> The encoder and decoder will be made of two linear layers, each.\n",
    "> The depth dimensions should change as follows: 784 inputs > **encoding_dim** > 784 outputs.\n",
    "> All layers will have ReLu activations applied except for the final output layer, which has a sigmoid activation.\n",
    "\n",
    "**The compressed representation should be a vector with dimension `encoding_dim=32`.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Autoencoder(\n",
      "  (fc1): Linear(in_features=784, out_features=32, bias=True)\n",
      "  (fc2): Linear(in_features=32, out_features=784, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "# define the NN architecture\n",
    "class Autoencoder(nn.Module):\n",
    "    def __init__(self, encoding_dim):\n",
    "        super(Autoencoder, self).__init__()\n",
    "        ## encoder ##\n",
    "        # linear layer (784 -> encoding_dim)\n",
    "        self.fc1 = nn.Linear(28 * 28, encoding_dim)\n",
    "        \n",
    "        ## decoder ##\n",
    "        # linear layer (encoding_dim -> input size)\n",
    "        self.fc2 = nn.Linear(encoding_dim, 28*28)\n",
    "        \n",
    "\n",
    "    def forward(self, x):\n",
    "        # add layer, with relu activation function\n",
    "        x = F.relu(self.fc1(x))\n",
    "        # output layer (sigmoid for scaling from 0 to 1)\n",
    "        x = F.sigmoid(self.fc2(x))\n",
    "        return x\n",
    "\n",
    "# initialize the NN\n",
    "encoding_dim = 32\n",
    "model = Autoencoder(encoding_dim)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Training\n",
    "\n",
    "Here I'll write a bit of code to train the network. I'm not too interested in validation here, so I'll just monitor the training loss and the test loss afterwards. \n",
    "\n",
    "We are not concerned with labels in this case, just images, which we can get from the `train_loader`. Because we're comparing pixel values in input and output images, it will be best to use a loss that is meant for a regression task. Regression is all about comparing _quantities_ rather than probabilistic values. So, in this case, I'll use `MSELoss`. And compare output images and input images as follows:\n",
    "```\n",
    "loss = criterion(outputs, images)\n",
    "```\n",
    "\n",
    "Otherwise, this is pretty straightfoward training with PyTorch. We flatten our images, pass them into the autoencoder, and record the training loss as we go."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# specify loss function\n",
    "criterion = nn.MSELoss()\n",
    "\n",
    "# specify loss function\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1 \tTraining Loss: 0.636266\n",
      "Epoch: 2 \tTraining Loss: 0.300271\n",
      "Epoch: 3 \tTraining Loss: 0.258592\n",
      "Epoch: 4 \tTraining Loss: 0.249542\n",
      "Epoch: 5 \tTraining Loss: 0.245499\n",
      "Epoch: 6 \tTraining Loss: 0.243327\n",
      "Epoch: 7 \tTraining Loss: 0.241832\n",
      "Epoch: 8 \tTraining Loss: 0.240754\n",
      "Epoch: 9 \tTraining Loss: 0.239949\n",
      "Epoch: 10 \tTraining Loss: 0.239308\n",
      "Epoch: 11 \tTraining Loss: 0.238760\n",
      "Epoch: 12 \tTraining Loss: 0.238279\n",
      "Epoch: 13 \tTraining Loss: 0.237813\n",
      "Epoch: 14 \tTraining Loss: 0.237417\n",
      "Epoch: 15 \tTraining Loss: 0.237069\n",
      "Epoch: 16 \tTraining Loss: 0.236747\n",
      "Epoch: 17 \tTraining Loss: 0.236448\n",
      "Epoch: 18 \tTraining Loss: 0.236169\n",
      "Epoch: 19 \tTraining Loss: 0.235894\n",
      "Epoch: 20 \tTraining Loss: 0.235611\n"
     ]
    }
   ],
   "source": [
    "# number of epochs to train the model\n",
    "n_epochs = 20\n",
    "\n",
    "for epoch in range(1, n_epochs+1):\n",
    "    # monitor training loss\n",
    "    train_loss = 0.0\n",
    "    \n",
    "    ###################\n",
    "    # train the model #\n",
    "    ###################\n",
    "    for data in train_loader:\n",
    "        # _ stands in for labels, here\n",
    "        images, _ = data\n",
    "        # flatten images\n",
    "        images = images.view(images.size(0), -1)\n",
    "        # clear the gradients of all optimized variables\n",
    "        optimizer.zero_grad()\n",
    "        # forward pass: compute predicted outputs by passing inputs to the model\n",
    "        outputs = model(images)\n",
    "        # calculate the loss\n",
    "        loss = criterion(outputs, images)\n",
    "        # backward pass: compute gradient of the loss with respect to model parameters\n",
    "        loss.backward()\n",
    "        # perform a single optimization step (parameter update)\n",
    "        optimizer.step()\n",
    "        # update running training loss\n",
    "        train_loss += loss.item()*images.size(0)\n",
    "            \n",
    "    # print avg training statistics \n",
    "    train_loss = train_loss/len(train_loader)\n",
    "    print('Epoch: {} \\tTraining Loss: {:.6f}'.format(\n",
    "        epoch, \n",
    "        train_loss\n",
    "        ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking out the results\n",
    "\n",
    "Below I've plotted some of the test images along with their reconstructions. For the most part these look pretty good except for some blurriness in some parts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABXUAAADuCAYAAAB28uR+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3WeYFNXW9vGNIkrOOQeRIEhWEVAU\nFUEQFBDFHDBhOOajHkVM14MRI+pzDKBHUQRFkgoiiooSBCRKkBmQHAWJ4rwf3uvZZ60lXXQXHaZm\n/r9Pa1+rp7uY2lNVXfS+u0BOTo4DAAAAAAAAAETDEZneAAAAAAAAAABA/LipCwAAAAAAAAARwk1d\nAAAAAAAAAIgQbuoCAAAAAAAAQIRwUxcAAAAAAAAAIoSbugAAAAAAAAAQIdzUBQAAAAAAAIAI4aYu\nAAAAAAAAAEQIN3UBAAAAAAAAIEIKJvLgAgUK5KRqQ5CwTTk5OeUzvRHxYN7kHjk5OQUyvQ3xYM7k\nKhxrEAbzBmEwbxAG8wZhMG8QBvMGCeM9OEKI+1jDJ3WjKyvTGwAgX+BYgzCYNwiDeYMwmDcIg3mD\nMJg3ANIh7mMNN3UBAAAAAAAAIEK4qQsAAAAAAAAAEcJNXQAAAAAAAACIEG7qAgAAAAAAAECEcFMX\nAAAAAAAAACKEm7oAAAAAAAAAECHc1AUAAAAAAACACOGmLgAAAAAAAABECDd1AQAAAAAAACBCCmZ6\nA4B0uvPOO9W4cOHCvm7atKnq9erVK+bzvPLKK2r8/fff+3r48OGHs4kAAAAAAABAID6pCwAAAAAA\nAAARwk1dAAAAAAAAAIgQ4heQ540YMcLXQZEK1l9//RWzd91116lxp06dfD116lTVy87Ojvs1kX/U\nr1/f14sXL1a9W2+91dcvvPBC2rYJ6VG0aFE1fvLJJ31tjy2zZs1S4969e/s6KysrBVsHAAAAREvp\n0qXVuEaNGnH9nL2e/sc//uHr+fPnq94vv/zi67lz5ya6iUBK8EldAAAAAAAAAIgQbuoCAAAAAAAA\nQIRwUxcAAAAAAAAAIoRMXeQ5MkPXufhzdG2u6WeffebrOnXqqF63bt3UuG7dur7u16+f6j3xxBNx\nvT7yl+bNm/va5jevXr063ZuDNKpcubIaX3vttb62c6Fly5ZqfO655/r6pZdeSsHWIZNatGihxqNG\njfJ1rVq1Uv76Z511lhovWrTI16tWrUr56yN3kdc6Y8aMUb0BAwb4eujQoap34MCB1G4YQqtQoYKv\nP/jgA9X77rvvfP3aa6+p3sqVK1O6XVbJkiXVuEOHDr6eOHGi6u3fvz8t2wQg87p27arG3bt39/Vp\np52mevXq1YvrOWVOrnPO1axZ09dHH310zJ878sgj43p+INX4pC4AAAAAAAAARAg3dQEAAAAAAAAg\nQohfQJ7QqlUrX/fs2TPm4xYsWKDGcsnGpk2bVG/nzp2+LlSokOpNnz5djU844QRfly1bNo4tRn7X\nrFkzX//xxx+qN3r06HRvDlKsfPnyvn777bczuCXIzc4++2w1Dlr2lwo2Wuiqq67ydd++fdO6LUg/\ne/3y8ssvx3zsiy++6Os33nhD9Xbv3p3cDUNopUuXVmN5HWwjDtavX+/rdMctOKe3Z9asWaonz6E2\nlmjZsmWp3TAEKlGihK9t5Nzxxx/v606dOqkesRn4PzLG0DnnbrrpJl/LiDLnnCtcuLAaFyhQ4LBf\nv379+of9HEAm8UldAAAAAAAAAIgQbuoCAAAAAAAAQIRwUxcAAAAAAAAAIiTjmbq9evVSY5mbsmbN\nGtXbs2ePr999913VW7duna/JVsp/Kleu7GubrSPzw2xe4dq1a+N6/jvuuEONGzVqFPOx48aNi+s5\nkb/IXDHnnBswYICvhw8fnu7NQYrdcsstatyjRw9ft2nTJvTzdujQwddHHKH/X3bu3Lm+/vrrr0O/\nBtKrYMH/Xop16dIlg1vy9xzL22+/3ddFixZVPZsFjuiTxxfnnKtWrVrMx7733nu+ltfnyLxy5cr5\nesSIEapXpkwZX9vM5Jtvvjm1G3YIDzzwgK9r166tetddd52veZ+XWf369VPjxx57zNfVq1eP+XMy\ne9c55zZv3pzcDUNk2XPNrbfemvLXXLx4sa/td+4geurVq+dreQ507u/ft3Taaaf5+q+//lK9oUOH\n+vrbb79Vvdx87uGTugAAAAAAAAAQIdzUBQAAAAAAAIAIyXj8wuDBg9W4Vq1acf2cXIbjnHM7duzw\ndSY+Qr969Wpf23/TzJkz0705+c6nn37qa/nxe+f03NiyZUuo5+/bt68aH3XUUaGeB/lXgwYN1Fgu\nZbbLIxF9zz77rBrb5T1hnX/++QetnXMuKyvL1xdeeKHq2WX1yD06duzo65NPPln17PVEqpUuXVqN\nZdRQkSJFVI/4heg7+uij1fj++++P+2dlbFBOTk7StgmHr0WLFr6Wy0ytQYMGpWFrYmvcuLEay6iz\n0aNHqx7XSZkll8c/99xzqle2bFlfBx0LXnjhBTWWMWTOhX+PhtzDLnuXMQp2KfvEiRN9vXfvXtXb\nvn27r+21ho2C+vzzz309f/581fvhhx98/dNPP6ne7t27Y74GcicZZWiPH/I9kZ2HiTjxxBN9/eef\nf6rekiVLfD1t2jTVk3N93759oV8/LD6pCwAAAAAAAAARwk1dAAAAAAAAAIgQbuoCAAAAAAAAQIRk\nPFP32muvVeOmTZv6etGiRarXsGFDX8u8KOd0ZtRJJ52keqtWrfJ19erV4942m6OxceNGX1euXDnm\nz2VnZ6sxmbrpJXMlD8ddd93l6/r16wc+Vmb2yBr4P3fffbcay3nKMSJvGD9+vK+POCI5/2e6efNm\nNd65c6eva9asqXq1a9f29Y8//qh6Rx55ZFK2B4dPZoI559x7773n6+XLl6ve448/npZt+j/nnXde\nWl8PmdWkSRM1btmyZczH2mviCRMmpGSbkLgKFSqo8QUXXBDzsVdffbWv5fuadJE5upMmTYr5OJup\nK78fA+l35513+rpMmTKhnsNm/Xfu3FmNH3vsMV/b/N1MZFQiPjLjVubbOufcCSec4OuePXvGfI7p\n06ersbzPs3LlStWrUaOGGsvvNUrW91cgc+S9wJtuukn15DGkRIkSMZ/jt99+U+NvvvlGjX/99Vdf\n2/fn8jtI2rRpo3ry2NelSxfVmzt3rq+HDh0ac9tShU/qAgAAAAAAAECEcFMXAAAAAAAAACIk4/EL\nkydPDhxLEydOjNkrXbq0r5s1a6Z68mPUrVu3jnvb9uzZo8a//PKLr200hPw4tl0+iWg499xz1XjQ\noEG+LlSokOpt2LBBjf/5z3/6eteuXSnYOkRNrVq11LhVq1ZqLI8nf/zxRzo2CUl26qmnqvFxxx3n\na7sELN4lYXbJjl3Ktn37dl+ffvrpqnf//ffHfN4bbrjB16+88kpc24LUeOCBB9RYLl20y1Fl3Eaq\nyOsXO6dZypi3BS3Tt+yxCLnH008/rcaXXHKJr+V7IOec+/DDD9OyTbG0b9/e1xUrVlS9t956y9fv\nvPNOujYJB2Hjna688sqYj503b56v169fr3qdOnWK+XMlS5ZUYxnx8O6776reunXrYm8s0sq+J/7P\nf/7jaxm34JyOkAqKW7Fs5IJkYy4Rba+++qoay5iOcuXKxfw5e8/w559/9vV9992nevaentS2bVs1\nlu+X3njjDdWT9xjtse6ll17y9UcffaR66Yg64pO6AAAAAAAAABAh3NQFAAAAAAAAgAjhpi4AAAAA\nAAAAREjGM3WTZevWrb6eMmVKzMcFZfYeiswekxm+zukcjxEjRoR+DWSOzTy1mUGS3cdTp05NyTYh\numw2pZWOfB0kn8xKfv/991UvKPtJysrKUmOZvfTwww+rXlBGt32e/v37+7p8+fKqN3jwYF8fc8wx\nqvfiiy/6ev/+/TFfD+H16tXL1126dFG9ZcuW+XrmzJlp26b/I7OYbYbuV1995ett27ala5OQJh06\ndAjs79u3z9dBmd3IrJycHDWWf8dr1qxRPblPU6Vw4cK+ttmGN954o6/tdl911VWp3TDEzX4/TfHi\nxX39zTffqJ683rXXFxdddJGv7VyoW7euGleqVMnXn3zyieqdc845vt6yZUvgtiP5ihUr5mv5PTLO\n6e+k2bRpk+o99dRTvuY7Z/Ive1y4++67fX3NNdeoXoECBXxt3yvL7wR58sknVS/s99OULVtWjY88\n8khfDxw4UPXk93vZ3PFM45O6AAAAAAAAABAh3NQFAAAAAAAAgAjJM/ELqVChQgU1fvnll319xBH6\nfvigQYN8zbKQ6Pj44499fdZZZ8V83LBhw9T4gQceSNk2IW9o0qRJYF8uh0d0FCz439NmvHELzumI\nlr59+6qeXa4WLxu/8MQTT/j6mWeeUb0iRYr42s69MWPG+Hr58uWhtgXBevfu7Wu5L5zT1xbpICNE\nnHOuX79+vj5w4IDqPfroo74mmiNvaNu27UHrg5HLGefMmZOybULqdO3aVY0///xzX9tIFbm0NRE2\nbuq0007z9UknnRTz50aOHBnq9ZB6Rx99tBrLqIxnn3025s/t2bNHjd98801fy/Ogc87VqVMn5vPY\npfrpiA1BbD169PD1vffeq3rZ2dm+bt++vept3749tRuGSJDnBOecu+uuu3wt4xacc+63337ztYw+\ndc65H3/8MdTry0gF55yrXr26r+09nvHjx/vaxq1KdruHDx/u60zElfFJXQAAAAAAAACIEG7qAgAA\nAAAAAECEcFMXAAAAAAAAACKETN0AN910kxqXL1/e11u3blW9JUuWpGWbcHgqV66sxjJPzuZHyZxL\nmSvonHM7d+5MwdYh6mR23JVXXql6P/30kxp/8cUXadkmZMbMmTPV+KqrrvJ12AzdQ5HZuDIn1Tnn\nWrdunZLXxMGVLFlSjYNyJcPmWIbVv39/NZbZ0IsWLVK9KVOmpGWbkD6JHAvSPTcRzpAhQ9S4Y8eO\nvq5SpYrqdejQwdc2E7B79+6hXt8+j8xftVasWOHr++67L9TrIfUuuuiimD2b0yy/nyRIq1at4n79\n6dOnqzHvuzIrKH9dvr9ZvXp1OjYHEWMzbe33N0h//vmnr0888UTV69Wrl68bNGgQ8zl2796txg0b\nNow5tu/JKlasGPN5pfXr16txpr+Dgk/qAgAAAAAAAECEcFMXAAAAAAAAACKE+AXjlFNO8fW9994b\n83E9evRQ4/nz56dsm5A8H330kRqXLVs25mPfeecdXy9fvjxl24S8o1OnTr4uU6aM6k2cOFGN9+zZ\nk5ZtQuoccUTs/xe1S4bSQS6BtdsWtK0DBw709aWXXpr07cqPbJxP1apVff3ee++le3OUunXrxuxx\nLZP3BS2B3rZtmxoTvxANs2bNUuOmTZv6ulmzZqrXuXNnX991112qt3HjRl+//fbbcb/+8OHD1Xju\n3LkxH/vdd9/5mmvr3Muep2Q0h41wkcugmzRpono9e/b0denSpVXPHm9k/9prr1U9OccWLlwYuO1I\nPrns3ZLHlIceekj1PvnkE1/PmTMn+RuGSPjyyy/VWEZ7yffOzjlXo0YNXz///POqFxTtIyMdbNxD\nkKC4hb/++kuNR48e7etbbrlF9dauXRv3a6YCn9QFAAAAAAAAgAjhpi4AAAAAAAAARAg3dQEAAAAA\nAAAgQsjUNbp06eLro446SvUmT57s6++//z5t24TDI3OgWrRoEfNxX331lRrbXCDgUE444QRf29yf\nkSNHpntzkALXX3+9r23WUqZ169bN182bN1c9ua12u2WmLpJjx44daiyz5GTepXM6f3vLli0p2Z4K\nFSr4Oigbb9q0aSl5fWROu3bt1Pjiiy+O+djt27er8erVq1OyTUitrVu3+lpmF9rxPffck5TXq1On\njhrLfHebo3nnnXcm5TWRWpMmTVJjeWywubky4zYo89I+50033aTGY8eO9fWxxx6rejK/Ul6HIT3K\nly/va3sNKb9D4MEHH1S9Bx54wNdDhw5VvenTp/ta5qg659yyZct8vWDBgsBta9y4sa/t/RnOYbnD\n7t271VhmbZcqVUr15Hdaye+6cs65zZs3+zo7O1v15DyU78edc65NmzYJbvH/99prr6nxfffd52ub\nCZ5pfFIXAAAAAAAAACKEm7oAAAAAAAAAECHc1AUAAAAAAACACMn3mbqFCxdW486dO/t63759qicz\nVvfv35/aDUNoZcuWVWOZf2JzkiWb+7Vz587kbhjynEqVKqlx+/btfb1kyRLVGz16dFq2Caklc2sz\nQeaaNWrUSPXksS7Ixo0b1ZjzWfLZ/LDly5f7+oILLlC9cePG+fqZZ54J9XrHH3+8GtuMy1q1avk6\nKPMwt+VE4/DZa6Ijjoj9eY4vvvgi1ZuDPMjmaMpjjM3ttecf5E42371Pnz6+tt8RUbJkyZjP88IL\nL/jazoU9e/ao8ahRo3wtczWdc+7ss8/2dd26dVVPnl+RGk899ZSvb7/99rh/Tp5vbrzxRtWz42Sw\nxxf5fTl9+/ZN+uvh8NlsWvu3H8awYcPUOChT134Hhpzfb731luodOHDgsLctVfikLgAAAAAAAABE\nCDd1AQAAAAAAACBC8n38wl133aXGzZs39/XEiRNV77vvvkvLNuHw3HHHHWrcunXrmI/9+OOPfS3j\nNYB4XHHFFWpcoUIFX0+YMCHNW4P84P777/f1TTfdFPfPrVy50teXX3656mVnZx/2diGYPL8UKFBA\n9bp27err9957L9Tzb9q0SY1txEK5cuXieh671AzR16tXr5g9u+zx1VdfTfXmIA/o3bu3Gl922WVq\nLJezbt68OS3bhNSaNGmSr+0x5eKLL/a1PabIaA4bt2A98sgjvm7YsKHqde/e/aDP6dzfr2mQfHJJ\n/IgRI1TvP//5j68LFtS3lqpXr+7roOifZJERZc7pufrAAw+o3qOPPpry7UH63H333b5OJGrj+uuv\nV+Ow1+GZxid1AQAAAAAAACBCuKkLAAAAAAAAABHCTV0AAAAAAAAAiJB8l6krs+ucc+5f//qXGv/+\n++++HjRoUFq2Ccl1++23x/3YAQMG+Hrnzp2p2BzkYTVr1ozZ27p1axq3BHnV+PHj1fi4444L9TwL\nFy709bRp0w5rm5C4xYsX+7pPnz6q16xZM1/Xq1cv1POPHDkysP/222/7ul+/fjEft3v37lCvj9yl\nWrVqvpZ5l9bq1avVeObMmSnbJuQd55xzTmB/7Nixvp49e3aqNwdpJvN1DzYOS55/bG6rzNTt2LGj\n6pUpU8bXW7ZsScq2QDtw4ICv7Xmifv36MX/ujDPO8PVRRx2legMHDvR10PffHA75HQYtW7ZMyWsg\nM6655ho1lpnJNtvZWrBgga9HjRqV3A3LED6pCwAAAAAAAAARwk1dAAAAAAAAAIiQfBG/ULZsWV8/\n//zzqnfkkUeqsVzqOn369NRuGDJOLtnZv39/6OfZvn17zOeRy01KliwZ8zlKlSqlxvHGSMglMc45\nd8899/h6165dcT0Hwjn33HNj9j799NM0bgnSRS7lOuKI2P8vGrQ89bXXXlPjKlWqxHysfY2//vrr\nUJt4UN26dQv1c0i9OXPmHLROphUrVsT1uOOPP16N58+fn4rNQYq1bdvW10HHqY8//jgdm4M8xp7f\n/vjjDzV++umn07k5yIM++OADNZbxCxdeeKHqySg9ohNzl8mTJ8fsyegpG7/w559/+vrNN99Uvddf\nf12Nb7vtNl8HxQ0h+tq0aeNre54pVqxYzJ+zEZvXX3+9r/fu3ZukrcssPqkLAAAAAAAAABHCTV0A\nAAAAAAAAiBBu6gIAAAAAAABAhOTJTF2bkztx4kRf165dW/WWL1+uxv/6179St2HIdebNm5eU5/nw\nww99vXbtWtWrWLGir20OVCqsW7fO14899ljKXy+/adeuna8rVaqUwS1BJrzyyiu+Hjx4cMzHjR07\nVo2DsnATycmN97FDhw6N+zmR98ksaFlbZOjmDfK7JKxNmzb5esiQIenYHOQBMoNQXtc659yGDRvU\nePbs2WnZJuRd9lpHXm+dd955qvfQQw/5+v3331e9X375JQVbh2T4/PPPfW3frxYs+N9bVNdee63q\n1atXT41PO+20uF5v9erVCW4hchv5/SDFixeP+Tib8y4zuZ1z7ttvv03uhuUCfFIXAAAAAAAAACKE\nm7oAAAAAAAAAECF5Mn6hbt26atyyZcuYj7399tvV2MYxIHrGjx+vxnaZTir07t071M/9+eefvg5a\nVj1mzBg1njlzZszHfvPNN6G2BfHp2bOnr23Uy08//eTrr7/+Om3bhPQZNWqUr++66y7VK1++fMpf\nf+PGjb5etGiR6vXv39/XNgYG+VtOTs5Ba+RNZ599dsxedna2r7dv356OzUEeIOMX7DFk3LhxMX/O\nLpEtXbq0r+VcBILMmTPH1w8++KDqPfnkk75+/PHHVe/SSy/19e7du1O0dQhDXsN+8MEHqtenT5+Y\nP9exY8eYvQMHDqixPDbde++9iW4iMsyeP+6+++64fu7dd99V46+++ipZm5Rr8UldAAAAAAAAAIgQ\nbuoCAAAAAAAAQIRwUxcAAAAAAAAAIiTPZOrWrFnT159//nnMx9kMxLFjx6Zsm5AZ559/vhrL/JWj\njjoq7udp3Lixry+88MK4f+6NN95Q45UrV8Z87EcffeTrxYsXx/0aSJ8iRYqocZcuXWI+duTIkb62\nuU7IG7Kysnzdt29f1evRo4evb7311pS8/mOPPebrl156KSWvgbznmGOOidkjZzD67LWN/W4Jac+e\nPb7ev39/yrYJ+Ye93unXr5+v//GPf6jeggULfH355ZendsOQJw0bNkyNr7vuOl/b94CDBg3y9bx5\n81K7YUiIvPa47bbbVK9YsWK+btWqlepVqFBBjeX77OHDh6vewIEDD3MrkW5y3y9cuFD1gu7jyL9v\nO5/yAz6pCwAAAAAAAAARwk1dAAAAAAAAAIiQPBO/0L9/f1/XqFEj5uOmTp2qxjk5OSnbJuQOgwcP\nPuznuPjii5OwJYgiuzx169atvh4zZozqDRkyJC3bhNzh66+/jjm2MUDyHNWtWzfVk/PotddeU70C\nBQqosV2KBMTjyiuv9PW2bdtU75FHHkn35iDJ/vrrLzWeOXOmr48//njVW7ZsWVq2CfnHNddco8ZX\nX321r//973+rHscbHK6NGzeqcadOnXxtI+/uueceX8tYEOQu69evV2N5nXzppZeq3kknnaTGDz/8\nsK83bNiQgq1DOp1++um+rlatmuoF3beTUT8yZiq/4JO6AAAAAAAAABAh3NQFAAAAAAAAgAjhpi4A\nAAAAAAAAREhkM3XbtWunxjfffHOGtgRAXmYzddu2bZuhLUGUTJw4MXAMpNOMGTN8/cwzz6jelClT\n0r05SLIDBw6o8f333+9rm0E3a9astGwT8pYBAwb4etCgQapn8+VfeeUVX8vvIXDOuX379qVg65Cf\nZWdn+3rSpEmq1717d183atRI9fiOgmgYPnx44Bh5i8xdD8rQffLJJ9U4v1/L8kldAAAAAAAAAIgQ\nbuoCAAAAAAAAQIRENn6hffv2alysWLGYj12+fLmvd+7cmbJtAgAAyG26deuW6U1AGq1Zs8bXV111\nVQa3BHnFtGnTfH366adncEuA2Hr16qXGc+fO9XW9evVUj/gFIPcpU6aMrwsUKKB6GzZs8PVzzz2X\ntm2KAj6pCwAAAAAAAAARwk1dAAAAAAAAAIgQbuoCAAAAAAAAQIRENlM3iMzPcc65M844w9dbtmxJ\n9+YAAAAAAIAU+f3339W4du3aGdoSAGE888wzB62dc+6RRx7x9dq1a9O2TVHAJ3UBAAAAAAAAIEK4\nqQsAAAAAAAAAEVIgJycn/gcXKBD/g5Fqs3JyclpleiPiwbzJPXJycgpkehviwZzJVTjWIAzmDcJg\n3iAM5g3CYN4gDOYNEsZ7cIQQ97GGT+oCAAAAAAAAQIRwUxcAAAAAAAAAIoSbugAAAAAAAAAQIQUT\nfPwm51xWKjYECauZ6Q1IAPMmd2DOIAzmDcJg3iAM5g3CYN4gDOYNwmDeIFHMGYQR97xJ6IvSAAAA\nAAAAAACZRfwCAAAAAAAAAEQIN3UBAAAAAAAAIEK4qQsAAAAAAAAAEcJNXQAAAAAAAACIEG7qAgAA\nAAAAAECEcFMXAAAAAAAAACKEm7oAAAAAAAAAECHc1AUAAAAAAACACOGmLgAAAAAAAABECDd1AQAA\nAAAAACBCuKkLAAAAAAAAABHCTV0AAAAAAAAAiBBu6gIAAAAAAABAhHBTFwAAAAAAAAAihJu6AAAA\nAAAAABAh3NQFAAAAAAAAgAjhpi4AAAAAAAAAREjBRB5coECBnFRtCBK2KScnp3ymNyIezJvcIycn\np0CmtyEezJlchWMNwmDeIAzmDcJg3iAM5g3CYN4gYbwHRwhxH2v4pG50ZWV6AwDkCxxrEAbzBmEw\nbxAG8wZhMG8QBvMGQDrEfazhpi4AAAAAAAAARAg3dQEAAAAAAAAgQripCwAAAAAAAAARwk1dAAAA\nAAAAAIgQbuoCAAAAAAAAQIQUzPQGALlFgQIF1DgnJycpz1uw4H//zP7888+kPCcyR86TZM0R68gj\nj/T1EUfo/3vbv39/Sl4TAJA3yHPIgQMHMrglAPKzo48+2tdHHXWU6u3cuTPdmwMgwtLxHjyq+KQu\nAAAAAAAAAEQIN3UBAAAAAAAAIEK4qQsAAAAAAAAAEUKmLvIcm9lUt25dX3fo0EH1+vfv7+sKFSqo\n3tq1a3393Xffqd7ixYvVeNmyZb6eM2eO6u3YscPXMufOObLuou5wcpjlz9p5Icd//fVXyK1DbmXn\njVSoUCE1tnNKZiqTJwXgYDhvAMgN5DWL/V6RsNfQ8rtKnNPvpbgugiQznStXrqx68v35tm3bVI/3\n57kTf9+x8UldAAAAAAAAAIgQbuoCAAAAAAAAQIQQv4BIOuII/f8RJUuW9PVtt92meldccYWvS5cu\nrXpyWYb9SH+VKlV83aJFC9Wzj503b56v77jjDtX74YcffL1v3z6HvMPOA7mUzMaANGzYUI1r1qzp\naznXnHNu1apVvpbzxznntm4gwEIqAAAgAElEQVTd6muWB0WHjNQoVqyY6nXr1s3XlSpVUr2mTZuq\n8T333OPrdevWqR7LkqLPLkeV46Clqofa90H9eF8jkedEZsW7b+z+PuaYY9RYxsHY3t69e30tl7E6\nx7kpN5NL1+08CZo36Y70CIop4tgTHUHzJmgfB50LbWSZPE7ZuSHfd9ltYR6FY/eNPKbY9+cyfiNV\nxxA5H+R7K+ecu/baa33dqFEj1ZPbM2HCBNUbOXKkr7ds2ZKU7QRSiU/qAgAAAAAAAECEcFMXAAAA\nAAAAACKEm7oAAAAAAAAAECFk6iKSZIauc849+OCDvu7Zs6fq2YxKSea+7dq1S/VkZpDNkrPZuHJs\ns+RknhCiISjnK+xz7Ny5U43lvGzcuLHqbdy4ManbgvSzuWJlypTxdefOnVXvySef9HWpUqVU7/ff\nf1fjTz75xNdjxoxRPY410RA0N+z+l/nLMk/bOee2bdvm6927d6ven3/+GfP17TFF5n/bLHCZOWif\nU47JUE0/OY8SyUaN9RzOOVe4cGE17tChg687duyoel988YWvv/rqK9X7448/Et4WJI/MuDzuuONU\nT14j23NGdna2r2fMmKF6a9eu9bU93iSSlSmPP3b+FS1a1Nc1atRQvapVq/p6yZIlqrd69WpfBx37\nkBpBuexBx6mgeRN0TLP7WM53+50F8vXt9ZTMBeccFkz+Hu3vWL6fsb9jeS6w77Pj/Z3b1+vVq5ca\nX3bZZb5u0qSJ6hUpUsTXNotZ6tSpkxoPGDDA1+eff77qLVu27BBbDKQfn9QFAAAAAAAAgAjhpi4A\nAAAAAAAAREjG4xfsMg350Xi7DFCyyzKClgGmYumX3e6gJdLy9VmGFp5c+lGnTh3Vk8st7LKcX375\nxddy+Zhzzn355Ze+tssHZeRC165dVc8uxZCPlUvEnEv9Po937iH15O/bHofs8Uwuc7XRDHJpoV3K\nlMgyR+Qe7dq18/Xzzz+vejJOxv49y6X5zullZt9++63qrVu37rC3E6khz19yOaBzzlWsWNHXrVq1\nUr1q1ar5eu7cuaonl0fLJY7OJbYcX/bstpUrV87X9ji1ZcuWuF8f4QQta471OCuRfXHyySer8UMP\nPeRre20jff3113G/BpLP/t1eeeWVvr7mmmtUTy5V37x5s+otWrTI1zKKwTl9LRIUxeJc8LJqOVdt\nlJo8TwYto7bLn+Xx1f4tcCxKPjmHnNPHhrPOOkv15NyYMGGC6m3YsEGN491X9jpYxojYXoUKFWI+\njzyn7dmzR/WI8YjNxrasWbPG1/b3Jh9r96+852PnlIz7efvtt1VPXpccinzNoHs3hQoVUr2GDRv6\n+oYbblC9gQMH+nrHjh1xbwsyx0ZvyHOG7SUjWiwT5x0+qQsAAAAAAAAAEcJNXQAAAAAAAACIEG7q\nAgAAAAAAAECEpCxTV+aU2KwKmRFYv3591atevbqvbS6LzOzZtm2b6snHrly5UvVWr17ta5u1Y/MK\npd27d6uxzMew2Ssym2PVqlWqJ/NWyOgJT84pm+22fv16X9vcwdmzZ/t67Nixqif3jZ0bcp/aXNPe\nvXur8bHHHuvrc889V/U+/vhjX+/du9clg9w2m0Mk5xhZYrGlOp/QPq5o0aJq3KxZM18vX75c9eQx\na9++faFen6zlzLLnlldffdXXNkcwkbkoc8aGDBmiejfffLOvN23apHpkMWeWPGbbPLgrrrjC18cd\nd5zqyesQm6G8detWXx/OdwnIx5YoUUL15Hyz1zazZs3y9a5du0K/Pv7L/r3LeWPJa2v79x1vDpy9\nfrj44ovV+Pjjj4+5baVLl/a1vbZh/6eefB/StGlT1ZPXqMWLF1c9Oafk37Bz+hp54cKFqifzRxPJ\n7LbkPCpVqpTqnXbaab62c2rSpEm+lhmezv39OgnJd/TRR/u6Z8+eqvfkk0/6umzZsqon32fJ90rO\nOffcc8+psbxuSeSaRR7v7LyRWbn2/CZ79pgp/064ftK/A3t/RJ5H7DlLzhvbq1Kliq9tFvN9993n\n66BcZLttNuNWfpeO/W4T+bx2H8vvCbCZ8TZ/GakTdE1kr1/kNYnMlXfOueuuuy7mY+33RXzyySe+\nfvfdd1VP3n+02dLyeybk9blz4bN5E8EndQEAAAAAAAAgQripCwAAAAAAAAARkrT4BfuRdrkE3H7c\nvlatWr5u0aKF6sll9XaJYqVKlXxdsWJF1du+fbuv7TKcYsWK+dp+hN8u/ZEfl5Yf2XdOf6xaPqf1\n2muvqbFczkT8QnhyHs2bN0/15Efgf/nlF9X79ddffZ3IEhr52JNPPln15JIR5/QyyJYtW6qeXB6Q\nSPyCXHJg/4bkUn4bbyL/FhCfRJa/x7vM0B4TW7durcYNGjTw9fTp01VPLvsJu6wx7HYjPPk7f/rp\np1VPLvUJmm/2GGX3W5EiRXzdpUsX1ZPnt/fff1/15NLVrKws1eO8FFuy/o7kcbpevXqqJ5ekyggq\n55x75plnfP3DDz+onjzvHc7ftzxWyeOSc86dfvrpvl6yZInq/fTTT6FfEwcXtB/tXEzGkmC7NL9d\nu3ZqLK9f7Ot9//33vmb5e/rJY4q9DpRRKfZaREYu/Pvf/1a9+fPn+zpouWjQOexQ5PVs8+bNVU9e\na3/xxReqN2PGDF/b5d9IPjun5DXs0KFDVU/GGti5Iedf586dY/acc278+PG+njlzpurJiJ+g6yT7\nPku+J7LLpeVzHk6kSH4nf1d2n8pziI2/kOeN7Oxs1ZOxikFRmc459+WXX/r6gQceUD15D8DG4FWr\nVs3XNlZzw4YNvl66dKnqpWMpfX5ijxny2GPvt8iooU6dOqle+/btfW2vZQsXLhzz9Y855hg1lu+t\n7P0fOYdsNKd83/X666+rXlDcZ7LwSV0AAAAAAAAAiBBu6gIAAAAAAABAhHBTFwAAAAAAAAAiJGmZ\nujanRrLZIzKDwmaoyJwkm3Ehezbvds+ePb6WubjOOVezZk1f2yxcm8u0YMECX2/cuFH1Gjdu7GuZ\nC+yczoyxmUHjxo1zOHxyHsl8Y+d0JqnMLXEufHaJnCv33Xef6tmsKZkLZHNUwmZ/yYwZm+0kM1dt\n3m6qslryMpvnE282rd0v8rE29/uss85SY3nMmDNnjuqFzTiNNxuYrLDUOPHEE33du3dv1ZP72/7+\n5f6WGW/O/f3vW+aV2eyyVq1a+bp27dqqd/bZZ/t60KBBqvfzzz/7mqwwLezfiv1blNml1113nerJ\nHHZ7bTN37lxfB2Wy23liyX+HfazMYpbzxDnnatSo4WuZceecPg9x3kkNud/kMcQ5Pcfs7z/ePFSb\n9W6zC+Xr2+uuTz/99KCPQ3rIv2M7N+R1p71eXbFiha8XL16semGP/4lkQZcvX97X9v2SPBfKY59z\n+niD1JBzyl5DvPHGG7622ahB157y/bn9uT59+qhxjx49fD169GjVe/bZZ31tz0Vy/tn7EfKeAOep\n1JDHGPn9Ec7p65bNmzernnzv/Ntvv6mefF/UtWtX1bN5/vJYIeebc3pu2PtD8jXtcVLOFeZNOEHX\npfL3bd8vX3311b62xwj5XVj2/or8zhG7z2z+rcydl1nyzunjVJs2bVSvZMmSvrbny3POOcfXw4YN\nUz25rfbnkvW+i0/qAgAAAAAAAECEcFMXAAAAAAAAACIkafELiZDLC9esWaN68qPTU6ZMUb1ChQr5\n2i5RlexH6Js3b+5ru7TMfuR66dKlvi5atKjqvfzyyzGfR37M225b2KXU0OQSChuxEOtxibDLh159\n9VVflytXTvXsR+UnTpzoazlPnIt/2UbQMgX7b5LPyXLpcILmiT2GSPL3beeMXA7fokUL1ZPHIef0\ncTArKyvubZPijVtI5DkRv7Jly6rx8OHDfW3jgyR7Tvj11199/f3336tepUqV1Lhy5cq+tssMZU+e\nL53T0RDdu3dXvSVLlvg6bFwMNPv3JyObZJSTc3rp9Lx581TPLkmM9RpBxwLn9N+/Pb6de+65vj7z\nzDNV7+ijj/b1rFmzVM8uZ0PyhT1PBc0HeWzq0qVLzJ5z+lpDxi049/cl0GEEbSfnrGDyPGKXNVet\nWtXXdjn0tm3bfJ3I9aO8RrVzL+h5bOzdPffc42t7XSSXUU+dOjXu10ByyP1qjw0yNsOSx4lNmzap\nnoz4sHOhfv36aizPhXbZszwX2WND0LGC9+DJZ4/b8n5JnTp1VE/ufxuNEPT+WC6Pf+uttwK3R0Y8\nJHLekI9lnhw+ex9DHk/sfpH3Vf73f/9X9U455RRf22ihnTt3+tq+X5LvZey1tI05XL16dcxtu/XW\nW319xhlnqJ58n2//vfL6SUaXOafnftA9rMPBJ3UBAAAAAAAAIEK4qQsAAAAAAAAAEcJNXQAAAAAA\nAACIkLRk6trMFJk5G9TbuHFj4PPEYrOe1q1b52ubQWjHQYoXL+5rm1e4detWX3/44YeqRw5UcqQ6\nX61169Zq3LlzZ1/v27dP9RYuXKjGN954Y8zHBok3E5FsueQL+n3Lv9mg7C6bpyMzwZs2bap6JUqU\nUOMvvvjC10EZ4YlgDqWezHd69NFHVa9mzZq+tvtC5nV9/fXXqvfII4/4etmyZaonc3Kd03PF5jaf\nd955vj7ttNNUT+bR2Z+Tc9NmnjFvwpG/b+d0jrE8Tjins0k/+ugj1du+fbuvg/bFoTIG5XyU1zLO\nOXfWWWf52s43OR9ldrxzZNClm92n8vo16PrYZtJVqFDB1yeccILq2etnmV83ZswY1Qu7/w+V/4z4\nyOsPmdntnHNNmjTxdcmSJVXv7LPP9rX9m5bvu+x+khnyMlfQub9nscu8xMGDB6teq1atfL1hwwbV\nGzJkiK+3bNnikHxB7ztk5m3FihVVT+Y2y/OSc85NnjzZ1zZ7u1q1ar6W+e3O/T1TVx5TbGa7fP1k\nXZfIv6F47zHAucKFC6vxLbfc4mu7bxYsWODrRH7H8nlkZq5zfz+nyf2YSN4ykivod22vieV9k1NP\nPVX15D02e09FXoc89NBDqifzvO3PBc0LO59lnrjdbsleA2VnZ/vaHr/S8R0UfFIXAAAAAAAAACKE\nm7oAAAAAAAAAECFpiV+w5Eee7ceh5Ufzw35k3n68Xy4ntb2g16hbt64aN27cOObPyeUF8+bNi/s1\nkFlVqlTx9bBhw1RPfvx/7dq1qvfcc8+psfzIf9D+tkud7PJ9iXmTPmGXXdn9J5e12mVmdinIjz/+\n6Gu75D1edj4FHVuZT8lRp04dX/fo0UP15HywsTsycqFfv36qF7Ss0C5PlXNVLvVxzrmqVav6+vTT\nT1c9uYSoQYMGqle+fHlf29gj5k385NJ1uS+cc+7YY4/1tV26+tlnn/n6+++/V72gKJiw5LWMc86d\neOKJB30955ybOXOmr+3cQHrZpX7xXmvYpfLNmjU7aH2w55TRID/88EPcrx+EY0pyyGN627ZtVU9G\nvNj9Lx97ww03qJ48/tj3QPI4Ya89gl7DRrrIc5iMoXLOuaVLlx70cUgeeZ1i95tchvzrr7+q3ogR\nI3wtzwvOOTd9+vSYzyn3f8OGDVXPRhnK6+SsrKyD/wOQEfL6Rka4OOdc3759fb1o0SLVSyTmMhk4\nv2ROUOxX7dq1VU++f7LHDElGQDnn3OjRo31tr6Xl6x8q5kkeBy+55BLVO+mkkwJ/9v/Ya+KxY8f6\nes2aNXE9RzLxSV0AAAAAAAAAiBBu6gIAAAAAAABAhHBTFwAAAAAAAAAiJC2ZujZjQ+ay2IywVL/+\nobJW5LZ17dpV9WTmh8xAdM65IUOG+NrmfyCzZK5KxYoVVe/jjz/2dfXq1VVP5uQ+++yzqjdx4kQ1\nljmEQbm5cn7Zx9rMVXKBMiferEK7P2vVquVrmWnn3N/zyb766itfJ3IcjDeHmfmTGhdccIGvS5Ys\nqXoyA3Dx4sWqd9FFF/k6kdxam3Eq7dq1S41l/m5QnpR9PZmpi/CCrm1kruC2bdtUb8yYMb7esWOH\n6iXr71huW7du3VRP5iiuXLlS9V5//XVf23MUUk/+HYfNSbd5dTJv22Za2vxDeY1kr3uRWfJ8M3Xq\nVNW79NJLfS2zd53T+7xz586qJ3MObS64vPaw73PkMcQ558qWLetrey6SP/vuu++qXrzHmKDvE0Aw\n+buzucVyH9trVplxa/P85fmuRo0aqifnWOnSpePezpo1a6pxmTJlfP3HH3+oXtj8ZXKb4yd//82b\nN1c9ua/s8UZeJ//++++hXvtQ34fEfsydgr4nS86noOO5/c4ZeW+uXbt2qiePX1OmTFE9e2195ZVX\n+rp3796qJ6+X7XbL7fn0009Vb/78+b7evXu3Szc+qQsAAAAAAAAAEcJNXQAAAAAAAACIkLTEL1jJ\niFywH9UuWDD2PyXo9ezzyCX43bt3Vz25DPbVV19Vvc8//9zXLAPILLtPGzZs6OsRI0aoXoMGDXxt\n54lcdmg/Yr99+/aYr2+XnhxzzDG+tsul5VLHsMvHbARA0HLt/CZoCbqUyO9ePqdd1tq4cWNflyhR\nQvWmTZumxlu3bj3s12fJYerZuAu5rNQuC5Lz4YknnlA9uXQ5WfvNzu9SpUr52i5zk8chGxshlzIy\np8ILOvbK5arr169Xvb179/o6Wb9/O29POOEEX/ft2zfmz7355ptqvGDBgqRsD2KLNyol7HmiePHi\nqte6dWtf22tne23z5Zdf+jqRa/dE4l8QjryeXLp0qerJSJeiRYuqnjyHrVixQvVq167ta7ufZDSL\nvb5p2bKlGgddp8hjynfffad68v2TnUNybI9v8tjL/AoWFBcnr2l+++031WvVqpWv5fsq5/S8Oe64\n41RPLs23S5Lt+2X5/skurR44cKCvb775ZtVLRuwhkR7B5HnE7ht57SuX1Tvn3Lnnnutre+8k3vsl\nNiZIXus6p2Nb7ByTPfZpesljjb0+nj17tq+bNm2qenIO2WuUPn36+LpYsWKqJ/fvDTfcoHr2/bqc\nU0F/+zYuT0Yn2ijOVatWHfQ50oVP6gIAAAAAAABAhHBTFwAAAAAAAAAihJu6AAAAAAAAABAhacnU\nDcqqSCTDRmYoFS5cWPVkNobN1gl6Tpux0aNHD1/LfF3nnFu0aJGvX3nlFdWzWanIHJtp+9JLL/na\n5kDJObVu3TrVe/bZZ31ts6XsnArKj5N5lTaTLmzmSrxZsfmd3L9hM9js71o+j80mPfHEE31tM01l\nNqFz+jiVyOsHZSySF5V8NrurXr16MR8rs5dsVmGystaD5p/MvLO5ZjJ72x4j7fENh69ixYpqLP/e\nq1Wrpnoy73bNmjWqJ+eUzSSTxwabrV6pUiU1vueee3xduXJl1ZOZvvb1ZQ+pkYzzVNBz1qpVS/Xk\n2F6TzJs3T41nzpwZ6vWRevKcYvPdhwwZ4muZKemcfr9i/77lfAi6Xi1fvrzqvf/++2rcpk2bmM/z\n0EMP+drmX0pBmbr2vRvfJRG/oL9j2StXrpzqXXzxxb5u27at6snMfjun5DFk1KhRqifzve3Y5jbL\n9+efffaZ6snvSwl7nOL4ptm/v+bNm/tavtdxTp9vbM7pHXfc4Wt7fTFjxgxf2/s6devW9bWdbzYn\nXh7T7LHhjTfe8LW9Lk/GdzwhNjmH7L25f/3rX74uXbq06slc7mbNmsXsye+xcU7fC7TXxHZeBN1H\nkd8tYO/3DRs2zNdbtmxRvR07dviaTF0AAAAAAAAAQCBu6gIAAAAAAABAhHBTFwAAAAAAAAAiJC2Z\numHzIoOyLG1P5pbu378/5uvZjA2Z++Scc1dffbWvbQ7Vq6++6muZc4fMk/NB5i45p/exzauTeV7X\nXHON6mVnZ/v6UNkosm/nTbKyNKWg/Kr8zO5f+fdu83RkBl0i2d7ysTZzrEmTJr6WxyTnnFu8eLEa\n23kS6zUSydRF8jVq1EiNa9eu7Wv5d+ic/lu3Gd1B5D62c9iSebgDBgxQvc6dO/u6SJEiMbdN5pg5\nF3++M+Jnr0NWrVrla3mccM65rl27+trmhy1cuNDXK1euVD05/2y+co0aNdT45JNP9rW9DipY8L+X\ngjbXjvmQfEFZbzbv2ubQxWLPE3Kfyqxt53R+nZ2nU6ZMUeOtW7f6Oui7BDhPpZ/8vdpsvw0bNhz0\ncc7pc0HYfWPfA8nrZef0eVNmqjrn3DfffHPQbTkU+XdjszDleZN83fjZ/S9/rzLT1I7t9YX8+7fn\nqYcfftjXWVlZqjdp0iQ1ljmb7du3Vz15bDzllFNUT2b1Bl1bI35FixZVY5lrK88vlj0XVK1a1dcv\nvvii6skMVJvFK3v2PGXfX8lzqPw555w7/fTTfd2nTx/Vk/OR81Tyyd+pPCc59/d8ZWnq1Km+tvNJ\nngfsnJHXOjK73bm/53dL9j6KnKfye5mcc27z5s2+zm3fa8MndQEAAAAAAAAgQripCwAAAAAAAAAR\nkpb4hSCJfFRZLqmxH72Pd7n0scceq3qvv/66GlerVs3X48aNU72RI0f6OhVL6hGeXE4ol+849/cl\n0tLy5ct9LT/u71xwFEjQMsRkzQ25xMAura1UqZKvV6xYoXoyViA/kL97uyRIju2SV7lcL5F9Jpf2\nnHrqqaon98uSJUtUL1nL8ZOxdBLxs8uhS5Uq5Ws7p+RyNXkucU4vvz/UawSRy/MvvPBC1StevLiv\n7TFLLl8bPXq06tnzKcKRf492Ceq8efN8XbNmTdWrUqWKr0866STVa9mypa/tfpLHgh9//FH17BJk\nedyyxw25XFXOb+f0POJ4E578PdooILlv7DEl3ogD26tQoYKvr7jiipivZ5e1yqXxzsV/brSvH2s7\nkTzy92qXj6b6d25jic4880w1lttz1VVXqd6uXbvieo2gpa32uog5Fo79vcnjwQ8//KB606ZN87V9\nX7V69Wpf//Of/1S9n3/+2dd2nsp4F+ece/DBB339xBNPqF7FihV9bWOC5PHOLusOOoZxftPk76NE\niRKqJ6M5tm3bpnryvVbQ77Fs2bJqLK99DxU9JgVFNdhzkbzvI2OonHPut99+83VQdCfCCYqmDEte\n29rjh4zTkNfVzv19fsntef7551Vv8ODBvrYRWLn5OMEndQEAAAAAAAAgQripCwAAAAAAAAARwk1d\nAAAAAAAAAIiQjGTqxptHkazcCpn10rNnT9Wz+XEyF8hmbMSbA4X0q1Onjq9tjkpQZtLYsWN9bTMI\ng/J9gvLj7PME5d7J/LwiRYqonswsa9CggerJDOEXXngh5rbkB/L3a/MIJbtfwmbTyv1k85nk648f\nP171tmzZosZB80LOPZsTLLfbZkDl5qyfqPr111/VWGZwlS9fXvVkrlenTp1U76effvK1zZWTWWV2\nH5YrV06NH3roIV/bjHg7VySZvf3hhx/GfH2EJ/fd9u3bVU9mtsssd+d0dp3NnJNsFqvcp/Y57fmk\nTZs2vrb5l3Le1K9fP+brI7yg85Ts2WN62PNU9erVfV25cuWYj5s/f74az5gxI+a22fOUHAf9m4Ky\nUZEcifxO481ptmT+5WWXXaZ69tg0ceJEXyfyfQJB5DVc0HU24mfzZuW1if2+jltuucXXNlNX/pzd\nN7Ee59zf95vM8b311ltVT47tdxa0bdvW1/baW753Z54Ek78fm1c6bNgwX0+YMEH15O/YXnvI74Sx\n+1S+h5LXz87peWTfP9lM5aDvzpH5u3KeOPf3a2FEi93vL730kq/tdY89Lsn7P4888ojqRfV7Rvik\nLgAAAAAAAABECDd1AQAAAAAAACBCMhK/kG41atTw9UUXXaR6O3bsUOM333zT17Nnz1Y9u0wFuYf8\nCH7QsizbO+mkk3zdq1cv1ZNLxuwS6KBl/nZ5kVxSYpeoNW7c2NcVK1ZUvaDlJHJJuF2GtGbNGl/v\n27cv5nPkRXYZudxPQfELQexyrQoVKvjaLmOWli5dGrhtQYKWR4YlIx04lsXPLh399ttvfV21alXV\nk8vozzjjDNVbuXKlr3fu3Kl6WVlZvm7RooXq3XbbbWosl8cHHYc2btyoxpdccomv7VI2JEfQuUYu\nT1y2bFnMn7Pk363d3/JcY5/DHvvlMvuOHTuqnoxfSNbxBrHZiAW5X+15ImxcmYyGsTFj8jUmT56s\nejZmLN65idzL7qd4/8btzxUvXtzXxx13nOrt2bNHjeUy+mQteWfpfOrJa0N7DpPnFLs8Wc4pu/xe\nOlQUi3xNe56U11BlypRRvSZNmvh6ypQpqifnpr325Vo4NnsNsWTJEl8vXrxY9YL+NuXc+Prrr1Xv\nf/7nf3x94YUXqp48T9lrHxkFcyjyOLZt2zbV45gSPfI89PPPP6uevB9iz1+rVq1S4/79+/s6qnEL\nFldkAAAAAAAAABAh3NQFAAAAAAAAgAjhpi4AAAAAAAAAREiezNQtXLiwGg8ePNjXlStXVr0VK1ao\n8TvvvONrm42K3MNmgsmcHJuRIx9rc3nat2/v61NOOUX15PPY3CWbNSWfV+YT2sfa55GZLzZLT+Y9\nf/PNN6r33nvv+dpmfua3HF3J/g7l7/5QWV6x2FyeG2+80dc2q1Dus7lz58b9erYXlGsmHxuUjRf2\n3wvN/j2NHz/e1127dlU9eX6Red3OOde2bVtf23kq97c9ftgc7iBbt2719TnnnKN68+bNi/t5kHxy\nHyfytyn//m0WayJkDpk9D8rXsFnMZOwmn92P8ngQ9rht99MFF1zga3sOk7m5H330keolkjEZdI2E\n3MPOITkfgq6XbU+e36pXr656NuNSZp7ac1p+vkbNDeQ+tscGe70Z5jnt94EcznlLCpo38lpcfu+A\nczoLdvPmzUnZlvwg7FwIYjNt5T2Xdu3aqZ78zgr53TTOJXZdLM+vM2fOVD3eF+V+NqP7u+++87U9\nD8njmT1e9OvXT43l+6W8gk/qAgAAAAAAAECEcFMXAAAAAAAAACIkz8QvyI9c9+3bV/XOOuusmD/3\n7rvvqrFcys7H8qND7vwsDZYAAAkGSURBVDcbqVGyZElf22VgdhyvRJasBS11C1qSu2jRIl9PnjxZ\n9RYvXuxru5Q7P0tF5EDZsmXVuGPHjr62S9fkcg67zCgRcl7Y15D/pkT+vRzPwrF/X7Nnz/Z1dna2\n6h177LG+Djq22OXviZD7cdOmTarXpk0bX2dlZcX8OWRWOvaFjaE64YQTfG2X6v/+++++Xr16terJ\n408qlmPmR6k4TxUtWlSNGzVqFPP55XnKHkMSIZ/XnqeIY8g97P6Xf8f2WCDHdsnzxRdf7OvatWur\nnj3ftWrVytcyisE559avXx9z25B8dh/LqAzbk1EJQe8t7M/JyAV7LEok8kA+r30eedyyEQ/lypXz\ndbNmzVTPXgshPnYfJ+Nv1T7HjBkzfD116lTVu+yyy3xtjy9BsVD2OkU+7w8//BC4Pcgd5N/3Sy+9\npHoNGjTwddB1x/Dhw1Vv2rRpydzEXIlP6gIAAAAAAABAhHBTFwAAAAAAAAAihJu6AAAAAAAAABAh\neSZTV+Y7DRo0SPVkLs/SpUtV75133lFjcsCiSWYtXXnllarXoUMHX19//fWqV6dOHV/LnClrz549\ngeOdO3f6eteuXaonM19KlCihen/88Yevv/jiC9W77777fL17927VI0c3tWReU4UKFWL27H6ReU1y\nThyOVOQvIrzt27f7un///qonzz39+vVTvXjzu+3+3bdvnxoPGDDA12+99ZbqcVzILHlsSCSPLhl/\n0zZbrEqVKmq8cuVKX1evXl311qxZ42t7jpLZZjJvEbmLzS2V+80eF2Smqb3mTVaOYlDmITIrKJdf\n5r3Xr19f9WSmbpEiRVTPziM554466ijVk3nf9udkHqadt1z7xC/o709+T0TlypVVb+PGjb6WWevO\n6etdm9kun9Nes8jzRiL70D6PfI9Wvnx51atUqZKvf/zxR9WT12z2PElOfGbJ99IjR45UvfPOO8/X\nNl/ZzqNVq1b5+qmnnlK9cePG+VrOhYM9DzLDfs+I/Htu166d6sm/YXv+WL58ua9vvvnmZG5iJPBJ\nXQAAAAAAAACIEG7qAgAAAAAAAECERCp+QS4nsct5rr76al+XKlVK9fbu3evr4cOHq55dXhIv+1Fx\nuW0sgU09u2RCLqFZuHCh6snx0KFDYz6nXa4kx/b1gpYo2uU9cllk1apVVW/btm2+/u2331SPpa6Z\nI/fvli1bVG/UqFG+tkuVR4wY4evD2X9ByyODekiv7OxsNZbnIRmf4pxzl19+ua8rVqyoenKp9Kef\nfqp6U6dOVeNkxXog+YLOA7Ee51zwuSaIfA0b72HPUW+++aavZeyPc3rpbM2aNVVPLrPdsWNH3NuG\n1JPXoXY5tFxGbc2fP9/XhQoVUr1ElicHnYvkskjOU9FUvHhxNZbHFDsv5Pss55wbP368r+21kFxG\nb+ftpk2bfM28CS/odyeXoDds2FD1Spcu7WsbfyCPDfb9inysjaBLJOJAbreNudu8ebOv7Xtw+Zpy\nDjmnz43EL8QvHX9/8jW++eYb1evevbuvixUrpnr2OnjevHm+5r1zNMjzib1+lccle70qjzWrV69W\nvWuvvdbX9vgRlj1myGNPbptrfFIXAAAAAAAAACKEm7oAAAAAAAAAECHc1AUAAAAAAACACMnVmbo2\nx0Lm6LZu3Vr1evbs6WubtSMzN0aPHq16MvfrcMg8EJtll9syN3BwQdmlh3qsZDOaZLZdUM4dcg95\nXFi/fr3qPf7447622d4y+ydZ2dp2PtnjC3IPua/WrFmjek888US6NwcZlMi1RdjsOvlz9jrDZh7K\nzEub2yx/duvWraon8xeRu8g5Zo83AwcO9LXN8J4zZ46v161bp3ph5yLfJZE3yDmVlZWlel9++aWv\nW7VqpXr22lbmzdv8S5l1aHNb7RjJJ69TZ8yYoXolS5b0dY0aNVRP7uNVq1apntxv9r172PfZdi5M\nmDDB10uXLlW9Nm3a+Fp+j4lzzhUpUsTXYb9HB6ln9/esWbPi/lnyt6NHvpe134XVo0cPX9etW1f1\nZH72sGHDVG/mzJm+TtacsMevZN03TAU+qQsAAAAAAAAAEcJNXQAAAAAAAACIkFwXvyCXbdiPTstY\nhfPOO0/16tWr52sZheCcc/PmzfO1XTIS9mPUdtvksttixYqpnl3OCCA67DFCLv2w0rEEiGVGQO6X\n6WOBXfK8e/duX9toBnnNZONeWA6de8n9b5cVy2XVdp/K5e+p2BZEl9yPNppDRk9VrlxZ9X7++Wc1\nlvMxKIKO2I70k/vYxuvI/Wajx+S5wP69y6XUNn4h7LHBHrfkti1YsED1ZBzDjTfeqHqFCxeO+ZzI\nvTin5G3ynl6jRo1UT97jK1SokOrJ69ehQ4eqnoyWsVGF+WE+8UldAAAAAAAAAIgQbuoCAAAAAAAA\nQIRwUxcAAAAAAAAAIiTjmboyU8O54IxbmfvWrFmzmD2b0fT888/7eseOHYHbI7OAgrbF9uTP2Ywi\nAHlTfsjoARB9MkvQ5goGZV4immSGctjvjkD+Zo8Ly5Yt87XMMHWOORZVNv9W7ke7/4Oud2UvWTnJ\nNhMz6Dt35LY+88wzqhf0PRiIpvyYlxp19lgjjxOLFy9WvSVLlvja5u2+8847vt64cWPM18uPc4JP\n6gIAAAAAAABAhHBTFwAAAAAAAAAiJOPxC3YZYJDff//d12eeeabqlStXztf79u2L+XOHEnYJUbKW\nmwDI3fLjkg4AQHSwHB7Jlsj7NURDIjGDQeKNLkyEXWK/d+/eg9bIf3gfFj1Bx4W1a9eq8amnnprq\nzcmT+KQuAAAAAAAAAEQIN3UBAAAAAAAAIEK4qQsAAAAAAAAAEZJopu4m51xWKjbkcG3atCnTm5Bu\nNTO9AQnItfMmn2HOIAzmDcJg3iAM5g3CYN4gDOZNEqQiwzuX54Izb5Ao5gzCiHveFCBsGgAAAAAA\nAACig/gFAAAAAAAAAIgQbuoCAAAAAAAAQIRwUxcAAAAAAAAAIoSbugAAAAAAAAAQIdzUBQAAAAAA\nAIAI4aYuAAAAAAAAAEQIN3UBAAAAAAAAIEK4qQsAAAAAAAAAEcJNXQAAAAAAAACIkP8H7XsufLg1\nT5oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1800x288 with 20 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# obtain one batch of test images\n",
    "dataiter = iter(test_loader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "images_flatten = images.view(images.size(0), -1)\n",
    "# get sample outputs\n",
    "output = model(images_flatten)\n",
    "# prep images for display\n",
    "images = images.numpy()\n",
    "\n",
    "# output is resized into a batch of images\n",
    "output = output.view(batch_size, 1, 28, 28)\n",
    "# use detach when it's an output that requires_grad\n",
    "output = output.detach().numpy()\n",
    "\n",
    "# plot the first ten input images and then reconstructed images\n",
    "fig, axes = plt.subplots(nrows=2, ncols=10, sharex=True, sharey=True, figsize=(25,4))\n",
    "\n",
    "# input images on top row, reconstructions on bottom\n",
    "for images, row in zip([images, output], axes):\n",
    "    for img, ax in zip(images, row):\n",
    "        ax.imshow(np.squeeze(img), cmap='gray')\n",
    "        ax.get_xaxis().set_visible(False)\n",
    "        ax.get_yaxis().set_visible(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Up Next\n",
    "\n",
    "We're dealing with images here, so we can (usually) get better performance using convolution layers. So, next we'll build a better autoencoder with convolutional layers."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}