Ch08_Convolutional_Neural_Networks/Convolutional_Neural_Networks(LeNet).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z4Mfv4K6dbhQ"
   },
   "source": [
    "# Convolutional Neural Networks (LeNet)\n",
    "\n",
    "We are now ready to put all of the tools together\n",
    "to deploy your first fully-functional convolutional neural network.\n",
    "In our first encounter with image data we applied a multilayer perceptron\n",
    "to pictures of clothing in the Fashion-MNIST data set.\n",
    "Each image in Fashion-MNIST consisted of\n",
    "a two-dimensional $28 \\times 28$ matrix.\n",
    "To make this data amenable to multilayer perceptrons\n",
    "which anticapte receiving inputs as one-dimensional fixed-length vectors,\n",
    "we first flattened each image, yielding vectors of length 784,\n",
    "before processing them with a series of fully-connected layers.\n",
    "\n",
    "Now that we have introduced convolutional layers,\n",
    "we can keep the image in its original spatially-organized grid,\n",
    "processing it with a series of successive convolutional layers.\n",
    "Moreover, because we are using convolutional layers,\n",
    "we can enjoy a considerable savings in the number of parameters required.\n",
    "\n",
    "In this section, we will introduce one of the first\n",
    "published convolutional neural networks\n",
    "whose benefit was first demonstrated by Yann Lecun,\n",
    "then a researcher at AT&T Bell Labs,\n",
    "for the purpose of recognizing handwritten digits in images—[LeNet5](http://yann.lecun.com/exdb/lenet/).\n",
    "In the 90s, their experiments with LeNet gave the first compelling evidence\n",
    "that it was possible to train convolutional neural networks\n",
    "by backpropagation.\n",
    "Their model achieved outstanding results at the time\n",
    "(only matched by Support Vector Machines at the time)\n",
    "and was adopted to recognize digits for processing deposits in ATM machines.\n",
    "Some ATMs still runn the code\n",
    "that Yann and his colleague Leon Bottou wrote in the 1990s!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Y_gnPrUNdbhR"
   },
   "source": [
    "## LeNet\n",
    "\n",
    "In a rough sense, we can think LeNet as consisting of two parts:\n",
    "(i) a block of convolutional layers; and\n",
    "(ii) a block of fully-connected layers.\n",
    "Before getting into the weeds, let's briefly review the model in "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 350
    },
    "colab_type": "code",
    "id": "tSH2X0qIdbhS",
    "outputId": "bee6ace3-6b4f-45ae-81eb-129d5f1d7250"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABNgAAAE5CAYAAABClp38AAAABmJLR0QA/wD/AP+gvaeTAACgOUlE\nQVR42uydCZgU1dm2C5iBmWGGVRBEEUFQFhEERRBlEQERcEUF3AAhLAIBF1REUVxwQeKGvyJilLgg\nasSISdSgxhVMNF802+eSLyYYjcFdENHzn6f6FNQ0PTPdM909Uz33c13PNdPVp6urq09Vnbr7fd/j\neQghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEII\nIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGE\nEEIIIYQQQgghhBBCCKGYDrNead0sDevq5NbVid2KktB465+EHs+0vpTdglDaVdd6jvWT1v8vhde1\nd+f0Lu7xwe7x7uxShBBCCCGEECqtM62N9V4pvm6g9WbrXqFlR7h1HcFuRUnoVut/hx4/br2R3YJQ\n2jXBnZt/Yb0whdcd6l43xD0+2T3mRxSEEEIIIYQQilNlAdtR7nV9QsuKrQ9yfxGqSPGATdEy+7Fb\nEEq7bos71pIVgA0hhBBCCCGUdRVaj7Q+wYtFddUpo50A1BjrE73EUEuvbWGdZz3MtT049Hy+a5Po\ntbu75/JDyxq57dJ6BlvXj3tNPGBra31ggnXvFVre2Xq6e92Zoc9b5MVSiYriXivgNsJtg8BcQdzz\nLbydkXB7uHbHu+Uos8pz+343970d4/b/IeW8Zi/Xf8e4/lxWX9/DHQ9BH45vFw/YtN59Qo+7u3XU\ndX1X6zmsjPerZz00btsF69rxFaNarPru+H7U+kP3fzt3TPXydk31jF8OYEMIIYQQQghlVbqp/4+7\n8fjc/f2TVzoaRzXOnnHPbbX+3nq79TVxwEDLF1q/6f7/xr3mCddOfsf6pQTb8YD1X0OPdTP0mXt9\nsJ53rXuE2sQDtlvdZ4mXamV97P5/w70mbAGORCmignufxG3DP9yNW6Bp1j9Yn+r2zTfus+v/4+he\nGVVL953oe99k/Z31FrfsBa805FTfu9b12+D7UbunrZvGrXehW9cP5ayvohRRAYEbrH/rXv+V+/ui\nVxoUd3XHm97rU9fmfutXrO/lK0a1WO0SnKvv9mI/euj/8+PaF7jl57nHADaEEEIIIYRQ1tTdgQTB\ng7ZumaK43rd+K9TuWS8GjiY4ONDE+pa4mxkvBC6us27soMZC1264a3Opgwl7h16nCLovvZ31dfp5\nMRCi7erg1tPf+j0HLnZz7SoD2Pa0PsO9TpFm7d3yeMAmkPet9QYvFvUmKZLpL16sflsbt2yae93f\n3Y2cYF1r6z96MZiIMqeWoRvvpe7GW1FtE10/fCrU9kJvJ4xr4vrxWa5f/zrU7pzQjXxzLxZROdb6\nCy8GhuuE+lpFgE3bcJdbT3jdp7vHiroTNP4/LxZNJ3V0x56OEQAbqs3Kc+dn/fjyL/d/Cw/AhhBC\nCCGEEKqButtBgPh0RkVeKbKndegm5bIEr1d0zifuRkj63ts1Oq3ALQ9uhvZ2j+fGvZ+AQgf3WBFv\nnyXYrsPdtsxzjysD2KRENdjiAZvgxjferumsPd32X+UeB4BtfFy7OW55Q7pZxhQAthcTPHe1e07A\nWNBzs2sXn6J5aagv6DlFwv3ei6WbhTXXtTsy1NcqAmx/ce8dlqLUrnf/n+bWeXRcm0PccgAbQp53\nhxeD0IEAbAghhBBCCKEap9e9WJRYeZribkq6J3gugA7d3GOBpxsStFMk2OWhx89Zvxp6vMorDeb+\na/1kGdsjgPaE+z+TgO0PXgwgJpKijp53/weArX1cm8lu+e50s4wpAGznJnjuYPfcOC9WGy0+2jLQ\nAe459eUO7v+FCdoF67g01NcqAmyJANkH1re7/wXaFEGaV0Y/B7AhBGBDCCGEEEIIRUD/9GIFpMvT\nAndT0irBc0EEzuHusQDbFQnaxQM2pZoqYq2duylS+t3U0POCDneXsT1KnwvAVyYBmwDJmjK24WUv\nBuCkaWXsHwBb5hUAtkkJnguA2Lnuew6nZiZah/ptT/f/tATtitxzN4b6WkWA7f8lWE8YsP3U9f1E\nUpo2gA0hABtCCCGEEEIoAhIoejXBckVjKQ1T9coCiNY3QbsgvS6YPTFZwKYbpC/dDdIoL5amGi40\nr4kEnkuwHqXtqVj8A+5xsoBNReNTBWwqMv/7BOtSGqHAShBhB2CrPgVw7KoEzw10z41x30FZac7B\nTfiP3Heo/69P0K6bVzpaLh2A7Qq3zvhUaNUv3OYB2BCS4gFbQy8xYGvrAdgQQgghhBBC1aSbvRgU\n6xq3XGmegl4CYR1cm/vj2hS5m54/h5YlC9ik+7wYkLjH+pG45+70YpMc9Ihbfra7QTrTPY4HbAIo\n33ml6541sv7ISx2waR8oyq5/3DYEN2kz3WMAW/UpAGyCViVxz/3U9cd27vHfXH8timv3M9fXAkis\nWWYFzhonuMn/IXSspAOwDXPbf3Fcm0s8arAhFD72/i9u2ddebGKTsGZ7ADaEEEKoyhoduiGqqgZb\nj2CXIoRqiQSmNJmAZmhTpJpm+rzaAYerQ+1ucDcmghHHerFZFX/nxWDWsFC7VACb3kvAQpFsJ8U9\np8g5wQtFo2nmxaPd67UeFarPd+3iAVt/9/hxdy7XZ3rT+n+90oDtINdO0FCRenW8XQGboooUSaei\n9D922zDfi018oMi2QtcOwFZ9CgDb516snuAJ1kOtl7nlt4TaHu369e9d/1U/XuXaLQ61G+j62dte\nbOIKRViu8HbOQBooHYBN/e5Jd9wsde93mxdLG/27B2BDSEoE2J5w147g+nClF5u1WT8MAdgQQgih\nKuiP1g9V4nXzE9zwrbXewC5FCNUiHWj9jIMKuvlQXbaLvJ0QKwABc9wNjPF2zhZ6VNy6/te1i5dm\nU/xx3DLNrvi6O4c3SPCajl6sBto37j01W6mASTiy6ER349U6tEw3Vx+712jmyEvdTVgYfijVVNDk\nXWdtyyFuXYeE2mnGU0HFr9z6BNsUXbdbqM14t474NL9T3PLd6GIZUwDY1OeWu+9bjze563v8DJ5H\nuX77vWv3jntt/MyiA7xYivJ210710C70Ss8sekVcn1ruxhCBXnU3/fHS+y8KPVbdqGu9GHgTUH7W\ni6Vjq9bgrXzFCPkp4PETzuwdd916zYtNWKLrydmuTU93Tj/MPT7GPd6HXYoQQgiVrcoCtl96u86e\nF0RmIIRQbZNgRHES7ZR+WT+L2yWo0bgSr2uUxm0QgGlCF6lxCgDblBS/9/pe6TTispSf5DFRWSld\ntX2C40l9TWB5Ll8xQhUeo4XsBoQQQjVRuolp6u1anyTRgFDtGiSxTrUrScONTdNybm7SCdgq+ixN\nk2jXwLWrT5dCCCGEMqZEgC1K6u7tWoNNYzGluCrKrhtfMUIIIYRQ9KT0mQ/dQC9Imzg+ro3SdQSl\ngpSJLV6siHCzUJvpXqyWj1Js/hhan4oG7+vaTPJiaRx94tZf7Lbhorjt2hS3XWPiXhcGbAVu3fGp\nSPluuQqgDnD/a4au79z/wQxk97rPGNZEL1aHJ9gGhZefEddGtYiULrLQi9VOUTuFrd/o7Zp6ghBC\nCKGqK+qATVrkPoNqvj3mxSZjEFybz9eLEEIIIRQ9BbPuPGzdz4sBqACkBYVB23ix+jeyCjqrnsEC\nL1ZMVPAsiGZTOoPAlWqJqHC06oioCPUX3k5w1dS97idx26HUTBW8DkDc5d7OYtUqoq1Cw791bcKA\nKwzYCt1rLoxbd75brm1SwWlBOhW+/pP7/0DXLr4G27nudb/yYgWz5V+7ZTNC7QT+/um2RZMuqLDq\nT1270XQxhBBCKO1q4K7h7SP+OVR0/UdujDItNA5CCCGEEEIRkmqQaAa5p73SxXs1aP2rFyvaK2lW\nK/2i2jvu9cEMbkFB0bnu8blx7TQbnaBaEM212otFpoULED/ixYr/Si1d+196pSPA8rwYxNJrgwLc\nqQK2QIlSRMOATamtX7htCu8b/f+KF4t8C9JpBdg0i9nucdv6Xy8WxYYQQgghhBBCCCGEclRK0xR4\nGl9BO0V7vZRguWCTAN1j7nEA2OJn6JnllgfFh49xj49wj5UeqoK+U93jse75kQnec4Z7LpjpLVOA\nrb97zbgE23C6V3rqbwG2dQnaaea8e+lmCCGEEEIIIYQQQrmrU7wYKDqsgnaCaPeV8ZxSRF91/weA\nLX7Gt3jApuguRaHd4h6f7MUi1oJJBH7s2ndN8H5Hu+eOc48zBdhOc6/pm2AbDnPPBamq75SxfwBs\nCCGEEEIIIYQQQjmuI70YKDo6wXOqAdLF/a+iu78uYx0CZUF9tWQBm3SDF5vUQGmiShl9OPTcBNd+\nUIL3C9JSB7jHyQC2Zl7qgO0o95pjE2zD8e65Ue4xgA0hhBBCCCGEEEKolqq1F6utdnPccqV+vme9\nyj0W/FI9slZx7VTMX5MOXOIepwLYurplShf92is9GcD+7rmbE2zzz71YOmmJexwGbNpuRcJdGfea\nkV7qgE37RhM93JNgGwTNNANpS/cYwIYQQgghhBBCCCFUi/Wg9bfWE71YNFkT62VeDLwNdG1U70xA\nSXXYOnuxlEulSQosfeztBE2pADZJtd0UHafZSevHPSeopxlJZ3oxmKb00WA6+2tC7cKATRIgez+0\nnarz9he3rjBge9S1O9h6T7csfhbRFW4/zHOfSftG0XGCireF2gHYEEIIIYQQQgghhGqxBK8EqEzI\nX1qfE9fuBOt/x7UTQOoVapMqYJvplR2p1tBt1w+h91NEmeq2hWcfjQdsSivdHHqN4KFqpamOXBiw\nneGeU5tgps94wFbgxSLYvg+tT//f5ZUGggA2hBBCCCGEEEIIIeS19WKzYirts0kZbRQR1s+1O8i6\nTtzzAlJNEyxvUMbyPLe8QTnbtad7P9WL2y3B822sW8Qt0/b3da9rEVpWENeuyL1/oXusSLw9ErxH\nK7cuefcEzwsoNkywvFEZyxFCCCGEEEIIIYQQQgghVEukSW+m5LiP4GumP2fZi61n14JjC+OqeLY7\nVtK93vHW/b1ds24QQgghhBBCKGNakZeXt71hw4ZbctENGjTYVq9evQ/4munP2bQXK4HxkfW7GOMy\n/bE7Vv6eRv/Di5V92Wj9uLdrhg1CCCGEEEIIZUQr999/fzNt2rSc9KBBg4wFbB/yNdOfs+mioiJB\ntrP4OhAqVzpGBMT2SKP392KAbaQXq9F8EbsZIYQQQgghVGuABIAN5VJ/BrAhlJR0jGQKsPWwPsn6\nO+vD2dUIIYQQQgihWgEkAGwol/ozgA2hpKRjJJOATbrF+p/erpOmIYQQQgghhFDuAQkAG8ql/gxg\nQygp6RjJNGBrYP269VPWddnlCCGEEEIIoZwGEgA2lEv9GcCGUFLSMZJpwCZ18KjHhhBCCCGEEKoN\nQALAhnKpPwPYEEpKOkayAdgk6rEhhBBCCCGEch9IANhQLvVnABtCSUnHSLYAm0Q9NoQQQgghhFBu\nAwkAG8ql/gxgQygp6RjJJmCjHhtCCCGEEEIot4EEgA3lUn8GsCGUlHSMZBOwSdRjQwihiKqldXuM\ns+x8Dj2EUKaAxOmnn27Gjx8fGR966KECbB+ncA5tSZegP1fVhYWFW+22nM+1G6EaB9gk6rEhhFDE\n1Mr6e3eCxzibvpXDDyGUCSAxdOjQ2nAO1bV7d7oF/ZlrN0I5C9gk6rEhhFCE1E4n93POOcfMnz8f\n46y4e/fuGlCs5PBDCGUCSCjdslGjRmbWrFk56bPOOiuAHe3oFvTnbLlz585cuxGALfuAjXpsCCEU\nNcB27rnnmiuvvBLjrPiggw5ikI4QyjiQuOCCC3LSP/rRjwBs9Oesu1u3bly7EYAt+4BNoh4bQggB\n2DAGsCGEAGwANgRgQwjAVgXAJlGPDSGEAGwYA9gQQgA2ABsCsCEEYKsCYJOox4YQQgA2jAFsCCEA\nG4ANAdgQArBVAbBRjw0hhABsGAPYEEIANgAbArAhBGCrAmCTqMeGEEIANowBbAghABuADQHYEAKw\nVQGwSdRjQwghABvGADYUSR1rfVfo8QTr69gtAAkAGwKwIQRgqwbAJmWjHtsI659b/8K6WQqvu9P6\nePd/iTtvDKAbIYQAbBgD2BBaaL099HiF9fvsFoAEgA0B2BACsFUTYMt0PbZOXixK7vfWt6YI2LZa\nX+n+b+k+5xS6EUIIwIYxgA2heMDW1roruwUgAWBDADaEAGzVBNikTNZjG+O2b/9KvBbAhhACsCUD\nRrp06WIKCwsxrpLz8/NN3bp1t1l/kYzz8vLWcqgiq17WrawLrIe5wV9/6zpltFfbE1y7Pl7Zv/Aq\nveI4166fdb0KAFtr631Dj7tY7+W2o79bz8AE6/Fcm0GuzeGhAfK+fL3VAyRmz55tmjZtagoKCiLj\nBg0a6Bwqf5XkefQT1zcR/bnSttdirt0IwFazAJuUiXpsB1jPd9s32rqbW965jPHK/nHLAWwIIQBb\nMoBNg7YjjjjCTJo0CeOsWP2tfv36H3Ko1nrVdQM01UJ7zwGvb9yy33mxqLKw9GvuNusfrLe4dq84\n6BbW7NDzZa0vHrDFp4j+0YvVG3nSvf4r9/dNL1Z7JNA+bt16brP7q9covWMdX3H1AIkgGmzUqFHm\n1FNPzUlbOLLN3SAi+nNW3KdPHwE5rt0IwJZ5wCalux7b+27bAr/nlm/0YvXY4qXx1RMANoQQqgRg\nO+2008ytt96KcVas/gZgQyHAJt9t3cQtUyTYl9YbvJ2RbGe7dve5wWaeF/uF9zM3OAwiy0517dY4\n8Kblo6wV7fOWdf0UAJsGkw+HAN5YB/fmuMd5Drh9bH2EW6aot1ddOwBbNQOJXC6VUFxcvAXARn/O\npk844QQAGwKwZQ+wpbsem35k/LHbvgHezh8dAWwIIQRgwwA2lEOA7U8JBo9z3XNBesS71m87qBXW\ndNfuGPf4Lde2QVy7M1y7U1IAbOqjhXHrecfbWa9ouFvn6XFtlFLxPYANIAFgoz8D2BACsFVB6a7H\nNs5tX/vQMgAbQggB2DCADeUQYLsmwXP7uOdmebGUTP2/OEG71u65q60be7HIsVsTtAvWcXMKgC0R\nIHvDi0W1SfPcOndL0O7PADaABICN/gxgQwjAVkWlsx4bgA0hhABsGMCGchywzSsHiAm+dXD/z0nQ\nThFtihZb5sXSM9VuQRnv97X1vSkAtjUVALYl7v0aJGi3EcAGkACw0Z8BbAgB2NKgdNVjA7AhhBCA\nDQPYUI4Dtv+X4Llu7rlpXqxu2nY3wIzXvq6dZsZSOqcKv69M0K6VVzpaLh2AbZZbZ+e4NtreTwFs\nAAkAG/0ZwIYQgC0NSlc9tkSAbUMZgO1dABtCCAHYMIANRQ+wCUbF/yobRIcd5B6/bP1f6+Zx7W6K\na/e09RdeLHU0rKtcuyPSCNh6uXXeGNdmqlsOYAOwAdjozwA2hABs6VA66rElAmxPurFNWD29WHYA\ngA0hhABsGMCGIgbYPvdiNctOth5ifZ0Xq6X2QKhtPze4+4sXm7BgpPUdrt1dcYPCr7zY9PMTrEd4\nsbpravdgqF06AJt0r/sM2pbx1jdYf2P9NwAbQALARn8GsCEEYEujqlqPLRFgO9ct0/hFkzfNsP7A\njYkAbAghBGDDADYUMcAm2KVItI/d4/+4gV5BXPvDrJ91YEzt/s+L1VurF9eut/WvvFi6qNqpbski\nL5a6GUhT1b8benyt9W9Dj5WGcVuCbVYaRXgSBdWAu9QNRD9x6zjK+rk4oIcAbAA2+jOADSEAW1VV\nlXpso93YZ6/QMpXXuNOLRf9r2/9uPcZ6uXOgv1qf5/5v7sZgY+lGCCEAG4ANA9hQzQJsF4eWNU7i\ndfnWxUm0E/wqyeD2CwC2d4PTsBo4SHg1XzGADcBGfwawIQRgS6PSVY8tXnUyPGZCVfzSRTtXZ8D6\nZfiDDFnpJGsysM1ve7F0kUxYv87/MwP7Quk6WzO0zaLjj0asb/wlyZu5nAdsU6dONT179sS4St57\n771N3bp1t6RwfOuaUp/La60AbFHSnl4smi6+BtuV7nMN4CuOBpAYOnSo6datW6RsQYf63oYUzqMj\n6W6Z788lJSWmQ4cOFbply5aCpJHqz23atDF16tTh2o0AbNUL2KR01GNDUbu5Hzt2rJkyZUpa3alT\nJ9O4cWPTuXPntHrfffc1mdpmDQbdgYadR48eHZm+IRDgtrs/gO1Wc+ihh5rWrVubwYMHY5wV9+vX\nLzgG23F5BbDVQCnVVPXdVHPtMeu33OMb+XqjA9h0HezYsaM57LDDctKCOV7i2XVRejXKi9VjTMbr\nGzZsuD1X+3Pv3r25diMAW2ZV1XpsKGo39++//75Jt8466yzf6Za2NVPbHMA7vNO//e1vI9M3tK0A\nttKA7YgjjjA/+9nPMM6Kf/KTnzBIz10pHUF1PjrnwLjnbOt51udYd+WrjR5gy+VSCbp2A9hq3s29\nBWxbcrU/X3755Vy7EYAt86pKPTYEYAOwAdgAbAA2DGBDCAHYAGwANgAbgA2hWg/YMlWPDQHYAGwA\nNgAbgA0D2BBCADYAG4ANwAZgQ6hWADaJemwANgAbgA3ABmDDGMCGEAKwAdgAbAA2hABsVRT12ABs\nADYAG4ANwIYxgA0hABuADcAGYAOwIQRgq6KoxwZgA7AB2ABsADaMAWwIAdgAbAA2ABuADSEAWxVE\nPTYAG4ANwAZgA7BhDGBDCMAGYAOwAdgAbAgB2Kqo2lCPbV/r8daTrIdatwawAdgAbAA2ABvGADaE\nEIANwAZgA7AhBGBLp3K1Hlsb63UJ2MIP1qutGwPYAGwANgAbgA1jABtCADYAG4ANwAZgQwjAli7l\nWj223az/1/pL61nWLa3rWXe0XmS93fpl6/oANgAbgA3AlrOAbfny5WbKlClm0qRJGFfaJ598cnAM\nKtx9ShKemOu/YtUQHZHk94F39SnWdehC0QRsl112mRk7dmyk3L59e+2Pl1Loo0fQNTN/c9+gQYNt\nw4YNMxVZ466o9edRo0Zx7UYAtupTofUfvFhNth9lYBxzifUVWRzT3OsgWllReT92+30CgA3ABmAD\nsOUsYJs6daqpV6+eadWqFcaV9u67726aN29udtttt63WWypy3bp1v7f9/3TGjZmVPbY/KCgo2FZS\nUrIFJ+/i4uKt7prSll4UTcDWt29fY/u+adGiRWTcrFkzfcbt9u+WilxUVLQtLy/vA7pmxtXfnkf/\nYff1pops230Uxf6s7bDemky/q1OnDtduBGBLrwa6bVEk29/T7M1u3V9kYUyj/aqU15+W06ah9YNe\nLLotkKDfGC+WVvoX6/+xXu7F6tQFUi23nyVY36nWq0KPFUF3lfVrXiyS7jnXf8Lq5tYvsPln68fT\nyQ8AbAA2ABuAzb/BERz55S9/iXHWrIF6goseSrPsTd+HJ5xwQlLnGLzTOi97pE1FGrDleqkE7bv6\n9et/SNdkXJdN223m2o0AbBk4b1j3SfNnDHya9UfuPU7N4Oc4zb3HcSm+7kovVp/tfut51jdYf+6+\n8zzXZrH1ljJe+03wm7L1773Y5BFL3LqCWnAzXZt9rD918G2h9Xzrt9y6DwawAdgAbAA2ABsGsCEA\nG4ANwAZgQ4zrAGwIwAZgK8tHuvdQBOod1sUZ+BxXlLFPO1sPiXNwv17X+iu3TWHNdutqnwJg6+Re\nMzeuzZNerPSCNMW16RV6XvviX9bXAtgAbAA2ABuADQPYEIANwAZgA7AhxnUANgRgA7CV5T7uPbQv\nVVrgXS+WmppO3RAHxQItT8AZ/h4CbGofruuo/+9y7fZPAbA1drDuHeuTvcS1Ioe49T7jPn/9jHyZ\nADYAG4ANwAZgwwA2ABsGsAHYAGwIwAZgQwC2nAVseq8mXixiLIhma5imz3GBe48hccv39mIRY4Gf\nCgE2qch6qlv+rtuuTZUAbNJwB9j0Wk22sNGLRdbtFredQW26r62fdu+fD2ADsAHYAGwANgxgQwA2\nABuADcCGGNcB2BCADcCWDGALNMKLTayQrmi2w917XFZBu1dDgE2w723rf3uxGUYPs97d+qQkAdv1\ncYAtkF6nWVnXOmD3R+sGoedVr62vg22/c+91J4ANwAZgA7AB2DCADQHYAGwANgAbYlwHYEMANgBb\nKoAtAFzpimar40DWJ9aty2gjqPVDCLAFEyMMjms3JQ6wXe7FZiitG9fulyHAdqADcbvHtQnquem9\nj7dekGC7HvNi0Wz1AGwANgAbgA3AhgFsCMAGYAOwAdgQ4zoAGwKwAdhSAWyBwtFsA6rwWfQ+W63/\nZN0l7rmh1rpmbQsBtjFuu8aG2u3rtkPLu7llZ7rHo0PtRjkwGAC2Hq7Norj3vcot7xqCbeE0VqWG\nrrf+OG1fJoANwAZgA7AB2DCADcCGAWwANgAbArAB2BCArdYBNild0WyasVSzcipSTRFtqnH2Z/f+\n91ifGwJszdx3q+i0571YzbRgVlG1f9O6uduW/3PtfuuWK1JuuVc6RfR297o/u/d92z2+zj1f7N5D\n2/aqa6N6b99aHwdgA7AB2ABsADYMYEMANgAbgA3AhhjXAdgQgA3AVhXAFigd0WwCWad6sZlFBcuu\n9WIpmpImHDgm1LaF9XQvlt55oXVHt3yW9UVebBIEqan1OdbXWM/zYqmgnaxPiHvvg63nu/WpxtoB\ncc/XdTBtofXV7r1bp/XLfOCBB8xzzz2XVg8dOtQMGzYs7evVtmZqm9u0aQNUi/NDDz1k3n333bT6\nxBNP9J3u9WpbAWzZAWwPPvigWblyJcZVcuPGjRVCfr4Xm547GeczxswsYLv88svN3LlzsfXZZ58d\nXFOOSLJ/tvVi9UdyFrC1b9/ejB8/vkL37ds3coBt1apV5uabbzZLly6NjMeOHWvs8f1xCufQlpwR\nszOu0/kjmfOMvQ5Grj/bY5trNwKw5TZgkzI102jOazgQCeegRwPYMgfYBLfr1KlDP8PV4Vu5bGcW\nsPXp04d+VjWPyeGudFsq+6Jhw4aRAmyzZs3K+f5pr93fe7sWf0bpVSt3Q5rUd5Kfn18b+jPXbgRg\nix5gCxREs73jfnBEFegkBsOl3a5dO/Pee++l1atXr47s/li3bp3ZvHlzWj1u3Djf6V6vImLcdp8E\nYMscYPvpT3+6I4p0/fr1GGfFw4cPV79byWU7s4DtoIMOMr179zbXXXcdTtElJSVbvdxOm1IUStMk\nPcMCtq1RAmzajhYtWpg1a9bkpJctW0bKc/bUKNljxQK2j3K5Pw8cOJBrNwKwRRuwSUSzAdgqb9Vh\ny2BtMGqwZbAG28MPPwxgyyJge+KJJ8zvfvc7jLPiUaNGMUjPEmDTuSOZcwwubZs2RV2i0M2QBWxb\nogbYcrkWaXDtBrDVLKmGXi7356OOOoprNwKwRR+wBVLdNE1eQDQbgA3ABmADsAHYMIANwAZgA7AB\n2ABsdE8AG4AN5QhgezZFnwNgq/I1gGg2ABuADcAGYAOwYQAbgA3ABmADsAHYAGwANgAbyiHAttH6\n9RR8LoAtbdcAotkAbAA2ABuADcCGAWwANgAbgA3ABmBDADYAG8oBwEaKaPUBNik+mq2IrgpgA7AB\n2ABsADYMYAOwAdgAbAA2ABsCsAHYEIANwJa6iGYDsAHYAGwANgAbBrAB2ABsADYAG4ANAdgAbCii\ngO3PKfoiAFvGrgGaGZloNgAbgA3ABmADsGEAG4ANwAZgA7AB2BCADcCGIgbYHrdem4KnRAywLfdi\nUWHJ+ulqBGyBRno7o9kOB7BhABuADcAGYMMANgAbwAzABmADsCEAG4AN1WzAluspopdYP5KCb60B\ngE0KR7PdZF0AYAOwAdgAbAA2ABsGsAHYMIANwAZgQwA2ABuq+YCtvXXnCtzeI0U0m9eAIJrtbetD\nAGwANgAbgA3AlgbANmjQIN14YlwlN2jQwNSrV2+b9RfJ2N60rGU8mlnANmfOHP+7KSoqwta238lb\nku2j9quZm8Pd7ow6der8YI/D7ypyfn7+d1G7Dj7yyCOmTZs2pqSkJDK2wDPoo18l0z/teeMT+730\n5wya8fPzB/YY2G6vcd9VZPu9fB+1/sy1G2URsN2WxL3tEgBb1n9kae3F0nMVzaaxz+Ys+N/Vef0C\nsAHYAGwAtowCtj322MOMGTPGXHvttRhnxepvdlD/IePRzAI2nYt0AzVr1iycojt37pzrUR2Nrcck\n6elRvQ5eeOGFOXsetcf2No+IzGyoXwrHyvpc7s9cu1EVAduJ1lfF+Qbr1dZbrVdZD4oYYFPa55YU\n/IcaCNgCCXrd78Xq4GXan1Tn9QvABmADsAHYMg7YFi5cSJojzprV3xikZwew7bbbbkmdY3Bp69zs\nkTZVa66DUbQ9tkl5rnlamcv9mWs3qiJgK89Drb+0Hh4xwKYfoG5OwZfWYMD2d+sfe5mL0Av7HwA2\nABuADcAGYMOYQTqADcAGYAOwAdgQgA2h9AI2+Vl33SVFFMAGYAOwAdgAbAA2jBmkA9gAbAA2ABuA\nDcAGYEMAtkr459aPRxiw7evFIvDK80AAG4ANwAZgA7AB2DBmkA5gA7AB2ABsADYEYOPajSoL2Mqa\nRbSL9WnW31jfFGHANiKJe/e/RBSwaYbRTmU8t7d1X+u2ADYAG4ANwMaNBYANM0gHsGEAG4ANwIYA\nbFy7UWYBW0WziH5kfVCEAZtAYXwx/6lerO7aBuu3rI+OKGD7j/X5ZTw3xG33oQA2ABuADcDGjQWA\nDTNIB7BhABuADcCGAGxcu1FmAVuiWUTlK6wnW3eIa59LNdjaWD9nfU+EANuIECj8wvpeL/GMoLe4\n7T4QwAZgA7AB2LixALBhBukANgxgA7AB2BCAjWs3Si9ga+sAWgClFJ12eAoQJtcmOZhp/VmEANvq\nFJjE7xxEBLAB2ABsADZuLABsmEE6gA0D2ABsADYEYOPajdII2Hq5+77R3s4U0Q21GLDNsN5aDoiq\naYDtAC9WW03ebH1N6HHY+p738pjkAMAGYAOwcWMBYMMM0gFsGMDGdRDAhgBsXLtR2gGbit9r4oLf\neLFItpetP/ASp4kGPs7LzVlET7d+1/oPXjRrsP3M+mSPWURz08XFxWbMmDFp9YABAyK7PwYNGpT2\n/dGuXTvf6V5v3759AWwANowZpNdAwHbPPff415M+ffpg6xYtWug8+r4XS5FIxhfkOmDr1KmT6dat\nW4UuLCyM3HVw5syZZsiQIZFy/fr1t7tokGT76EjOtpkHbDp3JHOOad++vWnevHmk+nPXrl1N3bp1\nt6TQ55Zb16db1FrAtoe7Nn6Rwr3tEi93ZxH9zN0DRxGw6bvoXIH3jApg6+a5qK3OnTun1Q0bNjRF\nRUVpX2+HDh38TqS/6V5348aNTZs2bcyUKVPS6pEjR2YUgtmbp7TvC/UJrfuUU05J+/7QIFpO93pH\njx4d7JNBADYAGwawMR6tOYDtJz/5iX/eGDhwoBkxYgROwd27d8/1/tzA+mbrO5Jxfn7+11G8Dvbq\n1cvoOMxF77333kRkZkejkj1OrNfbMeP2XO3Pw4cPN9UMC1DNAGzxs4jmcopoollEA5/kItz28KI7\ni2hFzCMys4j6X+b7778fmTRAbWvUtjnTKaKZSGuN+H7uD2ADsGEAG+PRmgfYdP5I5jyDd1rXB/rz\nTtnIqg+5DtYsjxo1CsBWA4FEs2bNtuRqf9a2AtgAbHE+zfrCHANsC6ynxwG2fbzKpU3WZMB2ibdr\nOq+A6W+tP7e+KAmACGADsAHYAGwANowBbAA2DGADsAHYEIANwIaqCNhSdRQA2+vWK7zSKaI9chCw\nlWel9aq2XGRmEQWwAdgAbAA2biwwgA3ABmADsAHYuA4C2ABsADYEYKs5gO0BF8F1n/VTrs0a9ziR\nb8hBwNbdbffRADYAG4ANwAZgA7BhABuADcAGYAOwAdgQgA3AhgBsqQK2g62fdXDqQ9fmA/c4kdfn\nIGBrb/299XgAG4ANwAZgA7AB2DCADcAGYAOwAdgAbAjABmBDALZUAVuiWURzMUW0rFlE9Vnvctvd\nF8AGYAOwAdgAbAA2DGADsAHYAGwANgAbArAB2BCArSqATamSF1h3yEHAVtEsog+l+FkBbAA2ABuA\nDcCGMYANwIYBbAA2ABsCsAHYUBUB2yHW86yvtP6R9YE5ANiq6qjNInqVWz6qEp8VwAZgA7AB2ABs\nGAPYAGwYwAZgA7AhABuADSUJ2H5mvSxu2cXW2+Lul7+xng5gi2QNtsoYwAZgA7AB2ABsGAPYAGwY\nwAZgA7AhABuADSUJ2J72SqcPqk6ZCuK/aX2cdTe37AXr76yPALDVOMCmlNeZXmwW1N+47/QO69Os\n9wSwAdgAbAA2AFstu7H4+c9/bubPn19pX3zxxf73f84555gpU6aYCRMmmHHjxpkTTzzRnH766Wbj\nxo1AMwBbrQNs119/vZk1axa2Puqoo4z9Dj+3+29Kkj7Fuk4uAzb10bFjx1bo9u3bR+46WNVrSnW4\ne/fu2h8vpdBHj+DMnHkgUVRUtC2Zc4zG5FHrzzNmzAjG/Bcl2ecmWjemW+Q8YFtirevlAQkgjmbf\nvAXAVqMA22Tr/3qlIw3D0Yd/tj4KwAZgA7AB2ABsOQrY1q9f7++D+++/39xyyy3mmmuu8W8s8vLq\nm8LiEtOwpLEpbtzU/1tU3MgUNiwxBUUNTYOCIlO/QYHJr9/A5OXnm3r18kzdunVNnTp1yjyug+fW\nrFkDNAOw1TrA1rp1a9OkSRPTpk2bWu9WrVqZFi1a/NCyZcstFbl58+Zb3TmkbQ7350ctZNuUjO15\n9puoXQePPfZYnb/8631UXFJir38NG2633lKR7WfbVq9evQ84M2dc/e0x8A+7vzdV5Pz8/I+i1p91\nfbD9Sd6aTL+zYypFNZ1Ot8h5wPag9bNlQJc11k8A2GoMYJvqxaIN37Oe5e2cHXUvLzZjqGqwfeY8\nEMAGYAOwAdgAbFkGbK+++qoPwNatW+f/Yrpq1Spz1113+RDswgsvNLNnzzZnnHGGOe6448zRRx9t\nDj/8cNOrVy/TuXNnP8phr7328mFBo0aN/EGbBoT2Rq5cCBYPxIpKmph9Ovc2nXr0N10OPtL0HnSC\nOXzUBDP4pGlm2Ng55rjJC824OUvNmfNuN5MW3G1mXvuIueC2Z8z85S+ahT/daK595G/mxif+4fuS\nFS+ntJ8xgC2XAJugEqnlpE2lQSujdh1UuuX+++9vpk2blpMeNGiQ/ZGpHufnGjgWzeX+bKP5tlTn\nDTfKGmBbbP1yGdDlt9b3RhSwjbQ+MfR4sPssr3qx6OH/Z31QhACbUnc/d9vfsZxtFmjTDwAbAGwA\nNgAbgA3AFhqIzZs3z3/t6tWrffC1YsUKs2zZMrN06VJz+eWX++H+5513nh/yr883cuRIM2TIENO3\nb1/TtWtX07FjR9O2bVt/G7UPCgsLjf1l1o8CS/V4EQjTa7WO4uJiYyM+/PVq/RoECjDofQcOHGiO\nP/54c+aZZ/qDM32GxYsXmzvuuMOPYFu7dq15+eWX/cHjwUeO2QHG0mUAG4ANwAZgA7AB2ABsCMAG\nYEPlArbNLjJNcO1c66+sx8a1U5rwD15sRtGoAbafuucCOKjaZNuclUL5F+vt1p9YHxoRwHahi17r\nmwQsm+G2+3AAG4ANwAZgiwRgu/fee/3vw6Yg+fBIqY9LlizxB08B+FKNjqlTp/rvfcwxx/j7efjw\n4WbEiBGmT58+pkePHma//fbzB10CVkoXEcCyg+hy+64AWeCyIsW0XOtSZJnWveeee5pOnTr5qZm9\ne/f2QZzgwfjx483kyZP97b3uuut8gCeQJ6inGyJFtwmIZaK4M4ANwAZgA7AB2ABsADYAG4ANwIay\nCtg0gcEi60ccaPrOXQN/H2qz3sG1F73SRfOjANhOdsuvt97Xuo0DVX+NazvA+t/WayMC2AQL/5Qk\nLOvotnsygA3ABmADsKUFsGkQo+ipuXPn+pBp6NChZubMmf7NwKRJk/xC+CeddJJf50LASwWxFX2l\nG19Bs549e/pAap999vHr+QRpkEp/rAh+qY2tzWEKCgp8yKW0ST0Oni/rdQJsgmGqo6T2ig5TVNiR\nRx7pwzBttz6DYJgi2BTJli0gBmADsAHYAGwANgAbgA3AhgBsKPKALd77WA+Pi2C73/oaL1abdI+I\nATaBtXdCj3uXA5sutv4iIoDtYeuNScIy1WRThN4sABuADcBWOwBbXeum7mBub93Vvf8Q61HWZ3ix\nsOTZ1vOsF1rf5MWmHxa9f1LbLPClAYstUr0j+qsiAKY0R4EvO2jwa4Y1btzYB2d6vSywpXWoTb5f\nWD9xNJmWq43+ajtUk0xRaapRJiAmcKcb6YkTJ/pRbJrpUq9btGhRUkAs12cRBbAB2ABsADYAG4AN\nwAZgA7AB2FDWAdth1vt5lSv6HwXA9hPrN0KPe7p2xyb4PJMdiNozAoDtZi+W0to2ie9poNvukwFs\nADYAW80AbHkhACb41cs9L/g1Jg6ACX4tjgNgyulXfr/Cit+2ftd6kxfL999SzveiE9zn7uTxD/e6\nv3mxkF75Les/ul8l/NmQgpm4BLsEz8oCYooQE0zTDagiw1Sgv1u3buaQQw4xhx12mB+9pki2du3a\n+bXLBMUUIXfppZf6N9DXXnutf3O8cuVKZhEFsAHYAGwANgAbgA3ABmADsAHYUBQBm87FF8YtO8b6\n1BwBbBO8WNrr0NCy/3X3qvGf534HsaIQwTbabcvcJL6n+11k3j4ANgAbgC2J/bx161bzr3/9y7z7\n7rvmrbfeMq+//rq/v1RAXpFRGsTqxkzF5S+77DI/VVL1wJQWOWbMGL8YvqCSZoMUUHL75JMK4Jdx\nz29y4EvA63UvNuvMSw6mPW/9a+tnrJ9zft49p7ZvWv+Pe/3HXmwK4a1lvNdW915vu9e+6MDdve6k\n4adPqraZosVOPPFE/8ZV9cSmT5/uf159bn1+ZhEFsGEAG4ANA9gAbAA2BGADsAHYEgK21V7pGmxR\nBmxKj9zgANO11kd4sbpzuvdcbn2S9SnWq7xYnbkFEQFs8m/cPbKCXNp4idN9l7ltXuwxiyiALVcA\n25tvvlkKgOmzPP300z4A0wBTRfHDAEwwSCmEiQBYly5d/Jpc5UVnyUprFEhSW71Gr9U6VMxeEEr1\nxk499VS/qP2ECRP8WR71vSlqy61DEWcPerH87icdHHvBi00F/KckItC2lAPE7nC/Gix0J4QpLgpu\nlIuM6+Ui5XRwFyZ7DOb6LKIANgAbgA3ABmADsAHYAGwANgAbgA0B2JIGbLJSYB/wdk7gsC1BUMe3\nXiydtE2EAFtnL1aHTdukCSrudiDtBheA8pF7bpUDjQA2AFv2AJsit8LRX4kAWDj6KwBggl+CVoJX\nAllKNRTUEsRRMfuy3k8pjGqn9npdAL+0PkE1rTuIulJa4sUXX2wWLFhgrr76anPFFVeYAw880Ldm\nqLzwwgv9tjNmzPCjtRJtV7BNel8vwQyTAYwLRbBtDAGxm9zBGgAxwbAxVQBiGTkGAWwANgAbgA3A\nBmADsAHYAGwIwAZgQwC2BO7hAjsuc/e3t1pfbT3D+sAkPm9NA2yyarBd4sXKKIXv8b+3fsX6TK9y\n9fWqH7A9//zzfoRSOq00Nznd69W2Rm2bH3rooYwCtkRW4XsVvNdFUOBJs0hqNknV6VKNruHDh/sX\nvbFjx/pQS9Bt9uzZ/qyOumjqr9YjMKcZHpcsWeLX7tINmIrbX3DBBT4UU+F7RZMpemzAgAH++rt2\n7Wr22msvfxZJbUei7VPRfQ1ktW2qH6YbusGDB/sRalqf0iMF3gTgrrrqKv/m7M477zSrVq0yjz/+\nuHnmmWf8gvpvvPFGov2cU7OIAtgAbIADABuArWy/9tpr/o9KeK1Zvnx5cB1UGkn7JKzBbZ1cBmwa\nl2gcU5E1HtJEQ1EDbBrDKaI/Ktb5xQK2j5Psn3JLzuLZGYvq/JHMeUbjxaj1Zzt5mCJ+zk+h3+XT\nLQBsNQCwHeXFUkL3SINrImAL+xAvNgPs8FDwS2VdrYDt4GyDH4yz4NEANgAbgA3Axni0dgA2RU5z\n3auSx+Rw978tlX2ha2yUANvQoUNrQ/9UJMPunMkzqlZuPyf1najUSy3oz7fSLQBsNQCw/cKLlSoK\nL+vnxeqxdcoxwJZOVytgU6SPWbdundm8eXNaPW7cON/pXq9qgnmuNlhUtlmzNWqb9z7sDNNx2Lmm\n4/Dz0urepywytz71QVp9/q2/9rf5kUceidx+9mIFHwFsADYAG4AN1QLApmNQUdnr16/HKdpGmm/1\ncjttSlEoTZP0DHuN3RolwKZ0S2UrKOI/F62yLR4pz9lSo2SPFXt9/SiX+3Pnzp3V51bSJSID2DQJ\nwM9Cfs/6P3HLAp+VA4DtDNe2F4CtBgM21e2iBlvmtvnhhx/2t7nDkeeY/Uddknb3O/NG88Dr36XV\nV676vYla3wj2M4ANwAZgA7AB2GoXYJPp+6nbfn/UJQrdwDVr1mxLFAGbynbkojU2AbDVPOn6msv9\nWaVrAGyRAmyp+EYAG4ANwAZgA7AB2ABsADYMYAOwAdgAbAA2ABsCsAHYAGw7rTTJ/VJwOwAbgA3A\nBmADsAHYAGwANgxgA7AB2ABsADYAGwKwAdgAbOkxgA3ABmADsAHYAGwANgAbBrAB2ABsADYAG4AN\nAdgAbAA2AFvOAbYB1ucA2ABsADYAG4ANwAZgA7AB2ABsADYAG4ANwAZgA7AhAFu6ANtn1i+F/CfX\ndmPccvnBHABsl1l/C2ADsAHYAGwANgAbgA3ABmADsAHYAGwANgAbgA3AhgBs6QBs91v/NwW/BmAD\nsAHYAGwANgAbgA3ABmADsAHYAGwANgAbgA3ABmBDALbMGMAGYAOwAdgAbAA2ABuADcAGYAOwAdgA\nbAA2BGADsAHYAGwANgAbgA3ABmADsAHYAGwANgAbgA3ABmADsAHYAGwANpQNwNbJ+h3r0wBskQNs\nZ3qx2nMANgAbgA3ABmDLZcD2yiuvmDVr1vig4LzzzjOnnnqq6d+/vykuLjZ169YzDYsbV8mFRSWm\noKDI1K9fYPLy65u69er5+1nvCQgAsAHYAGwAtozrjDp16vxQWFj4XUUuKCj4LmqAbfbs2f4Ywm57\nZGzP4/b6Wlf+yvqLJPyJu7dBGVT9+vU/sN/N9qKiou8qsr3+fB+1/my3WX1uW5J97gvbfi29osYB\ntgCeTQOwRQ6wpcMANgAbgA3ABmCrCV6/fr257777zDXXXGNmzJhhjj32WNOrVy9/39gbL/9za+DV\ntm1b069fPzNmzBgzYcIEc/7555trr722XC9evNhcdNFFZvLkyeb444/34dx+++1nmjdv7t9ABOsW\nWNCvpwMHDjTjx483GzduBAQA2ABsADYAW+bV2HpMkp4eNcAWRINp/fqBKBdt4cg2+nNW1C+FY2V9\nLvfnPn36aOzGeAPABmADsAHYAGwANgBb7QNsL730klm1apUPvGbOnGkEHTQ4atOmjcnPz/c/l0Ca\nHmu5nlc7tdfrBODKW//rr79u1q5da5YtW+ZHuYXXX89FpOkX+c6dO5shQ4b4x8Hll1+e1LoxgA3A\nBmADsNWsa3cUAVuy440o2kaU059rnlbmcn/WtRXABmADsAHYAGwANgAbgC1nAdvzzz9fCqIdc8wx\nPtDSoM31Dx92tW/ffgfkmj9/vg/FBMeUClrRe7zwwgtmxYoV/uv0eq1H67NpE/76FZEWQDr9whle\n/4YNG7ixB7AB2ABsADYAG4ANwAZgA7AhABuADcAGYAOwAdgAbNUH2JQyGUSJhQFXPERr2LBhqUix\nVCGXQFtFoE711w488MBSkW6rV69OCtJhABuADcAGYAOwAdgAbAA2ABtKO2Dby/oY6y45DNjWWz8O\nYAOwAdgAbAA2AFuFfvnll8tM5QyixKqSyplMSqfqoXmkdALYAGwANgAbArAB2BCADcAWJcCWSpRb\nVAHbb73UZtsEsAHYAGwANgBbLgO2+FTOAG7phj2IEBNMKysKLZUoMfXjZFI6Dz/88EpFu2EAG4AN\nwAZgA7AB2ABsCMCGAGwAtqQBWzfrS6yftH7btZH/bP2U9eXu+wGwAdgAbAA2AFuQyqlZMnXDkKlU\nzsBqq/TM8lI6mzVrViribenSpSnDOgxgA7AB2ABsADYAG4ANAdgAbAA2AFulAdsJ1v9x2/Sp9R+s\nX3B/Bdi+dc99bn0KgA3ABmADsNUKwJZMKqes7a5KKmeilE6BuGCK9HBKZ0FBwQ5gN2XKlEq/Fwaw\nAdgAbAA2BGADsCEAG4ANwAZgSytga2u9yfod6xOt9/QS19EbYf0/1v+27gBgA7AB2ABsOQHYHn30\nUXPLLbf4QGvixIlm4MCB/n7WDbBm4/QS1CkLR6G1bt065Rps6neCYqp3Fo5806QCXjkpndpebpox\ngA3AhgFsADYAGwKwAdgAbDkM2Paz7hRRwDbMbcvxSWx3X9f2RAAbgA3ABmCLBGATyBo7dqy/7lmz\nZpmTTz7ZB1cdO3bcAbTkoqIif9khhxziP54xY0ZSqZxl3VjEp3RqkKIZOcMpnUovTZTS+eqrr3Jz\njAFsADYAG4ANwAZgA7AB2ABsADa5q/Wx1lOcz3YRUJ1yFLCl6poE2E5y29Izie3e2/p76zMiBdh0\nQhszZkxa3a5dO9/pXu+IESP8E6r+RmWbBQgyCdi6Hb/QnDDv/rT6yLOXmKj1jb59+wLYygBs9957\nrw+yBNCUVjl48GDTrVs307JlS5Ofn78DaCnarGfPnv7xpWg1RYYpem3NmjWVmuRAKZ2KjDvjjDNI\n6cQAtmoEbDrmdMxXZJ1jchmwKSJ3+PDh/vmmIh955JFmwIAB5rDDDvP3ycEHH+wDyO7du5uuXbua\n/fbbz+y7777+9Wyvvfbyrf0CYKs9gE19I5m+VFJSEjnANnTo0KTOGTXJ9ly33X7GDdark/RIunLm\nAZv6f4cOHSq0xqT6YTdK/VnXVjur/ZYU+txy6/p0i7QCtlHWGx2EMQm8zfrXLhIKwFYzANshblsu\nS2K7z3FtB0YFsIno+tErurFNpzt16uQ73evVtkZtmwUzMgnYDjjhcjN+4WNp9dHTb/G3efTo0ZHZ\nz9pWdwAOqo2A7YYbbvAHHZMmTTLHHnusP1BRe/0NUjlVF22fffbxIZcG8Wp70UUX+ceT2lV2koOK\nUjrt4IOUTgxgqz6da31Hkv5rlACbImEFzI466ijz2GOPmTvuuMNcd9115pJLLvF/UNAPBYowOPro\no31YpvOfUsyLihubho2amuat2prGzVuZkia7maKSJqagqNjk5df325QxUDf18vJN/QaFpqBhiV1H\nM/v63U3Tlm3seTbPj7QFsNUKNbC+Odnjyl57v44aYNP4QVHrgoi5aI15BH/oyhnXqBSuP+vtxFfb\nc7U/9+7d21Qz4MhFwHac9XfWGvPdZD3D+nQvVhT/FDf+WWH9hRcrqN8VwFZjJjl4xH13q6wnWg8P\n+Wj3Xd7v2vzai9AkB/6XGaV0S1JESREtS9pWd+Lon4uAbdGiRaZx48Z+VIWAmG4oFW3Wtm1bP30z\nuPlTRJiWKb1SERUBRNMNcVUmOVi3bp1/86rUUL2PbmqTSenUehcsWAD4wQC2iEQbpBOwrVy50tx5\n553+OUDnCEH466+/3j+fCYBNnjzZjB8/3owcOdL/MUqRyIoS048BLVq08M9lgm1NmjTxgb3qPSrq\ntjwAFrbgvn5g0PoKCwt967UCYnt26GYGnzTNDBs7xxw3eaEZN2epOXPe7WbSgrvNzGsfMRfc9oyZ\nv/xFs/CnG821j/zN3PjEP8r1brvvmXK9SQBbLaFx9nwURcCWbMR8FK3zHICt5kEUC9i25Gp/1vUP\nwJZ2wPasFyuUv59XcR2vz6yXAthqDGDraH2vF4swLGscJ7j2sHVnABuADcAGYKsUYLviiivM3Llz\nzYQJE8wxxxzjpyQpbL5Zs2Y7Uip1w6ibWqUn6RcxRasJomnAociNqkxy8NRTT/k3xNdcc41/86t1\nC+IpdTR4f93g6m///v2TSulMpfYMxgC2rCnPuqm3s25JL3fufEa/8p9++uk+yD/ppJP8lHFFpQp+\n9erVyxxwwAGmffv2/g2L4JX+Cn4J7qcCvxJZ5xmdY7QOpRUp8lUpmF26dPHPaZpkZdiwYX5ZAJ33\n5syZ44N+nSt17grS2TXDcVl9QyCsIlhWGQPYEIANwIYAbAC2rAI2Lbs+SejyqFd61k0AW/UCtsDK\nqBxnfZ71Bc7nW59WCbAGYAOwAdhqKWArcTezY6wXa5v33ntvH6IFqZy6SRWYUn0HQTKBLkE3RbDp\npreqkxysWLHCH3BMnTrVH8zrZlnvGZ6lU1BN0Wia9ECwTdBNr02lBhuADQPYKn9P7gYJ7eMA2Ch3\n7lCx19nW86wXunPJTe7XwNVuEPmC9evu191/WW+u4JfCHdY5ILCgfnkRYooK07lJkWaKmtX5RHXK\nBOQCIKYUTc9NjqLvZ8mSJX5ErKCYziuavCQbNdgAbAA2ABuADcAGYAOw5Qxg+5v1XUlClxdcNFQu\nAbYjrRdEHLBlwgA2ABuALccAWx13UzzEi81is9jd8L7ubnCD7VSo8v/of83MGUC0efPmpWWAIDim\neg/hiQ1U+84OXnbcHCvtShAvqJemGwDd8P7iF79IqQYbgA3XMsBWJxT9FQCw/u6YH+Xg15Q4ACb4\ndUcIgD1h/aI7L7xtvcmdE7ZWAL9+cG2+8WI1RT53/sot/9Z6ezmv/dL6v9bvW79p/bL109YPWt+m\nc9Kee+7pR64pik1RYrqRUaSYaidq31511VV++ncUJzkAsAHYAGwANgAbgA3AljOA7W7rLS7aqSzY\n0tb6YjcGmp1jgO1cN6YDsAHYAGwAtsgDtnpxEO0md5P6rjvRazu+d4+fdjfW81zkSS93c77jGKzK\nAEGDdtU7U2SbokWUOqqb3iDqROlWwcQGqo0m4KZ1qS5SKjXYAGw4Kr700kvV7z/ydk1/TATAwtFf\nAQB7wh238QBsc+j4LsvbHewS9PrYixXe1Ws/cH83uWX/dbDrm3Kiyn5w0E2f5S8hGPaE286b3LYv\ndJ/lDHeOGeU+b1d3ntJ+KEzy3LYyl2cRBbAB2ABsADYAG4ANwJYzgG1/F8Vm3D3XU26M9EToh8wv\n3PPPWO8ZMcB2hfUfyrHGlt+FHj8BYAOwAdgAbDUZsBV7O1M557mb7wCiBTfE37gb8NXuRneKu4nX\nTW2DZI/BigYIAgbTp0/3o8+6du26Y3IDpWZ5Lp1LN7qKRtPkB4JtikrTADKVGmwANpwNv/rqq35f\nUlrg6tWr/Rp+SltWVJRq+mkQet555/ngWHX+dL5QFKbqEqoWmGajFTBW7S2lI6pOl/q/Ctl7Fac/\nbnXg61/uAixw9Zbz2+7vm+7vX63/1/o96386wPalixIrC4htdhDtbTewyzQQS7cAbAA2BGADsCEA\nG4AtCoAtyCS4zIulgL7jfpj8zP2QqXHcL61nWbdJAOdqOmCb6QCaxq33JfDLblwbPL4hIoBN99cX\npOhOADYAG4AtGoAt2VTOeIjWPxSFVuVjUAMETXAQpHMOHTrUL9qt2mwBRPNCtdGCyQ00wNCg/aab\nbkqpBhuADZfl119/3Z+wIgzABL+WLVtWJgBTn1V0ZBiACX4J8qYCwNS/dbOotnqN+r/WoXpeWp9S\nngXW1LeVVq2C+/qrY0XPCz7rxlR92UZwbncDkvLSLnMBiAHYAGwANgAbgA3ABmADsNVWwFZZRyVF\ndKAXKym0wYvNhpoLKaKjHThMZSKsQwFsADYAW80AbPUSQLQnvNRTOdOp8PZocOcXCA8mOBCEEJTq\n0aOHD9oELxS9pvpHqQwQAGy56Y0bN+4AYOHorwCAaQCnviIAFo7+CgBYOPorAGCCJLpJqwiAqY8G\nAEyv0+u1Hq3zyCOP9B0UtB89erQZOXKkf7Oo/7VMz6lPC8KpbwqMKRpTkZdKY65oO5T2HH5/vbc+\ni95fn02fU59XqdJ2Wz+vJUAMwAZgA7AB2ABsADYAG4CtNgM2zTY53LpbOW0O8krPShmlGmyqI7fE\n/Wh8QSgaL8o12Hp6O9N7pzl4WJ7bAtgAbAC2LO3nL7/80o+6UQSYO0gf99KfypmqgvTSM+Ii4z71\ndtZpUsTMk3qstM7Jkyf7YCRdAwQAW81Nf0wEwCpKf0wGgOn5IPorHoBp3WEIdfbZZ5uJEyf6/U4z\nO+pmTcv0V7PLqo1SjTWL7JgxY3aBc8mAuWSBmPaD9of2jfaR9pX2mfZfMLvkyy+/XNtmEQWwAdgA\nbAA2ABuADcAGYEPlA7aFXulMhRcTRHrt4aLAlnjRnuRgpBebpOo3DhhGfZKDU902neBRgw3ABmDL\n7n5+9913zdNPP23uuOMOf5ZN3fArPUwXzOCEGpod8zde+lM5E6mei4gJ12jTST2cXhqOjJsdgnr1\nw8dgJgYIALbK+/zzzzcjRozwoZRuYPr27eunIiraqkuXLv7NjyKuNNuiUnb1vegmp6CgwE9zLA+C\n5eXl+e0Utdi8eXP/9aqnp/UJiCmaS++j70+pwJr9VVFf2h5Foanvjxs3zj+GNMOj4JhSi/WcZnpU\n2qaWC44dd9xx/mfo16+fHxGpCTAE37S9mvyirG0sKiryt02fT9ukY61///4+CNb7qA8KzOl9BQs1\nq+SNN95obr/9dh8oPvroo+app54yzz33XK7OIprTgE19Uf2nIguWNmvWDMBWCcCmY+WUU07xj1cd\nQ+m0Zn+97777zCuvvAJgq0GATX00me9PKfFRA2yXXXZZUueMmmT96GP3x0turJiMj6AnZx6i2GNl\nm6LdK7LGt1HrzzoHuHHWRUn2uYnWjekW5QK2s9w+VS2yS6xvdlFeKv3ROwcBWxCt95gXK4PyeIQA\n27+tH/VK11ULZne93Uu9Jlt5/i+ADcBW6wHb9u3bK4RoAhcaEAkYKOJl8eLFfoSLotc2b96cyVlE\ny5otNIiO+9JFp93rpZ5eCmCrgS4sLPL7W15evsnLr2/qNyjw3aCgyBQUNTSFRcWmqLiR74YljU1x\n46ampHEz06hpc9O4WQvfTXfb3TRt0co0a9naNN/dRqG1amNatN4rY1a0mGBHAMQE56ICxABsNUpz\nbUTipmRs03A/bdKkyQ8AttQBm0C652Z6rmPPNfn1G5jChiX+eUTnjt1a7Vmp80CzFq13RI8K5nfq\n1MlP1VaEqGaOfv755wFs1SB7vDxqf1jZlIztcfVN1ACbfoTSD0ctWrSIjHW9tJ9xu/27pSLbH522\n2ePpA3pyxtXf9v9/2H29qSLbdh9FsT9rO6y3JtPv7HlcJWxOp1uUC9iedfdge4WWHeLFZl9/3is9\nsUGuALbA493nrOmArbX1Wi9Wkkng87046zP8X4LlVfE7GeABADYAW80DbF999ZUPw1QfSQBAkEyw\nTNAsSDFTQX89J7gmyCbYJugm+LZ169ZMziJaVkpnEI0WpHSme6IDAFsNdKvWe5ixP16SkZvwTDmK\n6WkAtugPeO05aQuALfVjUNt78JFjzFUP/NHMvPYRc9zkhabv8HFmn869TV79WHRpo2YtTace/c3h\noyaYcXOWmgtue8bc8Pj75W7DJSte9l/72GOP+T9AKSVJqdhKz1Zkqr9eC2T0WMuLi4v1Q9F867p0\n5xqjlVEDbKmMN6Jo7Ts7TuWaUrPULtf7s66v/ABSIWD7u/WiBMtPcpFR03IYsKXqbAO2Ou5+WTWR\nNcNrh1p1cgKwAdjSBdg2bdpUYSqnYJrSzwKIphuuIAotw7OIViWlMz+XBwgANgAbgA0B2LIP2BKt\n54afv+fDtDPn3W6GjZ1juhx8pA/btP/q2aja3ffqaHoPOsGHclMX3W+uWPXGLoCtrPOz9q1qHOoH\nLtVCtJG6P7hr3xfuehhc+3QdLaCLA9gAbAA2ABuArQYDNkGzO8sASne76KiuOQDYOrl70v1Dy070\nYhP3KYrvVi82aUBNAWyakODX1l87yFa31p2cAGwAtmQBWzpSObMwi+jJXuKUzm+98lM6m9TWAQKA\nDcAGYEMAtpoB2MpyMtFuBw8Z4z++7bbb/BmHk6zB9mN3zZztrouK1lYqxzb3/72hH5ya0e0BbAA2\nBGADsNUQwCZg9p31XG/XWSb39WITAmxwkC2qgG2W9VfueU3moKjzCV4su0qf/UP33H+sD6tmwBaO\nWnvNukutPTkB2ABsYd/z28/M9EU/87dZswkmSuVUqkllUznTsZ+DdFPdrAWQTwXcQ5FomUrpBLAB\n2ABsADYEYMs6YEsm2q3jgYftuA7m5+f7123N2KsabMuWLTPPPPNMsjXYGrlrpsBaEO0dzNK2yf1q\nvtB6lBerr4IAbAA2BGADsGUbsHVwIMc4CLUg7vm+7pr1lXPUANthLkDkl9bTrX/irsX/dNflIKJt\nuPUn1r+oRsAmwPkrt30KYKlXq09OALbaB9hWrP+Pufq+18y0y+82o8883xw65CT7a/hBpmGjpqVm\nExS0SncqZ7L7OYiWW7t2rR8FJ9Cnwu1BpJyKOIcj5VTA3dsZwZbPAAHABmADsCEAWy4DtrJqsD3w\nwANmxYoVPljTRCequRb8QGahmj8DbFwNtmQGwvkuCkC1Sm9yg/sv3XV3s3t8k3te7ajrBmADsCEA\nG4Atk4BtDwd2VGttpbv+xD+/n/Ut1m9YXxkxwHa1F0tzbRdatsK1HRjXVj96/bsaABtRawC22gHY\n7t/wrbnp8b+Zi5f90px98e0JIZpmLmvZZh9zwCFHmiNPONuMnXm1mb34AR+8zb/juazNIhpft00z\nnoWj5UpKSvxoudNPP71Uuumnn36azkkOAGwANgAbgA0B2HICsCU6P2/YsMG/fl577bWJarDpF/L4\ndNAWSfYDDdxHucG9ItuCdBXqugHYAGwIwAZgyzRgq6yjANhut94Yt+x8L5aptVfccu0jpYy2ySJg\ni49ay6OrAtgiDdg6jZhn2h0xyew34EwfjgmSCZYJmuXlx+BUg4IiH6oJrgmyCbYJugm+3ffK11mb\nRTRI6dTgvkePHqZdu3al6rbZKbdNly5dSqWc6r1TiZYDsAHYAGwANgAbgA3Alvz52aWIzvB2TQfd\n4u1MB1Ud03BkWjLRbnt41HUDsAHYEIANwAZgqwpgu8j9WLVfaNnDru0Ib9d6dB942YlgI2oNwBZd\nwNZx+Hk+RGvT60SzW6cjTMkenU1B49amXn7BjjTOvPqFOyDaiVMW7IhCu+s3H2d1FtFEKZ1K32zd\nunWplE4Bl06dOvk3XEHdtm3btqVzkgMAG4ANwAZg42YIwAZgSw6wJbqBy/N2zrodRKZt8qoW7UZd\nNwAbgA0B2ABsALbkAVt3L1aK4S8OoK1z1+B3rP9sPdh6b+uJDsQtzwJg24uoNQBbzQZsI+fb5TPM\nXoeON626jzDN9+23K0SzYCq/qKlp2GIf02Tvg0yLzoN94CbwJgCXzVlEq5LS+dlnn2V0PwPYAGwA\nNgAbgA3ABmBLC2ArS0299Ea7UdcNwAZgQwA2AFtlAdshXmwig2Td2YveLKL6oetj97wixlRvbrT1\n116obrr1e9YHZBCwhaPWgplZUVlf5vPPP+9HEaXTJ554ou90r1fbmqltPv74483w4cPNc889l1Zr\nwK1tbtl1qGlp4VjTdr1M8e4dTYNGLU3dvAY7IFpeQbEpbNrGNNqji2nW4VCze7ehZs+Dx5h9Bpzt\nQ7iy3OO4C80V925Mq3+06AF/uy666CJzySWXmPHjx5t+/frtiEaTdQPUt29fM27cODN//ny/oPJv\nfvMb87e//a1a+sZDDz0EYAOwAdgAbAC2LA547c391qVLl5qKvGDBghoB2LQdTZrvbuYvfzHt1nov\nu+yyXAdsiZSJaLfaXNdtpX601LirImsMFjXAtmrVKnPzzTebZM4bNcVjx45VKRPd4LZP0i25PGRn\n/Hz22WebuXPnVujGjRtHrj/r+urF6m0l2+/yc3m8UQZg+3ccZKrIUZtFNPA+DhC2i4OLmrThVhdJ\n1skrP2KvKoBNUWu/JGotOR2cYqfEOAoeDWADsAHYAGxc4jOuMamen7MB2F544QVz//33+8X8Nbu0\nZtE85JBD/HORLeqf0evPxIkTayNgK0vpjnarLXXdbkulzzVs2DBSgC0043vO2pZEUb/cnUtERtXK\nHf9JfSf5+fm1oT/fWgsB2zDrV9znV1TVBRV4eEQBWzpcGcBG1FolpIGPWbdunV9MPp1WVJOc7vW+\n+eab/klEf9O9bv1imMkT3913323eeOONtHr06NH+zcN7772XVusGJVP7OVN9Y+XKlcG+PgnABmAD\nsAHYuMRnXHUcREnGB6YTsKnNXXfd5bdRTc82bdqYbt267Zg4R27RooXp2bOnD7OUTnfVVVeZe+65\nx/ziF78w69evL9PPPvusD+iuu+46M3PmTHPccceZ3r17++dI1RLVups3b+5P2KMSCVOnTvXXfd99\n95mNGzcC2MpXuqPdcrGuW34Kx9UMC9i2RgmwaTt0bK5ZsyYnvWzZMuOlf6Y+VPbxn9SxYgHbR7nc\nnwcOHKg+t7IWAjZZaZ8qLfCAl3s12KoTsBG1VhXAlq6ZIqNeg23ffffNKGBjP2en1h2ADcAGYAOw\nAdhq5rkuWcCm85EmylEUmiCXfkjq06ePD9Lq1au3I3Knc+fO/jlG12+VKtDNrSba2bBhQ7l9RCBM\n7ZSGE16/BU47rtl6Ly3Tc+edd96Odb/22mtV7qO1GLCVpXRGu9Wmum5n2eNgS9QAW7LjjSg6GCMB\n2GqWVEMvl/vzUUcdVZsB2x4OBAHY0gPYgqi1zzyi1gBsADYAG4ANwAZgA7AB2Go+YHvqqaf8qONr\nrrnGT7E5+eSTzeGHH246duxoiouLE0IugTABN9W8UaRZMjXYfv3rX/tgTPBN1x29h9apdCGtv6io\nyId0xxxzTKn1q35qJvsogC0ppTvaLRfrugHYAGwIwAZg87yZ1rMAbFUGbEStAdgAbAA2ABuADcAG\nYAOw1VAVWvdygGSxzhtdu3b1J8sJAJei0dq2beuDL0E2wTZBN6XcaIKdZL7vESNG+Oc6wTFFvQ0Z\nMsSHZir+rvdQ3TWVgNByXXuCSXnCkC7bBrBVSemMdot6XTcAG4ANAdgAbJUzgG2nwlFrGz2i1gBs\nADYAG4ANwAZgA7AB2KpFjUIQbZ4DHgIc71p/F4oQ+h/9r0ixAKIpgk310JKpwVbdKZ0AthqvdEa7\nRamuG4ANwIYAbAA2AFtVAJui1p7yiFoDsAHYAGwANgAbgA3ABmDLuPSrZnsHJfTrpiLRVlu/7sXq\nWel88IMDak87IDHPgY5eLtpox7muohpsDzzwgA/fFH3Wv3//pFI6dZ4bPnx4pPoogC1rSle0W02t\n6wZgA7AhABuADcBWWcBG1BqADcAGYAOwAdgAbAA2AFuaVVhOFFoQufNdCKLd5AZlQxx8K0j2XKfz\nx89//nNzyy23+KmaOp/F115TSmdeXp6fTppMSmd5NdgAbLUesCVSOqPdqruuG4ANwIYAbAA2AFuq\ngG1PotYAbAA2ABuADcAGYAOwAdgqr7JSOTeFrlNfuMi01S5SLQzR8lN8v3rudaPc+92v92jSpMmO\na6JqsfXs2dOvoTZ16tRSaaPxKaIVwSoAG4AtDUpXtFs267oB2ABsCMAGYAOwpQLYLgpFrXXj7ABg\nA7AB2ABsADYAG4ANwLar6nrpSeVMVa3j3vMJ9x7fuvf8ym3Do3o8Y8YMP3ptzZo15Z5nAGwAthqi\ndEW7ZaquG4ANwIYAbAA2AFsyHhW6dhG1BmADsAHYAGwANgAbgK3WA7ZkUjm3x0G02V5qqZyJ1DD0\nvgsTgLvgPZ+Ii3zbI/5cV1ENNgAbgK0WRbulo64bgA3AhgBsADYAW0W+IHR9GcoZAcAGYAOwAdgA\nbAA2AFttAWyNvYpTOb/00pfKGSg+pTPR+26Ki34b5V5TP4n1t9U6CgoKvissLKzQ9vvLacDWrVs3\nU6dOHVPP1prLz69v6jcosPumoSksKjENixtX2oVFsTp2Xbp08ffJ9OnT/dRbnedfeOEFAFs0ot1S\nqes2yfajHyw8+K4i24lCvovaeOORRx7xJzkpKSmJjC3wNPXq1ZO/sv6iItt6k5+47xJl8uDMy/vA\nHgPb7bXlu4psv5fvo9afdc20270tmT4n23PCWgBbjQJsA0Pn+U/T7G/cuoMfYttyRgCwAdgAbAA2\nABuADcCWK4AtmVRO46U/lTNQ+L1vCkXAxad0rnY3+Ol6X81GOtqtr0Lbwf+nuQzYli1bZkaOHGn6\n9u1r9ttvP4Eu/6Zc34HAm2rVtW/f3hx88MFm2LBhZty4cf6sqdonmjm1LC9evNi/RkycONEMGTLE\nX7dmXg36VbNmzcyBBx64A77ZGzPVBbvMi6Uwopob7VZeXbcPrJ930StXWE8o47iaHtXxxoUXXlhu\nv4+y3TEI5M68+iV7/bFen8v9ecyYMVH8ETKXAZuyE/5g/Zr1ySn002Q90/o89/9oNx5DADYAG4AN\nwAZgA7AB2CID2CqTyhlANN1gV3W2wWL3/meUAfCSSemsVun7y2XAVpY1U+qqVav8myABtRNOOMH0\n6dPHj3oQeNP3Z6Mw/MdarufVTu31updeeinhel955RWzevXqXdZroZ7q8v3g+sUWB24SRUYyIK95\n0W6p1nXL+fFGFE0UaY3UylzuzxGN8s9lwKYfR/5iXcKhB2ADsAHYAGwANgAbgK22ArZEqZwvxkWh\nZSKVM1C90A16We9flZROAFsNsyDZ2rVr/ei3+fPn+9dRRakp0i2IfpMbNWpkOnfubI455phS8O35\n558v6+Z+coK+FABh4Ft2VdVot/LqumnWOHPQQQeZE0880cyaNcssWrQIwAZgQwA2AFv1fi6lcHbn\nsAOwAdgAbAA2ABuADcCWy4CtulM5A5WV0rnNy2xKJ4AtQt6wYUNC+CbQpppQ8fBNz6mNfU796DrX\nzxIV0S8AvlW70hHtppvJSXqNgGxxcaxWn+o27b333n4qsoDs5MmT/fMngA3ABmADsAHYMq6ubgx3\nDoccgA3ABmADsAHYAGwAtlwAbFVJ5UwnxIp8SieALZqpp3EpolvL6euFGYBvqGpKNdqtQ3i8MW/e\nPDNhwgQfrPXo0cO0bNkyNvGGjYbU/7YOoundu7ffX6677joAG4ANwAZgA7ClTw3dtfFnHG4ANgAb\ngA3ABmADsAHYogTYkknlDEeBxUOAdKVTJpPS+aEX0ZROAFukb+7P9hJHa2rw/51XdspxAN8aVRK+\nfVrOcYcqp/Ki3fyoV80wK6gmuHbRRReVGmMsWLDAj2LT84pua9GihbGzL/rff+PGjf3Zb0eMGOGP\nGa666ioAG4ANwAZgA7BVso951F0DsAHYAGwANgAbgA3AVtO8cePGXVLjunbtqkiMbV759cgylUpZ\nK1M6AWw5e3Of55WdKv156PjaXA4sqwN8q3Y1dWMj0717dz8tNABndhZL06FDB39G26A22xVXXLFj\nvHHTTTf559bTTz/dDBw40E8xFXzTazVDrR63atXKn7E2qPcHYAOwAdgAbAC2cj8LddcAbAA2ABuA\nDcAGYAOwVY9ffvnlMmdWVBqT+mPdunV3zKzYq1cvpTl9kUF4VVZK56fuHPS9V8tSOgFstfbmvqlX\ndpp1ZVJPgW9ZGm8IogmmnXrqqWbw4ME+IBNsUxulicqCZyeddJJ/3l28eHGpMYYi2DR2UESbxibB\nubigoMB06tTJX6eg3KWXXmruueceABuADcAGYAOwUXcNwAZgA7AB2ABsADYAWzasGQ7jIdqBBx7o\nF2UPzqGFhYWlCrQrqkKRa4pge/XVV9M9eCSlE8AGYKvi1+iVHf0W1Ar7rhz41qgS8G0z8K1q4w2d\nV5Umqui0jh07+pAtPz9/R5qoQJwi2QTP1Pbmm2/eMd7Q2ENpp3quf//+/o8eQV03/a9l/fr188cy\nGrMB2ABsADYAWy0CbNRdA7AB2ABsADYAG4ANwJa5VM5glsMwRFMEhZapBpBAW5BypCLtGRg81vEq\nTun82iOlE8DGzX26le7UU+BbBscbgmg6b0+aNMmPWFMtNsE2z0W76ccPjSEE1gTYbr/99h3ji7vu\nusuPYtNzimpTOqmgm17brFkzP+pY76Pj/f777wewIQAbgC1XARt112oTYBs0aJAZM2ZMWt2uXTvf\n6V6vLuzaZv1N97oV1p5JwDZgwAD2cwa3WfVDogrYlE6hAWtF1iAWwAZgq6mA7aWXXkoplVPPVxai\npTh4LPFI6YwcYFMtPUHYiqxzDIAt527uq5J62tUBvHTAt9a5BtjSOd7Q7KJz5szxx3StW7f21x1E\nuzVp0sRfz7Bhw8zUqVP98/x99923Y7xxxx13mPPOO88ce+yxft1MvZ9eV1xc7D/W8ilTpvjL9ONM\nMv1Z15Nkzhk1yfbaqFmiN7i+l4xHcoXIPAjRBB8ap1RkRXg2b948I+PnTPVnHV92LLYlhT633Kv+\nyPyoAzZtP3XXaok6qWONHTvWv4il07rIyuler7Y1U9uscPhMAraRI0eynzO4zaNHjw729aAo3Uda\n3+wG/BXaDly/BrAB2KoTsMWncirSLD4KTT9WJJPKmU5v2LDBrF692t8uva8dPCryjJTO6OrcZM+L\n1n8FsNWq6JnyUk+/8XbOmFlW6mlJLYVvWRlv3Hvvvf55WPXd9CNKz549fdimfamJFRQZp2uEoNs1\n11xjHnrooR3jC71e1zG9r+BFECUn+KaSAVqfoNyKFSvMK6+8knC8oRqcapeL1uQULgoGZVajUrj+\nrLd9enumxs/V3Z+HDx8ejJ/aAdgqLequ1TL5v2ZFKQ2QFFFSRMuS9q/b1/1z9YC1v3R+CGADsGUS\nsKWayqnnBLjTHYWWyK+//nqpbVNx7iBCLpgRTzdu6nPu19mFHimdtSLaAMBGelpIe3jpST2t1ZFv\n6R5v3HnnnX6qqGqwCZgpiiaIbFaqqECcotYE0BTdtm7duh3jjUWLFvnL9YOOIoYUCa1zvv7XMj2n\n64LST3N5Nm2d5wBsNU5n2f67JVfHz9pWAFuVRN01ABuADcAGYAOwAdhyHbCVl8oZpPdkI5WzLOtY\n1vtcfvnlpQCfUofC23b44YeXipBT/4xwfREEYAOwZV7lpZ5+78YQW7zS0W+zQ/Atr7bAt2yMN558\n8kkfpukcHkStCbZ5LtpN53n9r0kXdI5/5pln/H77wgsv+FFsAmvBhDgBrNOPP7o26IefpUuXml/9\n6lcANgRgA7BVF2Cj7hqADcAGYAOwAdgAbLkA2J577rkdEE03Gomi0KojlTOw0nsqmjU0KIAdAD7d\nLCWzbQA2ABsGsFVC6Uo9bVAF+NaK8cYvzZo1a8ySJUvM+PHj/WuBrlMBQLN91r8uKIJZP8KoNIAi\nr1977TXTsmVLc/zxx/vP6XoSlH7RdUWP418DYEMANgBbBgHbWR511wBsADYAG4ANwAZgiwZgW7L2\n72b+8hfN1EX3mzEzFpvChiV+yk08RFPUVxii6eYiG1Fo8SmdQaRBopTOdKeaAtgAbBjAlgGVlXr6\nmZd86mlk4FtNG2+E62zqWqEoNcE2Pa/Ia6WLKvVUNaPC0W5aj36cCV6jgvR6jeBbMnXdAGwIwAZg\nq4SouwZgA7AB2ABsADYAW00DbIvX/NXMXfqkOXPe7WbkmReZvsPHmU49+pvmrdqaenmxVM46der6\nj/PrN/CL4WYzlTNwkK4Trtmmm534mUPLSulkCnoEYAOwRVyppp7eFAff6tU0+BaV8Yauc0G6qKBZ\n27ZtK4x207qDH34qquuWresogA3ABmDLGcBG3TUAG4ANwAZgA7AB2KoLsF314Fs7INqwsXPMgf2P\nMXvue4ApLG6849yRX7/AX6bnBp80zY9YU+SaItiue/SdrMwiGp/SmWj20LJSOrMdFQBgA7BhAFsN\nUlVTT4tD6xiVInzTeKhJbRtvJBPtFoZoinYrr66bXlsT6roB2ABsALZIADbqrgHYAGwANgAbgA3A\nlinAFp/KKUCWCKIVFBUnhGgLf7oxa7OIlpfSWa9ePX87LbjK+uyhADYEYAOw5bCqmnqaEfiWi+ON\ncLRbIogWH+2mH4f0V49rQl03ABuADcBW4wHbWR511wBsADYAG4ANwAZgqxpgSy6Vs47/WMv1vNqp\nvV531QN/zOososmmdAY3G5lO6QSwIQAbgA0lVLKpp2+XAd+KKgvfbKTXv2vDeKMy0W7VVdcNwAZg\nA7DVaMBG3TUEYAOwAdgAbAC2ZAFbkMo5bs7SMqPQ8mwdtIpSObM1i2gyKZ2aACG4GQhqtwW/2kcZ\nGgDYAGwYwFYLVNXU0+YVwTf96KKaZj179jRHHXWUGTt2rJkzZ465/vrrc368kWq027p16zJe1w3A\nBmADsNVYwEbdNQRgA7AB2KIK2DSQ1SC3ImtAB2CrnJs138206dDN7NO5t2mxR3sH0erEortsqmRx\nk91Mq7adzL4H9LUwbaQ5bMQZZqitn3bCjxb5UK06XFDU0P++FY3WqVMnf3ZOz0Wj6TvQzcDJJ59s\nLrjgAnPbbbf531+m01kAbChbgK179+5+pCVOzQ0bNtwGYMtZNXXjocqknu5vwdDH++23nw+WND5u\n1aqV/6NM8FrBJtXdFIBr0qSJ6dixo7n00kvN8uXLc3a88eKLL/rbpGNnzJgxpkePHjv2iX680kRD\ngm6nnHKK32b27Nn+40GDBvk/cDVt2nTHdVn7QOctzXoqOAdgiy5gs5GL22bNmmUqsu59AGw5C9io\nu4YAbAA2AFsUZQe8j9pB7aZkbAdw3wDYKufd7Y2EUkMKCgvt4LnENG7S1EK3Fqbl7q1M6z32rJHW\nzY6g6siRI80555zjR6M9+OCDkY9GA7ChJDTXAuVNOHXb4+QfuXzNROXCt4pST7fZ0gbb7Fhii/WX\ntg7nZ3YM8l+BN+uP7OPN1l/Y57627b7XjzoqhaDXWnBr9tlnH/+HHUGhSZMm7YBvuTjeUO3SJUuW\nmGnTppn+/fvHJv1p0dq0aL3XLt6t1Z6mSfOWprhRU/+HMe2zs88+G8AWXfW3469/2HPppopsx5Uf\nAdhyErBp26i7hgBsADYAWy3QSgAbro0GsCGEUKWVTOrpt17p1NPXFaGlc69+3Dn33HP9yK3Bgweb\nbt26mZYtW+6Ab5ooRz9gqXaZoqqVNnnLLbeYRx55JKdS6i5Z8XLa66cC2HLjvhvAllOAjbprCMAG\nYAOwAdgAbBjAhhBCqFJKlHr6iWqNBePdRBFsKk+gemVKiVRa5YgRI/zabq1bt94B37RcqaYBfNMk\nAlr+0EMPAdgAbAA2AFtNA2zUXUMANgAbgA3ABmDDADaEEELpH2/ceeed5sorr/TrTSUbwaa2ixYt\nMtddd5257LLLzMSJE3fAtxYtWuwYQ6vGmaLkVGdUY0jBOk3ko1mxAWwIwAZgqwbARt01BGADsAHY\nAGwANgxgQwghlN3xxj333GOWLl3q1wkVLBNcE2RTxJqgm8YVioJTRJvgmiCb2viw6pJLdsyIrRk9\nFRnXpk2bHdAugG+Ftm6qaqAlgm8ANgRgA7ClEbBpe6i7hgBsADYAGwNeABsGsCGEEKpZ4401a9b4\ntdhUky2Ab+3atSs1jo6PYluwYIG58cYbzdVXX+3DNzuboz/ZTyL4Vl7kG4ANAdgAbCmIumsIwAZg\nA7Ax4AWwYQAbQgih6I03HnjgAR+KJYpis7OY7gBpSj3t2rWrP8a88MILzRVXXOFHv2lm7fIi39IJ\n3wBsCMCW04CNumsIwAZgA7Ax4AWwYQAbQnGqZz3AeoL1GV7yaR56XZMk22o2xnTUZtGAviDJto2t\n8/l6UW0Zb7z66qtm7dq1ZtmyZaZJkyZ+iqiAmcBZQUGB/3qBN4E1ATaBtunTp5t58+aZuXPnmmnT\npqUVvgHYEIAtpwEbddcQgA3ABmBjwAtgwwA2VCvGMw9bf239rfXvHEBLJM1+uMn6Zes7rO+y/pP1\nH62PreB9Zlr/N4ntqW/9hvVzaQCBH1ifn0Tbk9w1cDjdATHeiHn9+vU+GBMg09gzgG8lJSUJU081\n42kA3zSLaVnRckF7tVFknd5D41AAGwKw5Sxg0zZQdw0B2ABsADYGvP+/vfMAs6JIu3BNYGQAQaKg\ngIBiRlAEF0VFFxVBzEpQwCwKK2ZwUQGFH1wDoi5mzAFcE2YxYE6YMKw5rq6oa0ZQwf7rTJ12amq6\n770TBea8z/M9c7u6urr7Tt+uqlNffSWBTSaBTazSrG3tK4pkI43zSHve2hJrXbx8edautPa6ta0S\nytmJwtvYlPPgmM9yFNjOYn1UFYENXmvnspxsAltrfgcS2ITaGznWFbH4lm3qaSymDRs2LBo1alSJ\n19uIESOivfbaq1z+Ro0alfzdaMsdo12GHBeNGHtxdPz0e6Kpc96UwCYksK28ApviromK/dAfe+yx\n6P33369W22effUqsusvFtdbUNbdv375GBbbZs2fre67Ba8b3K4GtbIO3Z8+eJat2ZbMhQ4ZErVq1\nksAmk8AmVsp3nbVF1pp7aY2t/UChKwYi1QfWWnB7B2vnUMRqRwFtiLWl1rp7x80wzhstrmOyCWy9\nrH1t7ZFKCmxdrT1k7VvvnNkEttus3S6BTai9UT3tDYhv1113XTR16tSSGG577rln1L179xJx0Bff\nNt5442jnnXeOhg4dGh1yyCF/rHzauWvvaK0OdppqA+cpl59fEDVtuXbUaZOtoi132DvaafCYaNAx\nZ0dNmraMJkyYIIGtDvW7L7/88pJpzdkMz1pNtJ8XLFgQ3XPPPTldA6ZeS2BT3DVRMXrUpKAkk/1J\ntrt+2iX8syLfW9OmTSWwySSwiZUNTMVcHAhpMW2trekJbt9Z243bU639Zm2OtfuNi6nysbVdrF1D\nixlmnFcb7PEsAltT46Z0DmEZvsC2Hs93fHDMVkyPr629d77xOQhs8NqDV14nCWxC7Y2ab28888wz\n0a233hrNmDEjOumkk6LBgweXrHrasWPHknhvFW23QpiTwFYngKfx8lyfi6Kiomp5np9//vno3nvv\njWbNmlUiGA8YMKAyfas+dVhgU9w1USHg6VPyo/vmm2+q1TCSA6vucl955ZWSHzr+VnfZqBhrUvjR\n91yz13zVVVfF3/W++mmXUI+dvVxslG3wLpXAJpPAJlYyNuN7vy8HDadYu9DaodaKvXyDKXxhmig8\n1363NtTb/4BxHm9YmGAEBaskzs0isF1rbTY/hwIbuIDn3pXbzXhdT1orTChvtSwC2wYUDrcwbqqs\nBDah9safWP/AO+i+++4r8YDLxeBJ9MILL0hgqzs0zvW3Ytsxi/A8Q9CdN29edNttt5V4v02ZMiUa\nM2ZMdOCBB0b9+/ePttlmm6hLly4l/Vh4vTVp0iQqLi6OCgsL/1i0IzSkt26/ftRx4x7RJj37Rj12\n3Dfafo9Do35Dj4/2OmJSdMDx50eHn351NOjY81eU2UF/lsCG8yrumqi4wKbYYIrBtirEYLvlllsk\nsFWh4mrWrNkSCWwyCWxiJWMnvvenG+eR9qpxixX8TpGsJfNNsnaHJ6bdHZSDxQ7+xc+HsIyKCmyD\nrOG5a55BYINghgUYPmO+W42LndYupcxMAhtEjQXWzuC2BDah9sYq3N6QwLZSUkQxZxP2uwcat3r1\n6cZ5fmJxHoQTeM64OKKfc9BkabZ+ZUFBQclKuZiqjGnXHTp0iDbddNMI8RARH/Cwww6Lxo0bF517\n7rkl8Q/xm3r66adLfiOI/ZfLIhzHzXiwLgtsirsmJLBJYJPAJoFNDV6ZTAJbnWII3/vojOzopSNU\nAKbiXMFteLbFnmUQyIZ7efMoxo3g9kXGeaJVRGCDQIaYabt5aUkCG9jYuGmtCykE7p3h/jIJbBAN\nXzClnm8S2ITaGxLYRPVSP0EgQ/0xxtqZrGPmWnuKAtlH1r6kMLMsh/4h6oAlFNUgrr1t3CI9D9pY\nfz9DLMOqtqFIVtU2kgS2rAKb4q4JCWwS2CSwSWBTg1cmk8BW59iD7/1LEvbda9z0S3AExagiCm++\nELYf0+Dt1owdnV4VENgg0M2zNitITxPYQBxbbX6W+0sT2HB9P1KsMxLYhNobEthEOfI8cax7gkA2\n0bhFbK7nO/wlax9a+8barxkEseX866cv5zsZ4hoW08HCOPBMg4caPNVOtrYPr2ETXldxxpe/bcfU\nxPMsgS0ngU1x14QENglsEtgksKnBK5NJYKtzbMP3/nEJ+y5gpwceXu3YYVrXuCmaj1nrYq0/O1Pw\nHMCUS0wjnZLhfEkCW0teA8p43zN0tpbw80FefjTY37X2BTtp/SshsOE6lgbn+5h58ey/rkdDqL0h\ngW0lp8CU9x7bL0Egg8fxXdaeNs4DDCLXLyn9seXct5R/lyfs/5aDM/BIg2caPNTgqTaZgzUDKyKU\nVRYJbH+awIZzKe6akMAmgU0CmwQ2NXhlMglsdY5G7CRdmLDvAYpOMeiYwqttS+O8FPAug1cb4rh9\nxbRjrO1l3IqcuQpsDUzpqp++Qcj7gJ+39PLDo+FrNvDnU+BrX0GBbceE801h3itN+ZVKhVB7QwJb\nbVPPpE+vhDg2LRDIsNjLGxwkSItBtoziBxal+Z5/fzblp2Mu53v2PeNiVcJD7Qaeb2JtCmUS2FYq\ngU1x14QENglsEtgksKnBK5NJYKvTXG1cTLMeXhqmScI77DwvrYlxcc8Qi60xt/MCMessil71KiCw\npZE0RXQ466kh3O7IDiLyFVRAYEtCU0SF2hsS2KpV48kikPni2DxPIIM37xKTPL0Sohg8zBbRvuBf\nvFfh9ftrglCG/W+y/Lt4zlgow7XstyILZRLYVhqBTXHXhAQ2CWwS2CSwqcErk0lgq/NgNc5XjPNk\nw0ps8FaAN8MTxnm4+axhbY5xHhKT2TE7gJ/fp1jXIMO5qiKwrc8O5Jwg31Gsu8ZKYBNqb6i9UY0C\nW9MKCGSxOPZ+BoEM71V43P7HOO9cTMf8t7V3uP0J363fmvJTNJex3DfqmlAmgW2lEdgUd01IYJPA\nJoFNApsavDKZBDZh3EpvEMqwqts0diTzM+Tf1NoJxk0tRQfvWJM+LdSnq3ELK+RCT2t9vO3N2IFs\nGeTDdWIl0d0TysjnMRvkcL5i5m2tx0GovbFy2wsvvFDyHcyZMye68soro5kzZ0Y9e/bE9/F4FoEs\nTRyD/Wac9ximw8MjDIH4sWLlszRMa3+dghnEDEyxXCqhTAJbHRDYUL7irgkJbBLYJLBJYFODVyaT\nwCaEEGpvrEjtjeeff76MQDZ9+vRo0qRJ0fjx46MTTzwxOuKII6LBgwdHAwYMiLbddtuoa9euUadO\nnaIWLVpERUVFif2I/Pz8iCIAPMew4iUC8GOlyget3Ud7kGnPGDclHjHIFiWIbrkIZfB26yuhTALb\nKi6wYXVXxV0TEtgksElgk8BWrQzPy8v7vbi4+LdsVr9+/d8ksMkksAkhhFB7o6zdd999UWFhYWpf\nwN57iVCWn18QFRQUlOStV69eiahWVLSarbvqR/XrF0f2/qMGDRpYaxg1bNgoatRodZu/MBbGwnIx\nHTNePRgeblgNeZa1c6z93dpI47zKsDgKvHWwqEojPYorNvaZ+NS2ZZbZ5+C36jRbrn2G7HNYuFp5\nKyiK8grqRXn5hdYKImOfVz5jiBX6zZ9oEMAQw+/barTvTOliR3easjFZhZDAJoFNApsEtirRhI2v\nXOxoCWwyCWxCCCHU3ihrc+fOLWmLbr3rgVG/ocdHAw8eH+0zcnI05NjzohFjL66SNW2zbkQhTUJZ\n3WDrCvxWcjWsnjrJ2t3GeUBClP3MuIWCfMEWXo/wlERcU3hDDqqBa6mIjbB2ag2Uuxf1kCZ63IQE\nNglsEtgksP1ZdJDAJpPAJoQQQu2NsoZrxTWfeuXTOU3Bq4i122RblP2uHguRASzk09/aadZuNi7G\nHhalwBTh371+KRazwMqvlxq3wA7EJkwFrq+vUAgJbBLYJLBJYFODVyaTwCaEEGpvSGATqzLNrG1h\n7UhrM609ZNyiFJjyGE4f/tW4Va+x0usD1i6wdqhxK04LISSwSWCTwCaBTQ1emUwCmxBCqL0hgU2s\nchQYt5p1Xwpol1h7zLiFKiCgLU/oXyI2GVaFRey9y40L1I+ppWvo6xRCApsENglsEtjU4JXJJLAJ\nIYSQwCaBbVWj0BPQEAttmrV7jPNA+y4Q0PzpnBDX3jQubtpZ1oZa6y4RTQgJbBLY9D1LYFODVyaT\nwFY3aWVtsrUnjYv9glW7rrO2eUp+dB5OtHascfFk8nM8Dzout7HT0kZfuxBqb0hgE7VEA+PimPkC\n2hxrr1n7MegXLrX2m7f9lbUXrd1kbbxxMdEkogkhgU0CmwQ2CWxq8Epgk0lgE2XYhJ0HjNBfwk7H\nNcZNbVnKzkhMATskeJe/ZdzUF3x+yVrrLOdpz/JesXZRDQhs8Br4RsKdEGpvSGCrkzSk6DUwENAW\nsH7zvc9Qv31v3Iqccfrnxg0yXWFKFxZAeU311QohgU0CmwQ2CWxq8Epgk0lgE7mAlcoWW9sgSO9I\n4e0NL20U3+NjvLRBxgVvvjHLeQbw2M1r6D4OZvkKEC2E2hsS2FY9GlHw2o91kC+gfeP16TCVE9M2\nv2Ad9rO3b5G1h03Z1TklogkhgU0CmwQ2CWxCAptMApuoFr629kTKvrP53m7riXFvJOS70zgPgTTg\nJXcCy9rf2mbBfohi+1jb3dpaKWWsSZEOHSJMS23o7etqbSLL39Urf012ngqCsloF6Rvw3QnWDeop\n5NmW58Xfwhy+027GefRh6uyOPLabt7+zdx+NEo6vZ60P8+B7WSfY3473DDZivr3VSRRqb0hgW4lp\n7AloEL9mZBDQIJzBgxohDf5jnEdavP+/rKskogkhJLBJYJPAJiSwySSwiVrlTYpsG+WQ9wF2ekLm\nsMOTxhtBnfsF0/OsTTEu1g2m6iyjjQuOh3i2lPu+ZRm45i7c/1VQ/udMj0W9xkF5Y5i+OrexwttV\nTF/O6wEIeP0a88ZeEO+a8t5+IZh69A9rz7PjF8fyQeyeCTxH3CH80trG3rHbW/uU+1DOr/x8nJdn\nOu95Ovf9yHwoc6QeaaH2hgS2FZAmgYAGAeyuBAFtGd/hqJtetraQQtr3wTteIpoQonICW8uWLaO2\nbdtWqzVs2DAqLCyMiouLq9Xq169f8uJr06ZNtV8zrrcmBbZWrVrpe67Bay4qKpLAVosN3m222Sbq\n27dvVlt99dUlsMkksNVtdqc4s8Q4TzQsXNDTlPf6SqOFtf/x2DTgAXck64B+ptQj62/sTI22Vmxc\nwOjzTKmn2x9tIeOmA8Vea90oJl3PbZR3MvP18sqviMD2Gu8DnbU+1upb+zc7eLG32FbW3mYnerUs\nAhtEtdP4PaJj+RjTFnpiZi9+91dyGx5vn1CYi+8B1/4o/z+FnsAWdzJ7MQ3eeg8ZF19oOz3WQu0N\nCWy1zBoJAlrs9ewLaIv5Hn2R79751p6moPatRDQhRE2CRtUrfOFUt33Kl1hN2P/YKK3ua8Zo7S81\nZGi4vqXvucavGf/Dlvpp1yjo9F3ABklWs8LnYglsMglsdR7ERbsh6NygQzTTpE/ZNNz3jLUfTFkv\nrCT2ZLmbchtiEby3LkvICzHqOX7eie+rBkEeiFAPettJMdgqIrBhu7+X5xCmbRgc2yMQANMEtvlB\n2mGmdAqrz7PWHuHnprzX7YM88X00CwS2AUE+CHmYqnu7Hmmh9oYEtmpmrRQB7X32o+K6A4MfC/h+\nvtvardbuYN6XA7FN0zmFEEIIsQq1jq3QIYFNJoFNEHhbdaVYBZEG3lCYztkhyJdHAQodJcTB2SKH\nskOBbTNu32ecd5pvEO3g7ZXvHQ9xaReKTbN5bdUpsH0a5LnauGmp4bWdZZzX3T+yCGznB2mDeM71\ng3R0Lh8N0hCDDd5yhxvn0fe/BIFtCf8PIejMfqRHWai9IYGtAuBdginxvXMQ0H6kgIbQAFhx83Lm\nvZpC2gKJaEIIIYRQg1cmk8BW18C0wr4ZOjoHsIN0XiB03U0B7J/GeU3lQiiwbcnt19j5SjIIY2tQ\n7IOohWlFc42LZbawmgW2x4M8N1FgS7u2k7IIbOelCGydswhsKBdeaPDum2+ch9AVCQLbopRzX8br\nFkLtDQlsMfkU0PC+H55FQPveE9AwqICp7mcYFy/zfJO88IBENCGEEEIICWwyCWx1mp3ZORqUoVOG\ngPyzub0GBbEPjJsqWRFCgQ2iF7zQRmc5DnHWMAW1a5D+RA4C2/EpAtuUBIHt/iDPRHY68yvxvVZW\nYBvIPKcEeUYnCGzLvev3eZBCpBBqb9Qdgc0X0I7g+8sX0JZ6Yth3gYB2BEW3o4yLwTktRURDCIEn\nJaIJIYQQQkhgk0lgE+WB8ISA0wtTxJpt2bGaxO1z2eFapxLnCgU28KpxsdQKg7z3mtIYbIhF+0yw\nvxk7e/O8tIMSBLZhpnwctQJTujJoJoGtL/McGKRvQmEwWwy2yghsf2ee8Pu9xSTHYDs5yNfBuEUT\nrtajLdTeWKUEtoJAQIMIdq0noP0SCGGhgIbjtud7+AiJaEIIIYQQEthkEthE9TOSgtF71o42Ls4Z\nOmMTKBTBWw2L0+Sx8/UiO16hHVkJgW034zyx0KEbzvNeyOuJV5y+xrjpqPCs2JnX+yavDZ5aXZhv\nd5Z/sbVRTFuHgtNT1vagKPYwO8HZBDbc70PsuE7jteKa4sWFimtAYOvPPHP4f8Axs1lexO8ojwIb\nPFLg2TeZ39sgdrSRd3091kLtjZVKYCtMENDm8N0YCmifpwhoOL5DQhmhiPadRDQhhBBCCAlsMgls\nomboTxHK78RBvLnSuDhtoC07emk2P8s5dmK+UPzBqprPUlSLKJr5U1ZbGTftMb4udGAPoYj2O68R\n1DduJVSsCO57vA219gmPhbfeRRTKcC0Nmedmk+z11cg4r70vefxSCl5ts9wrPPPGB2kDeM4OQTq8\nUG70ts/kdcYruV5i3IIQ3/N/kk+B7St2pBdQpFxG8XBTPc5C7Y0VTmBLE9AWUDD7lb/53/meeDJF\nQMN7DlP1e0tEE0IIIYRQg1cmgU0C24oLpiG180S1Wn0dmcxeYdjfIEgrrkD5jUzl4qnFNKni8RUh\nn51onyL+f4wnsBlPXCzS4yvU3vhzBLZp/3o7ar1ed5T9WRbhazkFNHiuXpsioBmJaLXKNsYN5KSB\negahATZmPZSJ7aydzf/JJfz/Nc9QJ3VKsLUqcQ/NU8pqrn+vEEIIIYFNJoFNCJFOKLAJofZGDQps\n0255Kzp++j3RoafNivYbNS3acd+joq69B0Rt1+sSFTdqEnmiV+QJaBBZJgYCmi/QNJGI9qeD/wlE\nz71S9o8M/h9fWxuSkA/hCB7j//4M5kH8ToioCK1wXMIxJwfPTWxPVuI+5qSUNV3/YiGEEEICm0wC\nmxBCAptQe6NWBLapt/y7REAbMfbiaM/DJ6YKaPn5BVHz1u2j9bv1jnr1GxrtNuKUaOhx06ORZ94Y\ntencA3neC247VxFtAcUYiWi1A7yBsejDm/wfJAlsQ7lvqnFhASDGzTUuVIAfS7OjNbRp4LmW5EmM\nRXYQ+uD/gnT8vx/n/9q3DSpxP4gNeklCWe30rxZCCCFqucE7cODAaPz48TJZrRieNwlsQlSJ9dk5\nFELtjQrY6NGjS0SttdbrFq25zkZRk5Zto/oNG0d5+fl/iGj1ViuOGjVdM2qxdueobectonW79Yk2\n2XpgtMVOB0Zb73FUtO0+YxKtUfO2OP4rCh1YIOUj47yj4viNEEHuNC6uIzyj4NXWwdTe1HPhOMyU\n9/TaK0GAw//v1iC9hXEi6nAv7WlTGgsUoug51t4wLhYoYmJONG6BHTwD3bzjFlK8qyoInYAYnHvr\nXyuEEEL8yRQVFd1Wv379z2Wy2jQ8d/r1CSGE2hu1bF+YvLzfYXl5+c7yC5bn5Rcuz4cV1FtWWUM5\nxi1EMpci2lES0VZIIJLFHl77pQhsW5p0zzafHa39yDLhrQgR9XVrw4zzWMSiF79QeLvdlK4uXcT0\nIVmuE55vz5vS2Hygn3EekGOCa11P/1ohhBBCCCGEEEIIUdtg8YIkIe0ApiO22gRrz1l7wdplpuwq\n0piuH3u5Ifbaf0zZRYLgxTafnyca58EINmP5VxjnKQcPtHesTTJlp5n24b4LuI0FEOAleYe1PKZh\nVW14x11s3IrXEO5esnaw/r1CCCGEEEIIIYQQoqZJE9iON256LxYceMvaDGtXWVts7XNrbZgP4tpk\n48QueJUdE5QD77MT+Rki2U38fCDP+6q1g4wTw+5i2rVBGVOs/W5tgHFi3YembHy+GTzuXmuDjZt+\n/DLTjtG/WAghhBBCCCGEEELUJGkC20SmY/GJBl761saJXedz+0bj4qi1Y/5Nvbztrf1q3CIHAGLa\n0fy8hXGeZw2C817L8jfy0uoZ50EHcQ/ebH2CY3YxLsZbnpcGL7hXjBP96unfLIQQQgghhBBCCCFq\nikwebEgfkXAMRLdn+flU4zzHOjP/lkyH2IUVYz/g9q7WPrPWKMv19GY5hwbpscfbR9aKc7y343jM\nZvo3CyGEEEIIIYQQQoiaIlsMtt0Sjpln3GIGAN5pS6yta9yqoIh9doS1642Lh4apmhDXsKro1l4Z\nvUyy8NWd5z3cS2tunDj3mHEecf/09mG103ghjZCRLKub/s1CCCGEEEIIIYQQoqZIE9iwuAFisE0J\n0jGlE4sM3OKlYaGCh41bxXO2cXHX4D22ubWnjVsYoRvzNecxjxon0oWry57I69mK2/CEu8+4xRPW\nsPZ37t/P24+Va69PuDdcCxY8qK9/sxBCCCGqG7jlYyQSI4uDrK2tr0QIIYSocSACNDVlY0RVBYgU\ne7M+hzDSIsfjII4Ur8T3LaqfNIENYIrnz9aG8rlpbdwiBci/s5cP+yCwPWitU0I52xnnweZP+xzC\ncmZZa2at0LhFDL639rj3zBzFfAO5jXwQ8CDyrcW0c42L23YCrwWC2himTde/WIgViy784cMV9mb+\nyFdLyNfH2g3MBwV9z0qeDy+N0dbuNm4+O1xgk9xnV7c2ztr9zHu2Kbskchrzrf1D/1YhVgnw3sHy\n5xid+9G41Zc6pDSoz2ajZa61S61dZ+0L40YFN004BqOQd7DsJWzMDErItyXfK2iAYaQTS7jvUMn7\nwfv1fTakPuO7t12QB4Fqx1v7lPm+tXahtSZ6HIQQQqyg7MQ6q1XK/n2tvWbtN+OCsiPQe/OEfC1Z\nN6I+n836HCLI1xQ+2mW5jlfZHkhiD+Om+EW8DtTtm1fxvndlec1S2iYzeC/Is8i4YPmFelxqlUwC\nW1O2GyPPsNDAyIS8aJ+dbu1/7A+fx+cTwtrr1nZMOAZ92Z+C8u/3+rSbsg16Q3DcRkyHoAchroi/\ni2VeOWiTXpTSbxdC/Elsx04jlgLGssS38If7rCm7GkmskKNjeRn/RnyxVJTrWda9fCm9xe39vDzN\n+LJazPywH/hCWz9L+VhNZab+tUKs9OzK9xEEsmHGCfP/tfa2tYZB4+hFa7eZZLFqNN8f23vpaNhA\nfMMy7BgdP5ANnojvu5jN2TBCfA3EyjjIuOXccV09K3g/p7B8dBCwEtRYdjLes9bYyxc3oDAiiaXY\nJ/IaHtAjIYQQYgUEsaleNekCWxwnCgNfQyk6QHR6zJT1/OrIPgna8aH4BrFqsnFePRuliB/x1Lok\nga0X69aXeT0IcI+BLgh3rSt53wh6/5pJF9hu5zkhsg1nfwZ55QhQu0Cc6hS0HUOwHwLZDkGbLAl4\nj8G7Dd5qiOO2RZb8mPb5F+PiqIX92KY8d4OE49bmPl+QbWNtW2t/NaXebUKIFQh4hkBxX91Li91U\nYw+1tsaN8vzLuCCLhpXhJcz3lwqcb2se448KFLPD/KaXhopoOSvDGHibwJPjYf3bhFjlwbsGwhNG\n7vzYFX34Dunvpd1jnCdaARshEMvg8XUqG0GYMjrBuNgW8cpOk/mO2Sw45wtsbMfvusuYz+8woAwM\nTFxcgftBow4eeHODzsQuvJ+x3MYUmN95Xp9TmW8dPRpCCCFWEE5kX2K5KfWqCQU2eF9DTLs8SD+U\n+TfgNurvhewDGAoORxo30+VvxgloGIyH+LbAaxu0N86baJF3DUkC24PM46/wuBGv/ewK3jfq7GeD\n+w4Fti2ZPj5Iv9o40a2lHh8hhFi1gDspVik5LkjfkBVCnB4vGRx6a3Rj+mh2GEfxmIIgHyrHo5kH\nncSfTPlgj9cY5wYbA++1hxKuGS7iP2S5L4wkxNO3ilkJ4p7gHoz565fymgpYyR7Fyvos46bLhvTm\nPhwHt+6dU75LeNhcwLw9WGmPNWXddvEd7MvzIe8QIzdxIUzK7w7vl12DdLw7MJoXjy7CK20xG/QN\n+N6AV9gsNtTRoF5q3OghllE/mMfdaZyYFjItaChjCsuPpnxsFXi/3czPu/Pd1C/IczDTcb29WO4+\nQZ5CNrQv4XZb5js1oSytEiWEEGJFoj/burBbTbLANpzpGya0nX3PHdRzCI0Ab6M1WYdj+3LW3w9w\nuwnr/dgrvbl3DWPZnwgFMwy2IRzEPxPu4WmeCxzOejucNgqPO3ifNeX2bt75bjfJAls8MBbGgt2B\n6UP0+AghxKoFKrDuCRXCKL74+3gVwaWmfPyfvzLf/tzej9une3lib7h9uY1RplDEQsX6Diu4GLhO\nj0i4ZnivvZflvhDf6Cqv0o1d0iHMzTdOvItYYeMzvOceMW76KTriXb2yZjAv3N7nsYLH9hlenmas\nmOHlh6ljT7Cc25h3Da9yv4v50LHH6NtynlsrvwhRlmOME54gksObFtMlz2Wj1he7IEzFcSsm8Xe8\nCbcbcTueWon3wkXee227hPPid/utt72bKR1IiBnGtL25jVF1jODD860N03rxtx438lvxHRl2PDYw\nZT3YDN8j73hl4R3yFN9XBXo0hBBCrIAcYZIFtvOtfcK6EmIbBpgxYL1tkA8DYmfy87Vsz8f1YDu2\nmWPPcbTnj0+5jm9NeYGtM69tVEJ+9AcW8zMEtA/Z14g93Qbz2KNTzne0SRbY4jhyIa1M8kCaEEKI\nVYjV2cFDhYaRn1Oy5IcXyX2sOFoFlRQ6lXCL7sKyLkkpA55gE42bGooljNfLcs6e7HBni/uWJLCh\nso2FszxW4kg/x+usd2L5U7mNqViYqjXZKxudW6z68llQgf4cdNb78nvwBTZ4tv1kynoCopO/1GtQ\nCCEcWDIdUzmu43sEAnwc9P9m73eL+C3HGidS4310YlAOYjyO9hrsF2Y4Z3824M8P0o/luwDn/69J\nnvKB+DPfGSfmNeF7CEJZJg/V1Zgf74+OXno8cr+Y5eD+PwryCCGEECsSaQIb4o4iTup84wa7MRD9\nJfNO8/KhjoVHeHu2x/cO2t+oCwdwG/VrRQS27jzf0IT853BfPONkK+Nm+GBArgPLuz7DfacJbLey\n7k6q++M+iBBCiFUUdOjgUfUeK5UbTfqKdahA5rIjOizY14gdWlSkzxknnqUFlIS79cvsFMMzbFCG\n69udFdxCk30lvSSBLQwmejrTw1VJMcI2i59jjxM/RkIev5vvuF2fFf5lCdcRu8qvwU427jNJHLyU\nnXYt7y1EKXGMx3dN6aqhefwtI/1ApsGzC56ymzG9vVcGRr5/845HvMlDEs6FAYMxfPc9bMrGZ+nN\nDgGmnV5h3Oj55/zd7xaUM4LXsJDvq04Z7q8TOxmYsrJ/cC03sJyn+M7AgjC/s3PSQI+GEEKIFZA0\ngS0e1H7alC5cAJHpJqZvzTS0hRGXFF7rGHz2PbYhemGQupjHoo7tk3IdSQJbD57rgIT8cbvC769M\nYj/ndfZRmma47zSBDVNHP0rIX4/5z9cjI4QQdQPECMLI0dUJ+9ChhPcWRp72Tjm+G49Hh7BrDueD\nAPUAK7INE/ZdwYrodpO8nHdIksB2TJBnHNND7xIIjLOCzi487SDIwfvlLR4XC2zx9K7DE67jJFMq\nsK3Pzx8bJ2T69qlJX31IiLrKdJMco2Q1/v6u4zbEKIwC92V+XxxDrJXX+Lkv31vhylD4bWJ6J4Su\nCabs6smGv/mPgg4DyniHnYGiIP88XsfElPuCSAgxD95pmHq+ZbB/bx5/WpAex2A7UY+GEEKIFZA0\nge1Opm8VpLdh+iRuoy4+muUsCurNe1mOYZsbA1lpIROSBLbNea6kQbaL2Gfx+wQo+20eMzzLfacJ\nbHPYTghpYsp77wkhhFgFwGp13RM6lACr8n3tbaPSmcUK6Boem8YhzAcbF+zb1JR6k/jECyac4KVh\nKiU8RSDo7VuB+6ougQ0VJTzs4L0C1/Cx7PxeaEoFtvi6D0u4jnip8DV4z/j8ACvUJGuuR1KIPxjP\n30zS0ucQph7n56H83SKIMKZaIm5iZ75L4L0Gj9M12VDuH5SDbYyIY1R9o4TzxO+P/0vYd6Ipu/pZ\n/D7AqDsWQEDst7bBMfB4vYN5Tkl5957Dd2coBObxnXybHg0hhBArIGkC25WmbMgUHywidDU/IybZ\no8aJYagHTzYufAwG3DBwj5AtGIj/wGRe8CdJYGvNa0gKyXI3622fAbwGpGNArVGG86UJbGiPLE84\ntivzj6mjzwnaPu34PxFCiFWKOGhnj4R9c9jxjIEXGYSmnbOUiYoPnhkQoc5nB3drbz9GnOYlHBcH\nHx3vbUPEgtdakwreV3UJbP/HyjXseF9gSgW2RswzI+E67vIaFHlsRExJyIe4SjvqcRSiDDub5FU3\n41hr13gNNTR+IVgN4+8sHgg4jO8jTFnHaqQQ/+MplogRiSmhM016nLTGLOvchH3xNPMO3F6d7w/E\nhIOgh9F3xIjxR9hxTV+nvHON995BuS2D9EK+g2/UoyGEEGIFJE1gG8n0cGXOeNXsCdyGQAVxbBDr\n9KXsRyA28iTW2RDgMACPkAz5KdeRJLChHQ5vsvuCdNTRmEnyoJeGUBMIC3Ed+wBL2C9KI01gO5Tp\n4WIOsUf6DnXs+diQ3+My3n/ENtvZJnv4CyxO8XCO59mT/1OIm8Or+R7QFutkMguuQog6TFtT6uHh\nx//qzI7cPdzemp3M/bOUV8SO7Dt8UcLgNYK4ZnHsggt4zrCSvYAv2t7chjj1hik//SoXqktgm8lr\n9WMytOD9/OClPcSK2BfidvEqkHjE7hbjRsLWCl7UT7HBIIQo+9vA7xFTPNf0GsixALW7l7cbf4Pw\nMoXgFnp/4fj7TVlPNLxj/m2yxz5EnLTvTNnFSTCtFALaAi9tFhuK8UIE+wcdh834Hj0oy/l687jr\nvQZnPjsY/orMQgghxIpEmsAWC2cPecJEIes5tJX9cDIYDMMg2jDW52Ec5/XY1zgsw3UkCWzgfIou\nvrPACbzmuG6G4IaBsk+99nvcZxiWcr40ga0V2wUor9jrl7zDvkRhHXo2OrGd9hm/837GCWE38rub\nm+HYnuyPfZ3DefJ4HgxwYobDhtV8Hx1Neiw/IYQo4Uy+KCDwIOYPAnj/wJfYxsxzFvNg5GBegsUj\nM1NZUfoea72YFo/8rG1KBaoZ7Hw+wfJjYQseKhil+jjlfFdluafqEtj2Yp4F/G7gog6B7BV2lI/z\nOtv/5Qsdx2KFw6WsLHyBraMpjV93Ect81jgPm630KApRDnh6/Y+N7cfYKI34WwzZxNpL/H1i1U8s\nUIIRbiwisohpsZiWzzJ/4e82tM+9crE66If8zb/M3+wvzLMJ8wzhdY0Ormk2G4XbcF/E8yad8yTv\nuFN5Ptz7fL4zcewleiSEEEKsoKQJbGCgcZ5gX3r12jLWzSHbsz2OUBAjWZ9DULue7fBsA/5pAhtm\nxLzI8z7JNkO4Mvk41r87ecfls6/yI9v8IWkCGxjOPgHa/4+wf5VpgYZVlXil1iQP/su5L2naL+Lu\nLuT/LReBLY5v97caug8JbEKInMAIAgKFI7joPezc+ZXEKHZS02wLvtBmprzQ4uPX4TYqXsRRgJs2\nYh9h2tS+XuW2dpbznZHlfrAa0KH83JDHhFMw+zE9dC+fzE55DNzU4TYOkQ2xk/qzw/1E0NmFhwym\nf87h/WzPRsOS4BxrUhx43jjPNaw+uoEeQSFSacV3yFT+9rfIkBe/NUy5OJ2/TzSwMRWjaUKDbb8M\nFi7ggumfGAmdwGtAmb6X3F+Nm8oavk9asDwMNHTNcs6Ng2Ph5XsM7xvi2zZ6FIQQQqzAdGJ9tlrK\n/g7Wjjcu7jD6GutnKAszWAawXY72OjzQEdoml5W00a/ZLGVfIevzaWy39wv2wzt+14Tj1uG9JZW7\nLvelzbrBzKC/85zw3mpTB5+NW40TNpO89iCswYFil4R9CDmEwUrE48smsMEzcaJxAhj6bJjV4Me3\n3s44L8aZbFcmTfPciv+nS5lvT69/is+x08lNLB/95db8vF5C+9VPh0PKSH7enm3U9bz+6jE877k5\ntvn24XPXkM/VpWwztvf6unCcwSwxCILhoiBr8vpm8thjg/ZyZ+5vyXZxnO8IU7kZZkIIkTOD2cEO\nQdDU1/X1CCGEEEIIIeooEDMhTEHwyTV+GZwaMEMIg5W5CGyI0/chz/OVcTOa1uU+CGO/s18G5w44\nQMBL0l+MKg7FgTwQ/N7l9jncj9lHHzNtEcuHqNzdlA9dAuLFLAZy+wxe399MaQw6hAWBVxxmaGB2\n1+O8rsgkx+32waq6cI5BSCWEU8FMD8yw+JTXjNkR840LhxLxO4jBeX/i9zSP5Sxj3vj/syePw37M\ntoIHJmZxYJo1vD9b6bEWQtQU8FrDVK7tuI04C8fzBfR3fT1CCCGEEEKIOgo8DzEjCIINVny/1zjv\nRMwwKkjID88zTKuNRatcBDawhikfsiNe1M+PoQfPMQhRd3Mb3lwQmM7z8uRx/7deWtIU0YoIbAgT\n8gmvqTXP8Zy1V03pqqp57D9GJvMieHcyz0Qv7QCmQbBrwTTMrkD8Q3+lXGwjTrovdg7nsXGMwlhg\nQ0gUP5YdhE+EcrpWj7UQoqaAOy2m1v7OF+dvfEnDrbmevh4hhBBCCCFEHQfizZXWPjKlXlwQ0vw4\nueB241ZxjamKwIZ4e/MT8h7JvhtCE0HQw3TL1kGeK4wTk2KqKrBh+1Avz1Ypx+K7gAB2U4Z7hcAG\nDz9foGzH8sYFeeNzx2D6dY+E/w3yxGFS9jTJ8YUBxDV4y62mR1oIUZNgznsf40Yb1tTXIYQQQggh\nhBDlwPRMxPOKp2KOZ/pBxnmXNfHyVkVgw+IUmAq5ILC3mbcv8+WxD4d4u/+y9hb3V7fA5ot48cIg\nrydcH1awfyXDvUJgeylIW5PlHR6kn2rKCmwAMdYwXRUL9D3F+0wS2DZPOPdh3LepHmMhhBBCCCGE\nEEKImgfTQ7FwwE4p+7FoFAS1d7j9kXHCkb/AHWJ/LeHnYRnOlSSwIYzPq7yGJOvCa8BUTcQlg9fY\nKcYtwIeFCCorsG1uygtsmOXkL4h1MPNcnnJt4zLca1UENlwLQhkh5hum6h5inFdbksCWtLDHCO7b\nUo+3EEIIIYQQQgghRM2DKYwQxzJNd5xvXKwvALFpTmAQyH7h5yMzlJMksD1hXMy3EMQog/caVuE8\njceFK9VD5MomsHVj2h7BsbFA5QtsS4M8PU1ZLzofCHebZ7jXygps6/PztCBPLAiGAluSoHku9zXT\n4y2EEEIIIYQQQghRO9xgXHzqvRP2IQ4ZBLibMxxflSmif+e5ewV5Zxo3dRTxss/hcU2Dst7jsTHr\nMN9wL60V0yZ5aYhN9qjJLrDh3IijNs+UjWeGaaSI6316hnutrMDWg59HBnkuYvq+3I4FtpdN2em6\n+A6+NG7lUSGEEEJUMxj5w6q5842Lo/GCccFP/5rlOCxtPqUC50EDBguHYBWqYfrahRBCiFqru+G5\nM8raSdaGWFtddbeoYBsOXmgRn7npxnlQ3WKcZ9oHxsWyTqMqAhumqD5vnLh1A88LcQoLHIxgnn7c\nRsyzs/jM/ofbKG8y8xWzHKwEClGsLdMf4n1cbG2GtTeMW4E0m8BmuP8XHoPznsPyEQOueYZ7razA\nhu8D3oLf8HzwRnvcu9f7rK1rSgU2xId7n/nO57EQ/7rpsRZCCCGql42tfcwGw01stKCy/pCV8sUZ\njsUqUr9W4FxzjQv4iuP2rYF7WWQyx7oQQggh6mLdPYj19TfshC82Lq7VLqq7RQWAhxa8pu6wttDa\nv40TqU601ijLsXjWzsvhHBCPEKdthyAdwthY48QjxFqDt9y2QZ69rN1v3AIDeG4hMEH0m2/tai/f\nTqY0Nly8qB1iuJ3FvI/ynlowTxzDDDHa/ply3ZieiRVLsdjAI9bONGW96ZI42jjhzKcxz9k7SO/P\n9BhME72G38WT/A01Yh6cfwNTKrAhzhrE+Cd4fRdYW0+PsxBCCFG9YJQQI2wY4dsw2Fdk3PLqYewG\nNCD2MW7EMlyZKRtYxn1mDd4PrmWi/q1CCCFUd/9Rd2NqGKbR3WWtPtPaGLcCI8ooUN0txCpJLLBp\npVAhhBCiFpjIirdPyn5MP4EL+ZNe2q3GjYDDlpjcBDaMDnbnMTfx8xrefoyI7mzcyDhG2fISykAH\nACOV+zFfJ2/f2iwTKztdxs9NeEx3Uzo6GZMfpDfndh47Hxih7OjlX4fnxBSAVjncb1uvMdOJ19zf\n69g04Dkw0rpBShkbGRdsF8f2DjpA9b3vsDGvC/nk6i+EEKq7w7o7XlmwS5DvdKa3U92tuluskkhg\nE0IIIWqRN42LF5GJtdhITWvk5yKwHccK3rd4KfS/GLe0O9J+5t+nrbX0jscKTR9w3w/GxbnA5wnc\nPyWh/IFswOLzCcH1NGD68dwe5jVAPubnQ9iYR6wKLIO+lH9xv3/Lcr9YEh7xcMbzuJ9Y5pvsaHzG\n9GUs87SgY3Qn80PA/JGfEXskjpezMdMQS+NrXlOc7x7etxBCCNXdAKsqYmpdUZAH8doWs05U3a26\nW6x6SGATQgghagk0VBEMdk4VyphochPYMCKN0WDEe7mKnxuwA4BGJuJldDBupBcj7VjZaF7QmUCA\n1vW8huzdbOSi4dqMZWIUfIZXfkUb6YhLg5gemALbgo1sdAgONW6kHue5lN/brlka6biWF3iPaOwf\nxXOgcY6AvIX8Xh5h3niEfAK/0314HOwIr+PgN9Jhp/NYfHfHsKzr9HgLIYTq7gTg/YWVGM9kfTFa\ndbfqbrHKghVOm5rs08CFEEIIUUXWZiNvRhXKmGgqFoMNje/zvW0Ek/3elF06HBzIa+tqXFBbNIzD\nQMyHM08HLy2M41LRRvodQR6MuJ8THItG80tBJyKpkY7y/CkkBby+24K8hzJv3AHBCHtSwFvkGRs0\n0u9IOPdl7Ly01iMuhBCquwN8rzTUZT1Vd6vuFkIIIYQQVaMpG3o3VKGMiaZqAtszxi0VPi2wS3ht\nh3l5MWrcnWn/MG7VsepupPf38vT2GsLh9aGR/m2WRvrihPTvuM9nf55n/SAd97UvG+zPMM+4oJE+\nIuEcu3DfLnrEhRBCdXcCEI0grGGlRUxn7Kq6W3W3EEIIIYSoPAgKjFXFnsuSD0uOY5pHfsK+iaZq\nAhsauwiePC/FBjPfKOOmqMAet3aRccukV3cj3V+NbUemvZzh+gozNNJ/Smmk/yNLI31znhPTW140\nLrD00SmN9D0SztGD+4brERdCCNXdGfJ0ZH0xXXW36m4hhBBCCFE1ZhsXqLdzyn5M8UCcladS9k80\nVRPY0ABdmOWY7Y2LmzI1SB+eQyN99ZRGeseURrq/mEPbKjR2K9tIh1fBp2ycN/PyrJbSSD8u4Rz7\nct/2eryFEEJ1t3HB999MyAexDh5b16vuVt0thBBCCCGqxtZspM9nQzBkapaG6kRTNYFtFBvgYQyY\nvXheLF0/mp+7BHmuTGmkTwo6D1jd7Lzg2ONyaKTjWKx+hkDG+UH6EyZ7HJfKNNJb8PPkIM9OKY30\n14P/G64NAaQx5aeJHm8hhFDdzTplGesOnx2CukV1t+puIYQQQghRBU5mQxmj2wg+3Ne46Qu3m9I4\nJmlTTCaaqglsmO7xDhuvaID2My42yRfGrU4G4ngqd3H/ftau4TFIH2lKp3vgOExdQUDhOPAwGq0/\nskOAOC1TrL1r3DSOTI10MITfzUPWDmBj+Qp2VPrVQCMdYBT8P9YG8XpPMy7WDVYwwyppbb1GOqbd\nPGZtd+PitlxnSlcnE0IIobobtDHOo+0T41bF3NnasayTIUY1Vd2tulsIIYQQQlQPaHw+yQZpvLrY\nJ2wg1stw3Bhrb1fgPC+wTB+smHWVcat+RWxQI0ZLYy/POG//98y/ETsMP3p50cBHDJT3jYvDAjpY\ne9g4IRDHP2ttU+NGkA9hnj15zNoJ14wR+QXe9/K6SY6f4nMK84W8Ysp7CvTnuTtyextrb/Fcv1l7\n2tpfrN1qnMfCgV4jHUGjr/e+m4/ZGRFCCKG62weeZBCcfmI+1J+zU+o91d2qu4UQQgghRBVB3JNO\nJvtodk2xRoZ9eQnXhQ5EYY5lI29xFa6tfhWPryiNrBUFafH540b63tzO5/9OCCGE6m6TpS5trLpb\ndbcQQgghhBCifCNdCCGEEKq7hRBCCCGEEGqkCyGEEKq7hRBCCCGEELVDQ+MCWrfSVyGEEEKo7hZC\nCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQggh\nhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQtQ4/w+bJk8VhwUHiwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 2,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image\n",
    "Image(filename='../img/lenet.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bfA9pPftdbhY"
   },
   "source": [
    "Data flow in LeNet 5. The input is a handwritten digit, the output a probabilitiy over 10 possible outcomes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-HVC9sT6dbhZ"
   },
   "source": [
    "The basic units in the convolutional block are a convolutional layer\n",
    "and a subsequent average pooling layer\n",
    "(note that max-pooling works better,\n",
    "but it had not been invented in the 90s yet).\n",
    "The convolutional layer is used to recognize\n",
    "the spatial patterns in the image,\n",
    "such as lines and the parts of objects,\n",
    "and the subsequent average pooling layer\n",
    "is used to reduce the dimensionality.\n",
    "The convolutional layer block is composed of\n",
    "repeated stacks of these two basic units.\n",
    "Each convolutional layer uses a $5\\times 5$ kernel\n",
    "and processes each output with a sigmoid activation function\n",
    "(again, note that ReLUs are now known to work more reliably,\n",
    "but had not been invented yet).\n",
    "The first convolutional layer has 6 output channels,\n",
    "and second convolutional layer increases channel depth further to 16.\n",
    "\n",
    "However, coinciding with this increase in the number of channels,\n",
    "the height and width are shrunk considerably.\n",
    "Therefore, increasing the number of output channels\n",
    "makes the parameter sizes of the two convolutional layers similar.\n",
    "The two average pooling layers are of size $2\\times 2$ and take stride 2\n",
    "(note that this means they are non-overlapping).\n",
    "In other words, the pooling layer downsamples the representation\n",
    "to be precisely *one quarter* the pre-pooling size.\n",
    "\n",
    "The convolutional block emits an output with size given by\n",
    "(batch size, channel, height, width).\n",
    "Before we can pass the convolutional block's output\n",
    "to the fully-connected block, we must flatten\n",
    "each example in the mini-batch.\n",
    "In other words, we take this 4D input and tansform it into the 2D\n",
    "input expected by fully-connected layers:\n",
    "as a reminder, the first dimension indexes the examples in the mini-batch\n",
    "and the second gives the flat vector representation of each example.\n",
    "LeNet's fully-connected layer block has three fully-connected layers,\n",
    "with 120, 84, and 10 outputs, respectively.\n",
    "Because we are still performing classification,\n",
    "the 10 dimensional output layer corresponds\n",
    "to the number of possible output classes.\n",
    "\n",
    "While getting to the point\n",
    "where you truly understand\n",
    "what's going on inside LeNet\n",
    "may have taken a bit of work,\n",
    "you can see below that implementing it\n",
    "in a modern deep learning library\n",
    "is remarkably simple.\n",
    "Again, we'll rely on the Sequential class.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DHAr1_2J1jMC"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(0, '..')\n",
    "import d2l\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hp3vLYIbdbhd"
   },
   "outputs": [],
   "source": [
    "class Flatten(torch.nn.Module):\n",
    "    def forward(self, x):\n",
    "        return x.view(x.shape[0], -1)\n",
    "\n",
    "class Reshape(torch.nn.Module):\n",
    "    def forward(self, x):\n",
    "        return x.view(-1,1,28,28)\n",
    "    \n",
    "net = torch.nn.Sequential(\n",
    "    Reshape(),\n",
    "    nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, padding=2),\n",
    "    nn.Sigmoid(),\n",
    "    nn.AvgPool2d(kernel_size=2, stride=2),\n",
    "    nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5),\n",
    "    nn.Sigmoid(),\n",
    "    nn.AvgPool2d(kernel_size=2, stride=2),\n",
    "    Flatten(),\n",
    "    nn.Linear(in_features=16*5*5, out_features=120),\n",
    "    nn.Sigmoid(),\n",
    "    nn.Linear(120, 84),\n",
    "    nn.Sigmoid(),\n",
    "    nn.Linear(84, 10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OnaREs_Zdbhg"
   },
   "source": [
    "As compared to the original network, we took the liberty of replacing the Gaussian activation in the last layer by a regular linear layer, which tends to be significantly more convenient to train. Other than that, this network matches the historical definition of LeNet5. Next, we feed a single-channel example of size $28 \\times 28$ into the network and perform a forward computation layer by layer printing the output shape at each layer\n",
    "to make sure we understand what's happening here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 238
    },
    "colab_type": "code",
    "id": "yiamwgtvdbhh",
    "outputId": "88be586b-b793-48b7-f4e2-7e598cecd22e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reshape output shape: \t torch.Size([1, 1, 28, 28])\n",
      "Conv2d output shape: \t torch.Size([1, 6, 28, 28])\n",
      "Sigmoid output shape: \t torch.Size([1, 6, 28, 28])\n",
      "AvgPool2d output shape: \t torch.Size([1, 6, 14, 14])\n",
      "Conv2d output shape: \t torch.Size([1, 16, 10, 10])\n",
      "Sigmoid output shape: \t torch.Size([1, 16, 10, 10])\n",
      "AvgPool2d output shape: \t torch.Size([1, 16, 5, 5])\n",
      "Flatten output shape: \t torch.Size([1, 400])\n",
      "Linear output shape: \t torch.Size([1, 120])\n",
      "Sigmoid output shape: \t torch.Size([1, 120])\n",
      "Linear output shape: \t torch.Size([1, 84])\n",
      "Sigmoid output shape: \t torch.Size([1, 84])\n",
      "Linear output shape: \t torch.Size([1, 10])\n"
     ]
    }
   ],
   "source": [
    "X = torch.randn(size=(1,1,28,28), dtype = torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fzicoTnEdbhl"
   },
   "source": [
    "Note that the height and width of the representation\n",
    "at each layer throughout the convolutional block is reduced\n",
    "(compared to the previous layer).\n",
    "The convolutional layer uses a kernel\n",
    "with a height and width of 5,\n",
    "which with only $2$ pixels of padding in the first convolutional layer\n",
    "and none in the second convolutional layer\n",
    "leads to reductions in both height and width by 2 and 4 pixels, respectively.\n",
    "Moreover each pooling layer halves the height and width.\n",
    "However, as we go up the stack of layers,\n",
    "the number of channels increases layer-over-layer\n",
    "from 1 in the input to 6 after the first convolutional layer\n",
    "and 16 after the second layer.\n",
    "Then, the fully-connected layer reduces dimensionality layer by layer,\n",
    "until emitting an output that matches the number of image classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 326
    },
    "colab_type": "code",
    "id": "eoQfoVUEdbhm",
    "outputId": "4f9c42b9-32c5-49f8-d9cd-1f1820572794"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIcAAAE1CAYAAADNkvPAAAAABmJLR0QA/wD/AP+gvaeTAAAmjUlE\nQVR42u2dB1hV19KGvXq95pKisSIiogIWiggIKIICFuyIqChSRMDeNfbee5fYC1bEXqOxJbGbRI0F\nozFqNIkx1tjb/OebuFfOOZxDNPjfRM58zzMPu65d1rtmZm10yJZNJBKJ/qryZc+e457uJ4lZtr3k\nIJ8+HPbY0T9pA01cc1TMQg39/xIS+3RwzN31PW04S2IWauh/gUNM4BATOMQEDjGB4y2xHhOW0dJD\nv77WOaknHlL7obMEjjdta04+psHztlGf6Wto+qaTf+u9zNt9iSoE1jXYNmPLaVpx9LZaX37kFvWe\nmkojFu+i9Wkv1Paq9VvQ5LVfCRxvyqZuOE7WdiWpWqOW1KLLcO4Yr6p1KPX4g7/lfsITe1OvKSkK\nWniD/+R6h8au2Pf7S959kYo5ulCzjoMpKDSagdDOxTH66wJHJmzt6adUooy77qXuN9heL7ozRXUd\nYXAcRvTSwzf+cOM6eNB5ptYxmufv/cHgeNjqbx7RnJ0XOASYuyd0fMqxe7zcpG0/at5pCAOrwVG7\neTvqOWmFOt7drzqNX3VIXdemmCOtO/1M4MisDZ2/nSrVbJRu+5KD12nM8i94efiinVTS2YNqNEmg\n0uUrUcuPxv0+wlv3oS6jF6pzGrfpS51HLaCFn1/l44IaxpBnQC2qHt6K9/dPWk+OrhWoZtNEcnDx\noo8mr0x33WVHbjKsxtt9ghsoOODl9L1a28FJDJC27uEfwmFI4MiktRk0g5p1GKTnlvezS4eNTN7D\n2yLaD2SvoY38/Na2tP7Mc7NwdBuXTDUax6vtkZ2HcY5QsXpDGjRni4IvuvuodPeTtC2NPUFGcHyY\n39pgX98ZaykkorVaD2wQRaOW7BU4Mmtdxy6msPieBi86psdoCo3rTm6+QWo0x/edxCPeL6QxWb33\nAW8zBwcSxyr1mlNZz8rsZRBesH98ykFy9a7KHd1z4nJadex+uvuZv+cyn5cRHHkL2hjsQ1uhLbup\ndb+a4TQh9bDAkVmbufUMhwz9jF/brsGBzkIHrPjyjnLrgAP5QKeR89U56CDAgU5fe+oJewvAZ1ui\nNLenzTZmbT/HYQAhJl0OpDsPOUNGcDi5eStPBoPnazt4plpHyEo+8IvA8SYsOCyW6kS2V52PZA7h\nBnBgOXe+gtxp2Idwk+PfORmODsPnqHwC3yQK2RZnOBL6TTEIVf61m1K/mevYa2BmhG1IOPPkL8SJ\nrvH9IE8xzhn04cC9IWHGcvL+a1S4mIOCwVzOInD8RUMegZFcxN6JXyxGdLVGcZT0yVnejzCDba4+\ngRTbcwxPeQESPIR3UD0q4+HHgCG3wK+l0R6gQe4AIGpFtGG40OHo+PKVa5BzhQCDkGQQJnQzEYQo\n/W3wShpYABZwFHNyZa83eO5WdVxi/6nUccRcgSOrGpLdcpWqpZsGvwrkmrcTOMTkdytiAoeYwCEv\nSeAQOMReB47obiP5N4xilmnof1Nw5M75n1xpud6x+lHMsk3HwRnwIP+VK71GyCsQmVJxnT0zdqki\nETTkZbwdIq9CpK9/6ey7l3Ccf7kuErECjf5DcVV5JSJN83R26iUY+DlXXolI01idtdfZvZc/x8gr\nEaX7ECizFZHAIRI4RAKHSOAQCRwigUMkcLz1ypcjh9QhFctGLzlIX4d0w4YNdPToUTELNfR/NnP/\nTPD7778nkeUK/S9wiAQOkcAhEjhEAsdboDlz5tDDhw//0rkvXrygqVOn8k+BIxO6ceMG7dixg23v\n3r2Ulpb2t9/Tl19+STExMWr90aNHtGXLFlqzZo3Z93f37l06cuSIWh8wYAClpqYKHJnR9u3bqXTp\n0tSrVy/q2LEj1axZk1xdXenkyZN/2z1FR0fTF198wct37twhHx8f6tu3L02ZMoW8vb1p/fr16c5p\n0qQJ+fn5qfWrV69S5cqVBY7MwtG4cWODbegYR0dHevr0qdr2yy+/cEdpevbsGf3222+8fPny5XQh\n4ObNm/TDDz/Q8+fP041wtGVOuKaDg4MKCUuXLqV+/fqp/RcvXqQqVaoYnDN+/Hjq3bu3ARwQoAIk\nAscbhAOqV68eHTx4kB4/fkyNGjWi0NBQPq558+b04MEDOn78OLm7u/M+jHR7e3s6duwYn9umTRuq\nVasWxcbGkpeXF125coW3DxkyhPz9/fn46tWr06VLl9Jd99SpU3yupt27d/O1ASMErxEZGan2f/75\n5xyCAKIxHO3ataONGzcKHG8aju7du9Py5ct5VCJ+axo9ejRNnjyZ4ShUqJDyHosWLaLExET2INiO\nPAHaunWrymfQ6ZpH2LVrl0En68Ogn29ALVq0IBsbG3JxcaEiRYooqK5fv07VqlXjezAFx+DBg2n+\n/PkCx5uGIy4ujrZt28YdWr9+fe54GJZxPOCoUaOGOh7JILwNNHHiRPYYyGH279/PQAwaNIh8fX1V\nO61ataJixYqluy68FfIHTYsXL6YGDRooCFesWEGBgYHsSerWrUsnTpzg7abgQB4FwAWONwgHRn3J\nkiU5HMClo0O+++47trNnz9K5c+cYjpCQEIMZBjoLOcO9e/e48+Ad6tSpQ+PGjaORI0dSnz59VDvn\nz59XYUhf165dowoVKhgkmjt37jQ4BjkJoABc8BwwhKs8efLwMq6vnQvYBI43AAc69Ntvv+WR2rlz\nZ94GtwwvgtGP5BJxPCkpySwcSADLlSunkllMJxE+cDwSROQr+u2akoeHB0+xoWHDhlG3bt1UOAJQ\nmF0Zy9hz4F6dnJzoyZMnAsdfFUaWp6enMoy8BQsWGLzkgQMHsivXppTYBog6dOigjjtz5gx16dKF\nlxcuXEhVq1blsAM7ffo0b1+2bBnPNAICAigiIoJu375t8p4mTZpE06dP52UkxAhPCEm4h+DgYBVK\njD2OPmybN29mTyVT2SwmeBcApCW1f0Xwardu3RI4RG+PBA6RwCESOEQCh+hvhwMfg2bNmiVmoYb+\nNwVH7ly5cqVZWVn9KGbZpuNA6pCakdQhFZmU1CEVmZXUIRWZlNQhFZmV1CEVmZXUIRWZldQhFWUo\n+2xSvEUkcIgEDpHAIRI4RAKHSOAQCRxZQVKHVEzqkGZk+B9wUodU/g2pSP6BsUjgEAkcIoFDJHC8\nXUK5hdmzZ7/RNlH2IaMCdQKHCaE6j1Z/FKWZ/glV9yZMmGBQGwQFW4wr86Ai0OrVq3laqC8ci0pC\nmCrrl17ANv06IgLHKwhFWFBYBXWz4uPjuTgKSjTpl5P8XwoFYdzc3FTJSsCKyoUlSpRQx6AaEGqL\njhgxgqsPoc6YJhSZi4qK4iJxKCx34cIFtQ/1yVD6UuB4DThmzJhhsA3/VU+/kh9GI8opwd1rQkEV\ndCDKOqH+qH6tUq1TTblxVN5BDVJzOnTokEFtMkCwbt06BQeuY2trq+BFiSo7Ozsu9IKSUyhdqQk1\nzFCUThNqlCK8CByZgAP1s9ABgALFYFEtGLCgFNSYMWP4GJRkQtWc8PBwHr2o33X//n0emSjLhPpf\n2I796EB0HmqFYR01S9GePmyapk2bxoXl9AWYNDgQKlDPXD8/QflJwApPopWJggBv8eLF1fqqVau4\ndKbAkQk4IHT2zz//TLVr1+aaoJpQ9hpF3wCHVlJSc+cYqahNjsqDmnAcvA4K06J2qSbUNEXRe2Nh\nO2qZmoPDGGLU/+rRowevI6dISUkxOMba2loVmUMNVIQcgSOTcKDEJDzBu+++q2qGwhC3UUUQnT5q\n1Ch1/NixY7nuKDxEQkICVaxYkSsAopwkhNpeYWFhqh1Ah8rGxkJI+/jjj/8UDhSmRdE55Bta5yNv\nSk5ONjgOXkUT6qnqhxmB4y/AgdLSzs7OvFywYEGuGKjVDUWhfHgUwKGFGAihALMMwAFXj2KyK1eu\n5KTwwIEDnAvgl01aO/irDBo4+oL3+eijjzKEA+cCUrSnL0x/9assIxdCZURNgFofaIHjNeBA3P7s\ns8/4ryVgmgihpDSqB2udhDKTgMccHChg3759e7Udy3PnzuVcQptKIgeB19DaNU5Y9YvTGsOBcwEc\nymUjv9EMsxzULAXU8CoQaq/jLyxoQrFaJLwCxysKCZx+/VG4/n379qn9v/76KwOCZBSjFUmd9lFJ\nPzeAO1+yZAl3Hgrbok4oDOeikjBmGXD7QUFB3PlDhw41+0dykMsAQE0Ibyinre8NjE0rug9okC8B\ncNyHVkgf4CBRlu8cb7kQut504ohCtajSLHCI/vESOEQCh0jgEAkcor8dDqlDKnVIs0kdUjGpQ/r6\nkjqkIpOSOqQis5I6pCKTkjqkIrOSOqQis5I6pCKzkjqkogxln02Kt4gEDpHAIRI4RAKHSOAQCRwi\ngSMrKN+/sksdUrFs9JKD9HVIa47YQGGzjlqsVR+SatHPj/7PZu6fCTZb/j0l7iYxCzX0v8AhJnCI\nCRxiAoeYwCFmuXA0mL6PotddfyteVvyOx1Rn/A5lmO7FbryVyeffTy1Sr/JyZtvKUnA0X3mJcv73\nPSoX0fOtgCNqzc+U6/285N6sF1upWi3pA5uSVG1Qyl9us3TdBKo9dhvFf/qU/v3OuxQ+74TAAfOI\nHkB+nabRB0UcqNX2R7wtbtsDavXJwz9Gq+6ltdx8948OWnvNYB3747be0513XwfbZUrY+fxlO/ep\n2bILBm3BEnY+o+YrLqrraNeFtdzyGwOQERx5ipZKB3huWyfD9nXb9NvVbz9y1RWTcGjPlrjrhXom\nLOOZjNuK3/GEn42P0z0n1rMUHHiJeexK8wsr26CtGn3Vh66mkoFN1XGVu8wgl0aduTPtKtalElXC\nqXhAI3KqGcNuPnTGAbL1qkEFSnlR3hJu/PI9YwaRTflAKlM3kfIWd2HXjbYazT1GeYqVIYdqkWTj\nXpWvU6njFN7n03oMFXKuxO0W8QimiGXfvRIcuK93C9jycsOkw5SvZDlyrBFF+RzcqcpH89Vx5SP7\n6tqvSA7BzalgGR8G1BgO3FvspttUd+IuKljWl4r7h/G9flC4BEUsPc/H1B73CcOIa+A+i3qHUMjo\nzVkLjpCRG8kpJJaXQ5MOkW2Fmiquv1fQTnkHa9fK1GjO1+TVcgh5J4428DoBPeYwHDmt3lediRHn\nEtZReZB6k/fwS8YyOgfHa9cBUIAD1wdMGKl8P7pjtHOM4bDKZ8Mga1aiahMOixrsEUvO8bHwSvkd\nPXi91pitZOdbR90T8pWiPrUyhAPeFPeI7RXbTST3yD7sIbAdnolzFN2xVvkKZz04MPq94oap5O6/\neQqyC8U+54YdKLDPYr5+fsfyvK2wWwB3GLwBzM6nNsOFjixWqZ5B200WnmZ4MLpwnLWLH79obYQr\nwKL6MxyArkDpCqptdJhVXmuTcLyTpwAfD/NtM447HvswsuF5jMMmvIdb0x78PPr7MAAAizk4HIKb\nqWPhLfCsTRankbWbv0E7JYMishYceMnvW9urxA6GTvRqOfR39/zxER5Z3gmjqGL7SbzN1qs6/yII\nHgKGF4WfgEM/DMFdI7zA28DlAxTAgX3v5i9iEJ/L1GvNcOAabo27qbZhOP9VwopmkSk/KJA1AxRB\n/ZcyhP7dZ+mF1OcKVLNw6MLJH3Bs53CHnARhUv8aCC1ZCg7v+JEcJvS3tVj9EwMD94x1dPCHxcqq\nBBGJKzyKlq9gJGE0GsMRMmoTjyYtcStbv42CAxC6hndhsJDbvF+4OMPRZNEZyu/kyfkPjgsesNxg\n5L4KHAhJuA46UoMFnY0ODZv9FYew2M13VB6lhdTXgUPzuL5tx1PT5G/Jv9vH7OGyFBx48VqCZZC5\n14nnpBHLeHAkqvqzEnSuTfkgDgHsZXQdgqmfe/PeegnifX6ByFUwqmoMX6ebcsapDgzoOZccq7fQ\nQTGVPGMH83U4Edb9er5wuSpsSHpjNtxId3/YZioXUYDrvlcgxKENXL/+tC/UvmqDV3GSifDoVCNa\n5VR4Di1hxnUBKDwnvI12bujMg5xkcy6jm7kADjxDYN9khgxhWb6QZtLKt+jHXkNbt/dvyAnr2/QM\nAF3zQIAdsyItVxM4MmEAATMIl7BODAZG8Nv2DMhhCpb25hwJXlTzKALHGzC4bSSc/MHpLX0GfGbH\nM8RsuCm/eBOT38qKCRxiWQIOfI/AxxsxyzT0vyk4cmfPmSstRy6rH8Us23QcSB1SM5I6pCKTkjqk\nIrOSOqQik5I6pCKzkjqkIrOSOqQis5I6pKIMZZ9NireIBA6RwCESOEQCh0jgEAkcIoEjKyhfdqlD\nKqaz7ObqkPZP2kAT1xy1WOs9LdWinx/9n83cPxOcu+t72nCWxCzU0P8Ch5jAISZwiAkcYgJHFrEV\nX96hrmMXv/Z5wxZ+StM3nRQ4YMn7r9GwBTsMbOHnVzM8Z+yKfQbHYz2j4/Gye01JoRHJu2n1N4/+\nJ88VFv8RDZqzRa2vPfWEJq4+YnBM0rY0vq9pG0+obfN2XyIP/xCBA9Zp5HwqXb4S1WyaqGzsiv0Z\nnmNdtITB8U3bDTB53JqTjymwQRSV8fCjZh0GUb3oTlTUoSyNX3Xo//WZVh3TFZQpXY7Wn3nO63N2\nXiCvKrV1z1lRHdN51AJyrhBACf0mU/nKNSi6+yi1r1LNRjR53dcCR1TXEdRt/JJ02zHCU47dM+jo\nlK9/o3Wnn5F9KbdXajuu13gKDotVnQSbufVMuvOTD/xCSw/9ahgWjt7mn4v2/cQhQrsHdLx2zPq0\nF7T8yK1018XHpDotOvBy6omH5FTOh3pMWKbgwLOVdPag1OMPfj9G9zOgbjN1fpfRCymi/UCBo150\nZxo4axN7kDaDZrBb1cINRvnc3RfZJZdy96Xhi3bS4n0/U7lK1Xj0x/WeQANnb+ZOMtW2TTFHkyEK\nYQbAADaMaJ/gBuRVtQ751QznjgMEH+a3Zq8T1DCGCtkWpwEfb6QZm09RiTLuBvmBqRDQtF3/dPnG\nrO3nFByD522julEdacaW09R+2GwavezzdPeH+7J4OPxrN6WK1RtSz0kr+IUWKV6KRzf2IUdw9a5K\nTdr2UyNp6objlK9QEXbDAAOdVyeyfbp24WHQwRldOzyxN8X0GK3XqQN4HXBkz5GDwcR2gIgQgGUH\nFy+VMFYPb8X3bdwuQh0AMAdHh+FzGCocB4/iF9KYGsR2NfBkuI7Fw4GXhhGsrQMQ5Abaeq1mbamY\nkyutPf2U13EszlGuXecBMLJXfnU3XduAw5xXgTm6VlAAwDCS3f2qMxwFbOzUdlwTXgjLrQdMY88A\nD4PrImyY8oZ9Z6w1Cwe8hZObt7o3gFywiL0KZfCezl7+AgdGDsKGtj50/nbOE7CMPAD5ARJKbUaC\nDpyQetigDYwyU+EDo33cygMG29AhxRxdaNmRm9wBmpeCTVrzJVUIrMtwoLP0E8zCxRx+v6fDN6ho\nyTLUZ/oaBtfUMyX0m8IhzxwcmGH514kwC+rI5D0UFBotcFStF8n5htYJiP2Y3sEjwPX2m7mOFnx2\nhb3HkoPXedrn4OypvM3guVv5xZpqG6Bh1oApo5YIhrbsRvVjuvB6Yv+pHBoADEYvkkLMIjKCA4Yc\nxc7R2ewUGrmJZ0Ats3DACwJ6zQOOWrL399nNS0+CmRWSUouHAyMRMRduFtZ28Ezejm8EWifCek5c\nTpGdh/Fyt3HJPAOAx8C5WhJryjDCcSySW/xETgEQtJCEEFHWszJ7pxZdhnMHwZPBg2htIHT4VgtV\n6wDyzxJGtIeZjrY+f89l5RE5j0k5yEk2Zi1oa/aO8+qeMLXXZjLyhTQLGmZW+nC/qiHnQliSj2Bi\n8rsVMYFDTOAQEzjEBA59OKK7jaT2Q2eJWaih/03BkTvnf3Kl5XrH6kcxyzYdB1KH1IykDqnIpKQO\nqcispA6pyKSkDqnIrKQOqcispA6pyKykDqkoQ9lnk+ItIoFDJHCIBA6RwCESOEQCh0jgyArK9y+p\nQyqms3+Zq0Nac8QGCpt11GKt+pBUi35+9H82+Vv2Yq/9t+wFDoFD4BATOMQEDjGBQyzLw9Fg+j6K\nXnf9rXhZ8TseU53xO5Rhuhe78VYmn38/tUi9ysuZbStLwdF85SXK+d/3qFxEz7cCjqg1P1Ou9/OS\ne7NebKVqtaQPbEpStUEpf7nN0nUTqPbYbRT/6VP69zvvUvi8EwIHzCN6APl1mkYfFHGgVtsf8ba4\nbQ+o1ScP/xitupfWcvPdPzpo7TWDdeyP23pPd959HWyXKWHn85ft3Kdmyy4YtAVL2PmMmq+4qK6j\nXRfWcstvDEBGcOQpWiod4LltnQzb123Tb1e//chVV0zCoT1b4q4X6pmwjGcybit+xxN+Nj5O95xY\nz1Jw4CXmsSvNL6xsg7Zq9FUfuppKBjZVx1XuMoNcGnXmzrSrWJdKVAmn4gGNyKlmDLv50BkHyNar\nBhUo5UV5S7jxy/eMGUQ25QOpTN1EylvchV032mo09xjlKVaGHKpFko17Vb5OpY5TeJ9P6zFUyLkS\nt1vEI5giln33SnDgvt4tYMvLDZMOU76S5cixRhTlc3CnKh/NV8eVj+yra78iOQQ3p4JlfBhQYzhw\nb7GbblPdibuoYFlfKu4fxvf6QeESFLH0PB9Te9wnDCOugfss6h1CIaM3Zy04QkZuJKeQWF4OTTpE\nthVqqrj+XkE75R2sXStTozlfk1fLIeSdONrA6wT0mMNw5LR6X3UmRpxLWEflQepN3sMvGcvoHByv\nXQdAAQ5cHzBhpPL96I7RzjGGwyqfDYOsWYmqTTgsarBHLDnHx8Ir5Xf04PVaY7aSnW8ddU/IV4r6\n1MoQDnhT3CO2V2w3kdwj+7CHwHZ4Js5RdMda5Suc9eDA6PeKG6aSu//mKcguFPucG3agwD6L+fr5\nHcvztsJuAdxh8AYwO5/aDBc6slilegZtN1l4muHB6MJx1i5+/KK1Ea4Ai+rPcAC6AqUrqLbRYVZ5\nrU3C8U6eAnw8zLfNOO547MPIhucxDpvwHm5Ne/Dz6O/DAAAs5uBwCG6mjoW3wLM2WZxG1m7+Bu2U\nDIrIWnDgJb9vba8SOxg60avl0N/d88dHeGR5J4yiiu0n8TZbr+r8iyB4CBheFH4CDv0wBHeN8AJv\nA5cPUAAH9r2bv4hBfC5TrzXDgWu4Ne6m2obh/FcJK5pFpvygQNYMUAT1X8oQ+nefpRdSnytQzcKh\nCyd/wLGdwx1yEoRJ/WsgtGQpOLzjR3KY0N/WYvVPDAzcM9bRwR8WK6sSRCSu8ChavoKRhNFoDEfI\nqE08mrTErWz9NgoOQOga3oXBQm7zfuHiDEeTRWcov5Mn5z84LnjAcoOR+ypwICThOuhIDRZ0Njo0\nbPZXHMJiN99ReZQWUl8HDs3j+rYdT02TvyX/bh+zh8tScODFawmWQeZeJ56TRizjwZGo6s9K0Lk2\n5YM4BLCX0XUIpn7uzXvrJYj3+QUiV8GoqjF8nW7KGac6MKDnXHKs3kIHxVTyjB3M1+FEWPfr+cLl\nqrAh6Y3ZcCPd/WGbqVxEAa77XoEQhzZw/frTvlD7qg1exUkmwqNTjWiVU+E5tIQZ1wWg8JzwNtq5\noTMPcpLNuYxu5gI48AyBfZMZMoRl+UKaSSvfoh97DW3d3r8hJ6xv0zMAdM0DAXbMirRcTeDIhAEE\nzCBcwjoxGBjBb9szIIcpWNqbcyR4Uc2jCBxvwOC2kXDyB6e39BnwmR3PELPhpvziTUx+KysmcIhl\nCTjwPQIfb8Qs09D/puDInT1nrrQcuax+FLNs03EgdUjNSOqQikxK6pCKzErqkIpMSuqQisxK6pCK\nzErqkIrMSuqQijKUfTYp3iISOEQCh0jgEAkcIoFDJHCIBI6soHzZpQ6pmM6ym6tD2j9pA01cc9Ri\nrfe0VIt+fvR/Nvlb9mKv/bfsBQ6BQ+AQEzjEBA4xgePttTaDZtD6M88zPKbD8DmUevyBwGHK1p5+\nSsMW7DCw6ZtOZnjOzK1n0p2z+ptHZo9fvO9n6j01lQbN2UJLDl7/nzzX4LlbKSz+I4Nto5Z+Rn1n\nrKUFn11R27qPX0pRXUcIHOZurGARe6rZNFFZp5HzMzynSr3m5BfS2OCcZUdumjw2od8UKupQlhq2\n6kFN2w2g4qXLUdvBSf/vz+VRuSbN2n5OrVdr1JICG0RRqz4Tyb6UG4PCg+PUEyrm5EprTj4WOIxt\n3MoDVD28Vbrt604/oxVHb6t1uOflR27xsptvEHuDP2sbH3jw4rXzYCnH7lExRxeD0Zvy9W80f+8P\nBueu/OouXxP3kLz/mroH/XuCrfjyDt+r/jaACgD07wMwq/XVR6hFl+FqPahhDI1I3i1wGFv/pPXU\npG0/GvDxRorrPYFfnNYR5SvXoH4z1/F6eGJvatymLy+XKONOc3ZeoNYDplGX0QvNxuyQiNbUbVxy\nuu3zdl9Sndy0XX8q5e7L3sjJzVu9izIeftxpwWGx7G1wj4Agf+GiCkxAlbegDaWeeGjQ/vBFO9lT\naOsILwgnwxZ+yl4x6ZOzBscn9p9Kcb3GCxzG1n7oLO5s/MRn3LKelanLmEW8DyPWztGZXyhA0ZK7\nDwsUptC47gxUdPdR5ODsadItl6sYTFM3HDd77SHzPiHPgFqqXXQqwoEGR9exi3kZnZ8nfyG+Rlh8\nT2o9cDpvB3gA0LjdnhOXU6OEXmo9oG4z8qsZzt4Cz+bg4kV9pq/54/hJK9LlJwLHy2Rx0Rc/qnW4\ne4QCbb3XlBTK8e+cPNq1bZPXfW3QBkY4wDJu2ye4AY1dsd/stdEh8Dz62wrY2LGHAByzd5xX2x1d\nK3DombH5FO/DNq8qtTksGrcLb1cvupNa968TYQBL0rY0bk9b7zxqATXrMEjgMLaRyXto7u6Lah2j\ns5Bt8d9DS9oL8g6qRx7+IewhtBivhRr9TjYVPpp1HEzR3Uam244EFl4nov1Ag+QUHiS/tS1fFwAg\ndGn7nMr50Pw9lxUo41MOGkCsb9M2nqAKgXXVOsJh+2GzDfIZa7uSah33YQypwKGzTiPn8chHx6BT\n4HprNWvL+2J6jOYRiH2uPoHs9jGqbYo58nSWH0wHVtGSZVTH6dvSQ78yaANnb+a2NReOHAOzBHgg\nhCQAh30AJSg0WoUVc3Ak9JvM4c7cFBTXKmLvpEIdpubOXv6co2A9svMwqhvVUR3vXCHAYIAIHHov\nEskeOgmxGC8NMwqEF4w+LLMr1iVxyPjxXQTZP2DBCHb1rkqjluw12z5GsW+1UB6pOB45gjb7gOH7\nB2BBB2GqiVH9+9QzjhZ+ftUgdGmJaPKBX/heTQGpGcII2tbWO46YyzMYPCfuQXsuTHcr1Wwk3zks\nyZYevsEA/9kXUkD4Zx/9BA4xgUNM4BATOMQEDjHLhQMfivD5WswyDf1vCo7cOf+TKy3XO1Y/ilm2\n6TiQOqRmJHVIRSYldUhFZiV1SEUmJXVIRWYldUhFZiV1SEVmJXVIRRnKPpsUbxEJHCKBQyRwiAQO\nkcAhEjhEAkdWUL4cOaQOqVg2eslB+jqkGzZsoKNHj1qspaamWvTzo/+zmftngt9//z2JLFfof4FD\nJHCIBA6RwCESOP5GzZ8/n+7fv/+PuJdly5bRL7/8YplwPH36lH777bd/DBjffPMNNW/e3GAbpner\nV6+m7777zmA7Om39+vU89bt58+YrtX/+/HluC23q69GjR7RlyxZas2aNwfvftWsXdejQwTLhGDt2\nLDVo0OAfA0dCQgLt3LlTrbdp04bCw8Np9OjR5OPjwzBoEJUrV47vf9SoUeTq6krnzp37U49UuXJl\nGjFiBD/zoEGDePudO3fI19eX+vbtS1OmTCFvb29at26dOs/Ly+uV4ctScDx8+JBu3brFy3fv3qVn\nz57RjRs31Mt4/Pgx38+TJ08MzoPbv3DhAp+vL5x/8eJFevDgAe/DiNQED3Xt2jWz9/L8+XNycHDg\nNqCvvvqK/Pz81H54DnQUhNG8adMmtW/58uXUu3dvvl99T4j7xnPBQ9ra2jII2n3a2dnxfSJ0AAxN\nly5dooCAALWOdnGMxcGxatUq6tSpEy8HBwdTdHQ0xcbGkqOjI02bNo3q169PzZo1o/LlyyuIBg4c\nSIGBgZSYmEguLi504MAB3v71119TmTJlKDIykqpWrUpNmjThNiCMbnR0TEwMXwdgmXL5QUFBav34\n8ePKU0AnT56kihUr8vLIkSPZA+h34PTp0xkMeBSc++LFC6pVqxatWLGC733q1KnqeEBkY2PD8O7Z\ns4c9CQCCEKb0QxveUffu3QWORYsW8fLhw4cpT548ahT26tWLZs+ezesdO3bkF6/FZLh9CG7/4MGD\n6uV7eHgwHNiGtrVz9u3bp87R1/79+ykiIsLkfV6/fp3BQL6geS6EEicnJypZsiSHBc27AQx4GIQc\nhCVj4bi4uDjq0aOH2hYVFcWwAPYiRYqw99O0d+9e3m/xcJw6dYqX4f7Lli2rjpsxYwYNGzaMl3HM\ngAED+IVhZPr7+/MILFq0qEHbcNWAA6O8QoUK7GlgyCsKFy6c7l7geUzlP8eOHSNnZ2cFBtS2bVv2\nYAhFGPHo6D59+qj9yCcQRoxnPYCsSpUqvF+DNTk5ma+rDYSVK1ey59O0bds2vm+Lh+P06dMKDnSI\nMRy4Nzc3N+5IxGu4esCBF40Rp5+bAALAMWHCBHbLyBk0w/nGQp6DkKAv5BUAS4NWE0DE9fVnLvAk\nmlfx9PTkDkaI0JSWlsYeRX8b1LRpU/r0008NtiGsIleBkpKSOCwKHH8Ch348RujAiAIcUM+ePalr\n167cCWi3ePHiDAfaRKfcu3ePj1u6dCnnJaaE0PTTTz/x8uXLlxkC5CIAB3b79m3eV6dOHUpJSVHn\nzZs3T90XOnvWrFmcZwBk5DdIQBEytm7dqtqCwfMMHz6c71vzJAhLpUqVUm0jdzp06JDlwYERM27c\nODVt1BJFzFgaN25sANHcuXN5VIaFhfGUEDBhyhcfH69mG3PmzKEWLVpw8ge3jzwFQkfCncPQrrmp\n4cyZM9nTaF4DHkDfatSowfuuXr1KoaGh3B5mFgACIQPJMZJeraMREtDxAM24LdiVK1cYcgwQ5C1I\ntPFcJ06cUO8B2+QLaSaFHOPs2bNqHXH8s88+e6020FHo8H/KF1LkMdu3bxc4MivMXDBD6dy5M49q\nTIlFAofBhy4knP/r30cIHCKBQ+AQOAQO0evDgS+KmKeLWaah/03BkTtXrlxpVlZWP4pZtuk4kDqk\nIpEoE/o/NLu01jBF4s8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(filename=\"../img/lenet-vert.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1oj2RuIfdbhq"
   },
   "source": [
    "Compressed notation for LeNet5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BnKUIw6Ddbhr"
   },
   "source": [
    "## Data Acquisition and Training\n",
    "\n",
    "Now that we've implemented the model,\n",
    "we might as well run some experiments\n",
    "to see what we can accomplish with the LeNet model.\n",
    "While it might serve nostalgia\n",
    "to train LeNet on the original MNIST OCR dataset,\n",
    "that dataset has become too easy,\n",
    "with MLPs getting over 98% accuracy,\n",
    "so it would be hard to see the benefits of convolutional networks.\n",
    "Thus we will stick with Fashion-MNIST as our dataset\n",
    "because while it has the same shape ($28\\times28$ images),\n",
    "this dataset is notably more challenging."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 275
    },
    "colab_type": "code",
    "id": "ECMwOKTzdbhr",
    "outputId": "9a7f2ac9-c95b-4bbb-e902-dcfbfd16301a",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Gg8uZka5dbhu"
   },
   "source": [
    "While convolutional networks may have few parameters, they can still be significantly more expensive to compute than a similarly deep multilayer perceptron so if you have access to a GPU, this might be a good time to put it into action to speed up training.\n",
    "\n",
    "Here’s a simple function that we can use to detect whether we have a GPU. In it, we try to allocate gpu0 if available using `torch.cuda.is_available()` method. Otherwise, we stick with the CPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Y3zs_KyHdbhv",
    "outputId": "da77a9d2-55c9-4781-8132-7ebba7f6f3f6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda', index=0)"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This function has been saved in the d2l package for future use\n",
    "def try_gpu():\n",
    "    \"\"\"If GPU is available, return torch.device as cuda:0; else return torch.device as cpu.\"\"\"\n",
    "    if torch.cuda.is_available():\n",
    "        device = torch.device('cuda:0')\n",
    "    else:\n",
    "        device = torch.device('cpu')\n",
    "    return device\n",
    "\n",
    "device = try_gpu()\n",
    "device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nhhHxEjVdbhz"
   },
   "source": [
    "For evaluation, we need to make a slight modification\n",
    "to the `evaluate_accuracy` function that we described\n",
    "when implementing the softmax from scratch (`chapter_softmax_scratch`).\n",
    "Since the full dataset lives on the CPU,\n",
    "we need to copy it to the GPU before we can compute our models.\n",
    "This is accomplished via the `.to(device)` described in `chapter_use_gpu`.\n",
    "Note that we accumulate the errors on the device\n",
    "where the data eventually lives (in `acc`).\n",
    "This avoids intermediate copy operations that might harm performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "g4iEi1U8dbh0"
   },
   "outputs": [],
   "source": [
    "# This function has been saved in the d2l package for future use. The function\n",
    "# will be gradually improved. Its complete implementation will be discussed in\n",
    "# the \"Image Augmentation\" section\n",
    "def evaluate_accuracy(data_iter, net,device=torch.device('cpu')):\n",
    "    \"\"\"Evaluate accuracy of a model on the given data set.\"\"\"\n",
    "    acc_sum,n = torch.tensor([0],dtype=torch.float32,device=device),0\n",
    "    for X,y in data_iter:\n",
    "        # If device is the GPU, copy the data to the GPU.\n",
    "        X,y = X.to(device),y.to(device)\n",
    "        net.eval()\n",
    "        with torch.no_grad():\n",
    "            y = y.long()\n",
    "            acc_sum += torch.sum((torch.argmax(net(X), dim=1) == y))\n",
    "            n += y.shape[0]\n",
    "    return acc_sum.item()/n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qZj0UJ91dbh3"
   },
   "source": [
    "We also need to update our training function to deal with GPUs.\n",
    "Unlike `train_ch3` defined in `chapter_softmax_scratch`, we now need to move each batch of data\n",
    "to our designated device (hopefully, the GPU)\n",
    "prior to making the forward and backward passes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WQ8PZBb4dbh4"
   },
   "outputs": [],
   "source": [
    "# This function has been saved in the d2l package for future use\n",
    "def train_ch5(net, train_iter, test_iter,criterion, num_epochs, batch_size, device,lr=None):\n",
    "    \"\"\"Train and evaluate a model with CPU or GPU.\"\"\"\n",
    "    print('training on', device)\n",
    "    net.to(device)\n",
    "    optimizer = optim.SGD(net.parameters(), lr=lr)\n",
    "    for epoch in range(num_epochs):\n",
    "        train_l_sum = torch.tensor([0.0],dtype=torch.float32,device=device)\n",
    "        train_acc_sum = torch.tensor([0.0],dtype=torch.float32,device=device)\n",
    "        n, start = 0, time.time()\n",
    "        for X, y in train_iter:\n",
    "            net.train()\n",
    "            \n",
    "            optimizer.zero_grad()\n",
    "            X,y = X.to(device),y.to(device) \n",
    "            y_hat = net(X)\n",
    "            loss = criterion(y_hat, y)\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            \n",
    "            with torch.no_grad():\n",
    "                y = y.long()\n",
    "                train_l_sum += loss.float()\n",
    "                train_acc_sum += (torch.sum((torch.argmax(y_hat, dim=1) == y))).float()\n",
    "                n += y.shape[0]\n",
    "        test_acc = evaluate_accuracy(test_iter, net,device)\n",
    "        print('epoch %d, loss %.4f, train acc %.3f, test acc %.3f, '\n",
    "              'time %.1f sec'\n",
    "              % (epoch + 1, train_l_sum/n, train_acc_sum/n, test_acc,\n",
    "                 time.time() - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QCTjn-iRdbh6"
   },
   "source": [
    "We initialize the model parameters on the device indicated by `device`,\n",
    "this time using the Xavier initializer.\n",
    "The loss function and the training algorithm\n",
    "still use the cross-entropy loss function\n",
    "and mini-batch stochastic gradient descent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "SRwnEh4ndbh7",
    "outputId": "abcc9495-ed33-41a2-c6f5-7bcb8b82c3e4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training on cuda:0\n",
      "epoch 1, loss 0.0091, train acc 0.103, test acc 0.100, time 4.3 sec\n",
      "epoch 2, loss 0.0077, train acc 0.230, test acc 0.547, time 5.4 sec\n",
      "epoch 3, loss 0.0038, train acc 0.611, test acc 0.590, time 5.4 sec\n",
      "epoch 4, loss 0.0030, train acc 0.693, test acc 0.680, time 5.4 sec\n",
      "epoch 5, loss 0.0026, train acc 0.740, test acc 0.709, time 5.4 sec\n"
     ]
    }
   ],
   "source": [
    "lr, num_epochs = 0.9, 5\n",
    "\n",
    "def init_weights(m):\n",
    "    if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "        torch.nn.init.xavier_uniform_(m.weight)\n",
    "\n",
    "net.apply(init_weights)\n",
    "net = net.to(device)\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "train_ch5(net, train_iter, test_iter, criterion,num_epochs, batch_size,device, lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pRUx8zxq1XmD"
   },
   "source": [
    "## Summary\n",
    "\n",
    "* A convolutional neural network (in short, ConvNet) is a network using convolutional layers.\n",
    "* In a ConvNet we alternate between convolutions, nonlinearities and often also pooling operations.\n",
    "* Ultimately the resolution is reduced prior to emitting an output via one (or more) dense layers.\n",
    "* LeNet was the first successful deployment of such a network.\n",
    "\n",
    "## Exercises\n",
    "\n",
    "1. Replace the average pooling with max pooling. What happens?\n",
    "1. Try to construct a more complex network based on LeNet to improve its accuracy.\n",
    "    * Adjust the convolution window size.\n",
    "    * Adjust the number of output channels.\n",
    "    * Adjust the activation function (ReLU?).\n",
    "    * Adjust the number of convolution layers.\n",
    "    * Adjust the number of fully connected layers.\n",
    "    * Adjust the learning rates and other training details (initialization, epochs, etc.)\n",
    "1. Try out the improved network on the original MNIST dataset.\n",
    "1. Display the activations of the first and second layer of LeNet for different inputs (e.g. sweaters, coats).\n",
    "\n",
    "\n",
    "## References\n",
    "\n",
    "[1] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Convolutional_Neural_Networks(LeNet).ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}