pynq_dpu/notebooks/dpu_mnist_classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DPU example: MNIST Classifier\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aim/s\n",
    "\n",
    "This notebook shows how to deploy Convolutional Neural Network (CNN)\n",
    "model for hand-written digit recognition. The network was trained on\n",
    "the MNIST dataset,quantized using Vitis AI compiler tools, and\n",
    "deployed on the DPU.\n",
    "\n",
    "Compared to the other notebooks delivered in this folder, this notebook\n",
    "shows how to deploy a **user-trained** DPU model on PYNQ image; i.e.,\n",
    "the model used in this notebook does not come from the model zoo.\n",
    "\n",
    "## References\n",
    "\n",
    "* [Train your own DPU models](https://github.com/Xilinx/DPU-PYNQ/tree/master/host#train-your-own-dpu-models-from-scratch)\n",
    "* [Vitis AI model zoo](https://github.com/Xilinx/Vitis-AI/tree/master/models/AI-Model-Zoo)\n",
    "\n",
    "## Last revised\n",
    "* Mar 8, 2021\n",
    "    * Initial revision\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Prepare the overlay\n",
    "We will download the overlay onto the board. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "try {\n",
       "require(['notebook/js/codecell'], function(codecell) {\n",
       "  codecell.CodeCell.options_default.highlight_modes[\n",
       "      'magic_text/x-csrc'] = {'reg':[/^%%microblaze/]};\n",
       "  Jupyter.notebook.events.one('kernel_ready.Kernel', function(){\n",
       "      Jupyter.notebook.get_cells().map(function(cell){\n",
       "          if (cell.cell_type == 'code'){ cell.auto_highlight(); } }) ;\n",
       "  });\n",
       "});\n",
       "} catch (e) {};\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "try {\n",
       "require(['notebook/js/codecell'], function(codecell) {\n",
       "  codecell.CodeCell.options_default.highlight_modes[\n",
       "      'magic_text/x-csrc'] = {'reg':[/^%%pybind11/]};\n",
       "  Jupyter.notebook.events.one('kernel_ready.Kernel', function(){\n",
       "      Jupyter.notebook.get_cells().map(function(cell){\n",
       "          if (cell.cell_type == 'code'){ cell.auto_highlight(); } }) ;\n",
       "  });\n",
       "});\n",
       "} catch (e) {};\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from pynq_dpu import DpuOverlay\n",
    "overlay = DpuOverlay(\"dpu.bit\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `load_model()` method will automatically prepare the `graph`\n",
    "which is used by VART."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "overlay.load_model(\"dpu_mnist_classifier.xmodel\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's import some libraries as well. The `mnist` package\n",
    "requires some additional headers for URL requests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import time\n",
    "import numpy as np\n",
    "import mnist\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from six.moves import urllib\n",
    "\n",
    "opener = urllib.request.build_opener()\n",
    "opener.addheaders = [('User-agent', 'Mozilla/5.0')]\n",
    "urllib.request.install_opener(opener)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load test data\n",
    "\n",
    "The `mnist` package enables the following data for users:\n",
    "\n",
    "* `test_images()`: returns test images stored as a numpy array. \n",
    "Each image is a grayscale 28x28 pixels, representing a digit between 0 and 9.\n",
    "* `test_labels()`: returns a list of the true labels stored as numpy array.\n",
    "\n",
    "\n",
    "There are 2 pre-processing steps we need to do to the test images \n",
    "before we can use it:\n",
    "\n",
    "1. The raw numpy array delivered by `mnist` has a data type of \n",
    "uint8 (data ranges from 0 to 255); we need to normalize the elements to \n",
    "floating-point numbers ranging from 0 to 1.\n",
    "2. The VART API will expect each input sample to have 3 dimensions; \n",
    "so we need to expand the original numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of test images: 10000\n",
      "  Dimension of each picture: 28x28\n"
     ]
    }
   ],
   "source": [
    "raw_data = mnist.test_images()\n",
    "normalized_data = np.asarray(raw_data/255, dtype=np.float32)\n",
    "test_data = np.expand_dims(normalized_data, axis=3)\n",
    "test_label = mnist.test_labels()\n",
    "\n",
    "print(\"Total number of test images: {}\".format(test_data.shape[0]))\n",
    "print(\"  Dimension of each picture: {}x{}\".format(test_data.shape[1],\n",
    "                                                  test_data.shape[2]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAAD3CAYAAADmIkO7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAIrElEQVR4nO3dX4iVeR3H8c933ZF1YdF1ChkyF1QMFDRkZMOglhp0KWdMshRcdmGp2wjHPyAuRiRCBEUEXgQWjl0M08ikaUkXKUKOi5FawtwM6GwxgzvZmpNJ/vl1MccY7Dy/s/OcmTmf2Xm/QHDne57nPIflzc85P55zIqUkAH6ea/QFAKiOOAFTxAmYIk7AFHECpogTMEWcs0hEnI+Ib8z0sWgM4myAiLgZEW2Nvo4iEfFWRPwxIv4ZEX+NiO9HxPONvq65hjhRzYuSvi3pY5JelfRFSXsaeUFzEXEaiYiXI+LXEfF+RPyj8velzzxsRUS8W1nVfhURiycc/5mI+ENEfBAR1yLitTLXkVI6mlK6mFL6T0rpb5J+IemzpV8YSiFOL89J+pmkVyQtk/RvST955jFvSnpbUoukR5J+LEkR8QlJZyR9T9Jija90vRHx8WefJCKWVQJe9iGv63OSbkz61aAuxGkkpfT3lFJvSul+SumepMOSPv/Mw7pSSn9JKf1L0juSvh4R8yS9IelsSulsSulJSul3kq5I+lKV5xlKKS1KKQ3VuqaIeFtSq6Qf1PnyMEn8km8kIl6U9ENJr0t6ufLjlyJiXkrpceW/35twyC1JTRr/3fAVSV+LiPYJ8yZJv6/jer4i6YiktpTSaNnzoBzi9NIp6VOSXk0pjUTEpyX9SVJMeMwnJ/x9maSHkkY1Hm1XSumbU3EhEfG6pJ9K+nJK6c9TcU5MDv+sbZymiHhhwp/nJb2k8d8zP6i80XOoynFvRMTqyir7XUm/rKyqJyS1R8TmiJhXOedrVd5QqikivqDxN4G+mlJ6t/QrRF2Is3HOajzEp3++I+lHkhZofCXsl/TbKsd1Sfq5pBFJL0j6liSllN6TtFXSAUnva3wl3asq/48rbwiNZd4QekfSQklnK48bi4jflHmRKC+42RrwxMoJmCJOwBRxAqaIEzCV3eeMCN4tAqZZSimq/ZyVEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmOIrAKfBnj17svMFCxYUztauXZs9dvv27aWu6amjR49m55cuXSqcdXV11fXcmBxWTsAUcQKmiBMwRZyAKeIETBEnYIo4AVORUvG3/PEVgNV1d3dn5/XuRTbS4OBg4aytrS177NDQ0FRfzpzAVwACswxxAqaIEzBFnIAp4gRMESdgijgBU9zPWUUj9zEHBgay83PnzmXny5cvz87b29uz8xUrVhTOdu3alT32yJEj2Tkmh5UTMEWcgCniBEwRJ2CKOAFTxAmYIk7A1Jzc52xtbc3Ot23bVtf5b9y4kZ13dHQUzkZHR7PHjo2NZefz58/Pzvv7+7PzdevWFc6am5uzx2JqsXICpogTMEWcgCniBEwRJ2CKOAFTc3IrpaWlJTuPqPpJhf9Ta6tk8+bN2fnw8HB2Xo/Ozs7sfPXq1aXPfebMmdLHYvJYOQFTxAmYIk7AFHECpogTMEWcgCniBEzNyX3O06dPZ+crV67Mzu/du5ed37lzZ9LXNFV27tyZnTc1Nc3QlaBerJyAKeIETBEnYIo4AVPECZgiTsAUcQKm5uQ+Zy23bt1q9CUU2rt3b3a+atWqus5/+fLlUjNMPVZOwBRxAqaIEzBFnIAp4gRMESdgijgBU5FSKh5GFA8xLbZs2ZKd9/T0ZOe1vgLw9u3b2XnuftALFy5kj0U5KaWqH5TMygmYIk7AFHECpogTMEWcgCniBEwRJ2CK+znNtLa2Zue19jFr6e7uzs7Zy/TBygmYIk7AFHECpogTMEWcgCniBEyxldIAfX19hbNNmzbVde7jx49n5wcPHqzr/Jg5rJyAKeIETBEnYIo4AVPECZgiTsAUcQKm+GjMadDS0pKdX7t2rXDW3NycPXZ0dDQ737hxY3Y+ODiYnWPm8dGYwCxDnIAp4gRMESdgijgBU8QJmCJOwBT3c06D3t7e7LzWXmbOiRMnsnP2MT86WDkBU8QJmCJOwBRxAqaIEzBFnIAp4gRMsc9ZQkdHR3a+fv360uc+f/58dn7o0KHS58bswsoJmCJOwBRxAqaIEzBFnIAp4gRMESdgin3OKmrdb3ngwIHsvKmpqfRzX716NTsfGxsrfW7MLqycgCniBEwRJ2CKOAFTxAmYIk7AFFspVXR2dmbnGzZsqOv8fX19hTNuCcNTrJyAKeIETBEnYIo4AVPECZgiTsAUcQKmIqVUPIwoHn6EPXjwIDuv55YwSVq6dGnhbHh4uK5zY/ZJKUW1n7NyAqaIEzBFnIAp4gRMESdgijgBU8QJmOJ+zgZYvHhx4ezhw4czeCX/7+7du4WzWtdWa/934cKFpa5JkhYtWpSd7969u/S5P4zHjx8Xzvbv35899v79+6Wek5UTMEWcgCniBEwRJ2CKOAFTxAmYIk7AFPucDXD9+vVGX0Khnp6ewlmte02XLFmSne/YsaPUNbkbGRnJzg8fPlzqvKycgCniBEwRJ2CKOAFTxAmYIk7AFB+NWcXJkyez861bt87Qlcwtjx49Kpw9efKkrnOfOnUqO79y5Urpc1+8eDE77+/vz875aExgliFOwBRxAqaIEzBFnIAp4gRMESdgin3OEvbt25ed1/sVgTlr1qzJzqfztqxjx45l5zdv3qzr/L29vYWzgYGBus7tjH1OYJYhTsAUcQKmiBMwRZyAKeIETBEnYIp9TqDB2OcEZhniBEwRJ2CKOAFTxAmYIk7AFHECpogTMEWcgCniBEwRJ2CKOAFTxAmYIk7AFHECpogTMEWcgCniBEwRJ2CKOAFTxAmYIk7AFHECpogTMEWcgCniBEwRJ2CKOAFTxAmYIk7AFHECpogTMEWcgCniBEwRJ2CKOAFTxAmYIk7AVKSUGn0NAKpg5QRMESdgijgBU8QJmCJOwBRxAqb+C2Bgo5SWY4k8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(test_data[1,:,:,0], 'gray')\n",
    "plt.title('Label: {}'.format(test_label[1]))\n",
    "plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Use VART\n",
    "Now we should be able to use VART API to do the task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "dpu = overlay.runner\n",
    "\n",
    "inputTensors = dpu.get_input_tensors()\n",
    "outputTensors = dpu.get_output_tensors()\n",
    "\n",
    "shapeIn = tuple(inputTensors[0].dims)\n",
    "shapeOut = tuple(outputTensors[0].dims)\n",
    "outputSize = int(outputTensors[0].get_data_size() / shapeIn[0])\n",
    "\n",
    "softmax = np.empty(outputSize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can define a few buffers to store input and output data.\n",
    "They will be reused during multiple runs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_data = [np.empty(shapeOut, dtype=np.float32, order=\"C\")]\n",
    "input_data = [np.empty(shapeIn, dtype=np.float32, order=\"C\")]\n",
    "image = input_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also define a few functions to calculate softmax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_softmax(data):\n",
    "    result = np.exp(data)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Run DPU to make predictions\n",
    "\n",
    "We can now classify a couple of digit pictures. For each picture, \n",
    "the classification result (shown as 'Prediction') is displayed on top of \n",
    "the picture. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1kAAABOCAYAAADSOtyNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABBSklEQVR4nO29eXCb13nw+zvYCIAACIIEF3BfxZ3aJVu2bCdOlMRbHCdNMp2brplmuWnvN9M0bSdtc2++LzftTNu0t1/aJk3jbE3ijBMrcWy5XuQtsayFEiVSJMV930mQWIntvX+Q7xtSoiyJgkSKOL8ZjQjgxcHBg/Oec57zbEJRFCQSiUQikUgkEolEkhx0m90BiUQikUgkEolEItlOSCVLIpFIJBKJRCKRSJKIVLIkEolEIpFIJBKJJIlIJUsikUgkEolEIpFIkohUsiQSiUQikUgkEokkiUglSyKRSCQSiUQikUiSyKYqWUKIJ4UQ/3Pl73uFEF0bbOffhBB/ldze3blIuSYfKdPkI2V6a5ByTT5SpslHyjT5SJneGqRck0+qyPSaSpYQYkAIERJC+IUQkyuCsSW7I4qivKEoyo7r6M/vCiHevOy9n1IU5cvJ7tM6n/1vK3JQ/y0JIXwbbEvK9Tef/TtCiDNCiEUhxIgQ4u+EEIYNtCNl+pvPbhBCvCCEmBFCbLgYnpTpFZ//P4QQEytj9T+FEGkbbEfKdf1+vCyEUOT9f3PI+//WIsdpchBCpAkh/lEIMSaEmBdCfF0IYdxgW1Kuaz87ftl+9f4NtCNl+pvP3tBYvV5L1iOKotiA3cBe4IvrdOCGJ5s7jZUf06b+A34I/OQmmpRyXcYK/F9ANnAAeDfwpxtsS8p0mSjwFPAHSWhLyhQQQhwB/pzl8VkClAP/9000KeW6CiHEbwMb2mCtQsp0GXn/3yLkOE0qf87y928AqlmWxxWyuAGkXH/DW6v3q4qivLrBdqRMl9nQWL0hd0FFUUaB51c+hJWTnM8KIbqB7pXnHhZCnBNCeIUQvxZCNKnvF0LsEkK0CCF8QogfA+ZVr90vhBhZ9bhICPFTIcS0EGJWCPEvQoha4N+Au1Y0a+/KtZrZceXxJ4UQPUKIOSHEz4UQnlWvKUKITwkhulf6+L+FEOJG5LDSTjrwBPCdG33v5aS6XBVF+deVk4zIiix+ABzagChXt5nqMu1SFOVbQPtG5HeVNlNapsDvAN9SFKVdUZR54MvA796YFK9EyhWEEBnA3wB/doPiW5dUl6m8/+U4vUNk+gjwz4qizCmKMg38M/D7NyjGK5ByTT5Sphscq4qivOM/YAB4cOXvIpYn7S+vPFaAFwEXYAF2AVMsWyP0LG9KBoA0wAQMAv+D5VOgD7N82vY/V9q6HxhZ+VsPtAL/CKSz/GPcs/La7wJvXtbHJ1e18y5ghmUtMw34/4DXV12rAM8CTqAYmAbet/JaMeAFiq9DLp8A+gBxrWulXK9frivXPwN8Vcr05mUKVALKRsaolOkVsmgFPrrqcfZKe1lSrjc3VoH/vfIdSlfaMkiZyvt/q8kUOU6TPaeeBn5r1ePfXmkvQ8r1puT6u0Bgpf1LwF8hx+rNynRDY/V6hexf+fBB4OuAZVWH37Xq2n9Vf4BVz3UB9wGHgTFWKSXAr68i5LtWvvwVg+I6hPwt4O9WvWZb+TFLV/X5nlWvPwX8+QYG38vAl270fVKu15TL7wMjQLaU6c3LlORssqRMl6/tZWVCXnlsXGmvVMr1puS6FzgHGLj5zauU6drPk/e/HKdbWab/E/gV4AbygLdX2suXcr0puZYDZSx7qzUCF4G/kDK9/WP1ev0oP6goyktXeW141d8lwO8IIT636jkT4FnpzKiy0tsVBq/SZhEwqChK7Dr7txoP0KI+UBTFL4SYBQpYHjAAE6uuD7L8Q1w3QohilgfFJzfQv9VIua5CCPFB4P9l+eRkZgN9BCnTW4GU6TJ+wLHqsfr3hpLfIOWKEELH8sL9J4qixJLgDZPyMr0FpLxM5Ti9ZeP0f7FsVTgHLAHfZNkiMrmBfoKUq9pW36qHF4QQ/w/weZb3VzeKlOkyGxqryUjhvlpow8D/UhTFueqfVVGUHwLjQMFl/o/FV2lzGCgW6wfTKes8t5oxln9sQIudygJGr/VFboD/A/jVZQM52aSUXIUQ72N50D6iKMqFZLS5Dikl09tEKsm0HWhe9bgZmFQUZTYJbV9OqsjVwbKF4MdCiAng1MrzI0KIe2+y7ctJFZneTlJFpnKcrpDMcaooSkhRlP9TUZQCRVHKgVngjKIoiZtte72PW/X3tpbrVfpyK+K5UkamGx2rya6T9U3gU0KIA2KZdCHEQ0IIO/AWEAP+WAhhFEJ8CNh/lXZOsvyjfHWlDbMQQk2EMAkUCiFMV3nvD4HfE0LsFMvplb8CvK0oykCSviMsx2M9mcT2rsW2lqsQ4l0sJ7t4QlGUkzfb3nWy3WUqhBBmlk+SWOnXhtKN3wDbWqbAd4E/EELUCSGcLGcWejIJ7V6L7SzXBZZPH3eu/PvAyvN7WHbHuFVsZ5nK+1+O08vZijJFCFEghPCsfLeDLMcO/c3NtnsdbHe5vl8Ikbvydw3Lcj16s+1eg+0u0w2N1aQqWYqinGbZhe5fgHmgh5XsW4qiRIAPrTyeAz4K/PQq7cRZzuRRCQyxHKPz0ZWXX2H5RHlCCHGFS9mKWfOvgKdZ/qEqgI9dT/+FEMViOWvJ1TRshBB3AYXcXOr2GyIF5PpXQAbwnPhNTYfnr6ftjZICMi0BQvwmu1iIZf/oW8Z2l6miKMeAvwOOr/RrkNuwIdjOclWWmVD/seyLD8sWwsj1tL8RtrNMV5D3vxynq9vZcjJdoYLl2JwAy5ma/1xRlP++nrZvhhSQ67uB80KIAPDcSv+/cj1tb5QUkOmGxqpQlGtZ3yQSiUQikUgkEolEcr0k211QIpFIJBKJRCKRSFIaqWRJJBKJRCKRSCQSSRKRSpZEIpFIJBKJRCKRJBGpZEkkEolEIpFIJBJJErneYsTXjRDijs+koSjKragncFNIuSYfKdPkI2V6a5ByTT5SpslHyvTWIOWafKRMk4+U6ZVIS5ZEIpFIJBKJRCKRJBGpZEkkEolEIpFIJBJJEpFKlkQikUgkEolEIpEkkaTHZEm2NkKIK/5e/ZyiKKgFqmWhaolEIpFIJBKJ5MaRSlYKUVdXR11dHUajEaPRiMfjwe12U19fjxACRVE4efIkY2NjDAwMMDExwblz5za72xKJRCKRSCQSyR2FVLJSAKPRiMPhoLy8nObmZkwmE2lpaZSWluLxeNi7d6+mZBkMBgYGBrBYLBiNRtra2ojH49KqlQT0ej1ms5nMzEzC4TDhcJhQKEQ8Ht/srm1phBAYDAbS09NJS0vDaDQSi8WIxWIsLCwQj8dJJBKb3U2JRCKRSN4RIQQ6nQ6dbm20jqIoxONx0tLSMJlM2roWi8VIJBJyn3CHIpK9eZYpHG8NNyPX+vp6PvOZz9DU1ERDQ4PmHqjX6zEYDKSlpWnXqpv/4eFh3n77bb785S/j9Xrx+Xw3/R22mlxv51jV6/UUFBRw6NAhvvrVr/Laa6/xyiuv8OKLLzI6Orrhdre7TIUQpKWlUVlZycc//nF27txJY2MjAwMDDA4O8rd/+7eMj48zOzubtM/cajIFOa/eCqRMk4+U6a1ByjX5bIZM1cNCu92Ow+FY81o4HGZmZoY9e/awe/duRkZGWFhYYGRkhMXFRYaGhq5oT8o0+SRbptKStY0xGo1UV1fT1NREfX09RUVF2o2tKAqhUIhAIMD4+LjmQpiRkYHdbicnJ4f8/HyKiopIJBJJUbJSGZ1Oh9PpxO124/F4yMjIwGAwrImHk1yJEAKz2Ux2djYNDQ1UVFTg8XgQQmgur+FwOKlK1nYlLS0Ns9lMRUUFJpOJlpYWIpHIdb/f4XBgMBgIhULEYjGi0egt7O32QqfTkZmZSWZmJpWVlUxPT2v/QqHQZndvS6De0/n5+eTm5hIIBAiHw0xOThKJRG5orK5uU6/XY7FYMJvNLC4uEolEpGeG5LaRkZGBxWLB6XRis9lwu904nU6cTuea60KhEJOTkzQ2NtLQ0IDH4yEQCDA9PU1fXx/Dw8Ny3N6BSCVrG2O32/nsZz9LXV0d99xzz5oNfTweZ3x8nImJCVpaWnA6nbhcLvbv34/b7SYnJ4fKykoeeOABXn311ZuytkiWT7BKSkrIy8tDURQCgQCzs7Nyo3oNVOW0rKyMI0eOoNfrAfB4PFitVvbs2YPJZKK3t1cuQNfA5XLh8Xj4y7/8S/Lz83n88ceZnJy87veXl5eTmZlJf38/fr+fmZmZW9jb7YXJZGLXrl0cOnSIL37xixw7doznnnuO559/nqGhIenuCtoh38MPP8yHPvQhOjs7GRkZ4ejRo8zMzDA1NXXDber1eqxWK6WlpRQVFdHS0sL09DSxWOwWfAOJZC1CCKqqqigqKuLw4cMUFBTQ0NBAVlYWLpdrzbWBQICxsTEyMjJwOp0YjUZ0Oh2KonDs2DGOHTsmXQbvQKSStU3JysqioKCA2tpaSkpKEEIQCATw+/289tprDA8PMz4+zuLiIsPDw1itVmw2Gzabjbq6OrKysrQFymQybfbXuePR6/Xk5OTgdDqJRqNMTEzQ0dGB3+/f7K5tWdLS0sjIyOChhx5i165dV/iwS24Ml8tFVVUVGRkZGI3GG7KiCiEoKSmhqKiIiooKhoeH+e///m+p2F4HJpOJzMxM3vve99LY2Iher2fHjh0AtLS0MDo6mvJKluoS/JGPfISdO3dSWlrK3Nwcfr8fv99PIBC44TZNJhNlZWU8+uij5Obm4nK5WFhYYGlpCa/Xm7IyNxqNWCwW9u/fj91ux+l0cvbsWc6fP5+yMkkGQggcDgeZmZns2LEDj8dDQUEBRUVFOJ1OSkpKsNvtuN1urFbrFeuZ2WzG7XZrccfqgaIawyW5M9lUJUsIse5CvzqNuGRjZGZmUlBQQHV1NTk5OQD4/X7Gx8d56qmnOH36NF6vl2g0SjgcRq/Xk5aWxqFDh3C73WRmZmrPqTe7ZOOsVrIikQgTExN0dnZudre2LEIIrFYr2dnZvO9976O8vPyKuUJdfFbPI3LeuDpZWVlUVFRgt9tv+J4WQuDxeKirq8PpdHL+/HlefPFFKe/rQE128573vIeioiLtdLusrIxvfOMbKb+B0ul0WK1Wqqur+fSnP43FYsFkMjE4OMjU1BQ+n++GlSzVzbiyspJPfvKT2Gw20tLSOH78uBbrkoqoMa6ZmZk8+OCDFBQUUFJSAkB7e7vce90El3td7Nu3j927d69RmN4Jo9FIZmam9nj17yCV37VcrfzQ6v+3CpumZBUXF1NaWsqePXtwuVwsLi5qGcP6+/s1l5RwOMz09LQcZDeIw+HA6XSi1+u1gMpnn32WY8eOce7cOc1VTR2QtbW17Ny5k4aGBgoLC9Hr9Xi9Xk6fPs3Y2Ngmf5s7m9zcXIqKivjABz6A2WzmtddeY2BgYLO7tWWpqamhpqaGpqYmCgsL2blz5xVBwrDsgllTU4PZbEav1zM3N8fc3BwDAwPSQngZOp2OgoIC9u3bR0ZGBsFg8IberyiK5sqyY8cO8vLyMJvNLC0tSReWa1BaWqoddqnjeGBggN7eXqamplLaZdhqteJyufjDP/xDGhoacDgc9PT00N3dzfe+9z06Ojo2FA9stVr54Ac/yJ49e8jPz+f06dOcPHmS1tZWJicnU3I/UVpaSkVFBUeOHKGiooKmpibMZjNWq5WCggKsViuBQEC6Um4QvV5PZmYm1dXVPPbYYzidTkwm0zUPUeLxOJOTk/j9/qu6xHZ0dGw55eF2kpaWhsvl0uKK6+vrcbvdVFVVYTAYUBSFrq4uuru7GRoaYmFhYcvoDZumZGVnZ7Njxw72799PXl4es7OzWjC1zWbDaDTi9XoJBoPo9foN3fjxeJxYLKYFam8Fgd8u1NSfU1NTmEwmRkdHaW9v5+TJkywsLFwRRJybm0ttbS1ut5v09HSEEEQiEaanp294QyZZi81mIysri+LiYoLBIL29vczPz292t7YseXl5NDc3s3v3bjweD1lZWRiNxiuu0+v15ObmAmgumJOTk8zNzbG0tJTSm9fVqKUDsrKy8Hg8WkzgjS7aoVCIUCiEw+HAbrdjMBiIxWJSyXoHdDodOTk5FBUVYbVatXE8NzdHb28vgUAgpdalyzGbzTidTnbv3q0lZJmdnaW9vZ0LFy7Q09Nzw22qbu719fWUl5cDMD4+zrlz51I20YherycvL4+6ujoOHjxIVVUV2dnZmgKQlZVFdna2dj+n8ob+Wuh0Oi2ZSjweJx6Pa/uptLQ07HY7hYWFmmzV+zsSiWjXq3tT9b3Dw8NaJsH1GBoaSrnfRK/Xa/ey3W7X5tD09HSam5u1+DaTyYSiKGRkZGhK2MzMDAaDgWg0SjQaJRQKEQ6HN+V7bJqSdeDAAf7oj/6IwsJC0tPTSSQS2mB8+OGHiUajBINBQqEQ/f39N7xhisfjzM7O0tfXx3PPPadtwFKF9vZ2ent76e7uBiAYDGon/est6uXl5Tz44IPk5ORI98AkIoTA7XZTWFiI1WplcnKSl156ib6+vs3u2pbl4MGD/Mmf/InmZrGeggXLC9pdd91FIpEgGo3i9XqZnZ3la1/7GmfPnqWnp0eeyrJs1a6vr6e5uZn6+nqOHTvGhQsXbnjRsVqtWmzBzMwMZrNZKrLvgDp2jxw5wqFDh7Bardpr3d3dHDt2jOnp6U3s4eZTWFhIdXU1u3bt0tzaW1pa+Pa3v834+PiG2szLy6OsrIwPf/jDGAwG3njjDY4fP85LL72Ukm6Cq90Df+/3fo+cnBwsFssaV6u7774bk8nEd77zHTo7O/H5fCm3qb8e1Lir4uJijhw5wtjYGCMjI1y4cAG/308sFiMSiRAIBIjH40SjUdLS0ojFYpw9e5a5uTlGRkYYHx9nfHycgYEBzR02Fotd9UA7FTNiejwe3G43Dz/8MKWlpezbtw+73a7lCVDnV3UcV1VV8eijjxKJRFhaWmJwcJCZmRm6u7t5+eWXefXVV4nFYrddjpumZHm9XoaGhojFYlgsFs1iZTabtWJs6enppKeno9PpiEQiRKNRDAYDBsOV3VYLucXjce11t9uNyWSivb1dS4+ZKkSjUeLxOMPDwwDawLtcwbJYLOTm5lJQUEB2djZGoxFFUfD5fCwsLOD1ejftBGC7UFRURGlpKfF4HJ/Pp7kGSNbicDioqqqivLwcp9P5jvEBqy3TRqOR9PR0jEYjaWlp7N+/H7PZzPDwcEorWWpcW35+PgcOHCAvL49oNEpfXx+dnZ03rCBlZmaSk5OjLXBXi6mVLKOeaufl5ZGfn68dXimKoq1/qWhVgd/UaFTXHqvVSjgcZmBggIGBAWZmZm54fJpMJtLS0mhqaqKmpobMzEx8Ph+Dg4PanJuK84HVaqWyspLCwkKys7Mxm82alUWdX91uN7W1tdx999243W66u7tZXFxkamoqpS2tq9HpdJjNZnbt2kVFRQW7du3S3NR1Oh2JRIKFhQWGhoZ48803WVpaIhQKYbPZSCQSXLhwAa/Xy9TUFNPT08zMzDAxMUEgEND2a6l8aKVardTsirW1teTn59Pc3Ex+fj6FhYUYDAb0ej1+v5+lpSUikQhms5m8vDzi8bhWKgOW9wjp6ekA9PX10d3dzeTk5G2fczdNyXr77beZmZmhrKwMi8VCf38/ZrOZoqIi8vPzyc/Pp7GxkezsbMrLy4lGo0xPT5ORkXFFfIaqYAUCAYLBoOa7CcvasOrydunSpc34qptGIpG4Zv2g3NxcHn/8cfbv34/H48FoNBKLxejr66Orq4uuri6WlpZuU4+3H0IIDh06xKFDh7QUrT09PSm7uXonqqqq+Ju/+RtqamquuXkPhULaIpaWlkZOTo6WHfNzn/scIyMjvPDCCxvKSrZd0Ov1FBYWctddd/H5z3+eSCTC+Pg4L774Ir/61a9uaAwKIaitrWXXrl0y2+h14nQ6tWyMpaWlwPJalUgkGBkZ4cyZM5vbwU3EbDZjs9loampi586dmEwm+vr6+Od//mfOnDnD4uLiDbeZkZFBbm4un/rUpzhw4AAOh4OhoSFOnDjBwMBAys656hrf3NyMzWZbd24tKSmhuLiYPXv2MDc3x5NPPklbWxvPPvss0WhUKlosH+bl5OTwhS98gfLycsrLy4nH47S3tyOEIBaL0dPTw9jYGOfOnWNxcRGv16sd9k9OThKPxzVZpppl6lpYrVZKSko4cOAA+/fv595778Xj8WCz2bRDAZ/Ph9frpb29nenpacbHxykuLuZDH/qQprRWVVVht9u1ZDdut5tgMKilwlcND7eLTVOyVLe16elpTCYTc3NzGAwGhoaGsNls2O12Tpw4gdPppLi4mHA4zMjICNnZ2WRnZ69pS1EUotEoCwsL+Hw+fuu3fouKigrMZjOxWExaY9bBYDBQXFxMU1MTDz74IBUVFej1epaWllhYWOC1117j3Llzmh+x5MYxm82kp6eTn59PXl6eZo2VMl2LXq/XCl/n5OSQnp6uWbEuX4jGxsYYHR1lYGCAxcVFgsEgNpuN8vJyysrKKC4uBpatCOqmItUOV1RUd8rm5mbMZjMXLlzgxIkTjIyM3JD7icFgwGw2a7/Rep4EkitRrQM2m017LhAIMDo6mvLFsysrK9m5cyeHDh2ivLyc+fl5BgcHaWtr21A9LFhWskpLS7VD1osXL3Lu3DnOnTvHxMREkr/B1kcIgd1uJz8/n927d5Ofnw/A/Pw8S0tLBINBLBYLeXl52vVq2YwDBw4ghOCVV14hGAym9EGrWtD68OHD1NbWUlpaihCCN998k/Pnz9PX17dGgY9EIszPzxMOh0kkEvh8Pi2vgFRWr8RsNnPvvfdqMVZlZWWUlpbidrsxGo0MDw8TCoXwer1cvHiRzs5OxsfHCQQC+Hw+6uvr2bNnDy0tLZw9e5aHHnqI0tJSLU1+KBRix44dOBwO2tvbmZmZIRwO3zYld1PdBb1e7zteY7PZtNSuwWCQnp4ePB6PNlmoKIpCJBJhdnaWhYUF9uzZQ2FhISaTSRvwqXqKdTUMBgPl5eU0NjZy//33a1lwfD4fs7OzvP766/T09FyRIENy/ahB3bm5ueTm5mrusZFIRE62q1AV/uLiYrKzs7FYLOsqWYqiMDw8zKlTpzh79ixTU1MEAgFcLhdNTU0IISgqKgKWTx137tyJwWCgp6cnJeVtNps5cOAANTU1mEwmOjs7eeaZZxgZGbkhtxSj0YjVasXj8VBYWJiSstwI6ylZoVCI7u7ulE98o8YAHzx4kOzsbDo6OhgeHubixYsbOhAVQuByuTQly2Aw0NbWptV/SkWrgU6nw+Fw4PF42LlzJ2azGUVRmJubw+v1Mjc3p7kAq66/RqMRu93O3r17CYVCpKenE4vFUlrJ0uv1mEwm7r33Xg4ePEhRURGjo6O88cYbnDt3jr6+vjXyicVia2L/ZOKwq6O6tL/nPe9hx44d3HXXXaSnp2t7gHA4rMVWDQ4O8tJLL/HKK6+ssa76fD4ee+wxTp48yfPPP09ZWRlpaWns2LEDvV5PMBikqqqKnTt38oMf/IDOzk6Wlpa2v5J1PYRCISKRCG1tbSQSCYLBIMPDw1ecdKmbsby8PBoaGrSCbqOjo3R0dPDaa6+l5EnWO6EqWYWFhWsqi1+4cIHOzk7OnTvHzMzMZnfzjkbN2Gi327U4t1R2X1sPp9NJXl4ef/zHf0x5ebkW86OSSCSYm5tjZmaGixcv8vrrr/Pmm28yOztLOBwmHo+Tnp7OxMQEbrebpqYm0tPTsVqtfPzjH6ezsxOTyURHRwddXV0kEomU2HDV19dTUVHB/v37yczMZHx8nMHBQbq7u2940W9sbNRiuiKRCJcuXaKnpwe/3y8PYd6B3NxcGhoaNCUrHo8zODjI97//fVpbWze5d1sDNY7a6/WysLBww65pRqMRj8fDfffdx759+zS3d7/fzze/+U36+vpS4n5fD4PBQElJiZZ0aXZ2lqmpKf7lX/6FtrY2APLz8zl06BCNjY00NDRoVsCMjAx27tzJX//1X/P888/z4osvEgqFUsoDQ6fTYTKZaGpqoqmpiQceeIDq6mqWlpbo6+vj6aefZmJiYt1Yd8k7o9PpMBgM3HvvvVRXV/P+978ft9tNRkYGkUgEr9fLyZMn6e/v52c/+xlzc3OaAeDyOcJgMOBwODCZTJr1dWBggD/90z/FbrdjMpkwGAybFj+8pZWsy9NjwvIpwXobVZ1OR2VlJeXl5TgcDoQQTExMMDY2xtjYmNzcrsLhcOByuSgoKMDtdqPT6YhGo1pGlu7ubmZnZ2VyhpskPT1dUxoSiQTz8/MbqvmynUlLS9Oy3xUWFmKxWDT/a7VQ9sjICKOjo5w7d4729na6urrWWANVf+uJiQlmZ2cxmUxYLBYqKioQQtDc3MzCwgJ9fX0pkaVJCEFeXh6lpaXk5ORgMBgYHBxkdnb2hmJd1GLP6uGVzWYjFosxMjLCxMSEjNW4CqpFwOFwkJubq8UHx+NxFhcXaW9vT3l3wdUoiqK5qmVnZxONRonFYoTD4XUTVajy1ev12Gw2SktL2b17N01NTVRXV2MwGFhcXNTcilIVIQQ2m4309HT0ej0+n4+hoSFaWlpoaWnBZDKRl5eH0WjEZrNRUlKCw+HQEo9lZWXR3NxMa2srRqMx5UIuDAYDNpuN4uJimpub8Xg82O12BgYGGBoaor+/n1AoJOfADaB6R1RVVdHc3ExJSYkWJhAIBJientbW+paWFhYXF9c90FMTMEUiEaxWK263m0gkoq1zRqMRo9FIJBIhFAqtqQ17u9jSStb1oi5oDz74IL//+79PQUEBoVCIH/3oR7S1tbGwsJBSJzDvhE6n4w/+4A/Yt28f99xzj6aQdnd309bWxpNPPsnFixelUpoEXC4XFRUVWpHHp59+mvb29m2/yb8R1DSsTqdTywSk0tPTQ39/P1/5yleYmJjQ6l9drigtLS0xMTHBm2++iRCC3/7t36aqqoq0tDSqqqr47Gc/i16v17ILbffNghCCnTt3cuDAATIyMhgeHuZnP/sZXV1dN9SOuundtWsX733ve3G5XPh8Pp566ik6Ojo2ZcG6E7Db7drmobGxUcvYGg6HmZ+fp7u7O6WziF1OWloae/fupbi4mJKSEq2mzSuvvMLo6OgV1wshqKmpIScnh6amJvLy8mhsbMRkMmE0Gjl9+rTmEpTKrFZGAS5cuMDPfvYzTfGMRCKMjo7y7LPPYrfbqaioICsrS9vsxuNxwuGwFkucave6y+Xirrvu4rHHHuPRRx8lGAzS1dXF5z//eS31ulSwNkZ+fj4VFRV88IMfZPfu3VitVqLRKPPz8zzzzDMcPXqUixcvMjc3RzAYvELOQggMBgPZ2dnE43GOHTtGbW0tDz30EHa7HbPZvCZBXk9PD+3t7QwPD+P3+2/r77YtlCyr1UpZWRkFBQXk5OSg0+lYWlpiamqKubk5qWCtoPoWFxYWUlZWRmZmJkajkWg0ytjYGOfPn2d8fByv1ysnj5tAPQHLz8+ntLRUS9k8ODjIxMREyi1W10K1mKjm/HA4TDgcpqOjQ4vVmJ+ff0c3N7UuXm9vLzMzM+Tl5WG32zEajWRkZGCxWLRTr+2M6oaRlZVFbm4uer2eQCBAR0fHDScUsFgslJaWkpeXR0ZGBrFYjMXFRYaGhuQ4fgfUjFaqqwosW2XV+jjSArgck61uVCORiFbLSXXHUmsNrVd2RafTabFXlZWVOJ1O7HY7sVhMi3lrb29PeUVWLUCclZUFLK9L6jyoEo/HCQaDCCE0i5eKuukNBoObUl9os0lLS8Pj8ZCVlYXNZmN2dpbZ2VlGRkZkavubRE2mpMZfCSFQFIVYLKYVaXa5XFitVsxmszYu1b281Wpd8/vk5ORoMcNms1k7XEgkEsRiMcbGxrhw4QKLi4u3/XfbFkqW2+3mAx/4AI2NjTgcDgKBAIuLi4yNjcm4olVYLBYyMjIoLy+nqqoKs9msubCcOXOGp556ivHx8ZQ/AbxZbDYbO3bsYP/+/bz73e9mamqKoaEhWltbU6pW20aZnZ1lbGyMH/zgB5w5c4aJiYnrOigZGRkhGAxy//33Y7PZqK2tvWoh4+2K6oZRWlpKeXk5RqORyclJnnvuuRuuEeR2uzly5Ah1dXU4HA6Gh4cZHh6mra1NjuN3QHWDUTe3gGaZOX36dMptVtejq6uLQCDAzp07yczMJDs7G5vNRn19PbB88HLgwIF1ZXW1+mxqzMZzzz3HiRMnUj7hQFpaGgcPHqS2thYhBAUFBezdu5e33nqLkZGRNdeqCuvqgtl+v5+uri4mJia2vfV/PdLT06mpqSE7OxtFUbTiwVNTUxsqMSD5DTqdTssDoN7jqvXU5XJRU1PDPffco7mxqhltVSWsqKgIp9NJaWmplsFRLeGyGjXr+OnTp/nxj3/M6OjobTe63NFKll6vJzc3lx07dnD48GFKSkpQFIVXX32VtrY2hoaGrpnBMJVoaGjgrrvuorS0VDs9WFxcpLW1lf7+fubm5mQgexKwWq2Ul5eTm5uL1WrF7/czOztLJBJJyWKY14NqzQI4c+YMP//5z+no6GBhYeGGTp4SiYS2AVu9GSsrK+OBBx7ghRdeuGKDsZ1QLVnqCWFXVxcDAwMbOr0zGo1kZWVpGy+v18v09LT0DLgGZrOZgoIC7HY7sLzQ+3w+Wlpa6OrqkkoWsLi4yMjICD/5yU84efIkRUVFWia8kpISioqKiEQihMNhLl26tMYqpSgK/f39JBIJnnjiCc3NeGFhgeHhYSYnJ5mfn095S0M0GqWzsxOLxaLVDVO9faanp3G5XDgcDoqKimhsbMRisWgugmazmYyMDHbv3k1fXx/nz5/H6/WmlHXQbrdTX1+vZV/0eDwkEgkOHz6sxQn7fD4Zu74B5ufn6e/vp7+/n8zMTMrLyxFCkJGRQVNTk2bFMplMOJ1OzdKl/nM4HJjNZkwmkzafrk6YFY1GGRgYYHh4mDfffJMTJ04wMzOzKfvbO17J8ng8VFdXc+jQIYxGI4lEgldffZU33niD4eFhqTSsIISgsbGRj370o5SUlGA2m4Hlxe78+fMMDAykfFrhZGGxWCgvL8ftdmM2m9coWam+8F/O5a6CAC0tLXz729/eUHuqS6Dapjo5l5aWcu+99/L222+nhJKlLlDd3d0bLr5oNBpxuVxa6mepZF0fqpKlxgQsLS3h8/lobW1lcHBwk3u3NfD7/fj9fn76059qlj+1nhOAx+MhEAjg9Xp5++2311ilFEXh+PHjJBIJ3ve+92lKluqSPT09vSaFdqqiKlmZmZnaxrSwsJC8vDzm5uYoLy+nqKiIgwcPUl9fT1paGoFAgEQigclkwuFwsHv3bs6dO0dOTo6WOCBVsNls1NXVaePL4/GQnp7OvffeS19fH2+//fY7JlW7/DBFXYskaG6ofX19uFwucnNzsVgs2O126urqaGhouMLKlUgkNAvY5ayu3ZhIJFhaWtKSZnzjG99gcXFx05ThO1rJMplM7Nq1i5qaGgwGA5OTk4yNjdHe3k5PT4+0GqzgdDqpqamhqalJS8QQjUbp7u6mpaWF733veymdhSnZWCwWysrKcDqdKIpCW1sbp06dkgr/ZQgh2LdvH83NzWtOpDa6EBUWFtLY2EhNTQ0lJSXodDotZXtrays/+MEPtv04j0ajBINBRkdHGR0dJT8/X8uKpZbEuBZqEdOioiIOHTpERkYGgJZ5VM6r62MwGKioqGDXrl3cf//9mtyWlpa0gxbpZrQWdUyqa/a5c+d45plncDqdRKNRotEoMzMz6445tYhuMBhkcnKSl19+WasDJ1meC9ra2sjKyiIQCJCenk5ZWRlf+MIXCAQCWCwW0tLScDqd+Hw+zp07x9GjR5mbm+Mzn/kMOTk5OJ1OGhsbeeSRR3jqqadSympjMBjIyMjQXM71ej0Oh4PHH3+cQCDAxz72Mdra2jh//jwjIyOEQiEyMzOJRqNaLTLVk0qn0+HxeAgGg5pbeyq7s6oW0+9///s8++yz5OXlUVRUxH333Udubi5ut5tQKMTS0hLz8/MoioJOpyMrK0vLjH15oixY3jucPn2a3t5evvvd72qF3zdzzbqjlSyDwUB+fj7Z2dkIIfB6vfT392snWdJqgBbQWlFRQV5eHg6HA71eTyQSYWZmhtHRUXp6emQcVpLQ6XSYzWays7Mxm81Eo1HGx8cZHh6Wm9PLUOMEysrKMBgMG1KuVIuVwWDA7XZTWVmpxXeoJ4fxeJyZmRl6e3u3/cKWSCSIRqMsLi7i8/lwOBxkZWWRn5+v1SJaD71ej06nQ6/XYzAYcLlc5Ofnk5eXpymrPp9PzqvvgE6nIzMzE7fbTV5enna6GgqFCAQCKWcJuB4uL9NyvbF+5eXlpKena0muhoeH6e3tpbOzU2bGXSEejzM3N8fU1BSTk5NkZWXhdDppamrSrlGThYyNjdHV1cXJkyeZm5vjiSeewGKxkJ2dTU5ODtXV1djtdvR6fcpYshOJBOFwWMvSqP5fUlICLG/o1QQMLpeLQCCgpRCfnJxkdnZWywlgNBopLi4mEAig1+vxer1aSZdUnBPUdbm3t1fbo5aWlpKRkUFhYaFW6y4cDmsJm1QZKopCTk7OFe2Fw2GCwSCXLl3i4sWLXLhwAa/Xu+l72ztWyVJrO+zdu5eysjJisRgnT57khz/8IYODgykzEbwTQggcDgfV1dV84hOfoLKyck0AYX9/P8PDwylXZPBWoW6yCgsLaW5uBpZP/0+ePMlbb7216Tf7VqS+vp79+/ev8ae+XnQ6HQ6HA7vdTkFBAe9///t5/PHHyc3N1a6JxWIsLCykTIYs9TtOTU0xPT1NdXW1Vhfv5MmTnDlzZt33ZWdnaxsqm81GXl4e+fn5VyQY2O7yuxl0Oh12u13b/MOyvLq6umhra0vJzdStQAjBkSNH2LdvHxaLhe7ubr7+9a/T3t4uM+OuQlEUzU31i1/8Ih/84Ad55JFHsFqt6PV6rbbgSy+9xMsvv8zx48fxer1YLBbefPNNfD4fZWVlFBUVkZaWRmVlJTMzM9ediOhOZ2xsjB/+8Ifs27dPc2NdjRCC+vp6qqqqtHlXTeaQSCSIx+PaWBRCaIfbc3NzDAwMMDAwwL/+679y8eLF2/3VthRqbayuri6GhobQ6/Xa+FSVMViW4cMPP8xDDz1EWVmZ9n7VKnb8+HGOHz/Oiy++yMjIyJZJsX/HKllqeuzc3FzS09OZnZ1lfHycoaEhQqHQZndvS6BWfFd9r9UYAa/Xy8zMDB0dHRsOipdciV6vp6CggPz8fOx2O/Pz88zMzBAIBKSCdRVMJhNpaWkbSq1uMBi0BCP19fXU1NTgcrm0ukRLS0vMzs7S3t6uzQupMNYTiQTDw8NkZWVRXV2N0WiktLSUUCh0VWXWbrdjt9vJzMzEarWSk5Ojubupi93MzAzj4+MpscHaCGoipqysLIQQWvrg0dFRBgYGpCU7CdhsNhwOB5WVlVRWViKEIBwOa/NsKtzfN4KqaKl1MNX1yWq1EovF6Ovro6WlhZ6eHqanp7Ux2tnZidlsZnJyUjs8LCsrY2ZmhpmZmZSYAxYWFjhz5gyKohCJRCgrK9OS2RgMBq0u241ksI3FYlitVi05UVVVFYFAgJGRkZSQ6dVQFEVzD14Pi8Wiza1ut3vNOubz+ejq6uLChQu0t7czOTnJwsLCljkQvCOVLCEEhw4dYvfu3ezYsYNIJEJLSwsXLlygs7Nzs7u3ZbBYLLz//e9n586dVFVVaacsFy9epKurix//+MdMT09vmcF4p2M2mzl06BD79u0jMzOToaEhurq6UsqP/XZisVh49NFHaWxs5KGHHlqTQEN1ETx79iz/9E//RF9fH9PT05vc49uDmhigq6sLh8NBRUUFe/bsoaKigscee2zd96yOEdDpdDidzjUBxoqicPbsWd544w15YHAVTCYT+/fvp66uDlgu9hoMBjlx4gRvvfVWSqbBTjbFxcU0NDTwwAMP0NTURCgUIhgM4vP55Li8Cmq8lZrhUo178fv9dHZ28l//9V9rXFlDoRBHjx5lYGCA6upqamtrqaqq4r3vfS8ej4fW1taUkPXo6Cjf+c53ePHFF/F4PHz+85+nqakJvV6vuVKul4ThnTAYDJr3RUVFBYODgxQXF/Pkk0/i8/lu0Te588nNzeWJJ57gPe95D4cPH15Tz21oaIivf/3rnD17lvb29i23n73jlCxVk7377rvZuXMnQggmJyd59dVXZeamy0hLS2Pfvn1UV1evcV85deoUJ0+eZGFhQbqwJBEhBE6nU6vVEIvF5OnqLUAIwf33309VVRVNTU0UFRWtmXRh2ZozOzvLxMQE/f39KZc50+v1Eo/HeeaZZ8jPz6e1tRWbzbamDs5qRkdHGR8fJ5FIkJ6ezpEjR8jLy1vjlqGeNG61RWwrYLVacblc1NfXU1ZWhhCCpaUlFhYWmJiYkBbAJOFyuaiuriY9PV2L6ejt7WVsbEweZl2Dubk5uru7iUQi2Gw2LS47FAqtsbKqlhuv18uFCxc0i3h+fj5+vx+bzUYwGNz2iZxUVzV1Lv3Rj37E8ePHtTCM8vJyLZa1qamJnJwc3G73dSleagbc7OxsLe5VciVGo5EdO3bQ0NDA/fffT1lZmZZBOJFIMD8/z/DwMO3t7VvWYHDHKVnZ2dns2LGDAwcOsHPnTkKhEBMTE7z++utSyboMk8nE7t27KSws1J5TlaxXXnklKT6rVysMqX7WVhz0twqdTkdGRoaW9SYajRIOh+Xm6jpZbxyp42v1azqdjgcffJD9+/dTUlKiKbWrr1EtWaoLcaopuouLiywuLjI6Oordbqe8vByXy4XL5Vr3+vb2ds0LQC0OW19fT2lpqSZXNc5AciVWq5XMzExqa2u1rHeqkqUmHpDcPNnZ2VrR3EgkoilZ4+PjKbXWbIT5+Xnm5+fp6+u75rWqctHe3k5dXR2KopCbm8vS0hI2m42FhYVtr2TB8h5GnUt/8pOfaM+7XC6am5vR6/VaUoy6ujpcLtcVHgDquFxvr6SmL7/8kFCyjMlkorm5mT179nD48GHMZvMab5Xp6WlGRkbo6OjYsuPxjlGy9Ho9VquVu+++m4985COUlJQQi8VobW3l7NmzdHR0bPvMYclACIHZbMZutxONRtfdNCmKosWvqMVd1ax56mRgMBgwm83U1dVRW1t7xfvj8Thnz57l7NmzRKPRlNjkqiUF1FiB0dFRTp06JWu2vAOrlSh18iwsLGTv3r3AcqzQxz72MVwul5YxEKCqqoqMjAzS0tIwGAxXnAT6/X7+/u//np6enpQYe++EWo9keHj4qvEDq60AiUSCyclJPB6P9pwQgtzcXIqKihgbG5PK1mUUFxdTVVWlbbgAhoeHOXXqlJbGWXLzFBQUsGfPHqxWK4uLizz33HN0dHRIBesWMD8/zxtvvEFubq52uO3xeHjf+95HW1sbL7/8csrKXU0oou6NKioqCIVC2hyQSCTo6emhp6eH06dPEwgEuO+++ygsLFyT3VFydQoLCyksLOQTn/gEJSUlWK1WbZ1XE4j853/+J21tbUQikS27Jt0xSpbZbMbj8VBZWUl9fb12kjUwMMDQ0BBerzdlb/gbJScnh6KiIux2+7ob0EQiwdzcnOYepKZ1zszM1AIO1YKnDQ0NV2TeUbPreL1e+vr68Hq9W/aUIVkYDAYsFgs5OTlkZmYCEAgEmJ6e3vbf/WYIBoMEAoE19252dramuGdkZHDffffhdrtxOp3aNepku974DYfDeL1ezp8/v+3rYl0P8Xj8hvz91dTFatyFqjQYjUZMJtOGkpRsZ9S6Yi6XSzuEUt1V+/r6ZErxJKDX6zGbzWRmZpKbm4vBYCAcDtPX18fo6Ohmd29bsrS0xMTEhFYfr7y8nIyMDGpqaggEAhgMhjUZ9FIJtRYWLN//IyMjZGdnE4/HNetVKBRifn6e/v5+vF4vzc3NV3gSSO+Aq+N0OsnPz2fHjh3k5ORoc6uiKMzPzzM2Nsb58+fp6+vb0jK8I5QsnU5HfX09X/rSl6isrMTj8Wg1nr7zne8wMDAgFazrRKfT8Wd/9mdEIpGrTo6hUIg33niDhYUFvF4v6enppKenc/DgwTXpsXU6HSaT6YqMZepvobod/fznP9/2rpwVFRVUVVWRnZ2tZSCSvDOKovDrX/+aUChEQUGBlv3yyJEjHD58GEVRtA2smtJVRR27l9/3iUSCN998k7a2NplldIOsVqJW/z02Nsbg4KDMkrcOVqsVu92OEIJoNIrf7+f06dM89dRT0lUwCeTm5vKud72LXbt24XK58Pl8+P1+enp6GBsb2+zubVsUReHcuXMsLCxQU1PD3r17+fjHP05RURHHjx9nZmYm5S21iqLQ2dmpxbKp6drr6uqorKykpKQEv9/PgQMHroiJ9Xq9zM7OpqSi+k4IIWhoaGDXrl1kZGRgNpuB5Tj3paUl/v3f/523336bEydObHkPti2vZAkhsFgsWgpR1ed1cHCQS5cuMTo6qp0oSNYSj8eZnZ3VTllVVlsE1iMSiVBTU4Pf78fv92M2m7FYLBQXF5OZmXnFxlZRFBYWFrSim+qEMT8//47K3HZBCIHb7cbj8ZCWlkYikSAYDOL1ejWLoGR9xsfHcTqdhEIhLBaLZiG1Wq1XxPSt9m1XiUQixGIxwuEwk5OTTE5OcvLkSXp7e6Xck4yaklxyJU6nk+zsbK1YayAQwOfz4fV6pcySgM1m0060dTodXq+X6elplpaWSCQSWm0dnU6XEmvO7cTn8zE6Okpvby9ZWVlUVlaSk5NDXV0dXV1dLC4upry8fT6fVkNMr9eTmZmppXcvKCggFAphs9k0d211fujp6aG7u1uuVasQQmAwGCgtLaW6unrNIf7i4iIzMzP09fXR39+v1b/cymx5JUuv1+N2uyksLKSyshK9Xk8ikeDYsWP8+te/ZmBgQKbGvQrRaJTW1lYikQgHDx687vepqYhXb3BXb2wTicSaDXAikaCtrY3h4WG6uro097ihoSEGBgZSIjVpbW0te/bswWw2Ew6H6e3tpbOzk7a2tpRfgK6Goih0dHTg9/uZnZ3FbDZr1qzrIZFI4PP5WFxcZHh4mF/84hf84he/YG5ujnA4LC1ZG2Q9xVZ6ClwdIQRVVVU0NDRgNBqJRCJMT0/j9XoJBoNb2pXlTsHtdvOBD3wAj8dDIpHg0qVLWiyGTqfDYrFgsVgwm81MT0/LPUESmZ+fZ2FhgV/+8peMjIzwuc99TouVefrppxkcHEx5xVbNbPfrX/+auro6Dh06pO2ZVmdohd/UMezq6uInP/kJra2tW94aczsxGo1YrVYOHz7Mu9/97jVxxP39/Zw+fZozZ87cMeWatrSSlZ6eTmZmJh/72Mdobm5Gp9PR29tLd3c3Z8+epaenZ8trsZtJMBjk6NGjtLe34/P51tQRUklLSyMjI4Pc3Fzcbvea13w+H729vYRCIW3RisVi9PT0aP7GKqOjo9opgzrZer1ezcK1XdHpdBiNRoqKiigvL8dgMGgFcKempjSFVLI+fr+fyclJjh49SlFRERUVFRQXF6/JiHk1wuEwJ06cYGhoiFOnTtHV1cXs7KyWkljKfWOoJ4mr44tisVhKb6Kuhuoa5HA4yMzM1GoRxmIxYrGYFqMhuXkuP+jT6/XU1NRgNptpbGwkEokQDod56aWXZJxWklETOeh0OqamprDZbNTV1XHq1CmysrKYmZlJidpZVyMej+P3+3n55ZeZmprSirnbbDbMZjM6nU6zbPf19XH69GlOnz7N0NAQS0tLco5g+f42Go3cfffd3HPPPVq6dlj2WJmfn+fs2bM8//zzd1TNyy2tZNlsNvLy8vjwhz9MUVERQgh6enp48cUXaW1tZWBgYLO7uKUJhUL84he/oLS0FJ1Ot24WNpvNRnFxMY2NjVcEZc7Pz9PS0sL8/Lzmdx0KhXj11VeZmZlhZGTkdn2VLYuawrWgoIDS0lL0ej1+v5+LFy8yNTUlJ89rEAgEiEaj/OIXv6C4uJgDBw4ArMlsp7LaciqE0JSs8+fP89xzz0lZJxGDwYDBsLw8xONxIpGIlO86qNnF7HY7GRkZmpKlZm6Viumtw2g00tDQQH5+Po888gjT09NMTExw5swZqWTdAvr6+ohEIkxOTmKxWKipqaGkpITs7GwWFxelkuX38+qrrzI/P09zczOFhYV4PB5t3zU/P8/Q0BCvv/46x48f5/XXX9/sbm8p1Bj/u+66i09/+tNr0uGrSVjOnj3LCy+8cEe5V25ZJUsIwd69e2lsbKSgoECLI5qamuLChQsp4YKWLCYnJ3n66afXfU1Nxe5wOLT6TipLS0vMzs4SjUY1F0BFUWSc0SrU7EDBYJD5+Xmmp6e5cOECR48eZWpqarO7d0cQjUbp6upicHCQ8+fPay4Xe/fupaCggIaGBgKBAJ2dnVy6dImenh5g2ZL19ttvS2U2yZjNZu677z5KS0sJBAKcP3+ec+fOyUOVdVAzqY6OjtLX10dJSQnhcJjx8XG5Rt0ihBDs2bOHuro6Dh8+zOLi4hrrgEyEcWsIBoOMjIzwla98hbvvvpu/+Iu/oL6+nieeeILvfve7KT/eE4kECwsLnD17li996UvU19dTU1NDeXk5Op2OZ555hqmpKYaGhmQegXVwOBzU1dVRXl5OVlbWGjfBoaEh/uM//oNTp07dcSWBtqSSpVoH1NojqwMGVTc1ucm/fkKhkLT63SLUmmCzs7OMjIzg9Xrp7OxkYGBApm6/TtTEKQsLC0xOThKJRPB6vZhMJnw+Hw6HA5/PR0dHB+fPn+fChQvAsuvqyMiITJGdZHQ6HS6XC7PZzOTkpOaeneqbqKuhKApTU1OMjo5qWe+8Xq+MC0oi0WiU+fl5zWJoNpsxGo0YDAZCoRBDQ0P09vbS1dUl54NbRCwWw+/309LSoiUryszMpK6uDqfTicFgSPnwDTW1uxoXHA6HtVCNEydOMDc3J+fRq2CxWCgrKyM7O5u0tDQALYnY1NQUra2tTE5O3lEKFmxRJSsvL4+ioiIefvhh9u7di8Vi2ewuSSTrkkgkiEQi/OM//qNWhDAajRIKhaR1ZYMMDw8zPj5OS0sLBoMBk8mkpceNRCJrDlhk7FXyicVidHd3c/78eV544QXNeiiVhitR469++ctf0tLSAiy7DnV1dTEzM7PJvds+9Pf38w//8A8cPnyYu+++m0uXLjE2NsYvf/lLLYtrMBgkFArJRCO3kEQiwfT0tJaAIC8vj3vvvZejR48yMzPD8PCwlP8Kvb29DA4OrskoeKcpCLeT4uJiPvnJT1JaWqo9FwwG+elPf6pZqe9E48qWVLLUWCyXy4XD4UCn05FIJFhaWmJpaUkGE0u2HAsLC5vdhW1DPB4nHo+ntI//ZhKJRDh9+jThcJju7m4mJiakdeAaqDGrp06dApZLE0iXoOShemM4HA4SiQSDg4NMT08zMDCA3+8nHA6TSCTkJvY2kEgkmJ+f58SJE+zbtw+32016erq2V5NK1jLRaFQ7cJVcG4PBgMvl0mqJRaNRgsEgHR0d9PX13bEJQrakkuV2u6mtrcXlcmlmw6WlJcbHx/F6vYRCITmZSiQSyS1gcXGRr33ta4BM3X69+Hw+fD4f3/zmN7XnpOySh9/vp729nYsXL/LjH/9YlhbYZHp7e/nqV7/KZz7zGcrLy3E4HOTm5nLp0qU70tog2XyMRiN2u13b86tugseOHWNoaOiOvde3pJIFaFlFEokEs7Oz9Pb2cvToUc6cOcPMzIyMd5FIJJJbxJ26oG02Um63lssLlEs2h3g8TigU4vXXX8fv93PmzBkmJiZSPiZLsnHm5+d56623aGpqora2lvPnz9PR0cHs7OwdXUdsyypZata2SCTCxMQEra2tfOtb39JcAyQSiUQikUgktxc1RvZXv/oVv/rVrza7O5JtgNfr5a233sLpdFJVVUVrayunT5/WkojcqYhknwoJIW66QafTidPppKCgAIvFQjAYxOv10t3dfVtqjyiKIq591e0lGXLdbLaaXKVMk4+U6a1ByjX5SJkmHynTW4OUa/KRMk0+NyNTs9lMTk4Obrcbl8vF8PAwi4uLTE5O3tY4v2TLdEsqWZvNVhu4IOV6K5AyTT5SprcGKdfkI2WafKRMbw1SrslHyjT5SJleSdKVLIlEIpFIJBKJRCJJZXSb3QGJRCKRSCQSiUQi2U5IJUsikUgkEolEIpFIkohUsiQSiUQikUgkEokkiUglSyKRSCQSiUQikUiSiFSyJBKJRCKRSCQSiSSJSCVLIpFIJBKJRCKRSJLI/w98Jh6LsoeTogAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x864 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_pics  = 10\n",
    "fix, ax = plt.subplots(1, num_pics, figsize=(12,12))\n",
    "plt.tight_layout()\n",
    "for i in range(num_pics):\n",
    "    image[0,...] = test_data[i]\n",
    "    job_id = dpu.execute_async(input_data, output_data)\n",
    "    dpu.wait(job_id)\n",
    "    temp = [j.reshape(1, outputSize) for j in output_data]\n",
    "    softmax = calculate_softmax(temp[0][0])\n",
    "    prediction = softmax.argmax()\n",
    "\n",
    "    ax[i].set_title('Prediction: {}'.format(prediction))\n",
    "    ax[i].axis('off')\n",
    "    ax[i].imshow(test_data[i,:,:,0], 'gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also evaluate on the entire test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classifying 10000 digit pictures ...\n",
      "Overall accuracy: 0.9871\n",
      "  Execution time: 3.6281s\n",
      "      Throughput: 2756.2394FPS\n"
     ]
    }
   ],
   "source": [
    "total = test_data.shape[0]\n",
    "predictions = np.empty_like(test_label)\n",
    "print(\"Classifying {} digit pictures ...\".format(total))\n",
    "\n",
    "start = time()\n",
    "for i in range(total):\n",
    "    image[0,...] = test_data[i]\n",
    "    job_id = dpu.execute_async(input_data, output_data)\n",
    "    dpu.wait(job_id)\n",
    "    temp = [j.reshape(1, outputSize) for j in output_data]\n",
    "    softmax = calculate_softmax(temp[0][0])\n",
    "    predictions[i] = softmax.argmax()\n",
    "\n",
    "stop = time()\n",
    "correct = np.sum(predictions==test_label)\n",
    "execution_time = stop-start\n",
    "print(\"Overall accuracy: {}\".format(correct/total))\n",
    "print(\"  Execution time: {:.4f}s\".format(execution_time))\n",
    "print(\"      Throughput: {:.4f}FPS\".format(total/execution_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Clean up\n",
    "\n",
    "We will need to remove references to `vart.Runner` and let Python garbage-collect\n",
    "the unused graph objects. This will make sure we can run other notebooks without\n",
    "any issue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "del overlay\n",
    "del dpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "Copyright (C) 2021 Xilinx, Inc\n",
    "\n",
    "SPDX-License-Identifier: Apache-2.0 License\n",
    "\n",
    "----\n",
    "\n",
    "----"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}