Yuri_Di_Biasi_PPO.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Proximal Policy Optimization in a Custom Pygame Shooting Game"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Yuri Di Biasi 0001043300"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gym\n",
    "import numpy as np\n",
    "import gym_game\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import tensorflow as tf\n",
    "from keras.layers import Input, Conv2D, Flatten, Dense\n",
    "from keras.optimizers import Adam\n",
    "from keras import Model\n",
    "from keras import backend as K\n",
    "from scipy.stats import entropy"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENVIRONMENT = \"Pygame-v0\"\n",
    "env         = gym.make(ENVIRONMENT)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Box(0, 255, (300, 300, 3), uint8)\n"
     ]
    }
   ],
   "source": [
    "print(env.observation_space)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discrete(12)\n"
     ]
    }
   ],
   "source": [
    "print(env.action_space)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at the observation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13964141210>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAGiCAYAAABd6zmYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABHDElEQVR4nO3deXRc5Z3n/3ftqpJUKu0l2fK+rxgMjjBrcGzTDglLTwfa3SFMGg7E9OlAQifOTCCk+7Qz6TOZnM4Q0p2ehvCbhCRkmpAQcEJsbIMtGzA2XmSMLcuWLKu0q6qkUu3P7w/hAsU72NaV/Hmd85zjuvepqu9TuuVP1b1P3WszxhhEREQsyD7cBYiIiJyKQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELGvYQuqJJ55gwoQJ5OXlsWjRIt54443hKkVERCxqWELqF7/4BQ8//DCPPfYYb7/9NvPnz2fZsmW0t7cPRzkiImJRtuE4weyiRYu48sor+d//+38DkM1mqamp4W//9m/5+te/frHLERERi3Je7CdMJpNs376d1atX55bZ7XaWLFlCXV3dSe+TSCRIJBK529lslu7ubkpLS7HZbBe8ZhEROb+MMUSjUaqrq7HbT71T76KHVGdnJ5lMhsrKyiHLKysreffdd096nzVr1vD4449fjPJEROQiam5uZuzYsadcPyJm961evZpwOJxrTU1Nw12SiIicB4WFhaddf9G/SZWVleFwOGhraxuyvK2tjWAweNL7eDwePB7PxShPREQuojMdsrno36TcbjdXXHEF69atyy3LZrOsW7eO2trai12OiIhY2EX/JgXw8MMPc/fdd7Nw4UKuuuoqvv/979Pf388999wzHOWIiIhFDUtIfe5zn6Ojo4NHH32UUCjEZZddxtq1a0+YTCEiIpe2Yfmd1McViUQoKioa7jJERORjCofD+P3+U64flm9SF5LNZuPKK69kwoQJQ5ZnMhkSiQTd3d3s27ePnp4e8vLymDFjBuPHj6e9vf2Uv9OyotmzZ1NVVUVnZyc7d+4c7nLkIhrN23hhYSETJkxgzJgx+Hw+0uk0vb29tLS00NDQMNzlyXAwI1A4HDbASZvD4TDPPPPMCfdJJBLm6NGj5qWXXjJLly41drvdVFVVme9///umq6vL/OIXvzA2m+2Uj2uF5nQ6TUFBgSkpKTHf+973zI4dO8z3vve9Ya9L7eK20bqNO51Oc9lll5kf/OAHZteuXaahocHs37/frF271nzta18z+fn5w16j2vlv4XD4tP/fj7pvUseZP9mL6XK5qK6upqKigjlz5jB79uyzOluF3W4nm81eqDLPmtvtZsqUKdxyyy0sXryYT33qU8Tj8eEuS4bRaNvGZ8yYwe23387f/M3f8Oyzz/Lf/tt/46abbuJzn/scX/3qV+nq6uInP/kJqVRquEuVi2hE/Jj3o0in0xw8eJCysjLKysqYNm0a3/jGN+js7CQYDDJ58uST/vbKZrNxyy238OMf/5j33nuPjo4ODh48yFNPPcWtt95KZWUlv/rVr2hra2PVqlVMmjQJm81GUVERTU1NvPHGG9x5552UlpZy2223sWnTJpqamti7dy//9//+X2677bbcc40fP55vfvObNDY28m//9m/867/+Ky+//DKf//znT6jL7XYzYcIEPvOZzxAMBjHGnPCfFMDnP/95XnjhBXbt2sU999yj00aNYqNtGy8vL6esrIyuri5+9atf0d3dzebNm9m+fTsej4fFixfjcDi0jV9iRvU3qUwmQ3d3NwAOh4OBgQGy2Sx2ux273X7Sjfvyyy9nxYoVzJkzh3Xr1vHyyy9zyy23cNlll5HJZOjs7OTtt9/m05/+NJdffjnvvvsubW1tzJ8/n8rKSjZv3gzA9ddfz9e//nWOHTvGj3/8Y+bOncu8efN46KGHaGlpYfv27TgcDnw+H9XV1dx00014PB56e3vJy8s7oa6BgQHeeustHnnkEQCeeeaZk/5SOy8vD7/fTyAQIC8vD5vNdtIw4ylgJE2m/C3w5HAXYS2jbRvftWsX7e3t/PKXv2T37t0kk0mCwSAVFRVkMhlCoRDZbFbb+CVm1IaUw+GguLiYVatWAVBQUMDChQvxer2Ew2G6u7tPuosjFouxd+9eIpEIhw8fpqenB5fLRTAYJBKJUF5ezubNm0kmk8ycOZOamhreeecdFixYgMPhyL0xFy1axJw5c3juuef4/e9/T2trK8XFxfzFX/wFN9xwA7t27RpSayAQ4A9/+AMHDx6kvr7+hLoymQzt7e25y5nEYjEKCgpO6Ld3716ef/55ysvL2bNnz8nfvAA3AuM/wgs7XHTM/ASjbRvv6uqiq6srd3vy5Ml88pOfZPr06TQ1NbF582YymYy28UvMqA6pysrK3OVAjDGk02kikQg7d+4kFApRXFx8wv0aGhpIJBJMnTqVuXPn8td//ddcffXVlJSU4Ha78Xg87Nixg97eXiZOnMi4ceMoLi5m4cKFJBIJdu7cybhx45gzZw7GGJqamggGg8TjcTo7O3G73Vx77bX88Ic/zD1nJpOhq6uLxx9/nCNHjnysY02bN2/OfdKV0W20buMOh4Py8nJWrFjBsmXLyM/P59VXX2XdunVkMhlt45eYURtSxz9dZTIZYHD/fSgUYtu2bXzrW98acumPD5syZQpf+9rXuP322/H5fBhjSCaTuN3u3ONGIhG2bNnCTTfdxMSJE5kzZw5XX3019fX1HD58mJqaGoqLi/F6vfz0pz8d8kkvlUoRDAaHnJo+Ho/z5ptv0tDQoIPCctZG4zbucDgoKSnhH//xH/nzP/9zfv/73/OLX/yC3/zmN6TT6Y/9msnIM2pDKp1Oc+jQIS677LLcMmMM2WyWdDp9yl0Et912G5dddhldXV38x3/8B//2b//Gtddey/333z+k329+8xvmz5/PwoULKS0tpaamhn//938nHA4TjUbp7u6mv7+fa665hkOHDg15Y2azWZLJJGVlZbm69AaUczUat/GFCxfy+OOPU1tby7333strr71GW1ubJWYfyvAYtbP7js9+i8fjuZZIJEilUqfeh83gRRmz2SzhcJhdu3bR1tbGVVddRVlZGTabLXcgetOmTXR0dDB+/HiuvfZajDH84Q9/IBqN8u677/LGG2/gcDhYsWIFBQUFzJs3j/vvv5//+I//4DOf+UzuU+v5duutt/LEE0/wwgsvcPvtt2vm0yg22rbxSZMmcfXVV/OJT3yC/Px8/uEf/oHf/va3bN68mfXr1/Ptb38bt9utbfwSM2q/SX1U27dvZ+7cucyePZsbb7wRl8tFZ2cnhw4doqenB6/XC0B7ezsNDQ1MmTKF0tJSmpqaaGxsJJlMcvToUTZv3szLL7/M4sWLKS4uxu/3U1RURCqV4ujRoxfsk2FFRQWzZs1i8uTJrF279tQzn+SSZdVtfOzYsUydOjV3yrNp06bl1vX19dHW1obdbtc2fokZdSFljOHAgQNs3bqVlpaW0/ZNJpM0NjbyxhtvsH//fgDeeustqqurSaVSjBkzhtmzZ7Nu3Tq6urpwOp2Ew+Hcfffu3cvMmTPxer3s3LmTcDhMNpult7eXHTt28PTTT7Ny5UquuOIKYrEYhw8fzv3uI51OE4/HOXLkCNu2bePAgQPn9EbbuXMnxcXFHD58eMjyUCjEnj176O3tJRQKnfoxtwOnf3mspXG4C7CO0bqNG2Po7u5my5YtJ6wbGBjgvffeI5vNahu/1Jz2fBQWdbrTIl3MtnTpUvPzn//cHDlyxNx3333GbrcPe01qauezaRtXu9DtTKdFGrXHpC40r9fL7Nmzqa6uprOzkxdeeEEHd2VU0TYuVjDqdvddDDabjV/+8pdMmjSJQ4cO8fOf/5zOzs7hLkvkvNE2LlahkPoIjDE888wzeL1e2tvbOXjwYO63KiKjgbZxsQpd9FBERIbNJXfRww87fr4wGJzCmkwmR+xUVZvNhtvtJj8/n3A4nPtU63A48Hg8eDwebDYbyWQy91sZGf20jctoN2onTrhcLqZMmUJLSwsdHR2sXLmS0tLS4S7rI6uoqODuu+9m3759TJ06FYfDAcCsWbP4xje+we7duzlw4ADf//73ueGGG4a3WLkotI3LpWDUfpMaM2YMy5Yty/3qvba2liNHjrBu3bphrmyom2++mS996UsEg0EWL15MMpnMrbPZbIwfP557772XxYsXM336dJzOD/5kc+fO5a677uLOO+/kq1/9Ktlslr//+7/H7XYTjUbZunXrcAxJLhJt49rGLwWjNqSOvyEOHTpEOp1m6tSpTJ8+PfcGrq2tZeXKlaTTaXp7e5k6dSpdXV388Y9/ZMqUKblPcr/73e9Yu3YtEyZM4Prrr+e6667jnXfeYf78+USjUerr62lpaeHGG2+ksLCQ9evXs3nzZhoaGnA4HNx4440sW7aM8vJy0uk0R44c4Wc/+xmNjY1ks1k8Hg8lJSWUl5ef8vQu/f39dHR0UF5eTlVVVW751KlTmTx5MolEgg0bNmCM4b777qOqqorZs2frDTzKaRvXNn4pGJUh5fF4qKqqYtasWezYsYNkMsncuXOZOHEifr+faDTK2LFjWbZsGeFwmJ07dwKDJ970er309fWRn5/PrFmzKCwsZNOmTZSVlXH55Zdzyy23EIvFyGazXHvttUydOpVdu3aRyWSYN28efr+f7u5ujh49SjAYZOXKlZSVldHf34/L5WLSpEkkk0l+8IMfEIvFaGpq4ne/+x3FxcUnzJ4yxhCNRnnjjTdyZ6n+8Bu4oqKCQCBANBolFAoBg5NKKioqqK6uvmivt1x82sa1jV8qRuUxqdLSUiZOnMjYsWPZuHEjGzZsAGDChAlMnDhxSN+Ojg42bNjA6tWrKSoq4uqrr2bHjh08+eST7N69m9tvv52CgoLcZQdSqRT19fV8/etfp62tjenTp1NTU8M//MM/UFdXx9VXX82MGTNy19/5/Oc/z5YtW/jOd77Dj3/8Y5LJJA8//DDFxcU4HA7efvtt/umf/olHHnnkpGeJPv7J9/XXX6exsXHIuvz8fFwuFwMDA7lliUQCu91+0gsiyuihbVzb+KViVIbU5ZdfnrtA29q1a1m7di0dHR1MmDCBG2+8cUjfI0eO8NZbb9He3k40GmXv3r00NzfT0dFBc3MzLpcLr9ebO4gbj8d57bXXCIVChMNhmpubqa+vp6enh8OHD+d2bwSDQa6//noymQy/+tWv2LVrF/v27eN3v/sd5eXlzJgx47y9yT48m8u8f2ZsGd20jWsbv1SMypC66qqrqK2tpby8nB07dlBfX891113H5MmT+dSnPnVRarDb7TidTowxZDIZjDG5ZcfXf1y9vb0MDAwM+Y+goKCAbDY75DLcMvpoG9c2fqkYdcekxo8fz9SpU+np6eEnP/kJr776KgALFixgyZIlTJw4kRkzZlzwa9D09fXx7rvv4nQ6+fSnP826deuorq7mxhtvpLu7m4aGBmKxGNdccw133nkn5eXl/NVf/dU5/fbj4MGDtLS0MHv2bBYsWIDL5SIYDNLa2sqePXsu4OhkOGkb1zZ+KRl1ITV//nzGjBlDT08Pv//976mrqwOgs7OTcePGsWDBAq666qpTXlr7fIlEIrzxxhu89tprLF68mHHjxuHz+SgsLOTZZ5+ls7OTdDpNWVkZCxYsYMyYMef8yfPgwYO8+eabTJkyhS984QvA4DWAtm7dSn19/QUYlViBtnFt45eSURdSpaWltLW10dzczHvvvZdb3tzczJ49e1iwYAGlpaXU19dTX19PU1MT8XgcYwx79uyhsbGRaDRKIpGgtbWVHTt2kEgk6Ovro6mpiT179hCLxQA4dOgQkUiE1tZWANra2tizZ09uX/6OHTt45plnWLp0KZdffjmpVIo9e/bwwx/+kL6+PowxhMNh3nvvPXp7e0+7n72/v5+jR4+ya9cu4vE4AC0tLbz22mv4/X5uueUWEokEf/zjH1m/fv0J15mS0UPbuLbxS4nO3SciIsPmTOfuG5UTJ0REZHRQSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFjWeQ+pb33rW9hstiFtxowZufXxeJxVq1ZRWlpKQUEBd9xxB21tbee7DBERGQUuyDep2bNn09rammuvv/56bt1DDz3Eb3/7W5577jk2btzIsWPHuP322y9EGSIiMsI5L8iDOp0Eg8ETlofDYf7P//k//OxnP+OTn/wkAE899RQzZ85k69atfOITn7gQ5YiIyAh1Qb5JHThwgOrqaiZNmsTKlStpamoCYPv27aRSKZYsWZLrO2PGDMaNG0ddXd0pHy+RSBCJRIY0EREZ/c57SC1atIinn36atWvX8uSTT9LY2Mi1115LNBolFArhdrsJBAJD7lNZWUkoFDrlY65Zs4aioqJcq6mpOd9li4iIBZ333X0333xz7t/z5s1j0aJFjB8/nl/+8pd4vd6P9JirV6/m4Ycfzt2ORCIKKhGRS8AFn4IeCASYNm0aBw8eJBgMkkwm6e3tHdKnra3tpMewjvN4PPj9/iFNRERGvwseUn19fTQ0NFBVVcUVV1yBy+Vi3bp1ufX79++nqamJ2traC12KiIiMMOd9d99Xv/pVbrnlFsaPH8+xY8d47LHHcDgc3HXXXRQVFfHFL36Rhx9+mJKSEvx+P3/7t39LbW2tZvaJiMgJzntIHT16lLvuuouuri7Ky8u55ppr2Lp1K+Xl5QD8r//1v7Db7dxxxx0kEgmWLVvGD3/4w/NdhoiIjAI2Y4wZ7iLOVSQSoaioaLjLEBGRjykcDp92noHO3SciIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREcs655DatGkTt9xyC9XV1dhsNn79618PWW+M4dFHH6Wqqgqv18uSJUs4cODAkD7d3d2sXLkSv99PIBDgi1/8In19fR9rICIiMvqcc0j19/czf/58nnjiiZOu/+53v8u//Mu/8KMf/Yht27aRn5/PsmXLiMfjuT4rV65k7969vPLKK7z44ots2rSJ++6776OPQkRERifzMQDm+eefz93OZrMmGAyaf/7nf84t6+3tNR6Pxzz77LPGGGPq6+sNYN58881cn5dfftnYbDbT0tJyVs8bDocNoKampqY2wls4HD7t//fn9ZhUY2MjoVCIJUuW5JYVFRWxaNEi6urqAKirqyMQCLBw4cJcnyVLlmC329m2bdtJHzeRSBCJRIY0EREZ/c5rSIVCIQAqKyuHLK+srMytC4VCVFRUDFnvdDopKSnJ9flTa9asoaioKNdqamrOZ9kiImJRI2J23+rVqwmHw7nW3Nw83CWJiMhFcF5DKhgMAtDW1jZkeVtbW25dMBikvb19yPp0Ok13d3euz5/yeDz4/f4hTURERr/zGlITJ04kGAyybt263LJIJMK2bduora0FoLa2lt7eXrZv357rs379erLZLIsWLTqf5YiIyEh3DpP5jDHGRKNRs2PHDrNjxw4DmO9973tmx44d5siRI8YYY77zne+YQCBgXnjhBbNr1y7z2c9+1kycONEMDAzkHmP58uVmwYIFZtu2beb11183U6dONXfddddZ16DZfWpqamqjo51pdt85h9Srr7560ie6++67jTGD09C/+c1vmsrKSuPxeMxNN91k9u/fP+Qxurq6zF133WUKCgqM3+8399xzj4lGowopNTU1tUusnSmkbMYYwwgTiUQoKioa7jJERORjCofDp51nMCJm94mIyKVJISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREcs655DatGkTt9xyC9XV1dhsNn79618PWf+FL3wBm802pC1fvnxIn+7ublauXInf7ycQCPDFL36Rvr6+jzUQEREZfc45pPr7+5k/fz5PPPHEKfssX76c1tbWXHv22WeHrF+5ciV79+7llVde4cUXX2TTpk3cd9995169iIiMbuZjAMzzzz8/ZNndd99tPvvZz57yPvX19QYwb775Zm7Zyy+/bGw2m2lpaTmr5w2HwwZQU1NTUxvhLRwOn/b/+wtyTGrDhg1UVFQwffp0HnjgAbq6unLr6urqCAQCLFy4MLdsyZIl2O12tm3bdtLHSyQSRCKRIU1EREa/8x5Sy5cv55lnnmHdunX8j//xP9i4cSM333wzmUwGgFAoREVFxZD7OJ1OSkpKCIVCJ33MNWvWUFRUlGs1NTXnu2wREbEg5/l+wDvvvDP377lz5zJv3jwmT57Mhg0buOmmmz7SY65evZqHH344dzsSiSioREQuARd8CvqkSZMoKyvj4MGDAASDQdrb24f0SafTdHd3EwwGT/oYHo8Hv98/pImIyOh3wUPq6NGjdHV1UVVVBUBtbS29vb1s374912f9+vVks1kWLVp0ocsREZER5Jx39/X19eW+FQE0Njayc+dOSkpKKCkp4fHHH+eOO+4gGAzS0NDA3//93zNlyhSWLVsGwMyZM1m+fDn33nsvP/rRj0ilUjz44IPceeedVFdXn7+RiYjIyHdWc74/5NVXXz3pNMK7777bxGIxs3TpUlNeXm5cLpcZP368uffee00oFBryGF1dXeauu+4yBQUFxu/3m3vuucdEo9GzrkFT0NXU1NRGRzvTFHSbMcYwwkQiEYqKioa7DBER+ZjC4fBp5xno3H0iImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbGscwqpNWvWcOWVV1JYWEhFRQW33nor+/fvH9InHo+zatUqSktLKSgo4I477qCtrW1In6amJlasWIHP56OiooJHHnmEdDr98UcjIiKjyjmF1MaNG1m1ahVbt27llVdeIZVKsXTpUvr7+3N9HnroIX7729/y3HPPsXHjRo4dO8btt9+eW5/JZFixYgXJZJItW7bwk5/8hKeffppHH330/I1KRERGB/MxtLe3G8Bs3LjRGGNMb2+vcblc5rnnnsv12bdvnwFMXV2dMcaYl156ydjtdhMKhXJ9nnzySeP3+00ikTir5w2HwwZQU1NTUxvhLRwOn/b/+491TCocDgNQUlICwPbt20mlUixZsiTXZ8aMGYwbN466ujoA6urqmDt3LpWVlbk+y5YtIxKJsHfv3o9TjoiIjDLOj3rHbDbLl7/8ZRYvXsycOXMACIVCuN1uAoHAkL6VlZWEQqFcnw8H1PH1x9edTCKRIJFI5G5HIpGPWraIiIwgH/mb1KpVq9izZw8///nPz2c9J7VmzRqKiopyraam5oI/p4iIDL+PFFIPPvggL774Iq+++ipjx47NLQ8GgySTSXp7e4f0b2trIxgM5vr86Wy/47eP9/lTq1evJhwO51pzc/NHKVtEREaac5kokc1mzapVq0x1dbV57733Tlh/fOLEr371q9yyd99918CJEyfa2tpyff71X//V+P1+E4/Hz6oOTZxQU1NTGx3tTBMnzimkHnjgAVNUVGQ2bNhgWltbcy0Wi+X63H///WbcuHFm/fr15q233jK1tbWmtrY2tz6dTps5c+aYpUuXmp07d5q1a9ea8vJys3r16rOuQyGlpqamNjraeQ2pUz3JU089leszMDBgvvSlL5ni4mLj8/nMbbfdZlpbW4c8zuHDh83NN99svF6vKSsrM1/5yldMKpVSSKmpqaldYu1MIWV7P3xGlEgkQlFR0XCXISIiH1M4HMbv959yvc7dJyIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWQopERGxLIWUiIhYlkJKREQsSyElIiKWpZASERHLUkiJiIhlKaRERMSyFFIiImJZCikREbEshZSIiFiWQkpERCxLISUiIpalkBIREctSSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZZ1TSK1Zs4Yrr7ySwsJCKioquPXWW9m/f/+QPjfccAM2m21Iu//++4f0aWpqYsWKFfh8PioqKnjkkUdIp9MffzQiIjKqOM+l88aNG1m1ahVXXnkl6XSab3zjGyxdupT6+nry8/Nz/e69916+/e1v5277fL7cvzOZDCtWrCAYDLJlyxZaW1v5/Oc/j8vl4p/+6Z/Ow5BERGTUMB9De3u7AczGjRtzy66//nrzd3/3d6e8z0svvWTsdrsJhUK5ZU8++aTx+/0mkUic1fOGw2EDqKmpqamN8BYOh0/7//3HOiYVDocBKCkpGbL8pz/9KWVlZcyZM4fVq1cTi8Vy6+rq6pg7dy6VlZW5ZcuWLSMSibB3796TPk8ikSASiQxpIiIy+p3T7r4Py2azfPnLX2bx4sXMmTMnt/wv//IvGT9+PNXV1ezatYuvfe1r7N+/n//8z/8EIBQKDQkoIHc7FAqd9LnWrFnD448//lFLFRGREeojh9SqVavYs2cPr7/++pDl9913X+7fc+fOpaqqiptuuomGhgYmT578kZ5r9erVPPzww7nbkUiEmpqaj1a4iIiMGB9pd9+DDz7Iiy++yKuvvsrYsWNP23fRokUAHDx4EIBgMEhbW9uQPsdvB4PBkz6Gx+PB7/cPaSIiMvqdU0gZY3jwwQd5/vnnWb9+PRMnTjzjfXbu3AlAVVUVALW1tezevZv29vZcn1deeQW/38+sWbPOpRwRERnlbMYYc7adv/SlL/Gzn/2MF154genTp+eWFxUV4fV6aWho4Gc/+xl/9md/RmlpKbt27eKhhx5i7NixbNy4ERicgn7ZZZdRXV3Nd7/7XUKhEH/913/N3/zN35z1FPRIJEJRUdE5DnV0qqysYtq0mafts/3tbcT6+y9SRSKXLq/Xy7Tpp/+w3XSkkZ6e7otUkfWFw+HT7h07p5Cy2WwnXf7UU0/xhS98gebmZv7qr/6KPXv20N/fT01NDbfddhv//b//9yFFHDlyhAceeIANGzaQn5/P3XffzXe+8x2czrM7RKaQGlRdXcPE+eOY8okaKkoLTtlvw7PvcHDfIXq6uy5idSKXFr+/iGnVVfzlzAmn7NOedLLlaAv1LUfp7Oy4eMVZ2JlC6pwmTpwpz2pqanLfmE5n/PjxvPTSS+fy1PInAoFipsyfzOTFVVQvymNiTeCUffu7ZmMw7N+VIhrV9H2R883ny2dadZBPTRnHbaXuU/Y7NODGbhuDsdvZEY/T1xe9iFWOTB95dp8MH7vDwaLF11C9xMfYeV7Gjyk5bf/LP1eGJ38eLuOjbsumi1SlyKVj8pRp/PnECj5X7jltv0neJHZbHn5HDRmPl61vbCGbzV6kKkcmnWB2hHG53axYcRsT7vTh1Sx8kRFpSp6H+2qquHrx9Wd9mONSpVdnhDl+VNBmO/Uxws6ePto7+4Ys6+rVCXxFLqTOlJO9/XlDllW6U5S5Mif0tdlsnPzdK39KITVCxfuS5J3kE1h7Z5RMNkv5n0ykKLrS4HXbiETms3fPOxerTJFRb9KkqdxWXcTCIjsV7qEfBpNZO+1J2wnLU5kM/YnkxSxzxFJIjVCpRApHAuIDKaJ9cQryPUT7EnT19uNIOvHbXEP6O4DycX6uXDqbzo52OjrayWZP/IQnImfHbrcTCBSzdMo4JhS4SZsMRxND31MGD167C489i9+RJZqxE0sbEukM8XRqmCofWRRSI4wxkEwmySYhHk7REY+SiGWZPrmChoNdZLJZ+vaneHdX7IQ7lk70sOC/lBPadyUbXn2FeHxgeAYhMgq4XC6mT5/Jf63y8qPWOAcT5oRd8IsKnMzzuuhNu5iTH+fdARc9A0ni6TQJY0grqM7onH4nZRX6nRQsXfZp8vKO7/8e+sZwuT243UP3jScTcVKpBKlUipdf+vXFKVLkElB79XU4nU4KCwMUBYbOtA33dhGNhk96v0QiwRvbNl+MEi3tvP5OSixmfB+uiixup4f+ujwm3rCYSbNmkl9YyIeDa0xpKVtfWssf/99/8uabdcNXr8go9NabdcyaPY/FNy/nuts/y7GuD84mYUyWaDjMofp9vPOb37E834PPZudoOs1ricQwVj1yKKRGgDlzLyMvzztkmcvlhk4nDDjAnQckcXrcePLzySsooK+nh91/WM+i/3Ib3sICXG432WyWpN4YIudVKpUim83i9LjxFRbijsVY95P/j0/c+hkKS0pIZ7O43t/r0ewLkIj10ZPM4HQ6mTlzzpDHSiQTHGo4MBzDsCyFlIXZbDbGjh3H7LmzsDscpJIf7L9OxjPQ7yATd5B2OnC63MTau+hqOEIsP59Yb5ieIy10vHeI/sYjHNi7l/b2k1+vS0Q+np7uLg7u3Uv561tIGzvhI8foPHCYgeJeBvr7GejoIj+/kGPpDP2pNMlsFqfTyYTJE3KP4XK7yGYz9PVFaW/Te/U4HZOyKLvdji8/n08t/TOmzR1PLBYjGo6SSg1OZY2FU5js8VNV2fD6BqecFxT4cTqGfvZobz9Gff0umo40nvF54/G4Zv2JfIjdbsflPvWpjo6rqhrDlKkzmDhx2pDl6XSa/v7B0x+1t7VgMNjtNuwOOxVVg8ew3B43/kAhHo+Ht+t2U7fldeIDA2c8Fd1ocF5PMGsVl0JIFReXct31n2TCjCA2mw2v18tALMmhfc0A+EvzsNkHv1ENRM/fD3W3bn1Nn+JEPqSktIzZs+edt8crCxZT4PeSSWdpPjT4Xpt/1SwK/fkMvB9MO+rqefvtN+iLjv5z+2nixChxtLGNAVcU/6cGjyn1bYJs0gYGXN4CJt5wxxkfw+3xEigtp/1YE2XBsfRHejHGUFhUQkeomYqqGvxVP+Xt19ZSv3fXhR6SiOVNmjSVy65dyqLb7qWjtZmyqhqivd3YbDby/QG6Qi2UV9fQ29VOKhE/4+PteOHf6WrroautF6/dxl8UDh5r3rzzPULlAcZOOPmFXy9lCimLikbDbNmyCXfeTdhsNvKmZAiMc+AoGNyo+2xQNu0KfCVBMDaibb0AxPqjpNMpHA4nvoJCouFewJCX58Pj9RFp6yaVSBDvHSCTHvwG1tvamVuWHIgzZsw4AoHiXC273nlbZ2uWS4LPl8/kKR/srvN6fST6B2h+98DgeyQcJ5NOATZ6HZ2kkgni4QHSqRTxgRiJeAxsNgr9AWJ9ETKZDE6nC19+IQBTr/kMYAiHjtC9ZwtlPh8A13uyHIwlqd99CGNg964dDMRiJ6nw0qOQsiin00lxoIQ8r5tobwyvFwqDLgLvf/LqsEew2fKwOQrIZrOk4oMhkkmmMdks2WyG1EACkx48vpROprCZARLZfga6DwNg8xSBY+jvqVKJOLP9ceaXd2Ow8Zu+qTidQ89eITJaOZxOSgNF/JeyZmzAm31OjibidLW2DumXSfSR7O8EbBRXT8dmd5BJpTDpwTOapwYSZFMZTDZLxqRJxQf3gHgDhdjtdhyuDhw2qMkb7F+QdnCsL8VALEFpRYDCQj99/X1kMjo+rJCyKK83n5mz5lBSWUh/XxyXy0ZRgZcxwQAAe+xRYuEIqXQIMh/M+nPC4LntbRkymTRuTx6k45COk07HMSZDqvcILqebZCJOxuXH4fpgens2m6Xa2ce13mYMNl7sm3JRxy0y3Ow2w01F7dgwHEt6acpkSH5oV14q0Ucm1onpD5FOJfH6y7HZHQA47A4cbh+ZTBpbNoXdZCGdIt43eP94XzcOVx4D4SgOYIxn8L2bBdw2g8vlZOyEIB2tk+nu6SKV1Pn9FFIjRLDcz9iqwiHLYv1R6OvFRJtzy2w22+CMIKcXe8kMALLd+yA98P56O2NrJhGsmsDhQ/sIJzopLP/goHAk4uK1vrFs6h/zoWcacXNrRD4aY0hkbHzxwBXA4Pspv8SFvyiQ6xJ6by8Bn5tZV3+KUOsR9uz+I+b9a0LZPYUEJiwCoPfwVjKJfj58piRjDPnBWdhstiH/+dZ4UlS6U5DQudH/lEJqBDORwwA4HE6qx4wHoLx8DJ2dIVraWgmUVgDQGz5AprIfvGnMwQA9PV309HQDBlsqSde+Vz78qBw+3MCe3Ts/WDLyJoCKfCTRaIQtmz+4uvjkKdOostk4uPUXuWWlpRWAm331OwCDMYZP+jx0ZrK863RS/P77LtrsZOrMy6iuHk9nZwsA++rfoT9UD0DAoUA6Gwopi4pEetm0cR3FlbeQzXxw5c50JsuBxg6u+mI5duyE9sZ4b32USKQXgO7ujsGzSjjcQ0526fa6sAezxB09xA58cK3LtrYQe/fsHPLcyWRSwSSXrA9v+01HDtN6rGXI+qnTZlBSUpa7/Smfh4DdRtf779MPv+8aD71Le3sLfn8AgKzJckepmwUFDoyxsS+WxzRvnON5lclkCPeE2bnjTWL9/RdohCOLQsqi3G4PY8eOw5PnxmaL0xsZoKvHTnGRj5KAD3+JG4fdTrgliclmifUPXuQwXRzBOMDWV0h/6F0AsukEdDnJJmxkY0n27tqRe56+viiRyMlPgClyqUsmEySTQ08l1nTkMB3tbYM3bDBh2nScdjtdmSzpbIyOxrcBSCcHqCJNYSxCy/uPkc1kKHU5GOdxkDHQmUpjAzpTDnrTTmy2FG6Ph/KKIInEERI6jZlCyqry8rxMmjyVPG8e3vwEkfYkrYcM7ukOfHkuYrHBA6rxRApshlReFIfLgaMqTrbfTjbbx0DyPehz0dPTTTo0ON08k8nQ0PDecA5NZETr7u4ccntrUTEO++DeCYfTSSaVoNJhpxyY6HZSaIee2ODs235jiGftRNOD/X32LP0ZO/tiEDJOCovceH15jBlTQ1tbq0IKhZRlGWNIp9Nks4ayKj9dzRGOdQ2QsA+d7dPfbiAvg216mPzSfGx2N4lYkkRRGJOykXrbz84dbxGNRoZpJCKj24d/+J6fn89l86/g2oJC3LYPdv0tyR/8qceLfQMcSzjZYx96mqXf94YZKC5k3MQgmfTgzFzNVxqk0yJZmMPh4M9W3Ia/1ENiIENy4CS/mSgdwD6pn4LSwR8FHp/dl43Yibzu4ncvPq/jSyIXkc1mY/Hi61lR6KPYYR+yzhhDXTxJY+rE93JhIB9/IJ+jjW1s2byRbDZ7Qp/RSOfuG+G8Xh/XXHcDXm8+niCU1A5u9NMnVeB0OsBuwJnF7rCz72CIbHbwz2mykB2w0fjzKHWbX9MZI0QuAp8vn3lz53NvTTX5djvHI8pug1m+wZ+BJIwhbSBlbOyPDX7D+kNvhJZUmkQyzs6337qkdvPp3H0j3MBAjN3v7GTipKnUjKnA7bdTUxWgwO/Gbh/clZBMpmloaCfel8AYcLoduPJc2J2GMUsKmJucR8PeBl2qQ+QCKi4pZfrYcXy6soIihwObzUY8lSKZyWADDmQMM/w2Au9/u8oacOenaU64cdpsdHd3cezY0UsqoM6G/cxdZLiFQsfof/+bkM1mo7jIlwsogEzScHRnP4lYOvdN6njf/El2Jn1iDBNmjaO0tPyi1y5yKSgqCjC9uprasWOY6csbMg09m80SS6XYGk2T+NAePLsNSpyZ3DW0B2IxuruGTsoQhdSIkUqlSA4kyQwYUuksxhiSyTTxgSTxSJqBnS6yAzbcHhdO9+AXZGMMmVSGwvmGyYvGM236zGEehcjoNG78RBbVjKO2wEcmm80dB3Y7nbhdLvqyhtcHkkQyhnjGkMoazPu7/PqzWeKpJOnM+bvkzmii3X0jxL59u+nu7qS27xqM6xhzp1ez663DtB7txSTsQCnRjR7MoiS+sQ48+W4w0PpeBwCmJ394ByAyysVTKbrfP3N5WX4+xhhiySSHBuKs649js9nY2pWlwG4nmAcz/XZ29+fxVFsn23ftfP8sMPKnFFIj1Ktr9+BwOygZU4TT5iG0O8sfX3mJKyJXUbUwgHtilFjvAAVlPvIDPvpiNjrQ5AmRC8XrclHk9RIeGKDr/bNFvJNIsbW9g73v1nPlVbWU+HzY0mna4mm6k1n83jM8qCikRpKenm7erNvK4tJPkOj34Z2WIePNkkkkASfJVJI9O3cRS05kfPc4Mmkntk9kScaSpJP6U4tcSIlMhv5EgqwxbBpIYjDsb2nh4LGjpNODZzvvSyTIA0LpDPXJNK6BFLvqdxPV7NtT0v9cI4jD6cCbl0+mw40TN5m2OPH+NKlYisbGw5hslt7ebiLt5WQqXLhcPpJN/SRthraGLkKtx4Z7CCKjUldXB4dcTvJKSzDYiOTl0x/ro6Ovj75oFJvdTuuxFuqdNXhdTiI2B5G8wV3wnjwvDnsYHZE6OYXUCOHx5FFZWcWcuZfh9RXidLnoC/XS1x+htyeaO2u51+vDk5eH0+HE6y0kddhFPD7A0T2tNDYeHN5BiIxSx1qOYrfZiRcW4fMVUlpWQbotgyfPi8fjIZFI0NDwHj6fj4JCP4FAKZVVY0mnUkybPot0JkNnRzuplK4f9acUUiPE9OmzmDxlOr78QoqKAlRXT+LAgV0cPLCPnTvfyvVbcPmVVFXX4PMVUFRUzJixk3lnx+ZhrFzk0lFQUMQ1195MS0vD4DWjnE58Pl/uQ+Tu3TuZNn0mNeMmM2XKHFqPNWJMFrvdwaGG93RezZPQFPQRqrX1MPGBM5/Kv+XoIX06E7lIUqkELS2HzthvINZHqPXIRaho5NM3qRHGGEMkEmbipBkUl1SQXxDA5XLx5pt1Q/plMhkikTCTJ88mUFyO2+3F+6FPdCJy/kyeMo15865kxqwF5HnyaGioJ5M58fx8M2fNZeGV1zJu3BQcDgeNje/q3JpnMKJDqtjFkEszj2apnmP0tnqwj50MQHtbC3a7g3Q6SXEgQMn7J1XuDx0m4rLjLx9DMhGnra0ZsGGz2yjx+3P9ROT8KSrIJ5NJ0xZqxhhDV1cbGIh0HiPd25p73/kLCxgY6KO19QiZTIburnYAou3N2Po7L6n3pzHQkzpzvxEdUleMK8LlsJHOZOiPD5yw3m6zUej74EesyVSKgQ9dwKwgz4vD4Rhyn2gsRtacePZhnycPl/ODlyvS35c7k77DbqfA6xvS/7zXlO3EEQZPecHg8x9tw+fx4nM6GVcIjgmBwZriLTgjDnwBDwVeH6ZrcB93EWAvNKTLT/yTj6rXSTWppmGoqdCTwEQO09XXRIHXR+D9dW568bpjlLz/vgu44iQ69tPRMXjassD7NTlsPeR7E1R86P05Gl+nD9fkcXv5Y8OZL7g6os+C/ruHbsTncdIbjbDr0IHcept98MV2O118Ytbc3PJjnR0cPNacu71gygwKfT6MGTzZI8Cb7+5h4EMneHS8f13nWeMnUVYUGFxoYPPenWTeP5V+QZ6Xy6cNnnIomx28DIxqUk2qSTWpplPXNH38VG7+n6+O7kt13DjWQSyeATP0+mALphRQU+4568friaZ5fe9gon/41XA7bSxbWDzkZJFnsnFXmEgsrZpUk2pSTarpNOKpLP/4+57RHVIziyDPDt4PffvsSUJRng2vy4YBoikodkNfGlJZKPQ6+OScQl7cHiaTNXgcYDPQFcsO2R+czEJ/GoKFgxMgBzKDZy32OqD3/cly18wooKc/w97mwa/Nfhd09Gdx2VSTalJNqkk1na6mTNaws9OM7pC6rBiCxV4K8hxEIn2UlQaIZ6AnHGNgIEnWwLE41E4qIouNrAEbBreJk7ANnjSrry9OJBonbWBaTYCengherwe3x0M8bWhrH/y00JWEonw3Uyt8xN+ftOMyCQx20jYXxkBbWy/JLJQHVJNqUk2qSTWdrqZUxvDivvAZQ2pE/07KAGmbjaTNTtzYyM9zUJrvwG63YXPYKSrwkDTg8zgo8Tkoy3fgc9l4tzWRu13oseFyQBZI2R0MZMHhtOP3Dt4nnoFAgQe7ww52GwV5g/cry3fQ2psiGs8O3i5w4HGqJtWkmlSTajqbmkp9Zxc/I3p2X9RAV0d8cKeoMXgdXbidNlr7DF5vHuPHFLG3s42WUDdux+BXz3gGutJ22jt6MCZLxoDdCR1xOx0NPYMXKEv209/Xj81m51gCFlYH6EqHicXiNLcmcL7/2nYM2BhIZ3GbOJmMweOGSMJGu2pSTapJNamm09aUSJ/dTrwRvbvvroXlVBS6GYgnaGrp5O1O+OyCIO80R4hEY0wpGvxbvN0Jk/0Q8IDL5WTc2EoAjh5rp6ErRdLu5lOzB69a29TcxoHuNK68PD4xuZj/t72Vy0rB5YDOATjWD/PLBusYU1VOXp6bZDLFkaPtbO+A5fMqCXidqkk1qSbVpJpOU9Oh5nZ+38ToPiZ141gb0bSNWMowLt/wdidcXmHHbrJ4fXlUlPhpbmmnqrqSNxt76YgmyHPAnLLBWK8sL+G9zjjNHX3MKrXzThdM92dp7YeUgenFduKpLJPHVdDVEwGbDU9+IevrB3+AN73YRqHbhtvlory8mP/3doh5pXZ6kqgm1aSaVJNqOk1NxSUB/m1L2+gOqeUT7ByLZumMg98NJZ7BmSrleVDideD1esj35dHXP0B7NIHd6aIwP49UNExlRQnvdSRwuxz43RCPRtjeAUVuKHQNfs2NpWFiIeTn5+HzejAG+mJxWnsTVFYUk+6P4vM4cXi8vNcRZ1ypl1Q0zJFwRjWpJtWkmlTTaWqqbxtg57H4GUNqRB+T8ufnMZBNgj1Lvi+PYmL0paAvNXgBMufAANO9HvpjcVJpg8MO6cxgJrfHsvTF4pQXuCjyeHAbH2XRGAX5XgrtKbLpNANp6IxDZzxOTZkDl90QiyXIGEi9/zj9KUhlMgwMDFDkyiNZkEdxOqGaVJNqUk2q6bQ1xc/q//kR/U3qC1eWkIzHiSfTBEqKaW9tA6AlZqcnATaTZUoR+NwODvZm6YkbXA5YUOFgdxdMyM+Q77aRl+ehrKSIpqNtlFWU0d/Xz0BsgHQW9kcdpNMZqn1QkmcDu53dHRmSWZhTaieetdGTMEwuyOJ0OqgoL6avL6aaVJNqUk2q6TQ1jfNlR/8xqVsmO8lm0kSScDAMVwwex6OifPAXz82t3ezsgjuuqKKrs5vGrgQ9GScr5n1wEDDf58Xr9dByrANjYHc3BH1Q4QW73c7E8VUcbmrl3e4sxX4fV4zzc7gpxNudcMPMMioKPSSSKY62tDN54hiaW9pJJFOqSTWpJtWkmk5T06Gmdv7QPMpD6r9eVUZD1wDHuvqZFrAxeUIVAB2dvfT1x7Db7YwdG+To0TYymQyFhfkUFubT2toBQH23IeCBSSUeyitKeH57KzMDUFNZRJE/n0wmS9PRNowxlJcXY7KGru4wxhhqaqro7OwmHk8SSw8+1hUVNsZWlbOvrV81qSbVpJpU02lq2t1laI+N8pBaNtGJzRh6BjL0JGBecPAcHalUmp6BLO1xmFPhpjjgp7snQjqdJmtzUN8xeH74ci/E05C12ZhW4qI9mqRtAKoKnZT57GAgnkhyIAw1RU4qi/LI87hpa+8mL8/Nga4UfUmDyz74WMf6YWaZi3Qmq5pUk2pSTarpNDWVeGBL6MwhNaInTmQyGfLshmKfA0+Bl8auPqZU5pPncdNv0kQjceLxJHkeN+GUnUQ8i9ueJfr+NUxqCqDQ6yZjdxJPxPC7weHNZyCepKM/Q02Jl7w8N/1d/cTiabKFWVxuN639UGmS9CUHz3Plcw7OlLF7C8hkYjjJqibVpJpUk2o6TU2FZ3ntrBF9WqRCrxuH3U6+2860ci/N/eDx5lFY4MNfkEfA58LtdpFMpmiLZYmk7XjcTnzOwRcpz+2kKpDHpNI83G4XAJPL8jAOJ70pB4UFPgoLfBTnu/HlubDb7MQSKZr7weV24XXZ3n8sG3keFzODPgp9qkk1qSbVpJrOVJPbdXbfkUbkN6njeygLCgpIJOKEI/2E+zvIGmg+1oXbAQW+PK6bUY4xhiPNrfQPQHnAR3VFPpn04P7V6vIA/QNxwtEowWAph5tCNLd2EYkOntH3UPPgD9BqJ1XicDqIRmM0H+sia6CiohToIp5I4XY5KK8oo/FIK2OrysEeU02qSTWpJtV0mpoqKsugIcSZjjiNyGNSR48epaamZrjLEBGRj6m5uZmxY8eecv2IDKlsNsv+/fuZNWsWzc3Npz3oNppEIhFqamouqTGDxn0pjftSHDNcmuM2xhCNRqmursZuP/WRpxG5u89utzNmzBgA/H7/JfNHPe5SHDNo3JeSS3HMcOmNu6io6Ix9RvTECRERGd0UUiIiYlkjNqQ8Hg+PPfYYHo9nuEu5aC7FMYPGfSmN+1IcM1y64z4bI3LihIiIXBpG7DcpEREZ/RRSIiJiWQopERGxLIWUiIhY1ogMqSeeeIIJEyaQl5fHokWLeOONN4a7pPPqW9/6FjabbUibMWNGbn08HmfVqlWUlpZSUFDAHXfcQVtb2zBWfO42bdrELbfcQnV1NTabjV//+tdD1htjePTRR6mqqsLr9bJkyRIOHDgwpE93dzcrV67E7/cTCAT44he/SF9f30Ucxbk707i/8IUvnPC3X758+ZA+I23ca9as4corr6SwsJCKigpuvfVW9u/fP6TP2WzTTU1NrFixAp/PR0VFBY888gjpdPpiDuWcnM24b7jhhhP+3vfff/+QPiNt3OfbiAupX/ziFzz88MM89thjvP3228yfP59ly5bR3t4+3KWdV7Nnz6a1tTXXXn/99dy6hx56iN/+9rc899xzbNy4kWPHjnH77bcPY7Xnrr+/n/nz5/PEE0+cdP13v/td/uVf/oUf/ehHbNu2jfz8fJYtW0Y8Hs/1WblyJXv37uWVV17hxRdfZNOmTdx3330XawgfyZnGDbB8+fIhf/tnn312yPqRNu6NGzeyatUqtm7dyiuvvEIqlWLp0qX09/fn+pxpm85kMqxYsYJkMsmWLVv4yU9+wtNPP82jjz46HEM6K2czboB77713yN/7u9/9bm7dSBz3eWdGmKuuusqsWrUqdzuTyZjq6mqzZs2aYazq/HrsscfM/PnzT7qut7fXuFwu89xzz+WW7du3zwCmrq7uIlV4fgHm+eefz93OZrMmGAyaf/7nf84t6+3tNR6Pxzz77LPGGGPq6+sNYN58881cn5dfftnYbDbT0tJy0Wr/OP503MYYc/fdd5vPfvazp7zPaBh3e3u7AczGjRuNMWe3Tb/00kvGbrebUCiU6/Pkk08av99vEonExR3AR/Sn4zbGmOuvv9783d/93SnvMxrG/XGNqG9SyWSS7du3s2TJktwyu93OkiVLqKurG8bKzr8DBw5QXV3NpEmTWLlyJU1NTQBs376dVCo15DWYMWMG48aNGzWvQWNjI6FQaMgYi4qKWLRoUW6MdXV1BAIBFi5cmOuzZMkS7HY727Ztu+g1n08bNmygoqKC6dOn88ADD9DV1ZVbNxrGHQ6HASgpKQHObpuuq6tj7ty5VFZW5vosW7aMSCTC3r17L2L1H92fjvu4n/70p5SVlTFnzhxWr15NLBbLrRsN4/64RtQJZjs7O8lkMkP+YACVlZW8++67w1TV+bdo0SKefvpppk+fTmtrK48//jjXXnste/bsIRQK4Xa7CQQCQ+5TWVlJKBQanoLPs+PjONnf+fi6UChERUXFkPVOp5OSkpIR/TosX76c22+/nYkTJ9LQ0MA3vvENbr75Zurq6nA4HCN+3Nlsli9/+cssXryYOXPmAJzVNh0KhU66PRxfZ3UnGzfAX/7lXzJ+/Hiqq6vZtWsXX/va19i/fz//+Z//CYz8cZ8PIyqkLhU333xz7t/z5s1j0aJFjB8/nl/+8pd4vd5hrEwutDvvvDP377lz5zJv3jwmT57Mhg0buOmmm4axsvNj1apV7NmzZ8gx1kvBqcb94WOJc+fOpaqqiptuuomGhgYmT558scu0pBG1u6+srAyHw3HCrJ+2tjaCweAwVXXhBQIBpk2bxsGDBwkGgySTSXp7e4f0GU2vwfFxnO7vHAwGT5gsk06n6e7uHjWvA8CkSZMoKyvj4MGDwMge94MPPsiLL77Iq6++OuQid2ezTQeDwZNuD8fXWdmpxn0yixYtAhjy9x6p4z5fRlRIud1urrjiCtatW5dbls1mWbduHbW1tcNY2YXV19dHQ0MDVVVVXHHFFbhcriGvwf79+2lqaho1r8HEiRMJBoNDxhiJRNi2bVtujLW1tfT29rJ9+/Zcn/Xr15PNZnNv9NHg6NGjdHV1UVVVBYzMcRtjePDBB3n++edZv349EydOHLL+bLbp2tpadu/ePSSgX3nlFfx+P7Nmzbo4AzlHZxr3yezcuRNgyN97pI37vBvumRvn6uc//7nxeDzm6aefNvX19ea+++4zgUBgyOyXke4rX/mK2bBhg2lsbDSbN282S5YsMWVlZaa9vd0YY8z9999vxo0bZ9avX2/eeustU1tba2pra4e56nMTjUbNjh07zI4dOwxgvve975kdO3aYI0eOGGOM+c53vmMCgYB54YUXzK5du8xnP/tZM3HiRDMwMJB7jOXLl5sFCxaYbdu2mddff91MnTrV3HXXXcM1pLNyunFHo1Hz1a9+1dTV1ZnGxkbzxz/+0Vx++eVm6tSpJh6P5x5jpI37gQceMEVFRWbDhg2mtbU112KxWK7PmbbpdDpt5syZY5YuXWp27txp1q5da8rLy83q1auHY0hn5UzjPnjwoPn2t79t3nrrLdPY2GheeOEFM2nSJHPdddflHmMkjvt8G3EhZYwxP/jBD8y4ceOM2+02V111ldm6detwl3Refe5znzNVVVXG7XabMWPGmM997nPm4MGDufUDAwPmS1/6kikuLjY+n8/cdtttprW1dRgrPnevvvqqAU5od999tzFmcBr6N7/5TVNZWWk8Ho+56aabzP79+4c8RldXl7nrrrtMQUGB8fv95p577jHRaHQYRnP2TjfuWCxmli5dasrLy43L5TLjx48399577wkfwEbauE82XsA89dRTuT5ns00fPnzY3Hzzzcbr9ZqysjLzla98xaRSqYs8mrN3pnE3NTWZ6667zpSUlBiPx2OmTJliHnnkERMOh4c8zkgb9/mmS3WIiIhljahjUiIicmlRSImIiGUppERExLIUUiIiYlkKKRERsSyFlIiIWJZCSkRELEshJSIilqWQEhERy1JIiYiIZSmkRETEshRSIiJiWf8/BOysnJF6q6MAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "obs = env.reset()\n",
    "plt.imshow(obs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As stated in the paper, a series of preprocessing operations will be implemented to facilitate the task. To this end, we will utilize gym wrappers, which can be accessed via the following link: https://www.gymlibrary.dev/api/wrappers/\n",
    "\n",
    "The preprocessing operations encompass the following steps:\n",
    "\n",
    "-   Initial cropping of extraneous vertical screen elements.\n",
    "-   Normalization of values by dividing them by 255.\n",
    "-   Resizing the observation to a dimension of 84x84 pixels.\n",
    "-   Stacking four consecutive frames to enable movement detection by the agent.\n",
    "-   Additionally, reward normalization will be employed to enhance the stability of the learning process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENVIRONMENT             = \"Pygame-v0\"\n",
    "VERTICAL_TOP_CROP       = 190\n",
    "VERTICAL_BOTTOM_CROP    = -15\n",
    "FRAME_STACK             = 4\n",
    "RESIZED_WIDTH           = 84\n",
    "RESIZED_HEIGHT          = 84"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CropObservation(gym.ObservationWrapper):\n",
    "  \n",
    "    def __init__(self, env):\n",
    "        super().__init__(env)\n",
    "    \n",
    "    def observation(self, obs):\n",
    "        return obs[VERTICAL_TOP_CROP:VERTICAL_BOTTOM_CROP,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NormalizeObservation(gym.ObservationWrapper):\n",
    "\n",
    "    def __init__(self, env):\n",
    "        super().__init__(env)\n",
    "    \n",
    "    def observation(self, obs):\n",
    "        return np.divide(obs, 255.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_environment():\n",
    "  \n",
    "  env = gym.make(ENVIRONMENT)\n",
    "  env = gym.wrappers.NormalizeReward(env, gamma=0.99, epsilon=1e-8)\n",
    "  env = CropObservation(env)\n",
    "  env = NormalizeObservation(env)\n",
    "  env = gym.wrappers.ResizeObservation(env, (RESIZED_WIDTH, RESIZED_HEIGHT))\n",
    "  env = gym.wrappers.FrameStack(env, FRAME_STACK)\n",
    "  \n",
    "  return env"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = create_environment()\n",
    "obs = env.reset()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at the new observation space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 84, 84, 3)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.observation_space.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have created a function specifically designed to display the new observation, which consists of four stacked frames."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def showObservation(obs):\n",
    "    fig = plt.figure(figsize=(12, 8))\n",
    "    for i in range(FRAME_STACK):\n",
    "        fig.add_subplot(1, FRAME_STACK, i+1)\n",
    "        plt.imshow(obs[i])\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAD6CAYAAACms8QbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA2EklEQVR4nO3de4xcd33//9fnXObMzF59SdY28SYmhKYQboLgmlCohEWEUEUgoqBSiZa2KcHQJFRcLBFSUsAlamlEoFCqKoBKG0hVaEEqKJjiitZJSNJbvgkOgfyIS7xrEntnd+d2Lp/P749ZL3HsJGfXu5mZc56P6kPD7Oz6M5x9evM+O3PGOOecAAAAAADA0/L6vQEAAAAAAIYFQzQAAAAAADkxRAMAAAAAkBNDNAAAAAAAOTFEAwAAAACQE0M0AAAAAAA5MUQDAAAAAJATQzQAAAAAADkxRAMAAAAAkBNDNAAAAAAAOa3bEP2Zz3xG5513nqrVqnbu3Kk777xzvf4oAOuEjoFioGWgGGgZGAzrMkR/5Stf0Xvf+15dd911uueee/SiF71Il156qY4ePboefxyAdUDHQDHQMlAMtAwMDuOcc2v9RXfu3KmLL75Yn/70pyVJ1lpt375d73nPe/TBD37wKT/XWqtHHnlEY2NjMsas9daAwnDOaWFhQdu2bZPnrf35sDPp+MT9aRl4auvdsUTLwDNh0FumY+DpraTjYK3/8DiOdffdd2vv3r3Lt3mep927d+vgwYOn3L/b7arb7S7/95/97Gd63vOet9bbAgrr8OHDOuecc9b0a660Y4mWgTOxHh1LtAw80walZToGVi9Px2t+quzRRx9VlmWampo66fapqSnNzMyccv99+/ZpYmJieRE4sDJjY2Nr/jVX2rFEy8CZWI+OJVoGnmmD0jIdA6uXp+O+X5177969ajQay+vw4cP93hIwVAblaVm0DKzeoHQs0TJwJgalZToGVi9Px2v+dO7NmzfL933Nzs6edPvs7Ky2bNlyyv2jKFIURWu9DQBnYKUdS7QMDCJaBoqBf78GBsua/ya6UqnopS99qfbv3798m7VW+/fv165du9b6jwOwDugYKAZaBoqBloEB49bBLbfc4qIocl/4whfcfffd56644go3OTnpZmZmnvZzG42Gk8RisXKuRqOxHhmfUce0zGKtbK1Xx7TMYj2za1BbpmMWK//K0/G6DNHOOXfTTTe56elpV6lU3Mtf/nJ3++235/o8ImexVrbW8wf2ajumZRZrZWs9O3aOllmsZ2oNast0zGLlX3k6Xpf3iT4T8/PzmpiY6Pc2gKHRaDQ0Pj7e722cgpaB/Aa1Y4mWgZUY1JbpGMgvT8d9vzo3AAAAAADDgiEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIaUVD9L59+3TxxRdrbGxMZ599ti677DIdOnTopPt0Oh3t2bNHmzZt0ujoqC6//HLNzs6u6aYBnBlaBoYfHQPFQMvA8FnREH3gwAHt2bNHt99+u2677TYlSaLXvva1ajaby/e55ppr9I1vfEO33nqrDhw4oEceeURvetOb1nzjAFaPloHhR8dAMdAyMITcGTh69KiT5A4cOOCcc25ubs6FYehuvfXW5fvcf//9TpI7ePBgrq/ZaDScJBaLlXM1Go0zyZiWWawBWIPaMS2zWCtbg9oyHbNY+Veejs/oNdGNRkOStHHjRknS3XffrSRJtHv37uX7XHjhhZqentbBgwdP+zW63a7m5+dPWgCeWbQMDL+16FiiZaDf+JkMDL5VD9HWWl199dW65JJLdNFFF0mSZmZmVKlUNDk5edJ9p6amNDMzc9qvs2/fPk1MTCyv7du3r3ZLAFaBloHht1YdS7QM9BM/k4HhsOohes+ePbr33nt1yy23nNEG9u7dq0ajsbwOHz58Rl8PwMrQMjD81qpjiZaBfuJnMjAcgtV80rvf/W5985vf1L/927/pnHPOWb59y5YtiuNYc3NzJ50tm52d1ZYtW077taIoUhRFq9kGgDNEy8DwW8uOJVoG+oWfycAQWcmFDqy1bs+ePW7btm3ugQceOOXjJy588A//8A/Lt/3whz90Ehc+YLHWa63mIia0zGIN1hrUjmmZxVrZGtSW6ZjFyr/ydLyiIfrKK690ExMT7nvf+547cuTI8mq1Wsv3eec73+mmp6fdd7/7XXfXXXe5Xbt2uV27duX+M4icxVrZWs0PbFpmsQZrDWrHtMxirWwNast0zGLlX2s+RD/ZH3TzzTcv36fdbrt3vetdbsOGDa5er7s3vvGN7siRI0TOYq3TWs0P7Cf7WrTMYvVnDWrHtMxirWwNast0zGLlX3k6NkvxDoz5+XlNTEz0exvA0Gg0GhofH+/3Nk5By0B+g9qxRMvASgxqy3QM5Jen4zN6n2gAAAAAAMqEIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMjpjIboP/3TP5UxRldfffXybZ1OR3v27NGmTZs0Ojqqyy+/XLOzs2e6TwDrhI6BYqBloBhoGRh8qx6if/CDH+iv/uqv9MIXvvCk26+55hp94xvf0K233qoDBw7okUce0Zve9KYz3iiAtUfHQDHQMlAMtAwMCbcKCwsL7oILLnC33Xabe/WrX+2uuuoq55xzc3NzLgxDd+utty7f9/7773eS3MGDB3N97Uaj4SSxWKycq9ForCbjde2Yllmsla3VdkzLLNZgrUFtmY5ZrPwrT8er+k30nj179PrXv167d+8+6fa7775bSZKcdPuFF16o6elpHTx4cDV/FIB1QsdAMdAyUAy0DAyPYKWfcMstt+iee+7RD37wg1M+NjMzo0qlosnJyZNun5qa0szMzGm/XrfbVbfbXf7v8/PzK90SgBVa644lWgb6gZaBYuDfr4HhsqLfRB8+fFhXXXWVvvzlL6tara7JBvbt26eJiYnltX379jX5ugBObz06lmgZeKbRMlAM/Ps1MIRW8lqNr33ta06S831/eUlyxhjn+777zne+4yS548ePn/R509PT7pOf/ORpv2an03GNRmN5HT58uO/Pg2exhmmt9PVX69ExLbNYZ7ZW8zpKWmaxBm8NSst0zGKtfuXpeEVP537Na16j//3f/z3ptt/5nd/RhRdeqA984APavn27wjDU/v37dfnll0uSDh06pIcffli7du067deMokhRFK1kGwDOwHp0LNEy8EyjZaAY+PdrYPisaIgeGxvTRRdddNJtIyMj2rRp0/Ltv/u7v6v3vve92rhxo8bHx/We97xHu3bt0q/8yq+s3a4BrBodA8VAy0Ax0DIwfFZ8YbGn8xd/8RfyPE+XX365ut2uLr30Uv3lX/7lWv8xANYRHQPFQMtAMdAyMFiMc871exOPNz8/r4mJiX5vAxgajUZD4+Pj/d7GKWgZyG9QO5ZoGViJQW2ZjoH88nS8qveJBgAAAACgjBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJwYogEAAAAAyIkhGgAAAACAnBiiAQAAAADIiSEaAAAAAICcGKIBAAAAAMiJIRoAAAAAgJyCfm9gGBhjTvzT0v93cs71azt4hvziuPdwzIcfLZcTLRcPLZcTLRcLHZdTUTpmiH4axhjV6yOqVCoyxsgYT1mWqdlcVJomQ3vg8eSMMfL9QLVaTZ7nybneX+pJkqjb7XDMhxQtlw8tFxMtlw8tFw8dl0/ROmaIfhrGGEXVquq1uoznyfM8pUsHO8tSScN7BgWnMsZIxsj3fVWrNfmBL2ednLMyxqjb7fR7i1glWi4XWi4uWi4XWi4mOi6XInbMEP0UPM9TGAbauGVCG8+alB/4CkJfnVZX8aG2XMMqTVKladrvrWKN+J6nwPc1UYt0/sZxVcNQibXKrNOjCwvqtJpKsoy/2IcMLZcPLRcTLZcPLRcPHZdPETtmiH4SZulsSVSLNP3LWzT93G2qVANV6xXNP7aohfl52cyq1WwTeYGEQaBaFGnr+Kheec6UJmtVtVKrTuZ06OjPdeT4cdlYyoYs9DKj5XKi5eKh5XKi5WKh43IqYscM0U/CeKZ3ZqwSqDpW0cjGSJUoUK1eUZYlqo5UVKlW1O3G/d4q1lDgeaoGgUbCQBuj3mr6Tp3MaSwMFPq+Yi+TtXZoIi87Wi4nWi4eWi4nWi4WOi6nInbMEP0kgjDQ6MSoRjfWddazR7X1eaMKK76iaqBoo9WWHRtljJH7aabFhUVpOI43noKRtKEa6ZzJMZ0/Marnj4TaXAvUzqSulbrNqn4wMiLfD9RoNhVb2+8tIwdaLh9aLiZaLh9aLh46Lp+idswQ/SQ8z1OlWlG1VtHIZEWjm0MFoa+o6ivNQo1uqKp5vKZKNZQRjRdFNfA1UY20IQp1VsXX2aGnti/FzmhjJVAtrKidWvlPuDw/BhctlxMtFw8tlxMtFwsdl1MRO/ZW+gk/+9nP9Fu/9VvatGmTarWaXvCCF+iuu+5a/rhzTh/+8Ie1detW1Wo17d69Wz/60Y/WdNPPhDAKNLa5rrGzRlSpB/ICT57fO7DGGIVRoEq9Fz4Kwkih76ke+qr6vryljj0jBUaqB4HOGhnRppERVYLhP/9Ey7RcWCVquSwdS7RcSrS8/PGitEzHJVTQjlc0RB8/flyXXHKJwjDUv/zLv+i+++7Tn//5n2vDhg3L97nhhhv0qU99Sp/73Od0xx13aGRkRJdeeqk6neG6dHlYDTQ+NaaJLWOqjlTkB0be0v9anmdUqQWqjlSWIh+esyZ4ckZS6HkaDQNVA19m6bgGRgqNNFoJNTU2qrNHRxX5wxP56dBy72O0XExlablMHUu0XEa0XLyW6bh8itrxinb6iU98Qtu3b9fNN9+8fNuOHTuW/9k5pxtvvFEf+tCH9IY3vEGS9KUvfUlTU1P6+te/rre+9a1rtO1ngDHyfE9+4Ml4vcOdWafMZoqTTFnmZK2THZIXvyMPIycpc0aZM4qXVu8jS2fMPE+BMRqiZ5ucFi3TcrGVo+VSdSzRcinRslSwlum4hIrZ8Yp+E/3P//zPetnLXqY3v/nNOvvss/WSl7xEf/3Xf7388YceekgzMzPavXv38m0TExPauXOnDh48uHa7fgZ4nlEQ9a4eaJaed9DpJnrseFPH5ppabHXV7iRK00y8YqMYnKTEGbWtp0Xr6Vga6LEkUMf24veNp8j3VQkCecNU+WnQMi0XWVlaLlPHEi2XES33FKllOi6fona8oiH6Jz/5iT772c/qggsu0Le//W1deeWV+sM//EN98YtflCTNzMxIkqampk76vKmpqeWPPVG329X8/PxJayCY3mszjCcZ0/sGsNYpSTIlqZW1Tk4n8h6eA46nY+RkZGWU2N6ZMvu442uMkSctPxVlWNEyLRdf8Vtej44lWsagoWV+JmP4Fa/jFT2d21qrl73sZfr4xz8uSXrJS16ie++9V5/73Of09re/fVUb2Ldvnz7ykY+s6nPXk3OSs1ZZ5paeWmJlZBQFoVSTtmzfoPGRuo4fOda7FD9POxl6njHaXAt1wWRdm6sV1QKnwPTOhFonZW7pKUfOyQ352VFapuUiK0vL69GxRMsYHLTMz2Q6Hn5F7XhFv4neunWrnve855102y//8i/r4YcfliRt2bJFkjQ7O3vSfWZnZ5c/9kR79+5Vo9FYXocPH17JltaPc7KZk8usrO2dGfNkVPF91aJIm6cmNLV9g8Ymar3XdAzR0w9wOr1zX5NRqHPHqtparyjynHxjJeeUycg6J+vs0hvB93u/Z4aWabm4ytPyenQs0TIGBS3zM5mOh19xO17Rb6IvueQSHTp06KTbHnjgAZ177rmSehdB2LJli/bv368Xv/jFkqT5+XndcccduvLKK0/7NaMoUhRFq9j6+nLOyaZWaWLVaaZqzidKmk7deSsbO8WNTEnHySZGvu9LTsqyrN/bxip5nifP99SV0ZyVupmUxlLFc9rgG436TouZFGdW6ZCdKTsdWqbloipTy+vRsUTLGAy0zM9kOh5+Re54RUP0Nddco1e84hX6+Mc/rt/4jd/QnXfeqc9//vP6/Oc/L6n3fParr75aH/3oR3XBBRdox44duvbaa7Vt2zZddtll67H/deMyp6STKW6mavy8Iz9qqX3UavFhK1nJ94zknNKmp0oYKTXJ0hmU4Tn46DHGKAgCBYGvhgL9f4knZZKNe2/6fn7F09bA6EjXqplmaqepsiE/zrRMy0VUtpbL1LFEy2VCy8VtmY7Lo+gdr2iIvvjii/W1r31Ne/fu1fXXX68dO3boxhtv1Nve9rbl+7z//e9Xs9nUFVdcobm5Ob3yla/Ut771LVWr1TXf/HpyVrKpVRZbxW2ruGmVtKSs48lZSb7rnU1bPjnG002KIHVSu3fBfaVeIN94WnROC5lTyzrF1iotwF/mtEzLRVeGlsvUsUTLZUXLPUVpmY7LqYgdGzdgu52fn9fExES/t6HRiVFtnd6q2khN07+0XZNnTyo0I6qYCWVppubcccXdjg7de58O/b/7ZK3l6SZDzPM8eZ6v5z7nOXru+c9RpVbX2KbN8v1A3vFHZRYX9PNGQz85ckTNTkc/e/TnWmy3+71tSVKj0dD4+Hi/t3EKWkY/DGvLg9qxRMvoD1peW3SMfihyxyv6TXSZ2NSqu5jIZIHax6yqRvLHI1U3TChNUi1mC0o7UhpbpWk6VGdOcKoTTxWKndT2AnmVSOHkBgVBoOOLC1rM5jWXZmrEiTpJotRyvIcFLZcLLRcXLZcLLRcTHZdLkTtmiH6CIAjk+4Gq1Zqq1aqqtaoqtUhhPVJ9ckwTW6eUdLpabBxXksQygd/vLWMN+WGgsBqpOjaqiamzFFYqShYasq2mOt2OqlEk55yiqCLremdHOUM6mGi53Gi5OGi53Gi5GOi43IrYMUP0E1QqkarVmuojdY2Mjiiq11Qdq6syXtPo2Zt01rPPVbfZ0vGfz6obd+SH/E9YGMYoCENFo3XVN0xo83nTqlSrShvHpeaikiRRvVaXMUatTl1ORt24O/CRlxUtlxgtFwotlxgtFwYdl1hBO+Y79HGMMQrDULVaVdVqVZVKpEpYkafe1eRcamXjVFmSKkliJXEsO+AHGCuT2UxpHCtNEmVxIuv7slnvzeDleQoqoUKbKapGssbJ2kzdfm8ap6Bl0HIx0DJoefjRMYrYMUP04xhjNLlhUlu2blUlijQ6NqYgCOUngdJGplYwr2PBYbWbTT36f49o7rFH1VxY6Pe2sVacU2thUT8/MqMstXr0rMOq1mpqzM1rIU6Ver5GJicUpXWFI1XFSaKjs7NqNpu8ZmfA0HLJ0XJh0HLJ0XIh0HHJFbRjhugnqESRRsZGVQlDVaOqPN+XcUYudkrbsTrzi+q0muo0W+q0WsrSVDJGvfcGH9wDjRyMUZok6rRa6jSb6swvSEmmbqerJLNyxiioVOQFvpyRKlmqueOVpePPsR80tFxitFwotFxitFwYdFxiBe2YIXqJ53kKgkAjo3Vt3DQpyUiuF69NU9nMqb0wL+Oc4m5XLkkVGF8T4xNyW89RlqXqxl3ZLFO73VIcx/1+SHgaYRiqEkXyPV9hGMrzfdVrdSmzitttHZs5okol0uJ8Q912S85l8jwjzw9UrY3LeNLc3HH5vi+r3hUI0X+0XD60XEy0XD60XDx0XD5l6ZghWr2nmfi+Lz8INDI6og2bJpUmmVqLHdnM9iK3qdppqrjVVpqmskkq33gaH5tQrVpXEsdaWJxXEifKsozIh0AQhhodGVMQBKrXRxQEgYIwlMsyxa22jh2ZURAEStNEmc3k+0ZB6MkPfI1NjCishJo5MiPf9+WsU+9M6eCeMSsDWi4nWi4eWi4nWi4WOi6nsnTMEL3MyBgjY078VyeFVgokL6vIc6H8IFRQieRnVqN+1Dv4S5dgj7sd+XPH1O12tNhqSYu8lmPQVap1jW86W5VKpNGJDQorFfme3ztrGoaqjozK9zyZbkdeGst3qUKXyLdWxrnl12kY9Z5xMuDPOikRWi4bWi4qWi4bWi4iOi6bsnTMEL3E84w840nOyGZOzrfSWCpjPFXMBvkaV1gbVWVkQsbztMEYGRkZz5MxnrqtRR2bOaz24rwWm00df+xovx8SnsbExrP17IsuVm10XBu3Tiuqj8hZu3zWyzknZ63a848pbi0obDVUPT4jWas0yZR5npx1y98DEleSHAS0XD60XEy0XD60XDx0XD5l6Zgh+gmcc7LOyRknE/aeiuKZinxTk1dZWp4vPwhlPE++78vzfHl+oNpiQ845hVHUewqCcwP7PP4y650RNQqjqqoj46qNTqg+vlHV+qiyLJO1mZy1ytJUNksVRG3ZLFMYd1SRkXNOmXVy2dJfCFz0YiDRcvHRcjnQcvHRcvHRcfGVrWOGaPXCTtNUktRKmlqIGwoCX+FYVZ4fKbAT8u1GWSs1Gy3J9C6ScCJy3w+UdDtyqeTL16YNm5Vtn9Zic1GPPfYYoQ8Qz/M0MTGhWrWm8dFxec6TS526za5s6stmmbKlyNM0kXNOnj+i6nhdVRmNtY7LprG6i23FNlO62Fx+2tEgX4a/LGi5PGi52Gi5PGi5uOi4PMrYMUP0kizLJGMUp121k5aqqqpWH5EfVOUnI/LSUaWdjjqtRUnmcZEH8v1UWRLLZU5GnsZGxmQ3nyXP83T8+HEiHyDGGI3URzQ+Pq56rS4jI5c5JZ1ENuvK2mzpbJlVliaSjGqjI6pUI1XjtupRTakkLS4q67SVtntXjOQYDw5aLgdaLj5aLgdaLjY6LocydswQvcT3fQW+r7ASqlqLVK1WVI0C+b6vtJspjWO5LOu9rkNOclayTlkaK3OZlMUyaVtB1tGk11EtbCvzY3kD/DSEMvKNtCFMNBV1VfW7CrKuTGIUN+dk/IqMF8h4viTJyEim9xYMaVdSGquqTJlnVasEilVRpdK7dL/T4F6Cv2xouRxoufhouRxoudjouBzK2DFDtJYuwb90+fVavabR8bqqY5HGR6oyJtTcXKK41ZRk5HtLkdtUzlmlnUUlnQX5sqqZRL6NtcGfV1Sdlwvbut8Q+SAJjNM5UVcXjBh1w6ZaWVOZjdVqt5TJU1QfV6U+IeP1nkYkGWVxrCzuynTbGlMi52ear1fkokC1alVB0MvIPe6KgugPWi4PWi42Wi4PWi4uOi6PMnbMEC2pd06k92J4z1t6Tzvfk7d0OX6XJbJJRzKeZHxJTi6LJZfJ2UTGZTKycktXnMucU2qlbPCOd+k5SdZp6fgsXSFQVnKZJCuXxbJJW8YLJBdJxpOzqeSsbBrLyEpy8j2jwHnyvd73Tu9C/Og/Wi4LWi46Wi4LWi4yOi6LMnbMEP04RlLge4oqgYLAyDkrZ2MlCz9X97hk/IoUVJcO+KKcTRQGkepRRdY6xamRtVZHWlW152s62kqU2sE9+GWUWuknzVANP1LdCzU2buQHvqJKKM/zFHcbajVmZPyKvNqE5HnKuk3ZtKtxL1bmdeUZp0BS5Dn5nAkdSLRcfLRcDrRcfLRcfHRcfGXsmCH68YyWzpR58o2R1HsfM5t0lXVSKajKWCe5VFkyL9lYFW9CgV9VZpy6qVHiPM2lvuY6gRqpr8F8Fn95Zc5oLvaUtH1tSDxVMqniS1XfV+h7SppdJc3jMkHUe32O5yvrzCuLW0pCyVWdnCf5MvKNeE3OoKLlwqPlkqDlwqPlEqDjwitjxwzRknpv/G1lbW9lWaZAkh9aeb4UVjyFlUAmCKWgIicj5zlZZ5XYttyiUxwnml9YUDfu6tjPZ3T82KPqtDuydvC/CcrEOatWu6k0S2X8SGGlqkolkpuYUKUSysRtjRrJ+Z5cGMn5vpytyJlUdS9TzSQyrvfeddb1LnbgnB3I12qUEy2XBS0XHS2XBS0XGR2XRRk7ZoheYq1bDjxLU1lJfuRLziis+ooiT/IrUhjJyigLJWtTxYtNdRZb6rRbevTRo+p02jo6+4jmjh8b6ANfVs45NRcX1TLN3nvP2UTValUm26JataYJYzTuGVnfVxxFsn4oT13JzzSiWHV15ZyVsUbZ0vfMiQsecLwHAy2XAy0XHy2XAy0XGx2XQxk7Zohe1jtAzrne2S3j5EdGckbGt3LGSkrlbCKrRG7p4CZxrG4zVbvdVqu5qE63ozRJBvaAo8c5pzRJ1O20JWfVbjXlrFU1DJWFoZzNZNO4dxYsS+XZTL6xqvqSc0Ymc7JOck5Lx5rjPThouUxouchouUxouajouEzK1DFDtE4cqKUDn1nFSaq676u+KZCRkRe1lSmRzWwvYKVKXVeZyzR37Lge+9kxddptPXbsUSVxrDRN+v2QkEOn01aaJgqCUHEcK4oiacNG1SY3SmorXTgmeZ6CrCPZWNVI2hIZZTJ6ILFKrFXGX+YDhZbLiZaLh5bLiZaLhY7LqSwdM0Q/gbNLTx0wTn6o3iUFPSunrPccfedkXabMZrI2U9ztqtVqqtNuq9NuKU3Tfj8E5GStVRzHyjKrdrulLMvUrY8oyVIZ48klXcnzJNd7mwXfGUWeUeokY6RMpvf6nX4/EJwWLZcHLRcbLZcHLRcXHZdHWTpmiFbvgHmeJ8/z5SmQyULZ2FOzlUieU+LFsrVY1jq5TEq6iY4deUzdxY4eO/qY5uaOKU1TZZZrBQ4jazO12i3FcVdHfV+Zcxqr1rR9o1EUhvKUyTgrP/PVzHwlxqjr+0oqTjYI5HmerLUyxvA0oz6j5XKj5eKg5XKj5WKg43IrescM0ZIksxS5J8/58rJANjVqdxLJX4q8ksi53tmxNO6q8ehxLT7W0lzjuBbm5/v9AHAGnHPqdjrqSjLGU9dZTY2M6oKxEY15mZwkZ4w866mTeYo9o9j3lXpajtwYT1LW50cCWi43Wi4SWi4zWi4KOi6zonfMEC1J6l1Ov/eajExxJ5FZMGrOepLvlCZWqli51MpmmTKbqtNtq91uKU14fUaRZFmqbqejOAyVZalcFijwPfmSEuv0szhVbIwasuo4KUnT5cvwYxDQMnpoedjRMnpoeZjRMXqK2DFDtHpnSrI0lZHUaXbUnGup0/XUafsyFcnWE5mRVLaVKu0k6iYdzc/Pae74vLJsMM+OYHW63a7iJNGoMYq7XaW+r1Gvorrvq5k53bkQKzbSY8apLadWp6s0zZYvxY/+omWcQMvDjZZxAi0PLzrGCUXsmCF6yfLVA9NMSZzKBb7CzJOXGXmBkYmMstST5xsZr/ei+TTLep+IwrDWyjgnZ608Sb6kipFqnlHXOWXqnTFLXKbY9b4HBvk97MqIliHRchHQMiRaHnZ0DKmYHTNEL3HOylqjVrOl48fmNFGra+uOuiqjvsKxirxIas7FakQdWWcVRL3n6rsBPkOClTOm9/qdSuBrQ62ms+p1bat62hh6alnpLOvUSDL9x7F5HW911Gq2e+91x/fAwKBlSLRcBLQMiZaHHR1DKmbHDNGP07ske6JWs6VRF6i2yVNtMlQ04skPTe9F8R2raCGUH3jyjBnQl7pjtYwxMsYo8HyNhKHGKqE2VKTNoVHHSbVMqsrKdGM1my3FcTzQgZcVLYOWi4GWQcvDj45RxI69fm9g0CRxrFarqSSLVal7iuq+nHNKkkxWVl7VKKj7qo1WVR+tK4zCfm8ZaygMAo1UqxqJItV9XzXPk5zUzXpPQxn1pRHjZJNYnU5HaZLwjKMBRcvlRsvFQcvlRsvFQMflVsSOGaIfxzmp02lrYX5ecdJWbTRQfTyUk1W3k8jKKhiRKmO+RidHNb5xXFE16ve2sYZqlYomR0Y1Wa9rPAw05vuSM2pnTnJOGwJpwnfKum01m4uK41ga+LeDLx9aBi0XAy2DlocfHaOIHfN07pP0XsCe2UzOWp04eDaTbOxkrZOMZDyjIPJVqYXyQ7+/W8aaCjxf1TBUFPjyjSdjpFROzjqFS6eczNL3Ca/XGWS0XHa0XBS0XHa0XAR0XHZF7Jgh+gmstcqyTFmaKU2s0q5VMu/UnXeyFclUjYKKr7GzRuRSo1a7Kf1cg36yBDkYSeNRpGeNj2tzfUQV35cxRs1M6lqrxHgasU6xldKl75NBfv+6sqPl8qLlYqHl8qLl4qDj8ipqxzyd+wlOnAGx1slmTjZ1yrpOWUeyS+/7bjyjMAoU1TlTVijGKPQ91cNQ1SCQ5xlJRrFzalunrnNKnZQtXXJ/0C+9X3a0XGK0XCi0XGK0XBh0XGIF7ZjfRD9BkiRyzqm50FbjaEdJR5qb7ajdSOSPS6FvlLRTpUmmLLVydvAPMvJzTrLOKbNW3TSTb4yOxanmklRx5is0nua6VovdWN1uV1nG9SMHFS2XGy0XBy2XGy0XAx2XWxE7Zoh+gjRNlKbJUuRtxW1pbratdiNWNfNVrwdK2qmyxBJ50TjJyck6p9Q5dbJMxhgdjzMd7aaKrRT6gRpxL/I47vZ7x3gKtFxitFwotFxitFwYdFxiBe2YIfpJ2Myq207leYmSOO29hqMrJU2jtOMk23vaCYopc07NLFUqqZOmSrNM3dRoMcnUSjOlQ/A0E/TQcrnRcnHQcrnRcjHQcbkVqeMVvSY6yzJde+212rFjh2q1ms4//3z9yZ/8yUnPW3fO6cMf/rC2bt2qWq2m3bt360c/+tGab3y9Jd1M80fbahxtqbXQVqfbUbvRVfNIovbRVC5T7w3hfUIvDNP7D6PehQ4eaXd0uNXW8XZH7W5HjU5HR1odzbRjdbPBv+DBU6FlWi60krRcpo4lWi4lWl6+T1FapuMSKmjHKxqiP/GJT+izn/2sPv3pT+v+++/XJz7xCd1www266aablu9zww036FOf+pQ+97nP6Y477tDIyIguvfRSdTqdNd/8enKZlHWd0o5VlmS9KwomVlnbKutauaUr9A/RCRM8Hdf7DycptU7tzKqVZYqzTGmWKcms2tapY52yIT/utEzLhVaSlsvUsUTLpUTLy/cpSst0XEIF7XhFT+f+j//4D73hDW/Q61//eknSeeedp7//+7/XnXfeKal3luzGG2/Uhz70Ib3hDW+QJH3pS1/S1NSUvv71r+utb33rGm9//bjUyC74yhJfncVU3W5HaeTkqoFslqm9ECuJY6WdwX/hO/Jyaqepjnc6qoRW3SCU53nqdrtK4q4qVakT1dSWUdLvrZ4hWqblYitHy2XqWKLlcqJlqVgt03EZFbPjFf0m+hWveIX279+vBx54QJL03//93/r+97+v173udZKkhx56SDMzM9q9e/fy50xMTGjnzp06ePDgab9mt9vV/Pz8SWsgZEau7cu1PCWLmbqLseL5VMm8VTxvFTdTdZuJsiTjPewKwkmKM6tmkmghSdRIU82lmeaTRAtxrIU01YKMmvKUabifZkTLtFxkZWl5PTqWaBmDg5b5mYzhV9SOV/Sb6A9+8IOan5/XhRdeKN/3lWWZPvaxj+ltb3ubJGlmZkaSNDU1ddLnTU1NLX/sifbt26ePfOQjq9n7uvA8X57vqVKJVI1GFIaBssSpvdBROF5TtTKiNE20kM4ri61sSuFFkjmnbmZlKp7C2oh831er1VSz29V4bUTV+oi8sKKwUpHvB3LOytrhef3GCbRMy0VXhpbXo2OJljFYaJmfyRh+Rex4Rb+J/upXv6ovf/nL+ru/+zvdc889+uIXv6g/+7M/0xe/+MVVb2Dv3r1qNBrL6/Dhw6v+WmshDANVqzXVa6MaH92g0fqk0lam+UcXZDvSWH1SI9UJudhT0syUJUReJLHtvU4j8TzVJzZoZHKjujJ6dHFRHSeNTm7S2IZNqtVHFEUV+b7f7y2vCi3TctGVoeX16FiiZQwWWl4dOsYgKWLHK/pN9Pve9z598IMfXH7txQte8AL99Kc/1b59+/T2t79dW7ZskSTNzs5q69aty583OzurF7/4xaf9mlEUKYqiVW5/7RnjyfeWlu/L8zxZK2VpJuecfC+Q7/Vep+EsVz4oIuskGSPP93tnwyRl1sqpdybV96w8z5cxnowZnqedPB4t03IZFL3l9ehYomUMHlrmZzKGX9E6XtFvolutljzv5E/xfX/51+07duzQli1btH///uWPz8/P64477tCuXbvWYLvrz/d9BUEoPwiWD6C1mdI0lSSFYagwDIfi4GKVTO8v+0pYUaUSyRijLMtkrZXxjLylHwBBEMjzBv9M2enQMi2XQsFbLkPHEi1DtFyAlukYRet4Rb+J/vVf/3V97GMf0/T0tJ7//OfrP//zP/XJT35S73jHOyRJxhhdffXV+uhHP6oLLrhAO3bs0LXXXqtt27bpsssuW4/9rzljegfRM57M0v8555b/IvN8f+ksiVl+3zMUi9HS94Hvy1t6Ool1veNvjOktz8h4w3Gm7HRomZbLoOgtl6FjiZZBy0VomY5RtI5XNETfdNNNuvbaa/Wud71LR48e1bZt2/QHf/AH+vCHP7x8n/e///1qNpu64oorNDc3p1e+8pX61re+pWq1uuabX0/GM/KXzoTUanWNjo4piqqSegd/dHRczkndTleNxvF+bxdrwqharWlicqNGR8d7f5lLiiqRRuqjqlZrCsNIxvR+CAwzWqblYitHy2XqWKLlcqLlE4rSMh2XUTE7XtEQPTY2phtvvFE33njjk97HGKPrr79e119//Znura+M8RQEFQVBoHp9VGNjE4qqNRlJvudrfHxSYRhpYb4hY3pn0zDcjJFq9RFt3HiWarV676IGxiiqVjU6OqZ6fVSVqNp7rYY3PJGfDi3TcpGVpeUydSzRchnR8i8UpWU6Lp+idryiIfqZ0O9YTjy1JMsyxUlX1mbLz9fP0lRxEstZqzRNlWWprLN93zPWhnNOzjplWaYsS5UksYwxStNM1vVuT+KukiRWlvWO+yAc+0HYw+n0e1+0XF7D2HK///yn0u+90XJ50fLa6fe+6Li8itqxcf3e5RP83//9n7Zv397vbQBD4/DhwzrnnHP6vY1T0DKQ36B2LNEysBKD2jIdA/nl6XjghmhrrR555BE55zQ9Pa3Dhw9rfHy839tad/Pz89q+fTuPt6DW4/E657SwsKBt27adclXPQWCt1aFDh/S85z2P41xgZXvMa/14B71jiZZ5vMXDz+RyHGeJ7+2i6+fP5IF7OrfneTrnnHM0Pz8vSRofHy/FN8EJPN5iW+vHOzExsWZfa615nqdnPetZkjjOZVC2x7yWj3eQO5ZomcdbXPxMLo+yPWYe7+rl7XjwTpUBAAAAADCgGKIBAAAAAMhpYIfoKIp03XXXKYqifm/lGcHjLbayPd4Tyva4y/Z4pfI95rI93hPK9rh5vMVWtsd7Qhkfd9keM4/3mTNwFxYDAAAAAGBQDexvogEAAAAAGDQM0QAAAAAA5MQQDQAAAABATgzRAAAAAADkNJBD9Gc+8xmdd955qlar2rlzp+68885+b2lN7Nu3TxdffLHGxsZ09tln67LLLtOhQ4dOus+v/dqvyRhz0nrnO9/Zpx2fmT/+4z8+5bFceOGFyx/vdDras2ePNm3apNHRUV1++eWanZ3t447PzHnnnXfK4zXGaM+ePZKKdWzzouViHG9aLnfLdFyMY122jiVafiJaLsaxLlvLg9rxwA3RX/nKV/Te975X1113ne655x696EUv0qWXXqqjR4/2e2tn7MCBA9qzZ49uv/123XbbbUqSRK997WvVbDZPut/v//7v68iRI8vrhhtu6NOOz9zzn//8kx7L97///eWPXXPNNfrGN76hW2+9VQcOHNAjjzyiN73pTX3c7Zn5wQ9+cNJjve222yRJb37zm5fvU6Rj+3RouVjHm5bL2TIdF+tYl6ljiZYfj5aLdazL1PLAduwGzMtf/nK3Z8+e5f+eZZnbtm2b27dvXx93tT6OHj3qJLkDBw4s3/bqV7/aXXXVVf3b1Bq67rrr3Ite9KLTfmxubs6FYehuvfXW5dvuv/9+J8kdPHjwGdrh+rrqqqvc+eef76y1zrliHds8aLk4x5uWy9syHRfnWJe9Y+domZav6t+m1lDZWx6UjgfqN9FxHOvuu+/W7t27l2/zPE+7d+/WwYMH+7iz9dFoNCRJGzduPOn2L3/5y9q8ebMuuugi7d27V61Wqx/bWxM/+tGPtG3bNj372c/W2972Nj388MOSpLvvvltJkpx0rC+88EJNT08X4ljHcay//du/1Tve8Q4ZY5ZvL9KxfSq03FOk403L5WuZjnuKdKzL2rFEy7RcrGNd1pYHqeNg3f+EFXj00UeVZZmmpqZOun1qako//OEP+7Sr9WGt1dVXX61LLrlEF1100fLtv/mbv6lzzz1X27Zt0//8z//oAx/4gA4dOqR//Md/7ONuV2fnzp36whe+oF/6pV/SkSNH9JGPfES/+qu/qnvvvVczMzOqVCqanJw86XOmpqY0MzPTnw2voa9//euam5vTb//2by/fVqRj+3RouVjHm5bL2TIdF+tYl7ljiZZpuTjHuswtD1LHAzVEl8mePXt07733nvQaBkm64oorlv/5BS94gbZu3arXvOY1+vGPf6zzzz//md7mGXnd6163/M8vfOELtXPnTp177rn66le/qlqt1sedrb+/+Zu/0ete9zpt27Zt+bYiHVv8Ai3T8jAfW/TQcbE7lmi5LGi52C0PUscD9XTuzZs3y/f9U64gNzs7qy1btvRpV2vv3e9+t775zW/qX//1X3XOOec85X137twpSXrwwQefia2tq8nJST33uc/Vgw8+qC1btiiOY83NzZ10nyIc65/+9Kf6zne+o9/7vd97yvsV6dg+ES2fqkjHm5ZPVqRj+3h0fKoiHeuydCzRMi2fqkjHuiwtD1rHAzVEVyoVvfSlL9X+/fuXb7PWav/+/dq1a1cfd7Y2nHN697vfra997Wv67ne/qx07djzt5/zXf/2XJGnr1q3rvLv1t7i4qB//+MfaunWrXvrSlyoMw5OO9aFDh/Twww8P/bG++eabdfbZZ+v1r3/9U96vSMf2iWj5VEU63rR8siId28ej41MV6ViXpWOJlmn5VEU61mVpeeA6fsYvZfY0brnlFhdFkfvCF77g7rvvPnfFFVe4yclJNzMz0++tnbErr7zSTUxMuO9973vuyJEjy6vVajnnnHvwwQfd9ddf7+666y730EMPuX/6p39yz372s92rXvWqPu98df7oj/7Ife9733MPPfSQ+/d//3e3e/dut3nzZnf06FHnnHPvfOc73fT0tPvud7/r7rrrLrdr1y63a9euPu/6zGRZ5qanp90HPvCBk24v2rHNg5aLc7xp+ReKdmyfDh0X51iXsWPnaPkEWi7OsS5jy4PY8cAN0c45d9NNN7np6WlXqVTcy1/+cnf77bf3e0trQtJp18033+ycc+7hhx92r3rVq9zGjRtdFEXuOc95jnvf+97nGo1Gfze+Sm95y1vc1q1bXaVScc961rPcW97yFvfggw8uf7zdbrt3vetdbsOGDa5er7s3vvGN7siRI33c8Zn79re/7SS5Q4cOnXR70Y5tXrRcjONNy79QtGObBx0X41iXsWPnaPnxaLkYx7qMLQ9ix8Y559b3d90AAAAAABTDQL0mGgAAAACAQcYQDQAAAABATgzRAAAAAADkxBANAAAAAEBODNEAAAAAAOTEEA0AAAAAQE4M0QAAAAAA5MQQDQAAAABATgzRAAAAAADkxBANAAAAAEBODNEAAAAAAOTEEA0AAAAAQE7/P/UqJnAGSFmAAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1200x800 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "showObservation(obs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As observed, the initial observation is formed by stacking the initial frame four times. Subsequently, when the agent performs an action, the resulting new state will be placed in the fourth position, as demonstrated in the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs, _ ,_ ,_ = env.step([4,0]) # action 4 means shooting, 0 none"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAD6CAYAAACms8QbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9yUlEQVR4nO3de5BkZ32f8ec99+657kU7uyvtSkLIlkGAXVjIa7CdKm9ZRVEpMCrHrpAqJ06iAIsjgcsGVRkIxPYayhcV2DFxKgW4QoKtVCDBVbELi6CU4wUhEcdREAsCGa3ZnV1JO9Mz05dzed83f/TsoJUE6pmd0XSf8/24Xix6emZPc+bZ1u/06dPGe+8RERERERERkecV7PYGiIiIiIiIiEwKDdEiIiIiIiIiI9IQLSIiIiIiIjIiDdEiIiIiIiIiI9IQLSIiIiIiIjIiDdEiIiIiIiIiI9IQLSIiIiIiIjIiDdEiIiIiIiIiI9IQLSIiIiIiIjIiDdEiIiIiIiIiI9qxIfr3f//3ue6668iyjFtvvZUHHnhgp/4oEdkh6likHtSySD2oZZHxsCND9B//8R/zjne8g/e+9718+ctf5hWveAW33XYbFy5c2Ik/TkR2gDoWqQe1LFIPallkfBjvvd/uH3rrrbdyyy238Hu/93sAOOc4cuQIv/iLv8i73vWu7/m9zjnOnj3LzMwMxpjt3jSR2vDes7q6yuHDhwmC7T8ediUdX7q/Whb53na6Y1DLIi+EcW9ZHYs8v810HG33H14UBQ899BB33333xm1BEHD8+HFOnTr1rPvneU6e5xv//dvf/jYveclLtnuzRGrrzJkzXHPNNdv6MzfbMahlkSuxEx2DWhZ5oY1Ly+pYZOtG6XjbD5U9+eSTWGtZWFi47PaFhQUWFxefdf+TJ08yNze3sRS4yObMzMxs+8/cbMeglkWuxE50DGpZ5IU2Li2rY5GtG6XjXb869913302n09lYZ86c2e1NEpko43JalloW2bpx6RjUssiVGJeW1bHI1o3S8bafzr1//37CMOT8+fOX3X7+/HkOHjz4rPunaUqaptu9GSJyBTbbMahlkXGklkXqQf9+LTJetv2V6CRJeOUrX8l99923cZtzjvvuu49jx45t9x8nIjtAHYvUg1oWqQe1LDJm/A745Cc/6dM09R/72Mf8V77yFX/HHXf4+fl5v7i4+Lzf2+l0PKClpTXi6nQ6O5HxFXWslrW0Nrd2qmO1rKX1wq5xbVkda2mNvkbpeEeGaO+9//CHP+yPHj3qkyTxr3rVq/wXvvCFkb5PkWtpbW7t5BP2VjtWy1pam1s72bH3allL64Va49qyOtbSGn2N0vGOfE70lVhZWWFubm63N0NkYnQ6HWZnZ3d7M55FLYuMblw7BrUsshnj2rI6FhndKB3v+tW5RURERERERCaFhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERmRhmgRERERERGREWmIFhERERERERnRpobokydPcssttzAzM8OBAwd4wxvewOnTpy+7z2Aw4MSJE+zbt4/p6Wluv/12zp8/v60bLSJXRi2LTD51LFIPallk8mxqiL7//vs5ceIEX/jCF/jsZz9LWZb81E/9FN1ud+M+b3/72/nMZz7Dvffey/3338/Zs2d54xvfuO0bLiJbp5ZFJp86FqkHtSwygfwVuHDhggf8/fff7733fnl52cdx7O+9996N+zzyyCMe8KdOnRrpZ3Y6HQ9oaWmNuDqdzpVkrJa1tMZgjWvHallLa3NrXFtWx1pao69ROr6i90R3Oh0A9u7dC8BDDz1EWZYcP3584z433XQTR48e5dSpU8/5M/I8Z2Vl5bIlIi8stSwy+bajY1DLIrtNz8ki42/LQ7RzjrvuuotXv/rV3HzzzQAsLi6SJAnz8/OX3XdhYYHFxcXn/DknT55kbm5uYx05cmSrmyQiW6CWRSbfdnUMallkN+k5WWQybHmIPnHiBA8//DCf/OQnr2gD7r77bjqdzsY6c+bMFf08EdkctSwy+barY1DLIrtJz8kikyHayje97W1v40//9E/5n//zf3LNNdds3H7w4EGKomB5efmyo2Xnz5/n4MGDz/mz0jQlTdOtbIaIXCG1LDL5trNjUMsiu0XPySITZDMXOnDO+RMnTvjDhw/7r33ta8/6+qULH/zn//yfN2776le/6kEXPtDS2qm1lYuYqGUtrfFa49qxWtbS2twa15bVsZbW6GuUjjc1RL/lLW/xc3Nz/vOf/7w/d+7cxur1ehv3efOb3+yPHj3qP/e5z/kHH3zQHzt2zB87dmzkP0ORa2ltbm3lCVsta2mN1xrXjtWyltbm1ri2rI61tEZf2z5Ef7c/6KMf/ejGffr9vn/rW9/q9+zZ49vttv/pn/5pf+7cOUWupbVDaytP2N/tZ6llLa3dWePasVrW0trcGteW1bGW1uhrlI7NerxjY2Vlhbm5ud3eDJGJ0el0mJ2d3e3NeBa1LDK6ce0Y1LLIZoxry+pYZHSjdHxFnxMtIiIiIiIi0iQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZEQaokVERERERERGpCFaREREREREZERXNET/5m/+JsYY7rrrro3bBoMBJ06cYN++fUxPT3P77bdz/vz5K91OEdkh6likHtSySD2oZZHxt+Uh+ktf+hL/9t/+W17+8pdfdvvb3/52PvOZz3Dvvfdy//33c/bsWd74xjde8YaKyPZTxyL1oJZF6kEti0wIvwWrq6v+xhtv9J/97Gf9T/zET/g777zTe+/98vKyj+PY33vvvRv3feSRRzzgT506NdLP7nQ6HtDS0hpxdTqdrWS8ox2rZS2tza2tdqyWtbTGa41ry+pYS2v0NUrHW3ol+sSJE7zuda/j+PHjl93+0EMPUZblZbffdNNNHD16lFOnTm3ljxKRHaKORepBLYvUg1oWmRzRZr/hk5/8JF/+8pf50pe+9KyvLS4ukiQJ8/Pzl92+sLDA4uLic/68PM/J83zjv6+srGx2k0Rkk7a7Y1DLIrtBLYvUg/79WmSybOqV6DNnznDnnXfyiU98gizLtmUDTp48ydzc3MY6cuTItvxcEXluO9ExqGWRF5paFqkH/fu1yATazHs1PvWpT3nAh2G4sQBvjPFhGPq/+Iu/8IBfWlq67PuOHj3qf+d3fuc5f+ZgMPCdTmdjnTlzZtfPg9fSmqS12fdf7UTHallL68rWVt5HqZa1tMZvjUvL6lhLa+trlI43dTr3T/7kT/J//+//vey2f/JP/gk33XQT73znOzly5AhxHHPfffdx++23A3D69Gkef/xxjh079pw/M01T0jTdzGaIyBXYiY5BLYu80NSySD3o369FJs+mhuiZmRluvvnmy26bmppi3759G7f/03/6T3nHO97B3r17mZ2d5Rd/8Rc5duwYP/IjP7J9Wy0iW6aORepBLYvUg1oWmTybvrDY8/nd3/1dgiDg9ttvJ89zbrvtNv7Nv/k32/3HiMgOUsci9aCWRepBLYuMF+O997u9EU+3srLC3Nzcbm+GyMTodDrMzs7u9mY8i1oWGd24dgxqWWQzxrVldSwyulE63tLnRIuIiIiIiIg0kYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZkYZoERERERERkRFpiBYREREREREZUbTbGyAiIiIiIs0ShiFBcOn1PIP3HmsrvPe7ul2yc4wxBEGIMcP/7j1473DO7e6GbYGGaBERERERecEYY8iyFkmSYIzBmABrLd3uGlVVapCuIWMMYRjRarUIggDvPd57yrIkzwcTt881RI/AXDpcwqX/7yduR8vmfWe/D2mfTz613ExquX7UcjOp5fowxhAnMWkrIwgMQRBQlRWDvI+1FaD9WyfDAyWGMAzI4pgwDHHrQ7Txnjzf7S3cPA3Rz8MYQ7s9pSNlDVK3I2UypJabRy3Xk1puHrVcJ8OBOclijtx4iKsO7yeKQ5IkorvW5/Rff4POUysURUFZlLu9sbJNojAkjiL2TU/xsqsP0U4TCuspnePc0jJfLXKKqpqo07o1RD8PYwxpltFutTFBsH6kbPiXto6U1Y8xBowhDEOyrEUYhXjn8d5hjCHPB7u9ibJFarlZ1HJ9qeVmUcv1YgwEgSGOI/Yf3sORGw+SpDFZO6FzcZWzjy3SXxtgraVEQ3RdhEFAEsfMtTJevH8v8+2MXunIrcNby6PnL1BaizFmYv7+1hD9PQRBQBxH7D04x96r5gmjkCgOGfRyitN9fMdRlRVVVe32pso2CYOAKAyZa6XcsHeWLI4pncM6z5Orqwx6XUprJyZwGVLLzaOW60ktN49arpcgDEizlGwqY+ZAxvyRjCSNyFoxJkuZvapNvz/AecugrwMkddGKI/a0Ug60Uq7NQvZmIb0oILeetSyhnSQ4D3mRYyekZQ3R34VZP+qZtlKO/sBBjn7fYZIsImsnrDy1xurKCs46et2+nqxrJI4iWmnKodlpXnPNAvOtjF7lGFjP6QtPcG5pCVeA1RP2xFDLzaSW60ctN5NarpcoimjPtpnZO8WBF09z9cunidOQrBUzdR4W/t8evIOqKuksrYB26cQzwFyWcs3cDDfMtnnlbMxVWUTPQe4MRTfjr9ptCEKsrbATckq3hujvwgRmeIQ7ichmEqb2piRpRKudYG1JNpWQZAl5Xuz2pso2ioKALIqYiiP2psPVDT0D65mJI+IwpAgszjk9WU8ItdxMarl+1HIzqeV6CUJDnMYkWUTcColaIXESEKWGKA1IspAkiwjD4Pl/mEyMKDBkUThcgSELDB4IMLTCgCyOSCtLYCZnv2uI/i6iOGJ6bprpvW2uetE0h14yTZyEpFlEutdx8Pq9w/P2v2VZW13TkbIaMMCeLOWa+RlumJvmpVMx+1sRfQu5g7yb8aWpKcIwotPtUkzIkbKmU8vNo5brSS03j1qunziL2XN4lrn9U7RnU+IkIIyGg1MQGlozKdPzLZJWjMHgFfLkM5BFEXvShJkkJjAGA8QGTABzaczVs7Ok0YC1Xpd+MRmX6tYQ/V0MrxyYkLUSpuYTpvfHRHFImoVUNmZ6T0Z3qUWSxRj0XF0XWRQyl6XsSWOuSkIOxAH9EApv2JtEtOKEfuUIn/ExGzK+1HIzqeX6UcvNpJbrJYwDsumUbCYjSiLC0PCdT6wzxElEksUbg7VMPoMhNsNXopMgwKx/NGFghoNoGobMpCkD64mCydnvm97Sb3/72/yjf/SP2LdvH61Wi5e97GU8+OCDG1/33vOe97yHQ4cO0Wq1OH78OF//+te3daNfCHEaMbO/zcxVUyTtiCAKCMLhTjfGEKcRSXv4BC41YSAOA9pxSBaGBOt/qQcGIgPtKOKqqSn2TU2RRJN//Ektq+XaalDLTekY1HIjqeWNr9elZRMERGlElIYb/ZaVY62X0+0VDIqSoqywVmcV1IUHLIbCBwy8oesC1mxA6YandIeBIY0i0jAkmKCDYZsaopeWlnj1q19NHMf89//+3/nKV77Cb//2b7Nnz56N+3zwgx/kQx/6EB/5yEf44he/yNTUFLfddhuDwWRdYS/OImYXZpg7OEM2lRBGhksHR4LAkLQisqlk/cl6cna4fHcGiIOA6Tgii8KNI2WRGZ5yMp3ELMxMc2B6mjSc7CdrtTz8mlqup6a03KSOQS03kVquX8tBaIiziDiLNobooqxYWctZ7Q7oD0ry4tIQrfNJ6qLyw4uI9W3AShWyYkMKP9z/oRle9yCLookaojf1N84HPvABjhw5wkc/+tGN266//vqNf/bec8899/Crv/qrvP71rwfgj/7oj1hYWODTn/40P/dzP7dNm/0CMIYgHL5PwwTDv7at81hnKUqLtR7nPE4XsaiR4REx6w3WG4r1NfzK+pHvICAyTzv1aEKpZbVcb81ouVEdg1puJLUM9Wt5+NHfw317qdbAQBgGZO2E1nRFlESgN2bUggHSMGAmDmnHIaGBwPjLDnUaAGM2DpRNgk29Ev3f/tt/44d/+If5mZ/5GQ4cOMAP/dAP8e/+3b/b+Ppjjz3G4uIix48f37htbm6OW2+9lVOnTm3fVr8AgsAQpcOrgJr184cGeclTS10uLndZ6+X0ByVVZVHg9eCB0hv6LmDNBVysIp4qIwaXTjcxAWkYkkzYkbLnopbVcp01peUmdQxquYnU8lDdWt7ghwcIwjAgS2OmplMOXbuHozdexdy+tk4oqZH9Wcz3z7e5bjplJobUeALD0w6kTN7O3tQQ/c1vfpM/+IM/4MYbb+TP//zPectb3sK//Jf/ko9//OMALC4uArCwsHDZ9y0sLGx87ZnyPGdlZeWyNRbWj5KZYHjEzAPOecrSUlYO5/xE73j5bgweg8NQuuERb/e0/WuMIYCJOlL2XNSyWq6/+re8Ex2DWpZxo5Zr9ZwMeA8ej/d++M8OcBAQkKYx2VRCnEQTvU/lO4yBJAyYiiOy9QvGOTzOX/pdGP4+MGFnEW3qdG7nHD/8wz/Mb/zGbwDwQz/0Qzz88MN85CMf4ed//ue3tAEnT57kfe9735a+dycNo3ZY69dPEXMYDGkUQwsOHtnD7FSbpXMXh6ekTNiOl2cLjGF/K+bG+Tb7s4RW5InM8BUN58H69VMHvZ/4j1xQy2q5zprS8k50DGpZxodarvFzcuno9go6K336TznWvl3hSrB9h80NvohI05SqslRVqZ4nkTFEYUQUBvRMxKILWCoN57uOxHiuS0MOxCFdG1BNYMebeiX60KFDvOQlL7nsth/4gR/g8ccfB+DgwYMAnD9//rL7nD9/fuNrz3T33XfT6XQ21pkzZzazSTvHe5z1eOtwbniEO8CQhCGtNGX/whwLR/YwM9cavjdrgk8jErh0DHs+jbl2JuNQOyENPKFx4D0Wg/Me54e/D5P+d7laVsv11ZyWd6JjUMsyLtRynZ+TrXX0ByWr3ZylxQHnHxnwxOmc7nlP3gHKkCRJieNYLU8oA0RRSBTHDIKIp1zA31WGv+k5/rrn+LvCsFqF9KzBOj9xHW/qlehXv/rVnD59+rLbvva1r3HttdcCw4sgHDx4kPvuu48f/MEfBGBlZYUvfvGLvOUtb3nOn5mmKWmabmHTd5b3Hlc5qtIx6FZ0V0rKridfcbjCU3Qs5cDjSkMYhuDBWrvbmy1bFAQBQRiQY1h2kFuoCkgCz57QMB161iwU1lFN2JGy56KW1XJdNanlnegY1LKMB7Vc7+dkWzrKvqXoWuwATBXhrad0Du8d5aCiqkqsrfQq9ASz1mGMJbeONQ/GBNhoeGBkgGHVevrWUbjJ63hTQ/Tb3/52fvRHf5Tf+I3f4B/8g3/AAw88wB/+4R/yh3/4h8DwfSl33XUXv/Zrv8aNN97I9ddfz7vf/W4OHz7MG97whp3Y/h3jraccWIpuReeJAWHao3/Bsfa4Azf8TDO8p+oGJHFKZcr1IyiTs/NlyBhDFEVEUUiHiL8tA7DgCkdoDDckAYciw7nc0a0s/arCTvh+VstquY6a1nKTOga13CRqub4tX+q47FX0lkqitKTqxARFiisd/X6PsirpdnL6/QHeq+FJ5b2nLIcHQjql40JliOOQdtrCBCFLriAtKp4sLb2qmriONzVE33LLLXzqU5/i7rvv5v3vfz/XX38999xzD29605s27vMrv/IrdLtd7rjjDpaXl3nNa17Dn/3Zn5Fl2bZv/E7yjuGRssJR9B1F11H2wA6C4QUQwuEFEdzGQW6dalIHlYf+8IMzqIKI0ASsec+q9fScHx4pq8G/lKlltVx3TWi5SR2DWm4qtTxUl5af3nE18Ni+x5eGwER4LNZCVTqsdRqga2F48TjrPaXzGAw+jCAMKcqSnvX0nR+eUTJhHRs/Zlu7srLC3Nzcbm8G03PTHDp6iNZUi6Pff4T5A/PEZorEzGErS3d5iSIfcPrhr3D6/30F55xOG5tgQRAQBCHf9+IX8303vJik1WZm337CMCJYehKztsoTnQ7fPHeO7mDAt598grV+f7c3G4BOp8Ps7Oxub8azqGXZDZPa8rh2DGpZdoda3l7j2PHVL76a+avmiZkmZZ4yL3ji3Fn6a10e++bX+dtvPjpRQ5U8N2MM1xy9jmuOXktraor9hw6TRBFxZ5mo1+XJToe/XTxHb8I63tQr0U3iKke+VmJsRP+iIzMQzqZke+aoyoo1u0o1gKpwVJXerzHpLp3yV3joBxFBkhLP7yGKIpbWVlmzKyxXlk5RMihLKqf9PSnUcrOo5fpSy82iluvp6R0XHSijgGgqIprNcMbg8FSuwjm325sq28h5R+Xs8PPd05ggTuiuBuTOs2wtKxPYsYboZ4iiiDCMyLIWWZaRtTKSVkrcTmnPzzB3aIFykLPWWaIsC0wU7vYmyzYK44g4S8lmpplbuIo4SShXO7hel0E+IEtTvPekaYLzw1c59ErHeFLLzaaW60MtN5tarofv2fGeOfYePETeH7C6vITHE2c7e1G0MDBE4fAD5yvnNj6zWHZGkmXMzM0xs3cv+49cQ5KkdJylW1UT27GG6GdIkpQsa9GeajM1PUXabpHNtElmW0wf2MdVL7qWvNtj6Ynz5MWAMNb/hLVhDFEck063ae+ZY/91R0myjKqzBN01yrKk3WpjjKE3aOMx5EU+9pE3lVpuMLVcK2q5wdRybXyvjmcX9nPwxhvIe31Wly5iAkPWau3o9kShYSqN8N7TKxh+9vz654/LNjOG1tQU8weuYs/CAkduuok0yziX54SDwcR2rGeapzHGEMcxrVZGlmUkSUoSJwQMrwrpK4crKmxZUZYFZVHgxnwHy+ZYZ6mKgqossUWJC0OcXf9LNQiIkpjYWdIsxRmPc5Z8tzdankUti1quB7UsannyPW/H1uNKiy2HH2tVleWOn84dGEMcGpw3BMbiDEzQpytNHOcstiyxVYUrK1xU4dxkd6wh+mmMMczvmefgoUMkacr0zAxRFBOWEVXH0otWuBidod/t8uTfnWX5qSfprq7u9mbLdvGe3uoaT5xbxFaOJ686Q9Zq0VleYbWoqIKQqfk50qpNPJVRlCUXzp+n2+3qvXdjRi03nFquDbXccGq5Fsax4zQKmG3HWOcpKrt+JXBw+r3ZfjXtWEP0MyRpytTMNEkck6UZQRhivMEXnqpfMFhZY9DrMuj2GPR62KoCY9aPXo3vjpYRGENVlgx6PQbdLoOVVSgt+SCntA5vDFGSEEQh3kBiK5aXkvX9r30/btRyg6nlWlHLDaaWa2PcOg5DQxqHWOeIAkNphn+c7ICadqwhel0QBERRxNR0m7375gEDfhivqyqc9fRXVzDeU+Q5vqyITMjc7Bz+0DVYW5EXOc5a+v0eRVHs9kOS5xHHMUmaEgYhcRwThCHtVhuso+j3ubh4jiRJWVvpkPd7eG8JAkMQRmStWUwAy8tLhGGIA11Jckyo5eZRy/WklptHLdfPuHUchYbAGPbNZLz40Bxl5bAOVnoFa4OKUm8HuWJN6VhDNMPTTMIwJIwipqan2LNvnqq09NYGOOuGkbuKflVR9PpU6+fzhyZgdmaOVtamLApW11YoixJrrZ6sJ0AUx0xPzRBFEe32FFEUEcUx3lqKXp+L5xaJooiqKrHOEoaGKA4Io5CZuSniJGbx3CJhGOKdZ3ikdHyPmDWBWm4mtVw/armZ1HK9jFvHBojCgDgMNoboQWG5uDbAe09ZedbQEH2lmtKxhugNBmPMd07lMB5iBxEENiHwMWEUEyUpoXVMh+lw569fgr3IB4TLF8nzAWu9HqzpPVnjLsnazO47QJKkTM/tIU4SwiAcHjWNY7KpacIgwOQDgqog9BWxLwmdw3i/8T4Nw/CMkzE/66RB1HLTqOW6UstNo5braHw6NgbSKCRLQuamWxzYO8+gqNh7YY3Kenq5Bujt0JSONUSvCwJDYALwBmc9PnQwU2FMQGL2EDJL3JommZrDBAF7jMFgMEGAMQF5b42Li2for62w1u2y9NSF3X5I8jzm9h7gRTffQmt6lr2HjpK2p/DObRz18t7jnaO/8hRFb5W41yFbWgTnqEqLDQK88xu/A+jo5VhQy82jlutJLTePWq6fceo4CAzz0ylzUyk3XH2AV/7AjfTznLV+wfxUTF55zi71x3JgmyRN6VhD9DN473He443HxMNTUQKTEJoWQbK+gpAwijFBQBiGBEFIEEa01jp474nTdHgKgvdjex5/kw2PiBriNCObmqU1PUd7di9ZexprLc5ZvHPDy/Dbiijt46wlLgYkGLz3WOfxdv0vBF28Ziyp5fpTy82glutPLdffOHRsMCRRSCuJmGplzE0PL3Q2086YaSUkUbgDj7w5mtaxhmiGYVdVBUCv7LJadIiikHgmIwhTIjdH6PbiHHQ7PTDDiyRcijwMI8p8gK8gJGTfnv3YI0dZ667x1FNP6Ql7jARBwNzcHK2sxez0LIEP8JUn7+a4KsRZi12PvKpKvPcE4RTZbJsMw0xvCVcV5Gt9Cmep1robpx2N82X4m0ItN4darje13Bxqub7GqWNjhu+H3jvT4vC+Gfbv28/UviOE+YCr9p/DOsfM2VX0cdFb08SONUSvs9aCMRRVTr/skZHRak8RRhlhOUVQTVMNBgx6a4B5WuQRYVhhywJvPYaAmakZ3P6rCIKApaUlPVmPEWMMU+0pZmdnabfaGAzeespBibP58MPgrcU5h61KwNCaniLJUrKiTzttUQGsrWEHfar+8IqR2sfjQy03g1quP7XcDGq53salY8PwdO7pVsq+2RazM7NkM/sxSZ/5mVmKQY8sTdbvOZ5D2zhrYscaoteFYUgUhsRJTNZKybKELI0Iw5Aqt1RFgbd2+L4OPHgHzmOrAust2AJT9YnsgPlgQCvuY8OCQCGOldDAnrhkIc3JwpzI5pjSUHSXMWGCCSJMMDydx2DADD+CocqBqiDDYgNHK4koSEiS4aX7PeN7Cf6mUcvNoJbrTy03g1qut/HpeHhxszgKSeKYKE4xSZvAQZSkJHFMGOh07q1qYscaolm/BP/65ddb7RbTs22ymZTZqQxjYpaXS4peFzCEwXrkrsJ7RzVYoxysEuJomZLQFewJV0izFXzc5xGjJ+txEhnPNWnOjVOGPO7Ss12sK+j1e1gC0vYsSXsOEwxPIwKDLQpskWPyPjOU+NCy0k7waUQry4iiYUb+aVcUlN2hlptDLdebWm4OtVxf49Lxpas8B8aQJjHtLCNtTRG09hCahKw1RavVJo4jvRC9RU3sWEM0MDwmMnwzfBCsf6ZdGBCsX47f2xJXDsAEYELA420B3uJdifEWg8OvX3HOek/lwI7f/m48DzjP+v5Zv0IgDrwFHN4WuLKPCSLwKZgA7yrwDlcVGBzgCQND5APCYPi7M/xbV3afWm4KtVx3arkp1HKdjVvHw98Lc+lzk9bXd35j9DuzVU3sWEP001z6EPY0iYgig/cO7wrK1SfIl8CECUTZ+g5fw7uSOEpppwnOeYrK4JzjXC+jv9LiQq+kcuO785uocvDNbkwnTGkHMTOzhjAKSZOYIAgo8g69ziImTAhacxAE2LyLq3JmgwIb5ATGEwFp4An1isZYUsv1p5abQS3Xn1quv3HpeDiMO5xzeOfAVuCGV412zuH8pStCy2Y1sWMN0U9nWD9SFhCa4fkc3jlcmWMHFUQZxnnwFbZcAVeQBHNEYYY1nrwylD5guQpZHkR0qpDxPIu/uaw3LBcBZT9kTxmQWEhCyMKQOAwouzlldwkTpcP35wQhdrCCLXqUMfjM4wMIMYQGvbduXKnl2lPLDaGWa08tN8C4dOzB+0unB7v1Zdf///oXZUua2LGGaICnHZlyzmGtJQLC2BGEECcBcRJhohiiBI/BBx7nHaXr49c8RVGysrpKXuRcfGKRpYtPMugPcG78fwmaxHtHr9+lshUmTImTjCRJ8XNzJEmMKfpMG/BhgI9TfBjiXYI3Fe3A0jIlZv1IpfPDix1s/OUrY0AtN4Varju13BRquc7Go2PPcD523jMoStb6Awa9Lq6/jM17DPo9uv0+xfrHccnmNbFjDdHrnPMbgduqwgFhGoI3xFlImgYQJhCnOAw2BucqirUug7VhgE8+eYHBoM+F82dZXro41ju+qbz3dNfW6Jnu8LPnXEmWZRh7kFbWYs4YZgODC0OKNMWFMQE5hJYpCtrkeO8wzmDXf2cuXfBA+3s8qOVmUMv1p5abQS3X27h07PE47+kNcla6ffq9VWzvIlXep9tdZa3XoyiL7f8foCGa2LGG6A3DHeS9Hx7dMp4wNeANJnR444AK70ocJX5955ZFQd6t6Pf79LprDPIBVVmO7Q6XIe89VVmSD/rgHf1eF+8cWRxj4xjvLK4qhkfBbEXgLKFxZCF4bzDW4552WpDeRDNO1HKTqOU6U8tNopbrakw6Xv+9yEtLd1DSHwzI+6sMBgN6/QFr/ZyytOj35so0qWMN0VzaUes73jqKsqIdhrT3RRgMQdrHUuKsGwZMReVzrLcsX1ziqW9fZNDv89TFJymLgqoqd/shyQgGgz5VVRJFMUVRkKYp7NlLa34v0KdavQhBQGQH4AqyFA6mBovha6WjdA6rfykbK2q5mdRy/ajlZlLL9TJOHXugsp6nVnpY5zh7/jwXzj5GL895fPECZ59YZmmtP8Yj2+RoSscaop/Bu/VTB4wnjBleUjBweOzwHH3vcd5i16/mV+Q5vV6XQb/PoN+j0vspJoZzjqIosNbR7/ew1pK3pyhthTEBvswhCMAPP2Yh9IY0MFR++KkIFjN8/85uPxB5Tmq5OdRyvanl5lDL9TUOHV96Jbo3KOn1B3S7q/TygrXegNV+TlHZcX7hc2I0pWMN0ax/AHsQEAQhARHGxrgioNsrIfCUQYFrFTjn8RbKvOTiuafI1wY8deEplpcvUlUV1uman5PIOUuv36Moci6EIdZ7ZrIWR/Ya0jgmwGK8I7QhXRtSGkMehpSJx0URQRDgnMMYo9MFd5labja1XB9qudnUcj2MW8fee9YGJZV1/O3iEg+efpy8KHn07BJPdHp0usXYD26TpO4da4gGwKxHHhD4kMBGuMrQH5QQrkeelHg/PDpWFTmdJ5dYe6rHcmeJ1ZWV3X4AcgW89+SDATlgTEDuHQtT09w4M8VMYIdXdTSGwAUMbEARGIowpArYiNyYALC7/EhELTebWq4Ttdxkarkuxqtj56GfVxSl5exTHb7ytyFFZXn8iRU63YK1vs5a2U5171hDNADDy+kP35NhKQYlZtXQPR9A6KlKB4nDVw5nLdZVDPI+/X6PqtT7rOrE2op8MKCIY6yt8DYiCgNCoHSebxcVhTF0cAw8lFW1cRl+GQdqWYbU8qRTyzKklifZ+HXsvAcH3UHF+eU+lXV0BxWD0lLpzJUdU8eONUQzPFJiqwoDDLoDuss9BnnAoB9iEnDtEjNV4XoV1aAkLwesrCyzvLSCteN5dES2Js9zirJk2hiKPKcKQ6aDhHYY0rWeB1YLCgNPGU+f4UclVJXduBS/7C61LJeo5cmmluUStTy5xrFj6zwWzxMrA3p5hfOeflFRWY/VZ8jvmDp2rCF63cbVAytLWVT4KCS2AYE1BJHBpAZbBQShwQTDN81X1g6/UWrDOYfxHu8cARACiYFWYMi9xzI8YlZ6S+GHvwPj/Bl2TaSWBdRyHahlAbU86ca149I6ekWF91Da4cdv6Vdm59SxYw3R67x3OGfodXssXVxmrtXm0PVtkumQeCYhSKG7XNBJBzjviNLhufp+jI+QyOYZM3z/ThKF7Gm1uKrd5nAWsDcO6Dm4ynk6peWvLq6w1BvQ6/aHn3Wn34GxoZYF1HIdqGUBtTzpxrVj6zxF6fAMT/HWr8vOqmPHGqKfZnhJ9pJet8e0j2jtC2jNx6RTAWFshm+KHzjS1ZgwCgiMGdO3ustWGWMwxhAFIVNxzEwSsyeB/bFh4KFlIcNh8oJut0dRFGMdeFOpZVHL9aCWRS1PvnHs2Huo9Hvygqljx8Fub8C4KYuCXq9LaQuSdkDaDvHeU5YWhyPIDFE7pDWd0Z5uE6fxbm+ybKM4ipjKMqbSlHYY0goC8JDb4Wko0yFMGY8rCwaDAVVZ6ujlmFLLzaaW60MtN5targd13Gx17FhD9NN4D4NBn9WVFYqyT2s6oj0b43HkgxKHI5qCZCZken6a2b2zpFm625st26iVJMxPTTPfbjMbR8yEIXhD33rwnj0RzIUem/fpdtcoigL0qYJjRy2LWq4HtSxqefKpY6ljxzqd+zLDN7BbZ/HOcWnnOQuuGF50AAMmMERpSNKKCeNwdzdZtlUUhGRxTBqFhCbAGKjweOeJ1w85mfXfE73vbpyp5aZTy3WhlptOLdeBOm66OnasIfoZnHNYa7GVpSodVe4oVzz5isclYDJDlITMXDWFrwy9fheeYNwPlsgIDDCbplw9O8v+9hRJGGKMoWshd47SBEw5T+GgWv89GefPr2s6tdxcarle1HJzqeX6UMfNVdeOdTr3M1w6AuKcx1mPqzw299gBuPXPfTeBIU4j0raOlNWKMcRhQDuOyaKIIDCAofCevvPk3lN5sOuX3B/3S+83nVpuMLVcK2q5wdRybajjBqtpx3ol+hnKssR7T3e1T+fCgHIAy+cH9Dsl4SzEoaHsV1SlxVYOrw9mrxXvhx91YJ0jryyhMVwsKpbLisKGxCZgOXes5QV5nmOtrgM7rtRys6nl+lDLzaaW60EdN1sdO9YQ/QxVVVJV5XrkfYo+LJ/v0+8UZDak3Y4o+xW2dIq8bjx4PM57Ku8ZWIsxhqXCciGvKBzEYUSnGEZeFPlub7F8D2q5wdRyrajlBlPLtaGOG6ymHWuI/i6cdeT9iiAoKYtq+B6OHMquoRp4cMPTTqSerPd0bUUFDKqKylryyrBWWnqV1WcLThC13GxquT7UcrOp5XpQx81Wp4439Z5oay3vfve7uf7662m1Wtxwww3863/9ry87b917z3ve8x4OHTpEq9Xi+PHjfP3rX9/2Dd9pZW5ZudCnc6FHb7XPIB/Q7+R0z5X0L1R4y/AD4UOFXhtm+B+G4YUOzvYHnOn1WeoP6OcDOoMB53oDFvsFuR3/Cx58L2pZLddaQ1puUseglhtJLW/cpy4tq+MGqmnHmxqiP/CBD/AHf/AH/N7v/R6PPPIIH/jAB/jgBz/Ihz/84Y37fPCDH+RDH/oQH/nIR/jiF7/I1NQUt912G4PBYNs3fid5Czb3VAOHLe3wioKlw/YdNnf49Sv0T9ABE3k+fvgfHqicp28dPWsprKWyltI6+s4zcB474ftdLavlWmtIy03qGNRyI6nljfvUpWV13EA17XhTp3P/1V/9Fa9//et53eteB8B1113Hf/pP/4kHHngAGB4lu+eee/jVX/1VXv/61wPwR3/0RywsLPDpT3+an/u5n9vmzd85vjK41RBbhgzWKvJ8QJV6fBbhrKW/WlAWBdVg/N/4LqPy9KuKpcGAJHbkUUwQBOR5TlnkJBkM0hZ9DOVub+oVUstqud6a0XKTOga13ExqGerVsjpuonp2vKlXon/0R3+U++67j6997WsA/J//83/4y7/8S1772tcC8Nhjj7G4uMjx48c3vmdubo5bb72VU6dOPefPzPOclZWVy9ZYsAbfD/G9gHLNkq8VFCsV5YqjWHEU3Yq8W2JLq8+wqwkPFNbRLUtWy5JOVbFcWVbKktWiYLWqWMXQJcAy2acZqWW1XGdNaXknOga1LONDLes5WSZfXTve1CvR73rXu1hZWeGmm24iDEOstfz6r/86b3rTmwBYXFwEYGFh4bLvW1hY2PjaM508eZL3ve99W9n2HREEIUEYkCQpWTpFHEfY0tNfHRDPtsiSKaqqZLVawRYOV6nwOrHek1uHSQLi1hRhGNLrdenmObOtKbL2FEGcECcJYRjhvcO5yXn/xiVqWS3XXRNa3omOQS3LeFHLek6WyVfHjjf1SvSf/Mmf8IlPfIL/+B//I1/+8pf5+Mc/zm/91m/x8Y9/fMsbcPfdd9PpdDbWmTNntvyztkMcR2RZi3ZrmtnpPUy356l6lpUnV3EDmGnPM5XN4YuAsmuxpSKvk8IN36dRBgHtuT1Mze8lx/Dk2hoDD9Pz+5jZs49We4o0TQjDcLc3eUvUslquuya0vBMdg1qW8aKWt0YdyzipY8ebeiX6l3/5l3nXu9618d6Ll73sZXzrW9/i5MmT/PzP/zwHDx4E4Pz58xw6dGjj+86fP88P/uAPPufPTNOUNE23uPnbz5iAMFhfYUgQBDgHtrJ47wmDiDAYvk/DO135oI6cB4whCMPh0TDAOodneCQ1DBxBEGJMgDGTc9rJ06lltdwEdW95JzoGtSzjRy3rOVkmX9063tQr0b1ejyC4/FvCMNx4uf3666/n4MGD3HfffRtfX1lZ4Ytf/CLHjh3bhs3deWEYEkUxYRRt7EDnLFVVARDHMXEcT8TOlS0yw7/skzghSVKMMVhrcc5hAkOw/gQQRRFBMP5Hyp6LWlbLjVDzlpvQMahlQS3XoGV1LHXreFOvRP/9v//3+fVf/3WOHj3KS1/6Uv73//7f/M7v/A6/8Au/AIAxhrvuuotf+7Vf48Ybb+T666/n3e9+N4cPH+YNb3jDTmz/tjNmuBMDE2DW/897v/EXWRCG60dJzMbnnkm9GNZ/D8KQYP10EueH+98YM1yBwQSTcaTsuahltdwEdW+5CR2DWha1XIeW1bHUreNNDdEf/vCHefe7381b3/pWLly4wOHDh/kX/+Jf8J73vGfjPr/yK79Ct9vljjvuYHl5mde85jX82Z/9GVmWbfvG7yQTGML1IyGtVpvp6RnSNAOGO396ehbvIR/kdDpLu725si0MWdZibn4v09Ozw7/MgTRJmWpPk2Ut4jjFmOGTwCRTy2q53prRcpM6BrXcTGr5krq0rI6bqJ4db2qInpmZ4Z577uGee+75rvcxxvD+97+f97///Ve6bbvKmIAoSoiiiHZ7mpmZOdKshQHCIGR2dp44Tlld6WDM8GiaTDZjoNWeYu/eq2i12sOLGhhDmmVMT8/Qbk+TpNnwvRrB5ET+XNSyWq6zprTcpI5BLTeRWv6OurSsjpunrh1vaoh+Iex2LJdOLbHWUpQ5ztmN8/VtVVGUBd45qqrC2grn3a5vs2wP7z3eeay1WFtRlgXGGKrK4vzw9rLIKcsCa4f7fRz2/Thsw3PZ7e1Sy801iS3v9p//vez2tqnl5lLL22e3t0sdN1ddOzZ+t7fyGf7u7/6OI0eO7PZmiEyMM2fOcM011+z2ZjyLWhYZ3bh2DGpZZDPGtWV1LDK6UToeuyHaOcfZs2fx3nP06FHOnDnD7Ozsbm/WjltZWeHIkSN6vDW1E4/Xe8/q6iqHDx9+1lU9x4FzjtOnT/OSl7xE+7nGmvaYt/vxjnvHoJb1eOtHz8nN2M+g3+26283n5LE7nTsIAq655hpWVlYAmJ2dbcQvwSV6vPW23Y93bm5u237WdguCgKuvvhrQfm6Cpj3m7Xy849wxqGU93vrSc3JzNO0x6/Fu3agdj9+hMhEREREREZExpSFaREREREREZERjO0Snacp73/te0jTd7U15Qejx1lvTHu8lTXvcTXu80LzH3LTHe0nTHrceb7017fFe0sTH3bTHrMf7whm7C4uJiIiIiIiIjKuxfSVaREREREREZNxoiBYREREREREZkYZoERERERERkRFpiBYREREREREZ0VgO0b//+7/PddddR5Zl3HrrrTzwwAO7vUnb4uTJk9xyyy3MzMxw4MAB3vCGN3D69OnL7vP3/t7fwxhz2Xrzm9+8S1t8Zf7Vv/pXz3osN91008bXB4MBJ06cYN++fUxPT3P77bdz/vz5XdziK3Pdddc96/EaYzhx4gRQr307KrVcj/2tlpvdsjqux75uWseglp9JLddjXzet5XHteOyG6D/+4z/mHe94B+9973v58pe/zCte8Qpuu+02Lly4sNubdsXuv/9+Tpw4wRe+8AU++9nPUpYlP/VTP0W3273sfv/8n/9zzp07t7E++MEP7tIWX7mXvvSllz2Wv/zLv9z42tvf/nY+85nPcO+993L//fdz9uxZ3vjGN+7i1l6ZL33pS5c91s9+9rMA/MzP/MzGfeq0b5+PWq7X/lbLzWxZHddrXzepY1DLT6eW67Wvm9Ty2Hbsx8yrXvUqf+LEiY3/bq31hw8f9idPntzFrdoZFy5c8IC///77N277iZ/4CX/nnXfu3kZto/e+973+Fa94xXN+bXl52cdx7O+9996N2x555BEP+FOnTr1AW7iz7rzzTn/DDTd455z3vl77dhRquT77Wy03t2V1XJ993fSOvVfLavnO3duobdT0lsel47F6JbooCh566CGOHz++cVsQBBw/fpxTp07t4pbtjE6nA8DevXsvu/0Tn/gE+/fv5+abb+buu++m1+vtxuZti69//escPnyYF73oRbzpTW/i8ccfB+Chhx6iLMvL9vVNN93E0aNHa7Gvi6LgP/yH/8Av/MIvYIzZuL1O+/Z7UctDddrfarl5LavjoTrt66Z2DGpZLddrXze15XHqONrxP2ETnnzySay1LCwsXHb7wsICX/3qV3dpq3aGc4677rqLV7/61dx8880bt//Df/gPufbaazl8+DB/8zd/wzvf+U5Onz7Nf/kv/2UXt3Zrbr31Vj72sY/x/d///Zw7d473ve99/NiP/RgPP/wwi4uLJEnC/Pz8Zd+zsLDA4uLi7mzwNvr0pz/N8vIy//gf/+ON2+q0b5+PWq7X/lbLzWxZHddrXze5Y1DLark++7rJLY9Tx2M1RDfJiRMnePjhhy97DwPAHXfcsfHPL3vZyzh06BA/+ZM/yTe+8Q1uuOGGF3ozr8hrX/vajX9++ctfzq233sq1117Ln/zJn9BqtXZxy3bev//3/57Xvva1HD58eOO2Ou1b+Q61rJYned/KkDqud8eglptCLde75XHqeKxO596/fz9hGD7rCnLnz5/n4MGDu7RV2+9tb3sbf/qnf8r/+B//g2uuueZ73vfWW28F4NFHH30hNm1Hzc/P833f9308+uijHDx4kKIoWF5evuw+ddjX3/rWt/iLv/gL/tk/+2ff83512rfPpJafrU77Wy1frk779unU8bPVaV83pWNQy2r52eq0r5vS8rh1PFZDdJIkvPKVr+S+++7buM05x3333cexY8d2ccu2h/eet73tbXzqU5/ic5/7HNdff/3zfs9f//VfA3Do0KEd3rqdt7a2xje+8Q0OHTrEK1/5SuI4vmxfnz59mscff3zi9/VHP/pRDhw4wOte97rveb867dtnUsvPVqf9rZYvV6d9+3Tq+NnqtK+b0jGoZbX8bHXa101peew6fsEvZfY8PvnJT/o0Tf3HPvYx/5WvfMXfcccdfn5+3i8uLu72pl2xt7zlLX5ubs5//vOf9+fOndtYvV7Pe+/9o48+6t///vf7Bx980D/22GP+v/7X/+pf9KIX+R//8R/f5S3fml/6pV/yn//85/1jjz3m/9f/+l/++PHjfv/+/f7ChQvee+/f/OY3+6NHj/rPfe5z/sEHH/THjh3zx44d2+WtvjLWWn/06FH/zne+87Lb67ZvR6GW67O/1fJ31G3fPh91XJ993cSOvVfLl6jl+uzrJrY8jh2P3RDtvfcf/vCH/dGjR32SJP5Vr3qV/8IXvrDbm7QtgOdcH/3oR7333j/++OP+x3/8x/3evXt9mqb+xS9+sf/lX/5l3+l0dnfDt+hnf/Zn/aFDh3ySJP7qq6/2P/uzP+sfffTRja/3+33/1re+1e/Zs8e3223/0z/90/7cuXO7uMVX7s///M894E+fPn3Z7XXbt6NSy/XY32r5O+q2b0ehjuuxr5vYsfdq+enUcj32dRNbHseOjffe7+xr3SIiIiIiIiL1MFbviRYREREREREZZxqiRUREREREREakIVpERERERERkRBqiRUREREREREakIVpERERERERkRBqiRUREREREREakIVpERERERERkRBqiRUREREREREakIVpERERERERkRBqiRUREREREREakIVpERERERERkRBqiRUREREREREb0/wEAqTfpN8aiXgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1200x800 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "showObservation(obs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Network Model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The chosen network architecture for our implementation is an Actor-Critic network, which closely follows the structure outlined in the paper \"Playing Atari with Deep Reinforcement Learning\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_shape = env.observation_space.shape\n",
    "n_action    = env.action_space.n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_network():\n",
    "    \n",
    "  # Input layer\n",
    "  inputs  = Input(shape=input_shape)\n",
    "\n",
    "  # Convolutions on the frames on the screen\n",
    "  layer1  = Conv2D(16, 8, strides = 4, activation = \"relu\")(inputs)\n",
    "  layer2  = Conv2D(32, 4, strides = 2, activation = \"relu\")(layer1)\n",
    "\n",
    "  # Flatten and add densely connected layer\n",
    "  layer3  = Flatten()(layer2)\n",
    "  layer4  = Dense(256, activation = \"relu\")(layer3)\n",
    "\n",
    "  # Actor and critic layers\n",
    "  actor   = Dense(n_action, activation = \"softmax\")(layer4)\n",
    "  critic  = Dense(1, activation=\"linear\")(layer4)\n",
    "\n",
    "  return Model(inputs = inputs, outputs = [actor, critic])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      " Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      " input_1 (InputLayer)           [(None, 4, 84, 84,   0           []                               \n",
      "                                3)]                                                               \n",
      "                                                                                                  \n",
      " conv2d (Conv2D)                (None, 4, 20, 20, 1  3088        ['input_1[0][0]']                \n",
      "                                6)                                                                \n",
      "                                                                                                  \n",
      " conv2d_1 (Conv2D)              (None, 4, 9, 9, 32)  8224        ['conv2d[0][0]']                 \n",
      "                                                                                                  \n",
      " flatten (Flatten)              (None, 10368)        0           ['conv2d_1[0][0]']               \n",
      "                                                                                                  \n",
      " dense (Dense)                  (None, 256)          2654464     ['flatten[0][0]']                \n",
      "                                                                                                  \n",
      " dense_1 (Dense)                (None, 12)           3084        ['dense[0][0]']                  \n",
      "                                                                                                  \n",
      " dense_2 (Dense)                (None, 1)            257         ['dense[0][0]']                  \n",
      "                                                                                                  \n",
      "==================================================================================================\n",
      "Total params: 2,669,117\n",
      "Trainable params: 2,669,117\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = get_network()\n",
    "model.summary()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Proximal Policy Optimization"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameters"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have adopted the hyperparameters recommended in the paper \"Proximal Policy Optimization Algorithms\" with a few modifications. Unlike the paper, I have chosen not to multiply the learning rate and clipping parameter by an alpha value, which consisted in gradually reducing it from 1 to 0 over the course of the learning process. Additionally, I have decided to use a higher learning rate based on extensive experimentation and time constraints, which has yielded satisfactory results. According to the paper's recommendations, it is advised to allow the agent to accumulate experiences for a fixed number of frames before applying a network update. It is suggested that this number should not surpass the duration of an episode. In this case, a horizon value of 64 frames was selected, despite the possibility of some episodes being shorter based on the agent's chosen actions. Notably, this parameter choice has yielded satisfactory results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "T                   = 64\n",
    "ppo_epochs          = 3\n",
    "learning_rate       = 1e-3\n",
    "mini_batch_size     = 16\n",
    "gamma               = 0.99\n",
    "tau                 = 0.95\n",
    "c1_coeff            = 1\n",
    "c2_coeff            = 0.01\n",
    "clip_param          = 0.1\n",
    "\n",
    "optimizer           = Adam(learning_rate = learning_rate)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The compute_gae function computes the Generalized Advantage Estimation (GAE) returns in the Proximal Policy Optimization algorithm. The function begins by appending the next_value to the list of values to ensure consistency in the number of elements.\n",
    "\n",
    "Next, the function iterates in reverse order over the range of the rewards list. For each step, it computes the temporal difference (delta) by subtracting the estimated value at the current step from the sum of the current reward, discounted future value, and a bootstrap term that accounts for incomplete episodes. The GAE is then updated by adding the delta, plus additional discounting factors (gamma and tau) that weigh the importance of immediate and future rewards.\n",
    "\n",
    "Finally, the computed GAE plus the estimated value at the current step is inserted at the beginning of the returns list. Once the iteration is complete, the function returns the list of computed GAE returns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_gae(next_value, rewards, dones, values):\n",
    "    values  = values + [next_value]\n",
    "    gae     = 0\n",
    "    returns = []\n",
    "    for step in reversed(range(len(rewards))):\n",
    "        delta   = rewards[step] + gamma * values[step + 1] * (1 - dones[step]) - values[step]\n",
    "        gae     = delta + gamma * tau * (1 - dones[step]) * gae\n",
    "        returns.insert(0, gae + values[step])\n",
    "    return returns"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ppo_iter function generates minibatches from input batches for the Proximal Policy Optimization (PPO) algorithm.\n",
    "\n",
    "A random permutation of indices is generated using the batch size. This permutation is applied to shuffle the states, actions, action probabilities, returns, and advantages arrays accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ppo_iter(states, actions, actions_probabilities, returns, advantages):\n",
    "    states                  = np.array(states)\n",
    "    actions                 = np.array(actions)\n",
    "    actions_probabilities   = np.array(actions_probabilities)\n",
    "    returns                 = np.array(returns)\n",
    "    advantages              = np.array(advantages)\n",
    "\n",
    "    batch_size              = len(states)\n",
    "    p                       = np.random.permutation(batch_size)\n",
    "\n",
    "    states                  = states[p]\n",
    "    actions                 = actions[p]\n",
    "    actions_probabilities   = actions_probabilities[p]\n",
    "    returns                 = returns[p]\n",
    "    advantages              = advantages[p]\n",
    "    \n",
    "    for batch in range(batch_size // mini_batch_size):\n",
    "        start   = batch*mini_batch_size\n",
    "        end     = (batch + 1) * mini_batch_size\n",
    "        indices = np.arange(start, end)\n",
    "        yield states[indices], actions[indices], actions_probabilities[indices], returns[indices], advantages[indices]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ppo_update function performs an update step in the Proximal Policy Optimization (PPO) algorithm.\n",
    "\n",
    "Within each PPO epoch, the function utilizes the ppo_iter generator to obtain mini-batches that are used to compute the loss and update the model parameters.\n",
    "\n",
    "For each mini-batch, a gradient tape is initiated to record the operations performed. The current_actor and current_critic are obtained by passing the state_batch to the model. The old_probs and new_probs represent the action probabilities corresponding to the selected actions in the respective batches.\n",
    "\n",
    "The ratio is calculated as the exponential of the logarithmic difference between new_probs and old_probs. The surr1 and surr2 terms are computed based on the ratio and advantages_batch, with the latter being clipped within the specified clip_param range.\n",
    "\n",
    "The actor_loss is computed as the negative mean of the minimum value between surr1 and surr2. The critic_loss is the mean squared difference between returns_batch and current_critic. The entropy_values represent the entropy of the current_actor probabilities, and entropy_bonus is the mean of these entropy values.\n",
    "\n",
    "The total_loss is calculated by combining the actor_loss, critic_loss (multiplied by the c1_coeff), and entropy_bonus (multiplied by the c2_coeff)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ppo_update(states, actions, actions_probabilities, returns, advantages):\n",
    "    for _ in range(ppo_epochs):\n",
    "        for state_batch, actions_batch, actions_probabilities_batch,\\\n",
    "            returns_batch, advantages_batch in\\\n",
    "            ppo_iter(states, actions, actions_probabilities,\\\n",
    "                     returns, advantages):\n",
    "            \n",
    "            with tf.GradientTape() as tape:\n",
    "                current_actor, current_critic = model(state_batch)\n",
    "\n",
    "                old_probs = [prob[actions_batch[i]] for i, prob in enumerate(actions_probabilities_batch)]\n",
    "                new_probs = [prob[actions_batch[i]] for i, prob in enumerate(current_actor)]\n",
    "                \n",
    "                ratio = K.exp(K.log(new_probs) - K.log(old_probs))\n",
    "                surr1 = ratio * advantages_batch\n",
    "                surr2 = K.clip(ratio, 1.0 - clip_param, 1.0 + clip_param) * advantages_batch\n",
    "\n",
    "                actor_loss = -K.mean(K.min([surr1, surr2],axis=0))\n",
    "                critic_loss = K.mean(K.square(returns_batch - current_critic))\n",
    "                \n",
    "                entropy_values = [entropy(probabilities) for probabilities in current_actor]\n",
    "                entropy_bonus = K.mean(tf.convert_to_tensor(entropy_values, dtype=np.float32))\n",
    "                \n",
    "                total_loss = c1_coeff * critic_loss + actor_loss - c2_coeff * entropy_bonus\n",
    "        \n",
    "            gradients = tape.gradient(total_loss, model.trainable_variables)\n",
    "            optimizer.apply_gradients(zip(gradients, model.trainable_variables))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, we initialize several variables that serve the purpose of evaluating results and tracking the training process. These variables are essential for monitoring and assessing the performance and progress of the training algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "frame_idx               = 0\n",
    "max_frames              = 2e6\n",
    "agent_victories         = []\n",
    "opponens_victories      = []\n",
    "ties                    = []\n",
    "consecutive_wins        = 0\n",
    "early_stop              = False\n",
    "n_games_test_training   = 5\n",
    "positive_tests          = 30\n",
    "frequency_test          = 2000\n",
    "scores                  = {\n",
    "    \"Player 1\"      : 0,\n",
    "    \"Player 2\"      : 0,\n",
    "    \"Tie\"           : 0,\n",
    "    \"Limit exceeded\"  : 0\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The test_env function serves the purpose of testing the environment at regular intervals throughout the learning process. This function is executed after a fixed number of frames and provides valuable insights into the progress of the training. By evaluating the environment during these test intervals, we can assess the effectiveness of the learning algorithm and gain valuable feedback on its performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_env(consecutive_wins, n_games_test = n_games_test_training, vis=True):\n",
    "    for _ in range(n_games_test):\n",
    "        state = env.reset()\n",
    "        if vis: env.render()\n",
    "        done = False\n",
    "        while not done:\n",
    "            state = tf.expand_dims(state, axis=0)\n",
    "            dist, _ = model(state)\n",
    "            next_state, reward, done, info = env.step([np.random.choice(n_action, p=np.squeeze(dist)), 4])\n",
    "            state = next_state\n",
    "            if vis: env.render()\n",
    "        if info[\"victory\"] != None:\n",
    "            scores[info[\"victory\"]] += 1\n",
    "        else:\n",
    "            scores[\"Limit exceeded\"] += 1\n",
    "    env.close()\n",
    "    agent_victories.append(scores[\"Player 1\"])\n",
    "    opponens_victories.append(scores[\"Player 2\"])\n",
    "    ties.append(scores[\"Tie\"])\n",
    "    if len(agent_victories) > 2:\n",
    "        if agent_victories[-1] == agent_victories[-2] + n_games_test:\n",
    "            consecutive_wins += 1\n",
    "        else:\n",
    "            consecutive_wins = 0\n",
    "    return consecutive_wins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the training process begins by allowing the agent to interact with the environment for a predetermined number of frames, following the current policy. This interaction gathers data, which is then used to calculate Generalized Advantage Estimations (GAE). Subsequently, the network is updated using these estimations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "state = env.reset()\n",
    "\n",
    "while frame_idx < max_frames and not early_stop:\n",
    "\n",
    "    states                  = []\n",
    "    actions                 = []\n",
    "    rewards                 = []\n",
    "    dones                   = []\n",
    "    actions_probabilities   = []\n",
    "    critic_values           = []\n",
    "\n",
    "    for step in range(T):\n",
    "\n",
    "        t_state = tf.expand_dims(state, axis=0)\n",
    "        actor, critic = model(t_state)\n",
    "\n",
    "        action_probability = np.squeeze(actor)\n",
    "        \n",
    "        action = np.random.choice(n_action, p = action_probability)\n",
    "        next_state, reward, done, info = env.step([action, 4])\n",
    "\n",
    "        states.append(state)\n",
    "        actions.append(action)\n",
    "        actions_probabilities.append(action_probability)\n",
    "        critic_values.append(critic.numpy()[0])\n",
    "        rewards.append(reward[0])\n",
    "        dones.append(done)\n",
    "\n",
    "        if done:\n",
    "            state = env.reset()\n",
    "        else:\n",
    "            state = next_state\n",
    "        \n",
    "        if frame_idx % frequency_test == 0:\n",
    "            consecutive_wins = test_env(consecutive_wins = consecutive_wins, n_games_test = n_games_test_training)\n",
    "            print(scores)\n",
    "            if consecutive_wins >= positive_tests:\n",
    "                early_stop = True\n",
    "                model.save_weights(f\"player1_PPO_frame{frame_idx}\")\n",
    "        \n",
    "        frame_idx += 1\n",
    "    \n",
    "    next_state = tf.expand_dims(next_state, axis=0)\n",
    "    _, next_value = model(next_state)\n",
    "\n",
    "    returns = compute_gae(next_value.numpy()[0], rewards, dones, critic_values)\n",
    "\n",
    "    advantages = np.array(returns) - np.array(critic_values)\n",
    "    \n",
    "    ppo_update(states, actions, actions_probabilities, returns, advantages)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how our agent behaves at the end of the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.load_weights(\"player1_PPO_frame136000.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def play(episodes, show = False):\n",
    "    env     = create_environment()\n",
    "    wins    = 0\n",
    "    defeats = 0\n",
    "    ties    = 0\n",
    "    for _ in range(episodes):\n",
    "        state   = env.reset()\n",
    "        done    = False\n",
    "        while not done:\n",
    "            if show: env.render()\n",
    "            state = tf.expand_dims(state, axis=0)\n",
    "            dist, _ = model(state)\n",
    "            next_state, reward, done, info = env.step([np.random.choice(n_action, p=np.squeeze(dist)), 4])\n",
    "            state = next_state\n",
    "        if info[\"victory\"] == \"Player 1\": wins += 1\n",
    "        elif info[\"victory\"] == \"Player 2\": defeats += 1\n",
    "        elif info[\"victory\"] == \"Tie\": ties += 1\n",
    "    env.close()\n",
    "    print(f\"Games played: {episodes} - Results: {wins} victories, {defeats} defeats and {ties} ties.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Games played: 10 - Results: 10 victories, 0 defeats and 0 ties.\n"
     ]
    }
   ],
   "source": [
    "play(10, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results obtained from the agent's performance are encouraging. Although the agent may not have learned the optimal policy as it still fails to avoid all bullets, it demonstrates the ability to consistently win all games it plays. The observed outcome can be attributed to the reward design, which might not effectively incentivize the agent to avoid all bullets. By adapting the reward structure and incorporating variable frame skipping based on the players' positions, such as skipping more frames when they are far apart and fewer frames when they are close, it becomes plausible for the agent to acquire the capacity to play against itself and potentially even challenge human opponents. This adaptive approach holds the potential to enhance the agent's overall capabilities, enabling it to refine decision-making processes and optimize gameplay strategies. Through exposure to diverse scenarios and opponents, the agent can continuously improve its skills, leading to further advancements in its gameplay abilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources and references"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[PyGame Shooter Game Beginner Tutorial in Python](https://www.youtube.com/watch?v=DHgj5jhMJKg&list=PLjcN1EyupaQm20hlUE11y9y8EY2aXLpnv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "[Creating a Custom OpenAI Gym Environment](https://www.youtube.com/watch?v=ZxXKISVkH6Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[An introduction to Policy Gradient methods](https://www.youtube.com/watch?v=5P7I-xPq8u8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Proximal Policy Optimization | PPO Tutorial](https://www.youtube.com/watch?v=9DO63MSGeNA&t=301s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[OpenAI Gym](https://www.gymlibrary.dev/), [Keras](https://keras.io/getting_started/) Documentation"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}