ML-Intro-Course/18_Q_learning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5pjUZX5KM_mP"
      },
      "source": [
        "# Introduction to Reinforcement Learning\n",
        "\n",
        "<a href=\"https://colab.research.google.com/drive/1lledEJ2rc3SD39St7NnsM_vGmj4R-qrh\" target=\"_blank\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\">\n",
        "</a>\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle).\n",
        "\n",
        "Reinforcement learning (`RL`) is an area of machine learning concerned with how intelligent agents ought to take _actions_ in an _environment_ in order to _maximize the notion of cumulative reward_, being one of three basic machine learning paradigms, alongside Supervised and Unsupervised learning. An in this notebook, we will learn about $Q$-learning.\n",
        "\n",
        "![pacman-gif](https://miro.medium.com/max/1274/1*D7JNcbvhP5UOR6_Ul-WJaw.gif)\n",
        "\n",
        "[Source](http://ai.berkeley.edu/project_overview.html).\n",
        "\n",
        "$Q$-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. Model-free means that the agent does not require a model of the environment (the transition function that maps actions to world-states), and it can handle problems with stochastic transitions and rewards without requiring adaptations.\n",
        "\n",
        "The algorithm's core is the [Bellman equation](https://en.wikipedia.org/wiki/Bellman_equation.)\n",
        "\n",
        "$$Q(s,a)^{New}=Q_{old}(s,a')+\\alpha\\left[R(s) + \\gamma (\\underset{a'}{\\max}\\: Q_{old}(s,a') - Q_{old}(s,a')) \\right]$$\n",
        "\n",
        "This equation has guarantees with respect to the convergence of the $Q$-learning algorithm in finite [Markov decision process](https://en.wikipedia.org/wiki/Markov_decision_process), i.e., for any finite MDP, $Q$-learning can identify an optimal action-selection policy ($\\pi^{*}$), given infinite exploration time and a partly-random policy.\n",
        "\n",
        "### Frozen Lake 🥶\n",
        "\n",
        "FrozenLake is an environment from the [Gymnasium](https://gymnasium.farama.org/) (_a maintained fork of OpenAI's Gym library_), being a simple environment composed of tiles. In it, the objective is to move from an initial tile to a `goal`. Tiles can be a safe `frozen lake tiles` ❄️ or a `hole` 💀 (_terminates the game_). The AI agent has 4 possible `actions`: go ◀️LEFT, 🔽DOWN, ▶️RIGHT, or 🔼UP, and 16 possible `word-states` in the original environment (_if you want, you can customize your own more dangerous frozen lake_).\n",
        "\n",
        "```\n",
        "S F F F       (S: starting point, safe)\n",
        "F H F H       (F: frozen surface, safe)\n",
        "F F F H       (H: hole, stuck forever)\n",
        "H F F G       (G: goal, safe)\n",
        "\n",
        "👾 ❄️ ❄️ ❄️\n",
        "❄️ 💀 ❄️ 💀\n",
        "❄️ ❄️ ❄️ 💀\n",
        "💀 ❄️ ❄️ 🏁\n",
        "\n",
        "\n",
        "```\n",
        "\n",
        "One possible correct solution is RIGHT → RIGHT → DOWN → DOWN → DOWN → RIGHT\n",
        "\n",
        "```\n",
        "\n",
        "👾 ▶️ ▶️ ❄️\n",
        "❄️ 💀 🔽 💀\n",
        "❄️ ❄️ 🔽 💀\n",
        "💀 ❄️ ▶️ 🏁\n",
        "\n",
        "\n",
        "```\n",
        "\n",
        "There are different ways you can render the `FrozenLake-v1` environment. You can generate, for example, a `rgb_array` to plot a nice-looking image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "R1yjOs13M_mW",
        "outputId": "9c204380-4326-4bb7-f280-6e8b98fcfec4"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAADjhUlEQVR4nOz9WbBtSXoehn1/5hr2dMY71dxTNXpAN9jdIAARBEEQHE2AEi3StCJkWxEWrXCIDtsRlB18cIRo69UK2bLFMF/IF1OUaJiUSIIgAFLmBKAbQGPoeaiuru6qunXHM+9xrczfD7mG3HvlmvZw7rnV60Nc9K591s7xy5X5//kPxMyMDh06dOjQAYB41g3o0KFDhw43B92m0KFDhw4dMnSbQocOHTp0yNBtCh06dOjQIUO3KXTo0KFDhwzdptChQ4cOHTJ0m0KHDh06dMjQbQodOnTo0CGD1/TBX/zCe7tsR4cOHTp02DF+7iderH2m8aYAAGG/h6AXrN2gka8QyKID9cCXIKK1y901IqWxULrymYuFRKy30AdmXJ1fgZlBRBgdjIA1x0YS4yBUhe89IRB6N1tInEQKVc72sSZcLORW6lrM5phP5wA6jleh4/h28aw4XodWm0Kv72O0N4BiAGg/iUe9GD1/mXgE4LjvQSakEISNF49mhmus1y17FitMoiLxtFWHnEgs4mUSMtB6rFgzxhfjbMHs7w+gQeA1xtsTjFuDqFB9zxPYDwzZiACx4Xgz89JYpFi3bGbG+TyGdkxiWs88JtCkSF8NQHPLOpmzBdNxfLWe/HPH8eL3zyPH69BqUzgIFW4PI9y/8ts1KMHJTAIz184XwRNmXo/6/hrUWMZ4oTB3nHqOej7kGoWHUiCUxVPHlVXP7b4CsLyoGMC7l76TTE0gBOPFvQinMw/jqP2JIdZwztXQ11CDCADQ9wSGQSsaFKAZOJ1Fhe9DKbAXrlf2geN3ioGzpJ5AMl7aK9Z5sRA4m63fn47jy+g4bvB+4ngdWpVM2f9bF64fu088m2DbEf6anLxcj2zaL8KGw73xq6cZrmu8yRrQXWliOo5XPeNoR8fxtfAsOV6H3W03LRBpgmYzCLFm0MoUEBkd4So0M1RyRJGCMjFOEuCJ8hFlZsTrHm1W6k8hHWI7MxBKhmqxclgwtk/BZWgmzGPTVk8YffIq7PHM2samL2l3PEEgIhDc421/pTQ7ReV27c4/U9LGVQQCCGS1bnwV82tQPXcct37Tcbyi3fnnZ8XxG7ApEB4nejMz8BFWx8EThIOQCoScxxrjRA868CUGvsw+O2uyfn8xj7dKy1HgOUmjOWolWjMz3iOjN9wVprHANFkwB7FG3IsLz+yHHgKHHuJ8ZsbN6MmN2C4FOUVhG3OlnTrrdVFWZyA1el6xP5WYKpxuqV1udBy30XG8GZ4Vx2/ApgDkIuB2KPysrDyKpyhOvm9exvVlt9juGD2LMS8VwW+kkU/H8fw322xRFTqOr4PWm4IAMPA1sHL7HTOwUJvL4ZNIgIghAPQ8BpERYReqKHLa4vE0BubJ5hl6uuayjRFIAU50vVFSDgHwHT9UGrXicaQZmlfPPnk9S99adTZBKBlcKBuYxutZbNiIFGEcUVZPasUXKe1YvLz06WxmPhMx+l59f9JTWaxzSw5PUOHUDCCZ73IwM+ax+6zpOv3Zddah47gbHcffPxyvQutNQQrgzqAoIo0jwpPJZguGQXg6NU2SxHg5uXnXDFwuqsWlkwlwsTCfb/c1hkG1cHrc9yGIoDRnVgWCCHuBV9idp5HKRPgylImNaT02Yq1xNmsu/u2FGnvhcn+YgfeuPEQb2o1PYoFJYmZ43I+xl4zbNNaoEvCVBr5/aRaOJ4CXRnHl6aXvCeyHRhS/WsSYJWQf+BLBitULs5mTKoKrEk7Y9di4mEe1izBFx3E3Oo6/fzhehZvt3dGhQ4cOHa4VrSQFXwr0LC9Bzci8IH0B7AXVJ41pLBp7RDIDl4vinuUJxsAv7oahp7GXtTP/eyBFJrrNY13Q6BKZnZdR7oTiCVrqtwsLpZ27/izWxuyOjE0zkbF4SMvL1QZu9DwBYcmedj3DQGeWKS5oBsaRQFPd6iwmMBf7OfQ1Vk3YiYC9wIynpLwNgpCdipRmpwrBFyJjXpkBTc+TlZYcNvdsxJoxTU60vqTMosfwwJTn4kHWto7jpeg4bvC8c7wOrTaFQAqMLCcQ2zU+kIzjfvWCeTymxgtGg3DqcNDoexoDvyhSDXzGwC/W3/cEfCnAzFg4dIiCqNaxxZemjCqczSLnBKciNwEI+yKrMx3HhdJYKLeYTUnbbLO086QeIuAgrFYfLFS6YJphEklMCr4yjJ7HkCsUEwQcOebb7tssVogWxWdCTyCsEFKJqNS6JkVZWIZIMyJt6hxCZvrjnmfKS001y8w1O46Xo+N4+v3zzfE6dOqjDh06dOiQYSOTVCkIw2S3izU73e5tDAOF0GtmnUwgDHwjFmpmTGOzM3qi7IRHRmSDEWdTS4pprLN22Ycc16WZTETedUzP+p6ovDRiwHmRR0A2hpFmzHT1SbTnSQRJRyaRqhQRJQFHveZ204EU1hiqbAxt0dnGwJcgIJkfM8aaGVfJ5Zgt9kc6/361P1VOWGUQlHOvDHZbbARSIJRmTqY1l6sdx3N0HH9/cnwVG20Kggj9pNFzi5hlMHrSZiKNIOCoJ0AERIpxPq8u2xMia0ukGSrRYZZFfpw5TL0k1etVyxB69ZN3Mi3GMfEF4aCXODbFCo7wKiv1mPYx89KLwQUpgP0a8dvGKCAkTcH5TFeaFBLMS4KIjI4zWzDusS0TZ41utv2CkYLQF9VjbluA2DgIvUzdMourF0zH8Rwdx9+fHF9Fpz7q0KFDhw4ZtubR7Alj/9wU4yiuFEVtu+0mHpDzWCPWZscsu2AZ+hKCCAzGleNySDPX2oqHUq4Vp50A5/jYUqUvRGJD3qzMYSAbe4fGWmcnnTJMo/xiq+6SilE/P54g9JPT5UK5T9mTSFWeZAiEUbBeLoKel6sKbNiXmqPAa3xy7jhejY7jzz/HgS1uClKQM3hTGaYx1QaPauOIoZihaqSkQApIQWBmjFHUVXKDOsv0vXUgIoRe9fhIQQg8ARN+q7oeInJ6NZY+r1C7YJqMoY26sZJE2ctFMa9GXQZQvzBN4Lj1Eo14QqBqLaRj2JS3Hcer0XH8+ec40KmPOnTo0KGDhVaSwjRSWQIIwOyS64o9oxZi4Sa4WijnRdV+En2wSXgBG7NYOS/2RoF0hj5ug3msMV7EWZAxZsb5LFpy7Bn6stae3AWvQZTHbSDW7LRACaWAn/RjGqvGJ2QGcD4vzo8naMmfYB1wokqxT5cdxzuO1+F553gdWtWoV2O0b8CPTcnVFESqIKUSURYUrMpb0gXNcKoEtrH4GWZ87aJizUvDvG41gghinZRcreEmnxQEmVhgGL1r855sIy9AVdnaKr/jeMfxejzfHK/DVrZVQ5YK067klFWVpJqInH+vOqGtPt/0NFfVDvfzDZ6pKHfbIXfrxjuts268TVnNx7Dt/NT9tvzZps9tabwbPN5xvON4HZ53jqdotSkMfImD0MtEnUkEvH1h/tb3GMd9l2u+zNyvx5FbLD0I87yyZ7MYDIagdkkt7Hps2NYQ9l2LXc9hr5gzN9I6s96YRAJn8+qLoFjHCFdC6xKMfXbT+QikwGEvjTjJIBCOej6mKhdF3zoD5onkem8YFy4F7XHTDJzP3Xllh5mbvs6cpoa+rLVFT5ONl82PJ0ybgeWY73Y9fU9mz9i4mMeZGuTh2IOqSE4eSobmYt96nqwNH2DjIPRx6i2y/+44Xo6O4wbPO8fr0C5HMy2nr2M2sUcAwCM4ze/sjU5zdbxvTp7h7L+qwZzXWVZu2a27XY8rzaA9WZpRG89GsasN7U5rgmg5KBcBQhBI5ZYakcrHPNbFYFv2yYLhHhf7O/uZJq1ViehfdYJxSfCrbVmdl9XyYk2VC0aSm0ttTmtpW+2p7zhejo7jSZOfc47XYS31UVo+UT5hgty5pdJF0LjshBtN+kBWnW2F1yb12P1czakLpP1MRdQiOSn93koTso6YbfdTCEAkB9GqMWfmxuRp06Js3NbQFtjjWVY2sXnQfC4f86qXQco3wvpqjY7jBh3H2+F54ngZ2mdeI+Con4tFd4emQQQCkfl+Fuci7yRSWeyNuukjAIcOkasMoRQIkqiMbYelrh5fUNbPvq8x8Iti3NOpxCQyYlyag9eGIIARQSYvljZ9szG0cvMe9vJxFJTXeTqNzEJJPjdF38tDRTcZw3X70KSe/SVRvZiAJNKEB1dp5E3COxfFtuyHGkfK9H/gyywsRBt0HM/Rcbw5nieOV6H1pkBESx3OJaT829UBaXqGWi1728+3+Z39DMEdE93+ypUyULNJhcglaoemWGqvY7xXT0ttqmozhuuOd5Pf2n9nZud4i6Weuc615tS6wVAX2gJ0HE/RcXyz394kjleh9abAls7UiNbFntkitw2bNIRcxNJG/ixFWT3bRpO+tYViAmtzaaWNzFv5vMt0LF14QCJOO9qVhjawsUqedE6I6s0Uy+rZNpr0rVV5MHpowOjAm5hjrk5Jx/F26DhejZvI8Sq03hQ0I8v3mjqLrHYylAKhw/nkchFnFgb7oZeFk63LVVpWz7ahmLO8soEkZx7UNmCYHLOACfFLiGr1lFpzQT89Xqgspsp+6Dld/w97xam0++MLykTXudLZHJahrJ5t48rixGHPg7fhHF8tBK6SbGbHPYW9sF7NMFmJS9NxvDk6jtfjJnK8Cjtx/2tK7Do742eJ7TRrWfjeFUrHe/VS8BpORW2x/em3VA6NG7FGLR3HE3Qcr8PzxvFWm8JCGbvtuSKAAW65y0qiLHiTZkZUEZueGVhoU4+WqccfJyZtu5p4yk52TerwBBBI04dIkVPnmoJhLo7qsGRKx2a8ZzFlv+WgtogMBGT9EUTZeJedWGMNqMQsMfIYlOg7PEE7W2xS2HrT6jqITEpMgKG5Pu2l0oR5XN9u2yyw4/gyOo5vjpvG8Tq02hQejoErQdnPhj5we9C8Mtvh4nweV7p2K0Z2Cx9IBiFOBmxzkbcMbWOnHIQKB6H5/HDsmRdJCTQTHozry2ZtngUYSgPffAoQCaTxFvZCoGkTpdWfSLMzvoqN85nEVZQmW4/R97cn8pbBzixVtyg9AbwwNH2YK8LDcTUPLhYCF4v6UBPjef5Mx/FldBzfHDeN43VYU3203uCttxMb8lwX2rSxfXea/MDV1/XJmvensX3M2nWtg7acaPf4Jn3pOG6ebV16g2c6jlc/3+rpVmU3wUZ3CrEGzufmsyROxB6TIq8uGJgvKLMUqIsmqJkwTUSkhcrzxJ49fA+P3vrO+h0AEAQhfuQnfhSior1S5HHdY+32Mgw9hiAjis7iajF7E1wtUtGY0fM4s3Dxa8RfkZxAAUBpVKY4BIC5EuBEtLY9OPdDt9XNtuELAYax2nClTJTE6Ce5kBUbXuwCz5rjgQQGuxEaltBx/AeX46vYaFOYK+A7p+bzwGPcScSenicwCqo7kMYlYeZay4xYEx5Piivjt3/l8/iVv/l/X6/xCY7v3Mbf+Id/t3JTCKRI8qwC45IwtGnycGbg/pWPFpFqW+G9q/zziyOVvaSO+8XYNjY8IbAf5gnLXVm5bJzPJdLEH48n5jsC8Km75kW1SxAR9rLYNiV5fyVwN+HbeCHwZLqbBfOsOX5nALx2sF7b26DjuPnuB5Hjq+iS7HTo0KFDhwxbM0mNNHAxT3dpgUmy8Y2C7Yq/Zw/ewxu//XkAwNNvfhUvDgYAgGEo0K85uaVgBk7HJn9uj4Bf+YX/HkJK7B8e4Cf/5B8DEUFpzqJdGtHalO0JAYe5NBZKZyfBoa+g2VgcjBdiZ2L2OBKYxaZ0pXOHqlv9dXTB9WAAT6coBAMTxBj6OvlMa+X3ZealMex5IvMAdeWX1cyZSsaTjL3AnAoXijDfkZh9XRy3MY2BR+Pi931PZ9FDQ0+s5YTWcbyIH3SOA1vdFAROZ8WGvrK33QXz6K3vZCqjFwcDvH64DwB44cDH3f1mtmxKM77x3gRKA3Ol8Lf/8/8nGMCHPvY6fvJP/jHzDOfZlXpeLlqHnkDoELDOZhE0M4iAw15uFjeJxI6ybxEurFDHJzPzv54Ajvu7u0q7f1n8zhPASyMFImPdss6CAYBJpDM9cCBFFtnTlX0qUhoLZUTrUDLCvpmr87nY4aZwPRy3cbUw/1Zxu68xDHJzynWSy3Qcd+MHmeNApz7q0KFDhw4W2iXZ8TQOe7kdsNKEy4VA1Z59NjdiN2BEzvTSqOfJzHlm4MtMpEojTwoyF1tcUc8wFHjhwBzRRmHzmyFBwL39AJoZ01iAHuax0MeRMqKqdfSJFGPsyHEbeiKzQOl7AqmByTQy7SYYO+/VQ5QgQt8z/VHMmFku6KwZT8j8gohxGCqQtXVfLWSlQ4tm4N1LU7ckxn5oypaCsgQtnqDMnj5SOrN8GPgaXjI/dfWs1nk2k0nf3CdbGwNfI5TFW0o79ME0VgVWUTJuac6DtA9KcxYioSd5iaMpelJACgGAMYv10vzqKSM9HN40jpdhHInM2WsSUaW1jEh4sPpIx/EfTI7XodWm0PM0DsK8o/M4JXI5bPH39kBj6FuiE4yJWTqRmjkjmyATIraqnn4gGquMbBARbu/5SfvizEw8rb+YjIQxjR0mYyL3Xk2zOTEz5rGCYqPz3A+LxJBEOOwJEBlTxYt5/ozWuXbWjIFaSmo+iwTiiheH5lwH7QvAEzoz58sXjMjaPQEQ6SRblM/oJ8t7FosWC4ZwsbA25Xn188c9jT3HuNiYOUxbBCF50ZixHwhT5zxWWaau0ONCdjAA2AsEQs/cJ53P9ZJT2cRavDeN42WYxgLT9D9qXlCSAD/hQRk6jlfj/cTxOnTqow4dOnTokGGji2ZPMG73m0ffC6W1c0UKVNgpVwPjFut5ULP7boJYE55MvIIE3/MYe8ml3nghMIlTBxnGwjdi3MCXWQyWYeCZrFAw0R9X+6SZcZnYULfJ2AUABz2FkW42BoLysmPNuHCEACgLu3sQKowSa4vTmWwVO8WFvqcz642FJjyelKv7CEatIoU5FZ5MJRjmss8TiaqAcj8ATwjsJQLjQuVito1pnEfhbBJqOMWz4PhCrZxK1wLjuK9glAnAk2l1eR3Hf3A5voqNNgUpkFlAtEWUBP9qW0/o7W5TMHcKoqA+EpS/FBaaskxUfS+GTPIGpvpTotwzlJkxRvGFwoAzuXsT9Lzm47ZJnWk9RhQVGy8YX3I2hzMrm1dZaw8tR6lJZEwePcGYx1FmAZJCCoJMxGzFCo4hz4LNtcWz4DhFBGy4KRDM/YgUxiv76dRH1X1Fx/EfXI6volMfdejQoUOHDO3TcWI1z2g1prFe+8TwLBFInbn1S0tEvTMAXhiZz76QkJRfCrnQZqwAI/atXgj2LRvyOmg2iV62iVsDBeZyFYrSlKgnNjlpMW71VeaQlZrdSwF89NgUTQBCacaz7NI0lAJ+y8A1pys5bp81x0PJuDesTpxyPpOYbWir3nE8xw8ax6uwRo7mdrHHXbqv5wGSUvFyGcsByghVJCEi+C2digQxaCVqphQEv/GC2a4XEdGyntyFSDHSGDKbIJB5wLmsfhiPYUO36vEGEjG75cJdnaJnzXEpACmqx/xSsFOF0KqejuMAfjA5XoWdZF6zsSuvQ8BQys5q1XQRL2XCWuECub7s0BBtxu39M8a75Hg1mo+heQW/f8b82eH9z/Gdbwp9X2a2w22guT5hxtk4xjcfGGvtO3s+bo2axRpgBr7zaAaV2Fune4QvGC/tGbHd3l8CKbJEGTcw298SjFXDejEXxpFaSw1iQgBUqztsafcwVNgPisdcO3LAQehdSyL7bWCXHC/DcU/hMKwWFdIxlwS86JifjuPN8YPE8Z1vCoJovaNUg3lTGlgkDjdtTLBSKwWll+PcG7UBO6yPmqUuvAmw48K3/u3adZowv00hRb0gvtuUlNvFLjlehiZjmKLJ/HQcr6vzB4fja20KLsunNHrhNrGsdcy/S8FgKJ3b5Vblw7WhmaG0+beqn6xTQa2ThH3T3K+azQa4CkHbPdXdlEPL0hw/g/EGni3HnzU6ju8eN4HjZWi9KSwU8OVHxe8Pe8AHD7fQogSCgKN+UUS0owk+nEzxZGpCJ4pH1GrCU8nCXvueIBw76rSLncYa06jZDZ+JJlmdGKQJ3rlAFqbZxsdvwxnieF0MfbmUY9iF02m08Uts4EtnqGAb6ZgxTHTOpuum54nM4WddPGuO25jFOouVtC4I9fV0HM/xg8DxKrQumQG4MgvWZBtsjTTOePH75bbE6UhuySKBUL0Lp16cTbAtIwnN7vHddrjisjHP69veGLfRpW7gnLkWnjXHl57ZSj31vLbRcXwL9eBmc7wKa/kpGH1e0awqjdooaPlCZVcQRPCSgY85VwUFvR6kX+za7GqcTbovRCa6Rw1d6gEk0Qurn7EnWDMXiN12kQpatiNPEet8zH1xPaKxINrYiqVtO+vGmzlnI8N9v2TUEM0qvkkc34bKqu1lZsfx9z/Hq9B6U5AEvLxXlPMmscBXHpni7g6BV/Y3blst7vV7+PCBqeiti0u8OzaJVn/0z/xxvPrJjy0/rBn/5G/+bVydnkEQ4XN3bsETAgul8NuPnjSmQN8TWRTDMpzNouzUczYrWpeYCJLNh/6opzInIxtvnfmIkmxUn7rT7iJsXbRp9zbQxNJkrnSWj3cWa2f0yb3AQ+g1WzA3ieOhFAgb2u9vCx3H3/8cr8Jazmtu5HvrbrIwuUDOUxAJgpTL7GFaHsQ0XnnbnXUXF9H1dbrqMSPOoGu7qdzl5dYmdW67VTeJ4zdxzDuOX3+d19mqjbZEzcjikdtxyWOdXxr5ovnuzsxLiSGk46XthyGOXrhrPmvG1cJU1EYFBGaMoxgeEWICDu/dBQjYv3WreRkwIpxLzKzjLyPp56rlU4OpjxTAoOSCPH9+GptELwRzMdeU29pSuwlyb7LbwDbqWeVHiia6fs2M2MERWdOOZ8HxbWBb9XQcb47nleOr2GhTiBThwbhYxMksz6f6yh5wb9SsPAZwPouzBCQukereB17Fn/7L/wsAwLd/+3fxxX/6z1u3WwP48tMTAEB/b4Q/97/5jyCkwNHhQatyJpFaK8SByeLkyHIlRa1lxMnUw0wtTzID+LbpDjwBfPpu85OFbd0yCtZzwmpbz9CX6K+hBygbtyYYRwpwWLfUie3PguPbwjbq6TjeHM8rx1exJeXZ9YpcWz9ZbemC5vrwPLX1/YJuzK8X3Xg/K7TaFKQg+IKyfKdNMIuB8+RENfCrxWwC4EsBZi4VvcIwxAv37gAALl99GS++/uHCM/2R49hGhLsffBV7t46Xvh6Mhnjx3l0IKTAcDLPvjShW3c9ZDEzjzcgrCVlqPc2MhdJL4rovBRjc2GTNxIY3MdEF2YG2yutPIy7u0uXersflxcnJeFd1M9aMaUQbq5d7HmfWHpHWS5YcN4Hj20JdPR3Ht4vnheN1aJmjWWIUeDidVccAsfFkav4BwIcPgaN++bNEVBuG99bxEX7q3/pxAMALd+9i74V7jdpBRPiJn/8zhe97vRA/9Yd+HEIsW1uUZXGy8XQqMY42E7Z6nsY9z9QTacY8jjM1bDoebUR4xcCbp2nZwCdvVz8fejLLvbtLNKnnaqGc+tQUkQIeTaqTxdSD8cIwzl5SVwu1ZMlxEzi+DTSpp+P4dvG8cLwOXZKdDh06dOiQYaMjgBCMUZJ+LtaEWVy9x1wsjNUGAAx8vZRfdR0EgwFefPnl1r8jINvRfd/DLNYgMmJ2IClz3nG5qUcqtw7oeQyiinAAbNJ7csWurzThcm7q8SUjqImj3/c1vCQe+yQS0BUpBJUGnhjXDUjB6O8wlWkdPCtefqR0prYIpMhE7dAThVhUzHm+AqMqqO7DQhEWNclnJrEwgRAJWU7dMjxrjq8LkyPApJZlNjl9mdFxfId4Xjle6Eerp1fgC+BWkmx8HNUvmHTyAOCFocrEm3Ux2D/Exz55uNZvj/s+BBGU5kxVIIkQyDxR9igo9me8iDFNIrOOAo0qoxPNZnKqzLojTTiZmTpHvkLQq57A/dD8nRmYx1S5YCINfP/CfO5Jxr3RhllZNkDfE9mCmavc+SZNGEJETqsUzYz5NH0251sZzueiZsEQLuZpPfUvkWfN8U0Q9I3XvmZkjk8dx3eH55Xjq+jURx06dOjQIcNaHs3pbqeZW11g2LhcCEw2TLMaSo2BXzyijBcCi8TRaC9QlTFq7P40iTETJCL5KmaxKlhPEICD0CTx0SBczAXWuUAKvVz8dNXTBJEmnM42u2wjAPuhKoyT0sDFwpTtCcZejfgbSJFZgdQ51hBQerpyca8nGYehIZYRodufezqOdxx/v3O8Cq03BWGJQJFyx+BognG0uTXAXgAM/KKoNYkJk6T8vqezRNkuiBKRrgy+FE6Tw4XSjtwMuSgcayQLpj0CKRAkdUaOeppAsS1SrgvGKCguGM3IXgah1I0WTNCwKWUidxn3Qo8zlY2aEhZraBM6jnccf79zvAqd+qhDhw4dOmTY6KJZCsIoSEUqgNmINLOYtnJKciEQGnvJycS3TkfjhcicbALJ6Cd20Qdh8dRD2L6/5MCXlacbzTD5oFe+jxRlYulMCTyZepWic9+XCJN65kpnsU5OprLSAmR9MA5DBSnSGPHmW82mTsCcFtPLsUAi44QNb8spy2zulbacGYEs6m/OZxJxcnl5MvUwjsrPRh3Hc3Qcf39yfBWbmaQSZXFECBpxkqhas8C4ue9PK0jhNtlaqHyR3vZiDJNnep6XWQTsEkFNHZoZw6A4KNMoXzCxJkRxqpN1r5q0HmbG0I+hklj2pzO5s8idfZ8RyKIZ3SQxRfQE47gXJTmuCT1v985ZNvfKEOsYtBIdl9no+qHMa3MaUxavv66ejuMdx9+PHC/Uu05jO3To0KHD+xNb2+58SThI3Op9wQhk8cRwNpOYb3hTPleEB1emnoGvs0uuUaDQ94sidwpmrnUxb4KeJ9aKsmisNEy7VdIWwKgB7g3zsdKa8aShw1MqWjKASMcFkTzWhKdTiU0VCU+nMsmiBdzux5DCiNh3h0ZsLSt9obaTX3g/9NYKWNj3ZJagZhwpxJpBBBz3FZjzdj2eaVw2KK/jeDU6jq+Hm8RxYIubgiCCkIm+Uyv0HE47cgsqN82Eucr1qil8Cfg1YaQirTfOhbru74kIfjIAZMU8l8J4Ymbl62ZaU7s8ZkYoi4G2FooBbKr3tkXPvA4iOOfYRpOAa/W1r4/UaQgARJwvkHBFTVBluWOj43g1Oo6vW/v62DbHgS1uCs1hN65cr7jTFmxYZf778oJuVijujOYr/90OjRZyWQvWftEkrc3/X8lzN3G8gY7j14WO49vCTjaF0HL3nlrRD4/6MQ6TPu+FHjxhQsRezOK17JLty/6+J5wRCl0nNwbwcOxVus9X1Zn+6qinMHDEFdkP/coTo8lfW0x8sVAaVzVRK8tw2DNnSGbGeVKGL4CXRkZs94TAXiLaz2PGJFqvnrRfBOCg52F1iZR1+2ohssvGNiAA712Zz54A7g6jQh2+FBgF1VQe+h4GhSGvjxJaho7jHcdX8X7h+E42BZEE2wKW45zbXpehNOIwMzCVzWOpl4GIWpmExbo6pkoTRNqdJq+uK0QEV37tNX2kQEQZke1xJMpj+/uCkecjZ0RbiBvWJsWjndZyXSg2nqWrVcoG3HHHt1+/LR3Hq9FxfD3cBI632xQ4T9rddKBKRS1sNwF4M7HPiPKpDfdqKgu2nmwK90LnpclYW+xLxpu5XRllCoutjneD5tiPEBXHG0jb2bxvhjcr3/Fy39qO1VL1HccL6Dhe3Q77+eeC4zVotSlMIoWzWYzDXvOfDaxcpZfzOMtodWmJNK5pFIRM/CxLBhJKgWGQ2pDX4yBpNzPAiAoDH2nCQ0c+3jKcTKUz1ormGL4wVgCHPb/xfIRS4CiJ3mouvEwE1/3Qa5wkhAAc9c24KZ2L2ZFmnEyrDeuHvkSYHHXtubJx2PMaZ68KpMBx35TX8zSGfrH+J1MPs4aZvWINvHtZVEkMfI1Ym7L7vmwV0uGg5+PEX2T/3XF8GR3Hq/E8crwOrTYFTv6vDYgoI4w9zk1Koez37qeJmqfXs9vBYEgCViVrl5hcBQY5xDM2Jx+0F93SGPdLpXG7KzO7n6uSbL3IXz+eRm3SfswFGSuUwjONSsqfdp1atTVGbU6KafuWBYWO4zY6jjdvy/PC8TpsdKeQEiNF3UASysm/CrssInenymqzL/TSRVcsPxeL7RYZveVyG82ENB9WxQRKzO50iUVBWbvq0KRv9t8FradXFI4XVVlVTXhgE3P5WYZcsVlnpOPWbHyYKdNVK4bzQnfd8e447kbH8fcPx1ex0aYQW6JbIEVtTljjiNL+dl4S4bhfFKnKMF7k1iAHoZfZOttIxXbNyJLs+ILx8l5R/LtaiCxJSBOk4rkAQBQVTkaeyJ2g2sIWeQ97vvMyL4Ug4MhhAdIEbeZqeQwJB446jUOUOUZdWfNj4sksO/8wgPuXfuNT7TSmTOTeDzUWqjiHe4GHsGqwStBx3I2O4+8fjq9ii9ZH9b1bdxdr+7t6EdKW8e3Lm1bVlJVutSO5jHOIy9uBo3C7JRt0qM1vueRzWXl20c5qWo9PS53N2ug4npRitaPjuKu855fja24KylyFQ3OeQMIWpzRzJt7U6eeYOdktzfNtTMDKIClvV11RZD271C7kJoRCMHxhdn3Fzc38GOZib/U04Il8DFehHGZ0xtRNJ+0ly47afFgeQ/P3qjFsMz9NQMjHsC6hSFqn6zk7PIMvGUKb/hjTwmZtVAxn8K9YM6R22yi6dLgdxzuO23g/crwMrTcFzcDZzIjTniCnlcY81hgn8UCGlmVGGS7mETTnVgWbHmYGvnQ4cbhBgLMPkWUNMvAYg5H5fDaXrRJ5uCw9fMEgxM7FrDUX9IWTSGGSSIv7oQffYY98PouyM1WdGmKhdBaXpu8JDGucYupgrGial9H3BPqOVGFnszhbNHcHZrw1G2uMppweR8IZJjjWsTODGWCcz2x0HO84vor3G8ersBX10c1yd29pw1v67DZE7vqy1yr1OR7vsue5TMWxiZi9RTzPY95xfHP8IHA8xcbWR+ZCqHiznu70TcQ2Xwho5lILjLKgU4LaeXjeFDCT025ZCsCriR5pxqEoIvqSGjsACaJsflxekKYed2A1X2yu+ngWWChyciusCXjWcXw9dBy/fqzL8VVstCkohtPhpucJ5+28C0SUxSsprafEsafn1ccDuYmImfBoUhyfka9w1Kv2zS8L03ucOQTVw+SPrQ7vPF4op2PP8RZUH88C53MXT9yWODY6jq+HjuPXj3U5voouyU6HDh06dMiwkyOI0oxZvFniidXy2mChdHaRFUixluWBIGTJK5Ql2odSY5QcgGZKbBwAy0akCVcLsZY2dh7rLZkbGrQZcs2MRWJSIohqT2hlCD2RzXVq401kUlMyG4XCJEpTOW4H40hkuQvaoOP4eug4fvM53i7MBRvLgTrMNWO+bjjEFtCanYtpslDZBO6H6+WvJVAWX2QWKyyS/vQkoyfN5ydTL8k3ux3MNGG6kIk5oLH/1ilTanC5ZjjitlCai6ETNOPCstZpqlZZRSgFII0efx6rbNEehmlsG8J4IbZpBI/TiVyy5Og43nH8/c7xOhA3DKTxi194D0I0j8NyHTCXdm7767RXZc+0gV2ejTZu6s3rAtiyNfa2kcpriyib//TUuo3xtssrfr/98TBROhOb9o7jS+g4nuP9wPGf+4kXa59tJSloDehdu9O1RpP27LLNuyk7CAMEvWCjMgQBe0FRxeGJ9cXf68I0VpWHJcVGDbEtMDPGF+OO49deNhC3jdK3c7y/x7sOrTYF2tATUxA7dYKEEju9mwJGbeRMxbS1udRaI+gFGB3sbVSOLxgvjqLCmIdS1FrDPEswm3DKVVqcWUzAeD0R3gWtNSaXEwCb2ch3HG8GbUkKQmy2ucuS/MM32qy0wXgb1dp2+mBLw3Vo9WYYHYww2Buu1SgAuN2Pnan9jno+xA22xZ7FCuNF9aXiw7GHudr85MrMeHz/8cbldFgPHcfLsW2Os9YQQuD2S3fWfoGnB59VhM+BKe/pNKpM0TpXhIdbOviML8e4Orts9GyrURsGjMO+wtlMYp1jz1UkMVshFQHwPQ2PjK5u6MuNd/hppJyOQMNArqUvDj3pdICZxTqr57Cnl+KaAGanP53JViGJkYRC6Hsat/rm8ulqIdZajLEGnk6L4QpMeIQ88meadGQWK0SJKN/3BbyVExwzYxypxIEonyvN7HyheIJqwz+UYejLwjlKc27D7gvOxsfGNCZMovXqBDqOr2KXHGcARIxbA4VJJNfiuGI4o7sOfECQ4cpN5DgzYy+s5ngA4PagWOc6HG/DiFabQig1Br5OFkx7zJXAvNBHxn4UwZem4QNfbixlRzo3H7MxYLmWCO8JgieKfY4UI0ZKsKLXq2YTR2YdQ4JAMkaB6cMsJse41YNBGDvJkydjFwSEibtKpDgzkwsdcVsAYxaYxp8ZJotBc25et4z1TpVE5MzCpTRnC0YKZOOz9AyLLIbOOug4vlLPDjkOGP6NfI2Fc9zq0XF8+7jZt40dOnTo0OFacSOUbk+nXpI1CBAUFxJ2SEFOkXse68yBqOeJbOcd+HmyCxtpucyMy0W8sSmwLb4P/aL4zQxEKm5igp3/RjOe1MSGcfwKtwdxlmnrycRD0+PiyRR4N1E1Ki2h2Yzbk0kxbSIARNrLQtw/naa1A7EqUqksPWFZH477Cn7JpWFaTwpJlOUutuEJRiDbH6O0YjzZocq/47j1m47jFU/k2CbHH880mt0o3IhNgTJvO4IJPlbI4VryS50FKwPsiLGrekIXItU2E281PEEFByJmRs9rV4/WrbSzGXqSIYWJVZ/qa5tgroDLLKc3IVtoTlHe+juAWcn362IUFNUTZSCC06w21gr9lgHAACCmXTqidRy30XH8+jletRGtovWmkA+Lq5LNB42x7My3nECq3oQrL+V6Ydr9LGyXrRlpUf1Om7omVue+DCkndjXeHcdL6kbH8U1xUzhehdabgiQkJmDLi2MWE05b5Hh1gQE8uPJBYEgC7g6NdUmsOUt6YsM253o4Rnb5ctSr30n3Qw+UiKJpdEpJcNrvz2ONaU1Ig6tF7PRyTOuxoTRwuWjusn/YU9gPi/U/nnhZMu8HYzNuXEhFXsQkElnmJrUD78l18WTiVbZeijwxSRknAulOilMWEdNZT8dxJzqOb46bwvEqtJcUCDBqruXKo61I35QQgKBXdI6rpnCriBWQjl+kjPVOFaQwKfqW48qQM1Vi1ED/aYopPpfWs/J0bXk2jOq4aApImY1EPm5NwCBEWwxyth3U92FVSeHmBDlVK0TNTVs6jrvRcXxT3ByOV2Frdwqmm012qeYTVcJBd6REsh5ueYmV20yX/H0DbjEvT3RT+/QmT1WP+WYLIraicO4SnhCtnLrKOGH/3SVyb6MrHcfdWJfjTdBx3P33XXEcaLkpDHy5lBs1thKD9D3GKzXJHJ5OPUwd2ZhcUGzylq6i72mnQ8dBqLCfxPmxx38v8OAnAbfOHKETBJmcuVUIpUDQr77Yu5jHTmeis1mU1JOLfJIoG8dIaVw6nGL6K2PtqidVPdhQTHjvavO9/r/8lW/gy++cblxOHf53f/oT+Mxrx42ejbWbEzb2Q43DsMjDpuul43g5ts1xwIT/OOr7S1ZNHcd3y/E6tI59ZIuJtHQ6MPrKyt+3aho5Y9+UxQIRJcYBggxZyy5siOrjHTZ5pgxprfaJxC6v6lTl8kzNf1cy3ollx6YEuZpHON+lh0yCuFX4aTcnbGi9Wd87jrfHJhxH0vblMU9/13HchU05XoeNttvVfLN1DSViiBZirzk8mOlPB6rq92lb7Cc0u0PUlulHCeuJv4KqXwiCzGSvXjIZvWlzlOX4XZcks0hh5kh/uBS5Unpr6Rfsvi5NgVbZF+OFwtlkgSYQRNjreZXzo2EuOYu/XU9F0nF8ua0dx5fxfuD4KjbaFGwRcVEhIqa41Vdgt3Gwo2zgsOeDaPkWvqzPA1+inzjzXC5iLJIJL7OASEVeG54gHKwZPXSvJviWZlPnKrF9kY/hLNa1iURc9TBMcK11Fs0v/t47+Htf+F7h+9iKYhm+/gmIfvsgcUJriKScKDL/AEA/+R74wgT9+69+9RuNX1AvHPTwX/z7P1ZJ/KuFMElKVnB7EGPgtx+hjuM5Oo4X8X7g+Co2lBTsltd3uuwEUPZsuvMRlnWo9e1ZT7jc5KKmbtKpYFdQ/G0T3jjr2aDhWjMiZ0wXu04BWjO8cTYjwvxb/jb1mG3W/qhR3P16c8U26Djuqrvk7x3Hn0uOr2J71kcN9K026vrOnJhjcbNcqmzlmC3Trdrir7t+Nt9XEDDV364DScWFvKxLpeSZ9og1QTFDs5sws0jh8cWs8P35ND9Nku8bMXoVDRYLOR4hyuVZweYfACD0ocNedXnJHDByHbuSAd45mRReLIPAw629sLK8WFN2svZE/gJOOcGN9O4dx+vwfua4jZTvzwvH28zn1jYFXxAOW+QtPS+xZEjBgNNxowzTBs43B6GfDdSJQxRV7Ba5bQx8meW1bQMCnA4nNoxTit96QTKAB+PcyceFb713gb/+D36/shz/xdcgb91tVTdgFkhvr3rufQDpElF7r0HNXiovD4CnIoCNo9Jl5IMZOInm+N//nd8qPP+HP3oHf/XP/nBl/aczCcDM2wvDGGHi+JVyIlb1oRc6jlejPcebn3efNcdtCEkIR8V+3mSOn/nN7jSArUoKW7jhuAZQhZXGrutt9EzrQGHApvbahTbs+rcO56klZLlwycpDXC4yNxsx1xVtXnaTUjqO19fb6JnnmOONy7hhHG+DVpuCWonhTkAhQFbjisU2Ul/XI9ZuPacvKXG6QeVpbhWrY5DCc3p1toNKdJ8MhubqephNyAPFifOQowuzSOHbDy7AAN58bMVI9HyI3gAAcDAADoam3f7dEPLAfH50zqg5UDrBmqHr9CYEkF/Dm6QMYmRx/ll6wGg/62x6T3gRe/jS26eFIl467OP2XrUIH2sNxZTxpOP49XEcAMDGj8EORbFtjtsgf7O850Ci9qtzb79xHK9uio1Wm8IsVpkjD5BYMoj18jZfV6q8s1lUWBBEhP3QqAGUNvmAm2KutDPZxkHoQbRRODsQaY2LuQl3PIv10lin2A89BEk9b53nYQ9ceHwxw3/694vitNw/RPjBjwIAPvtDAj/18eJc/MJvRHjrcfsTh9aM+bhaJeL3Jfxhs8VJAEbJZ1YBovDjAAClgPncfP+1y6f4645+/od/9HX83GdeqSw/tSaKlUms0nH8+jgOGE/oi3m85PG7bY5vG/wcctxlkluGLslOhw4dOnTI0Ooo4wmBQOY33CYHcPo3Rs9ziZwCXnIKWJTEGgmlyPSgC5WmwiMEsvqEFmvObI1NOsHiHhdIAUmp/s58Z9eT1r8KO459pFCbPzaQDMXF3TjtWxNIouR505+0XZHWmXXKxTx/3mVlt4gVfuONx9AaOBlbD3s+5P6hqWe0n3399Ar4xoNiOWPrXuoj9wi9wIjz3zuPEK9Moa0kEZIRjqpPJV7AkAlXVCTBSdKTDxwDPd/Mz7ff44J5HmuGTsfSI/ie+cw0gFrcNuXNJtDTCQDg2w8u8f/7muncp149xB2HmB1IMhYxSC7/Oo6XYtscz37vicSqyPz3tjleD8b+kTFKYgCzmd44jpDN8cWUoFWz8dkVx70WEl67HM2ewND3sFBGFJ3HwIOx+dvAY9wZFgkz9HN92SxW2WKz4fcEJJkJuVooMIwZVSCrLT0ipTFOxKKBL+FKuVpmRWHXc9TzC6ReKI0oUSPMYuFMDm7DExF6jlDGQV801iv7UmAUUvKyEFmI48t5nInzD6+AWcU7d7JQ+Bv/7FsFu2zRHzjF6e8+Bd6dFMuZjfPPf/jjEncPBJRm/H+/OcN0ZVfQijC9MOZy0meMjptb1Myu+lCRmbgf/yBwZ2ReBG98P8aiaF0ISDOfQhKGicWTGkmofh8AMH/0HubJgvlX33yIf/XNhwCAv/bzn3IumKHvQQpCFGsQOo5XYdscBxhEhFHgYRKpnXG8Ce69xugPzWb66FEMXXNl0AazSw+LaQtrrh1w3HUoKEOnPurQoUOHDhnWyqeQutoHMveUCUT+faxzsdQWYWucCkEw0UGZuZE46gnK6vRbhKZFg3ok5WUbJ6Pi0WUSESJtnrlaCMxWTtBEQM/TmUNRz2suZtsIpMgdUQQV0ggqzfjF33sH8yTGSxY/3/Pg3X7B/M5ypOkNGPtJ0EbhaUjPnBbnc40oMr+dvPsEUeII9MUvSAxD06HXP3IHJCUUgO9PY2PZEglML0x5UhJGo0TtFTHm82o5fP7wERbnRlf1zZdu4/6+sdf2RgqDnhm3wdCMm4qBpw8AgMCaESXHSY5zkVuM9hDcNfbh8eU59NQc83/t24/wvSdXAIA//9l7uL3ngwAchKlld46O4zmeV47b8EKNoF89MbM5Zz4QtupoODSqLp8Ir/YMU6aa8d7cNLAJx8ORhheWP3MdHG+D1puCIMLQsqrYzyoUSAWPaaQQ6WTQNGef60BErZxmfCnWMhdsUo8UeT8FKWcCi1jLLPHKOCqWR2CM/AhS5AtmHYSeQJiMrUs1qLTGP/jt7y95bgIAeT78F18tLNL+EHjhtZSkjDQhytlZjOk0MYU7e4jooXnT/9Z75kkpCf/bz76Avb0QC804O2FoAIs54SnyZ/b2ksUz1ZjPq+d+/uAhpg/OAQBf/3AfwdSYEfp7xhmICLh7lyAEMJ8CTx8kOlZGtmCSigEAYrQPf2BsOaZKZQvmX3/zUfboZz54DD8wovgrjrXacTzH88pxG0FfY3hUPT+zOYD58ndEwGhkciEMJOEzhwGICCeRxlVykGnC8d6oLsTG9XO8Cp36qEOHDh06ZGgtKWhmjBPbbpmItqu7tC8JIyqeKqaRztLL9T2RJdaYxqrytr+snm1DacYkudTzBKG/hqu/DYZxPSeY0CqBrD9NauZCWLEnE+AqsQZKbbaZGf/N59/CyXgOrc3lGwBASPgvf8CMlRXjJewz7rxkyg1KfF0GA4EgMa3+6E+9il4UA8z4R3/7v8PJ4xMIIfHL/+QAYa8HDeC9ubnIHPRD/Mwf/AgAYKwZ3z0zjZw8HuPEOr24sDjPb7l/5G6AozshmIHfeTjHIok+fHFh+qYVYWQMMOBZaqrJFeHkYWqlQVCJhY68dQc0NCeq6PF74OnqjbqZ74Uyc5+Oesfx5rhJHLfhhRq9vaSfQV5Xv08IguZn4XQ6FprxtSvTsHGkcZZwXJV0t64eZsblpc6c8nbNcZffSRlabwrMyCrwONdJLhUqhNNKYqF0NohBIhYzMyZRMVvUUnkl9WwbDM76pnnzBQNQJnJLYszCqDZKpNZceHlcLXKzSBuff+Mx3j5ZIYEQ8G7dLUR89APg+F513UGQbwo/9MIxbgdGJ/0P/x/3Mb7/NkACX/69PwAZLnuK3rrVxyd+/iMgAh5cMb52YjowPpni6q2nq9WU4rV9Dy8eB1DM+NKjBRbJiyNVaQGM/l7SH59wfNv0UUjOFgyIwJnudR8iMU1U5ydQhQVjuKw4d14DOo63w83huA3pM/p7xRdhEAgMBu3HOWbg3URNNJ9ri5Nu+D5V1sPMuLrKTV93zfG4hUtzpz7q0KFDhw4ZWkkKj8dAdApE2vys7wEHYXNxd+BLpEEUZ7HOxFjXCUppk++WAQSCQTDimi/FWhEcm0AQYT9ML97qn98PNYa+OY2czmRmpeGCYuDRpH64WXOjMMpLIELwgY+CpDQyfHKK8APGyx8x0T81GCcnZryDgDAaJdEUQ4GXwuJ47lun1v/ZX/0PMJtMoWKNv/M3/iEmVzMIz8etH/5pkJC4uJjjv/5/fwkAMFfAk6mpRzUIrfBHf+aDePkVc9L59pTwle+ak87McbIRAjg4MG3d9wU+tW9suB97DCkNP86eEk4fNefkk4kHIkBrc2rrOL6M54njUcS4vExyWFvN+NCBh48cGa54HqX3tUv4+lWMqWYwM87OqlV9B4HAn/hgv7K592OF04TDHxl4OEic0L5wf47LheGayxdiVxy/ciTlKUO72EcKuFwQUhu9tmFQbG/MSaRqwwqbBOgELTUWik0yEtIoGhFuB4Ioi7nSBIHkrCnn8+pnAcKsQUJ31sbXtS3k3n4h2JeQwN6hWT/zOeMkUevYUvdAEG4H5eNJRHj908YhKFpEUOO/hdnJOYQfGnNHAFGk8Z3vFIN1NcGLL47w+uvHYGZ8/Y0JTiq8loiAMDQewAMvb3fMCntH5pnppM34EeaJp2m6QDuOL2O3HG+3MzTh+NxxmNgLBF7eK3/VMfPSPC8WXOm8Rh4qywOA8yvGaaJH3E+4yszQNSasu+J4pHfk0VwGs6sWA3JtG1kQLauqmxXN2B6D3TSsWUjkDX30G9fJ2d82me91wzzbv8s/L4eN3hYPO46nuAkc3y6/m9e7bsHb5/gu52GjTWEaAV9/Yj73PMZRz+yMoVcv/u4FXmaBcDGPK8XJSBHeuzJNJaJsVz/qAy+Oyn+3LYSegJ9UOo3cESRv9WOkaVoejb1WoWrb4P/1P3wL33pg/AfGr30Ehx8JAQJErw8QQUjgpdcW5sRpSQS+T7h924zhvZ7AxxL3+aBGh8DM+Jffn+FioaGVxv6n/zT6kzlYRXj0xX8KZgWvN8LtH/njjV+SL//Bl3H8uvGe+zoF+MYbRmV0MS+Oqy+AP/GhATwyL0cvEcPPZxr/KPnd5YNzvPd77wAAYitPrn/vZXjHdxq1qQwdx3M8C44/OC/ePpMADl6ITKyqAEAWuSrHK3sSn71nHEz6Xj0vz04VTpOL5FRKIAKOjyWICH1B+EyiyvFaOhF+4b05dJTyYD2Ox5oxTQikiHCU5O+ZnEvMx9uVKjfaFDSAaWI+RkBmiucKCLYKY6rXLBkIwy3+1MRd2xoE5XHkBbnlSvN+MFYVDXO2rIUH51N874lxVjn8cA/eaDnZuJSM3qCYXVAIyr7reYS9FpYuFwuN05kGKw1veAiSEdRihmh8CtYK3NBxK0UwDDC8bdp9chJjPiuX1YmAozB/YaWYkGkTAIyvIkyeFq0uOG4eg6kMHcdzPAuOl8HzGSQAr+QNFkrCcb/5yzJWDBddPI8ghFG5HfXWMxm+WuhalVEdx5d/wPASh0qHVfTG2FrAd8ay6Jueigi7E38Z7gu81fxF60zk6kJuU0Yxf9Lu5P9cfZN9A1a68lTKJZeFmtmaQ84WPSuzIbBm9wuOYTaGhmOktZE6TNmmvaVtBUEpDeLlsrX1O7Y7axHOSErbe3N1HC+vf2ccT9tEics0uDDWBALY/DnPS+xuj83xJlULMqH7yo5QzLnHha1h5JYqI83IQ3dY39W20bCycT1NsLVNYRoT3r3MIz6mk/PiCLgzLPnRhjiZAueOSJrHfYV+ErZ2P/QKO3ATRJpxmUSQ7HkiCwfQ96XTtvvCysd7bxhnU/Xgym9vadEQ86sIpIxd+NFLC5AE9CLG7/+dr4MrbsriT93Fp/9sMZrkFx/MM8ezJ7/5XcySTFazmJMXIkM7Ev/E00vc//VfaN7u05/A/S/+EAC3BcYq/qtfLmYw08xIowvYm8roQ3dw8HETC4ekBwjjEfXkvoY6b9xEJzqOXz/Hw9c/mahGgePXGCSNajQlhNACh2NzbB6GwM9+zDSkTMVjc9zG8EDg7v7y69AXhJ88DiCpPIPeo4XG16+MvnIy0bi6MnMSM2fqNZvjx8cyUw+lYAb++zfGhXe7LY2GIWVWSXY9gyOF/oFZCNvgOLDFTQGgJR1jNiA7Iktadpmp3zaq5ZX/BcpPICmIcouVXfYdMGQiNg5JJBhCAAyNaLJYPj2vIHa82AETUz8Ni301iTC7aprsm6EXDs+jEkTTOaJJ80xg1YqEZQgp4PWLGa+2c5LvOA5cL8fJ843FETGEt4BY2auIAdJGyvEADPzqttsctzEkglzZWCUBoSDIivHQDCyS4maKnWXbEKJYj9ZsDl4VPyVC9ju7OSIPx7U1abXVpkBgCCvpttEwVLdEM7Log2SJfkbkTnWYud51mZzmmyb1LNWJPFplrOsGi5322su3/m4d8nIfcgHa1QfXb1MrAvuvLPL/0ktj0rD/BIjQAysNIkLqaa81Z4G7VKwxcbyUF7MIKt0wdr3arwHMADK1k2v8rLnPeNlx3MauOZ5a5rTi+EolnjS/LIsbaBIVJZe0S5Zp+TO+AFb3E79CBZXmzIj0cnnpTzzKN9fI8svQmpcusl0qO1e1xpcm4Y89fA053mbDaLUpHPYU7u3lL5OFIjwceyWNMXjvKk9Scruv0E8cYQ4skfcgcaZhAKfTCAyzS7+c1NWkHhsnU4nTxLj63cvqAZEEvLRXfWqdK435tKjn2AskQk8mn/OhPJ2ZkAYE4MVRsWxPUNbnSPNSTmCtGY+JMY0UTpKIkItYoqndugg8vPSnPgWAcRAK/PzrRq/x1ltn+G/+6y8DAL72tcf41reK4SfSBOkAKvX8zwsW0xDxPElS4ojwCQAvDE2ET6UYj6jj+Cp2yXENs12cTiMsYg/r+Gb0fOBHXjWfy8bg608W+P1HRuq1zzqHhxJBQCACfvwwxJ7DSsm1zzyeKPzqd41k3OsL7CdqncFAoN83v/j4yMcrSajtf/G9Kd69MgeykxOVtfPOHa/gSCclMitBG/O5Sf4DLG8KTTn+eK5w4fxrES0lhWUvyCb0tS/ntOs0Yc0kr+ziVftfXUuXTiZVYlny93VEL7vYtB+pQ5f5zt32dByJzIt46RRHxbLbnNmJCJSQW3gCvm8sJjzL2ojZOJz9YKBeFSII0JSfVTuOW+Xa5WyZ43Ydm8ildSdhDZSaz4qkYZJQqSayYXJPmM9qSVKgrB2S8nuN1WLr7p/dEkSVeqme422mfqM7BSIkQcGaTam962pmh7enuxy7HjMha7B7pZ70HSmIEWlyWFPkEJSLpkrnpw2lkfVBWhMpKBeZ3VqY5BKKuZFpY2m7vCQPbIPhICngjYrZNmxT1WgSQS2WbSCZGWo2Lpid6rjWvbUUejFFND6vaCzg9fYqA57ZCAKJvT1zjxDu+dnpiyhfgHt7IcAmNIHXwhy34/jzw/FYM8ZRsa5Iwxnaou8R+okk52KEZsbVYjWeKzCJOStv9X4gxTRmnCc+CWXnr74wuZOZgP3ABE4kqzxmzoIr6hU1lbDuETbl+Co22hR8wXjJITo2weVCoZBeqUE905jweFKd17YOBCNSieQUY1uUuDAKNG71TVsvFhIXczPgc6UwCky7bFVBGlsGAE4SVYENxcBZg7hAdTi4F8HfbzaF4fEQL/2pHy58PxpJ7O2Z/rz5P7yJx19/XHjm8Zf+OaKrk80aa+H8u7+H8+/+XvkDJPDST/5FeL1mJj2vv36Mv/A/+SQA4K2pwrfGZmwfRAJjk4gKP/8/+ig+kvix3b1EU+p1HH+OOP54ovAr3y0aPIxGAnfuFMv4zEGA2375y3OhGP/ojXF2X5QiCMhZno3ff7TAZFIujQsAP3YQ5BfjSSbEiWL82plRdUUR4+nTIn8GA5Gt2W1wfBUbSwrXgV3Vs+yA06aS8mfLVAW7QuOxqbjUyr4vLeuGXzpbfSCU9ZOy+W410x3HHWXeTI5XtcTJiZLvl8osKXQrIU4ca5Ko2Xhuk+OrWGtTaOPmrXmNiIgVEASEslofHmmC3lD8FsTwEysJ37KWCCUwSiweex6V6g1T+JJa9V9XlNUWsWY8HJvjwql1iShE7ko/9AWOks8v3Rli8AFzSnnnO+9gfGmOIKw29wxuB8bi/BHiaUk2oBVE4/wUHMWE6dj0R8W5ffmIgcNkCPpSI4RZOL4gk0qSxdJC6jhuPr/fOG7DZV3EzHg80cbCaMXsNwjM8/2AsvIWDIyTCwalGHGcf05xGAqEK/ULuIMtMgOLJIpqHNtzQjjsGelA0Tocbz7ga+RoNmJk053yahFjtip/bYCex3hhVP2SejyRmJTcxDdFKBl3h8V6bg+AQfYOqrYKIiLsh+3UAEozyl1l2mEcMX7ZIU6HIeHw0Ez9h/sSrw+TNv6xDwL4IADgv/ir/xzf/Z0vb6UdrcGMJ1/5F40fPz/4cQB/GABwdS7x7vfyOfEThr+ugR+NAIDRDxVk8hI87PmQghDFOhv1juPXxfHNd9LWHK+BZuBffH9a8DeQMo+DdOQRfuzQ3M+9N1P4cuK8Np1qXF4WefC5F0K82lAFphScKqMXRxJ/9DVzX/C7bwO/8d38b004ft7ijqFLstOhQ4cOHTJs0aO5pAJBCMu8SirAyB1E2iKUDKq5ZbHV6GkSERu+zOsWBPhpTtTrUjI3QK9HCHpGj3g3kBAwovmjRU2SkJ7EC6Hpz+gaUkCu4oMf/xDuvfpCo2c1M75/Hhub9lhh8t554YAZD+/gzTNzWns6FVDanGw/OgJeTnKhvNwHXIfFsxkAMie0dc+tHcd3h11y/L2ruCARaAb8gMArb8bQI7wYGi85ycj4dhZpTJOgdZFl+eT7ZFkoFetmZnz/Ii6Yyk4d1lOACaq3CcdbBBDY/abQ8/JMVG2gNGOh1rNe2A+bi/JSALcH1YsrkAKjYOdD1Rp7exKDIw8CwB86DhEKwlwz/vXJHFUj8FIo8Km9YiiI68JP/dxP42f+/M82enYeM37hm1eINRBdzXD/l79aeGZ2fIR//bYJEDSb+NmC+elbwJ9LQgzfTVQizCZZTKzN53cujaWM1mWmlfXoOL477JLjX3m8wP2r4rjcvethuKLwH0jCp/d8EBEeXMX45e9Vh3Xp9wnDoeGh7wi9oQH85v05JjVhMVI8meqNOH7RPApNpz7q0KFDhw45Wh8NlAYeOiKUhR5w1MxYpBGIgH6NaiPWvBR7ZF3U1WOnWIyUblVn31svBnsZwnv30Kd9gAARGFlRM/CNJwsImFPv5VSBYRzT+n0q1H860/jy1DifzZ+OMXtyWajn7LTF0WILYGa8cVq8sI05P8H3ej5+8qdeAwEYRxpvnppTtrc3wmJmToU6lpn3bKxNkhwAeDIB0qyj01gYZyAyl6qAkRSeJL/rOP7+4ngQiMxyyMa9PenMuTAaCuOJrYFvniygGZgT4SuRscy7XORj43lAmKipDn2Bo8TvoRcS/NRaKSGkzXEGWo2xVgJxZF7X63CcZ0BxlbvRelOIGbjvKP2wt90FI4iyUL5lmEYKUcsEL8V6TLL1pqReKI1pC0uTnie2ZEuUlPfiixjuAwCD/AWMZprxpcdRIQy15wH9fnEMn0413jw343b29Sc4/9r94jNPr3dTAIBvPF3gpCLhzmDg4Y9/5kMgIjwcxzh907QxXniYj3Nv7SRUD2Kd61JznSrBtqb51B3zso9i4OvJdx3H318cH40E9vaKL//P7fuV+clnscYXL3Wm9z+5KM6D5xH2900ZH+hLfLTGyqmO42XQSiCars/xxRgornI3OvVRhw4dOnTI0FpSkAQc94o7JhHhe+dmd9oPTG7ZXcOXhOEakRVttJV6Aylq481PIpVZsowXqlCHINqKyH362INILEg0YhMzSACfuxdCEDAeL/Bb//ItrEZvUTEwT+KyLM6KaSx3ja98+SHG3rcAAPc+fQ+DW0aHM6sRp2cx4/P3jUrAthoJexr7SRTQyVhgfFnNiXtD46AFAC6tSsfx54fjEYD7XDx5BwFhf99Mrn3R+2IocJhM+tBhMaaZ8bsPF1gohtJup8S9gPDDt426UnqEMFETlaW4/f40xlUibtRxvAzb5ngV1nJe23NYPowjwpOJaYXA9SwYT4jWHd4UvhSoytfOzJjFKhM5XQnQZbJgNgITri7ShjAGhwQihkeEjx778AThsYrwj7/ycLN6doDvfe8cJ8oIsx+7t4ejYQhmxqJmwUQa+NZJ0VrHDxgHR3nS9boFc9QDhhWGKR3Hnx+OX8Ua98+KyaA8j+B5xU4c+QKvVJiKMQPfOY0qk+UMfIGP3WpuvfdkofEk0o04XoZtc7wKnfqoQ4cOHTpk2MgwOVLA6cwUYTthnM2BeRJU8/bAXNA1ATPjcqHAzBBEGAXNL8faYFv1TCOFheOUVOePpHk56UiKQAoEDpF2P9QYJM5Hf+kliZOFAAP4u/eBk+SQNBv3QATMVYy/999+FcSMycU5Hv/+PysELesdv4T914oRUy/f/hqmT98FACwui0l4doF3fusdPPyqkWZGn3gFcrAc3psIODiQSeYp4Py8+aXrvzoBvppEkPy5u8CHBwDAGEcKsTYZ0kaBV6kq6Th+QzhuNX2hGf/ie1MTNloQ0EdjHdn3pwqP5sX+XFwoKGUUrfOkczb3bOyFeftPIo23JtUhSS5qLu2vg+NtsNGmoJkwjQmrMfkWyvwDgDY+UgxjDsdAkhJxM11qFbZRz7rmgmXmaIIYLmOIQObfvz4EZj0j5v59gWToCTpOzNUWGm98+wTQGtHVGaZP3i6UJ333G2xxdYLZ03da92cTjB/ltp/9j7wIOSg+E4YEIWgpQFgT3J+bfwDwR47z7yOVp9xkRmVIyY7jN4XjOTQjy2TmecDtvtfY+ulKcabfz9rKjCdXMWLHuz3l3lJbreB2c8V4soWEVTvneAtciwuj3agb5EFfiaYhgbcROXg7Y+JoSFm5dD0hj1uDi+3Ks31Zjzmavvyd/R/1g7uNseg4Xo2bOCabzHub3+5mrbXjeBu02hT6vsR+6DnFwjI8HAOpyfsr+8B+MflXBgJwsE68gDVQV0+sGVeLajHuZCowjjZrb+hxltxkoTQWyoTt7XsSh0kbJ5GqjpFDjJdejSClsZgIex8DGBCk8Kkk+cyBJ/CJkbGh/spvfg3/+O/8w0Ixau7w2LpGPP7NN0ErqgXheZj/wR8BSQkwEMfFY/n8Crg8TcIkDzVe/ZDRqZ0+9ayLSjdSLittTtUdx5exS44DAIFw0PMxV3UcN1FKpWeykJ2cFNu97xE+mXD88ULjO3VqnQuNxcKc8l1Sgo2eAD6zb7g3VozPp4lwGrzwy+o5OpJZ5rZ00xz6hJ943YjLl7HGV6/ipb8DwN6hwmBoymvK8TZ+J61mWxC1DpYV6TwdXV0ueCKCdw1Hiib1MABVM+Gmb5vd1UvLG4eRp90z6RlN2ZSkOC8DAQh6Gp5n4rgHB4ZUngcc3j4CEeHYF/jAgSH1O995e6uZ1LaF+KqY4pM8D4spgWosWdKUD4OhRtg3YyVl/YJN5zjWnKhaOo7b2CXHASQmpoSFqg6lTQA8nxKOu5/xiLCf8OSygRrG5D9o0AEYXuxJ4zkds8Zli0CGZfWYoHkreRaIcNwzprwiIvhzl8Mc4HntOK5bqAC3dGRJK6wnYdq2qnyx1wUiKlVXNEeTftSMy5pDsVSqQ/VSBkEEmZjrsW2L7bD33gmY85zPVG/LLkRR/cCVicwbNSFDsynvOF5TU3UJ18zx5brdvynNqpb+I2OenP43J2WVl1fftux2yirXRotcOLVYd8w32hQCyXh5L80rK3AyrS7u++fAOxfm891hjKDBLrcLGBHezyY6zSUribCfJFfxBeGoV3RZn8Yqi89z1FM4dDg5pWAGHoz9yuibs5iy/LlDX+MgbK62CDwgrf7p0xhomMrvcz/zB/Gxz30CAPCbX3gHv/5r5jL67I3fwuThdxvXvy7O3vwdXHzvSwCAW5/8I+gdv1T6bOAB/9MfBYIVrdF5zPj9i6J9OtUeagmPJvnF5AujCI6kXHn9HcefS47bmEw0rq6KBx47ZMbxsYTnmdQ/P3EYIEjezmnGsplm/JvT+dKmv049f/yD/SyDmisb2naEyCLH22DjHM3pghINJktxbsqmNLChVLoVMNsnuxxE5EyXZ39VF0K/mcRG2Zi0NfJITzOAEanrX4gGYS9E2DOK78H+Bbze0JQnr0fXzfECKjYvdK6J60MwTjjhip4+EoC3Zm77PI1lg5Ndx/FK3FSO29AahZhJqxDCqHIIxjktXHlhK2bUhSxirq+n5xGG/u5J0Ybjq1jrLWAP17rnIM31+tfadpBb3NKci06Cqndforw/TXZpIneyzLJxMO0zqo70aq0t7DbaP+9JoJ/wa45E+0NG3ZK2lRORW2nG3KFn1SQgEvdV4QcQXhJtVEXbMTtJISSESOPLS8jkTev3A8jEH99zjG2v52GhCLxy2FnE5brlFJs0v+N4Ec8Tx2OV69FL1UQlzYw0F7x67aVjVEjF35WpJD1rfpzjygzXFUis3XcBROuoAJtjrTAXx31zRIu020GlCR5PNj+VjgKN437xzXAylZhEZlrvDmP0vOoLrLQ/TdD3hNN9/3weI16ZQALwYiK6KQ3cv1rvaDv0JYbJi/vpFJgl3/+VV1N7cOA//fYQlzEgPcZrH1mAiKEU8OiRmZ/HAN6gorpFHR7i5T+7DwDo3dvD1VufBQA8+dI/x+z0vbXa68LeKx/HwYdM2X/6z7yOz37uRQDAl8YaT5NsUz9+GGC0cnRdKMIv/L5AvDLNJu9YhZkP1r8i6Tj+/HOcGVkSnrJN4fBQZiG103csA/iCI2yGXUQUua2f7Hr29gT6yW72B/YD3E6kA5e9RMzAr53OCw6Bi4jx+GmRe8OhO+rrttCatfYOtclFGm/Ztna57Oblt91x2zxvn370Bt111UkEpCHiBShnLSenS0HGQ5Ozr92iOxFEcuksPAmR6mTWkdMr+5CX7YcBegPjQOdFC0gyy9cPJMKVVcMREKvcuscqcavtW25rx/Hmzz4HHK/AqmMagJokp0gkk+Zle8IENqyC4mK9qoFEsgtsfKeQjqlHBE+Y1mq2dVrbBYEzPaet41VWOkUBZG2Roih+u4XjzSCJoCt0zlIQfIFCxFJmQCVjpQFEycpimHSNQDrO1W22F2e0IJDIzTSvE37g4+jOUeH78O4BwgNzspclbvdXc0BHZo4937joLzSw1ycotdz/mBkTRz5bZgDcfEOLNSWqB/f4dhzPsW2Omz8anl8Xx21LtjZ7JTNn6kr73oAsVZZPpv8AMPIFesmkxBqYVOgRYwb6kgqSAm3JFCnW1IqrG20KknLrhaHPCKQRIy8XIosXs230PMadQZEJF3OJi4WZkdt9heO+acthz4O3TTuvEowCiapQAiYJUlTY5Wcx4dHEjOEkEhgvhAmtECmcJhYjo0Ci54j4aCP0gFTofftNcy8gpEZvL7pWb9IPfeLD+E/+y79W+P67U4U3xmbebu8VVQzMwC99FVjMABDwgdcX8HzAJ+AvfDYsXIg+nmj80neKYb+jebCUjKQOD8dJeBDtPpV1HM+xbY4nhsU4m0XY73nXwvG9PYl+v/1YKQU8flyckzAkHB6adn9o4OGjgyInfu8ywpcdVkkpfAL+yFGR448mCvcddbbFw7GH8bz5QWlDSWH5Ou66Xj7l9ZDzmV1eyjStI1VDVD9GG8mGRABbJ4JnEsmCACGKBCTStScfTv7ZB01G2WVrGd/adLr+OrnjePM6nheOb3us0vKopOy6NnLy493MYfsyt3bUMR64pgGBBEJZ3BmjlmKMC5rNyQMAJHEW990TnNUphEutwKU6ujYQRE774ibwBSUOMLknqaDlsWJhXo2SKBtPW6zue7kKYRoZcZwI+EAPmGijm3zjKn2ZEnQsE8IxhGNObESTS8zPTMTS3p1jBIfmxH3cF4WTqGbg6VSZC70owtXbxtchZuA0uQAIRJ7IpCcIh4nFkbKecYUJIAL2PYHAM+OQ1hxrxslUg8E4n+ul5ClZuyJCaqh02wcOEqFkaB1CfaELm0yVWsRuV8fxarTheApP0LVxPI45CzlRhtt9kwM54ziWLd2ITL4GABgEIuN1zxqzidJIg7GWhcLY9wgCaM1xZsJ0Yr4/EpTlGa/ieAtBYXubgh0SN5QKPa8o9jwee5jEmy2YuRJ4ODb17AUqs8zYC7UzMYqNi3nc+iJqFQNfYiDa3/wLMjFeACDWGmczMz6hx3hhlI+V1oz7BISewKHDsejVA/O/zMDXngCz2Ji8/Uevmu8vY+CvfydNeC8wuzKMEV6M/l513uWL730JD7/4GwCAH/4r/zGOf9jETfpzrw8KCc5nscYvfGMMxcDk4UN88f/8nyX1a/zWuRHyXwgFfiQJIfpST+Klninja1cRvnVetPBIQSB8Zj/AaEULNIkYv/SmURn5PuH27SJ9T58ITJNQwn/iNvCzt1efMHF4whVrnVhpvFtDzY7j1WjLcfOyN7+xN6Fdcnw81hhXhPgSBPzsx3oY+MJw/N1FQdfv+4Rbtwz3bI7beGuq8M6s/MqaAHxmL0BvRWfUjOMS775rxvnfexH42YLvZ5HjPFU4LW3NMm6Aa02HDh06dLgp2MlNmRSUnaiU1tlOG3oaqZu6L9I8sIyF4oLFQhPYIQRkichbpuObxrSkm2yKWBOSAxBCqTMLEBt+TY5bAjkTjWgrXZ9ixnzVOH+l7P3QiNoM4Hxm/tcj4DN75hQ1VcCXL/I644WZbhIa0ktyNF9MEV2Y01U8dp/e718pnK8kJkkDyK0imsZ4+m2ToGd0uw/smSDvV7HO4thPnMHEGNHJKeaXCiSAt88O0AuXx2iuGL2e6btnxaaIImA2Mc9OJwJR0g3bSmToI4vX3/dT1RxXR+asQMfxzTluGmoipwprrrbNcRu3+gJ7gWnXg7HCbMVrjBl4+yJGIGmJ40TmUhkwKqN7SRmHlgl1HceZGYsFQ2sjKXxfxljVDt0Eju9kU7DF7KtFDJXEUdm3RN+DkOBL4414Ottc5A09gUFVYlkLDODp1NtY93urrzEKisQ77InKwFZSmPgzq5jFCotkrOaxxqUjrPF+SAgkgQh41ficQTPw5UfG9K0vgf95Ik4+mAPfTAx0lJaYj01SYS+IID3jHjR59wznXzP5kmePL5zt/eKDYvTSMkxPpvj2L38bABB++i7wUbMpPFpovFETynj8xncweTIBiPDrr3wK3mBZLJcSuHPHK7wEZxOBh++aZxczYJ5oEGLLC/qFUZodjZDSnplxOovW4l7H8c05btrJuJzHS/4C2+a4jR869vFDx4Yrv/rdCe5fLa8xBvD5+0W+C2Gc3YgIRx7hD+wXVUZNOH5xobKIqb9xVlzfu+J4G3Tqow4dOnTokKGVpBApjVmsEMr6cMcpfCFASS1zpbPT0lzpLF2f63KeYE5GBHNKmDucPySZE4Wpp749PU+C2YSK3g904eSmGBhHjvx/JZjGBOVwlAqkhidMKT2v+VhJouT58mfmsc5CDfQ8I2YTgDuDojdn4AM/m6Toe7oAfvvcfBYswIvkElvlJ89P/fincdh/HQAwe/EOXFeag0GxfcHdPXzsz/8pgBmLSYzT734ZAPC29wp+/ddGAEwu2yc1Vh/RNDnRMOPyu48hfAkSAr2XXgKESVd49rQ43tFcZOEDPrAPfPCW+fwjt4B7JtYfkvBKYOYlHobJaT9OxrHj+DLeDxzXnAcpfOeMsVBG93JVwkcXx20r65lGlpd56BHuOPKLzucakcO5sixgXlqnXU8cAZfnpuxNOe7XRTa00GpTWCiNSaSyhdQEoScQJgKJmkeZfmvWIJn10DfiWqS0c8H4kjBsmJSaiDLRm5mxUFFBw7tQlCyYZphEEpOCZMYYeFFmRpgu+iZIdalV3qhzpTN/+EAaUz4i4KW94rO3IuDfSdr3zTHw+1fpXyR4kYyFtSn86B/7cXwuiUnkEq0BYDQShcQg2D/Arf/gLwAAHn3lW/jur/7nAIDF5QnO46OaXrtx8Y0HAADyPBwffADkmXl2JYeTZEIsA8CnjoB/5675/Oo+cHdYfH5mvXSOEsuXKNYgdBxfxW457tanbZvjsTJqJwB482SGt0vuzlI4OW5hqhnfSjaFF0Lh3BRmM8Zk0iz4FpGJlbQaciOKCE8fmY1tU44HNWE2bHTqow4dOnTokGGji2alGdPEQsYTVOum3vMkfIclgwu2+CYFZREUbZS59s9ile2SfU86LTaGgcxC/U4i0wdJjCNHQpG5Ikyi5nbb53OZZWwKpCpcyAki9FuI3GWYRhpEGsymTs0EQYyDUIHIxOZ5dd+oCvwA+AuO++IvPAR+I/n87UfA+A3z+fFFgHniYOb3Fpmz1OWlqmy3Hh7jw3/pL5rfDQ7QO34ZADB7eIHpAyPbD145Qnhr1KyTJGqD+r/SA/5IokJ4faRx1DPtJgBXjkOhanHj23HcjeeJ41+9Ar5yaT7ryMNcFfnUhuM2YqkxvTR8u4g1zhOVUZ2DnA1m4OnDok9SbPm77JLjq9hoU9DMmYgcyPoFE0hRFTqlFIII/YZWFwCwUIxFIoqHUkCuCLdEeVuV5nzBiGXrkRSXc+EQoctAGCeLi8DYj6LCO00mC2ZTpOoGZuDxREKzcb4JpPFmlATcSXSVPQ/46eNiGQ97+abw7hnw9H76l5QaDC+IgGTBTKfL3qgF9A7w8s/8TOFrjlW2KfTu7GHvw3ca9ZE1MDmvXqC3g7xv+wEvzeFsw9AxHcddeL44PtXANxILpUXsFZzRWnPcwgTA49q4qtVgDZyfS1Td8+yS46vo1EcdOnTo0CFDa0lBM3C5MFsTgbCX3H4sxdmINeZJsJBQFmPk22BmjCMFzQwCYRTIjUXOvieyi8K6GC6CkPXBRqw1pskJse9r3BHmGDVeSEziZnspA3gy9Qr7vy8ZUri3d62LoYdtnM0kFkkY6eN+nFkk3O7HJhYMrPj21lwxEz58ZPqZWtgAwF/60dv4Yx81N1V//3QPv10MPIrFNAS5YgM1OUwljYkXuXgczX3MrnoNfgyEgvCXX0XBycfGK0Pgw8l99ngh8GhcPedHvTi7JL1axCAymbrS7nQcf39x/OdfFvipF8yEPxwbJzgA+MePgLeTz9vg+Lq4Do7Pa4webKylPkqtKzyBxMlkuYGacy+6JvrVRWI+RWBwQRBujzbmV0SE0JG5nRSyBeOJPHb9XDHQWFyjLLCZjVhrzGN3xM2yEM4p5jFhpgQAzsIPEwF93+FBCVjzABwl7+FZzFmc/o/d6+Fzrxr9/m9+FY5NgaBjB02aqiyTPmpLj6uVgIoaZuiSwI/smbSMZTjq5X2bx4RpZewhxgHnli+pyWis9FKXOo43Lv3Gc/xWnzFMfM3e8k1mNwD4Vyd5H7bB8bVxDRxfzZhXha14NPM1xGguq+M6QgbvCusP23o/bDdPZc/ucrzXr7PtWKbPN6VPx/H10HF8e3XumuMpNtoUYs04d+Sv9QXhIHFxbxKCdz/0wLycWWm1nqtFsZ5ANnf7v0mINeHBuDj0A09jL6i+tDrux1noAldMmvI687nyrPmx49f8hx8G/t1XzOfxQmWn6BQEYC/0ajNkuaA/+wL0zMi/oteDKEQJYFwuVMFqggAE2kPVormYA98w4ZYQ1d75EZ5OvUxVcHcQVxo3dRxfDzeV4y+OjBMcAPwfQoWLaHscr8ez43gbbCwpuMQST1Bj8ZaI4NVMALNb/GlDmJsEBmU6UxtBg/6Y90P7fjPyuZLknp+X+uYfAJzPuLApAMBx35XwpgnC5F9J+xg4m3EWgz+F0sC7l9U9VowWljNpOkgjXjcZyY7j7XFTOR56OQtfG2yb4zXtu8Ect7GbfIK4HnG7ad3bFL9NSVz4Zhtobgi3hbq2OD/bGm/K1aCuWuwn1yrfWSqvr+LoON4eHcfxTDjeBjvZFOaxzmyot4G2nRpHKqt/L/Dgt3DxTuGLPDfvPAl9AAD7ocIoEX9Ppl7NhU87TCKBaeTvLCF8irnSiGbV89PG90UzcD43xxhPCGd0zCZIVSwAcDYzIRoEAS/tmbJjTVlO5W3h4djMMWvdimcdx9fDMsd3tz10HM/xcOzjal4d2sNGq9ovxsWk3M8SvnTHbF8KqOXLQkLstog1O026xpFwisibgHE9J9BNwzjbYKs8vWbbKU1JSKb/JhdvkvowGeLtjwtljkzpRtxxfBm75rjSjAdPZzu92m0FAs69ReFyVjOMZzubO6TeFhzzJpEqcE0zYdwmd2ZDLHa1KZxeRDi9aBebu8N6YDZmkjcJBBM4bvUSTjFnbSUWiFrYRJchVlzYYJQmaL0bBQQnL9iO49cLrYG3HzicYzo8M+zsTqHDZnjvyRQPnhaThDxr2I5DKezXtOvvbcEr/7v0tx2e4p/lHUGHDjcFrTaFsN9D0CtmHGqKka+W0gumGPibe3juEpGq1x9fLCRivYU+MOPq/Cp7QY0ORu0NjRPIJHDYKjwhKj1wbwKMaF3+ko414WKxHVPNxWyO+dREUus4Xo5dcJyIOo6XYFccr0OrTaHX9zHaGyR62PaTeNSL0fOXiUcAjvseZEIKQZtbUmh2e0yuW/YsVtkl3HI9+Wc5kVishAZgoPVYsWaML8bZgtnfH0CDwGuMtycYtwZRofqeJ7CfxIAnwsY22WlSl1WsWzazsTd33U9kCWxiAk2K9NVA+4t65mzBdBxfrSf//H7guD1urrFa5XL6zPuJ43VotSkchAq3hxHuXzUMUbCCk5kEZq6dL8qyOB31/Y3VD+OFciYsOer5a13IhVI4k65cWfXc7itgJVoiA3j30l/7UlcIxot7EU5nXhaVsg1iDedcDX0NNTB6874nGidxKYNmOPPAhlJgb00rjQPH7xQbiw3AJLRPLTZsXCwEzmbr96fj+DLebxy3x20/9JzJZ87nURKSBDju59nb3i8cr0Orkin7f+vC9ePqOCjrYOs2Kg1OAq5HNu3X5vr561FXXNd4kzWgu9LEdByvesbRjvchx119ej9xvA434qI50mR2ZjKmcbQyBURGR7gKzZy5jEtBmRgnqTw5CVDuPdoWtugnnaIoEMqiB2MVWOzevUczYR6nYQSMPnkV9nhmbWPTl7Q7nqDMlNQ13vZXShetidq3O/9McIeXCISJtd8GO7AALKDjuPWbtTjOxhs6cW9Y6ObbSaTzRDSzOPe0nghg4QgZMVeUmUOPk4O6ZmRrxkY6p437IKuVZDeB4zdgUyA8TvRm5uUSFVzMTRyTYqTKeawxTvSgA19mMWLKYsXYv7+Yx1t99Y4Cz7lINUetRGtmxntk9Ia7wjQWmUPSQawR94qxUUpF61kevjgVraUVZ6YMtnPUNlBWZyA1el7LrCNThdMttcuNjuM21uX43WEMQebF+e6l37hvTybAeaZOF8jSyDjDTxPs1+J7V+T8fl28OHIbIrjwrDh+AzYFIN/xt0PhZ2Xl4bq0Mt83L+P6rCK3O0bPYsxLRfAbaeTTcTz/zSb1Y8MhbNLQsmd+MDjeelMQAAa+BlZuv2MGFo7cp20xiQSIGAJAz+MkD2sau36ZDbZ4PI2BNJhl6OmayzZGIAU40fWmQbEIcIYLUBq14nGkGZpXzz55PUvfWnU2QSgZXCgbmMbrWWzYiBRhHFFWT2rFFylX2Ade+nSWuFEQMfpefX9SySPWuSWHJ8gZfGxRzJm43BJ2e+Da9diw66xDx3E3njXHp7EZN+ZV5VsRsc55HW3DjHZLmMaEqEJEsjmxS45XofWmIAVwZ1BUA4wjwpPJZguGk5CvgLE/fjm5ebezK5XhZAJcJHrD232NYVAtnB73fQgiKM2ZVYEgk2VrdXeeRioT4ctQphpJ67ERa42zFolV90KNvZW8uszAe1fexoSfxCLLsnXcj7GXjJtJvlI+hkoD308iO3oCeGkUV55e+p7AfmjUTVeLOMt7PPBlIYwDs5mTKoKrEk7Y9di4mEe1G02KjuNuPFuO5+PWBHMlMN9wrrYPqrUa8gTjpZGZq11yvAo3bdQ6dOjQocMzRCtJwZdiKRCUZmRekL5AbfKMaSwae0QyA5eL4p7lCcbAkZYv9DT2snbmfw+kyNQT81gXxE4is/Myyp1QvAYBsNJ0i6uYxdqY3ZGxaSYyFiRpeXZaRxd6noCw9Ct2PcNAFxJ22NBsApo11YXOYgJzsZ9DXxcS0RABe4EZT2nlthWE7OSvtDtevS9ExrwyA5qeJyutlWzu2Yg1Y5qcaH1JmUWP4YEpz8WDrG0dx0txUzjOAK4WzXldhi+88RiPLncfSubHP3wb9w76jZ7VJZywoTQgqMjDbYVKa7UpBFJgZDk62a7xgWQc96sXzOMxNV4wGoRTh6jV9zQGflGkGviMgV+sv+8J+FKAmbFw6MkFUa3zli9FbUKVs1nkfImlIjcBCPsiqzMdx4XSWCi3mE1J22yztPOkHiLgIKxmwUKlm0IzTCLpSOTB6HkMufKqEQQcOebb7tssVogcNn+hJxBWCKlEVJttrCwsQ6QZkTZ1DiGzO5KeZ8pLTTXLzDU7jpfjpnBcaZO8flNFyS9/5T5+73u7tTsDgBcP+i02BTcnbCitIMX2LPlW0amPOnTo0KFDho1MUqUgDJMTXazZ6XZvYxgohF4zGYdAGPhGRNTMJpY5ytMTBpKMWgJGnE0tKaaxztplH3Jcl2YyEXnXMffre6LyYpQB50UeAdkYRpox09UngJ4nESQdmUSq8rQkCTjqNT9RBFJYY6iyMbTVQzYGvgQByfyYMdac5xq2xf6oJAdxz5OVTlhlEJRzrwx2W2wEUiCUZk6mNZerHcdzPAuO22Wv6+fye987we9+76Tw/bsnechu7+5LIL99IERiBiVWU1oBKmmiHp8CsysAwD/76gN8+Z2zRuXt9338u3/wtcr5mSnCybTI/VGgG/tAVGGjTUEQoZ9M9twiZhmMnrRZowUBRz0BIiBSjPN5ddmeEFlbIs1QiQ6zLPLjzGHqJWn95BmhV/+COpkW45j4gnDQSxybYgVHeJWVekz7mHnpxeCCFMB+jYrJxiggJE3B+UxXmhQSzEuCiIweP9sU3GNbprIx9w/tX1BSEPqiesxtKycbB6GXqVtmcfWLpuN4juvmuA1mc1e0zivv2w8u8I9+953KZ7zjOxCDYeuySWtIbcY5igBO+sbRHJxsCr/55pPG5b1w0Mf/+Edfq7TkWyjh9MTueRrBFoKqduqjDh06dOiQYWsezZ4w9s9NMY7iSlHUtttu4gFp0hOaHbvsEnHoSwgiMBhXjq1WM9faiodSrhWnnQDn+NiaE1+IxIa8WZnDQDb2Do21zk7zZZhG+aVqXdwcRv38eILQT06XC+U+ZU8iVXlaJxBGwXq5CHperg6zYV9qjgKv8cm543g1dsHxFAzg6dSD0stJnWy89eQKf+8LbxW+X1IT3b4HsXdYbHsYNm4LCYLfS8Jyg7MgdkIDqdGY8l+APtovL4QBqfN5mMYeGMAFafxff+lrWO3hx188wL/9uVcr23U+lxgnfiyHvRipdnXoy1bS4dY2BSnIGbypDNOYagOktXHEUMyZPq8MgRSQwsRGH6MoinKDOsv0vXUgIoRe9fhIQQg8AUqjftWU5/JqLH1eoXZTaDKGNurGShJlLxfFvBp1GUD95mN8V9eTiT0hULUW0jFsytuO49XYNsdXSsc0JlRR+GIS4fNvVKtqxGAE7+hWi3odLSHAC4rEsr+J9Qja75UXwgxPRcmdHDCLfBAI88UMn3/jzWLZDXbRhRIwewJjn/PxTTnRFJ36qEOHDh06ZGglKUwjlSWAAMxJcF3RftRC9bEJrhbKeRm7n0QfbBJewMYsVs6LvVEgnaGP22Aea4wXcRZkjJlxPouWHHuGvqy1J3fBaxDJdBuINTstUEIp4Is0ho1qfEJmAOfz4vx4gpb8CdYBJ6oUW4LqOH5zOM4MvHUGzJXhgUtKeHg+xf/tl78OZmBi9VHsH8J/oahuES3URGXQijG7rL4t93wB76C6LmLzdwFglMTZ4thDFHwSYAZrYJGog746nuKv/be/Uyjj3/7cK/jJj96trOdiHtdqCZba3vhJGH3kkri/AT82JVdTEKmClEpEWVCwKo9gFzTDqRLYxuJnmPG1i4o1Lw3zutUIIoh1UnK1hpt8UhBkYmVk7haa92QbeQGqytZW+R3HbxbHJzFQFUJpEWt8872Lwvfk+ZCjPccvtgNdF7AxEBA1JtM20hcxewQejAAY81ZKiri8XOD8YbGfp+klQgUUL3O8aVs2giFLhflicsqqSlKd5kEt+6273uXnm57mqtrhfr7BMxXlbjvMcd14p3XWjbcpq/kYtp2fut+WP9v0uS2Nd4PHO44/C46X/S2PBlp6Z9O6Pds9eJBVZqup4JLWlPSHOd/0K/NztxiOVpvCwJc4CL1MnJ9EwNvJ5tX3GMd9l2u+zEIMjCO3WHoQ5nllz2YxGAxB7RK32PXYsK0h7LsWu57DXjFnbqR1Zr0xiQTO5tW7fqxjhCvhownGPrvpfARS4LCXRpxkEAhHPR9TlatbUnEaAO4N48KloD1umk2+2VWEMs9ZO4t15jQ19GWtLXqabLxsfjxh2gws89iup+/J7BkbF/M4U4M8HHtQFcnJQ8nQXOxbz5O1ITJsHIQ+Tr38tNVxvBzPguMuf7XLWYz/0y/8LqJYL0l1YriH4IMfNe2q8WFZBuODn2CEfQAMPH0atzpZO0FRxv/LJx6iWUOpkQGWpu0iIOwdJ2t5fgfqYAAAWJw8xuLxewCA/89vvoVf/D3jg/G//uM/hD/w2nGhyIPQw3nNurbRLkczLadoZM7T2XkEp/mdvUtqro73zckznP1XNZhhnRjcz5Tdutv1uNIM2i8kzaiNZ6PY1YZ2xBJEy4HnCBCCQCq3JIhUPuaxLgaUs09yDPe42N/ZzzRprUpE/6oTo0tLtdqW1XlZLS/WVLkpSHJzqc0JOW2rPfUdx8vxLDjugmbGw/NZMY2sEBBhhcVPBfwACHuGP9Jn0MbB5exFZu4HGiOdJyJIPxksLcFJ30jmr+3LWYzLRL82L0nUIIhaCU5rqY/S8onyl5Igd26pdBE0LjvhRpM+kFVnW2GxST12P11pPdh6ih0LhtLvrVQ464jZdj+FMPbQQPWYM3PjF2SbFmXjtoa2wB7PsrKJzYPmc/mYV214Kd8I66s1Oo4bPAuOuxrMzJglHs2z1R0jkQqo9A6Ha/nKDKd0YP8utbZlmA0y/V0dSABUEirGfiYt3LiirKgj2eKcIPMiSBuQPDOPVXbZrgbrJ9xpn3mNgKN+LvrfHZrGEwhE5vtZnIu8k0hl8WXq2kgADh1qhTKEUiBIojK2pWFdPb6grJ99X2PgF1UVT6cSk8gQMs3Ba0MQwIggkxdLm77ZGFq5eQ97+TgKyus8nUaZU8+pI9RAGfpeHiq6yRiu24cm9ewvqVKKSXYiTXhwlUbeJLxzUWzLfqhxpEz/B77MwkK0QcfxHM+C40+nwGpA60gx/pO/+0WcTxZgIJMSKOyh97EfMQ+VvPn7BwqDg2oHj4tL4MJEpche9ETAnTvG0W4gCT9xGIIAnEYav5tkO5rNGOfn1WXv3Y4rNw8h0noIixnw7S+ZfmjFmJ5bc5KolbzbLyA4um3qf+9tRCePAQB/4599M5Ma/7O/+Bl8+M4IROad0QatNwUiWiKncGzvq1PTdMNaLXvbz7f5nf0MwR333/7KlRZTs0mFyCVqh6ZYaq9jvFclgjZVtRnDdce7yW/tvzOzc7yXgyW7UzIyt+t/XVuAjuMpro3jTjAm8xiTVSmBCJDVJsNEmTBRUTqck0iJeksIQiBMO31NmQltnQQAJJJC1d+T9glR0860j0KA0lc35dKRnbrTWHiRLUg0RutNwb75N6J1sbu2yG3DJg0h76M28mcpyurZNpr0rS0UE1ibyz5t9DqVz7tE2HThAeUWBmloAxurL8h0TojqzRQrLRm2iCZ9a1Uecnt2xc3MMVenpON4O2yD47Eu3p0wM84mC8SKESm9pKJLI5qWRjYlzjQs1nvTqKkadlmI/FlmYKYBgDFXeTDCsgvpJvVovfyZmaE1IDwCmJMNKf+7iq2DYFqP52VjwHGM4uWFme82G0PrTUEzsnyvqUPU6kIOpUDocLC6XMSZhcF+6GUhk+vy8ZbVs20o5iyvbCDJmQe1DRgmxyxgwlgTogZE4YJ+erxQWdyg/dBzhrc47BWn0u6PLyhTz8yVzuawDGX1bBtXFicOex68Def4aiGSrFzAcU9hL6xXpU1WYi91HG+ObXH8nQujNlrF/+W/+xLeOZkAjNxBz/PQ++Rnk13ZXVk41Ni7XbQU29sTGAza+49MNePfnM4BAPOFxtOnNSqjBvU8ehQnmwHw+HHe1uNXzP8GAeFWYn109hR4+9vJPQMRVKJKki+9BvmScdJbvPlNqPPTQj3n86jA8SrsxMW1KbHrbOmfJbbTrGXhe1coHe/VS8FrOIm2xfanv3iaqm/EGrV0HE+wW45rzQ5pzxzDiapfuq4pIqK110Haiqbj1tbnJ/+d9VkUv7P/g6z/X4ldSQoLZey254oSe9p2gyuJsgBlmrloUmaBGVhoU4+WqVcrJyZtu3q5UXaya1KHJ4BAJrHUFTl1rikY5nK0DkvmomzGexZT9ltukQeEgKw/gigb77ITa6wBlZglRh6DEn2HJ9ZfSHWQwr4bqK6DCEkSEYbm+rSXShPmcX27bdPMjuPL2DXH24D6A8NDyySTiNEb5OWq2BQuLX+KvkcY+KZdo0Cg5wjadxUzNMzLOo5rXtoM3OpXb0jSisTYF4TUsvRspms99P2kraFP2E/aGodAf2h+F0dAtGjOj4WqNu9eRatN4eEYuBKU/WzoA7cHzSuznYrO53Hl4ChGZmkSSAYhTl4Km4u8ZWgbH+ggVEjDmzwce+ZFUgLNhAfj+rJZm2cBhtLAN58iOREZVu2FQNMmSqs/kWZnDCEb5zOJqyhNth6j729PrVMGO3ta3cbjCeCFYWKTrQgPx9U8uFgIXNQkQQeA8Tx/puP4MnbN8cYgQu/1TwJe6hhp2hH0gNd/xJSzKFHrfPTIx2fuVZ+mfuNsgatE7XdyEi/p+1dxbyjxpz8yqCzv6+MY78xMWz4+8nA72RX+8eMJTmblhUsJ3LplOHTkC/zYgWn3w76GHBmV5uP7wIPvNeUk4fHEW+J4HdZUH633gljvtNmSPBuiTRvbd6fJD1x9Xf+FnPensX3M2nWtg7acaPf4Jn3pOG6ebV16g2fW72tZ29Ovy31g1lPltG1H1fNtHStX61n/XNbuhxvdKcQaODd3L5DEWX5QKeqDgfmCMuuNuoiZmk0sdfNsnic2kMBgNweqJUiR5y6ItdspJPQYInGDnMXVYvYmuFqk4jej53Fm4eLXqHhEcgIFAKVRmcYTAOZKgBP1ke2lvB+6rW62DV8IMIxlkistqCRGP8mFrNjwYhfoOJ7jujj+jfvnuEj8bqZO12ZG0NcAAXnKguW2DHzCrSQRzkGvnhuLBWMW6YIJZxgmqjYCbiX5tA8blGfj8URhmkgHLh4QgJf2JAQIQgIHoWm3D+D7F0Y6OIsYs6QMBiEYmHGJFwK6gYq0DTbaFOYK+E5y2T3wGHcS0b7nCYwcSShspLF3mLnWMiPWhMeT4sq4MwBeO1iv7W0QSJHkEgbGC3cY2qNe4rzEwP0rvzIZyCZ47yr//OJIZS+p434xto0NTwjsh2nSd+XMymXjfC6RJrd5nCSuIgCfuout5IGtAhFhL4vfVJL3VwJ3E76NFwJPprvZFDqO57gujv/dz38XX377rPTvJID9uzFIAJ4HuF5jLw4lfurVfuM6Ly4UTlfUOkTA4aGEEISBJPzhw2AtSfBLjxaYz8sn35fAH321n0W1TfFwHOOfvukwyRLAwT3z8fKph9nFdhdkl2SnQ4cOHTpk2JpJaqSBi3l6EhWYJIe7UbA78XcaA4/Gxe/7ns6ih4aeWMtBR2nOol0a0dr0zRMCDpcALJTOToJDX0GzsaoZL8TOxOxxJDCLTelK5w5Vt/qb6B/LwTC25KsGOYIYQ18nn2mt/L7MvDSGPU9kXq6u/LKaORPFPcnYC8wpdqEI8x2pkjqOXz/H5dFtkOeZi+YDgKQCCc60RcSEXmQa2yeBO8emD3cG7tPze1cxzufLEgEDIJ8wWFEHSgG82veM6rVEZzpWGk8XiXVWxIgiM0CP5grjJECdslRGvR4tJRQCzHr61mlUWFeXi7ydUgJhIunb9QQ9nV3RRB67Mt62xhY3BYFTR3jYV/Z2t2CuFubfKm73NYZBbk65TnIZxXkGsZ6Xi9ahJxA6BKyzWQSdeCEe9nLTz0kkdpR9i3BhhTo+SYLFeAI47u/uuvj+ZfE7TwAvjRSIzHivsykAwCTS2V1HIEUW2dOVYS1SGgtlVDmhZIR9M1fnc7HDTaHj+PVyHPBfeAWiPwCIMbq9gJDLFnRCEwYz4/S31wM+9lJ1ed89i/Ht06I68vZtDwf+cp99MpZDsmLDvYgY3xibNl1dKVxeVuvURiOZmZym0JrxxQfzyjH0fcLBgczqSTeFcKgRDk2d04DRPOpZOTr1UYcOHTp0yNAuyY6ncdjLd2qlCZcLcyNfhrO5EbsBI3KmF6M9T2bOMwNfZmJpGnlSkLnY4ob12BhHInOimURUaS0jiLEfqkLJtnVOpBhjR47b0BOZBUrfE1k43WkS4pdg7LxXDwCCCH3P9EcxY2a5oLNmPEmCbBExDkO1FLvlaiErnbY0A+9emrpl0jfAqAfSBC2eoMyePlI6s+4Z+BpeMj919azWeTZLEoOQ+2RrY+BrhLJ4orJDH0zj4pxQMm5pzoO0D0pzFgakJ3mJoyl6UkAKAYAxi/XS/OopIxWAOo4vY9ccZ2ZMIoVY5744beBJ4JVj046ylN33r2LcvzR9ezzN29HvE7zEOexDQw+DFac2O1S6jcu5xjdPDMnnAqk9BoKAsLdn+nAvkJnk8cZplKmsxmOVqY9GI1FQJRGZ71fBbC7DAWRSAgDECw86Ng3QTgmZsR9qaD/neB1abQo9T+MgzBfzPE6JXA5b/L090Bj6lnoAxowyfVlp5oxsgkwY5Kb12JjGAtmdfc0LShLgC12pg1fMmMYOs0iRe6+mGcuYGfNYQbGZ4P2w+PKTZMzaiIyJ2oWl49Q6186aMVBLxJlFAnHFi0NzroP2BeAlffOXNgWRtXsCINJJRjSf0U+W9ywWLTYFwsXC0uHOq58/7mnsOcbFxsxh2iIIyYvGjP0gCSk5j1WWqSv0uJAdDAD2AoHQM4vrfL7sVTqxNqiO4yu/3THHAbPBNOXaKjwJvHBQfYf2eKLw1SeOLH09gV5iXvrqQGKvodpzHOmsvF6PcHSUOCAGAkHiI/fa0MOrffP9w3GcmTVPp8ZrHQCGw2J9QpjvV62cplONs7PijYGKJeJ5EhBPu9u/FyjMHIewMnTqow4dOnTokGGji2ZPMG73m993h9I6nUUKVDgNrgZ/LtazUCun0rXAOO4rGGUC8GRaXV7PY+wll3rjhcAkTp3AGAvfiKUDX2YxZYaBZzKfwUQ4Xe2TZsZl4ifQJmMXABz0FEZVPvgWhBXrPdaMC0eYi7LQ0gehwiixKDqdyVaxU1zoezqzUFpowuNJ+ZgTjFpFCiP5nEwlGOZC2xOJOoxyPwBPCOwlJ7SFylVJNqZxHmm2STjtFB3Hr5/jNobHMbyRibxaE/8OpzOFLz0qik0zMA4Pi/3/oQMfB4EwFm4O/VukGF+4P8Oqv1kMZOXJkmH99mmENx8baeLptMhHAvCJoY/QJ4CBWcLliIG3VOKYFjMuL81nZVEwDAn9JPbSSUSYJX5E/9YfehW3Q+PAcHzU3EdjFRttClIgs4Boi0jnYlSbeigiYMMFQzC6YymMx+rTqY8qXa6gfEYWmrJMVH0vhkxyY6b6U6LcM5SZMXYYiTHgTO7eBD2v+bhtUmdaj1G3iI03BV9yNoczK5tXWWsPLUepSWRMHj3BmMdRZuWUQgqCTFRJihVcdnlxC77Z6Dh+/Ry3EfQ1glGzcqYx463z4sFnNBLY2yuO5+1Q4HaFJ6ZiU97qphAEhFu3ql+dT6cak0l5uwnAnVBgmFo8Jd7XE6Xx1mmymWrGbFbkj+flm4KQlN1VvfraIT5yBwAD/Us410ETdOqjDh06dOiQoX06Tqzm0q3GNNZbOTGkCCXj3rDaGvd8JjHb0FY9kDpz65eWGubOAHhhZD77QkJSfvHpQpuxAoy4vnpp1rdsyOug2SR62SZuDRSYy48dSlOinthEmmDc6qvMISs1u5cC+OixKZoAhEnY5LKLxVAK+C2DM52u5HHuOH79HF8XVwudhYKw4wqFIWVWPK8MPHxgWGzjyOHboZnxL74/wzw2iYDSIoUwKiMiYN8X+HQSvfTpQuHNxKJpMsmlA9th7XMvBLjrcKbrOepXCnj6NE5iMOVlvDCUWaTXN58Svv5d89vFgpDc/+OjCvjsAkglbS9xKTzsebhskat8jRzN7eLru/S7m0AKQIpqkfxS8NqiU1YPpSqUZSwHKCNUvQiJqBDPpA6CGLQSNVMKgt94U9iuFxHRsp7chUgxMru8DRDIPOBcVj+Mx7ChW/V4A4kqqeXmtDpFHcevn+PrQjHwaFIcCCGMNRAAjHzCkd9s/TADTyaqYIlFZNRGxiw6L2+mGOlEaM1L5qIpDkOJe45Nqaz+xaJYRs+jrIz7J8DMComU7tVDAIfJT/uSMw75ggocr8JOMq/Z2JVnbT2aE87Qc0cumT9QaDOG75/x7ji+Q+xwcF2hrMtGiMgKY22FwWarnDbnsbIw2mVF2PWY31PerrSNzauvxM43hb4vMxvtNtBcnxSmDMc9hcOw+hiV7q6SgBdHRVHdPiQGUmTJYG5gRsslGMud9WIujCO1lhrEhLmoVnfYmofDUGE/KM6PbSZ+EHrXksh+G+g4vjscH0n0jz0IAD92GMAnQsSM3zpfVEadfaEn8fG9xDqtpEO//u4MD8fFMRweSKz6jw0k4ceOAhCAk6nGP/iWcQaKGVgkDbGNAkcjkV0Gh0Gxfgbwy29OC9Foy/r07mWc1Xlx5SNSJvPRv/cq8LN3zTOv7wOHiRXe1cJIUczAt54Cp1euUt3Y+aYg7K2sDTaQyKVorswgMqFrqyCoWerCmwA790Hr365dZ/0Y2mgyP7tNSblddBzfHaQ0XscCJpVmKAhzzfAkVQ5fIIFBjcp1EjEuHaqavjT12vAkYZCoFC+gnb+zIQQyb2mXKS3D3IdMHA6DLsQaWZ1z6zf7HvBiklNi6Jl55sSpMBU75gqtwpyvtSm4LO3SCJ3bxHa0jtvFdWRpWoVmEwV1FYK2e6q7KSdEuxnPYryBjuNtsc0c3iQkIBKvXqvcWDMEAzEbU+k0OJ+rfs3pXVdShKN5gpal0xSSzIbLMJfiDEBT+mLlgolq1gbk/PAEZZt2WjUnF9fM5jzQStm69DA5/2a/JzTnUkfb90TrTWGhgC8/Kn5/2AM+eNi2tHIIAo761WqQWayzODLrglBfjz2e01hj2rBOE02yOvlNE7xzgSxMs42P34YzxPG6GPpyKcewC6dJRqxNMPClMxy2jWwhwUTnbPqe6nkic2pbFx3Hr5/jNvZ++BM4fi2xQksyjClm/OM3JtDJmy6NNeV5lOU0tvG98xhffcfElhgNJYaOeEI//EKIn3LwkIQZj7li/MNvjxGbJG9496FZhDYXe708eukH+h4+3M+t49IXsS10/LO3pjidpU6KDQYjgYo8zCeJSMC55DdTwGly6Xw2S+eRwNar/WO3gPsKuDhtVlfr1cNAYadEyXebII2lX/nMVupJ7CsabqWpF2ezZ9du1hI0u8d32+GK68Z8nROksx6g1X1BCwfkraDj+PVz3AZJD8IHzEzkZ+qFYqw685dx0qRoNZ8XmhE6HvMEEHrlY6K0qY85dcZzP5fGJqsrDzDSS5vNYAmWA6mdej3tWpk0IVtKuGv5KZidr2g6mEZtLBPLto1tiPNtLzNNhM7qZ+yXmE5E3eUy2i1SQct25ClinY+5L65H/SOINrZiadvOuvFme2HAHcZCUPPx7jh+/Rwvg1KEOGIw01Idfc9sqKLkDUZk9Prp5xQe5Sd3Z0zRJDCg2QTcPBeU+xiEHiHJfVN6ob2wcl6ve41EBMjEfJg1oGsCCHoiHf/26631piAJeHmvqMuYxAJfeWSKuzsEXtlvW3J7hFIgbGi/vy30PZFF6izD2SzKTpVns6J1iYkg2Xzoj3oqczKy8daZjyjJuPapO+0ue9dFm3ZvA4R6a6q50lnO6VmsnRFW9wKv9hSXouP49XPcCQbuf8/MPQHoH8QgwehJwp//oSEkAWPF+Px5Md5Rr0foOer/6MDDy0lICRcbNAP/5DsT46fA7pf4nb7En/xQElvIsjEoY9dXryI8SbKojddMbD3cUzh81ajDzp5KPH1UvSY+cpT7mrTdltdyXnODS8SY3WGbl1vbqnNbKpblOl31mBFn0LXdVN7E8Qa2b8recfz6OV7Skozbq9Jpai0lSnSLZX0wkld1/+xL2rJmtbHUSiXZNmo5Z7UtqGBferfFRlu5ZmRx0O146LHOL0Z90fwEa27n82GT1NyrtA22VY+xTGjuAJP/Pal/ZXHVa5iBSAEMkxvXjko/jc3lG8FcPjftjmbOvKBFkrxmF9hGPavzlqKJrl8zI3ZEl61KtWh+13H8OjjuCZPzAQBe6BHGiZruwcxEDrWhwTidaQgCZi0vnGaKceE4rcdxatG0vCF4jjek7ZISaca0pg1xg03Uridu4bpyGgHfTy6abwdAGk1DMWc5Q+o4XmhLq6dXECnCg3GxiJNZnjP4lT3g3qhZeQzgfBZnCUjWdcJqgm3UM4nUWiEOTKYyR5YrKWqtf06mHmZqeZIZwLdPzGdPAJ++2/z0bFu3jIL1nLDa1jP0Jfpr6LrKxq0JxpGCK4Ft3dx3HL8ejh+ECgfGHwv/y1c9TJPQ3X/928B7MywReqGMigcwL9Pbt73GG96b0zxWUQZmPH4SF17GRMCtW17hgGXncn6y0Pjy1WaZke16lAIeP27O8V9+Yv4BwH/8AeAz+wDAuJzH2Z1I27nfkoL4+kXcH2x043396MZ8l1h98RJZQsaOh77uHH9dKjyyPc6eIVptClIQfEFZTt8mmMXAeXKiGvjVYjYB8KUAM+88xEFdPdoSv8owi5GdaNaFJGTpIzUzFkovieu+FGBwY7NMk//AWFYIsoPJldefRhXd5Zjb9bj0sZyMd1U3Y82YRrTxsul5nOlbI62XrJU6ji/jJnE8CAlClAeN8wk4SC7IZ5pxVaNXXCx0ZuJap+GRQBYELxSEx4lhw0UDKaqsHhNgb9lCyBPAy0nuh0gD54mqy/fzOfADxmCU+DrMBeKoen5WOV6HljmaJUaBh9NZc3HpydT8A4APHwJVCYGIqHUY3nXQpJ6yTGU2nk4lxtFm7e15Gvc8U0+kGfM4zoiTtrONCK8YePM0LRv45O3q50NPZrl3d4km9VwtlPPOIEWkgEeT6mQx9WC8MIyzl9TVQi1ZK3UcX8ZN4TiRCV3teSYs9aNHxXbveQKfS0JavztT+GqNWufqSmM+b/ayDCXhs/s+iAgnkcZvOyye2tZzeCgL4TT6nsBPfSAAEeE00vgtRz2jfY3Rvhmrx+95OD+tnp9VjtehS7LToUOHDh0ybHQEEIIxSlIIxpowi6v3mItFHphp4OulHMLXCSJz4UVJCNy50mA26pNAUua84wrFEKncqqPnMYgq3BMZGCepJMugNOFyburxJSOoiaPf9zW8JOfAJBLQFWkylQaeJPlbpWD0ve3G/W8Dz8oJESmdqS0CKTJ1UuiJQj4I5jxfgVGHVfdhoQiLmuQzk1iYSzhClje6DB3Hr5/jNjxhgv8RgOlUQ4iio1wdooixWBTn2U6E0+tR5pn8YigSay1gr28S6wgA7yThKSYlaqkm9by276GX+Mvs9UVWZwpHQNW10Ibjq9hoU/AFcCtJNj6O6hdM+oICgBeGKhPhnwWCvknYrRmZ45MkQiDzZPCjoNif8SLOEnCMAo0qoxPNZnKqSBxpwsnM1DnyFYJe9QTuh+bvzMA8pspNIdLA9y/M555k3BttmJVlA/Q9kW0Kc5U7mKVJcYjIaZWimTGfps/mfCvD+VzUbAqEi3laT/1G2XH8+jluwxMm4ikAXF5oUIsNJcVspnF1VV3naCTh+8Zg9seOQ4QrL+txrPFrZ9Uqo/lc4/Kyup4fuRvgVn/X6tp2HF9Fpz7q0KFDhw4Z1vJoTk90mrnVBYaNy4XAZMNUwqHUGPjFk8N4IbBIHI32AlUZo8buTxMPwCARyVcxi1XBeoJg7K9NqFzCxVxgnUvS0MtVLK56miDShNPZZicUArAfqsI4KQ1cLEzZnmDs1ah4Aikyi5g6xxoCSiUIF/d6knEYGmIZEbr9uafj+LPluBCUpRr9o8fAODb/+StnAebMIAEc3Yozs9X0dB5Jjd+ZmgZexhoXiSrHlSITAPp9ynIepHGSGMB3J3EhJ4ldRBxzlot56RnroTAkBEGqjpIYJgUOHKFWFDPemqqC6nQcMS4uipJxEBB6vd2d51tvCsIS8yPljjPTBONocxFqLwAGfnHQJjFhkpTf93SWDN4FUaK2KIMvhdPkcKF0YVKJcnVPrJEsmPYIpMhE6MhRTxMotkXKdcEYBcVNQTOyl0EodaNNIWjYlDK1Uhn3Qo8zlY2a0loRKTuOP1uOpy9kIuAnD83nSBN+9VGASAFSMoZDhhDmRfzkidl5xwCetEhc3esJ58v1+7PqMpRijMfVnAhDwnBoOvT6vo87FYTXDLw1iQstXyx0ST0CvV5l9RuhUx916NChQ4cMG100S0EYBanaAGA2O/Yspq2cklwIhMZecjLxrdPReCEyJ5tAMvqJXfRBWDz1ELbvJDnwZeUJXjMwj1XB8SpSlKleZkrgydSrVA/1fYkwqWeudBbP52QqKy1A1gfjMFSZBUgqJWg2dQKJm35yGRtIZJyw4W05ZZnNvdKWMyOQRf3N+UwiTi7oT6YexlH52ajjeI5dcJyZcbWIMfBlZohQBl8CPgzfHt33kw5qgOLGne33CUFyuW47hNVBKcblZZLwxzrS+z5hMDDl3Q0F7iVcCQLKfBBOFhoP5uXSB4Hw8VExUdHZTOHp082NQ+o4vorNTFKJslg5BI04ScauWWC8WTiQUkjhNktcqHyR3vZiDJNnep5XS7ZtIKipQzNjGBQHZRrlCybWhChOdbLuxZfWw8wY+jFUEsv+dCZ3Frmz7zMCWTQVnSSmiJ5gHPciEJmXf88VRWzLsLlXhljHIFrmCrPR9UOZ1+Y0pixHQl09Hcd3w/F5rBF6AnUReqQAPAaYCVcXyZwIoL/ffAMMApG9xNuAGZhOiwtMSmTl3elLfGRY7MXvnCs8icrVTR4BHx95hXwMcitZneo5vopOfdShQ4cOHTJs7UjnS8JB4lbvC0YgiyeGs5nEfA1rEBtzRXhwZeoZ+Dq75BoFCn2/KHKnMGJqdRiFJuh5Yq1IosZKw7RbJW0BjBrg3jAfK60ZTxo6PKVqDQYQ6bigdoo14elUYlNFwtOpzLI43e7HkMKoke4OjfqirPSF2k5+4f2weRRMG31PZglqxpFCrE2i9+O+AnPersczjcsG5XUcr8Y6HCeYMBe2FPLKfp6A/q1zExVVEvCXXzZWSBMF/K13AONKITC7MnFFhNQIB/PKNl5dKUyn5ad2QcBPv9pH6BEixfiX359C8XLMIs8D9vfN+NzpSfzQvpEO7Dvrt6YxHicqI1cMJgLw6T0fgTC+EelojyONf/P2zITGJ+D4uDgP07GHd94y4/zTe4RPf8h8/3J2+cw47in4lnTflOPAFjcFQQSR6NACrdBzOO2smnmtA82Eucr1qimMvrF6MURab5zvd93fExH8ZADIiusvhfE2zsrXzW4G7PKYGaEsBpMzoXM31XvbomdeBxGcc2yjScC1+trXR+oYBwAizjeBcEUVVmW5Y6PjeDXW4jgZayc7aF+aMcx4YJvPgoAPDczny9iUafIfEHScvsbq7X+VWvYwXoUg4MAjDHyBGWksFlzI1yEEIUxycA4CwrFfPARMFOM0rh7IQ09kaT2z9mngwdhw1fcJt/eKr+jJJWE2MXUeHwAfc3gXBpKXHCebchzY4qbQHHbjynXnO23BhlXmvy8v6FlkzCpH9ipf+e92aLRZlbVg7RdN0tr8/5U8dxPHG+g43rS+VVPXev8V+3/zcuwytteeunY0/U36DBFl2dhcZa5+u/rYLpPf7WRTCK2QBlMr+uFRP8Zh0pm90IMnzMBczOK1bO9tg5a+J5xROF0nNwbwcOxVhoioqjP91VFPYeCIK7If+pUnRpO/tnghtVAaVzVRK8tw2PMzkp0nZfgCeGlkxHZPCOwlov08Zkyi9erJbMgBHPQ8rC7Lsm5fLUR22dgGBOC9K/PZE8DdYVSow5cCo6CaykPfy06gOeqjhJah4/h2OM7MOJtFS/0cBV4mcbx+ZPqiGfjWUxMFeCiB/+MHzfdPIuBvvp2mu5SYXhhxwvNjBINiWIrP3AvwwQPDlV9/Z45Hk2X1pmbgn745AYGS7HFJfyRwdGQS4Rz4Ap858LN+pnhnpvC9xMdh4RC3mBmnpwpxbCSlX3oaFzLRkTBJg0DLm9pkLPDkQbJ+J4Tp2HwfWVGQX90H9kMAIEhKkwO15/hONgWRBNsCljtme12G0ojDzMBUNs8XUAYiamX2GOvquEFNEGl3Ksi6rhARXDnk1/SRAhFlC1SvnJRSU0VfMPI85owKY4jGaJPi0U5ruS4UG/F6tUrZgDvuHA7rt6XjeDXacFxpBlv9slWUafRvzck4Jyqlu2H+LCWbmGIC6+Sejd0E73uEgzA3MXbhcsFw9dDzTL8CCQxdgQS5PFhe1lfFmUnrpSrWIyVwxyt6lGsFRAtTZxzDnZ9BwlrjVPh7U7TbFHhZ/GmCUnUCtpsAvJlqw7AqteFeTdfC1pNN4V7ovCLGrrkwk/FmbldGmcJiq+PdoDn2I2bhOk5PhSerYXiz8h0v963tWC3rADqOr2KnHC+Baz5tCSaFIBNZNN0cbNnANfTmcGL+0GZm7O4xkF3mE9zJqdad96V62OqDVZwkII1j6An3TKb1Mxwcr0GrTWESKZzNYhz2mv9sYOXjvZzHWUarS1uEdPxOEDLxsywZSCgFhkFqQ16Pg6TdzAAjKpAm0oSHjny8ZTiZSmc8Ic0xfGEsXQ57RaeUMoRS4KjvJyQzCdNPZxH2Q69xIhwCcNQ346Z0rkqKNONkWm1YP/QlwuQEZM+VjcOe1zhjWCAFjvumvJ6nMfSL9T+Zepg1zOwVa+Ddy6JKYuBrxNqU3fdlq5AOBz0fJ37+Kuk4voxdc9wFe9wOej68ZDP4xO3ii/71GEjdDt6YAH/r3eQP7IHHQwCGN6lk/AW1wO89NLqXsux6t297WSykFDblz2PGvzoxVk73QolPjoqcvLzUTisn7RBgiNI808v1zGeE+983SYMEcingp24Df+aW+fyRI+BF002kBlzMhkvp5nfQ81qtiVabAif/1wZEudZsdbet/W32e/fTRM1TSNrtYDAkGZMvG219RRjkOI2wOd2jveiWxrhfKo3bnmjyfq5qa+pF/vrxNGqT9mMuKCft0jONSsqfdq1jbY1RmxNa2r5lQaHjuI3r4Hix9NX/MrkNytRRo+Qt1pf2+Oczy5aWJlKoHQQhUMiIttqiNPZdXDIAzO4NoKrO1dwKzIBOrNCIAEr6GUpg3+wV6El3+ldbMUWFm4tqbHSnkBIjRR15CeXkX4VdVqozLJbnhn2hly66Yvm5WGy3yHBhuY3mpdN8WBUTKDG7M3UU+1zWrjo06Zv9d0Hr6RWF40VVVlUTHtgv3+VnGZKKrwEzbs3Gh5kyXbViOC901x3vjuNu7JLjq+3SiQo1bSPB5NhOLdOMRSihJ4F9xwtyzECWToEB7bjbIuKM3+Zl3mwOoxiYJGLIIubM3LWt+kjFVDjEKcsL2RN53/qSly75XQelTbSWG20KsaWeCKSozQlrnK3aW6BIIhz365zgc4wXuTXIQZhbMthIxXbNyPLx+oLx8l5RxXG1EFmSkCZIxXMBgCgqnIw8kTtBtYWtnjhMROsyCAKOHBYgTdBmrpbHkHDgqNM4RBlR4cqaHxMzadkChAHcv/Qbn2qnMWVqpf1QY6GKc7gXeAirBqsEHcfd2CXHbaQqNeacE5KAl/ZMWBVJhE/dNfW8NgdeDopl/OpT4JeemM/RPMRiGhae6e9NQEkymjTqahM8Royv0Txr4zrvYq2A738nLP7W+uKHh8BfftV8Puhp3Orla+ZkukalFdii9VH9cKx7amj7u3o1ycptTvZ9q2rKSrfakVzGOVRC24GjcLslG3SozW+55HNZeXbRzmpaj09Lnc3a6DielGK1Y9ccz6GRSJGwmZ9cqpM5BLksilKpwjzuJty6zV1HheYsp0YyJqtvEtuaRzfW2hRUci2uObfTtUVhzZyJt3U6aObUFtg838bMsQyS8nbVFUUEZ6IXRm5CKATDF+YUobi5mR/DXOytnng9kY/hKpRDD2ksJnTSXrJ8BVKdqT2G5u9VY9hmfpqAkI9hXdKctE7Xc3Z4Bl8yhDb9MdJ5szYqhjP4V6wZskTJ6xK/O45fL8dtRNqoBAFkl9npZ828ZIXEsOsh9BNp0J6f2yHhw0Pz/SRClmPjyQKYp6adWkBvmBBpXRAIL/WqGX43APrJ21oIN8dteIIz6U1xO3Po1puCZuBsFicVk9NKYx5rjJOYN0PLMqMMF/PI2CHDWM5sugkOfOlwVHKDAGcfIssaZOAxBiPz+WwuWyWrcVl6+IJBiJ2LWWsu6MQnkcIkkfj3Qw++46bufBZlp6c6NcRC6SwuTd8TGNY4ftXBWNE0L6PvCfQdR7qzWZxtDHcHZrw1G4ujppweR8IZJjjWsTODGWCcz2x0HL9+jts4mXiYJS+9F0dxFurj3rD41rbnyheET9w2gzJXGpdJ7KF//0jgf+WbNn7vPM+j/V++BXz1CgAI87Eja01T0m04mQMJ/LWPAWFFyKyjPvDhQ/P54Vjg3cvq+FovDOMszMXFPC5wvApbUR/drBAD7dpT/uw2RO76stcq9Tke77LnuUzFsYkqaYt4nsf8eeQ4VrxtmravTDVZq7LcEW/aoKqPtPT3urZuNvYbWx+ZS8+i9Uh6mm2imvCFgGYutcAoC6wmqJ2H500BMzlt86UAvJoIqWYcivK3L6mxk5sgyubH5elr6nEHVvPF5qqPZ4GFcpvlhTVB/TqOr4dNOB54DCR5METDiMGAkRoWiW6KGU6O973cnPPj+xqrd+EEIx1mHG9SfePpMVxaFZJCkar3yguKNHCeBIBtErVirnLz7TqOr2KjTUExnA43PU84LVBcIKIsJk9pPSWOPT2vPubNTUTMhEeT4viMfIWjXrXCtSwU9XHmEFQPkw+3WvwcL5TTued4C6qPZ4HzuYsnbkscGx3H18MmHD/qrRduXXE+hqF0z8+9kfkHAH9lWM7xXezDzCab2mpoc6XrVaRXC+CNk6Y1EU4zS7J6jq+iS7LToUOHDh0y7OQIojRjFm+WXGW1vDZYKJ1dZAUrsdqbQhCyBC3KEu1DqZF6tc+U2DjIm41IE64WYi2N4DzWWzVTazPkmjkT2wVRrRRShtAT2VynNvhEJjUls1GaTaI0leN2MI5ElrugDTqOr4dNON4Giuvnp+O4G+3CXLCxHKjDXDPm64b8bAGt2bmYJguVTeB+uF7+WgJl8UJmscIi6U9PMnoycXKZekm+2e1gpgnThUzMAVMvTnZdIRRwuWYI6LZQmouhEzTjwrLWaapWWUUoBSCNHn8eq2zRHoZp/CbCeCG2agR/OpFL1kodx28ux9tgoTnrT1u83zleB+KG/ti/+IX3THyOG3TJaC7t3PbXaa/KnmkDuzwbbUIxNK8LYMue3ttGKq8tomz+01PrNsbbLq/4/fbHw0SiTXwOOo4voeN4jvcDx3/uJ16sfbaVpKA1oHcu+LVFk/bsss27HY+4bQSzneP9Pd4dx6+77I7j11t2PVptCrShJ6awgk4tlYsSO72bAq53hFdMW5tLbZ2ixGoM35aQJblZb7RZaYPxNmqH7fTBlhQ6jpej4/gW8Qw5XodWm8LoYITB3nCtRgHA7X7sTO131PMLYWNvEmaxwnhRfWn1cOxhrjbXvTIzHt9/DNYaQgjcfunO2uT2BePFUdEcLXwOzBxPp1Gl1+tcER6O19PrrmJ8OcbV2SWAjuNV6Di+XTwrjteh1agNA8ZhX+FsJrHOsecqkpitkIoA+J42iTTIhAwgImjmjKRSEPqOFHULpbPLvtDLbe+nkXI6Ag0DuZa+OPSk08lrFuusnsOeLtgfMwOnM9kqJDE49+UkYtwaKEwiudZiVAxn5MuBDwgyYxtIkSXWWRf2XNnwBNWGf3CBmbEXysI5SnPupxEAuD0o1jmNCZOoXZ327Fwnx///7b1pzDXJdR72nKpe7vbu3zorZzjkDCnuEocyLcqII1uwASFxkB8BIiAIgsAIYv8QIiDInyT65x9BYCdBYgSBAoOJFcVBIluSJVqQJZmSTVELSXEbzr5wvv1717t2d9XJj+ql7u393vt+MyP1Awzmfv1216mqPl2nzqmzbIKOxw06Hm+GNhzRSij4UmPg6viDaY+FEljkxsjYDUO40nR84ErDMJy5bDlMhblylOb0HilM7VTA5HQJCrJuDViupcI7guCI/JhDxYji19p381Gvmk0emXUdCQQBI1cjKJy3ejAIk0LmyYqxCwL8DcNVtPWulrFeu0RUWGlOaU4/GCmMG1/uHhZpnqh18Ch5fBN0PG7Q8fj20QWvdejQoUOHFJdidBs4GiPfSLvxQmLa0M9ZM/DaSVasItKOUTUJuDcpul9A6Sy/SZoqVgtoztN8OGu+ifKlxn5NuL2tvg/dvPrNDIQqauWCzZrxoEW+l3UwjwTujk1fd3xO6xvbGLgy5/vOzLgIFDjO4bPjOWkq76LiM/Z0LCKF+YZ+/fasSKK0drENRzA82W4bdX+u0czauj4ezhwr73+US6MgBRWalRaRToOweo5Id5cDNytaZCNp17yraGN3947H/+Lx+KUIBSkY/TgJ0yxs/vIZwMUimZjqBFFYvWdp/S4RQi3U08gx9uWmcAQVMljPaVfAQ+tW1tm1oJjSUn++Ks7/UtbnUOlcaR8iglfja664vFD6OiBCYVRppFXKe03hlnivbA+URpRSnBQtV4u75EmdJuQD7MzfTgOPnVCtXzymCB2P/8Xg8dZCIZuWIiLWX1vOzWVUatoUTfpkJ1NsW5e1ZW/KerB5y1bTywW76sdjbnlvXt5lzXdTHl8XhlesFlvMefbnRz/nHY8/elzufBejtVCQhNgFbPlFzaMsM98kFGnaXNXAz/be1AWBwY1Lnl8+5hHh9rh6evZ7Ki3cMg6iwijHXd/J+a0rDVwEzdNS7PcUdv28Wnp/6mDTTAsXgcA0NB3c6ykMK8eT7QgVA99/YD4VR2RFccrgyaxYzTTU6SHp0JWF9YXP5lElL0Sa0+IqZXRslGV9LUITHl8XDODO2PC7JOBaXDimbDy2y+LdCdIDxoNe/W4x4T3NWaZXSSjM2LqINGY1zNTx+J8fHq9Ce02BEHv5LBMPrRenW5TzAyh+6e8vH24GIaxJBKY5qRabpAPIvxBz1rHaTjsuN6bjvCsg5ZTc9rDfVZPx2PTnUfbBRLo4aCsDpSYPQdn4paCcKSSx59ZtklbdI1fpLF2l5rbDJjy+PjJ+1yt29eLxZIiUmXMACBXSimRlSHhvOXcSFZYDDRvY+Dse//PD41XY2pmCmavLUf80M6K6wq5bAFHeZlqHupdqDs3zzL4NVM95/s5acHnfCqN0KWbu941+lzdxbDOqdevznbRW0GRht8m6ueVBbRYXUPL3DaZplcff13Pe8XgtWgmFgSuX6v9GVmGQvsN4oqCYw+lCYhys5/Od4O0HE/zCr3xrozaa4OmjEf6bv/WpxpN8PJM4qfBnN62EaxfsIBAO+u6Sx8f5Iko9Qq4V1Ky1EWoqrJ9bhpO5xGlBbd7rwxCrThCCgMcKIkkdQamXhl3H2IapL2waLJua/ZoslKHiQvPEPMoCGoeeRC/21rGjW8+sWtBFfWvL4zYezhzMCiqOFUGxKa6yir6jC4OW9nyFXc9ct3lqx3NS88TpPMylhBZk6kJXwZcCXr96Q2Tzno3TeRjTycwakiidx1BpXJRES3c8Xo7L4vE6tM59ZKuJtLQ7MPbK3DNrdWsZkWacXWa0RoyLYTsaDKrc9RPYFGtfdxLIfGjLcx7/v2S+bbRliuLxJOr78h+q6Cf9LRt3XcQtUX0Oyqoyjclf7LEkNBO1vWzjtw6PLz1f/efc3UUm4DLTqyhxyBMxn5QdSjaZzyb3lCGhqld2r2T9Lifc8XgZLovH67CR+ais3uy6ihYz43wW5p4fz63FmgiQ63U7/cDZ6iMD0EYaK804mzUXDANPwiuISrShGIXmVUHrCQt7zuvmmZDsKJu9EUL2ATPXuxna3bfvTRYHm2EZy4tGEc22aPJUKU2rX5U0qHycxfdzq7rC5v2YLzgREFXPF/Vbc/E4y2zm6865KPnes78bIbdqajFnA83R8Xgx/TJsyuOr2Ego2CqijVmkS2sJV2EeKvzc//lHmKzEu9s7ILGzD//Z59t3FoBUUWqKnc3iF6oCqLe+DYDxzsMJ/vYvfq1xe3/nrz2PLz1/vfTvDOD2OF/T2JWMGzVqcRl2LBXxdB6iKuuwI9CqPuvIUkXPFxGCuPEy8/ZBXK9ZceYlEWnGcYFgnUe6MLBn13dq/b+L4Ihi3rMxCVVhX/Z8B05srjiZV8+PzeNBhRkkwVFfgRsGxEgyJgSiZU+TstkYuDJN93IRZO+nzMvntGBsjiDs1dSLLsNOTYI5zYbmKku61ruaR7q2IFTH4waPisdzdFvdvYIy6beJGhpEGmHFoTIRQOuk2mUGkchO/OOdOmur6hZQSXsV9RW6io/JNvE9tnc5RSrv8r3t3oUxQ8TPYtluXdh+0p/34Axu08M1qjC3lNOpp9lmzokyjbHJfC/3Zz37wCZODnVzTjmXivyzTV5bx+Mx7UfE46u4nBrNDCQbqrJ18+7ZLFcubx4qi2kJ1OvlniPPb9UXStd8AkEAYBAD0k2EgwD3erVfC8V/15ZN8nim8M7DfP6NG/v9Si8m4+5qfgtK3PEMA8oWjCAFGnv+GdW2+h7NWVqDJqyk4nwkZel/7Y9Oc3GbSjOiGnOLpHXNHVR8zmVdk1TkTlmMJjZuG3W1Y5jtOaxvjzmro1z2sdsmnmL6bK5X8HtyRrEOZIExe/m8gDoeL8D7hceBSxIKk1Dg9rh6N//3v/J9/OD2eenfyXXRe+HTG5zSGngDB6Jg1lKlTPsI+5+olAmCNaQyquNCScyVUT//j2++ji9/9ZXlfhPwP/zsi3j8cFDaXqSNWQkAeg7jemxK8qTAwElSH9cIKaJadd5GaHnRlGEaqsZmPwYKA2tseFJgFLt0lJkUJ6EyTvclIKBWhS5D3xGF2XVt7PoO+m4zzdMVVOsxYuOsxFsnQZM5tDFrEGC257vpInVccD6nuNisZMP2nGkDAgqDqmyYwCu343EL7yceBy5JKDRR6BppNWtKz1wzFW1w7HlQ6SDBy2p7+rtoDCW7hZUWlx8o+1NdK63m5j3Qf7Gd93dZtNv27X1dycvCumaDbdBtdE+Lg/iOxzejvU7fWgkFtZLDnYDUTBIqYBYL1TJh/daDcerdM7UOx8Rwx7bzmLZdF+ufTmTQigGu2F0xg2qkKGkAsZugkAJOfA5BwwE0B4ZOSoLwyt0xjieLpTZcKfD8zd1KNU4zx8m4Mi+pUOmlVCFOYfRoPQSZne5lQ3Oxm6BNXzE3MpckMKH7qyaJZknhqpCYY2wzSxWPt4UjtlHevR6RLrblu5KsjMPNJ3x1DhKsy3urbb+XPH7+7huYH98v/TuRwNELn4Zwynfu6/I4M+Pi9e9BzafLz0kHo09+DrRSz+KyeLwOrYTCPFJL6pkjCHvC7JzHAfD6afXzv/yHb+Jrrz7IXfc+9BEIP39+sA2Es2pVkQjo7fqNJaof/wcA0eAJ6MVNAMB8bgQDM+N//J1vA+F86bmDgYd/+B//OIRTTifUjEWUpTtmGHXYLuO4vieDwF7v8stnhErjrGBX0HMyr49xEDVOMcxAoUnAEYT9LYznIogwt1T7Kh5vi0dVDvJ0HuYWfSLCrm8WNqW5lQfKQunCgjJ7frEptg1CrXG+eO94/JXf/CW8+du/Uvp34bj4G//w19EfXS29ZxMef/kX/x7Gb7283O/hDob/4J/B6S2bnC+Lx+vQFdnp0KFDhw4pWm1lHCHgSUp9eyNtCtcAKC0P9+rdc7zz0KhL988zk4rY3Qc5LkCUU5uq4HqM0Z75HUWMoEW9hiIQAY5v1HytgGDWXE4qBjjeQTp9keaxifYPgCgEmBGeHQPMWEQKv/fSXVNMpefg888c5Xafgoypwr7sO2JJFU2mkMDou/lAKYI5/ErqXBeZASRlOZ4indXgdYUorNObgOP2EkfBhM4SfSL4cduO1ZZNh4D0HhtJ28zANCJwRVJFTwI9J7/7scfWBJ4UcKRtuijncUcwek6RWUWkYw2ULvRU8eO5Wp5Dk6O/SguJNCOKbZNOQWK1ZAwy5oP0uMuik9BfhV2rIVSorZHsSYbi/Jz7BXxQBhnzx2Xz+O3b9/H9774KALgqZ7jqmJfo6Fu4+RGvtH8kJG5//bcg/BEUC7wS7EGvGAGVZsziwkeOoFQ7sHGIC4wwz13fP5xhx1umTw7h/r/5Ckh6CFjg1dAscIcHu/iJv/TpXBub8ngd2tVodgSGroNAGQmwiIA7BRXRbHz1B/fwq9/4Ye66e+NJyNFOG/IAgP4QePIjhkkmE4Xz820kyjOqYLggjB+2OcsgQBqG8EcOpCPAzAjlU4BisFaILs7AKsI0UPiff/sHAICnjob4/DNHudZcIeL6vcYzg4gw8hxMQ5Wq83fHwDz+Lm+OVC5TpvFkMAyjGRgXBFv5UqRMFaisJuzIA2SNgB4HKhUKhwW5chxBhamZbTpDV+YqSjEzTuec2mpPZk5l2nVfangyr8L3HdH4g0nm114wq3h84DCuDvPzOXSR1jeeRyoVKDbcnoCM3d2TORQEeLIu/02WW2fgShQ5mpR5Ctl0DnpubuEOlEYYm0HmkcBxTVpwR4ToFaTr9vqi8RfjSoGRT5fO43/0jZfwD/77/x0A8IXeXXxxcBcA8PGfGOCZvzIs7Z9WjK9++b9DMGPMtMT/dvoxRGsYVP7q4If4dO84d/0L/+4Odq8s0w8XGl/9R38PKgJOlId/dPY8GISPffw5fOZHP5FrY1Mer0NnPurQoUOHDinWqqeQ+cUW7w9evnOOb711nP5OIPePQL2+edIrUuEY/T1V6R4qe8DFRazyWqYjzyN4nnnwmicxitWlWwuFuU40C10djyAZg/3qAxlDx4z//BiYTw2dKNDQURbgBgJYSHhXb4C1BrRC8MDsVs5mAf7J198CAXj8oIe/8ckrAIBdX2BQsHH0pEh9z68Os8C3HU+kqnXiv84wvtiJal0HV1D6PpsEFfVdGSfcancQaNNxSkxUPUekfd71dc57QzEwDgQAQqQJp/OCUoUe0rzybkH5yCawedyTWTSYJ7Lrkc5ML7aZpi4gntBuDh1r3tp6j9XRkZS1bQLp8rw/DQlh7G03DgTm0eqBNtBzdBo013Oam5JsbIPH7955gN/9HZOm5uLNl/GFnvnePvHkAs88aRxZdo6qNWEi4OlP9BCFjEVEePFP7kEpwsIb4OUP/3jls1cfvoUb914DAPzIMxGeuZ53nvEHBXW1JeFDn+5BK+DKXODFb9wDGOidE375l34NAHDj8cfw6Rd/DMB2eLwKrYWCIMIw9qooW3J+cPsMv/S1N3PX5cEVOAd5s0kCImCwp1B3xDAe56/5PmE0Mg8+t+Pipm9+h6cLnEYMZsZspqEq1nzpAMODaqGwsyMwGsX24zkh8S5Tgc4y3pAAJABmeNceM+cVYYDg4T2ATcbX/yuen08/dYAvfvQGABPZfNQvsEc7An6s1A1TWUowRIzpZR5lC1NdgJMNV7ZTRdcJampCh4jQd7Px7PfyKaBDlQgFU9HvbJFnX2YFR2RmqnW6a/M4AOymQfQCiXI9CxVCbeiEmtPfdWg7h23eT1s6UmTjFKRQVKQl0jJdoCdhvj0CY+SGkCITCutgGzx+5/Z9/OMv/1MAwMe9Y/z0yAiFZ57s4bkf6zfqBwmzQANAMNf4Sy/dg46A89EVvPujn6oMpn3mpWN8dmxovvDsAE9+rBlN6RCe/ay5d3qm8MWXTBvvnl2k4/nsj7+ID33aCKVt8HgVOvNRhw4dOnRI0VpTCDVw6zwJiMmuv3b3Ar/57XcBIPU2AozJSO7uAwDEoOiAhzE8UCDJJvlUkgeIzK68qSrqutl9P5wrHAemc/fOFabxb12wgW5CJwg0ZjOzR5nPGVEU17vtEUbG8oPRUMJxzLzcfZsQxTVhdeIt4nrwnnzWjDgMEN5+J0cn0jo+HFzeIs8jhTA+vOy7stD8snpwC5hDuORwVxLSnbhm1Gaq7Lti48CZJpiFKvVKGriy0vupUXsRQU/NOKMeI9L1aSRWffI1Mybx4aWMzTer/OFKwojycz4LdXpY3ncyb65ZpCpNl2V0tg2lswN/R2Ta2bpgmMI1BEAIwJP1GpNmvhQeb4KLOzPMT40TwdmZwCLIewA+92IfricgHcLHvjgAa2DGEfDD/xsAIQiA09N8Xx4P38XVq3FBpn5+sWFmvPGNOWaT5b8JAo6OjNl8kXdYymEbPF6F1kJBa+DBLH/93vkcv/3dO7nrYrgD50p5emkA8IcKq04YRMBgsN5HchJqnMS/z6d66exhFU3oGNNTdo4Rxu63wmUkKUv2Dgm+L8AMPLhlPFJBlLqsQoh0HvRsUigUlGYsovy5R6g4fam+Y2zqy2ModouLrAVAWPfMI4VpDZP4a5oB2iJQOrXNm/wsmwZHidTc4coIjqj/GKIVbyHm7CNymApzyxg31HxbgcpMlF5s+mFmTMO8OWypvRI62wYj4yXNmwsFgFKzkiTG3A9r05VpzZfC400wPw0xvmNW3ru3JKbTlUwKAnj2c33AM2adxz5qbIfhNIL8+tcBALMZ4d1380vnwYHC3pEZmFfiVHbvrRAXD5f7KwTD+1AEIYAgAMyyXBXkujmPV6EzH3Xo0KFDhxSXEocv9w7TXXHibQQA155gDHeMxDo/V4hiT4bkYFkS8KUneyZnDCH1JrJxGmq8HqeumM81ptNqKfmZqx72vHLZFzLjldifuy8IHxuZKXkw1fjmPRNFo0qk7HAo0j7+yL6LXdcEsKmPRJiFxnvmnVcIWjXb/c4igXDipDtKpYHXjgEFAc1mDD2HcrVkyyAJaS1ZzYyzONVB2Y71fCEwi+KDXK3T4LCR52xs1inD0JNWxbFqGlIA1wZxTIkmnNT41V8EMh1PFZLDawC4PwHCEyDUpu2+A+z5zcc+cCWSRKFzK2tm0ZwrbWo6MwBPMCiOl3GlWPtAvw6CsvrCTV7prq8xdM03djKXqSdSERQD96b1SwprvhQeH5Q8NLk3x51vmW+5f+jh+qdMcBi7U0xPzZzfvy8RRQTWwLd/ZwIhjTns+jVj1gkCxq1bpv3+nsRnfzpvCn/4zgK33jR0AswxfWB+Hx8LzBdmsod+iNHN5TWLKDu/1hzhfHYbzIyJPgTwXOX41+HxOlyKUCDPh9w7yF3vDxk78eWFYogVs44g4OaoOu+JYSbzoSnFWCyq1aKjnsD1Yfkw58oIBQBwCDhyjSlpPq9v23EIvTg3yb4ncegaU8HuHuAoEyHdxvqlmBCpRHU2KvZ5YJJ0JWiTSI6I0rm0zTRlCBVhHiVBbQpSmPs5DVfbPtqcWwgC+m4cudsg43SkjetqHULrnrkCLgJCMt62KXjs8Uyt85IiMIx9GCBoqREojhcIjcTrZtsQFk80gSc57crZovpegDCP6ttmnfDTdnm8zNU5nCnMTuJC99d8DA6NSWgwmoFC0webDU9uG+YSgrHjZGad6dSYdfxdiStP5u1DFw+j1Bw1O1Pw4kIQJ7cTMxXjyScV/MqSMBpBZNwrI13vvbQOj9dhI6HQLD3vZmko1qd7+e0vLfjMS89lVaOA5TnYfHHNyKz6jG934V6qP2v9fv9kkN7uvFZSSnKY2BQvYSLSRHHvy/kGHtWcXxqPc9X3XXy96PYma0TTZWq1znN5Tx4NNhIKbz+c4O9/5fsAgGmJN0t/V6E3MhLT7WV+3jb+7af7GLjGZFR31hYEjAcPDC3bm2g4FOjHoe8fHTo4itNh7/jNd6JnC41ffdV4ToUlJqMXjlx85MDsEqSkdIfxjdsLnM5NhyaKsyCs6ya3gdbA2Z31CmnYeOss2b0yrg4iOMIsGru+U/mJmgIx5nXbKSfKcDxzQHHQ0L0pxWUJgecOsHW/6CKYDJ9mF1eUkdKTjJsjc30WEU5rTEnrYhYC348T+/YcxkHPzJvv1Jt4djwn9bI5X0SVO+BQEW6PzRjIqqZ10AdujjYbQxP4joAbE52FxVlSj/oROOayexOnVTrmNliXx5vg5K0Jzt81njLhPPsGbtyIcou41oR335VgpqW/LS5C3PrjE6zi/B6QCMrjY4Gzs7isQE2CWmakdMa9Q/zBz/wcQIT5wwvgd7/TeoybYqMvaRFqvPWgOvmRcADHj1X+gvWZAOz5ovHirZkRFcgfITK31JEvcOi3X7kUAyfz6jOKvkM47OfbnoZc+KzjmbGr5gW2KmGneQk1zMLd4OMkIjjxLitqEGgVMaU1JBJPB6PwXz5MXw3Fsqhskzco9ghreGazDjSyOiGELI9+k2hxcw7TrOCNKdGaH8eouWPNRhBWyUZBxd+AkYFGUDconLY21uXxJlALDbXI2/SLEiwoxQiCfGJGrRjBpCDQbyGQ2NqUospAWRvMSOnMXQ8Xhx8CSCAM7gP4gAmFHFL3y2KOiYV/ektaT7bgm+aSQixl3xdR1kzREsG87BmdtKOZ40Ik3PDjNa6jTfu12sdtm76ycVh0cHkmB+Yimy8vzfm6Kv7q3KTF3lGzJhCv3HE5g2csm3eSebjU+UaxjZ22YFYpm+8mWKX/KExJbXicSUDFHiyaGBwX2mrz/SX1UVYfsflg+Xr9GqK1cck1oHhNALQwDia6Mp3DozHdbU0oiN0D+E9/GADQ2wOGR6YimX2AM5i72FWG5Oee8DCIpXOvoPDMNGL889emucl3PMK1a/luPzd08NTAXC+rY/NHZwFmyrzkBw8jsDbTHKTF0LN7PY+wv59/QW9MI/zgB3l9cGHp0kdHEjKuevXggVFLhQQOnjBzEp6HmH+/uI9tcHeSqdO3xoZNHAG8cOVyWIYBvPQw37YjgOtDMyeuzIq7tGqbTbGVSJvD1v2ei+S496Cgfm2oGBdx9b6+w3h8x9AfBxJni8uxb80iwrsXWV+Sc82bI+BqeeLNjXA8A84KApoO+wr9OI33ru+kpp82CDWnQYw9R6QpL/quLIxfOLdqTl8fRkhE8Z2x2+pwuA3W5fG3H/8EfuWLPwsA+Pjrv49PvPJ7AICH4wcYzwvy5JSgaLffc3uIwhu566eTMxyP82YlG++emv8TAY8fPAEpHARuD7/2U38XkfSgqThWp+/oR8LjwBaFAgkCuWaVF04EWZDrngCIWBXrSSpM/paAGZiF+TKDvkSa7M6GKwh+jY9doIGAjaSehfkdwFJfyZwZrCLSwCyq/gKEoFgocLoTIDK5lQBAyW18QbT8IfLS/y4NRWmVkt0sUTvPkVVoznZhyTk9UXFJSzvHfqJ1AkjPQS4HtFy6s2AHu23oQs3MuHJug2zRwWadazBR5pV1mWPfhMcj6WLW3wUALIQPFZsYw5ARtAh2K4LSKm1viabSUDWmWfuoJorMREZCYNbbQeSUuyXZ562Xy+MthQIvhae335nIpYEVtx8oo4at5qRP7peC4BY8WyYPAuvQ1xYxQhhVjghxexQXHUn6Yqt5WCoXuNqn1Wtac6x6rniTsKFTXjyGIcTyuAXxEh0dt1GFSCfzwUvzIixzTKFfFGWLbRM6NpIyyoKKBYcNAteaWxjF5kNjNsjGkd2f/X21KEt2L6VtL/2N7PuWi7oY02J1ZzVnY7bHZvdVUHa2sLwAxz1qQGeJJrIFJtJ15isu/D6WveWKz0mWx1DMN8kYip5NXU9tupfE42WbPCVdzD1T6jLojRAmvYlU+pDHAahil0iUWT1cR+S+UwCQQkAUHJwKkb2fOXtxFmWTeTWSDkK3WeK8tC/YjMfr0EooTEOF03kIYD0vmqevAE8Z4V3IpJEG/ukrk1QgJMNyHODoyHT1mi/x6d08/bIx/+7bM9ybmACUgyMHjmPseFevOmA2GssX930IAo5nGr/xuvE+CgLGvXtJHhOBvb28ura7K9Hr5Sk/fBhBqWUmZSbMzocAA1GJ9jp0Nfb9CPeJoWH8pB/fCZdsvfcmDhYVB6uRBr5zL9tZ3BzFZh1B2OuZefOkSIuUTEOVZpw86Cnsx94196dOI5/zhKZtVqljwIOewsirlhyn8/zJvKTMg8oRhMPYrLRQOi20suPpwrZ3fSdOPc04s8wgAHBnoZAkeN/vKVzfycyDgSLcnVSnHbg9zgrxXOkr9ONgrz3LrLPnZ5mFT2ZmWZKE1CTQhI6N45nESXyo+e5F9ZxLAh7bqXaBWSiNxSw/bzuehB+nl9ixMseezE3aDkLGYzYcQemYQ81Lda+15kvh8cScuIqXnvvLePnZOMMoOF1Y9PdfBZ9eAGD8zeN/iYPorJSm6zKefDJKF14qCJQ5ORlhdLybu37jhsJwYOb2f53+ddzRBwAR/vRHfyQ94VY1xZZsbMrjdWhtPtpEcRFkolKr2o4051zdiCgO6qFYdW0u9pSO1Xz7kCrZaVLGWIIo17cmZ1KrGkRyMFX4bM0OyOwAlv9ddK0Otgkm9RSx6VgXln+vexaxfOxYN2/rnrXbu5+lfts9KRkDwfAe87K5Kfc81p9vANBFO2arr/buvM45ohrN55xQzAdNsKxJxrtQznSaqvkW8feleGUTeFk8XjIHLGR66GxDCQ8QLsAMouJdfgJBDClE5RwSUWEbUmhIafKisXCgYcoQR9IDOeWlQcvpbMbjddjemQIBMj74KkgemcM40LnFP9IMKQl6RTVyJTCM7U5l5wbzSGNRZM6jtGJmITQD5wsNIsIkKN69OgQM4h0fuwQZp7awo+qVKj6jSGgLyj5cKQWOjozKuLtbGd6Yb08wnIarqrRU3ETgrqLMtdIhhhM/bxSJ9cRFAkGWmg9TG6EKiW86W6YZKWKbfrwoJak3bL/+5FxiFZo5HX8boZTZcps9ZC8JNs0Mxe3YdJhRWYq0GTg11QpihJoKPIYy2Bs2pbOzAqUzvpG0bA5LxHTxuQKn76qJ+66NdXm8LQ77c3ihUfF2ImCgBMBx/ELcrIkxMJtRd1BTn8Kya0vJmbmpJ+DEFqKrtADrCUAEKU8A4UKDcF/vV5oPCbg0Hl/F1oSC19c4eKImSsPCv3pnhvsreYuIgGvXnJw03nUEfny/WqJ+536A7z7I0z86lLi6U5HmImL8s1emlZ/8DV9m9Pez698eh7gT+zyfnioEQb6VoyMHQphDpeP75mUd7ffxH/7nL4II2AkB1NS5XmqvrwC0PyiLNMemv2Y4tOjcHjutwuSLMPI09n3T3vFc4mRcrS4/vhPCIfMB3B67YMSCAmG8gBL2Y3OYa/2eWfmGbExCVS+JCuAKxmMF5pEmuAiavyubziwi3J9uFuhIAG4MQ4h4p26b94ow8nTMW8B5IHG+MCvaQimMPNMv2xy2a9XhPp6Fue9HMVrxm411ebwt/pNPfR+fdN8AABh3hl2wBt752kNTNEsBb71lzMxOX+KJz+9W7o2ib81x/65xFTs81NjdNWvD9U/uYXBk1o+fx3etJ/41AGDCPfwXZ/8ZFihf4zwpLo3HV7G9OAVaTz0tbGqlofTgbh0CVP9cE6Fa1EbT3uSfTcxh7efsUaU8SOhsM6xiue9VAykiWm4fuMwaBI96vi+l3XTa2hCp2LWWmMO2hUc555nhIVlkuPzbbLCWrLaftFzkHJGgqbJz2bU2ElxOboASnM41fGkOaOyoRTdOcSEIOHBF7hB6WJL7YhpqnMc79YmVXM9xMlv/rivQjwMXziJzuMXMCGOXVDtVhimHWXCgXJJlNYqAxSIp4JPRP+qLNDnXIq4jHKnMtdIl4CA+5xiC4ccuqp6DNEOsDdmSGauSsK0DT3JplCsQx3qoxDq8LhheUmjJaokIGHlID2bdmuRnksr/VoaiDLBt2tAlgZbrQhDgy+qD+FAT9IYmJkEMN16RXGtl8qWZc8DEEDmpma64HVdSq/HrgrYeFY/fEA+xJ4xqPqJ8AAgDmM8J0YIQaYG77lVoJuw6jDrtZSwHuOPuAAA8dQZvZpxWZhcRKF6D/JEDsbKeCdL4qPNDBHAQsoPX1U1UfUvb4vEyPFKh8M17QeH1/X0JxzF24c/teY0LlL99HuEPb+VTN45GMs2D9Mk9Dwdx9tLfPwkwiwt8HB/nK2HtegI//Uy/MXOOxwrHp3lG+dKTPez5EqECvvx1INGikxohIwI+F8TnMMToxzl8fCkwcN1Ylc0613dlYYGRImhmHM/WU9uLQARcGVR/DKECbtWYg5rgqK/S1BUJJAEfOUwWEUKd55vvZN4yTXHsLPOQIGMqacwHQYR5i7rYdeg5jBuj6rwo96cS04KayW3gS8a1YZ7OlQGsGCKJqoytRO2DFZXm94zH/2bv6/gr/p+V38DA7dsSwUxgTh7+xdUvQZGDG8MJfgb/pnLb80bvaXzl0KS6/vHzP8Hzx6+b/uophkMz1sd+9AD+zrJQ6FOAn9/5JwCAu2of/+X5f1p5vrANHq9CV2SnQ4cOHTqk2F6aC0Gpz/5QCuzG6tK5YkwqdlFEwHVfwnUACSqMX1hEjHfH+R3NaaAL4wSOegK7cYK9BxOFExitYDxVCArcRZM2/JJiKucLjQez/G55XHCwDADvnCs8dDWUBkLlQGmCL4AvHJpd6DU/U8+XxqmAebR5lKhf5fdbgVDrtWgTIS3EUgZvyTzBYDc/n3ZAjicJxbHM7z84gtaa86IgzabwJYNqzBm2Tbvo/biWViYIcGN3mTYu3+8ViuZ7KdXHfAF97yEA4Lo4ww15AgAYTO9g7OTNRpMJxXmJCG84j2PREwjJaRVQaOOBewiXzZp1zAp+bN7uPTzHzrS8EHOkL/DJ2TfAAE4vZvjTtahvhu15H3mEgwPT3FM9iRdGRqX8wTjEW/Nq5v3YroNBxUd1EWh89Z38RA4GIqVp48M7Lm7GWVJ/47Up7k3L6QthzFdEhJ2S/DG3xxG+VmCmKsOf3DH3MgPzcAhmgi+Bv/thwBXmvyt989EGCjhfJOMEHk42EwqCCDv+eq/1YhG1KvCdwBH1JiYbI08XCkUbQ/fyqr1tGz0nq7bWBkozArWeGWTXb/6eZIP340mBkfdIrclro4zH7XxNfD4Gv/wmAOB57zv4KctkdH/lOWbgnXccBAFBQeAPrvxbmA3aRRmv4tX+M3i1/0xMAMDc/Lj62m/iMVkVRnaOfw//GADw2sUQf4oXNurHOujMRx06dOjQIcVGWwPhe+g/9RQAwLvaS68/nCl8O45BuB9ojGPzUa8n4KymMGXg5eMQks2h1WCQjxqMFOMTV/NbS88l+LHp585YpSaeN0LgQezlNAmzHVXStiDgyZ6EiCOkR4O4Nqy1Mz1faLx1ZnZxDwrC/8vADESBa/IbcZzzBkYjmYVAJICAgLtxbILSWQlMKQSuj4CHBNgUz+bAaQGto4HROrYFT+Y9v1Yx28KBqiuo1nsic4llzKOiOOFiOILgrWk+S6B09n5s+A5w0MtfXxdEQL+mqlSkubaMahPU0bHLiIYNSreutr2pu+Rl8vjb6gr+1eLjueufcN/CYeyJ9N3FAU7mLpgEQs4T1BHj7O1pZQDg7LTYkcbGN8Jn8Wo0AxHjRfcV+BQhZIGvhx+FWqH7IMz+rTgrjLUNHq/CZkKh18PwuWcBAP6OAmAW0XtThVfP84uHlJQTCgzGd+6HUCoLXltNHbHrEL5w3atkvPNzhYsLQzP5/yqGQyOUJIDPHPqVXk7nC40/vVv/kosQzl2wNqps8u4EAdMwY/CLtGlC8hqO+sBjO4yXVrp1MgcmBVaGPX+7QsF3BPwK5bHtAl0GT4rC1MyFNBEH5TRE3xEbfzARA7cu8tf3e9sVCoIoTVddhlmoEDYoilRNBxi4svHCHSjdSvj3HLHx6c9l8vgb6gbeUPlU1zflSSoUvjG/gndnA4AI+yxzX4EKNE5eH1fGUMwWdesF4Wvh8wAAAY1POW/CpwgRHPzLxScRrHjWheEUwDsAzOYgEQrb4PEqdOajDh06dOiQYmsnS4s54d6tLBMk4mCnp3cd3ByZXeF91pgWPLuzI6E1pwnvAJNv6LmBKbDhlezo74wjvHFmzET3Sw6TP37FTYPPen2RJukqanKuGK/HtRdPaw7Hy0AEHF6NIEiDGXh43wFXpIjoOcC1Qfa7CCNPpwFuNu5NJEDGR+fxnepkg9vC0JMbRzm3KQhDAEY1NCPN6QF5oBhckC2z54glE0kVJAGHvYJ6IER468zw8q5n6idfNlxJGFbECTRBW8uOMSNWPzQNVaoxTgKVoyGIWpmVtsHjTz91Ez//c/8RAOA7L72Gf/5bfwAAcDwfrpfPMfbbzh7+WJjd+WkYm6eZMb1/H0Qm1fl8emEyG8sJvjc6rjQfvbV4Fw/m5mN2/R4ct9ybggD8insTHkVQLHAWPYBmArPGfDoGGHj8sWv42/F4rly/kj67DR6vwtaEQhQInAemQ46v4ceVqK4OJJ6P837MzgNMV5LOERH6/YKIUgKeiO3+ZThdaLx8XO298dSug+vDZsMMmPHDWBjMS5LjNcHOnobjmJoKxw+cSnOLJ5ardhU5//QcRs9ZboUZuD0WcaIzU/3r8moxGRBR4wCjR0lzEak0GaJihiooguQKgRqzegpBwE6Bd88kJDyYmr4IPBqh4LTo97bgSoEq654xI6o0oWWRx5qMhUJTbIPHr109xN/6d/4qAEAgwi9/2QSE+f0ResOd3P3fxhBAvmRecGa8g7TWuDi+B4BBYoYf7lxUCtj784e4mJv2esNd+P3qcnxfx4H1L0NTqQjjE+Mf1b95kI5nEak4l9Z2eLwKnfmoQ4cOHTqkaK0pSAKuDuK0DC7h7zwtARBenQK/eS++yZKmr5yEuDMx9+selRdQXkGggW+e57WAMGSMx0Zi2sFj/T6h18vLuORgm5nxvXGERYVXRdTALmLTmUx0YWbUIkwU8L+8babmigf8BzeNWh9pjbNYO/Fk/QHSyVwijAuQRLFZigG8cRqnPybg6f3ySnSrWEQqTdHQd+WlHWBtg45mxjiIcqakJo4ys0hhHi2bhExepepPIFTAydzcY8eZnS6AxbH5fWVgDqGbgJlxESgwMwQRRl7zA+A22BadWagQFGgCdTF3mpcL6yR4L3jc293B6ObjzW5eAmNw8xrAjEMxxmdHtyrNR/PgQ7g1N15OwnEhCmo41CFazFNNQWnGWZwj5zJ5fBWthYLxZMjyq38qLjQUMgpzOJ0tNM7i4Kx9KdPkdHXQAB6EeWaczzVOLvL2XscpFgq2J9NxqDHb0L3PpjOfNzcxRQx8J/ZoedxaQDSQuv8J4qUaDUUIIsJc5ceZeDO1VR8VZ/T9S8h4WUTH07yWrYsZa0cAlyVQqxuyZsIsIqwyd6CypI47LeqkMIzLJyOJ4L48c9w26KzrEstA4XPvBY9Lz4O3M2r3UIzk1Q7Jw9Vhv9J8tLfYxSC4Un5DA5A1OFMeuPncr8vjq/hAhDBump636fOXkQY4bjn+/6OL0E2G8gHIWJCi+XvanNa258Xu0wdlzj/I8w18MHm8LTaZ+3XnpZVQ6LsSu75TqBYu3dcXOLximh6PFebz/Mh+ZORixyEwA9+8CLCo2HRHEXB6amg2maSPDBwcxR5Hb0wjvBTvLuc1Ulcp4OQkT6fXI4xiDyr7cP+z130cxpL9e+MQ5/HhT1JtjQh4/OkAYEApwq23XVQJhkBpBEqnlaoIpq7yQqnGO+RIAy89ML99B3h2v5o5eo6AF3sD1XmcbII6Oom5Q1WZ9zRwZ+JsvEhdGWTZWM8X0ZJPflMet3F3Ajycmd9P7AJVxfQIwN46OTHWQB2dSHNa27oMxzOBSbhZf32H0wI+7wWPf1ARaMLt8XpzX8XjdWhFURA1SpYlhKmRAJiAtaLiKENJ2HVMSmtRUUAFMAtG1PwbRS9uGwAWmnHRkNmYUUhHCErHY2PoChz1jAToLSLMVm4hAvyeoR2FXFUnxtDHcl0GkIleNLUKmq+EsVdtoycE0aUKgzZ0lGaoihU/0tup22CTUMxLc96Ux22E2vwHFHuP2SAiOI9gvpvQYaByvoFkbJudM0mraMl7weMfVDDTmnNfzeN1uLQtS6KattnVtTXf2MuDoOWloqoubBmd1evJ6yhbhrb1abcaNgEr5dSr247brzoge1SwC78XXafG60LdTfVvpr220cwEyMgOBd8vc142383RZBw18/KIeXzJnAeGwJou5nE7VUWmluiwti+0hsjNddnEXQaPG2xNKEjKgq/Chca9e83NPQmYgYcPo6VqaKvwPML+vtmdP+ZLPBfHIJigNDNRr04jvBzHzBeFGzShc2Mo8RNPmhNhE+yWfwnbSDExjyitnzt0Nfb8apXoSj9K2eTuxEWVVjiPgO/EHmG+w6nX2HuBniMxiJ3fp6FKfdtHnpOalXb9vGmIwTidm347wtRursJFIHG+qDrJJNybOukndaOm/rInOaU5iwSOZ9WfzNtnwA/jJJjXhlGuaNCjgjFTualymtRLlkTYjQsIuYJw0MsXyJlZnmIHPYX9gkC+BMzAnYlb6R3zXvC4nRrlC97L+JnhtyppFkFrwjvvGJ70XV0U0rCE52evYe/4XQDA0ZHCzqi9ILpPEr+AHTBoifdsbJvHV7FVTcGu61u14FZBqWpBQpSYpADPIQwLVmbNXHlG0YSOFChse/ug1L2viYaXePNlO6PqHUNi1pBq8zoNm8Derdq76QQUR62uDmfJ0tDAo9muyVCGrIxl/b02zSZtK87cNZUGNrS8bAXMtvaSgchUO1yFfanOc7gZTz16Hrf5zacQe2LWpKNLUAwMtQNmgtug466OMNSGzg4U9kT7RXBGEoAJtCNwoafVtnl8FWsJhYScvXmWBMQVMKEoe1Eklu9LbFuhYiyIUxuj1lwrDFZpai52e7NfRZEtbZWOXYwkeQllyfJUST1excW0yNJg1gVZprF113VGvb27viPGpLY6nKVFh5r5j7eZEUHFArxoLgjZR2P6tN7c20+tO+eaN5/zsvnUnM1Jkrqlqo2ib7b8/uLyRmXzIGJzD3NyT/s53waPazCCWK2oclj4oGNbPF6GteIUDvtGFZyEwO2x6dDHR8AvmPKk+J2HLv7fO+aeo+sR9g6MSnd2pnB2ZgbzlXtRyqD2OXDRx+84wNGR6arN1LcWCncWedXW/g5PThTCsFwwCAFcvWra3nEEXtzzcnRsvDlTeHOaV3/vH0e5QDYi0/am54pDV2IYm17OF9FafuMLlanw64IIeGwU5naXkQZuxzWafYdxvaDurw17PLU0gUITR6iLg6N2PI2RZzjgZC4xDtr759s8XkanCe5PN1fER57GYT/P48cziWmcWvnaMMqliLBByMbTBH1HFKaoOFtEOV94AnAzNk8ovX6t7m3w+MlFgD/7wR0AwPffOVurHx8EbIPHq9Caa8t2vZKQLharXg92AFmyGCug8ZYgSZS3SpuTdirAXG0mMu2btgUZTaFqZ6+ZC2kW0WlCuwm2E/G6+bEnM5e2kewQmzgLtBlP2b1lo1naFTemUk5zk1lbt5Rjs7abt9+Wf9q9n2yeK/I+bpVmGZgzDaGNt80HDdvg8Sq0EgqaGUrz0qFuGVJGUYQgMP9qe86Q+Pu3TfynVLEpKo0fgHFbJRiBlZQC9QTFEc/VDNUv0Oe3YTo20c3V82pcOzl25TMHsQDiw7jLYBGGpIwRUxWfkR4AKqbY7GZsoEXmjm27vTYxU0kBOCL/Lu25ijRZ9tdqOg5R2p5mVD63CQhs1eHI+q90ZqYTyMYmC+b8MupbSyLoCnu2FATX4skEzIZHgPeGx2fs4aFuH9GsNOFcyricrkKStK4MgfBwLk2WRAcK0RoHq6fWIZRmxiRxspCEfkEo+DZ4fBWthMIsVDhbhIXq/FKjwgSVAMDFscTJAzMYf6jheM1UcdusA7TbSZyfFwfMHR46kNIw908e+lgNPbhQjN8/qS6U8dzAwU8c5PMafOVUYV4TCFSHaSgwCUT8Aos/vpEnkaQsIBh1ngG8e1HtAbIJrg0juCuMp9mYjBjmfT8WmxBcSdjzNzNTNYGkYs8ZG55U2PXynhd3Jk7sFw/cnTiYLMpFuk1n6DI8adq7CESaE2nb6JV4ip0vZJqJ+Epf4bBv+rLfc2or2W0DNu8VQTNACHMbsnlEuDc1c/he8PjXg4/g1ckz7R8EwIfm/1fEOT6OX6+89+X+s/gt8WkA8XJcUL2vDuFsBsbXAQBnkxC/921jDvvQ1SG+9LFrufu3weOruLQ4hWwN3/SQdds7TEp3vPm2m3HcZSQwi1uutTet+vo39+3fDOVDzjqQOQNc/gLVlEbRobj115Z0+JFFzlbPd/6e98Oc0woflN316Hmc1jflxY81ed6c+VL2ew1UmWdLTakb8vgqWkc02zsSScAw3qxFGmlO+z0HeCbONX9vARzHm2/WAiqK00XIfGGOVQQBgyrU1b4jsOOZRsYBYxrbM2ytzXGyM4k91+SJF5Z3hWbGeWR2ItMSVxGPgEF8YGLXcT5b6NSfmwm5qGciYDE3daFJEz48AMCAXW6awEu+7CzqzVfp2ASB4nMLX3IuYI/ZhMpvKpgDRekOzZccn8GY3wyGLFBfgczcuCkcQWstetLiV6Uzo4YnGGS5IzTdRBEhbc+TgC/z/BK2VNWLoNnsrgFAEqe1DRzBKU1RMOfMJiJ803MsQQS5pubhCkqDyZJoaUHLc/V+5PEPKrbN48BauY9c69/AC3FSwJMZ8Pqp+f3ZXfMfAPx/d4HfPYk7NvfTxaW3M4F0ym1uWgPHx9XmmOcPBV68ZhLN/PHtOd55mL9/b0/Ci/MgfXbfTc8PEgQa+KOzoJJFr/kSHx/lTRV/dm+B1+OcTIeHEld2lqdTa+DNVzxoRdhzgP/q+XzA26q3jtaMW4Ta+Esiwk5so2NmaA5zYwhU5hW0PggP04AtE0zjxHb266NqU2CgdG1unXrq7TxnbPRdmdaCPl+ECJQRaEeD5T7xTOGkQXt22mdfKvScAi+0iYNptNkCtVACdyeGzo6nUu+jHV8XFv+xcb6INjYjDlyJwRppnwWZPEaASQmfBB36DuPG6P3M4x9cbJvHga7ITocOHTp0sNBKU4g0Y2EVcSCidOfkSuAgrhMwj7KEVY/3gM/ElfBemQDJOa6OHHBs55FulDMlSQKe2DE+/gvFuD3O7zgvAo03Ts0hy5kVwuy6lHoaXfEl+q4xFyWJzjQzHgR6qZbBKpTiNO7gOALeKNgUBzAZVIHM7ZYZmE4EdBwxHS6MxmBXz5OUZdJ0JdI51MwI1tzmeTLJmsJptklT+yI2N4DgxnQU81qeEeYcxqaZV9vLDjxDBQQFOfJraVJGUxCjX6BdCsrGVgZTu3Z1bnnt+gxSZLyvtE5jbXxHA7HJ0xVJrWNDp9yhtxy2aVGWmHWKTGvMwCwi8BqmrEgT4k0+fKkLvVvcmjrOBCospnOZPB5ZvLF7dITnP/ejAIB5GGJ8/x42AdEMf3jeT99tEV4PQ4yjzehIorTfh9evt3p2WzzeSijYdUJNJ0z+FCLCyANGsa387iTL//LinvkPAH7xXeBb8XvLTEmM/u4EtJIjpucQfvKpHgQRHkwVfn08zfXn1ljhVoGwGA4F+nF49fO7Lg5WbDYawHfGIQrKnKYIQ8bpqWn7FAqvIX/Cv7cncXCQn8IHdxyEgTAf5th8oHPrHMGTwDP7yWInkChs80ilEZltYKvZmhnHM9NXRwBXYzXSFYS9pDhQtB2zzo7nNLb1T0OB08V6fg3341dvvJzyZ1GeFLVCYVAQLMfMOJm3ywtj00wWvXEQQcXvbdcy7+z5BFea9PAn883NOr4jCsdRBAbwcOZsfL5x1M8CpWzs90SlS7AUJsfSKi6Tx8nKd/TURz6Kpz7yUQDAv/6NX8fv/9qvtqZp4z6A/2mppnIRJgC+txGd/atX8bP/9S9AtPXDx/Z4vDMfdejQoUOHFBu5pGrmtHiDJIIfh8YPXOB6QUbBH9sHjuIzoa+dAGeJSSZywYrT/DwMgCLCd++HIAKmBWU5AeNZ5PvldZkB4PZC4Sx+/vGezOU00poxnebbj0rUCNcleLHHk+1tdHEmEIXm3xQHcwkAf/kxwCVg6AA3R8Z0ZCsuSnOaMRQwxWjSnTCb2JCeK2LVsB4EpCkKNCNtWzFjGu+kysr2uSLzZFgoXbiz7TmisZ+HIyjtS+QBmvPayTQSaR3eOmhG6qdvo+8Ajsg0ojqtwUbPkUv3h0pjHin4UjTWglwhQPGXZM/bQunUPFnkEUQwu3/C8ruyIYnSLLJl+bhWx8Nxfq5dL/8OFQOTUKCpt84sIijOz6cntYn+R8KzzdqTRJfG431f4pkbJkhtPItw/2wOAHjsmWfx+Z/6a43afq8xGI7SuXQcib24jOhg4Kbf76Y8XocNhQLSjnoyEwo7XnHd2p9UwKdiYfH6FJgax14g9FM3tiAWFBFp/MmdSaXbqucJ7O5Wq9M/nGcL0TVfwF35GJiBi4vm6qxdhc3G2YmD+TT2THHMWYFLwL//NDByjCB4cjcfeWov1r4UGHpOHI1qbNDTUMGRScRwPYgIw7hQd2QJHPtdlcF3BHqOGVs0DwtrUgxc2Tg62RECTuz5RaTShdtGOKEWQoFwWhAwFrgKnjRt9516U1ICIsLAlemiCxiPqWlohEJT+I6AHyvdKvYAAZC6K5fTNzl/iAih0oVCwZXZ+6xDMh7AmA0CVeStQ7FQaIZpKDHNWR8YAydMXWX9FhuF5CziMnh82HPxsSf3AQA/fDBJhcKHXvgYPvTCxxr28P0D13Fw4+oRAGDgqvT73ZTH69CZjzp06NChQ4qtRTSbeq9mmz+PRJrBceBq9GKvgH1fYCfeOf70VeDBYrmNkIFfvRen3SZCMMsXuxVSw/XjAjqBxtlZ8z5+Yx6YXCoMnCxUnH643enf5EJidpGftjDIJPFfPzIBfJIY1wcKXpybZhLkow+rKsQVYRaqNCiobtcu4p1oU9jqe8+R8Ar6VkRNW9qOJEr9pm24gtK+LJROTVgjT6X8YeN0LhtHoQZK4DhOlz/yAEbeVaznNDdP2FCaMYs97hxBqSZVhp4jcylBymC/OmnNj40yb655pNI57Duy0Ctp6Mk0nXX2fhgHBUVzFoowDZvzytlCptUOPalyGrAgYzpcJ+hwGzz++EG/lbZXB2ZOzdiSgF7NdxVEujbT6+lcVjofOBavXSaP5+hu3EIME4VpJu18AZzMTecOehpJmMqOJ9KcSJ/fN6m3bcwV8C8emuhgzYTFIm+Dkm6YCoUoAqIWngyvFpwdtMV8Rghrct58fg+44ZsP5sDXaXKzgizfrRFYduqeU+0WKEoW6Cbwm+ryWH73jigRClKk5gbjEpt89PnoVmaT56epJ12kCRdBkitHwSuINHaFaGyesKGZ07F5sl4oeFJUpQcqRdt3FShGEJtNfCkgVwQoUdZXpS2hLZY9pBJcLESBmagMhEmYzDdjNwxzxXgkUWH67SbYBo/3XYmrO7216Bch0pxWr3MEYb8m79Y4iCrNh8yAc+GmiQLr6V8ej6+iMx916NChQ4cUG2kKoUKaKdJ3NPbiHUjf0XAGcdZMS5WeRQoLZXYsN0ZObB5gTAIFzQzNhP/24xIahFADb54aiXo/AP4fkywQOpKYjwt2AE12lVtIj/LijsRn8skKMxIEfOoIGLmmT3cnTqWK6EmN/Z6Zt1BrXCzyQU6zUKe7Qk8K9GLvqmQHxcwYBwoMBoEw8mSl2h7EHjZZe5sV6ZBkYhZMn+rv7zkCboGaOwkzX/6juE6vZsLxrLkpaRoJhJP8vUpr9N1iVW2xsqPTDFzEplACFY5tEemUl30pKzUrZsYkNDze5P00Qd8RqXmkLk+RsN6PjUjr1Huw72pcFeabnQQS06jZfpEBPJg5ubfjSoYUxWlQtM6n1+54/NHyeBU29D4izCIT1UoAEAsFVxqmWIXtCrnncxrYw6zjtLuML+4ZW2WggP14HG9acWvMAqrIe6KpaX5DwXDdBT67V33P1T7Qc8z8vH1OlYXHTbczD6FI65z7omJGvP6g5xRHigZKx9VsGXX2C81ZlKNsUO+1DkQEv654sgWnQM01NluzSSAC+q7pl0mo1/yDjnSxN9NCqdLEfUUuusn8OMKYjVYXIHsOm5whBCrjcc4Ze9qjjYth2fshhVQoOFZe/oViFJisy1pPk/fZiLTGIirOKltUerfj8UfP42XY0pnCepPe7pC37N4tbP9b0WtGs83Q2ma1bHs4vg7KaDyKFM2XhcJYgYbD6eZ8Paw7bd18r4dNeDzBRkLBlYwbcYZP0VIaj4MonfyR56R5dZL+OwJ4wbjo4qn95SyLCXwp1j5IrUOojVlrFQ4LgKtpvnYSq5iMSi0BABYR4c7EvIaBo7HjVZ9GT6x5s7HjZ3NYB08K7PnL6nmOTqgKdxe7vrP1KmqPAidzB2cFPHqtoJiNjUgzzgpqNLuCsBd7TTRJM73rO2Be5vFVOon3ng1PNk9t8X5CpDO+ttHx+OVhXR5fxUZCQZBJi7sOVBKthuJEX4KAYex85CnG8zt5Oj2H03xL20aggPNFnubJ3HhXVWHe4h0wKK2Q5DUwQ9jzZsMR1JiRBRFETTCL0txK5Xy/IyrMud8sRV3RPDgtokqJKFe3fBXMxfNdlIzugwCbr210PH552ITHbVxa5bW22Ka6uNrWNtXB+Ngrd2UbyDtntniW13n928G25ru6wtYlzfkGU/YoTBxNaXc8frn4i8Tj7wuhcLaodpBuO6hJqFJPhh3PgdsixDuBK7LavIs49QEA7PoKo1j9PZ458UH7djANBWahW1m/tgx1c9gWbTZQmjP6jhCF2TGbIDGxAMDp3KRoEAQ8thPHpWjC3QKTxCa4O3ExXlTX5S7CIsq8ZbaBjsfr0fH4emjL462on0/yRbnfS7iy2EthEWURsz1XYo3vZQmmjkR+AZiEolBF3gRGc868Eu48nL9/Cg0ScOYEBVHZxt0YbOzrvS1E0ExDleM1zdSqAHlTBNYH0/H4MjoeN/jzxON1aCUUTs5DnJxvV1p3KIfWwDt38nUkOlweOh5/tOh4/P0H4vfSMNqhQ4cOHd5X6NJcdOjQoUOHFJ1Q6NChQ4cOKTqh0KFDhw4dUnRCoUOHDh06pOiEQocOHTp0SNEJhQ4dOnTokKITCh06dOjQIUUnFDp06NChQ4pOKHTo0KFDhxT/P6OHX48qEuDHAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import time\n",
        "import gymnasium as gym\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython.display import clear_output, display\n",
        "\n",
        "%matplotlib inline\n",
        "\n",
        "environment = gym.make(\"FrozenLake-v1\", is_slippery=False, render_mode=\"rgb_array\")\n",
        "environment.reset()\n",
        "img = plt.imshow(environment.render())\n",
        "plt.axis('off')\n",
        "for i in range(10):\n",
        "    img.set_data(environment.render())\n",
        "    display(plt.gcf())\n",
        "    clear_output(wait=True)\n",
        "    environment.step(environment.action_space.sample())\n",
        "    time.sleep(0.5)\n",
        "    if i < 9:\n",
        "        clear_output(wait=True)\n",
        "\n",
        "environment.close()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bVAiYX5UM_mZ"
      },
      "source": [
        "Or you could generate a `string` to print the environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g3nel2pkM_ma",
        "outputId": "5b5da2dc-e668-4f03-cc5e-7f9fa46a660e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Left)\n",
            "SFFF\n",
            "FHF\u001b[41mH\u001b[0m\n",
            "FFFH\n",
            "HFFG\n",
            "\n"
          ]
        }
      ],
      "source": [
        "environment = gym.make(\"FrozenLake-v1\", is_slippery=False, render_mode=\"ansi\")\n",
        "environment.reset()\n",
        "print(environment.render())\n",
        "for i in range(10):\n",
        "    print(environment.render())\n",
        "    clear_output(wait=True)\n",
        "    environment.step(environment.action_space.sample())\n",
        "    time.sleep(0.5)\n",
        "    if i < 9:\n",
        "        clear_output(wait=True)\n",
        "\n",
        "environment.close()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X9aclIqUM_mb"
      },
      "source": [
        "You could also open the environment as a pop-up using the `render_mode=\"human\"`. However, we will now stick with the simple `ansi` mode. For more information on the different environments in the `Gymnasium`, read [their documentation](https://gymnasium.farama.org/).\n",
        "\n",
        "To recap, `FrozenLake-v1` is a discrete environment, with a space of possible actions equal to 4, and 16 possible world-states."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Idl3HuDM_mc",
        "outputId": "b8ae6b4a-3df0-413d-b1bb-8e82b04ab286"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "<class 'gymnasium.spaces.discrete.Discrete'>\n",
            "Action space (a): Discrete(4)\n",
            "Possible world-states (s): Discrete(16)\n"
          ]
        }
      ],
      "source": [
        "print(\"\\n\" + str(type(environment.action_space)))\n",
        "print(f\"Action space (a): {environment.action_space}\")\n",
        "print(f\"Possible world-states (s): {environment.observation_space}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47u2LBBcM_md"
      },
      "source": [
        "One way to solve Frozen Lake is to learn which action you should choose in every state. To know which action is the best in a given state, we would like to assign a quality value ($Q(a, s)$) to our actions. Since we have 16 states and 4 actions, there are 64 values to be learned.\n",
        "\n",
        "$Q$-Table\n",
        "\n",
        "```\n",
        "    ◀️LEFT    🔽DOWN    ▶️RIGHT   🔼UP\n",
        "S1     ❓         ❓         ❓      ❓\n",
        "S2     ❓         ❓         ❓      ❓\n",
        "S3     ❓         ❓         ❓      ❓\n",
        "S4     ❓         ❓         ❓      ❓\n",
        "S5     ❓         ❓         ❓      ❓\n",
        "S6     ❓         ❓         ❓      ❓\n",
        "S7     ❓         ❓         ❓      ❓\n",
        "S8     ❓         ❓         ❓      ❓\n",
        "S9     ❓         ❓         ❓      ❓\n",
        "S10    ❓         ❓         ❓      ❓\n",
        "S11    ❓         ❓         ❓      ❓\n",
        "S12    ❓         ❓         ❓      ❓\n",
        "S13    ❓         ❓         ❓      ❓\n",
        "S14    ❓         ❓         ❓      ❓\n",
        "S15    ❓         ❓         ❓      ❓\n",
        "S16    ❓         ❓         ❓      ❓\n",
        "\n",
        "```\n",
        "\n",
        "In this $Q$-table, each cell contains a value $Q(s, a)$, which is the _\"quality\"_ of the action $a$ in the state $s$ ($1$ is the best action possible, 0 is a really bad action).\n",
        "\n",
        "```\n",
        "s           ◀️LEFT    🔽DOWN    ▶️RIGHT   🔼UP\n",
        "S1        Q(1,◀️)   Q(1,🔽)   Q(1,▶️)   Q(1,🔼)\n",
        "S2        Q(2,◀️)   Q(2,🔽)   Q(2,▶️)   Q(2,🔼)\n",
        "S3        Q(3,◀️)   Q(3,🔽)   Q(3,▶️)   Q(3,🔼)\n",
        "S4        Q(4,◀️)   Q(4,🔽)   Q(4,▶️)   Q(4,🔼)\n",
        "S5        Q(5,◀️)   Q(5,🔽)   Q(5,▶️)   Q(5,🔼)\n",
        "S6        Q(6,◀️)   Q(6,🔽)   Q(6,▶️)   Q(6,🔼)\n",
        "S7        Q(7,◀️)   Q(7,🔽)   Q(7,▶️)   Q(7,🔼)\n",
        "S8        Q(8,◀️)   Q(8,🔽)   Q(8,▶️)   Q(8,🔼)\n",
        "S9        Q(9,◀️)   Q(9,🔽)   Q(9,▶️)   Q(9,🔼)\n",
        "S10      Q(10,◀️)  Q(10,🔽)  Q(10,▶️)  Q(10,🔼)\n",
        "S11      Q(11,◀️)  Q(11,🔽)  Q(11,▶️)  Q(11,🔼)\n",
        "S12      Q(12,◀️)  Q(12,🔽)  Q(12,▶️)  Q(12,🔼)\n",
        "S13      Q(13,◀️)  Q(13,🔽)  Q(13,▶️)  Q(13,🔼)\n",
        "S14      Q(14,◀️)  Q(14,🔽)  Q(14,▶️)  Q(14,🔼)\n",
        "S15      Q(15,◀️)  Q(15,🔽)  Q(15,▶️)  Q(15,🔼)\n",
        "S16      Q(16,◀️)  Q(16,🔽)  Q(16,▶️)  Q(16,🔼)\n",
        "\n",
        "```\n",
        "\n",
        "For now, our $Q$-table is just filled with zeros since we have yet to learn the value of each action in each state.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 568
        },
        "id": "D7Q9ZZuMM_me",
        "outputId": "7666a2b4-4046-43b5-b8f7-43fe03f99ab5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Initial Q-table\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |   ◀️LEFT |   🔽DOWN |   ▶️RIGHT |   🔼UP |\n",
              "|:---------|--------:|---------:|---------:|-------:|\n",
              "| state 1  |       0 |        0 |        0 |      0 |\n",
              "| state 2  |       0 |        0 |        0 |      0 |\n",
              "| state 3  |       0 |        0 |        0 |      0 |\n",
              "| state 4  |       0 |        0 |        0 |      0 |\n",
              "| state 5  |       0 |        0 |        0 |      0 |\n",
              "| state 6  |       0 |        0 |        0 |      0 |\n",
              "| state 7  |       0 |        0 |        0 |      0 |\n",
              "| state 8  |       0 |        0 |        0 |      0 |\n",
              "| state 9  |       0 |        0 |        0 |      0 |\n",
              "| state 10 |       0 |        0 |        0 |      0 |\n",
              "| state 11 |       0 |        0 |        0 |      0 |\n",
              "| state 12 |       0 |        0 |        0 |      0 |\n",
              "| state 13 |       0 |        0 |        0 |      0 |\n",
              "| state 14 |       0 |        0 |        0 |      0 |\n",
              "| state 15 |       0 |        0 |        0 |      0 |\n",
              "| state 16 |       0 |        0 |        0 |      0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from IPython.display import Markdown\n",
        "\n",
        "qtable = np.zeros((environment.observation_space.n,\n",
        "                  environment.action_space.n))\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "print('Initial Q-table')\n",
        "display(Markdown(df.to_markdown()))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ubcuouTnM_mf"
      },
      "source": [
        "We can randomly sample actions from this environment with the `environment.action_space.sample()` method.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4QwJCJUAM_mf",
        "outputId": "0722ee58-1c9a-4ba9-f5fc-f2836823ea83"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0\n",
            "3\n",
            "0\n",
            "2\n"
          ]
        }
      ],
      "source": [
        "for i in range(4):\n",
        "    print(environment.action_space.sample())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5zbsPyf9M_mg"
      },
      "source": [
        "_Translating..._\n",
        "\n",
        "```\n",
        "\n",
        "◀️ LEFT = 0\n",
        "🔽 DOWN = 1\n",
        "▶️ RIGHT = 2\n",
        "🔼 UP = 3\n",
        "\n",
        "```\n",
        "\n",
        "Now that you know the commands to control the agent, you can manually specify a sequence of actions that lead to the goal.\n",
        "\n",
        "> Note: _the agent is not rendered on top of the goal because as soon as it chooses the action to reach its goal (in the final state), the episode ends._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bG70ldOrM_mg",
        "outputId": "703bbebf-5231-49ad-99a1-977fba48bf46"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Down)\n",
            "SFFF\n",
            "FHFH\n",
            "FFFH\n",
            "HF\u001b[41mF\u001b[0mG\n",
            "\n",
            "You win!\n"
          ]
        }
      ],
      "source": [
        "environment.reset()\n",
        "for_the_win = [2, 2, 1, 1, 1, 2]\n",
        "for i in range(len(for_the_win)):\n",
        "    print(environment.render())\n",
        "    environment.step(for_the_win[i])\n",
        "    time.sleep(0.5)\n",
        "    if i < len(for_the_win) - 1:\n",
        "        clear_output(wait=True)\n",
        "\n",
        "print('You win!')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PuS6FeeUM_mh"
      },
      "source": [
        "In Frozen Lake, the agent is only rewarded when it reaches the final state 🏁. It is a very simple and sparse reward function (1 when the agent reaches 🏁, and 0 otherwise).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rIMTFQ7nM_mh",
        "outputId": "644fad35-4f51-4011-dee8-b7188adb2e34"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Down)\n",
            "SFFF\n",
            "FHFH\n",
            "FFFH\n",
            "HF\u001b[41mF\u001b[0mG\n",
            "\n",
            "Reward = 1.0\n",
            "You win!\n"
          ]
        }
      ],
      "source": [
        "environment.reset()\n",
        "for_the_win = [2, 2, 1, 1, 1, 2]\n",
        "for i in range(len(for_the_win)):\n",
        "    print(environment.render())\n",
        "    new_state, reward, done, info, prob = environment.step(for_the_win[i])\n",
        "    print(f'Reward = {reward}')\n",
        "    time.sleep(0.5)\n",
        "    if i < len(for_the_win) - 1:\n",
        "        clear_output(wait=True)\n",
        "print('You win!')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hMh6OL1QM_mh"
      },
      "source": [
        "The scarcity of rewards is exactly one of the problems that RL needs to solve. This phenomenon, called _reward sparsity_, makes it very challenging to train agents on problems where the only reward is at the end of a long sequence of actions.\n",
        "\n",
        "Once we get lucky enough to find the goal 🏁 by accident, we need to backpropagate the information to the other world-states to update our $Q$-table. The $Q$-learning algorithm offers a solution to this issue. It tells us to update the value of our state-action pairs (each cell in the Q-table), considering the reward for reaching the next state and the highest possible value in the next state.\n",
        "\n",
        "Thus, when the agent reaches the goal and receives a reward, the next time the agent interacts with the environment, the pair $Q(s, a)$ responsible for leading the agent to the goal will now be evaluated with a score higher than zero, and so on for all pairs $Q(s, a)$ that direct the agent to 🏁.\n",
        "\n",
        "The updated formula for our $Q$-table is:\n",
        "\n",
        "$$Q(s,a)^{New}=Q_{old}(s,a') + R(s)+ \\underset{a'}{\\max}\\: Q_{old}(s,a')$$\n",
        "\n",
        "The new value is the current one + the reward + the highest value in the next state. If the agent were in the world-state previous to the goal, the $Q(s,a)$ for the action that would get it to the final state would be:\n",
        "\n",
        "$$1 = 0 + 1 + 0$$\n",
        "\n",
        "Since our $Q$-table is empty, the old value of $Q(s,a)$ is 0, and the reward it gets from going to the final state is 1.\n",
        "\n",
        "If we repeat this process until the value of the final state reaches the initial-state, we get a $Q$-table like this:\n",
        "\n",
        "```\n",
        "s           ◀️LEFT    🔽DOWN    ▶️RIGHT   🔼UP\n",
        "S1             0          0         1        0\n",
        "S2             0          0         1        0\n",
        "S3             0          1         0        0\n",
        "S4             0          0         0        0\n",
        "S5             0          0         0        0\n",
        "S6             0          0         0        0\n",
        "S7             0          1         0        0\n",
        "S8             0          0         0        0\n",
        "S9             0          0         0        0\n",
        "S10            0          0         0        0\n",
        "S11            0          1         0        0\n",
        "S12            0          0         0        0\n",
        "S13            0          0         0        0\n",
        "S14            0          0         0        0\n",
        "S15            0          0         1        0\n",
        "S16            0          0         0        0\n",
        "\n",
        "```\n",
        "\n",
        "However, the original $Q$-learning update formula adds two (hyper)parameters:\n",
        "\n",
        "- the `learning rate` $\\alpha$ (between 0 and 1) is how much we should change the original $Q(s, a)$ value. If $\\alpha$ = 0, the value never changes, and if $\\alpha$ = 1, the value changes extremely fast. This paremeter controls _the importance of past and new knowledge_.\n",
        "\n",
        "- the discount factor $\\gamma$ (between 0 and 1), which determines how much the agent cares about future rewards compared to immediate ones. If $\\gamma$ = 0, the agent only focuses on immediate rewards, and if $\\gamma$ = 1, any potential future reward has the same value than current ones. In Frozen Lake, since the only true reward comes at the end of the game, we want a high discount factor.\n",
        "\n",
        "The real $Q$-learning algorithm looks like this:\n",
        "\n",
        "$$Q(s,a)^{New}=Q_{old}(s,a')+\\alpha\\left[R(s) + \\gamma (\\underset{a'}{\\max}\\: Q_{old}(s,a') - Q_{old}(s,a')) \\right]$$\n",
        "\n",
        "Again, it the agent is next to the goal 🏁 for the first time, we can update the state-action pair to win the game using $Q$-learning algorithm:\n",
        "\n",
        "$$Q^{new}(s_{before \\; goal}, a_{\\rightarrow}) = 0 + \\alpha \\times (1 + \\gamma \\times 0 - 0)$$\n",
        "\n",
        "With an $\\alpha$ = 0.5 and $\\gamma$ = 0.9, we get:\n",
        "\n",
        "$$Q^{new}(s_{before \\; goal}, a_{\\rightarrow}) = 0 + 0.5 \\times (1 + 0.9 \\times 0 - 0) = 0.5$$\n",
        "\n",
        "The second time the agent is in this state (since the $Q$-function would be updated) we would get:\n",
        "\n",
        "$$Q^{new}(s_{before \\; goal}, a_{\\rightarrow}) = 0.5 + 0.5 \\times (1 + 0.9 \\times 0 - 0.5) = 0.75$$\n",
        "\n",
        "Now let's do all of this in code.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "ZN12ao46M_mi",
        "outputId": "310cebb0-1c61-4ce3-a6a6-0c1929ae2b8b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table before training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |   ◀️LEFT |   🔽DOWN |   ▶️RIGHT |   🔼UP |\n",
              "|:---------|--------:|---------:|---------:|-------:|\n",
              "| state 1  |       0 |        0 |        0 |      0 |\n",
              "| state 2  |       0 |        0 |        0 |      0 |\n",
              "| state 3  |       0 |        0 |        0 |      0 |\n",
              "| state 4  |       0 |        0 |        0 |      0 |\n",
              "| state 5  |       0 |        0 |        0 |      0 |\n",
              "| state 6  |       0 |        0 |        0 |      0 |\n",
              "| state 7  |       0 |        0 |        0 |      0 |\n",
              "| state 8  |       0 |        0 |        0 |      0 |\n",
              "| state 9  |       0 |        0 |        0 |      0 |\n",
              "| state 10 |       0 |        0 |        0 |      0 |\n",
              "| state 11 |       0 |        0 |        0 |      0 |\n",
              "| state 12 |       0 |        0 |        0 |      0 |\n",
              "| state 13 |       0 |        0 |        0 |      0 |\n",
              "| state 14 |       0 |        0 |        0 |      0 |\n",
              "| state 15 |       0 |        0 |        0 |      0 |\n",
              "| state 16 |       0 |        0 |        0 |      0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table after training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |   ◀️LEFT |   🔽DOWN |   ▶️RIGHT |   🔼UP |\n",
              "|:---------|--------:|---------:|---------:|-------:|\n",
              "| state 1  |       0 |  0.59049 |    0     |      0 |\n",
              "| state 2  |       0 |  0       |    0     |      0 |\n",
              "| state 3  |       0 |  0       |    0     |      0 |\n",
              "| state 4  |       0 |  0       |    0     |      0 |\n",
              "| state 5  |       0 |  0.6561  |    0     |      0 |\n",
              "| state 6  |       0 |  0       |    0     |      0 |\n",
              "| state 7  |       0 |  0       |    0     |      0 |\n",
              "| state 8  |       0 |  0       |    0     |      0 |\n",
              "| state 9  |       0 |  0       |    0.729 |      0 |\n",
              "| state 10 |       0 |  0.81    |    0     |      0 |\n",
              "| state 11 |       0 |  0.39375 |    0     |      0 |\n",
              "| state 12 |       0 |  0       |    0     |      0 |\n",
              "| state 13 |       0 |  0       |    0     |      0 |\n",
              "| state 14 |       0 |  0       |    0.9   |      0 |\n",
              "| state 15 |       0 |  0       |    1     |      0 |\n",
              "| state 16 |       0 |  0       |    0     |      0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"5e2cf404-74c4-478a-857d-c0fc1c7a6af9\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"5e2cf404-74c4-478a-857d-c0fc1c7a6af9\")) {                    Plotly.newPlot(                        \"5e2cf404-74c4-478a-857d-c0fc1c7a6af9\",                        [{\"marker\":{\"color\":\"red\"},\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,1193,1194,1195,1196,1197,1198,1199,1200,1201,1202,1203,1204,1205,1206,1207,1208,1209,1210,1211,1212,1213,1214,1215,1216,1217,1218,1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253,1254,1255,1256,1257,1258,1259,1260,1261,1262,1263,1264,1265,1266,1267,1268,1269,1270,1271,1272,1273,1274,1275,1276,1277,1278,1279,1280,1281,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888],\"y\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"hoverlabel\":{\"bgcolor\":\"rgba(0, 0, 0, 0.8)\"},\"title\":{\"text\":\"Success rate = 88.9%\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('5e2cf404-74c4-478a-857d-c0fc1c7a6af9');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import plotly.graph_objects as go\n",
        "\n",
        "# Creating a Q-table with all values initialized to 0\n",
        "qtable = np.zeros(\n",
        "    (environment.observation_space.n, environment.action_space.n))\n",
        "\n",
        "# Creating a Pandas DataFrame to represent the Q-table\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "# Setting the number of training episodes, learning rate, and discount factor\n",
        "episodes = 1000\n",
        "alpha = 0.5\n",
        "gamma = 0.9\n",
        "\n",
        "# Creating an empty list to store outcomes or results\n",
        "outcomes = []\n",
        "\n",
        "# Printing the initial Q-table before training\n",
        "print('Q-table before training:')\n",
        "\n",
        "# Displaying the Q-table in Markdown format using the display function\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "# Training\n",
        "for _ in range(episodes):\n",
        "    state, _ = environment.reset()\n",
        "    done = False\n",
        "\n",
        "    outcomes.append(0)\n",
        "\n",
        "    while not done:\n",
        "        # if you want to render the experiment, lower the number of episodes\n",
        "        #print(environment.render())\n",
        "        if np.max(qtable[state]) > 0:\n",
        "            # Choose the action with the highest value in the current state\n",
        "            action = np.argmax(qtable[state])\n",
        "        else:\n",
        "            # If there's no best action (only zeros), take a random one\n",
        "            action = environment.action_space.sample()\n",
        "\n",
        "        new_state, reward, done, info, prob = environment.step(action)\n",
        "\n",
        "        # if you want to render the experiment, uncomment the next lines\n",
        "        #time.sleep(0.5)\n",
        "        #clear_output(wait=True)\n",
        "        #if done == False:\n",
        "        #    clear_output(wait=True)\n",
        "\n",
        "        # Update Q(s,a)\n",
        "        qtable[state, action] = qtable[state, action] + alpha * \\\n",
        "            (reward + gamma *\n",
        "             np.max(qtable[new_state]) - qtable[state, action])\n",
        "\n",
        "        state = new_state\n",
        "\n",
        "        # If we have a reward, it means that our outcome is a success\n",
        "        if reward:\n",
        "            outcomes.append(1)\n",
        "\n",
        "\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "print('Q-table after training:')\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "fig = go.Figure()\n",
        "fig.add_trace(go.Bar(x=list(range(0, len(outcomes))), y=outcomes, marker_color='red'))\n",
        "fig.update_layout(\n",
        "    template='plotly_dark',\n",
        "    hoverlabel=dict(\n",
        "        bgcolor='rgba(0, 0, 0, 0.8)'\n",
        "    ),\n",
        "    title=f'Success rate = {(sum(outcomes)/episodes)*100}%',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)'\n",
        ")\n",
        "fig.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p2jdHui1M_mi"
      },
      "source": [
        "Now that the agent has been trained let us evaluate the agent's performance on 10 episodes.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lnA1Sk9WM_mi",
        "outputId": "a244346d-56a4-408d-b08f-a17eb8387c0f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Right)\n",
            "SFFF\n",
            "FHFH\n",
            "FFFH\n",
            "HF\u001b[41mF\u001b[0mG\n",
            "\n",
            "Success rate = 100.0%\n"
          ]
        }
      ],
      "source": [
        "episodes = 10\n",
        "success_rate = 0\n",
        "\n",
        "# Evaluation\n",
        "for _ in range(episodes):\n",
        "    state, _ = environment.reset()\n",
        "    done = False\n",
        "\n",
        "    while not done:\n",
        "        print(environment.render())\n",
        "        if np.max(qtable[state]) > 0:\n",
        "            action = np.argmax(qtable[state])\n",
        "\n",
        "        else:\n",
        "            action = environment.action_space.sample()\n",
        "\n",
        "        new_state, reward, done, info, prob = environment.step(action)\n",
        "\n",
        "        time.sleep(0.1)\n",
        "        if done == False:\n",
        "            clear_output(wait=True)\n",
        "\n",
        "        state = new_state\n",
        "\n",
        "        success_rate += reward\n",
        "\n",
        "# Let's check our success rate!\n",
        "print(f'Success rate = {(success_rate/episodes)*100}%')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OseLJnGuM_mi"
      },
      "source": [
        "## _Getting Greedy_ 🤑\n",
        "\n",
        "With the implementation above, the agent will always choose the action with the highest value. So whenever a state-action pair starts having a non-zero value, the agent will always choose it. The other actions will never be taken, which means we'll never update their value. We can use an $\\epsilon$-Greedy approach if you want to incentivize **exploration**.\n",
        "\n",
        "In RL, we can divide the \"_behavior_\" of our agent between exploitation (_taking the action with the highest value_) and exploration (_choosing a random action to try to find even better ones_).\n",
        "\n",
        "A trade-off between these two behaviors is important: if the agent only focuses on exploitation, it cannot try new solutions and thus doesn't learn anymore. On the other hand, if the agent only takes random actions, the training is pointless since it doesn't use the knowledge of the $Q$-function.\n",
        "\n",
        "The $\\epsilon$-Greedy algorithm is a way to find this balance.\n",
        "\n",
        "- Every time the agent has to take an action $a$, it has a probability $\\epsilon$ of choosing a random action, and a probability $1 - \\epsilon$ of choosing the one with the highest value.\n",
        "- We can decrease the value of $\\epsilon$ at the end of each episode by a fixed amount (_linear decay_), or based on the current value of $\\epsilon$ (_exponential decay_).\n",
        "\n",
        "In this way, we are incentivizing our agents to be more explorative.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "uEl9dCZ9M_mj",
        "outputId": "b169faa4-424d-492d-f74b-d586f3322190"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table before training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |   ◀️LEFT |   🔽DOWN |   ▶️RIGHT |   🔼UP |\n",
              "|:---------|--------:|---------:|---------:|-------:|\n",
              "| state 1  |       0 |        0 |        0 |      0 |\n",
              "| state 2  |       0 |        0 |        0 |      0 |\n",
              "| state 3  |       0 |        0 |        0 |      0 |\n",
              "| state 4  |       0 |        0 |        0 |      0 |\n",
              "| state 5  |       0 |        0 |        0 |      0 |\n",
              "| state 6  |       0 |        0 |        0 |      0 |\n",
              "| state 7  |       0 |        0 |        0 |      0 |\n",
              "| state 8  |       0 |        0 |        0 |      0 |\n",
              "| state 9  |       0 |        0 |        0 |      0 |\n",
              "| state 10 |       0 |        0 |        0 |      0 |\n",
              "| state 11 |       0 |        0 |        0 |      0 |\n",
              "| state 12 |       0 |        0 |        0 |      0 |\n",
              "| state 13 |       0 |        0 |        0 |      0 |\n",
              "| state 14 |       0 |        0 |        0 |      0 |\n",
              "| state 15 |       0 |        0 |        0 |      0 |\n",
              "| state 16 |       0 |        0 |        0 |      0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table after training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |    ◀️LEFT |   🔽DOWN |   ▶️RIGHT |     🔼UP |\n",
              "|:---------|---------:|---------:|---------:|---------:|\n",
              "| state 1  | 0.531441 |  0.59049 | 0.59049  | 0.531441 |\n",
              "| state 2  | 0.53144  |  0       | 0.6561   | 0.590489 |\n",
              "| state 3  | 0.590397 |  0.729   | 0.590183 | 0.65603  |\n",
              "| state 4  | 0.656028 |  0       | 0.580974 | 0.436915 |\n",
              "| state 5  | 0.59049  |  0.6561  | 0        | 0.531441 |\n",
              "| state 6  | 0        |  0       | 0        | 0        |\n",
              "| state 7  | 0        |  0.81    | 0        | 0.656091 |\n",
              "| state 8  | 0        |  0       | 0        | 0        |\n",
              "| state 9  | 0.6561   |  0       | 0.729    | 0.59049  |\n",
              "| state 10 | 0.6561   |  0.81    | 0.81     | 0        |\n",
              "| state 11 | 0.729    |  0.9     | 0        | 0.729    |\n",
              "| state 12 | 0        |  0       | 0        | 0        |\n",
              "| state 13 | 0        |  0       | 0        | 0        |\n",
              "| state 14 | 0        |  0.81    | 0.9      | 0.728934 |\n",
              "| state 15 | 0.81     |  0.9     | 1        | 0.81     |\n",
              "| state 16 | 0        |  0       | 0        | 0        |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"37c061e3-1213-440f-8d0a-09d00425c5a7\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"37c061e3-1213-440f-8d0a-09d00425c5a7\")) {                    Plotly.newPlot(                        \"37c061e3-1213-440f-8d0a-09d00425c5a7\",                        [{\"marker\":{\"color\":\"red\"},\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,1193,1194,1195,1196,1197,1198,1199,1200,1201,1202,1203,1204,1205,1206,1207,1208,1209,1210,1211,1212,1213,1214,1215,1216,1217,1218,1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253,1254,1255,1256,1257,1258,1259,1260,1261,1262,1263,1264,1265,1266,1267,1268,1269,1270,1271,1272,1273,1274,1275,1276,1277,1278,1279,1280,1281,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028,2029,2030,2031,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047,2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061,2062,2063,2064,2065,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,2076,2077,2078,2079,2080,2081,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109,2110,2111,2112,2113,2114,2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126,2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137,2138,2139,2140,2141,2142,2143,2144,2145,2146,2147,2148,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2159,2160,2161,2162,2163,2164,2165,2166,2167,2168,2169,2170,2171,2172,2173,2174,2175,2176,2177,2178,2179,2180,2181,2182,2183,2184,2185,2186,2187,2188,2189,2190,2191,2192,2193,2194,2195,2196,2197,2198,2199,2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,2221,2222,2223,2224,2225,2226,2227,2228,2229,2230,2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,2241,2242,2243,2244,2245,2246,2247,2248,2249,2250,2251,2252,2253,2254,2255,2256,2257,2258,2259,2260,2261,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,2278,2279,2280,2281,2282,2283,2284,2285,2286,2287,2288,2289,2290,2291,2292,2293,2294,2295,2296,2297,2298,2299,2300,2301,2302,2303,2304,2305,2306,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,2323,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,2337,2338,2339,2340,2341,2342,2343,2344,2345,2346,2347,2348,2349,2350,2351,2352,2353,2354,2355,2356,2357,2358,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2374,2375,2376,2377,2378,2379,2380,2381,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2399,2400,2401,2402,2403,2404,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,2426,2427,2428,2429,2430,2431,2432,2433,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2448,2449,2450,2451,2452,2453,2454,2455,2456,2457,2458,2459,2460,2461,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2497,2498,2499,2500,2501,2502,2503,2504,2505,2506,2507,2508,2509,2510,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2530,2531,2532,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2550,2551,2552,2553,2554,2555,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2576,2577,2578,2579,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2597,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2609,2610,2611,2612,2613,2614,2615,2616,2617,2618,2619,2620,2621,2622,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2636,2637,2638,2639,2640,2641,2642,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,2721,2722,2723,2724,2725,2726,2727,2728,2729,2730,2731,2732,2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,2753,2754,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,2780,2781,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,2810,2811,2812,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,2857,2858,2859,2860,2861,2862,2863,2864,2865,2866,2867,2868,2869,2870,2871,2872,2873,2874,2875,2876,2877,2878,2879,2880,2881,2882,2883,2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,2993,2994,2995,2996,2997,2998,2999,3000,3001,3002,3003,3004,3005,3006,3007,3008,3009,3010,3011,3012,3013,3014,3015,3016,3017,3018,3019,3020,3021,3022,3023,3024,3025,3026,3027,3028,3029,3030,3031,3032,3033,3034,3035,3036,3037,3038,3039,3040,3041,3042,3043,3044,3045,3046,3047,3048,3049,3050,3051,3052,3053,3054,3055,3056,3057,3058,3059,3060,3061,3062,3063,3064,3065,3066,3067,3068,3069,3070,3071,3072,3073,3074,3075,3076,3077,3078,3079,3080,3081,3082,3083,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,3096,3097,3098,3099,3100,3101,3102,3103,3104,3105,3106,3107,3108,3109,3110,3111,3112,3113,3114,3115,3116,3117,3118,3119,3120,3121,3122,3123,3124,3125,3126,3127,3128,3129,3130,3131,3132,3133,3134,3135,3136,3137,3138,3139,3140,3141,3142,3143,3144,3145,3146,3147,3148,3149,3150,3151,3152,3153,3154,3155,3156,3157,3158,3159,3160,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,3173,3174,3175,3176,3177,3178,3179,3180,3181,3182,3183,3184,3185,3186,3187,3188,3189,3190,3191,3192,3193,3194,3195,3196,3197,3198,3199,3200,3201,3202,3203,3204,3205,3206,3207,3208,3209,3210,3211,3212,3213,3214,3215,3216,3217,3218,3219,3220,3221,3222,3223,3224,3225,3226,3227,3228,3229,3230,3231,3232,3233,3234,3235,3236,3237,3238,3239,3240,3241,3242,3243,3244,3245,3246,3247,3248,3249,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,3262,3263,3264,3265,3266,3267,3268,3269,3270,3271,3272,3273,3274,3275,3276,3277,3278,3279,3280,3281,3282,3283,3284,3285,3286,3287,3288,3289,3290,3291,3292,3293,3294,3295,3296,3297,3298,3299,3300,3301,3302,3303,3304,3305,3306,3307,3308,3309,3310,3311,3312,3313,3314,3315,3316,3317,3318,3319,3320,3321,3322,3323,3324,3325,3326,3327,3328,3329,3330,3331,3332,3333,3334,3335,3336,3337,3338,3339,3340,3341,3342,3343,3344,3345,3346,3347,3348,3349,3350,3351,3352,3353,3354,3355,3356,3357,3358,3359,3360,3361,3362,3363,3364,3365,3366,3367,3368,3369,3370,3371,3372,3373,3374,3375,3376,3377,3378,3379,3380,3381,3382,3383,3384,3385,3386,3387,3388,3389,3390,3391,3392,3393,3394,3395,3396,3397,3398,3399,3400,3401,3402,3403,3404,3405,3406,3407,3408,3409,3410,3411,3412,3413,3414,3415,3416,3417,3418,3419,3420,3421,3422,3423,3424,3425,3426,3427,3428,3429,3430,3431,3432,3433,3434,3435,3436,3437,3438,3439,3440,3441,3442,3443,3444],\"y\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"hoverlabel\":{\"bgcolor\":\"rgba(0, 0, 0, 0.8)\"},\"title\":{\"text\":\"Success rate = 72.25%\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('37c061e3-1213-440f-8d0a-09d00425c5a7');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "qtable = np.zeros((environment.observation_space.n,\n",
        "                  environment.action_space.n))\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "\n",
        "episodes = 2000\n",
        "alpha = 0.5\n",
        "gamma = 0.9\n",
        "epsilon = 1.0         # start with full exploration mode\n",
        "epsilon_decay = 0.001  # decay epsilon by this amount\n",
        "\n",
        "outcomes = []\n",
        "\n",
        "print('Q-table before training:')\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "# Training\n",
        "for _ in range(episodes):\n",
        "    state, _ = environment.reset()\n",
        "    done = False\n",
        "    outcomes.append(0)\n",
        "\n",
        "    while not done:\n",
        "        # Generate a random number between 0 and 1\n",
        "        rnd = np.random.random()\n",
        "\n",
        "        # If random number < epsilon, take a random action\n",
        "        if rnd < epsilon:\n",
        "\n",
        "            action = environment.action_space.sample()\n",
        "        # Else, take the action with the highest value in the current state\n",
        "        else:\n",
        "\n",
        "            action = np.argmax(qtable[state])\n",
        "\n",
        "        new_state, reward, done, info, proba = environment.step(action)\n",
        "\n",
        "        qtable[state, action] = qtable[state, action] + alpha * \\\n",
        "            (reward + gamma *\n",
        "             np.max(qtable[new_state]) - qtable[state, action])\n",
        "\n",
        "        state = new_state\n",
        "        if reward:\n",
        "            outcomes.append(1)\n",
        "\n",
        "    # Update epsilon\n",
        "    epsilon = max(epsilon - epsilon_decay, 0)\n",
        "\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "print('Q-table after training:')\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "fig = go.Figure()\n",
        "fig.add_trace(go.Bar(x=list(range(0, len(outcomes))), y=outcomes, marker_color='red'))\n",
        "fig.update_layout(\n",
        "    template='plotly_dark',\n",
        "    hoverlabel=dict(\n",
        "        bgcolor='rgba(0, 0, 0, 0.8)'\n",
        "    ),\n",
        "    title=f'Success rate = {(sum(outcomes)/episodes)*100}%',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)'\n",
        ")\n",
        "fig.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PTBKbA5VM_mj"
      },
      "source": [
        "Our success rate during training is _much lower_ now, and it takes _longer for the agent to find optimal solutions for Frozen Lake_. However, our $Q$-table is much _more complete_, and our agent has probably learned _new strategies/paths_.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yQQhhLR6M_mj",
        "outputId": "0fe37e74-c650-4e70-87d3-8239bc1f8825"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Right)\n",
            "SFFF\n",
            "FHFH\n",
            "FFFH\n",
            "HF\u001b[41mF\u001b[0mG\n",
            "\n",
            "Success rate = 100.0%\n"
          ]
        }
      ],
      "source": [
        "episodes = 10\n",
        "success_rate = 0\n",
        "\n",
        "# Evaluation\n",
        "for _ in range(episodes):\n",
        "    state, _ = environment.reset()\n",
        "    done = False\n",
        "\n",
        "    while not done:\n",
        "        print(environment.render())\n",
        "        if np.max(qtable[state]) > 0:\n",
        "            action = np.argmax(qtable[state])\n",
        "\n",
        "        else:\n",
        "            action = environment.action_space.sample()\n",
        "\n",
        "        new_state, reward, done, info, prob = environment.step(action)\n",
        "\n",
        "        time.sleep(0.1)\n",
        "        if done == False:\n",
        "            clear_output(wait=True)\n",
        "\n",
        "        state = new_state\n",
        "\n",
        "        success_rate += reward\n",
        "\n",
        "# Let's check our success rate!\n",
        "print(f'Success rate = {(success_rate/episodes)*100}%')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GGmYups6M_mj"
      },
      "source": [
        "## Frozen and Slippery❄️\n",
        "\n",
        "The Frozen Lake environment has a variant where, even if the agent chooses to take action $X$, there is a probability that the action performed is $Y$. This is a slippery mode (`is_slippery = True`). In the slippery variant, the agent's action only has a 33% chance of succeeding. This feature makes things more difficult for our agent."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "8PFmKcg7M_mj",
        "outputId": "08e1961d-d0d7-49cb-b296-8c1e71c5cbd4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table before training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |   ◀️LEFT |   🔽DOWN |   ▶️RIGHT |   🔼UP |\n",
              "|:---------|--------:|---------:|---------:|-------:|\n",
              "| state 1  |       0 |        0 |        0 |      0 |\n",
              "| state 2  |       0 |        0 |        0 |      0 |\n",
              "| state 3  |       0 |        0 |        0 |      0 |\n",
              "| state 4  |       0 |        0 |        0 |      0 |\n",
              "| state 5  |       0 |        0 |        0 |      0 |\n",
              "| state 6  |       0 |        0 |        0 |      0 |\n",
              "| state 7  |       0 |        0 |        0 |      0 |\n",
              "| state 8  |       0 |        0 |        0 |      0 |\n",
              "| state 9  |       0 |        0 |        0 |      0 |\n",
              "| state 10 |       0 |        0 |        0 |      0 |\n",
              "| state 11 |       0 |        0 |        0 |      0 |\n",
              "| state 12 |       0 |        0 |        0 |      0 |\n",
              "| state 13 |       0 |        0 |        0 |      0 |\n",
              "| state 14 |       0 |        0 |        0 |      0 |\n",
              "| state 15 |       0 |        0 |        0 |      0 |\n",
              "| state 16 |       0 |        0 |        0 |      0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Q-table after training:\n"
          ]
        },
        {
          "data": {
            "text/markdown": [
              "|          |      ◀️LEFT |      🔽DOWN |      ▶️RIGHT |        🔼UP |\n",
              "|:---------|-----------:|------------:|------------:|------------:|\n",
              "| state 1  | 0.115783   | 0.00582996  | 0.00570754  | 0.00566233  |\n",
              "| state 2  | 0.0028443  | 0.00282399  | 0.00277072  | 0.0578619   |\n",
              "| state 3  | 0.00216543 | 0.00156305  | 0.00213138  | 0.0338292   |\n",
              "| state 4  | 0.00111277 | 0.000851175 | 0.0007101   | 0.028117    |\n",
              "| state 5  | 0.136431   | 0.0062986   | 0.00445035  | 0.0067661   |\n",
              "| state 6  | 0          | 0           | 0           | 0           |\n",
              "| state 7  | 0.0734774  | 0.000419049 | 0.000421992 | 0.000356506 |\n",
              "| state 8  | 0          | 0           | 0           | 0           |\n",
              "| state 9  | 0.00967801 | 0.0128841   | 0.0119077   | 0.195107    |\n",
              "| state 10 | 0.02405    | 0.282775    | 0.0181691   | 0.0152109   |\n",
              "| state 11 | 0.245288   | 0.0151233   | 0.00818122  | 0.0141798   |\n",
              "| state 12 | 0          | 0           | 0           | 0           |\n",
              "| state 13 | 0          | 0           | 0           | 0           |\n",
              "| state 14 | 0.0428629  | 0.0431293   | 0.519502    | 0.0488329   |\n",
              "| state 15 | 0.0925838  | 0.942191    | 0.0944371   | 0.0912259   |\n",
              "| state 16 | 0          | 0           | 0           | 0           |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"e469a4a2-cf3a-458c-b791-4365d7724363\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"e469a4a2-cf3a-458c-b791-4365d7724363\")) {                    Plotly.newPlot(                        \"e469a4a2-cf3a-458c-b791-4365d7724363\",                        [{\"marker\":{\"color\":\"red\"},\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,1193,1194,1195,1196,1197,1198,1199,1200,1201,1202,1203,1204,1205,1206,1207,1208,1209,1210,1211,1212,1213,1214,1215,1216,1217,1218,1219,1220,1221,1222,1223,1224,1225,1226,1227,1228,1229,1230,1231,1232,1233,1234,1235,1236,1237,1238,1239,1240,1241,1242,1243,1244,1245,1246,1247,1248,1249,1250,1251,1252,1253,1254,1255,1256,1257,1258,1259,1260,1261,1262,1263,1264,1265,1266,1267,1268,1269,1270,1271,1272,1273,1274,1275,1276,1277,1278,1279,1280,1281,1282,1283,1284,1285,1286,1287,1288,1289,1290,1291,1292,1293,1294,1295,1296,1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028,2029,2030,2031,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047,2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061,2062,2063,2064,2065,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,2076,2077,2078,2079,2080,2081,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109,2110,2111,2112,2113,2114,2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126,2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137,2138,2139,2140,2141,2142,2143,2144,2145,2146,2147,2148,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2159,2160,2161,2162,2163,2164,2165,2166,2167,2168,2169,2170,2171,2172,2173,2174,2175,2176,2177,2178,2179,2180,2181,2182,2183,2184,2185,2186,2187,2188,2189,2190,2191,2192,2193,2194,2195,2196,2197,2198,2199,2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,2221,2222,2223,2224,2225,2226,2227,2228,2229,2230,2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,2241,2242,2243,2244,2245,2246,2247,2248,2249,2250,2251,2252,2253,2254,2255,2256,2257,2258,2259,2260,2261,2262,2263,2264,2265,2266,2267,2268,2269,2270,2271,2272,2273,2274,2275,2276,2277,2278,2279,2280,2281,2282,2283,2284,2285,2286,2287,2288,2289,2290,2291,2292,2293,2294,2295,2296,2297,2298,2299,2300,2301,2302,2303,2304,2305,2306,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317,2318,2319,2320,2321,2322,2323,2324,2325,2326,2327,2328,2329,2330,2331,2332,2333,2334,2335,2336,2337,2338,2339,2340,2341,2342,2343,2344,2345,2346,2347,2348,2349,2350,2351,2352,2353,2354,2355,2356,2357,2358,2359,2360,2361,2362,2363,2364,2365,2366,2367,2368,2369,2370,2371,2372,2373,2374,2375,2376,2377,2378,2379,2380,2381,2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395,2396,2397,2398,2399,2400,2401,2402,2403,2404,2405,2406,2407,2408,2409,2410,2411,2412,2413,2414,2415,2416,2417,2418,2419,2420,2421,2422,2423,2424,2425,2426,2427,2428,2429,2430,2431,2432,2433,2434,2435,2436,2437,2438,2439,2440,2441,2442,2443,2444,2445,2446,2447,2448,2449,2450,2451,2452,2453,2454,2455,2456,2457,2458,2459,2460,2461,2462,2463,2464,2465,2466,2467,2468,2469,2470,2471,2472,2473,2474,2475,2476,2477,2478,2479,2480,2481,2482,2483,2484,2485,2486,2487,2488,2489,2490,2491,2492,2493,2494,2495,2496,2497,2498,2499,2500,2501,2502,2503,2504,2505,2506,2507,2508,2509,2510,2511,2512,2513,2514,2515,2516,2517,2518,2519,2520,2521,2522,2523,2524,2525,2526,2527,2528,2529,2530,2531,2532,2533,2534,2535,2536,2537,2538,2539,2540,2541,2542,2543,2544,2545,2546,2547,2548,2549,2550,2551,2552,2553,2554,2555,2556,2557,2558,2559,2560,2561,2562,2563,2564,2565,2566,2567,2568,2569,2570,2571,2572,2573,2574,2575,2576,2577,2578,2579,2580,2581,2582,2583,2584,2585,2586,2587,2588,2589,2590,2591,2592,2593,2594,2595,2596,2597,2598,2599,2600,2601,2602,2603,2604,2605,2606,2607,2608,2609,2610,2611,2612,2613,2614,2615,2616,2617,2618,2619,2620,2621,2622,2623,2624,2625,2626,2627,2628,2629,2630,2631,2632,2633,2634,2635,2636,2637,2638,2639,2640,2641,2642,2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2699,2700,2701,2702,2703,2704,2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,2721,2722,2723,2724,2725,2726,2727,2728,2729,2730,2731,2732,2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,2751,2752,2753,2754,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,2766,2767,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,2780,2781,2782,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,2810,2811,2812,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,2857,2858,2859,2860,2861,2862,2863,2864,2865,2866,2867,2868,2869,2870,2871,2872,2873,2874,2875,2876,2877,2878,2879,2880,2881,2882,2883,2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,2944,2945,2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,2962,2963,2964,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,2993,2994,2995,2996,2997,2998,2999,3000,3001,3002,3003,3004,3005,3006,3007,3008,3009,3010,3011,3012,3013,3014,3015,3016,3017,3018,3019,3020,3021,3022,3023,3024,3025,3026,3027,3028,3029,3030,3031,3032,3033,3034,3035,3036,3037,3038,3039,3040,3041,3042,3043,3044,3045,3046,3047,3048,3049,3050,3051,3052,3053,3054,3055,3056,3057,3058,3059,3060,3061,3062,3063,3064,3065,3066,3067,3068,3069,3070,3071,3072,3073,3074,3075,3076,3077,3078,3079,3080,3081,3082,3083,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,3096,3097,3098,3099,3100,3101,3102,3103,3104,3105,3106,3107,3108,3109,3110,3111,3112,3113,3114,3115,3116,3117,3118,3119,3120,3121,3122,3123,3124,3125,3126,3127,3128,3129,3130,3131,3132,3133,3134,3135,3136,3137,3138,3139,3140,3141,3142,3143,3144,3145,3146,3147,3148,3149,3150,3151,3152,3153,3154,3155,3156,3157,3158,3159,3160,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,3173,3174,3175,3176,3177,3178,3179,3180,3181,3182,3183,3184,3185,3186,3187,3188,3189,3190,3191,3192,3193,3194,3195,3196,3197,3198,3199,3200,3201,3202,3203,3204,3205,3206,3207,3208,3209,3210,3211,3212,3213,3214,3215,3216,3217,3218,3219,3220,3221,3222,3223,3224,3225,3226,3227,3228,3229,3230,3231,3232,3233,3234,3235,3236,3237,3238,3239,3240,3241,3242,3243,3244,3245,3246,3247,3248,3249,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,3262,3263,3264,3265,3266,3267,3268,3269,3270,3271,3272,3273,3274,3275,3276,3277,3278,3279,3280,3281,3282,3283,3284,3285,3286,3287,3288,3289,3290,3291,3292,3293,3294,3295,3296,3297,3298,3299,3300,3301,3302,3303,3304,3305,3306,3307,3308,3309,3310,3311,3312,3313,3314,3315,3316,3317,3318,3319,3320,3321,3322,3323,3324,3325,3326,3327,3328,3329,3330,3331,3332,3333,3334,3335,3336,3337,3338,3339,3340,3341,3342,3343,3344,3345,3346,3347,3348,3349,3350,3351,3352,3353,3354,3355,3356,3357,3358,3359,3360,3361,3362,3363,3364,3365,3366,3367,3368,3369,3370,3371,3372,3373,3374,3375,3376,3377,3378,3379,3380,3381,3382,3383,3384,3385,3386,3387,3388,3389,3390,3391,3392,3393,3394,3395,3396,3397,3398,3399,3400,3401,3402,3403,3404,3405,3406,3407,3408,3409,3410,3411,3412,3413,3414,3415,3416,3417,3418,3419,3420,3421,3422,3423,3424,3425,3426,3427,3428,3429,3430,3431,3432,3433,3434,3435,3436,3437,3438,3439,3440,3441,3442,3443,3444,3445,3446,3447,3448,3449,3450,3451,3452,3453,3454,3455,3456,3457,3458,3459,3460,3461,3462,3463,3464,3465,3466,3467,3468,3469,3470,3471,3472,3473,3474,3475,3476,3477,3478,3479,3480,3481,3482,3483,3484,3485,3486,3487,3488,3489,3490,3491,3492,3493,3494,3495,3496,3497,3498,3499,3500,3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,3512,3513,3514,3515,3516,3517,3518,3519,3520,3521,3522,3523,3524,3525,3526,3527,3528,3529,3530,3531,3532,3533,3534,3535,3536,3537,3538,3539,3540,3541,3542,3543,3544,3545,3546,3547,3548,3549,3550,3551,3552,3553,3554,3555,3556,3557,3558,3559,3560,3561,3562,3563,3564,3565,3566,3567,3568,3569,3570,3571,3572,3573,3574,3575,3576,3577,3578,3579,3580,3581,3582,3583,3584,3585,3586,3587,3588,3589,3590,3591,3592,3593,3594,3595,3596,3597,3598,3599,3600,3601,3602,3603,3604,3605,3606,3607,3608,3609,3610,3611,3612,3613,3614,3615,3616,3617,3618,3619,3620,3621,3622,3623,3624,3625,3626,3627,3628,3629,3630,3631,3632,3633,3634,3635,3636,3637,3638,3639,3640,3641,3642,3643,3644,3645,3646,3647,3648,3649,3650,3651,3652,3653,3654,3655,3656,3657,3658,3659,3660,3661,3662,3663,3664,3665,3666,3667,3668,3669,3670,3671,3672,3673,3674,3675,3676,3677,3678,3679,3680,3681,3682,3683,3684,3685,3686,3687,3688,3689,3690,3691,3692,3693,3694,3695,3696,3697,3698,3699,3700,3701,3702,3703,3704,3705,3706,3707,3708,3709,3710,3711,3712,3713,3714,3715,3716,3717,3718,3719,3720,3721,3722,3723,3724,3725,3726,3727,3728,3729,3730,3731,3732,3733,3734,3735,3736,3737,3738,3739,3740,3741,3742,3743,3744,3745,3746,3747,3748,3749,3750,3751,3752,3753,3754,3755,3756,3757,3758,3759,3760,3761,3762,3763,3764,3765,3766,3767,3768,3769,3770,3771,3772,3773,3774,3775,3776,3777,3778,3779,3780,3781,3782,3783,3784,3785,3786,3787,3788,3789,3790,3791,3792,3793,3794,3795,3796,3797,3798,3799,3800,3801,3802,3803,3804,3805,3806,3807,3808,3809,3810,3811,3812,3813,3814,3815,3816,3817,3818,3819,3820,3821,3822,3823,3824,3825,3826,3827,3828,3829,3830,3831,3832,3833,3834,3835,3836,3837,3838,3839,3840,3841,3842,3843,3844,3845,3846,3847,3848,3849,3850,3851,3852,3853,3854,3855,3856,3857,3858,3859,3860,3861,3862,3863,3864,3865,3866,3867,3868,3869,3870,3871,3872,3873,3874,3875,3876,3877,3878,3879,3880,3881,3882,3883,3884,3885,3886,3887,3888,3889,3890,3891,3892,3893,3894,3895,3896,3897,3898,3899,3900,3901,3902,3903,3904,3905,3906,3907,3908,3909,3910,3911,3912,3913,3914,3915,3916,3917,3918,3919,3920,3921,3922,3923,3924,3925,3926,3927,3928,3929,3930,3931,3932,3933,3934,3935,3936,3937,3938,3939,3940,3941,3942,3943,3944,3945,3946,3947,3948,3949,3950,3951,3952,3953,3954,3955,3956,3957,3958,3959,3960,3961,3962,3963,3964,3965,3966,3967,3968,3969,3970,3971,3972,3973,3974,3975,3976,3977,3978,3979,3980,3981,3982,3983,3984,3985,3986,3987,3988,3989,3990,3991,3992,3993,3994,3995,3996,3997,3998,3999,4000,4001,4002,4003,4004,4005,4006,4007,4008,4009,4010,4011,4012,4013,4014,4015,4016,4017,4018,4019,4020,4021,4022,4023,4024,4025,4026,4027,4028,4029,4030,4031,4032,4033,4034,4035,4036,4037,4038,4039,4040,4041,4042,4043,4044,4045,4046,4047,4048,4049,4050,4051,4052,4053,4054,4055,4056,4057,4058,4059,4060,4061,4062,4063,4064,4065,4066,4067,4068,4069,4070,4071,4072,4073,4074,4075,4076,4077,4078,4079,4080,4081,4082,4083,4084,4085,4086,4087,4088,4089,4090,4091,4092,4093,4094,4095,4096,4097,4098,4099,4100,4101,4102,4103,4104,4105,4106,4107,4108,4109,4110,4111,4112,4113,4114,4115,4116,4117,4118,4119,4120,4121,4122,4123,4124,4125,4126,4127,4128,4129,4130,4131,4132,4133,4134,4135,4136,4137,4138,4139,4140,4141,4142,4143,4144,4145,4146,4147,4148,4149,4150,4151,4152,4153,4154,4155,4156,4157,4158,4159,4160,4161,4162,4163,4164,4165,4166,4167,4168,4169,4170,4171,4172,4173,4174,4175,4176,4177,4178,4179,4180,4181,4182,4183,4184,4185,4186,4187,4188,4189,4190,4191,4192,4193,4194,4195,4196,4197,4198,4199,4200,4201,4202,4203,4204,4205,4206,4207,4208,4209,4210,4211,4212,4213,4214,4215,4216,4217,4218,4219,4220,4221,4222,4223,4224,4225,4226,4227,4228,4229,4230,4231,4232,4233,4234,4235,4236,4237,4238,4239,4240,4241,4242,4243,4244,4245,4246,4247,4248,4249,4250,4251,4252,4253,4254,4255,4256,4257,4258,4259,4260,4261,4262,4263,4264,4265,4266,4267,4268,4269,4270,4271,4272,4273,4274,4275,4276,4277,4278,4279,4280,4281,4282,4283,4284,4285,4286,4287,4288,4289,4290,4291,4292,4293,4294,4295,4296,4297,4298,4299,4300,4301,4302,4303,4304,4305,4306,4307,4308,4309,4310,4311,4312,4313,4314,4315,4316,4317,4318,4319,4320,4321,4322,4323,4324,4325,4326,4327,4328,4329,4330,4331,4332,4333,4334,4335,4336,4337,4338,4339,4340,4341,4342,4343,4344,4345,4346,4347,4348,4349,4350,4351,4352,4353,4354,4355,4356,4357,4358,4359,4360,4361,4362,4363,4364,4365,4366,4367,4368,4369,4370,4371,4372,4373,4374,4375,4376,4377,4378,4379,4380,4381,4382,4383,4384,4385,4386,4387,4388,4389,4390,4391,4392,4393,4394,4395,4396,4397,4398,4399,4400,4401,4402,4403,4404,4405,4406,4407,4408,4409,4410,4411,4412,4413,4414,4415,4416,4417,4418,4419,4420,4421,4422,4423,4424,4425,4426,4427,4428,4429,4430,4431,4432,4433,4434,4435,4436,4437,4438,4439,4440,4441,4442,4443,4444,4445,4446,4447,4448,4449,4450,4451,4452,4453,4454,4455,4456,4457,4458,4459,4460,4461,4462,4463,4464,4465,4466,4467,4468,4469,4470,4471,4472,4473,4474,4475,4476,4477,4478,4479,4480,4481,4482,4483,4484,4485,4486,4487,4488,4489,4490,4491,4492,4493,4494,4495,4496,4497,4498,4499,4500,4501,4502,4503,4504,4505,4506,4507,4508,4509,4510,4511,4512,4513,4514,4515,4516,4517,4518,4519,4520,4521,4522,4523,4524,4525,4526,4527,4528,4529,4530,4531,4532,4533,4534,4535,4536,4537,4538,4539,4540,4541,4542,4543,4544,4545,4546,4547,4548,4549,4550,4551,4552,4553,4554,4555,4556,4557,4558,4559,4560,4561,4562,4563,4564,4565,4566,4567,4568,4569,4570,4571,4572,4573,4574,4575,4576,4577,4578,4579,4580,4581,4582,4583,4584,4585,4586,4587,4588,4589,4590,4591,4592,4593,4594,4595,4596,4597,4598,4599,4600,4601,4602,4603,4604,4605,4606,4607,4608,4609,4610,4611,4612,4613,4614,4615,4616,4617,4618,4619,4620,4621,4622,4623,4624,4625,4626,4627,4628,4629,4630,4631,4632,4633,4634,4635,4636,4637,4638,4639,4640,4641,4642,4643,4644,4645,4646,4647,4648,4649,4650,4651,4652,4653,4654,4655,4656,4657,4658,4659,4660,4661,4662,4663,4664,4665,4666,4667,4668,4669,4670,4671,4672,4673,4674,4675,4676,4677,4678,4679,4680,4681,4682,4683,4684,4685,4686,4687,4688,4689,4690,4691,4692,4693,4694,4695,4696,4697,4698,4699,4700,4701,4702,4703,4704,4705,4706,4707,4708,4709,4710,4711,4712,4713,4714,4715,4716,4717,4718,4719,4720,4721,4722,4723,4724,4725,4726,4727,4728,4729,4730,4731,4732,4733,4734,4735,4736,4737,4738,4739,4740,4741,4742,4743,4744,4745,4746,4747,4748,4749,4750,4751,4752,4753,4754,4755,4756,4757,4758,4759,4760,4761,4762,4763,4764,4765,4766,4767,4768,4769,4770,4771,4772,4773,4774,4775,4776,4777,4778,4779,4780,4781,4782,4783,4784,4785,4786,4787,4788,4789,4790,4791,4792,4793,4794,4795,4796,4797,4798,4799,4800,4801,4802,4803,4804,4805,4806,4807,4808,4809,4810,4811,4812,4813,4814,4815,4816,4817,4818,4819,4820,4821,4822,4823,4824,4825,4826,4827,4828,4829,4830,4831,4832,4833,4834,4835,4836,4837,4838,4839,4840,4841,4842,4843,4844,4845,4846,4847,4848,4849,4850,4851,4852,4853,4854,4855,4856,4857,4858,4859,4860,4861,4862,4863,4864,4865,4866,4867,4868,4869,4870,4871,4872,4873,4874,4875,4876,4877,4878,4879,4880,4881,4882,4883,4884,4885,4886,4887,4888,4889,4890,4891,4892,4893,4894,4895,4896,4897,4898,4899,4900,4901,4902,4903,4904,4905,4906,4907,4908,4909,4910,4911,4912,4913,4914,4915,4916,4917,4918,4919,4920,4921,4922,4923,4924,4925,4926,4927,4928,4929,4930,4931,4932,4933,4934,4935,4936,4937,4938,4939,4940,4941,4942,4943,4944,4945,4946,4947,4948,4949,4950,4951,4952,4953,4954,4955,4956,4957,4958,4959,4960,4961,4962,4963,4964,4965,4966,4967,4968,4969,4970,4971,4972,4973,4974,4975,4976,4977,4978,4979,4980,4981,4982,4983,4984,4985,4986,4987,4988,4989,4990,4991,4992,4993,4994,4995,4996,4997,4998,4999,5000,5001,5002,5003,5004,5005,5006,5007,5008,5009,5010,5011,5012,5013,5014,5015,5016,5017,5018,5019,5020,5021,5022,5023,5024,5025,5026,5027,5028,5029,5030,5031,5032,5033,5034,5035,5036,5037,5038,5039,5040,5041,5042,5043,5044,5045,5046,5047,5048,5049,5050,5051,5052,5053,5054,5055,5056,5057,5058,5059,5060,5061,5062,5063,5064,5065,5066,5067,5068,5069,5070,5071,5072,5073,5074,5075,5076,5077,5078,5079,5080,5081,5082,5083,5084,5085,5086,5087,5088,5089,5090,5091,5092,5093,5094,5095,5096,5097,5098,5099,5100,5101,5102,5103,5104,5105,5106,5107,5108,5109,5110,5111,5112,5113,5114,5115,5116,5117,5118,5119,5120,5121,5122,5123,5124,5125,5126,5127,5128,5129,5130,5131,5132,5133,5134,5135,5136,5137,5138,5139,5140,5141,5142,5143,5144,5145,5146,5147,5148,5149,5150,5151,5152,5153,5154,5155,5156,5157,5158,5159,5160,5161,5162,5163,5164,5165,5166,5167,5168,5169,5170,5171,5172,5173,5174,5175,5176,5177,5178,5179,5180,5181,5182,5183,5184,5185,5186,5187,5188,5189,5190,5191,5192,5193,5194,5195,5196,5197,5198,5199,5200,5201,5202,5203,5204,5205,5206,5207,5208,5209,5210,5211,5212,5213,5214,5215,5216,5217,5218,5219,5220,5221,5222,5223,5224,5225,5226,5227,5228,5229,5230,5231,5232,5233,5234,5235,5236,5237,5238,5239,5240,5241,5242,5243,5244,5245,5246,5247,5248,5249,5250,5251,5252,5253,5254,5255,5256,5257,5258,5259,5260,5261,5262,5263,5264,5265,5266,5267,5268,5269,5270,5271,5272,5273,5274,5275,5276,5277,5278,5279,5280,5281,5282,5283,5284,5285,5286,5287,5288,5289,5290,5291,5292,5293,5294,5295,5296,5297,5298,5299,5300,5301,5302,5303,5304,5305,5306,5307,5308,5309,5310,5311,5312,5313,5314,5315,5316,5317,5318,5319,5320,5321,5322,5323,5324,5325,5326,5327,5328,5329,5330,5331,5332,5333,5334,5335,5336,5337,5338,5339,5340,5341,5342,5343,5344,5345,5346,5347,5348,5349,5350,5351,5352,5353,5354,5355,5356,5357,5358,5359,5360,5361,5362,5363,5364,5365,5366,5367,5368,5369,5370,5371,5372,5373,5374,5375,5376,5377,5378,5379,5380,5381,5382,5383,5384,5385,5386,5387,5388,5389,5390,5391,5392,5393,5394,5395,5396,5397,5398,5399,5400,5401,5402,5403,5404,5405,5406,5407,5408,5409,5410,5411,5412,5413,5414,5415,5416,5417,5418,5419,5420,5421,5422,5423,5424,5425,5426,5427,5428,5429,5430,5431,5432,5433,5434,5435,5436,5437,5438,5439,5440,5441,5442,5443,5444,5445,5446,5447,5448,5449,5450,5451,5452,5453,5454,5455,5456,5457,5458,5459,5460,5461,5462,5463,5464,5465,5466,5467,5468,5469,5470,5471,5472,5473,5474,5475,5476,5477,5478,5479,5480,5481,5482,5483,5484,5485,5486,5487,5488,5489,5490,5491,5492,5493,5494,5495,5496,5497,5498,5499,5500,5501,5502,5503,5504,5505,5506,5507,5508,5509,5510,5511,5512,5513,5514,5515,5516,5517,5518,5519,5520,5521,5522,5523,5524,5525,5526,5527,5528,5529,5530,5531,5532,5533,5534,5535,5536,5537,5538,5539,5540,5541,5542,5543,5544,5545,5546,5547,5548,5549,5550,5551,5552,5553,5554,5555,5556,5557,5558,5559,5560,5561,5562,5563,5564,5565,5566,5567,5568,5569,5570,5571,5572,5573,5574,5575,5576,5577,5578,5579,5580,5581,5582,5583,5584,5585,5586,5587,5588,5589,5590,5591,5592,5593,5594,5595,5596,5597,5598,5599,5600,5601,5602,5603,5604,5605,5606,5607,5608,5609,5610,5611,5612,5613,5614,5615,5616,5617,5618,5619,5620,5621,5622,5623,5624,5625,5626,5627,5628,5629,5630,5631,5632,5633,5634,5635,5636,5637,5638,5639,5640,5641,5642,5643,5644,5645,5646,5647,5648,5649,5650,5651,5652,5653,5654,5655,5656,5657,5658,5659,5660,5661,5662,5663,5664,5665,5666,5667,5668,5669,5670,5671,5672,5673,5674,5675,5676,5677,5678,5679,5680,5681,5682,5683,5684,5685,5686,5687,5688,5689,5690,5691,5692,5693,5694,5695,5696,5697,5698,5699,5700,5701,5702,5703,5704,5705,5706,5707,5708,5709,5710,5711,5712,5713,5714,5715,5716,5717,5718,5719,5720,5721,5722,5723,5724,5725,5726,5727,5728,5729,5730,5731,5732,5733,5734,5735,5736,5737,5738,5739,5740,5741,5742,5743,5744,5745,5746,5747,5748,5749,5750,5751,5752,5753,5754,5755,5756,5757,5758,5759,5760,5761,5762,5763,5764,5765,5766,5767,5768,5769,5770,5771,5772,5773,5774,5775,5776,5777,5778,5779,5780,5781,5782,5783,5784,5785,5786,5787,5788,5789,5790,5791,5792,5793,5794,5795,5796,5797,5798,5799,5800,5801,5802,5803,5804,5805,5806,5807,5808,5809,5810,5811,5812,5813,5814,5815,5816,5817,5818,5819,5820,5821,5822,5823,5824,5825,5826,5827,5828,5829,5830,5831,5832,5833,5834,5835,5836,5837,5838,5839,5840,5841,5842,5843,5844,5845,5846,5847,5848,5849,5850,5851,5852,5853,5854,5855,5856,5857,5858,5859,5860,5861,5862,5863,5864,5865,5866,5867,5868,5869,5870,5871,5872,5873,5874,5875,5876,5877,5878,5879,5880,5881,5882,5883,5884,5885,5886,5887,5888,5889,5890,5891,5892,5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904,5905,5906,5907,5908,5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920,5921,5922,5923,5924,5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,5936,5937,5938,5939,5940,5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,5952,5953,5954,5955,5956,5957,5958,5959,5960,5961,5962,5963,5964,5965,5966,5967,5968,5969,5970,5971,5972,5973,5974,5975,5976,5977,5978,5979,5980,5981,5982,5983,5984,5985,5986,5987,5988,5989,5990,5991,5992,5993,5994,5995,5996,5997,5998,5999,6000,6001,6002,6003,6004,6005,6006,6007,6008,6009,6010,6011,6012,6013,6014,6015,6016,6017,6018,6019,6020,6021,6022,6023,6024,6025,6026,6027,6028,6029,6030,6031,6032,6033,6034,6035,6036,6037,6038,6039,6040,6041,6042,6043,6044,6045,6046,6047,6048,6049,6050,6051,6052,6053,6054,6055,6056,6057,6058,6059,6060,6061,6062,6063,6064,6065,6066,6067,6068,6069,6070,6071,6072,6073,6074,6075,6076,6077,6078,6079,6080,6081,6082,6083,6084,6085,6086,6087,6088,6089,6090,6091,6092,6093,6094,6095,6096,6097,6098,6099,6100,6101,6102,6103,6104,6105,6106,6107,6108,6109,6110,6111,6112,6113,6114,6115,6116,6117,6118,6119,6120,6121,6122,6123,6124,6125,6126,6127,6128,6129,6130,6131,6132,6133,6134,6135,6136,6137,6138,6139,6140,6141,6142,6143,6144,6145,6146,6147,6148,6149,6150,6151,6152,6153,6154,6155,6156,6157,6158,6159,6160,6161,6162,6163,6164,6165,6166,6167,6168,6169,6170,6171,6172,6173,6174,6175,6176,6177,6178,6179,6180,6181,6182,6183,6184,6185,6186,6187,6188,6189,6190,6191,6192,6193,6194,6195,6196,6197,6198,6199,6200,6201,6202,6203,6204,6205,6206,6207,6208,6209,6210,6211,6212,6213,6214,6215,6216,6217,6218,6219,6220,6221,6222,6223,6224,6225,6226,6227,6228,6229,6230,6231,6232,6233,6234,6235,6236,6237,6238,6239,6240,6241,6242,6243,6244,6245,6246,6247,6248,6249,6250,6251,6252,6253,6254,6255,6256,6257,6258,6259,6260,6261,6262,6263,6264,6265,6266,6267,6268,6269,6270,6271,6272,6273,6274,6275,6276,6277,6278,6279,6280,6281,6282,6283,6284,6285,6286,6287,6288,6289,6290,6291,6292,6293,6294,6295,6296,6297,6298,6299,6300,6301,6302,6303,6304,6305,6306,6307,6308,6309,6310,6311,6312,6313,6314,6315,6316,6317,6318,6319,6320,6321,6322,6323,6324,6325,6326,6327,6328,6329,6330,6331,6332,6333,6334,6335,6336,6337,6338,6339,6340,6341,6342,6343,6344,6345,6346,6347,6348,6349,6350,6351,6352,6353,6354,6355,6356,6357,6358,6359,6360,6361,6362,6363,6364,6365,6366,6367,6368,6369,6370,6371,6372,6373,6374,6375,6376,6377,6378,6379,6380,6381,6382,6383,6384,6385,6386,6387,6388,6389,6390,6391,6392,6393,6394,6395,6396,6397,6398,6399,6400,6401,6402,6403,6404,6405,6406,6407,6408,6409,6410,6411,6412,6413,6414,6415,6416,6417,6418,6419,6420,6421,6422,6423,6424,6425,6426,6427,6428,6429,6430,6431,6432,6433,6434,6435,6436,6437,6438,6439,6440,6441,6442,6443,6444,6445,6446,6447,6448,6449,6450,6451,6452,6453,6454,6455,6456,6457,6458,6459,6460,6461,6462,6463,6464,6465,6466,6467,6468,6469,6470,6471,6472,6473,6474,6475,6476,6477,6478,6479,6480,6481,6482,6483,6484,6485,6486,6487,6488,6489,6490,6491,6492,6493,6494,6495,6496,6497,6498,6499,6500,6501,6502,6503,6504,6505,6506,6507,6508,6509,6510,6511,6512,6513,6514,6515,6516,6517,6518,6519,6520,6521,6522,6523,6524,6525,6526,6527,6528,6529,6530,6531,6532,6533,6534,6535,6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548,6549,6550,6551,6552,6553,6554,6555,6556,6557,6558,6559,6560,6561,6562,6563,6564,6565,6566,6567,6568,6569,6570,6571,6572,6573,6574,6575,6576,6577,6578,6579,6580,6581,6582,6583,6584,6585,6586,6587,6588,6589,6590,6591,6592,6593,6594,6595,6596,6597,6598,6599,6600,6601,6602,6603,6604,6605,6606,6607,6608,6609,6610,6611,6612,6613,6614,6615,6616,6617,6618,6619,6620,6621,6622,6623,6624,6625,6626,6627,6628,6629,6630,6631,6632,6633,6634,6635,6636,6637,6638,6639,6640,6641,6642,6643,6644,6645,6646,6647,6648,6649,6650,6651,6652,6653,6654,6655,6656,6657,6658,6659,6660,6661,6662,6663,6664,6665,6666,6667,6668,6669,6670,6671,6672,6673,6674,6675,6676,6677,6678,6679,6680,6681,6682,6683,6684,6685,6686,6687,6688,6689,6690,6691,6692,6693,6694,6695,6696,6697,6698,6699,6700,6701,6702,6703,6704,6705,6706,6707,6708,6709,6710,6711,6712,6713,6714,6715,6716,6717,6718,6719,6720,6721,6722,6723,6724,6725,6726,6727,6728,6729,6730,6731,6732,6733,6734,6735,6736,6737,6738,6739,6740,6741,6742,6743,6744,6745,6746,6747,6748,6749,6750,6751,6752,6753,6754,6755,6756,6757,6758,6759,6760,6761,6762,6763,6764,6765,6766,6767,6768,6769,6770,6771,6772,6773,6774,6775,6776,6777,6778,6779,6780,6781,6782,6783,6784,6785,6786,6787,6788,6789,6790,6791,6792,6793,6794,6795,6796,6797,6798,6799,6800,6801,6802,6803,6804,6805,6806,6807,6808,6809,6810,6811,6812,6813,6814,6815,6816,6817,6818,6819,6820,6821,6822,6823,6824,6825,6826,6827,6828,6829,6830,6831,6832,6833,6834,6835,6836,6837,6838,6839,6840,6841,6842,6843,6844,6845,6846,6847,6848,6849,6850,6851,6852,6853,6854,6855,6856,6857,6858,6859,6860,6861,6862,6863,6864,6865,6866,6867,6868,6869,6870,6871,6872,6873,6874,6875,6876,6877,6878,6879,6880,6881,6882,6883,6884,6885,6886,6887,6888,6889,6890,6891,6892,6893,6894,6895,6896,6897,6898,6899,6900,6901,6902,6903,6904,6905,6906,6907,6908,6909,6910,6911,6912,6913,6914,6915,6916,6917,6918,6919,6920,6921,6922,6923,6924,6925,6926,6927,6928,6929,6930,6931,6932,6933,6934,6935,6936,6937,6938,6939,6940,6941,6942,6943,6944,6945,6946,6947,6948,6949,6950,6951,6952,6953,6954,6955,6956,6957,6958,6959,6960,6961,6962,6963,6964,6965,6966,6967,6968,6969,6970,6971,6972,6973,6974,6975,6976,6977,6978,6979,6980,6981,6982,6983,6984,6985,6986,6987,6988,6989,6990,6991,6992,6993,6994,6995,6996,6997,6998,6999,7000,7001,7002,7003,7004,7005,7006,7007,7008,7009,7010,7011,7012,7013,7014,7015,7016,7017,7018,7019,7020,7021,7022,7023,7024,7025,7026,7027,7028,7029,7030,7031,7032,7033,7034,7035,7036,7037,7038,7039,7040,7041,7042,7043,7044,7045,7046,7047,7048,7049,7050,7051,7052,7053,7054,7055,7056,7057,7058,7059,7060,7061,7062,7063,7064,7065,7066,7067,7068,7069,7070,7071,7072,7073,7074,7075,7076,7077,7078,7079,7080,7081,7082,7083,7084,7085,7086,7087,7088,7089,7090,7091,7092,7093,7094,7095,7096,7097,7098,7099,7100,7101,7102,7103,7104,7105,7106,7107,7108,7109,7110,7111,7112,7113,7114,7115,7116,7117,7118,7119,7120,7121,7122,7123,7124,7125,7126,7127,7128,7129,7130,7131,7132,7133,7134,7135,7136,7137,7138,7139,7140,7141,7142,7143,7144,7145,7146,7147,7148,7149,7150,7151,7152,7153,7154,7155,7156,7157,7158,7159,7160,7161,7162,7163,7164,7165,7166,7167,7168,7169,7170,7171,7172,7173,7174,7175,7176,7177,7178,7179,7180,7181,7182,7183,7184,7185,7186,7187,7188,7189,7190,7191,7192,7193,7194,7195,7196,7197,7198,7199,7200,7201,7202,7203,7204,7205,7206,7207,7208,7209,7210,7211,7212,7213,7214,7215,7216,7217,7218,7219,7220,7221,7222,7223,7224,7225,7226,7227,7228,7229,7230,7231,7232,7233,7234,7235,7236,7237,7238,7239,7240,7241,7242,7243,7244,7245,7246,7247,7248,7249,7250,7251,7252,7253,7254,7255,7256,7257,7258,7259,7260,7261,7262,7263,7264,7265,7266,7267,7268,7269,7270,7271,7272,7273,7274,7275,7276,7277,7278,7279,7280,7281,7282,7283,7284,7285,7286,7287,7288,7289,7290,7291,7292,7293,7294,7295,7296,7297,7298,7299,7300,7301,7302,7303,7304,7305,7306,7307,7308,7309,7310,7311,7312,7313,7314,7315,7316,7317,7318,7319,7320,7321,7322,7323,7324,7325,7326,7327,7328,7329,7330,7331,7332,7333,7334,7335,7336,7337,7338,7339,7340,7341,7342,7343,7344,7345,7346,7347,7348,7349,7350,7351,7352,7353,7354,7355,7356,7357,7358,7359,7360,7361,7362,7363,7364,7365,7366,7367,7368,7369,7370,7371,7372,7373,7374,7375,7376,7377,7378,7379,7380,7381,7382,7383,7384,7385,7386,7387,7388,7389,7390,7391,7392,7393,7394,7395,7396,7397,7398,7399,7400,7401,7402,7403,7404,7405,7406,7407,7408,7409,7410,7411,7412,7413,7414,7415,7416,7417,7418,7419,7420,7421,7422,7423,7424,7425,7426,7427,7428,7429,7430,7431,7432,7433,7434,7435,7436,7437,7438,7439,7440,7441,7442,7443,7444,7445,7446,7447,7448,7449,7450,7451,7452,7453,7454,7455,7456,7457,7458,7459,7460,7461,7462,7463,7464,7465,7466,7467,7468,7469,7470,7471,7472,7473,7474,7475,7476,7477,7478,7479,7480,7481,7482,7483,7484,7485,7486,7487,7488,7489,7490,7491,7492,7493,7494,7495,7496,7497,7498,7499,7500,7501,7502,7503,7504,7505,7506,7507,7508,7509,7510,7511,7512,7513,7514,7515,7516,7517,7518,7519,7520,7521,7522,7523,7524,7525,7526,7527,7528,7529,7530,7531,7532,7533,7534,7535,7536,7537,7538,7539,7540,7541,7542,7543,7544,7545,7546,7547,7548,7549,7550,7551,7552,7553,7554,7555,7556,7557,7558,7559,7560,7561,7562,7563,7564,7565,7566,7567,7568,7569,7570,7571,7572,7573,7574,7575,7576,7577,7578,7579,7580,7581,7582,7583,7584,7585,7586,7587,7588,7589,7590,7591,7592,7593,7594,7595,7596,7597,7598,7599,7600,7601,7602,7603,7604,7605,7606,7607,7608,7609,7610,7611,7612,7613,7614,7615,7616,7617,7618,7619,7620,7621,7622,7623,7624,7625,7626,7627,7628,7629,7630,7631,7632,7633,7634,7635,7636,7637,7638,7639,7640,7641,7642,7643,7644,7645,7646,7647,7648,7649,7650,7651,7652,7653,7654,7655,7656,7657,7658,7659,7660,7661,7662,7663,7664,7665,7666,7667,7668,7669,7670,7671,7672,7673,7674,7675,7676,7677,7678,7679,7680,7681,7682,7683,7684,7685,7686,7687,7688,7689,7690,7691,7692,7693,7694,7695,7696,7697,7698,7699,7700,7701,7702,7703,7704,7705,7706,7707,7708,7709,7710,7711,7712,7713,7714,7715,7716,7717,7718,7719,7720,7721,7722,7723,7724,7725,7726,7727,7728,7729,7730,7731,7732,7733,7734,7735,7736,7737,7738,7739,7740,7741,7742,7743,7744,7745,7746,7747,7748,7749,7750,7751,7752,7753,7754,7755,7756,7757,7758,7759,7760,7761,7762,7763,7764,7765,7766,7767,7768,7769,7770,7771,7772,7773,7774,7775,7776,7777,7778,7779,7780,7781,7782,7783,7784,7785,7786,7787,7788,7789,7790,7791,7792,7793,7794,7795,7796,7797,7798,7799,7800,7801,7802,7803,7804,7805,7806,7807,7808,7809,7810,7811,7812,7813,7814,7815],\"y\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,0,0,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,0,0,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,1,0,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,1,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,0,0,0,1,0,0,1,0,0,0,1,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,0,1,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,1,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"hoverlabel\":{\"bgcolor\":\"rgba(0, 0, 0, 0.8)\"},\"title\":{\"text\":\"Success rate = 56.32%\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('e469a4a2-cf3a-458c-b791-4365d7724363');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "environment = gym.make(\"FrozenLake-v1\", is_slippery=True, render_mode=\"ansi\")\n",
        "environment.reset()\n",
        "\n",
        "qtable = np.zeros((environment.observation_space.n,\n",
        "                  environment.action_space.n))\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "episodes = 5000\n",
        "alpha = 0.5\n",
        "gamma = 0.9\n",
        "epsilon = 1.0\n",
        "epsilon_decay = 0.001\n",
        "\n",
        "outcomes = []\n",
        "\n",
        "print('Q-table before training:')\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "# Training\n",
        "for _ in range(episodes):\n",
        "    state, _ = environment.reset()\n",
        "    done = False\n",
        "\n",
        "    outcomes.append(0)\n",
        "\n",
        "    while not done:\n",
        "        rnd = np.random.random()\n",
        "\n",
        "        if rnd < epsilon:\n",
        "            action = environment.action_space.sample()\n",
        "        else:\n",
        "            action = np.argmax(qtable[state])\n",
        "\n",
        "        new_state, reward, done, info, prob = environment.step(action)\n",
        "\n",
        "        qtable[state, action] = qtable[state, action] + alpha * \\\n",
        "            (reward + gamma *\n",
        "             np.max(qtable[new_state]) - qtable[state, action])\n",
        "\n",
        "        state = new_state\n",
        "\n",
        "        if reward:\n",
        "            outcomes.append(1)\n",
        "\n",
        "    epsilon = max(epsilon - epsilon_decay, 0)\n",
        "\n",
        "df = pd.DataFrame(qtable, index=[f'state {i+1}' for i in range(16)],\n",
        "                  columns=['◀️LEFT', '🔽DOWN', '▶️RIGHT', '🔼UP'])\n",
        "\n",
        "print('Q-table after training:')\n",
        "display(Markdown(df.to_markdown()))\n",
        "\n",
        "fig = go.Figure()\n",
        "fig.add_trace(go.Bar(x=list(range(0, len(outcomes))), y=outcomes, marker_color='red'))\n",
        "fig.update_layout(\n",
        "    template='plotly_dark',\n",
        "    hoverlabel=dict(\n",
        "        bgcolor='rgba(0, 0, 0, 0.8)'\n",
        "    ),\n",
        "    title=f'Success rate = {(sum(outcomes)/episodes)*100}%',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)'\n",
        ")\n",
        "fig.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2Gr3z1MtM_mk"
      },
      "source": [
        "Let us test our new agent in this slippery environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3hkSM76zM_mk",
        "outputId": "7e9250b1-17a6-49a9-9913-8d7080aa086c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  (Left)\n",
            "SFFF\n",
            "FHFH\n",
            "FFFH\n",
            "HF\u001b[41mF\u001b[0mG\n",
            "\n",
            "Success rate = 70.0%\n"
          ]
        }
      ],
      "source": [
        "episodes = 10\n",
        "success_rate = 0\n",
        "\n",
        "# Evaluation\n",
        "for _ in range(episodes):\n",
        "    state,_ = environment.reset()\n",
        "    done = False\n",
        "\n",
        "    while not done:\n",
        "        print(environment.render())\n",
        "        if np.max(qtable[state]) > 0:\n",
        "            action = np.argmax(qtable[state])\n",
        "\n",
        "        else:\n",
        "            action = environment.action_space.sample()\n",
        "\n",
        "        new_state, reward, done, info, prob = environment.step(action)\n",
        "\n",
        "        time.sleep(0.1)\n",
        "        if done == False:\n",
        "            clear_output(wait=True)\n",
        "\n",
        "        state = new_state\n",
        "\n",
        "        success_rate += reward\n",
        "\n",
        "# Let's check our success rate!\n",
        "print(f'Success rate = {(success_rate/episodes)*100}%')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SRczFlAFM_mk"
      },
      "source": [
        "You can try to tweak the hyperparameters to get a better result. Since we are dealing with an extremely stochastic environment, decreasing the initial epsilon value (e.g., 0.7) might be better. However, one problem RL faces is that the final result of an agent is _extremely sensitive to the choice of hyperparameters_, something that makes RL more difficult to get right when compared to, for example, vanilla supervised learning.\n",
        "\n",
        "---\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.13 64-bit",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "aca09746cf57686f00a55ae76e987247ecfb5dd0b3b2e2474d8dbbf0c5e3377e"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}