3_Multi-Armed_Bandit.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>.container { width:90% !important; }</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "display(HTML(\"<style>.container { width:90% !important; }</style>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.- Multi Armed Bandit (MAB)\n",
    "\n",
    "\n",
    "* El problema del \"***Bandido Multi-Brazo***\" es un problema clásico del aprendizaje por refuerzo basado en el ***juego de las máquinas \"tragaperras\"*** donde se \"tira del brazo\" (palanca) de la \"tragaperras\" y se obtiene una recompensa por el juego, positiva si se gana dinero o negativa si perdemos el dinero.\n",
    "\n",
    "\n",
    "* A este problema se le conoce como el problema del \"Bandido Multi-Brazo\" al denominarse una máquina \"tragaperras\" como \"Bandido de un solo brazo\". Cuando se juega a varias máquinas \"tragaperras\" se denomina \"Bandido de Multiples Brazos\", de ahí el nombre de \"Multi Armed Bandit\" (MAB):\n",
    "\n",
    "\n",
    "<img src=\"./imgs/015_RL.png\" style=\"width: 700px;\"/>\n",
    "\n",
    "\n",
    "* El objetivo de este problema es el de ***jugar un 'P' partidas a las 'K' \"tragaperras\" y obtener el mayor beneficio posible*** (la mayor recompensa posible).\n",
    "\n",
    "\n",
    "* Para ello tendremos que ir jugando partidas en las 'K' \"tragaperras\" y descubrir cual es la distribución de probabilidad de obtener premio en cada una de las \"tragaperras\". Con esto conseguiremos saber a ***que máquinas jugar y en que orden para máximizar nuestros beneficios (o recompensas)***.\n",
    "\n",
    "\n",
    "\n",
    "* Para resolver este problema definimos el $Q(a)$ como la ***recompensa media (beneficios medios) recibida por partida en la \"tragaperras\"*** $a$ y se calcula como:\n",
    "\n",
    "\n",
    "$$Q(a) = \\frac{R_a}{N_a}$$\n",
    "\n",
    "\n",
    "* Siendo:\n",
    "    + $R_a$: Suma de las recompensas (beneficiós) obtenidos en la \"tragaperras\" $a$.\n",
    "    + $N_a$: Número total de partidas jugadas en la \"tragaperras\" $a$.\n",
    "    \n",
    "    \n",
    "* El ***objetivo es encontrar la máquina \"tragaperras\" que mayor beneficio (recompensa) dé***:\n",
    "\n",
    "$$Q(a^{*}) = max Q(a)$$\n",
    "\n",
    "\n",
    "* Siendo:\n",
    "    + $a^{*}$: El conjunto de las 'K' \"tragaperras\" a las que se juega.\n",
    "    \n",
    "    \n",
    "* En este problema es muy importante aplicar correctamente los conceptos de **\"Explotación\" y \"Exploración\"** ya que si solo nos dedicamos a \"*Explorar*\" obtendremos el valor medio de recompensas que dén las 'K' \"tragaperras\" y si solo nos dedicamos a \"*Explotar*\" obtendremos la recompensa que nos dé la primera \"tragaperras\" a la que jugemos.\n",
    "\n",
    "\n",
    "* Para ello debemos de seguir una política conocida como \"***epsilon-greedy policy***\" la cual seleccionará una \"tragaperras\" al azar con probabilidad $\\epsilon$ para jugar y *explorar* o seleccionará la mejor \"tragaperras\" con probabilidad $1-\\epsilon$.\n",
    "\n",
    "\n",
    "<img src=\"./imgs/016_RL.png\" style=\"width: 300px;\"/>\n",
    "\n",
    "\n",
    "* Veamos a continuación un ejemplo:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "def gen_bandits():\n",
    "    \"\"\"\n",
    "    Función que devuelve una lista de probabilidades ordenadas al azar\n",
    "    \"\"\"\n",
    "    bandits = [0.1, 0.1, 0.1, 0.2, 0.6]\n",
    "    random.shuffle(bandits)\n",
    "    return bandits\n",
    "    \n",
    "    \n",
    "def multi_armed_bandit(num_games=1000, epsilon=0.1, verbose=False):\n",
    "    \n",
    "    bandits = gen_bandits()\n",
    "    total_reward = 0\n",
    "    acum_reward_bandit = np.zeros(len(bandits))  # numerador\n",
    "    num_selected_bandit = np.zeros(len(bandits)) # denominador\n",
    "    q_bandits = np.zeros(len(bandits))            # Q(a)\n",
    "    \n",
    "    if verbose:\n",
    "        print(\"Initial Bandits Distribution\\n  {}\".format(bandits))\n",
    "    \n",
    "    for game in range(0,num_games):\n",
    "        \n",
    "        # Hago una copiar de los Q(a)\n",
    "        old_q_bandits = q_bandits.copy()\n",
    "        \n",
    "        # Selecciono \"tragaperras\" a la que jugar\n",
    "        if np.random.random() < epsilon:\n",
    "            bandit = np.random.randint(len(bandits)) # Exploro\n",
    "        else:\n",
    "            bandit = np.random.choice(np.flatnonzero(q_bandits == q_bandits.max())) # Exploto \n",
    "            \n",
    "        # Obtengo el reward\n",
    "        reward = 1 if (np.random.random() < bandits[bandit]) else 0\n",
    "        \n",
    "        # Actualizo reward total\n",
    "        total_reward += reward\n",
    "        \n",
    "        # Actualizo valor (Q) de la \"tragaperras\"\n",
    "        acum_reward_bandit[bandit] += reward\n",
    "        num_selected_bandit[bandit] += 1\n",
    "        q_bandits[bandit] = acum_reward_bandit[bandit] / num_selected_bandit[bandit]\n",
    "        \n",
    "        if verbose:\n",
    "            print(\"\\nGAME {game}\\n  Old Q_Bandits = {old_q_bandits}\\n  Selected Bandit = {bandit} \\\n",
    "                  \\n  Reward = {reward}\\n  Q_Bandits = {q_bandits}\"\n",
    "                  .format(game=game+1, old_q_bandits=old_q_bandits, bandit=bandit, \n",
    "                          reward=reward, q_bandits=q_bandits))\n",
    "    \n",
    "    return bandits, total_reward, q_bandits, num_selected_bandit\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ratio de Exploración del 10%\n",
    "\n",
    "\n",
    "* Realizamos una ejecución con un ratio de exploración de un 10% ($\\epsilon = 0.1$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial Bandits Distribution\n",
      "  [0.1, 0.1, 0.2, 0.1, 0.6]\n",
      "Bandits Distribution games\n",
      "  [ 26.  22.  29.  20. 903.]\n",
      "Q(a) Bandits\n",
      "  [0.1538 0.1364 0.2414 0.1    0.6124]\n",
      "Reward: 569\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYgklEQVR4nO3dfbRddX3n8ffHBHlGQALF8BDUCEJbRFNAxDqKCmgrWJ9wlAJF6bQoODoqdFxjbVcsM62toxUVBUVlZLIsVKpWQQQqrYLhSYXAmBKEQISAKA/SCOE7f5x9N4fLvTc7Ieec5N73a62zztm//dt7f7fB+zn76XdSVUiSBPCUURcgSdpwGAqSpJahIElqGQqSpJahIElqGQqSpJahoI1SkrckubBvupI8e5Q1DcJ03S9tuAwFDVySW5I8lOSBvtffP5l1VtU5VfXK9VWjepJsmuSsJPcl+VmSd0/Rd+ckFyS5owmveUMsVQMye9QFaMb4/ar69qiLGKQkAVJVj466lifhz4H5wO7AbwCXJLmhqr45Qd9HgW8CfwX829Aq1EB5pKCRSnJskn9N8vEkv0xyY5JDxs2/Ocn9SZYleUtf++WTrPNpSb6QZGWSnyb5QJKn9C+X5G+S3Nus8/Ap6puV5CNJ7m76vqP5Vjy7mX9pkoVJ/hX4FfDMJHsluSjJz5PclOSNfevbtNn2rUnuTPKpJJv3zX9vkhXNt+8/6mv/nab/7L621yW5dh3+Z5/KHwJ/WVX3VtUS4DPAsRN1rKo7q+p04AfruQaNkKGgDcEBwM3ADsAHgfOSbJ9kS+BjwOFVtTVwEHBth/V9HHga8EzgJfT+0B03bns3Ndv7X8CZzbf8ibwdOBx4HvB84MgJ+hwNnABsDawELgL+D7Aj8Gbg9CT7NH3/J/CcZn3PBuYC/wMgyWHAfwNeQe/b+svHNlBVPwDuaeaNeSvwxYmKTnJKkl9M9ppkme2AZwDX9TVfB+wzUX9NU1Xly9dAX8AtwAPAL/peb2/mHQvcQe+0y1j/K+n9od2y6fs6YPNx6zwWuLxvuuj9kZ0FrAL27pv3x8Clfcst7Zu3RbPsb0xS+3eAP+6bfnnTf3YzfSnwF33z3wR8d9w6Pk0v7AI8CDyrb94LgWXN57OA0/rmPWdsv5rp9wPnNJ+3p3dksvN6/HfatdneZn1trwBuWcNys5vl5o36vzVfT/7lNQUNy5E1+TWF26v569L4KfCMqnowyZvofXs+szlF856qunGK7ewAPLVZR//65vZN/2zsQ1X9qjlI2CrJi4F/Hlumqvah9835tr5l+z9P1LY7cMC4b+Oz6X2jn0MvhK7qOzAJvSCj2dZV4+ru9yVgSZKtgDfSC58VE9Szrh5o3rcB/qPv8/3rcRvawHn6SBuCueNO3+xG7+iBqvpWVb0C2Bm4kd457qncDTxM749z//puX1MRVfXdqtqqeY2dMlkB7NLXbdeJFu37fBtwWVVt2/faqqr+pKntIWCfvnlPq6qt+rbVv/7dxtV3O/A94LX0jqQmPHUEkOTPxt3t9bjXJPt/b1PDvn3N+wLXT7YdTT+GgjYEOwInJdkkyRuA5wLfSLJTktc01xZW0fsmu3qqFVXVamARsDDJ1kl2B95N71v2ulgEnJxkbpJt6Z3CmcrXgOckObrZn02ai8TPrd5dSZ8B/i7JjgDNeg/t29axSfZOsgW9U07jfQF4H/BbwPmTFVFVH+4LuCe8pqj/C8AHkmyXZC9611Q+PzYzvduLj+2b3gzYtJnctJnWRsxQ0LD807hvq/1/0K6gd2H1bmAh8Pqquofef5/voXfU8HN6F43/tMO23knv3P3NwOX0LvqetY51fwa4EPghcA3wDeARJgmnqrofeCVwVFP3z+hdXB77w/l+YCnw/ST3Ad8G9myW/Wfgo/SuYyxt3sc7n95R0PlV9eA67tNUPgj8O71TV5cBf13N7ahJngo8Hfh+X/+HeOy0043NtDZiefypXGm4mm+db6uqg0ddSxfN7aufqqrd19h5cDX8O72L30N97iPJwcCJVfXmYW5Xw+WRgjSFJJsneVWS2Unm0vsmPelpmyHU8zp61zAmOooYqKq63ECY/rz7SJpagA8B/5feqZGv0zxXMPRCkkuBvYGja+N+alobME8fSZJanj6SJLU26tNHO+ywQ82bN2/UZUjSRuWqq666u6rmTDRvow6FefPmsXjx4lGXIUkblSTjn5ZvefpIktQyFCRJLUNBktQyFCRJLUNBktQyFCRJLUNBktQyFCRJLUNBktTaqJ9olqS1Me+Ur4+6hPXmltNePZD1eqQgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoNNBSS/Nck1yf5cZIvJ9ksyfZJLkryk+Z9u77+pyZZmuSmJIcOsjZJ0hMNLBSSzAVOAhZU1W8Cs4CjgFOAi6tqPnBxM02SvZv5+wCHAacnmTWo+iRJTzTo00ezgc2TzAa2AO4AjgDObuafDRzZfD4COLeqVlXVMmApsP+A65Mk9RlYKFTV7cDfALcCK4BfVtWFwE5VtaLpswLYsVlkLnBb3yqWN22Pk+SEJIuTLF65cuWgypekGWmQp4+2o/ftfw/gGcCWSd461SITtNUTGqrOqKoFVbVgzpw566dYSRIw2NNHLweWVdXKqnoYOA84CLgzyc4AzftdTf/lwK59y+9C73STJGlIBhkKtwIHJtkiSYBDgCXABcAxTZ9jgK82ny8AjkqyaZI9gPnAlQOsT5I0zuxBrbiqrkjyFeBq4BHgGuAMYCtgUZLj6QXHG5r+1ydZBNzQ9D+xqlYPqj5J0hMNLBQAquqDwAfHNa+id9QwUf+FwMJB1iRJmpxPNEuSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKk1e00dkmwC/Anwu03TZcCnqurhQRYmSRq+NYYC8ElgE+D0Zvropu1tgypKkjQaXULhd6pq377p7yS5blAFSZJGp8s1hdVJnjU2keSZwOrBlSRJGpUuRwrvBS5JcjMQYHfguIFWJUkaiTWGQlVdnGQ+sCe9ULixqlYNvDJJ0tBNGgpJ/mCSWc9KQlWdN6CaJEkjMtWRwu837zsCBwEX0ztSeClwKWAoSNI0M2koVNVxAEm+BuxdVSua6Z2BTwynPEnSMHW5+2jeWCA07gSeM6B6JEkj1OXuo0uTfAv4MlDAUcAlA61KkjQSXe4+ekdz0fnFTdMZVXX+YMuSJI1ClyOFsTuNvLAsSdPcpNcUklzevN+f5L6+1/1J7uuy8iTbJvlKkhuTLEnywiTbJ7koyU+a9+36+p+aZGmSm5Ic+uR3T5K0NiYNhao6uHnfuqq26XttXVXbdFz//wa+WVV7AfsCS4BTgIuraj6921xPAUiyN73rFfsAhwGnJ5m1rjsmSVp7a7z7qPlmP/61SYfltqE33PaZAFX166r6BXAEcHbT7WzgyObzEcC5VbWqqpYBS4H913aHJEnrrsstqVcDK4H/B/yk+bwsydVJXjDFcs9s+n4uyTVJPptkS2CnsVtcm/cdm/5zgdv6ll/etD1OkhOSLE6yeOXKlR3KlyR11SUUvgm8qqp2qKqnA4cDi4A/5bHfWJjIbOD5wCeraj/gQZpTRZPIBG31hIaqM6pqQVUtmDNnTofyJUlddQmFBVX1rbGJqroQ+N2q+j6w6RTLLQeWV9UVzfRX6IXEnc1T0WNPR9/V13/XvuV3Ae7otBeSpPWiSyj8PMn7k+zevN4H3NtcBH50soWq6mfAbUn2bJoOAW4ALgCOadqOAb7afL4AOCrJpkn2AOYDV679LkmS1lWX5xT+M/BB4B+b6cubtlnAG9ew7DuBc5I8FbiZ3u8wPAVYlOR44FbgDQBVdX2SRfSC4xHgxKryx3wkaYi6PNF8N/DOJFtV1QPjZi9dw7LXAgsmmHXIJP0XAgvXVJMkaTC63JJ6UJIb6H2DJ8m+Saa6wCxJ2kh1uabwd8ChwD0AVXUdvecPJEnTTJdQoKpuG9fkuX5Jmoa6XGi+LclBQDUXjE+iN1yFJGma6XKk8F+AE+k9XbwceF4zLUmaZrreffSWIdQiSRqxSUMhyceZYJiJMVV10kAqkiSNzFRHCouHVoUkaYMwaShU1dn900m2rKoHB1+SJGlUujy89sLm4bUlzbQPr0nSNNXl7qOP4sNrkjQj+PCaJKnlw2uSpJYPr0mSWj68JklqdbqmIEmaGQwFSVLLUJAktbo8vHZykm3Sc2aSq5O8chjFSZKGq8uRwh9V1X3AK4E5wHHAaQOtSpI0El1CIc37q4DPNU80Z4r+kqSNVJdQuCrJhfRC4VtJtgYeHWxZkqRR6PJE8/H0Hli7uap+leTp9E4hSZKmmS5HCgXsTW94C4Atgc0GVpEkaWS6hMLpwAuBNzfT9wOfGFhFkqSR6XL66ICqen6SawCq6t5mYDxJ0jTT5Ujh4SSzaH6vOckcvNAsSdNSl1D4GHA+sGOShcDlwIcHWpUkaSS6jJJ6TpKrgEPoPZ9wZFX5ewqSNA2tMRSSHAhcX1WfaKa3TnJAVV0x8OokSUPV5fTRJ4EH+qYfbNokSdNMp2EuqqrGJqrqUbrdtSRJ2sh0CYWbk5yUZJPmdTJw86ALkyQNX9ffaD4IuJ3ebzQfAJwwyKIkSaPR5e6ju4CjhlCLJGnEuvzIznOSXJzkx830byf5wOBLkyQNW5fTR58BTgUeBqiqH+KRgyRNS11CYYuqunJc2yNdN5BkVpJrknytmd4+yUVJftK8b9fX99QkS5PclOTQrtuQJK0fXULh7iTP4rGxj14PrFiLbZwM9D8BfQpwcVXNBy5upkmyN70jkH2Aw4DTmzGXJElD0iUUTgQ+DeyV5HbgXfTuSFqjJLsArwY+29d8BHB28/ls4Mi+9nOralVVLQOWAvt32Y4kaf3o8hDaT6vq5Um2BJ5SVfevxfo/CrwP2LqvbaeqWgFQVSuS7Ni0zwW+39dvedP2OElOoLkldrfddluLUiRJa9LlSGFZkjOAA3n8cBdTSvJ7wF1VdVXXRSZoqyc0VJ1RVQuqasGcOXO6liNJ6qBLKOwJfJveaaRlSf4+ycEdlnsR8JoktwDnAi9L8iXgziQ7AzTvdzX9lwO79i2/C3BHp72QJK0XawyFqnqoqhZV1R8A+wHbAJd1WO7UqtqlqubRu4D8nap6K3ABcEzT7Rjgq83nC4CjkmyaZA9gPjD+ridJ0gB1GtguyUuANwGHAz8A3vgktnkasCjJ8cCtwBsAqur6JIuAG+jd8npiVa1+EtuRJK2lLr+nsAy4FlgEvLeqHlzbjVTVpcClzed76P1gz0T9FgIL13b9kqT1o8uRwr5Vdd/AK5EkjVyXawoGgiTNEF3uPpIkzRCGgiSp1eVC87bAHwLz+vtX1UkDq0qSNBJdLjR/g97wEz8CHh1sOZKkUeoSCptV1bsHXokkaeS6XFP4YpK3J9m5+S2E7ZNsP/DKJElD1+VI4dfAXwP/nccGqCvgmYMqSpI0Gl1C4d3As6vq7kEXI0karS6nj64HfjXoQiRJo9flSGE1cG2SS4BVY43ekipJ00+XUPjH5iVJmubWGApVdfaa+kiSpoeuQ2dP9LOY3n0kSdNMl9NHC/o+b0bvR3F8TkGSpqEuQ2ff0/e6vao+Crxs8KVJkoaty+mj5/dNPoXekcPWA6tIkjQyXU4ffaTv8yPALTy532iWJG2gutx99NJhFCJJGr0up482BV7HE39P4S8GV5YkaRS6nD76KvBL4Cr6nmiWJE0/XUJhl6o6bOCVSJJGrsuAeP+W5LcGXokkaeS6HCkcDBzbPNm8CghQVfXbA61MkjR0XULh8IFXIUnaIHS5JfWnwyhEkjR6Xa4pSJJmCENBktQyFCRJLUNBktQyFCRJLUNBktQyFCRJLUNBktQyFCRJLUNBktQaWCgk2TXJJUmWJLk+yclN+/ZJLkryk+Z9u75lTk2yNMlNSQ4dVG2SpIkN8kjhEeA9VfVc4EDgxCR7A6cAF1fVfODiZppm3lHAPsBhwOlJZg2wPknSOAMLhapaUVVXN5/vB5YAc4EjgLObbmcDRzafjwDOrapVVbUMWArsP6j6JElPNJRrCknmAfsBVwA7VdUK6AUHsGPTbS5wW99iy5u28es6IcniJItXrlw50LolaaYZeCgk2Qr4B+BdVXXfVF0naKsnNFSdUVULqmrBnDlz1leZkiQGHApJNqEXCOdU1XlN851Jdm7m7wzc1bQvB3btW3wX4I5B1idJerxB3n0U4ExgSVX9bd+sC4Bjms/HAF/taz8qyaZJ9gDmA1cOqj5J0hN1+TnOdfUi4GjgR0mubdr+DDgNWJTkeOBW4A0AVXV9kkXADfTuXDqxqlYPsD5J0jgDC4WqupyJrxMAHDLJMguBhYOqSZI0NZ9oliS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1DAVJUstQkCS1Zo+6AGnY5p3y9VGXsN7cctqrR12CppkZHQrT5Y+DfxgkrS8zOhRmsukSiGAorq3p8m/vv/tgeE1BktTa4EIhyWFJbkqyNMkpo65HkmaSDSoUkswCPgEcDuwNvDnJ3qOtSpJmjg0qFID9gaVVdXNV/Ro4FzhixDVJ0oyRqhp1Da0krwcOq6q3NdNHAwdU1Tv6+pwAnNBM7gncNPRC184OwN2jLmJEZvK+w8ze/5m877Dh7//uVTVnohkb2t1HmaDtcalVVWcAZwynnCcvyeKqWjDqOkZhJu87zOz9n8n7Dhv3/m9op4+WA7v2Te8C3DGiWiRpxtnQQuEHwPwkeyR5KnAUcMGIa5KkGWODOn1UVY8keQfwLWAWcFZVXT/isp6sjeZU1wDM5H2Hmb3/M3nfYSPe/w3qQrMkabQ2tNNHkqQRMhQkSS1DYUBm8nAdSc5KcleSH4+6lmFLsmuSS5IsSXJ9kpNHXdMwJdksyZVJrmv2/0OjrmnYksxKck2Sr426lnVhKAyAw3XweeCwURcxIo8A76mq5wIHAifOsH/7VcDLqmpf4HnAYUkOHG1JQ3cysGTURawrQ2EwZvRwHVX1L8DPR13HKFTViqq6uvl8P70/DnNHW9XwVM8DzeQmzWvG3M2SZBfg1cBnR13LujIUBmMucFvf9HJm0B8G9SSZB+wHXDHiUoaqOX1yLXAXcFFVzaT9/yjwPuDREdexzgyFwVjjcB2a3pJsBfwD8K6qum/U9QxTVa2uqufRG5Fg/yS/OeKShiLJ7wF3VdVVo67lyTAUBsPhOmawJJvQC4Rzquq8UdczKlX1C+BSZs71pRcBr0lyC71Txi9L8qXRlrT2DIXBcLiOGSpJgDOBJVX1t6OuZ9iSzEmybfN5c+DlwI0jLWpIqurUqtqlqubR+//8d6rqrSMua60ZCgNQVY8AY8N1LAEWTYPhOjpL8mXge8CeSZYnOX7UNQ3Ri4Cj6X1LvLZ5vWrURQ3RzsAlSX5I78vRRVW1Ud6aOVM5zIUkqeWRgiSpZShIklqGgiSpZShIklqGgiSpZShIjSSrm1tIr0tydZKD1tN6542NGJtkQZKPTdLvliQ7rI9tSutqg/o5TmnEHmqGZyDJocBfAS9ZnxuoqsXA4vW5Tml98khBmtg2wL3QG8coycXN0cOPkhzRtM9rfjfhM81vB1zYPMVLkhc0RxzfA04cW2mS/zQ2zn6SpzfLXJPk0/SNmZXk3Ul+3LzeNbzd1kxnKEiP2bw5fXQjvaGP/7Jp/w/gtVX1fOClwEea4SwA5gOfqKp9gF8Ar2vaPwecVFUvnGJ7HwQur6r96A2Dshv0AgU4DjiA3m8yvD3JfutpH6UpGQrSYx6qqudV1V70BnH7QvPHP8CHm6Ebvk1vGPSdmmWWVdW1zeergHlJngZsW1WXNe1fnGR7vwt8CaCqvk5zZAIcDJxfVQ82v01wHvDi9bWT0lS8piBNoKq+11z0nQO8qnl/QVU93IyCuVnTdVXfYquBzemFSNfxYybqN9HQ69JQeKQgTSDJXsAs4B7gafTGyX84yUuB3adathky+pdJDm6a3jJJ138Zm5fkcGC7vvYjk2yRZEvgtcB3n8TuSJ15pCA9ZvPmF8Og9239mKpaneQc4J+SLAaupdtQ0McBZyX5Fb3RcifyIeDLSa4GLgNuBaiqq5N8Hriy6ffZqrpmHfZHWmuOkipJann6SJLUMhQkSS1DQZLUMhQkSS1DQZLUMhQkSS1DQZLU+v/TVKf1SrNKHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "np.set_printoptions(precision=4)\n",
    "\n",
    "bandits, total_reward, q_bandits, num_selected_bandit = multi_armed_bandit(num_games=1000, \n",
    "                                                                           epsilon=0.1, \n",
    "                                                                           verbose=False)\n",
    "\n",
    "print(\"Initial Bandits Distribution\\n  {}\".format(bandits))\n",
    "print(\"Bandits Distribution games\\n  {}\".format(num_selected_bandit))\n",
    "print(\"Q(a) Bandits\\n  {}\".format(q_bandits))\n",
    "print(\"Reward: {}\".format(total_reward))\n",
    "\n",
    "# Mostramos en una gráfica de barras el número de veces que elegimos cada \"tragaperras\"\n",
    "plt.title('Epsilon-greedy = 0,1')\n",
    "plt.bar(range(0,len(bandits)), num_selected_bandit)\n",
    "plt.xlabel('Bandido')\n",
    "plt.ylabel('num veces elegido')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ratio de Exploración del 100%\n",
    "\n",
    "\n",
    "* Realizamos una ejecución con un ratio de exploración del 100% ($\\epsilon = 1.0$); es decir que solo exploramos.\n",
    "\n",
    "\n",
    "* Esto significa que vamos a seleccionar la \"tragaperras\" a la que jugamos de manera aleatoria sin importarnos lo buena o mala que sea en función del conocimiento previo adquirido.\n",
    "\n",
    "\n",
    "* Como resultado esperamos que se realicen (más o menos) el mismo número de partidas por \"tragaperra\". "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial Bandits Distribution\n",
      "  [0.1, 0.2, 0.1, 0.1, 0.6]\n",
      "Bandits Distribution games\n",
      "  [195. 200. 197. 205. 203.]\n",
      "Q(a) Bandits\n",
      "  [0.0923 0.195  0.1269 0.1512 0.6059]\n",
      "Reward: 236\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAauklEQVR4nO3df7gdVX3v8ffHgIKAInKgMRAiGFBoNegpKIIK+ANoK+APhFIERKMtFHykKlqfWtuL9baiXkXQUFJQAUkLKFUqvxSQFsQkBAQDV0iiBlISCAoCNzXhc/+YOePOYZ9zJieZPSfZn9fzzLP3rPmxvhP0fPdaM7OWbBMREQHwrLYDiIiIiSNJISIiKkkKERFRSVKIiIhKkkJERFSSFCIiopKkEJsMScdKuqZj3ZJe0mZMTdhUrysmhiSFaIWkJZKekvSbjuXs9Tmn7Ytsv3lDxRgFSUdJ+i9JT0q6oe14olmbtR1A9LU/sX1d20E0SZIA2X667VjWw0rgC8BLgYPaDSWalpZCTDiSTpD0n5K+JOnXku6RdPCw7YskPS5psaRjO8pvHuGcz5f0NUkrJP1c0ickPavzOEmflfRoec5DR4lvkqSzJD1c7ntK2aWzWbn9BklnSvpP4ElgV0kvlXStpJWS7pV0VMf5nlPW/QtJD0n6iqQtO7Z/WNIySQ9Kek9H+R+W+2/WUfZ2SQvG8c8+ItvX2Z4DPLghzxsTU5JCTFT7AouA7YFPApdL2k7SVsAXgUNtbwPsByyocb4vAc8HdgVeD7wbOHFYffeW9f0jcH75K7+b9wGHAjOAVwJHdNnnOGAmsA2wArgWuBjYATgGOEfSXuW+/xvYvTzfS4ApwN8ASDoE+CvgTcB04I1DFdj+MfBIuW3InwFf7xa0pDMk/WqkZYRrjX5jO0uWni/AEuA3wK86lveV206g+FWqjv1vo/hDu1W579uBLYed8wTg5o51U/yRnQSsAvbs2PZ+4IaO4+7r2Pbc8tjfGyH27wPv71h/Y7n/ZuX6DcDfdWx/F/DDYef4KkWyE/AEsFvHttcAi8vvs4HPdGzbfei6yvWPAheV37ejaJlMbui/2XuH/s2ybLpL7ilEm47wyPcUHnD5l6j0c+BFtp+Q9C6KX8/nl100p9u+Z5R6tgeeXZ6j83xTOtb/e+iL7SfLRsLWkg4A/mPoGNt7AS8CftlxbOf3bmW7APsO+zW+GcUv+gGKJDSvo2EiikRGWde8YXF3+gawUNLWwFEUyWdZl3giakn3UUxUU4Z130yl7NO2fbXtNwGTgXuA88Y418PAbyn+OHee74GxgrD9Q9tbl8tQd88yYKeO3XbudmjH918CN9retmPZ2vafl7E9BezVse35trfuqKvz/FOHxfcAcAtwJEVLqmvXEYCkjw972mutZax/i+gPSQoxUe0AnCppc0nvBF4GXCVpR0lvLe8trKLogloz2olsrwHmAGdK2kbSLsCHKH5lj8cc4DRJUyRtS9GFM5rvALtLOq68ns3Lm8Qvc/FU0nnA5yXtAFCe9y0ddZ0gaU9Jz6Xochrua8BHgD8ArhgpCNuf7khwz1hGOq68sb4FRevmWZK2kLT5GNccG6kkhWjTvw/7tdr5B+1HFDdWHwbOBN5h+xGK/82eTtFqWElx0/gvatT1lxR994uAmylu+s4eZ9znAdcAdwK3A1cBqxkhOdl+HHgzcHQZ939T3Fx+TrnLR4H7gFslPQZcB+xRHvsfFI+Dfr/c5/tdqriCohV0he0nxnlNozmOojVzLnBA+X2s1llspLR2t21E+ySdALzX9v5tx1JH+fjqV2zvMubOzcVwP8XN7036vY9oXloKEetI0paSDpO0maQpFF06I3bb9CCet1Pcw+jWiohYJ3n6KGLdCfgUcClFV8p3Kd8r6HkgxbATewLHeeN+azomiHQfRUREJd1HERFR2ai7j7bffntPmzat7TAiIjYq8+bNe9j2QLdtG3VSmDZtGnPnzm07jIiIjYqk4W/GV9J9FBERlSSFiIioJClEREQlSSEiIipJChERUUlSiIiISpJCRERUkhQiIqKSpBAREZWN+o3miFh30874btshbBBLPvNHbYewSUpLISIiKo21FCTtTDF37O8BTwOzbP8fSdtRjEM/DVgCHGX70fKYjwEnUUxreKrtq5uKLyL6z6bSSoLmWkpNthRWA6fbfhnwauBkSXsCZwDX254OXF+uU247GtgLOAQ4R9KkBuOLiIhhGksKtpfZnl9+fxxYCEwBDgcuLHe7EDii/H448E3bq2wvppikfJ+m4ouIiGfqyY1mSdOAvYEfATvaXgZF4pC0Q7nbFODWjsOWlmXDzzUTmAkwderUBqPetKUZHRHdNH6jWdLWwGXAB20/NtquXcqeMVeo7Vm2B20PDgx0nSMiIiLGqdGkIGlzioRwke3Ly+KHJE0ut08GlpflS4GdOw7fCXiwyfgiImJtTT59JOB8YKHtz3VsuhI4HvhM+fntjvKLJX0OeBEwHbitqfiif6XrLGJkTd5TeC1wHPATSQvKso9TJIM5kk4CfgG8E8D23ZLmAD+leHLpZNtrGowvIiKGaSwp2L6Z7vcJAA4e4ZgzgTObimm4TeUXY34tRsSGkjeaIyKikqQQERGVJIWIiKgkKURERCVJISIiKkkKERFRSVKIiIhKkkJERFSSFCIiopKkEBERlSSFiIioJClEREQlSSEiIipJChERUUlSiIiISpJCRERUGksKkmZLWi7pro6ySyUtKJclQzOySZom6amObV9pKq6IiBhZk9NxXgCcDXxtqMD2u4a+SzoL+HXH/vfbntFgPBERMYYmp+O8SdK0btskCTgKOKip+iMiYt21dU/hAOAh2z/rKHuxpNsl3SjpgJEOlDRT0lxJc1esWNF8pBERfaStpHAMcEnH+jJgqu29gQ8BF0t6XrcDbc+yPWh7cGBgoAehRkT0j54nBUmbAW8DLh0qs73K9iPl93nA/cDuvY4tIqLftdFSeCNwj+2lQwWSBiRNKr/vCkwHFrUQW0REX2vykdRLgFuAPSQtlXRSuelo1u46AngdcKekO4B/Az5ge2VTsUVERHdNPn10zAjlJ3Qpuwy4rKlYIiKinrzRHBERlSSFiIioJClEREQlSSEiIipJChERUUlSiIiISpJCRERUkhQiIqKSpBAREZUkhYiIqCQpREREJUkhIiIqSQoREVFJUoiIiEqSQkREVJIUIiKi0uTMa7MlLZd0V0fZ30p6QNKCcjmsY9vHJN0n6V5Jb2kqroiIGNmYM69J2hz4c4opMwFuBL5i+7djHHoBcDbwtWHln7f92WF17EkxTedewIuA6yTtbnvNmFcQEREbTJ2WwrnAq4BzyuWVZdmobN8E1J1n+XDgm7ZX2V4M3AfsU/PYiIjYQOrM0fyHtl/Rsf59SXesR52nSHo3MBc43fajwBTg1o59lpZlzyBpJjATYOrUqesRRkREDFenpbBG0m5DK5J2BcbbrXMusBswA1gGnDV02i77utsJbM+yPWh7cGBgYJxhREREN3VaCh8GfiBpEcUf712AE8dTme2Hhr5LOg/4Trm6FNi5Y9edgAfHU0dERIzfmEnB9vWSpgN7UCSFe2yvGk9lkibbXlauHgkMPZl0JXCxpM9R3GieDtw2njoiImL8RkwKkt42wqbdJGH78tFOLOkS4A3A9pKWAp8E3iBpBkXX0BLg/QC275Y0B/gpsBo4OU8eRUT03mgthT8pP3cA9gOup2gpHAjcAIyaFGwf06X4/FH2PxM4c7RzRkREs0ZMCrZPBJD0HWDPoW4fSZOBL/cmvIiI6KU6Tx9N67gPAPAQsHtD8URERIvqPH10g6SrgUso7gUcDfyg0agiIqIVdZ4+OqW86XxAWTTL9hXNhhUREW2o01IYetJo1BvLERGx8RvtkdSbbe8v6XHWfrtYgG0/r/HoIiKip0Z7+mj/8nOb3oUTERFtqjN09nZdih+vMXR2RERsZOo8kjofWAH8X+Bn5ffFkuZLelWTwUVERG/VSQrfAw6zvb3tFwKHAnOAv6CYXyEiIjYRdZLCoO2rh1ZsXwO8zvatwHMaiywiInquziOpKyV9FPhmuf4u4FFJk4CnG4ssIiJ6rk5L4U8p5jf4VrnsXJZNAo5qKrCIiOi9Om80Pwz8paStbf9m2Ob7mgkrIiLaMGZLQdJ+kn5KMdcBkl4hKTeYIyI2QXW6jz4PvAV4BMD2HcDrmgwqIiLaUScpYPuXw4rGnBVN0mxJyyXd1VH2T5LukXSnpCskbVuWT5P0lKQF5fKVdbmIiIjYMOokhV9K2g+wpGdL+itgYY3jLgAOGVZ2LfD7tl9O8TLcxzq23W97Rrl8oMb5IyJiA6uTFD4AnAxMAZYCM8r1Udm+CVg5rOwa26vL1VspnmqKiIgJou7TR8c2UPd7gEs71l8s6XbgMeATtn/YQJ0RETGK0YbO/hJrD5m9FtunjrdSSX8NrAYuKouWAVNtP1KOp/QtSXvZfqzLsTOBmQBTp04dbwgREdHFaC2FuU1UKOl44I+Bg20bwPYqYFX5fZ6k+ynmgX5GDLZnAbMABgcHR0xaERGx7kabT+HCznVJW9l+Yn0qk3QI8FHg9baf7CgfAFbaXiNpV2A6sGh96oqIiHVX5+W115Qvry0s12u9vCbpEuAWYA9JSyWdBJwNbANcO+zR09cBd0q6A/g34AO2V3Y9cURENKbOgHhfoHh57UooXl6TNObLa7aP6VJ8/gj7XgZcViOWiIhoUGMvr0VExManTkthrZfXgFOp9/JaRERsZBp7eS0iIjY+bb68FhERE0ytewoREdEfkhQiIqKSpBAREZU6L6+dJul5Kpwvab6kN/ciuIiI6K06LYX3lAPTvRkYAE4EPtNoVBER0Yo6SUHl52HAv5TTcWqU/SMiYiNVJynMk3QNRVK4WtI2wNPNhhUREW2o80bzSRQvrC2y/aSkF1J0IUVExCamTkvBwJ4Uw1sAbAVs0VhEERHRmjpJ4RzgNcDQqKePA19uLKKIiGhNne6jfW2/spw/GduPlgPjRUTEJqZOS+G3kiZRztdczpKWG80REZugOknhi8AVwA6SzgRuBj7daFQREdGKMZOC7YuAjwD/ACwDjrD9r2MdJ2m2pOWS7uoo207StZJ+Vn6+oGPbxyTdJ+leSW8Z3+VERMT6qDPMxauBB2x/2fbZwFJJ+9Y49wXAIcPKzgCutz0duL5cR9KewNHAXuUx55RdVhER0UN1uo/OBX7Tsf5EWTYq2zcBK4cVHw5cWH6/EDiio/ybtlfZXgzcB+xTI7aIiNiAag1zYdtDK7afpt5TS93saHtZeZ5lwA5l+RSgcx7opWXZM4ORZkqaK2nuihUrxhlGRER0UycpLJJ0qqTNy+U0YNEGjqPbWEruUobtWbYHbQ8ODAxs4DAiIvpb3Tma9wMeoPgFvy8wc5z1PSRpMkD5ubwsXwrs3LHfTsCD46wjIiLGqc7TR8ttH217B9s72v5T28vHOm4EVwLHl9+PB77dUX60pOdIejEwHbhtnHVERMQ41Xn6aHdJ1w89Wirp5ZI+UeO4S4BbgD0kLZV0EsU8DG+S9DPgTeU6tu8G5gA/Bb4HnGx7zXgvKiIixqfODePzgA8DXwWwfaeki4H/NdpBto8ZYdPBI+x/JnBmjXgiIqIhde4pPNf28K6c1U0EExER7aqTFB6WtBu/G/voHRRvNkdExCamTvfRycAs4KWSHgAWA8c2GlVERLSiTlL4ue03StoKeJbtx5sOKiIi2lGn+2ixpFnAq1l7uIuIiNjE1EkKewDXUXQjLZZ0tqT9mw0rIiLaUOfltadsz7H9NmBv4HnAjY1HFhERPVenpYCk10s6B5gPbAEc1WhUERHRijFvNEtaDCygeOP4w7afaDqoiIhoR52nj15h+7HGI4mIiNbVuaeQhBAR0Sdq3VOIiIj+kKQQERGVOjeatwXeDUzr3N/2qY1FFRERrahzo/kq4FbgJ8DTzYYTERFtqpMUtrD9ocYjiYiI1tVJCl+X9D7gO8CqoULbK8dToaQ9gEs7inYF/gbYFngfsKIs/7jtq8ZTR0REjE+dpPA/wD8Bf005p0L5uet4KrR9LzADQNIk4AHgCuBE4PO2Pzue80ZExPqrkxQ+BLzE9sMN1H8wcL/tn0tq4PQREbEu6jySejfwZEP1Hw1c0rF+iqQ7Jc2W9IJuB0iaKWmupLkrVqzotktERIxTnaSwBlgg6auSvji0rG/Fkp4NvBX417LoXGA3iq6lZcBZ3Y6zPcv2oO3BgYGB9Q0jIiI61Ok++la5bGiHAvNtPwQw9Akg6TyKG9sREdFDYyYF2xc2VPcxdHQdSZpse1m5eiRwV0P1RkTECOoOne3h5bbH9fRRec7nAm8C3t9R/I+SZpR1LRm2LSIieqBO99Fgx/ctgHcC261PpbafBF44rOy49TlnRESsvzpDZz/SsTxg+wvAQc2HFhERvVan++iVHavPomg5bNNYRBER0Zo63Uedj4aupujvzxzNERGboDpPHx3Yi0AiIqJ9dbqPngO8nWfOp/B3zYUVERFtqNN99G3g18A8OkZJjYiITU+dpLCT7UMajyQiIlpXZ+yj/5L0B41HEhERravTUtgfOKF8s3kVIMC2X95oZBER0XN1ksKhjUcRERETQp1HUn/ei0AiIqJ9de4pREREn0hSiIiISpJCRERUkhQiIqKSpBAREZU6j6RucJKWAI8Da4DVtgclbQdcSjHG0hLgKNuPthFfRES/arOlcKDtGbaHZnY7A7je9nTg+nI9IiJ6aCJ1Hx0OXFh+vxA4or1QIiL6U1tJwcA1kuZJmlmW7Wh7GUD5uUO3AyXNlDRX0twVK1b0KNyIiP7Qyj0F4LW2H5S0A3CtpHvqHmh7FjALYHBw0E0FGBHRj1ppKdh+sPxcDlwB7AM8JGkyQPm5vI3YIiL6Wc+TgqStJG0z9B14M3AXcCVwfLnb8RST+0RERA+10X20I3CFpKH6L7b9PUk/BuZIOgn4BfDOFmKLiOhrPU8KthcBr+hS/ghwcK/jiYiI35lIj6RGRETLkhQiIqKSpBAREZUkhYiIqCQpREREJUkhIiIqSQoREVFJUoiIiEqSQkREVJIUIiKikqQQERGVJIWIiKgkKURERCVJISIiKkkKERFRSVKIiIhKG9Nx7izpB5IWSrpb0mll+d9KekDSgnI5rNexRUT0uzam41wNnG57fjlX8zxJ15bbPm/7sy3EFBERtDMd5zJgWfn9cUkLgSm9jiMiIp6p1XsKkqYBewM/KotOkXSnpNmSXtBeZBER/am1pCBpa+Ay4IO2HwPOBXYDZlC0JM4a4biZkuZKmrtixYpehRsR0RdaSQqSNqdICBfZvhzA9kO219h+GjgP2KfbsbZn2R60PTgwMNC7oCMi+kAbTx8JOB9YaPtzHeWTO3Y7Erir17FFRPS7Np4+ei1wHPATSQvKso8Dx0iaARhYAry/hdgiIvpaG08f3Qyoy6areh1LRESsLW80R0REJUkhIiIqSQoREVFJUoiIiEqSQkREVJIUIiKikqQQERGVJIWIiKgkKURERCVJISIiKkkKERFRSVKIiIhKkkJERFSSFCIiopKkEBERlSSFiIioJClERERlwiUFSYdIulfSfZLOaDueiIh+MqGSgqRJwJeBQ4E9KeZt3rPdqCIi+seESgrAPsB9thfZ/h/gm8DhLccUEdE3ZLvtGCqS3gEcYvu95fpxwL62T+nYZyYws1zdA7i354Gum+2Bh9sOoiX9fO3Q39ffz9cOE//6d7E90G3DZr2OZAzqUrZW1rI9C5jVm3DWn6S5tgfbjqMN/Xzt0N/X38/XDhv39U+07qOlwM4d6zsBD7YUS0RE35loSeHHwHRJL5b0bOBo4MqWY4qI6BsTqvvI9mpJpwBXA5OA2bbvbjms9bXRdHU1oJ+vHfr7+vv52mEjvv4JdaM5IiLaNdG6jyIiokVJChERUUlSaEg/D9chabak5ZLuajuWXpO0s6QfSFoo6W5Jp7UdUy9J2kLSbZLuKK//U23H1GuSJkm6XdJ32o5lPJIUGpDhOrgAOKTtIFqyGjjd9suAVwMn99l/+1XAQbZfAcwADpH06nZD6rnTgIVtBzFeSQrN6OvhOmzfBKxsO4422F5me375/XGKPw5T2o2qd1z4Tbm6ebn0zdMsknYC/gj457ZjGa8khWZMAX7Zsb6UPvrDEAVJ04C9gR+1HEpPld0nC4DlwLW2++n6vwB8BHi65TjGLUmhGWMO1xGbNklbA5cBH7T9WNvx9JLtNbZnUIxIsI+k3285pJ6Q9MfActvz2o5lfSQpNCPDdfQxSZtTJISLbF/edjxtsf0r4Ab65/7Sa4G3SlpC0WV8kKRvtBvSuktSaEaG6+hTkgScDyy0/bm24+k1SQOSti2/bwm8Ebin1aB6xPbHbO9kexrF/+e/b/vPWg5rnSUpNMD2amBouI6FwJxNYLiO2iRdAtwC7CFpqaST2o6ph14LHEfxK3FBuRzWdlA9NBn4gaQ7KX4cXWt7o3w0s19lmIuIiKikpRAREZUkhYiIqCQpREREJUkhIiIqSQoREVFJUogoSVpTPkJ6h6T5kvbbQOedNjRirKRBSV8cYb8lkrbfEHVGjNeEmo4zomVPlcMzIOktwD8Ar9+QFdieC8zdkOeM2JDSUojo7nnAo1CMYyTp+rL18BNJh5fl08p5E84r5w64pnyLF0mvKlsctwAnD51U0huGxtmX9MLymNslfZWOMbMkfUjSXeXywd5ddvS7JIWI39my7D66h2Lo478vy/8fcKTtVwIHAmeVw1kATAe+bHsv4FfA28vyfwFOtf2aUer7JHCz7b0phkGZCkVCAU4E9qWYk+F9kvbeQNcYMaokhYjfecr2DNsvpRjE7WvlH38Bny6HbriOYhj0HctjFtteUH6fB0yT9HxgW9s3luVfH6G+1wHfALD9XcqWCbA/cIXtJ8q5CS4HDthQFxkxmtxTiOjC9i3lTd8B4LDy81W2f1uOgrlFueuqjsPWAFtSJJG648d026/b0OsRPZGWQkQXkl4KTAIeAZ5PMU7+byUdCOwy2rHlkNG/lrR/WXTsCLveNLRN0qHACzrKj5D0XElbAUcCP1yPy4moLS2FiN/ZspwxDIpf68fbXiPpIuDfJc0FFlBvKOgTgdmSnqQYLbebTwGXSJoP3Aj8AsD2fEkXALeV+/2z7dvHcT0R6yyjpEZERCXdRxERUUlSiIiISpJCRERUkhQiIqKSpBAREZUkhYiIqCQpRERE5f8DViS3KTby8HsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bandits, total_reward, q_bandits, num_selected_bandit = multi_armed_bandit(num_games=1000, \n",
    "                                                                           epsilon=1.0, \n",
    "                                                                           verbose=False)\n",
    "\n",
    "print(\"Initial Bandits Distribution\\n  {}\".format(bandits))\n",
    "print(\"Bandits Distribution games\\n  {}\".format(num_selected_bandit))\n",
    "print(\"Q(a) Bandits\\n  {}\".format(q_bandits))\n",
    "print(\"Reward: {}\".format(total_reward))\n",
    "\n",
    "# Mostramos en una gráfica de barras el número de veces que elegimos cada \"tragaperras\"\n",
    "plt.title('Epsilon-greedy = 1')\n",
    "plt.bar(range(0,len(bandits)), num_selected_bandit)\n",
    "plt.xlabel('Bandido')\n",
    "plt.ylabel('num veces elegido')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ratio de Exploración del 0%\n",
    "\n",
    "\n",
    "* Realizamos una ejecución con un ratio de exploración del 0% ($\\epsilon = 0.0$).\n",
    "\n",
    "\n",
    "* Esto significa que solo vamos a explorar hasta que una \"tragaperras\" nos de una recompensa. Una vez que tengamos la primera recompensa vamos a jugar siempre a la misma \"tragaperras\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial Bandits Distribution\n",
      "  [0.1, 0.6, 0.1, 0.1, 0.2]\n",
      "Bandits Distribution games\n",
      "  [  0. 997.   0.   1.   2.]\n",
      "Q(a) Bandits\n",
      "  [0.     0.6229 0.     0.     0.    ]\n",
      "Reward: 621\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZsklEQVR4nO3dedRddX3v8ffHQJlRkKEYkKBGEWxxSAEVR1RAW8FaFS8iUCq9FgWvVoVe1/XarnjptVqrFW0cUSncLKuVOoIRUHpVGiYVAteUIAQiBBwYtJHhe//YO/UQnif7JHnO8OS8X2uddc7+nT18t8Tnc/b0+6WqkCRpfR426gIkSePPsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLDQRkhyT5Pye6UryuFHWNAib635p9AwLjZ0kNyT5VZK7e15/vynrrKqzq+pFM1WjGkm2SvKJJHcm+UmSN4+6Jg3GFqMuQJrGH1TVN0ZdxCAlCZCqemDUtWyC/wnMB/YGfhu4MMk1VfW1kValGeeRhWaVJMcn+dckH0zyiyTXJjl0ne+vT3JXkhVJjulpv2SadT48yaeTrE7y4yTvSPKw3uWS/E2Sn7XrPGI99c1J8t4kt7fzvqE9NbRF+/1FSRYm+Vfgl8Bjkuyb5IIkP01yXZJX9qxvq3bbNya5NclHkmzT8/1bk6xKckuSP+5p/712/i162l6e5MqN+J99fV4L/FVV/ayqlgEfBY6f4W1oDBgWmo0OAq4HdgHeCXw+yc5JtgM+ABxRVTsAzwCu7GN9HwQeDjwGeA7NH8AT1tnede32/jfw8faoYCqvA44Angw8FThqinmOBU4CdgBWAxcA/wjsBrwaODPJ/u28fw08vl3f44C5wP8ASHI48OfAC2l+3b9g7Qaq6t+AO9rv1noN8Jmpik5yWpKfT/eaZpmdgEcBV/U0XwXsP9X8muWqypevsXoBNwB3Az/veb2u/e544Baa0zdr57+U5g/wdu28Lwe2WWedxwOX9EwXzR/fOcAaYL+e7/4UuKhnueU9323bLvvb09T+TeBPe6Zf0M6/RTt9EfCXPd+/Cvj2Ouv4B5oQDHAP8Nie754OrGg/fwI4o+e7x6/dr3b67cDZ7eedaY5k9pjB/057tdvbuqfthcANo/435GvmX16z0Lg6qqa/ZnFztX+ZWj8GHlVV9yR5Fc2v7Y+3p3reUlXXrmc7uwC/1a6jd31ze6Z/svZDVf2yPajYPsmzgK+uXaaq9qf5pX1Tz7K9n6dq2xs4aJ1f71vQHAHsShNOl/UcyIQm4Gi3ddk6dff6LLAsyfbAK2lCadUU9Wysu9v3HYH/6Pl81wxuQ2PC01Cajeaucxro0TRHG1TV16vqhcAewLU059DX53bgXpo/2r3ru7mriKr6dlVt377WnnpZBezZM9teUy3a8/km4OKqekTPa/uqen1b26+A/Xu+e3hVbd+zrd71P3qd+m4GvgO8jObIa8pTUABJ/mKdu88e9Jpm/3/W1nBAT/MBwNXTbUezl2Gh2Wg34JQkWyZ5BfBE4CtJdk/y0vbaxRqaX773r29FVXU/sBhYmGSHJHsDb6b5Vb4xFgOnJpmb5BE0p4LW50vA45Mc2+7Plu3F6SdWc5fUR4G/TbIbQLvew3q2dXyS/ZJsS3Pqal2fBt4G/A7whemKqKp39wTfQ17rqf/TwDuS7JRkX5prNp/q2GfNQoaFxtW/rPPrtvcP3fdoLujeDiwE/qiq7qD59/wWmqOMn9JcrP6zPrb1RpprA9cDl9BcbP7ERtb9UeB84PvAFcBXgPuYJrSq6i7gRcDRbd0/obmovVU7y9uB5cB3k9wJfAN4QrvsV4H301wnWd6+r+sLNEdNX6iqezZyn9bnncC/05wCuxh4T3nb7GYpDz71K423JMcDf1JVh4y6ln60t9l+pKr27px5cDX8O81F9836uRUNlkcW0gxKsk2SFyfZIslcml/e057+GUI9L6e5RjLVUYfUN++GkmZWgHcB/4fm4vSXaZ+LGHohyUXAfsCxNbufEtcY8DSUJKmTp6EkSZ0229NQu+yyS82bN2/UZUjSrHLZZZfdXlW7rtu+2YbFvHnzWLp06ajLkKRZJcm6PQEAnoaSJPXBsJAkdTIsJEmdDAtJUifDQpLUaWBhkWYQ99uS/LCnbed2+Mgfte879Xx3epLl7bCSh/W0Py3JD9rvPrCeEcokSQMyyCOLTwGHr9N2GrCkquYDS9ppkuxH0+vm/u0yZyZZO8DLh2mGoJzfvtZdpyRpwAYWFlX1LZpuonsdCZzVfj6L34xPfCRwblWtqaoVNN0tH5hkD2DHqvpOOzLap5l6TGNJ0gAN+5rF7muHdWzfd2vb5/LgoSZXtm1z28/rtk8pyUlJliZZunr16hktXJIm2bg8wT3VdYhaT/uUqmoRsAhgwYIF9pC4Eead9uVRlzBjbjjjJaMuQdpsDPvI4tb21BLt+21t+0oePJbwnjSjhq3kweMZr22XJA3RsMPiPOC49vNxwBd72o9OslWSfWguZF/anqq6K8nB7V1Qr+1ZRpI0JAM7DZXkHOC5wC5JVtKMGHYGsDjJicCNwCsAqurqJIuBa2jGKz65qtaOWfx6mjurtgG+2r4kSUM0sLCoqldP89Wh08y/EFg4RftS4EkzWJokaQP5BLckqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE4jCYsk/y3J1Ul+mOScJFsn2TnJBUl+1L7v1DP/6UmWJ7kuyWGjqFmSJtnQwyLJXOAUYEFVPQmYAxwNnAYsqar5wJJ2miT7td/vDxwOnJlkzrDrlqRJNqrTUFsA2yTZAtgWuAU4Ejir/f4s4Kj285HAuVW1pqpWAMuBA4dbriRNtqGHRVXdDPwNcCOwCvhFVZ0P7F5Vq9p5VgG7tYvMBW7qWcXKtk2SNCSjOA21E83Rwj7Ao4DtkrxmfYtM0VbTrPukJEuTLF29evWmFytJAkZzGuoFwIqqWl1V9wKfB54B3JpkD4D2/bZ2/pXAXj3L70lz2uohqmpRVS2oqgW77rrrwHZAkibNKMLiRuDgJNsmCXAosAw4Dziunec44Ivt5/OAo5NslWQfYD5w6ZBrlqSJtsWwN1hV30vyOeBy4D7gCmARsD2wOMmJNIHyinb+q5MsBq5p5z+5qu4fdt2SNMmGHhYAVfVO4J3rNK+hOcqYav6FwMJB1yVJmppPcEuSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKlT53MWSbYEXg88u226GPhI21WHJGkC9PNQ3oeBLYEz2+lj27Y/GVRRkqTx0k9Y/F5VHdAz/c0kVw2qIEnS+OnnmsX9SR67diLJYwD7ZpKkCdLPkcVbgQuTXE8ztsTewAkDrUqSNFY6w6KqliSZDzyBJiyurao1A69MkjQ2pg2LJH84zVePTUJVfX5ANUmSxsz6jiz+oH3fjWYkuyU0RxbPAy6iGeFOkjQBpg2LqjoBIMmXgP2qalU7vQfwoeGUJ0kaB/3cDTVvbVC0bgUeP6B6JEljqJ+7oS5K8nXgHKCAo4ELB1qVJGms9HM31Bvai93PapsWVdUXBluWJGmc9DUGd3vnkxe0JWlCre/W2Uuq6pAkd9GcfvrPr4Cqqh0HXp0kaSys726oQ9r3HYZXjiRpHPXTRfnOUzTfZRflkjQ5+rl19nJgNfD/gB+1n1ckuTzJ0wZZnCRpPPQTFl8DXlxVu1TVI4EjgMXAn/GbMS4kSZuxfsJiQVV9fe1EVZ0PPLuqvgtsNbDKJEljo59bZ3+a5O3Aue30q4CfJZkDPDCwyiRJY6OfI4v/AuwJ/HP72qttmwO8clCFSZLGRz9PcN8OvDHJ9lV19zpfLx9MWZKkcdJ5ZJHkGUmuAa5ppw9I4oVtSZog/ZyG+lvgMOAOgKq6Cnj2IIuSJI2XfsKCqrppnab7B1CLJGlM9XM31E1JngFUkt8CTgGWDbYsSdI46efI4r8CJwNzgZXAk9vpjZbkEUk+l+TaJMuSPD3JzkkuSPKj9n2nnvlPT7I8yXVJDtuUbUuSNlxnWFTV7VV1TFXtXlW7VdVrquqOTdzu3wFfq6p9gQNojlROA5ZU1Xya8b5PA0iyH82AS/sDhwNnts94SJKGZH1dlH+QB3dN/iBVdcrGbDDJjjQXyI9v1/Nr4NdJjgSe2852FnAR8HbgSODcqlpD0yfVcuBA4Dsbs31J0oZb3zWLpQPa5mNoOiP8ZJIDgMuAU4Hd1471XVWrkuzWzj8X+G7P8ivbtodIchJwEsCjH/3owVQvSRNofeNZnNU7nWS7qrpnhrb5VOCNVfW9JH9He8ppGpmqvKlmrKpFwCKABQsWTHtUJEnaMP08lPf09qG8Ze30pj6UtxJYWVXfa6c/RxMetybZo93GHsBtPfPv1bP8nsAtm7B9SdIG6uduqPczgw/lVdVPaG7HfULbdCjN0+HnAce1bccBX2w/nwccnWSrJPsA84FLN3b7kqQN189zFlTVTcmDzgZt6kN5bwTObp/buB44gSa4Fic5EbgReEW77auTLKYJlPuAk6vKhwIlaYhG8lBeVV0JLJjiq0OnmX8hsHBTtilJ2ngjeShPkjS79NtF+TFDqEWSNKb66khQkjTZDAtJUifDQpLUqZ+H8k5NsmMaH09yeZIXDaM4SdJ46OfI4o+r6k7gRcCuNM9EnDHQqiRJY6WfsFj7NN6LgU+2T3BP1V+TJGkz1U9YXJbkfJqw+HqSHYAHBluWJGmc9PME94k0D+JdX1W/TPJImlNRkqQJ0c+RRQH70XTzAbAdsPXAKpIkjZ1+wuJM4OnAq9vpu4APDawiSdLY6ec01EFV9dQkVwBU1c/aDgUlSROinyOLe5PMoR2dLsmueIFbkiZKP2HxAeALwG5JFgKXAO8eaFWSpLHST6+zZye5jGasiQBHVdUmjWchSZpdOsMiycHA1VX1oXZ6hyQH9YyhLUnazPVzGurDwN090/e0bZKkCdFXdx9VVWsnquoB+hy7W5K0eegnLK5PckqSLdvXqcD1gy5MkjQ++h2D+xnAzTRjcB8EnDTIoiRJ46Wfu6FuA44eQi2SpDHVz+BHj0+yJMkP2+nfTfKOwZcmSRoX/ZyG+ihwOnAvQFV9H480JGmi9BMW21bVpeu03TeIYiRJ46mfsLg9yWP5Td9QfwSsGmhVkqSx0s/zEicDi4B9k9wMrACOGWhVkqSx0k9Y/LiqXpBkO+BhVXXXoIuSJI2Xfk5DrUiyCDiYB3f7IUmaEP2ExROAb9CcjlqR5O+THDLYsiRJ46QzLKrqV1W1uKr+EHgKsCNw8cArkySNjX6OLEjynCRnApcDWwOvHGhVkqSx0s94FiuAK4HFwFur6p5BFyVJGi/9HFkcUFUvq6pzZjIoksxJckWSL7XTOye5IMmP2vedeuY9PcnyJNclOWymapAk9aefaxZ3DmjbpwK9w7OeBiypqvnAknaaJPvRdC+yP3A4cGaSOQOqSZI0hb6uWcy0JHsCLwE+1tN8JHBW+/ks4Kie9nOrak1VrQCWAwcOqVRJEiMKC+D9wNuAB3radq+qVQDt+25t+1zgpp75VrZtD5HkpCRLkyxdvXr1jBctSZOqnwvcjwBeC8zrnb+qTtmYDSb5feC2qrosyXP7WWSKtpqijapaRNM1CQsWLJhyHknShuunu4+vAN8FfsCDjwQ21jOBlyZ5Mc1tuDsm+Sxwa5I9qmpVkj2A29r5VwJ79Sy/J3DLDNQhSepTP2GxdVW9eaY2WFWn04yPQXtk8edV9Zok7wGOA85o37/YLnIe8I9J3gc8CpgPrNtluiRpgPoJi88keR3wJWDN2saq+ukM13IGsDjJicCNwCva7VydZDFwDc04GidX1f0zvG1J0nr0Exa/Bt4D/Hd+c62ggMds6sar6iLgovbzHcCh08y3EFi4qduTJG2cfsLizcDjqur2QRcjSRpP/dw6ezXwy0EXIkkaX/0cWdwPXJnkQh58zWKjbp2VJM0+/YTFP7cvSdKE6gyLqjqrax5J0uat3y7KH/I0dFVt8t1QkqTZoZ/TUAt6Pm9N8/zDzoMpR5I0jvrpovyOntfNVfV+4PmDL02SNC76OQ311J7Jh9EcaewwsIokSWOnn9NQ7+35fB9wA47BLUkTpZ+7oZ43jEIkSeOrn9NQWwEv56HjWfzl4MqSJI2Tfk5DfRH4BXAZPU9wS5ImRz9hsWdVHT7wSiRJY6ufjgT/b5LfGXglkqSx1c+RxSHA8e2T3GtoxsSuqvrdgVYmSRob/YTFEQOvQpI01vq5dfbHwyhEkjS++rlmIUmacIaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKnT0MMiyV5JLkyyLMnVSU5t23dOckGSH7XvO/Usc3qS5UmuS3LYsGuWpEk3iiOL+4C3VNUTgYOBk5PsB5wGLKmq+cCSdpr2u6OB/YHDgTOTzBlB3ZI0sYYeFlW1qqoubz/fBSwD5gJHAme1s50FHNV+PhI4t6rWVNUKYDlw4FCLlqQJN9JrFknmAU8BvgfsXlWroAkUYLd2trnATT2LrWzbplrfSUmWJlm6evXqgdUtSZNmZGGRZHvgn4A3VdWd65t1iraaasaqWlRVC6pqwa677joTZUqSGFFYJNmSJijOrqrPt823Jtmj/X4P4La2fSWwV8/iewK3DKtWSdJo7oYK8HFgWVW9r+er84Dj2s/HAV/saT86yVZJ9gHmA5cOq15JEmwxgm0+EzgW+EGSK9u2vwDOABYnORG4EXgFQFVdnWQxcA3NnVQnV9X9Q69akibY0MOiqi5h6usQAIdOs8xCYOHAipIkrZdPcEuSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6zZqwSHJ4kuuSLE9y2qjrkaRJMivCIskc4EPAEcB+wKuT7DfaqiRpcmwx6gL6dCCwvKquB0hyLnAkcM1Iq5K02Zh32pdHXcKMuOGMlwxkvbMlLOYCN/VMrwQOWnemJCcBJ7WTdye5bgi1baxdgNtHXcQIDXz/89eDXPsm8b/95O7/bPh3v/dUjbMlLDJFWz2koWoRsGjw5Wy6JEurasGo6xiVSd7/Sd53mOz9n837PiuuWdAcSezVM70ncMuIapGkiTNbwuLfgPlJ9knyW8DRwHkjrkmSJsasOA1VVfcleQPwdWAO8ImqunrEZW2qWXG6bIAmef8ned9hsvd/1u57qh5y6l+SpAeZLaehJEkjZFhIkjoZFiMwyV2XJPlEktuS/HDUtQxbkr2SXJhkWZKrk5w66pqGJcnWSS5NclW77+8adU3DlmROkiuSfGnUtWwMw2LI7LqETwGHj7qIEbkPeEtVPRE4GDh5gv7brwGeX1UHAE8GDk9y8GhLGrpTgWWjLmJjGRbD959dl1TVr4G1XZdMhKr6FvDTUdcxClW1qqoubz/fRfOHY+5oqxqOatzdTm7Zvibm7pokewIvAT426lo2lmExfFN1XTIRfzD0G0nmAU8BvjfiUoamPQ1zJXAbcEFVTcy+A+8H3gY8MOI6NpphMXx9dV2izVeS7YF/At5UVXeOup5hqar7q+rJND0wHJjkSSMuaSiS/D5wW1VdNupaNoVhMXx2XTLBkmxJExRnV9XnR13PKFTVz4GLmJxrV88EXprkBprTzs9P8tnRlrThDIvhs+uSCZUkwMeBZVX1vlHXM0xJdk3yiPbzNsALgGtHWtSQVNXpVbVnVc2j+f/7N6vqNSMua4MZFkNWVfcBa7suWQYs3gy6LulbknOA7wBPSLIyyYmjrmmIngkcS/PL8sr29eJRFzUkewAXJvk+zQ+mC6pqVt5COqns7kOS1MkjC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQupDkvvbW12vSnJ5kmfM0Hrnre2BN8mCJB+YZr4bkuwyE9uUNsasGFZVGgO/aruqIMlhwP8CnjOTG6iqpcDSmVynNFM8spA23I7Az6Dp5ynJkvZo4wdJjmzb57XjVny0Hb/h/PbJZZI8rT1C+Q5w8tqVJnnu2rEOkjyyXeaKJP9AT59iSd6c5Ift603D221NMsNC6s827Wmoa2m6mf6rtv0/gJdV1VOB5wHvbbv1AJgPfKiq9gd+Dry8bf8kcEpVPX0923sncElVPYWmO5hHQxM0wAnAQTRjYrwuyVNmaB+laRkWUn9+VVVPrqp9aTrA+3QbCgHe3XZj8Q2a7uZ3b5dZUVVXtp8vA+YleTjwiKq6uG3/zDTbezbwWYCq+jLtkQxwCPCFqrqnHR/i88CzZmonpel4zULaQFX1nfZi867Ai9v3p1XVvW3Polu3s67pWex+YBuacOm3j52p5puqi3tp4DyykDZQkn2BOcAdwMNpxiq4N8nzgL3Xt2zbPfcvkhzSNh0zzazfWvtdkiOAnXraj0qybZLtgJcB396E3ZH64pGF1J9t2lHeoPl1f1xV3Z/kbOBfkiwFrqS/brdPAD6R5Jc0vQ9P5V3AOUkuBy4GbgSoqsuTfAq4tJ3vY1V1xUbsj7RB7HVWktTJ01CSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnq9P8B+fQG6v4GOyQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bandits, total_reward, q_bandits, num_selected_bandit = multi_armed_bandit(num_games=1000, \n",
    "                                                                           epsilon=0.0, \n",
    "                                                                           verbose=False)\n",
    "\n",
    "print(\"Initial Bandits Distribution\\n  {}\".format(bandits))\n",
    "print(\"Bandits Distribution games\\n  {}\".format(num_selected_bandit))\n",
    "print(\"Q(a) Bandits\\n  {}\".format(q_bandits))\n",
    "print(\"Reward: {}\".format(total_reward))\n",
    "\n",
    "# Mostramos en una gráfica de barras el número de veces que elegimos cada \"tragaperras\"\n",
    "plt.title('Epsilon-greedy = 0')\n",
    "plt.bar(range(0,len(bandits)), num_selected_bandit)\n",
    "plt.xlabel('Bandido')\n",
    "plt.ylabel('num veces elegido')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reward por Ratio de Exploración\n",
    "\n",
    "\n",
    "* A continuación se muestra el reward que obtenemos por ratio de exploración"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABFtklEQVR4nO3dd3xb13n4/88DgAQXuPcQSe1tWZZkW/JWHDu2E8VJ2jhpPJI4Tr+127Tf/r5JnKZt0lTNbto0cRs7y5mOm2XF8ZYjL8mWJVtbokhNUtx7DxDn98e9gAASJEEJpETweb9efAm4uPfiXEh68PA5554jxhiUUkrFFseFboBSSqno0+CulFIxSIO7UkrFIA3uSikVgzS4K6VUDNLgrpRSMUiDu7ooiMg9IvLqhW5HNInI0yJy94Vux1QRkR+LyL9e6Hao8DS4zwIiclJE+kSkW0Tq7f+UKRe6XRcLEblORHz25xP8c+X5nNcY8y5jzKPRaudIIrJGRJ4UkTYRaReRQyKyWUQypuo91cyhwX32eLcxJgVYBVwKPHihGiIirovwvWuNMSkjfnZMa+MmQUTWA9uA14DFxph04GbAC1wyxjEX7HNX00+D+yxjjKkHnsUK8gCIyBUist3O/vaKyHX29utFZH/Qfi+IyM6g56+KyHvtx58VkWMi0mVnkLcH7XePiLwmIt8SkVbgCyKSJSJbRKTTPue8sdosImUiYkTkPhGpFZE6Efn7oNcdQe/fIiKPi0jmiGM/LiKngRcn+5mJyDYR+bKI7BSRDhF5Iuj8CSLyM/t920XkTRHJCzru3qA2fl5ETolIo4j8RETSRrTxbhE5LSLNIvIPEzTra8CPjDFfNsY0ABhjThtj/tkYs22cz90tIt+w36dBRP5HRBKDrvU2EdljX8t2EVkZ9NqlIvKW/Xf8KyAh6LUDIvLuoOdx9nWsmuznraJDg/ssIyLFwLuAKvt5EfBH4F+BTOD/A34jIjnADmC+iGTbWd9yoFhEPHZAuAx4xT71MeBqIA34IvAzESkIeuvLgeNALrAZ+C7QDxQAH7N/JnI9sAB4J/BZEXmHvf1vgPcC1wKFQJt9/mDXAkuAmyJ4n3DusttYiJUdf9vefjfWNZcAWcBfAn1hjr/H/rkemAukAN8Zsc9VwCJgI/BPIrIkXENEJBm4EvhNBO0e+bl/FViI9eU+HygC/sk+72rgh8An7Wv5HrDF/kKIB34P/BTr38n/Au8Pep+fAB8Jen4LUGeM2RNBG9VUMMboT4z/ACeBbqALMMBWIN1+7TPAT0fs/yxwt/34FeB9wBXAc8DjWL/+Xw/sG+c99wCb7Mf3AKeDXnMCQ1jlBP+2fwNeHeNcZXa7g/f/GvAD+/FhYGPQawX2+V1Bx84dp63XAT6gfcRPsv36NuArQfsvBQbt6/gYsB1YGea824B77cdbgb8Kem1RmDYWB72+E7hjjPYWj/F5tAM9wOfH+NzFfn1e0LYrgRP24/8GvjTivSqwvhivAWoBCXptO/Cv9uNCrH9fqfbzXwOfvtD/9mfzj2bus8d7jTEerEC2GMi2t5cCf2b/Gt4uIu1YGaQ/637JPuYa+/E2rP/s19rPARCRu4J+nW/HyvL97wFQHfQ4ByuoBW87FcE1jNy/MOgafhf03oeBYSBvjGPDqTXGpI/46RnnveOwru+nWF+Gj9klo6+JSFyY8xcSeo2nsD6D4DbWBz3uxcrukdBO3jlYv5n4OPt3hDHm08aqu//OPm+4ducAScDuoM/qGXs7WJ/j34/4t1Bit70QOGPsyB10Df73r8Wq/79fRNKxfjv8eZjPQU0TDe6zjDHmJeDHwDfsTdVYmXtwUEs2xnzFfn1kcH+JEcFdREqBR4AHgCw7yBzAyhQDbx30uAmrtFEStG1OBM0fuX9t0DW8a8Q1JBhjzozx/udi5HsPAc3GmCFjzBeNMUuB9cBtWCWckWqxgmfwObxAw0RvbEI7eU/bXzpvYP1GNeHhQY+bsUpGy4I+pzRjdbSD9TluHvE5JhljfgnUAUUiEvx3OvLv7FGs0syfATtGfP5qmmlwn53+A7jR7uz6GfBuEblJRJx2B+F1dm0erF+9FwHrgJ3GmINYQepy4GV7n2SsINIEICIfxcrcwzLGDAO/xergSxKRpVi164n8o73/MuCjwK/s7f8DbLa/ZBCRHBHZFMkHMQkfEZGlIpIE/Avwa2PMsFidzitExAl0YgX94TDH/xL4OxEpF2sY6r8BvzLGeM+xPZ8GPmZ3JOdCoD+lfKwDjDE+rC/hbwUdUyQi/n6IR4C/FJHLxZIsIreKiAer/8UL/I2IuETkfVj/JoL9HlgNfAqrBq8uIA3us5AxpgnrP98/GmOqgU3A57CCczXw/7D/bdhZ4lvAQWPMoH2KHcApY0yjvc8h4Jv29gZgBdav6ON5AKvsUI/1m8SPImj6S1gdwVuBbxhjnrO3/yewBXhORLqA17G+fCajUEaPcw/uMPyp3c56rFEif2Nvz8eqL3dilYNewvrCHOmH9jleBk5gdSb/9STbGGCMeRW4Aes3qqNBJZZtwH+Nc+hnsD7D10WkE3gB68sbY8wu4BNYHb1t9n732K8NYv2mcI/92gexvqCD29SH1clbPvI1Nf0ktISm1MVHRMqwAmLceWS65/P+24CfGWO+P93vPdOIyD8BC40xH5lwZzWl9KYGpVRUiDX2/+PAnRe6LUrLMkqpKBCRT2CV9J42xrw80f5q6mlZRimlYpBm7kopFYMuipp7dna2KSsru9DNUEqpGWX37t3NxpiccK9dFMG9rKyMXbt2XehmKKXUjCIiY97ZrWUZpZSKQRrclVIqBmlwV0qpGKTBXSmlYpAGd6WUikEa3JVSKgZFFNxFJF1Efi0iR0TksIhcKSKZIvK8iFTaf2YE7f+giFSJSEXQdKJKKaWmSaSZ+38CzxhjFmOtrH4Y+Cyw1RizAGsK1s8C2HNz3wEsw1qO7SF7ruuoO9Pex1eePkJ9R/9UnF4ppWasCYO7iKRizRn9A7DmdTbGtGPNAf6ovdujWAsUY29/zBgzYIw5gTUn9MhJ/aOid8DL/7x0jK1HJlzMRimlZpVIMve5WIs4/EhE3haR79urr+cZY+oA7D9z7f2LCF23scbeFkJE7hORXSKyq6mp6ZwaPz83hZLMRF483HhOxyulVKyKJLi7sJbO+m9jzKVYq6d/dpz9Jcy2UVNPGmMeNsasMcasyckJOzXChESEjYvzeLWqmb7BcCubKaXU7BRJcK8Baowxb9jPf40V7BtEpADA/rMxaP/gxYSLObuQcdRtXJLLgNfH9mPNU/UWSik140wY3I0x9UC1iCyyN20EDmGtWelf1Phu4An78RbgDhFxi0g5sADYGdVWB1lXnklyvJOtR7Q0o5RSfpHOCvnXwM9FJB44jrXyvAN4XEQ+DpwG/gzAGHNQRB7H+gLwAvfbq91PCbfLyTULc3jxcCPmvQaRcFUhpZSaXSIK7saYPcCaMC9tHGP/zcDmc2/W5NywOJenD9RzsLaT5UVp4+7b0TeEQ8CTEDdNrVNKqekXE3eoXr84FxHYGsGomQd+8RYf+f4b+Hy6vKBSKnbFRHDPTnGzqiSdFyMY797UNcDemg7+sG/K+niVUuqCi4ngDvCOJXnsremgsWv8u1X7h6zy/zeeq2DAq8MnlVKxKWaC+/WLrHuoXqsaf0jkgNfHnMwkqlv7+MUbp6ejaUopNe1iJriXZCYC0Nw1OO5+/UPDXLMwm6vmZ/NfL1bR1T80Hc1TSqlpFTPBPTnehQgTBusBr48El5PP3LyY1p5B/vH3B3j5aBN1HX0Yo52sSqnYEOk494uewyGkxLvo7PeOuY8xhv6hYdxxDlYUp3HXlaX8ZMcpfr/H6lx1OiQwd0JJZhLP/d01xDlj5vtPKTWLxExwB/AkuOgaJ7gPDRt8BhJc1gzE/7JpOZ/auIDKxm4qG7qo77Q6Y0809/DU/npONPewMM8zLW1XSqloirHgHjduWcY/OiYh7uz08lkpbrJS3FwxNyuw7VBtJ0/tr6eivuu8g3tVYxdJ8S4K0xPP6zxKKTUZMVVz8CS46B4YO3PvH/IB4I4b/7Ln5iTjdAiVDV0h2+s7+rnj4R0TDrcM9omf7OYLWw5GvL9SSkVDzAX38coygczdNf7CUAlxTkqzkqgYEdxfPNLI68db2VvdEVF7ega8nGju4VBdZ0T7K6VUtMRYcB+/LBNp5g6wKM/D0YbukG37atoB6y7XSFQ1WsfXtPXRM85vFEopFW0xFtwjy9zdE2TuAAvzPJxq6Qnc0Qqwt8bK2CMN7sGZf2Vj9zh7KqVUdMVUcE+ZILhPJnNfmOfBZ85m3/1Dwxy1g3Vzd2TB/Wh9V9jHSik11WIquKcmxDE47AvJtoNFWnMHWJSfAhAI6AdrOxm2Z5KMNHM/2tjNkoJUEuIcgfMopdR0iLGhkNbldA94Q4Y7+g1MInMvzUomzimBuru/3j43J5mmSWTu6+dl4XQwqnNWKaWmUkxl7v7gPlZpZjKZe5zTwbyclEDGva+mg1yPmxVFaRFl7h29Q9R39rMw38PCPI9m7kqpaRVRcBeRkyKyX0T2iMgue9sXROSMvW2PiNwStP+DIlIlIhUictNUNX4kj9taXWmsETOTqbkDIUF5X007K4vTyElx09Q1MOE8NEcbreMW5XlYlOehoXOAjl6dpEwpNT0mU5a53hgzcj7dbxljvhG8QUSWAncAy4BC4AURWTiV66j6RZy5hynZhLMwL4Ute2up7+jneHMPm1YV4XY56BsapmdwmBT32B+f/0thQV7K2W2NXawty4zovZVS6nxMRVlmE/CYMWbAGHMCqALWTcH7jJISCO7jZ+4Jrsgzd4DfvX0GY7Ayd48bmLhT9Wh9F8nxTorSE1mYb52nQkfMKKWmSaTB3QDPichuEbkvaPsDIrJPRH4oIhn2tiKgOmifGntbCBG5T0R2iciupqamc2r8SKn2otdjzQzpH0XjjjBzX2QH5V/vti5nZXF6ILhPNByyoqGLhfkeRITCtARS3C6tuyulpk2kwX2DMWY18C7gfhG5BvhvYB6wCqgDvmnvK2GOH1WgNsY8bIxZY4xZk5OTM+mGhzNxWWZymXtJRhIJcQ6ONfVQnJFIZnJ8xJl7ZUM3C3OtLwcRYUFeigZ3pdS0iSjKGWNq7T8bgd8B64wxDcaYYWOMD3iEs6WXGqAk6PBiYFpWo/bXwLvHydydDsEV4RztDoewwA7QlxSnA9Zi3DB+cG/uHqClZzBQjgGrY7WivksXBFFKTYsJo5yIJIuIx/8YeCdwQEQKgna7HThgP94C3CEibhEpBxYAO6Pb7PBcTgdJ8c4xa+7WKkyT62bwd4iuLE4DICMpHqdDxg3u/rtRFwVNF7wwz0Nb7xDN3eMvA6iUUtEQyWiZPOB3IuLf/xfGmGdE5Kcisgqr5HIS+CSAMeagiDwOHAK8wP3TMVLGb7z5ZaxVmCKrt/v5A/QKO7g7HUJWcvz4wd0uvywMGinj75ytbOgKlHb82nsH6R7wUpyRNKm2KaXUWCYM7saY48AlYbbfOc4xm4HN59e0c5PidtE1EL3M/daVBZxq7eWy0ozAthyPe9y7VCsauklPigsJ4gvt6QwqGrpYPz+b9t5BnjvYwJP769he1UxinJO3/+nGiEtGSik1npiafgD80/5GL3Mvzkji325fEbItx+OeMHNfmGeNlAkck+ImIymOP+yt5aWjTbxa2YzXZ5iTmcTaskx2HG+hoWuAIl2xSSkVBTEY3MdeJHvA68M9ycw9nJwUN0fqwo98GfAOc6Suk/etLg7ZLiIsK0zj1apmijMS+fjV5dy6ooAVRda2HcdbqG7t1eCulIqKmAvuqQlx1Lb3hX3tXDL3cHI8blp6BvD5DA5H6MjPbRVN9AwOs3FJ7qjjvvFnl9DcPcCywtSQrN5fa69pC99upZSarJgL7uN1qJ5LzT2cHI+boWFDR98QGcnxIa9t2VtLVnI8G+ZnjzouPy2B/LSEUdsL0xMQgerW3vNum1JKQYzNCgkTBPeh4YjnlRlP4EamEZ2q3QNeXjjUwK0rC4ibRMeo2+UkPzVBM3elVNTEXHBPccfRNzTM0LBv1GvRrLnD6BuZnjtYz4DXx6ZVhZM+Z3FGItVtmrkrpaIj5oJ7YMGOMNl7f5Qy9+wxpiB4Yk8txRmJrJ6TEe6wcZVkJHFGM3elVJTEbHAPV5qJWuYeJrg3dw/walUz77mkMKSzNFLFGYnUdfSF/Y1DKaUmKwaDu71gR5gbmaKVuXvcLtwuR0jN/an9dQz7DJtWjZoAMyLFmUn4DNS19593+5RSKuaCe+o4mXv/UHQydxEhx+OmOShz37Kn1lp1KWiysMkozrDGt2vdXSkVDTEX3AOZ+4jgboxhwBudzB1CpyCobOhi16k2Nl06+Y5Uv5LAWPezwb26tZd7frRzzHH7Sik1lhgM7uFXYxoaNvgMUcncgcBaqgAPv3ychDgHd6ydc87nK0hLwOkQqlvPBvI/7q9jW0UTD/52v04VrJSalJgL7iljlGUmu37qRPzzy9R39PP7PWf44JoSMkfc0DQZLqeDgrSEkMz9zROtOB3CS0eb+N/dNdFotlJqloi54D5W5u5fP9UdF6XM3eOmtXeQR145zrDPcO/Vc8/7nNZYdytz9/kMu0618f7VRVxensmXnjxEXYeWZ5RSkYm54O52OYl3OegaGCNzd0UvczcGfrrjFLesKKAk8/znYi/JSApk7kcbu+joG+Ly8iy+9oGVeIcNn9PyjFIqQjEX3MEaMTOyLBP1zN2+S3Vw2Mcnr5kXlXMWZyTR0DlA/9AwO0+0ArCuPJPSrGQ+ffMi/lTRxCuVzVF5L6VUbIvJ4B5uTnd/5u6OYuYOsH5eVmCVpvNVkmkNh6xt72PniVbyUxMCQyQ/tG4ObpeDl442ReW9lFKxLaLgLiInRWS/iOwRkV32tkwReV5EKu0/M4L2f1BEqkSkQkRumqrGj8WaPCx8zT0hSpn7vNwUlhSk8n9vXBiV88HZqX+r2/p482Qra8szA3e7JsQ5WVOWwfZjLVF7P4An9pxhw1depH9o2lZCVEpNg8lEuuuNMauMMWvs558FthpjFgBb7eeIyFLgDmAZcDPwkIhEJ12OUIp7dFkm2pl7akIcT3/qataUZUblfHA2c99+rJmGzgHWlYXOUbN+XjaH6zpp7ZncItu9g17+uK+O14+P/mL405FGzrT3cayp+9wbrpS66JzPfO6bgOvsx48C24DP2NsfM8YMACdEpApYB+w4j/eaFE+Ci+bmnpBtA1HO3KdCnieBOKfwxNu1AKwrzwp5ff086/mOYy3curJg3HMZY9hW0cSvd9ew9UgD/UM+cjxudn5uY8jcN/tqOgBracBlhdEpLymlLrxII50BnhOR3SJyn70tzxhTB2D/6V96qAioDjq2xt4WQkTuE5FdIrKrqSm6deTpqLlPBYdDKEpPpL6zn7TEOBbkpoS8vqIojRS3i+3HQjtVX61sZuvhBqpbe/H5DFsPN/Ce77zGR3/8Jm+caOHPLivh7itLaeoa4HTQgiCd/UMct78EjzaMnbnXdfRx+0Ovsb1KO3OVmikizdw3GGNqRSQXeF5Ejoyzb7gpEUeN3zPGPAw8DLBmzZqoju/zJLhGTfkb7Zr7VCnJTOJkSy9ryzJGLeHncjq4vDwzpO5e2dDFnT98A/8IyTinMDRsKMlM5GsfWMntlxYR53RwuK6TR3ecYvepNkqzkgE4YGftInC0PvyasAD//txR3j7dzoO/28+zf3tN1G4EU0pNnYiCuzGm1v6zUUR+h1VmaRCRAmNMnYgUAI327jVASdDhxUBtFNs8IU9CHN2D3pA1Tv0dhtFYQ3Uq+UfHrB2jlr9+fjZbjzRS295HYXoi336xiqQ4J9+7cw3Vbb0ca+xmYb4nENT9FuZ58Lhd7DrVFli8e68d3K+an01FQ/jgXlHfxW/eqmFdeSY7T7TyyMvH+euNC6J5yUqpKTBhGisiySLi8T8G3gkcALYAd9u73Q08YT/eAtwhIm4RKQcWADuj3fDxpCa4MAa6B89m7wNeO3OP0twyU8U/YmZt+RjB3a67bz/WQlVjF0/uq+Wu9WVctSCbD62bw+dvW8qfrykZtcyf0yFcWprB7pNtgW37z7QzJzOJdWWZ1LT10TMweibNrz5zhGS3i+995DJuWZHPd7dVhUyRoJS6OEUS6fKAV0VkL1aQ/qMx5hngK8CNIlIJ3Gg/xxhzEHgcOAQ8A9xvjJnWcXYp7tHzy8yUzP3WFQV84upyVhaF79xclOchKzme7cea+fbWKhLjnHwiwqkP1pRmBO58Bdhb3cHK4jQW2tMUVzaG1t1fP97Ci0ca+T/XzSMjOZ7P37oUQfjSk4fO4wqVUtNhwrKMMeY4cEmY7S3AxjGO2QxsPu/WnaOz0/4OAVaZY6Zk7mXZyfzDrUvHfN3hEK6Yl8ULhxroGvDyyWvmRTxh2ZrSDIyBt063sbIojTPtfdy9vpRFeVZwP1rfxaqSdMAabfPlp4+Qn5rAxzaUA1CYnsgDN8zn689W8FpVMxvmZ5/fxSqlpszFHenOUbil9vqHhnE5BJdz5l/yhnnZdPZ7SXA5+cTV5REfd0lJOk6H8NapNvadsertK4vTKclMIiHOEVJ3f62qhb3V7fzdjQtCOlDvvbqcpHgnzx9qGHX+vsHhwKgkpdSFdT7j3C9a4RbJjtb6qReDq+yM+a71pWTZc9xEItntYkmBh10n23A5HIjA8qI0nA5hfm4KR4OC+5P7aklxu0YtG+h2OVmc7+FQXeeo83/kB28wPyeFr35g5TlemVIqWmI0uFtlmc6gKQiitX7qxWBOVhK/+6v153TT0ZrSTH71ZjXxLgfzclIC/RML8zy8Zo9jHxr28ezBejYuyQ37mS0tTOWJPbUYYwI3RPUOenn7dNuoIaiTEXw+pdT5iY1UdoRw66jGUuYOcOmcDOLP4XouK82gb2iYVyqbWBk04dmiPA8NnQN09A7x+vEW2nqHuGVF+Ltglxak0dXvpabt7PzyB8504jNwoqWHYd/kb1vYU93O4n98htMtOhJHqWiInWgXJNxqTLGUuZ+PNfZ8NT4DlxSnB7Yv9HeqNnbx1P46kuOdXLswJ+w5lhRY+waXZvbVtAMw6PVxpm3yi4q8VtXMgNfH29VtE++slJpQTAb3xDgncU6hve/sBFsDXt85ZbqxpiAtkcK0BICQqYr9wyEP1Xby7MEGNi7JG/PLcHF+Kg6x9vXzz1EDcKx58pOQ+c91dIybqZRSkxOT0U5EyEp209p9Nrhr5n7WZWWZuBzC0oLUwLbCtARS3C5+9vopWnsGuWVF/pjHJ8Y7Kc9ODsnc95/pYK39W8Hxpp6xDh3TgVrry6FynDlulFKRi8ngDpCVEh8yNW6s1dzPx9++YwHf+fClIV92IsLCvBQqG7tJindy3aLccc4ASwvTOGwH946+IU4093DdolzSEuM4Psnpgzv7hzhl19pH3killDo3MRvtslLcNAcHd83cA+blpHDz8tGdpf66+w2Lw4+SCbakwENNWx8dfUPsr/GPmU9jXk7ypDP3w3ZJZmVxGqdaenThEKWiIHaDe3I8Ld0Dgef9Q5q5T8Qf3G8dY5RMMH9J53BdJ/vOtAOwsiiduTkpoxb++M3uGj73u/1jnuugHdzfu6oInzm3so5SKlTMRjsruAeXZTRzn8htKwv45LVzuWHJ+CUZsMa6g9URuq+6g9KsJNKS4pibk0xj10DIMoeP7jjJL944HdIBG+xAbQc5HndgOoPKRu1UVep8xW5wT3HTNzRMrz0zZP+Q76Kfy/1Cy01N4MF3LYloQZNcTwLZKW4O1XWyr6adlfawyrnZ1gIjJ+xFQDp6hzhgT3Xw2Junw57rUG0nywpTKctOwumQiDtVm7sHws5kqZSK6eBuTablz94HvMMX9SpMM9HSwlS2VzVT29HPJfawynk51kIg/tLK6yda8BmYm53M7946Q99gaD29f2iYysZulhWm4nY5KctKiihzN8bw3u++xpefPhzlq1IqNsRscM/2B3e7U1Uz9+hbUuChtqMfIJC5z8mysm//iJkdx1pIiHPwL5uW0zXg5cl9oeu2VNR3MewzLLenUliQ64kocz9U10lNW9+4ywMqNZvFbLTLSrYm1GrpHsAYo5n7FPB3qjoEltk1eLfLSUlGIsfszP21qmbWlmWyYX4Wc3OS+eXO0NKMvzPVP0/OwrwUTrb0TDi75LYKa93d2vbJ3w2r1GwQs8HdP8d5S/cgQ8MGn7n410+dafwBfX5uCsnus3PQ+UfMNHb1U9nYzYb52YgIH143h7dOt1MRtF7rwdoOPAkuSjKteffn53msOWqaxx8x89JRK7jXd/Sf01w2SsW6mI12/pp7c89AIAvUzD26yrNTSIp3Bhb48JubnczJlp7ALJP+pQHft7qYeKcjJHs/aHem+meDXJhndciOV27p7B9i96k2slPceH2Gxq7+aF6WUjEh4uAuIk4ReVtEnrSff0FEzojIHvvnlqB9HxSRKhGpEJGbpqLhE0mKd5EU76S1e5D+IXsVJs3co8rpEH768cv5+3cuCtk+NyeF/iEfv9l9htQEV6Dkkpkcz83L83l8VzVP7DmDd9jH4brOkKmLy7OTcTqEqnHmmHmtsplhn+HP11gLfZ/LRGVKxbrJRLtPASOHJnzLGLPK/nkKQESWAncAy4CbgYdE5IKkzFkp8bT0DGrmPoUuK80gLzUhZNtce8TMq1XNXDE3C6fj7Bztn3nXYhble/jUY3v48CNvMOD1Bco7YP0dlWYljZu5b6towuN28e5LCgE4o3V3pUaJKLiLSDFwK/D9CHbfBDxmjBkwxpwAqoB1597Ec5eV7Ka5eyCQubs1c58W83JSAo9HrrNalJ7Ir/9yPZ+7ZTF77GmCRy46siA3ZczhkMYYXjraxFULspmTmQRocFcqnEij3X8AnwZ8I7Y/ICL7ROSHIpJhbysCqoP2qbG3hRCR+0Rkl4jsampqmmSzI5OdYt2lqpn79MpOiQ8sdeivtwdzOoT7rpnHU39zNV97/8pAnd1vYZ6Hky29YUfMVDR0Ud/Zz3WLckh2u0hPitOyjFJhTBjcReQ2oNEYs3vES/8NzANWAXXAN/2HhDnNqOEMxpiHjTFrjDFrcnLCLwpxvjKT42npGdCa+zQTEebmpJDjcTM/N2XM/ebnpvDna0tGLa03PzeFYZ/hZPPoVZn8QyCvXWhNkVCUnqjDIZUKI5I1VDcA77E7TBOAVBH5mTHmI/4dROQR4En7aQ1QEnR8MRB658o0yUpx09ozGJhlUDP36fN/b1xI3+DwOa2J6p/ArKKhi0X2IiJ+2yoaWZzvId9ecKQwPZFTLTrRmFIjTZjKGmMeNMYUG2PKsDpKXzTGfEREgqcOvB04YD/eAtwhIm4RKQcWADuj3O6IZCXHMzRsaLZnh9TMffpcuzCHm5ePveDHeObmWCNmKupDJxob8A6z+1Qb1wQt/1eUnsiZtj6MmXis+9Cwb9SMlUrFqvOJdl8Tkf0isg+4Hvg7AGPMQeBx4BDwDHC/MeaCTNCdnWLdpepfyFlnhZwZ3C4nc7OTqagPDcSVDd0MDZuQhb2L0hPpGRyms2/iCcT+7anDvOs/XqEzaMZKpWJVJGWZAGPMNmCb/fjOcfbbDGw+n4ZFg/9GJn9NVudznzkW5XvYa4+m8fMv6xe8PGBRhnVna017L2lJoaNugp1u6eVnr59iaNhwqrk3ZP1YpWJRTEc7//wy/uCumfvMsTjfQ3VrH91BU/oeruskKd5JaVZyYFthuhXca9vHv0v1689V4LWnKTjVqjV6FftiO7gHMnfrP75m7jPHonwrOz8adKfqodpOFuV7Qm6KKrKD+5m20SNr/PbVtPOHvbV8bEM5QGC9VqViWUxHu4wkK7if0cx9xllsj5LxTzJmjOFQXWdISQasTvN4lyMw9fBIxhi+/NQRMpPj+dt3LCDH49bRNWpWiOngHu9ykJYYF/jVXjP3maMoPZHkeGcguNe09dHV7w0s7+fncEhgxEw4Lx1tYsfxFv7mhvl4EuIoy0ripGbuahaI+WjnL824HILLGfOXGzMcDmFhvocj9nDIw3Zn6pIRmTvYwyHHuJHp0e0nKUhL4MOXlwJQmpWsmbuaFWI+2mXbnaqatc88i/M9VNR3BUoyImfLNcEK0xPCBveW7gFermxm06oi4u2//9LMJBo6B0Yt96dUrIn5iOfP3LXePvMsyvPQ1jtEU9cAh2o7Kc9OJil+9OjdovQkmroGAnci+z11oJ5hn2HTqsLAttJsa6TN6VYtzajYFvPB3b8ik2buM89CO0s/Ut8VtjPVrzDdmoqgfkSn6pY9Z1iYlxKS7ZfaM0lqaUbFupiPeFn2Xaqauc88i+3hkLtOtlLT1he23g5nb2QKLs2cae/jzZNtbFpVFDK/TZk9Rl6HQ6pYF/PBPdsuy7g1uM84mcnx5Hjc/H6PNe/cyJEyfoGx7kHB/Q97rWPec0lhyL5pSXGkJcbpjUwq5sV8cM/SDtUZbXG+J1AfXzZG5p6floBI6HJ7T+ypZfWcdErsMkywsqwkzdxVzIv5iHe2QzXmLzUmLbKn/81OsbL4cNwuJzkpbo439zA07ONoQxeH6zrZtGrUGjEAzMlKPqfg3tI9wNDwyPVqlLo4TWrisJkoK9ChqmWZmcg/n/uSgtRx54Yvz07mD3treeZAHWmJcTgdwi0rCsLuW5aVxB/31TLo9QWGSE6ksaufd3zzJZYVpvGzey8PmQIh2IB3mP5BH2lJcRGdV6mpEvPp7NkO1Zi/1Jjk71Qda6SM37c/dCnf/LNL+NhV5awsTucvr507ZqZfmpWMz0xu7dWvPHWE7gEvO4638O2tlWPu97nfHuCWb7/CsG/i+eWVmkoxn7mnJ8bhEM3cZ6pF+R5uWpbHrSvDZ+F+eakJvP+y4ojOWZp1djhkeXZyyGv9Q8M8tO0Yt64oCPzW8ObJVn779hnuv34ede39fPvFSi4vz2T9iMW/Gzv7eWLPGbw+w57qNi4rzYz0MpWKuphPZx0OIT81gbRE/TV5Jop3OfjenWtYWZwetXOeDe6j6+6PvHycb2+t5H0PvcbWww14h3380xMHKUxL4P7r5/Ol9y5nbnYyn/rVHpq6BkKO/fkbp/H6DC6H8OzBhqi1V6lzEfPBHeAH96zlrzfOv9DNUBeJnBQ3SfHOUcG9pq2X726r4tqFOZTnJHPvT3Zx7092cbiuk3+8bSlJ8S6S3S6++xer6ewb4u9+tSdQfhn0+vjFztNctyiHDfOzefZgfURL/yk1VSIO7iLiFJG3ReRJ+3mmiDwvIpX2nxlB+z4oIlUiUiEiN01FwydjSUEquZ6EC90MdZEQEeZkJo26S/VLTx5CEP7tfSt4/JNXcvOyfLZVNHHV/OyQ9WAX56fyxfcs49WqZh76UxUATx+oo6lrgHvWl/HOZXmcaunlaIOu16ounMlk7p8CDgc9/yyw1RizANhqP0dElmItpL0MuBl4SES04K0uKqVZSZwKml/mpaNNPHuwgQdumE9ReiJJ8S6+++HV/NeHLuVbH1w1aqTOB9eWsGlVId964SivH2/hx9tPUp6dzDULcrhxaR4i8NzB+um+LKUCIgruIlIM3Ap8P2jzJuBR+/GjwHuDtj9mjBkwxpwAqoB1UWmtUlFSlpXM6dZe6jr6eKWyiS9sOUh5djL3Xl0e2MfhEN59SWHYUTciwubbV1CWlcwnf7qbt0+3c9eVpTgcQq4ngUtL0nn2kAZ3deFEmrn/B/BpIPgOjjxjTB2A/Weuvb0IqA7ar8beFkJE7hORXSKyq6mpabLtVuq8lGYlM+j1ceWXX+TOH+ykpq2Xf9m0bFKjqlLcLr7z4dX0DQ2THO/kA0GjdW5als+BM52TGm6pVDRNOBRSRG4DGo0xu0XkugjOGe7ujlE9S8aYh4GHAdasWaM9T2pa3bw8n9OtvRSmJzA/N4XF+amBGUQnY2lhKj+6Zy39Q8N4Es6OyHrnsny+/PQRnj9Yzz0bysc5g1JTI5Jx7huA94jILUACkCoiPwMaRKTAGFMnIgVAo71/DVASdHwxUBvNRit1vjKT4/nsuxZH5VwbRox3B+uO2QW5KTx7sEGDu7ogJizLGGMeNMYUG2PKsDpKXzTGfATYAtxt73Y38IT9eAtwh4i4RaQcWADsjHrLlbrI3bg0j50nW+kd9F7opqhZ6HzGuX8FuFFEKoEb7ecYYw4CjwOHgGeA+40xuqaZmnUunZPBsM8E1n9VajpNavoBY8w2YJv9uAXYOMZ+m4HN59k2pWa0FUVpAOyv6dCpCNS0mxV3qCp1IeSluslOcbP/zPln7v1DwzrdsJoUDe5KTRERYUVRKgfOdJz3uf78ezvY/MfDE++olC3mZ4VU6kJaUZTGS0eb6BscJjE+/Bj6jr4hjjZ0cbShi+rWPu68sjSwdCBYWfv+Mx14h3XEsIqcBnelptDyojR8Bg7VdXJZaUbIa6daevjOi1X89u0zIfO/u10O/u7GhYHnx5q6MQaqGrsZGvYR59RfuNXENLgrNYVWFFudqgfOdASCe9/gMF/YcpBfv1WDyyHceUUp1y7MYUFeCnf9YCdH6kNr9JX2BGSDwz6ON/UE5pkHqKjvoqVngPXzRo+1V7ObBnelplB+agLZKfHsD6q7//yNU/xqVzX3rC/jr66bR27q2RlLlxSkcqA2tEZf2dgVeHykvjMkuH/pyUOcaO7htc/eMIVXoWYi/f1OqSkkIiwvSgt0qhpj+MXO06yek84X3rMsJLADLM73cKqll56Bszc+HW3opiwriXing0NBY+a9wz7ePt1GXUcfg14dSaNCaXBXaoqtKEqjsrGb/qFh3jzZxvGmHj60bk7YfRfba8UeqT+brVc1drOkIJX5uSkcqQvO4rvoGRzGZ6C+o39qL0LNOBrclZpiy4vSGPYZDtV18sudp/EkuLhtZWHYfRfbJRd/3b1/aJhTLT0syPOwuMATcrfr7lNtgcc1baOXDFSzmwZ3paaY/07VVyub+eP+Om6/tGjMYZHFGYl43K5Ahn68qQefgQW5KSwtSKWxa4CWbmvt1l2n2nC7rP/CNW06tbAKpcFdqSlWkJZAVnI833vpGINeH3esDV+SAatGH5yh+ztTF+Z5WJxvlWwq7JLNW6fauG5RDg7RzF2NpsFdqSnm71TtGRxmVUk6SwtTx91/cX4qR+q7MMZQ2dCN0yGUZSexuMAq2Ryq66Suo48z7X1cXp5FfmqCZu5qFA3uSk0Df2nmw2N0pAZbUpBK94CXmrY+jjZ0UZqVhNvlJDvFTY7HzZH6LnadtOrta8oyKM5IokZXfFIj6Dh3pabBbZcUcLy5m9suKZhwX3+Gfriuk6rGbhbmnR3Xvjjfw5H6TlLcLhLjnCwpSKU4I5E3TrROWdvVzKSZu1LTYHF+Kg/9xWUkxU+cTy2yg/nemnZOtvSwIC8l8NqSglSONnTzxolWVpWkE+d0UJyRSF1Hn84aqUJocFfqIpPsdlGalcTT++utkTJBmfuSAg+DXh+Hg+aqKc5ImnCsuzE66dhso8FdqYvQkvxUjjf3ANYwSD//iBmAy8r8wd2aQbJ6jBEz26uaWfmF52jo1BudZpMJg7uIJIjIThHZKyIHReSL9vYviMgZEdlj/9wSdMyDIlIlIhUictNUXoBSschfd3cIzM1JDmyfl5NCnFMAWD3HCu5FdnAfa8TMr3ZV0zXgZV/N+c8rr2aOSDpUB4AbjDHdIhIHvCoiT9uvfcsY843gnUVkKdZC2suAQuAFEVmo66gqFTl/hl6WlYzbdfaGp3iXg3k5KfiMIS0xDoCCtERE4EyY4N4/NMzWw42ANWb+xqV509B6dTGYMLgbq1jXbT+Ns3/GK+BtAh4zxgwAJ0SkClgH7DjPtio1ayy155gJ7kz123z7ckACz+NdjjHHur98tIluexKyqsbuUa+r2BVRzV1EnCKyB2gEnjfGvGG/9ICI7BORH4qIfyWCIqA66PAae9vIc94nIrtEZFdTU9O5X4FSMag4I5GSzEQuL88a9dplpZmjFv4ozkgMe5fqU/vrSE+K4/LyTI5pcJ9VIgruxphhY8wqoBhYJyLLgf8G5gGrgDrgm/buEu4UYc75sDFmjTFmTU5Ozjk0XanY5XAIL/+/6/nYVeUR7V+ckTQqc+8fGuaFw43ctDSfJQWpVDV266iZWWRSo2WMMe3ANuBmY0yDHfR9wCNYpRewMvWSoMOKgdrzb6pSs4tIuDwpvOKMROo7+/EGjXV/pbKZ7gEv71qRz7zcFHoGh6nTqYFnjUhGy+SISLr9OBF4B3BERIJvtbsdOGA/3gLcISJuESkHFgA7o9pqpVSI4oxEhn0mJHg/tb+OtMQ4NszPDgynHKvu/vrxFh74xVt858VKuvqHpqXNampFMlqmAHhURJxYXwaPG2OeFJGfisgqrJLLSeCTAMaYgyLyOHAI8AL360gZpaZWUXoSAGfa+yjJTGLAO8wLhxq4eXk+cU4H8+3gXtnYzTULz5ZBd51s5RvPVfD68VZSE1w8ua+O7796gk9cPZe715eR4tYZSmaqSEbL7AMuDbP9znGO2QxsPr+mKaUiVTxirPufjjTRNeDllpXWL9hZyfGkJ8WFZO6nWnr44MOvk5kczz+/eykfWjeHyoZu/uOFo3z92QoeeeX4tAX5h7ZVceXcLC6dkzHxzioieoeqUjGgID0Bsed1b+sZ5It/OEhpVhIb5mUDVv1+fk5KyIiZ5w81MOwz/Pb/rOejG8pJiHOyojiNH9yzli0PbGD1nAy+/mwFV3/1RbYebpiytvcPDfO1Zyr4zVs1U/Yes5EGd6VigNvlJM+TwOnWXv7+f/fS0j3Idz60mnjX2f/iC/JSAot/AGw93MiiPA8lmUmjzreyOJ0f3rOWJ+7fQFK8i++/cmLK2u7/baOpa2DK3mM20oKaUjGiOCORJ/fVMej18cX3LGNFcVrI6/NyUmjrHaKlewCX08GbJ1u575q5457zkpJ0VpWkc7i+c9z9zod/ThwN7tGlmbtSMaI4I5FBr4+bl+Vz15Wlo16fHzRi5uWjTXh9ho1Lcic8b26qm8bOqQu81a1WcG/uHpyy95iNNHNXKkasn5/NieYevvqBlWHHyPunDq5s7GbXyVYyk+NZVTJxB2ZeagLdA166B7xT0rHqD+5NXQMYYyY1vl+NTYO7UjHiz9eU8OdrSsZ8vTAtgaR4J0cbuth2tIkbFufidEwcSPNTEwBo6OwnJWf0XDfnq7rVqrn3DQ3TMziswy+jRMsySs0SIsK8nBT+sLeW9t4hNi6ObIbI3FQ3wJTNBx88D73W3aNHg7tSs8iCXKtT1eUQrlmYHdExeUGZ+1Sobu1lbrY1Z70G9+jR4K7ULDLP7lS9fG4mnoS4iI45G9zHDrzbKhq599FdDHont45rR+8Qnf3ewM1LGtyjR4O7UrOIf8TMDRGWZABS3C5S3K5xM/df7jzNC4cbeHLf5OYI9JdkVpemA9DUpRObRYsGd6VmkQ3zs7nrylLed+moJRbGlZvqHjO4e4d9bK9qAeDhl49Palph/0iZlUXpOB1CU7dm7tGiwV2pWSTF7eJfNi0nIzl+UsflpyaMWZbZW9NB14CXjYtzOVLfxUtHI198x5+5z8lKIis5nuYuHeseLRrclVITyktNGDNzf6WyCRH48vtXkJ+awPdeOh7xeatb+0hLjCMtMY4cj1sz9yjS4K6UmpD/LtVwJZdXK5tZWZRGrieBj19Vzo7jLeyraY/ovKdbeynJtGa0zPG4tUM1ijS4K6UmlJ+awOCwj7be0IU8OvuHeLu6nasWWMMq71hXgsft4qE/HaOpa4CmroFxF/+obuulJMOauCwnZfzgboxhaHhyo3FmM70VTCk1oeCx7plB9frXj7Uw7DNcvcBaAMSTEMdfXFHK/7x0jGcO1gPgEHj6U9ewKN8Tck6fz1DT1sc7llgjd3I8bpq7B/D5DI4Rd872Dnq599FdNHUN8MQD1kyVanz6CSmlJpRn36Va39nPkoLUwPZXKptJineyOmiRjQdumE9ZVhJDPsPA0DD/+sfDvFLZNCq4N3UPMOj1UZJxtizj9Rna+4ZCvkB6B7189Edv8ubJVnwGvvZMBV94z7KpvNyYEMkaqgkislNE9orIQRH5or09U0SeF5FK+8+MoGMeFJEqEakQkZum8gKUUlPPn7k3juhUfbWqmSvmZoXMG5/idnHHujnceUUp9149l9KsJHaeaB11Tv8wSP988jke6wskuDTTO+jlYz+2Avu3PriKu68s5dEdJ8OeT4WKpOY+ANxgjLkEWAXcLCJXAJ8FthpjFgBb7eeIyFLgDmAZcDPwkL3+qlJqhvIH3vqOs4G3urWXE809XDV//GkM1pZl8ubJ1lGdsadHBveU0cH9878/wM4TVmDftKqIT9+8mOKMRD796730DerSzOOZMLgbi39trjj7xwCbgEft7Y8C77UfbwIeM8YMGGNOAFXAumg2Wik1vdwuJ5nJ8TQE3UH6alUzwIRz1Kwry6Stdyhk/VY4OxtkUfrZsgxAsz0c0hjDn440cvulxWxaZd10lex28dX3r+RkSy9ff7YiClcWuyIaLSMiThHZAzQCzxtj3gDyjDF1APaf/ln/i4DqoMNr7G0jz3mfiOwSkV1NTZHf9KCUujDyUhNCyjIvH22iIC2BeRNMA7y2PBOAnSdDSynVbb3kpbpJiLN+sR9Zlqlu7aOtdygwNYHf+nnZfGhdCY/uOEl7r970NJaIgrsxZtgYswooBtaJyPJxdg83QfSowbHGmIeNMWuMMWtycnIiaqxS6sLJS3VTbwf3oWEfr1Y2c+3CnAkX1yjLSiI7xc2bI+rk1a29zAlavzXF7SIhzhG4kent6jYALilOH3XOD66dw7DP8OKRxvO5pJg2qXHuxph2YBtWLb1BRAoA7D/9n3INELxiQDEwudmElFIXnTzP2SkI3j7dTteAl+sWTZyYiQjryjN482RbyPbq1rNj3P37ZQeNdd9b3UFCnGPUKBuAlUVp5Kcm8NzBhvO5pJgWyWiZHBFJtx8nAu8AjgBbgLvt3e4GnrAfbwHuEBG3iJQDC4CdUW63Umqa5aUl0Nw9gHfYx7aKRlwOYf0Enal+68oyOdPex5l2q87eO+ilvrOf4qDMHULvUt1T3cbywjTinKPDlMMh3Lg0j5eONtE/pB2r4USSuRcAfxKRfcCbWDX3J4GvADeKSCVwo/0cY8xB4HHgEPAMcL8xRj99pWa4vFQ3xljj07dVNLG6NIPUCOeE99fd/aWZbzx7FJ+Ba0d0xvrvUh30+jhQ28mqkvQxz3nTsnz6hoZ5pbL53C4oxk14E5MxZh9waZjtLcDGMY7ZDGw+79YppS4aeR5rrPv+mg4O1XXy6ZsXRXzs4vxUPG4Xb5xopSgjkR9tP8FdV5ZyWWlmyH45Hje7TrVRUd/FoNfHqjnpY57z8rmZpCa4ePZgPTcujXx++tlC71BVSkUkP80K7v+7uwaA6xbmjrd7CKdDuKwsgx3Hmnn9eAtF6Yl85ubFo/bL8bhp7Rlk1ykrww/XmeoX53SwcUkeWw834B324QpTvpnN9NNQSkXEv1D2i0cayfW4WVIwuqNzPGvLMjnZYt349LX3ryTZPTq39A+HfPFII9kp8RTbUxOM5Z1L82jrHRrVWas0uCulIpSV7MbpEIZ9JqIhkCNdMdcqwfzF5XPG7Ij136W641gLlxSnT/ge1yzMId7l4LlD9ZNqy2ygwV0pFRGnQ8i1M+vrFkVekvFbPSeDh++8jH+8bemY+/gzd6/PjNuZ6pfsdnHNgmyeO9gwqeX9gt3+0Gs8tK3qnI69mGlwV0pFLDc1AadDAvO3T4aI8M5l+YE7UsPxB3eASyII7mCNmjnT3sfemo5Jt6m2vY+3T7fz1qn2SR97sdPgrpSK2OXlmdy8LJ+0xMiGQE5WdkpQcB+nMzXYTcvziXc5eGLPmUm/35v2lAh1HX2TPvZip8FdKRWxz92yhO/+xeopO39CnJPUBBdzs5NJS4rsCyQ1IY4bFuXyh711DPvOlmaON3Vz07deprKha8xj3zjhD+7h14edyTS4K6UuKksKUidd09+0qpDm7gF2HGsJbPvPrZVUNHQFhm6G47+pqrVnMObudNXgrpS6qDx23xV8/tYlkzrm+sW5eNyuQGmmqrGbP+ytxekQ/rivLmxna1vPIJWN3czNSQZiL3vX4K6UuqiIyKg1VCeSEOfkncvyeeZAPf1Dw3znxUrcLiefvmkRZ9r72Bems9Vfb3+vPVd8rNXdNbgrpWLCplWFdA14+eFrJ9iyt5Y7ryzljrVziHMKT+2vG7X/mydbiXc5uGlZPgB17Zq5K6XURWf9vCyyU+L5+rMVxLsc3HfNXNKS4tgwP5s/7h9dmtl5so1VxemUZlkzU2rmrpRSFyGX08FtKwsxBu68ojQwrPKWFQXUtPWx/8zZ0kzPgJcDZzpYW55BQpy1hGCt1tyVUuridOeVpVy3KIdPXjsvsO2dS/NwOYQ/BpVm3j7dzrDPsLbMmhIhPzWBunbN3JVS6qI0LyeFH390XcjNUOlJ8WyYn81TQaWZnSdbcQhcVpoBQGF6go6WUUqpmebWFQVUt/bxXy9W8VpVM9urmllamIrHXmykIC0x5oK7zueulIp571yWx39uTeTfnz8a2PbRDWWBxwXpCXT0DdE76CUpPjbCYiRrqJaIyJ9E5LCIHBSRT9nbvyAiZ0Rkj/1zS9AxD4pIlYhUiMhNU3kBSik1kfSkeF79zPXs/IeN/Pzey9l8+3L+T1BdvjDNmje+dozhkI+8fJzbH3qN3kHvmO+x+1Qb6za/wKmWnug2/hxF8hXlBf7eGPOWiHiA3SLyvP3at4wx3wjeWUSWAncAy4BC4AURWajrqCqlLiQRIdeTQK4ngQ0j5pMvsFeZquvoY35uSshrvYNe/uvFSjr7vXz92Qr++d3LRp3bGMO//vEQjV0DvHGildKs5Km7kAhNmLkbY+qMMW/Zj7uAw0DROIdsAh4zxgwYY04AVcC6aDRWKaWmQmG6lbmHu5Hp92/X0tnvZV15Jj/efpKd9nw0wZ45UM/bp9sBOFzXOaVtjdSkOlRFpAxrsew37E0PiMg+EfmhiGTY24qA6qDDagjzZSAi94nILhHZ1dTUNPmWK6VUlOSlWpl77YgbmYwxPLr9JEsLUvnRPWspzkjk07/eS9/g2ULE0LCPrz1bwYLcFFYUpXGkbuxZKKdTxMFdRFKA3wB/a4zpBP4bmAesAuqAb/p3DXP4qFl7jDEPG2PWGGPW5OTkTLbdSikVNfEuB9kpbupHjJh5/XgrFQ1d3LO+jGS3i6++fyUnW3r5t6cOM+j1AfCrN6s50dzDZ25ezPKiVI7Ud57zqlDRFFG3sIjEYQX2nxtjfgtgjGkIev0R4En7aQ1QEnR4MVAbldYqpdQUKUxPGHWX6o+3nyAjKY73rCoEYP28bO66spSf7DjFlr213LQsjxePNLGuLJONS3I5097HL3dW09A5QL5dx79QIhktI8APgMPGmH8P2l4QtNvtwAH78RbgDhFxi0g5sADYGb0mK6VU9BWkhd6lWtPWy/OHGvjg2jkhSwP+87uX8f271rBxcS5P76+npWeAz96yGBFhcb4HgMP1Y9fdz7T3Tcvc8ZFk7huAO4H9IrLH3vY54EMisgqr5HIS+CSAMeagiDwOHMIaaXO/jpRRSl3sCtISea3q7GIfP339FGBNaRDM6RDesTSPdyzNo39omMbOAebYk48tLkgF4EhdF9eHWXDk8V3V/MPv9lOencwP7l5LSWbSVF3OxMHdGPMq4evoT41zzGZg83m0SymlplVhegLdA146+4fw+Qw/f/0071peQJE9kiachDhnILADpCXGUZSeyJERmfuwz/DVZ47w8MvHWVuWQUV9F5u++xrfu/OywPw20abTDyilFFbmDtZwyB+8eoLuAS9/vXH+pM+zON8TMhzS5zP81c938/DLx7nrylJ++Ykr+P39G0hLjOPDj7zO/+6qHuds506Du1JKYWXuAEfqO/nxaye5ZUU+i/NTJ32eJQWpHGvqYcBrVaO3H2vh2YMN/L+bFvEvm5bjcjqYm5PC7/5qPWvLMjk0RePiY2MSBaWUOk/+zP0bz1XQNeDlbzYuOKfzLC7wMOwzVDV2s6wwjV/uPE1GUhz3Xl0esl96UjyPfmxd2Jp3NGjmrpRSQK7HjUOgurWPm5edW9YOBI47UtdFU9cAzx6s5/2ri3G7nKP2jXM6cDmnJgxr5q6UUlgrOeV6Eqjv7D/nrB2gPDsZt8vB4bpOmroH8PoMd6ybE8WWRkaDu1JK2daVZxLvcrC08NyydrCGSi7K93CorpPaw32sK88cNRnZdNDgrpRStm9/6NKonGdxvodf767BZ+Bv37EwKuecLK25K6VUlC0pSMVnrHHvNy/PvyBt0OCulFJR5u9Uff/q4pCpC6aTlmWUUirKVpemc+9V5XzimrkXrA0a3JVSKsrcLiefv23pBW2DlmWUUioGaXBXSqkYpMFdKaVikAZ3pZSKQRrclVIqBmlwV0qpGKTBXSmlYpAGd6WUikFijLnQbUBEmoBTkzgkG2ieouZc7Gbrtet1zy563ZEpNcbkhHvhogjukyUiu4wxay50Oy6E2Xrtet2zi173+dOyjFJKxSAN7kopFYNmanB/+EI34AKardeu1z276HWfpxlZc1dKKTW+mZq5K6WUGocGd6WUikEXdXAXkZtFpEJEqkTks2FeFxH5tv36PhFZfSHaGW0RXPdf2Ne7T0S2i8glF6Kd0TbRdQftt1ZEhkXkA9PZvqkSyXWLyHUiskdEDorIS9PdxqkSwb/1NBH5g4jsta/9oxeindEkIj8UkUYROTDG69GJa8aYi/IHcALHgLlAPLAXWDpin1uApwEBrgDeuNDtnqbrXg9k2I/fNVuuO2i/F4GngA9c6HZP0993OnAImGM/z73Q7Z7Ga/8c8FX7cQ7QCsRf6Laf53VfA6wGDozxelTi2sWcua8Dqowxx40xg8BjwKYR+2wCfmIsrwPpIlIw3Q2Nsgmv2xiz3RjTZj99HSie5jZOhUj+vgH+GvgN0DidjZtCkVz3h4HfGmNOAxhjZtO1G8AjIgKkYAV37/Q2M7qMMS9jXcdYohLXLubgXgRUBz2vsbdNdp+ZZrLX9HGsb/mZbsLrFpEi4Hbgf6axXVMtkr/vhUCGiGwTkd0icte0tW5qRXLt3wGWALXAfuBTxhjf9DTvgolKXLuYF8iWMNtGjtuMZJ+ZJuJrEpHrsYL7VVPaoukRyXX/B/AZY8ywlcjFhEiu2wVcBmwEEoEdIvK6MeboVDduikVy7TcBe4AbgHnA8yLyijGmc4rbdiFFJa5dzMG9BigJel6M9e092X1mmoiuSURWAt8H3mWMaZmmtk2lSK57DfCYHdizgVtExGuM+f20tHBqRPrvvNkY0wP0iMjLwCXATA/ukVz7R4GvGKsYXSUiJ4DFwM7paeIFEZW4djGXZd4EFohIuYjEA3cAW0bsswW4y+5dvgLoMMbUTXdDo2zC6xaROcBvgTtjIHvzm/C6jTHlxpgyY0wZ8Gvgr2Z4YIfI/p0/AVwtIi4RSQIuBw5PczunQiTXfhrrNxZEJA9YBByf1lZOv6jEtYs2czfGeEXkAeBZrF71HxpjDorIX9qv/w/WiIlbgCqgF+tbfkaL8Lr/CcgCHrKzWK+Z4TPoRXjdMSeS6zbGHBaRZ4B9gA/4vjEm7DC6mSTCv/MvAT8Wkf1Y5YrPGGNm9FTAIvJL4DogW0RqgH8G4iC6cU2nH1BKqRh0MZdllFJKnSMN7kopFYM0uCulVAzS4K6UUjFIg7tSSsUgDe5KKRWDNLgrpVQM+v8BkRdim5I7zJoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epsilons = np.arange(0.01, 1, 0.01).tolist()\n",
    "rewards = list()\n",
    "for epsilon in epsilons:\n",
    "    bandits, total_reward, q_bandits, num_selected_bandit = multi_armed_bandit(num_games=1000,\n",
    "                                                                               epsilon=epsilon)\n",
    "    rewards.append(total_reward)\n",
    "\n",
    "plt.title('Reward per Epsion-Greedy')\n",
    "plt.plot(epsilons, rewards)\n",
    "plt.show()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "*Este Notebook ha sido desarrollado por **Ricardo Moya García** y registrado en Safe Creative como ***Atribución-NoComercial-CompartirIgual***.*\n",
    "\n",
    "<a href=\"https://www.safecreative.org/work/2005103928013\" xmlns:cc=\"http://creativecommons.org/ns#\" rel=\"cc:license\"><img src=\"https://resources.safecreative.org/work/2005103928013/label/standard-72\" style=\"border:0;\" alt=\"Safe Creative #2005103928013\"/></a>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}