Aprendizaje Reforzado/Cuadernos/ar_Agente_simple.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5e6f8f0e",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<img src=\"../Imagenes/logo-final-ap.png\"  width=\"80\" height=\"80\" align=\"left\"/> \n",
    "</figure>\n",
    "\n",
    "# <span style=\"color:blue\"><left>Aprendizaje Profundo</left></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b718a9ce",
   "metadata": {},
   "source": [
    "# <span style=\"color:red\"><center>Agente simple</center></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d2dca78",
   "metadata": {},
   "source": [
    "<center>Introducción</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f5846fe",
   "metadata": {},
   "source": [
    "##   <span style=\"color:blue\">Autores</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2d09f7",
   "metadata": {},
   "source": [
    "1. Alvaro Mauricio Montenegro Díaz, ammontenegrod@unal.edu.co\n",
    "2. Daniel Mauricio Montenegro Reyes, dextronomo@gmail.com "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c072351c",
   "metadata": {},
   "source": [
    "##   <span style=\"color:blue\">Asesora Medios y Marketing digital</span>\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6b23f0c",
   "metadata": {},
   "source": [
    "4. Maria del Pilar Montenegro, pmontenegro88@gmail.com "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0e0a794",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Asistentes</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dfc4de4",
   "metadata": {},
   "source": [
    "5. Oleg Jarma, ojarmam@unal.edu.co \n",
    "6. Laura Lizarazo, ljlizarazore@unal.edu.co "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cec7f62",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Referencias</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a75ac7d2",
   "metadata": {},
   "source": [
    "1. [Alvaro Montenegro y Daniel Montenegro, Inteligencia Artificial y Aprendizaje Profundo, 2021](https://github.com/AprendizajeProfundo/Diplomado)\n",
    "1. [Maxim Lapan, Deep Reinforcement Learning Hands-On: Apply modern RL methods to practical problems of chatbots, robotics, discrete optimization, web automation, and more, 2nd Edition, 2020](http://library.lol/main/F4D1A90C476A576238E8FE1F47602C67)\n",
    "1. [Richard S. Sutton, Andrew G. Barto, Reinforcement learning: an introduction, 2nd edition, 2020](http://library.lol/main/6502B74CE247C4CD4D4FB54747AD7C7E)\n",
    "1. [Praveen Palanisamy - Hands-On Intelligent Agents with OpenAI Gym_ Your Guide to Developing AI Agents Using Deep Reinforcement Learning, 2020](http://library.lol/main/E4FD128CF9B93E0F7A542B053330517A)\n",
    "1. [Turing Paper 1936](http://www.thocp.net/biographies/papers/turing_oncomputablenumbers_1936.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffd63d24",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Contenido</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "683ee4c4",
   "metadata": {},
   "source": [
    "*  [Introducción](#Introducción)\n",
    "* [Anatomía de un agente](#Anatomía-de-un-agente)\n",
    "* [Clase Environment](#Clase-Environment)\n",
    "* [Clase Agent](#Clase-Agent)\n",
    "* [Ejecución de un episodio](#Ejecución-de-un-episodio)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be027523",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Introducción</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4cc08e0",
   "metadata": {},
   "source": [
    "En esta lección exploramos una implementación en Python de un agente en un situación muy simple."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41eb2d39",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Anatomía de un agente</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "068d77f3",
   "metadata": {},
   "source": [
    " El ambiente entregará una recompensa aleatoria al agente durante un número limitado de pasos. El ejemplo es extremadamente simple, pero muestra la interacción entre el agente y el ambiente."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdcc2c48",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Clase Environment</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1cd5183f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List\n",
    "\n",
    "class Environment:\n",
    "    \"\"\"Define una clase Enviroment básica. \n",
    "    se adimitiran 10 recompensas aleatorias\"\"\"\n",
    "    \n",
    "    def __init__(self):\n",
    "        self.steps_left = 10\n",
    "    \n",
    "    # regresa una lista de floats (type annotations)\n",
    "    # usualmente esta es una función del estado actual del Environment\n",
    "    def get_observation(self) -> List[float]:\n",
    "        return [0.0, 0.0, 0.0]\n",
    "    \n",
    "    # returna una lista con las siguientes posibles acciones\n",
    "    def get_actions(self) -> List[int]:\n",
    "        return [0, 1]\n",
    "    \n",
    "    # determina si un episodio ha terminado o no\n",
    "    def is_done(self) -> bool:\n",
    "        return self.steps_left == 0\n",
    "    \n",
    "    # recibe la acción que el agente ha tomado\n",
    "    # esta es la pieza central en la funcionalidad del agente\n",
    "    def action(self, action: int) -> float:\n",
    "        if self.is_done():\n",
    "            raise Exception(\"Game is over\")\n",
    "        self.steps_left -= 1\n",
    "        \n",
    "        #retorna la recompensa\n",
    "        return random.random()\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "585e295c",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Clase Agent</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "151a1bed",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "class Agent:\n",
    "    \"\"\"Define una clase Agent básica. \n",
    "    lleva la cuenta de la recompensa acumulada\"\"\"\n",
    "    def __init__(self):\n",
    "        self.total_reward = 0.0\n",
    "     \n",
    "    # acepta la instancia de Environment y hace las siguientes tareas:\n",
    "    # 1. Observa el ambiente\n",
    "    # 2. Toma un decisión sobre que acción tomar basado en las observaciones\n",
    "    # 3. somete la accion al ambiente\n",
    "    # 4. recibe la recompensa del paso actual y la acumula\n",
    "    def step(self, env: Environment):\n",
    "        current_obs = env.get_observation()\n",
    "        actions = env.get_actions()\n",
    "        reward = env.action(random.choice(actions))\n",
    "        self.total_reward += reward"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd7cc9eb",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Ejecución de un episodio</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e64a2d66",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recompensa total recibida: 3.0791\n"
     ]
    }
   ],
   "source": [
    "if __name__ == \"__main__\":\n",
    "    env = Environment()\n",
    "    agent = Agent()\n",
    "    \n",
    "    while not env.is_done():\n",
    "        agent.step(env)\n",
    "    \n",
    "    print(\"Recompensa total recibida: %.4f\" % agent.total_reward)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}