Adds Reflexion (solves #2316)

docs/extras/modules/agents/how_to/max_time_limit.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "75c041b7",
    "metadata": {},
@@ -12,7 +13,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 4,
    "id": "986da446",
    "metadata": {},
    "outputs": [],
@@ -25,7 +26,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 5,
    "id": "b9e7799e",
    "metadata": {},
    "outputs": [],
@@ -35,7 +36,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 6,
    "id": "3f658cb3",
    "metadata": {},
    "outputs": [],
@@ -50,6 +51,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "5e9d92c2",
    "metadata": {},
@@ -61,7 +63,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 7,
    "id": "aa7abd3b",
    "metadata": {},
    "outputs": [],
@@ -73,7 +75,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 8,
    "id": "129b5e26",
    "metadata": {},
    "outputs": [],
@@ -89,7 +91,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 9,
    "id": "47653ac6",
    "metadata": {},
    "outputs": [
@@ -124,7 +126,7 @@
        "'foo'"
       ]
      },
-     "execution_count": 6,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -134,6 +136,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "285929bf",
    "metadata": {},
@@ -143,7 +146,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 10,
    "id": "fca094af",
    "metadata": {},
    "outputs": [],
@@ -159,7 +162,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 11,
    "id": "0fd3ef0a",
    "metadata": {},
    "outputs": [
@@ -182,10 +185,10 @@
     {
      "data": {
       "text/plain": [
-       "'Agent stopped due to iteration limit or time limit.'"
+       "'Agent stopped due to iteration, trial or time limit.'"
       ]
      },
-     "execution_count": 8,
+     "execution_count": 11,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -195,6 +198,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "0f7a80fb",
    "metadata": {},
@@ -204,7 +208,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 12,
    "id": "3cc521bb",
    "metadata": {},
    "outputs": [],
@@ -221,7 +225,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 13,
    "id": "1618d316",
    "metadata": {},
    "outputs": [
@@ -236,10 +240,6 @@
       "Action: Jester\n",
       "Action Input: foo\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3mfoo\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m Is there more I can do?\n",
-      "Action: Jester\n",
-      "Action Input: foo\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mfoo\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m\n",
       "Final Answer: foo\u001b[0m\n",
       "\n",
@@ -252,7 +252,7 @@
        "'foo'"
       ]
      },
-     "execution_count": 14,
+     "execution_count": 13,
      "metadata": {},
      "output_type": "execute_result"
     }

docs/modules/agents/retries_with_reflection.ipynb

@@ -0,0 +1,180 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "5436020b",
+   "metadata": {},
+   "source": [
+    "# Retries with Reflection\n",
+    "\n",
+    "Sometimes Agents will not succeed in their task. This might be e.g. if their reasoning was wrong (i.e., they made the wrong plan) or inefficient (i.e., they are taking too long to get to an answer). In this case, we can use the re-try method Reflection (from https://arxiv.org/abs/2303.11366).\n",
+    "\n",
+    "With Reflection, the Agent will think about why their plan failed and what it can do better next time. This reflection, along with the previous trial(s) will be available to the Agent in the next trial. This enables the Agent to solve harder problems.\n",
+    "\n",
+    "This notebook will shows you how to use Reflection."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "3899f708",
+   "metadata": {},
+   "source": [
+    "**Caution:** Reflection can cause your context size to blow up, because previous trials are appended to the prompt. With each re-trial, the prompt increases in size. If you encounter this issue, lower `AgentExecutor.max_trials` or `Reflector.max_iterations_per_trial`"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "5a4d1b61",
+   "metadata": {},
+   "source": [
+    "First, we import libraries and load an LLM and tools; just like we would for an Agent without Reflection."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "10db20b9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "os.environ[\"OPENAI_API_KEY\"] = \"...\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "d01216c0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import load_tools, initialize_agent, AgentType\n",
+    "from langchain.agents.reflexion.react import ReactReflector\n",
+    "from langchain.llms import OpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "0728f0d9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "ba4e7618",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tools = load_tools([\"wikipedia\", \"llm-math\"], llm=llm)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "5b75e62a",
+   "metadata": {},
+   "source": [
+    "Next, we create a `Reflector` object. The default implementation is `ReactReflector`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "57072c39",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "reflector = ReactReflector.from_llm(\n",
+    "    llm=llm,\n",
+    "    max_iterations_per_trial=2\n",
+    ")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "cdf00e33",
+   "metadata": {},
+   "source": [
+    "Reflection only kicks in, wenn the Reflector determines that the current trial failed. For this, we have to pass in conditions that specify when a trial failed.\n",
+    "These can be:\n",
+    "- `max_iterations_per_trial`: how many steps can the Agent take in a trial?\n",
+    "- `max_execution_time_per_trial`: how much take can the Agent use in a trial?\n",
+    "- `max_action_repetition`: how often is the Agent allowed to suggest the exact same Actions?"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "0b50fc9b",
+   "metadata": {},
+   "source": [
+    "Finally, let's initialize an agent with the `tools`, the `llm`, the type of `Agent` we want to use and our `Reflector`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "03208e2b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent = initialize_agent(\n",
+    "    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
+    "    reflector=reflector,\n",
+    "    max_trials=3,\n",
+    "    verbose=True\n",
+    ")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "373361d5",
+   "metadata": {},
+   "source": [
+    "Now let's test it out!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5901695b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "question = \"At what age did the main character of One Piece set sail to find the One Piece?\"\n",
+    "agent.run(question)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

langchain/agents/__init__.py

@@ -6,6 +6,8 @@
     BaseMultiActionAgent,
     BaseSingleActionAgent,
     LLMSingleActionAgent,
+    Reflector,
+    ReflexionOutputParser,
 )
 from langchain.agents.agent_toolkits import (
     create_csv_agent,
@@ -47,6 +49,8 @@
     "ConversationalAgent",
     "ConversationalChatAgent",
     "LLMSingleActionAgent",
+    "Reflector",
+    "ReflexionOutputParser",
     "MRKLChain",
     "OpenAIFunctionsAgent",
     "ReActChain",