docs/docs/examples/agent/openai_agent_tool_call_parser.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "24103c51",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/agent/openai_agent_tool_call_parser.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99cea58c-48bc-4af6-8358-df9695659983",
   "metadata": {},
   "source": [
    "# OpenAI Agent with Tool Call Parser"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "673df1fe-eb6c-46ea-9a73-a96e7ae7942e",
   "metadata": {},
   "source": [
    "Unfortunately, the tool calls by OpenAI are not always valid json, especially from older versions of the API. Up to and including the OpenAI API version 1106, this issue is relatively frequent if the argument is a long string (e.g. a python script), see for example [here](https://community.openai.com/t/malformed-json-in-gpt4-1106-function-arguments/685884).\n",
    "\n",
    "With the default tool call parser, the OpenAI Agent will fail to parse these tool calls and tries to fix the tool call in the next step. This needs another llm call, which is slow and expensive.\n",
    "\n",
    "This notebook demonstrates how to define a custom tool call parser that can handle certain kinds of malformed function calls.\n",
    "The following steps are copied from the OpenAI Agent notebook, with the addition of a custom tool call parser."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54b7bc2e-606f-411a-9490-fcfab9236dfc",
   "metadata": {},
   "source": [
    "## Initial Setup "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23e80e5b-aaee-4f23-b338-7ae62b08141f",
   "metadata": {},
   "source": [
    "Let's start by importing some simple building blocks.  \n",
    "\n",
    "The main thing we need is:\n",
    "1. the OpenAI API (using our own `llama_index` LLM class)\n",
    "2. a place to keep conversation history \n",
    "3. a definition for tools that our agent can use."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "41101795",
   "metadata": {},
   "source": [
    "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4985c578",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-agent-openai\n",
    "%pip install llama-index-llms-openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c61c873d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d47283b-025e-4874-88ed-76245b22f82e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from llama_index.core.tools import FunctionTool\n",
    "\n",
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fe08eb1-e638-4c00-9103-5c305bfacccf",
   "metadata": {},
   "source": [
    "Let's define some very simple calculator tools for our agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dd3c4a6-f3e0-46f9-ad3b-7ba57d1bc992",
   "metadata": {},
   "outputs": [],
   "source": [
    "def multiply(a: int, b: int) -> int:\n",
    "    \"\"\"Multiple two integers and returns the result integer\"\"\"\n",
    "    return a * b\n",
    "\n",
    "\n",
    "multiply_tool = FunctionTool.from_defaults(fn=multiply)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfcfb78b-7d4f-48d9-8d4c-ffcded23e7ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "def add(a: int, b: int) -> int:\n",
    "    \"\"\"Add two integers and returns the result integer\"\"\"\n",
    "    return a + b\n",
    "\n",
    "\n",
    "add_tool = FunctionTool.from_defaults(fn=add)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30016529",
   "metadata": {},
   "source": [
    "## Definition of the Tool Call Parser\n",
    "\n",
    "Sometimes, OpenAI tool calls are not valid json\n",
    "\n",
    "When defining your own Tool Call Parser, you need to define a function that takes a OpenAIToolCall and returns a dictionary. The dictionary will be passed as **kwargs to the tool function.\n",
    "\n",
    "The Parser should throw a ValueError if the tool call can't be parsed. This will be returned to the agent and it will try to fix the call on the next step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ffe547b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict\n",
    "from llama_index.llms.openai.utils import OpenAIToolCall\n",
    "import re\n",
    "\n",
    "# The same parser is available as\n",
    "# from llama_index.agent.openai import advanced_tool_call_parser\n",
    "\n",
    "\n",
    "def custom_tool_call_parser(tool_call: OpenAIToolCall) -> Dict:\n",
    "    r\"\"\"Parse tool calls that are not standard json.\n",
    "    Also parses tool calls of the following forms:\n",
    "    variable = \\\"\\\"\\\"Some long text\\\"\\\"\\\"\n",
    "    variable = \"Some long text\"'\n",
    "    variable = '''Some long text'''\n",
    "    variable = 'Some long text'\n",
    "    \"\"\"\n",
    "    arguments_str = tool_call.function.arguments\n",
    "    if len(arguments_str.strip()) == 0:\n",
    "        # OpenAI returns an empty string for functions containing no args\n",
    "        return {}\n",
    "    try:\n",
    "        tool_call = json.loads(arguments_str)\n",
    "        if not isinstance(tool_call, dict):\n",
    "            raise ValueError(\"Tool call must be a dictionary\")\n",
    "        return tool_call\n",
    "    except json.JSONDecodeError as e:\n",
    "        # pattern to match variable names and content within quotes\n",
    "        pattern = r'([a-zA-Z_][a-zA-Z_0-9]*)\\s*=\\s*[\"\\']+(.*?)[\"\\']+'\n",
    "        match = re.search(pattern, arguments_str)\n",
    "\n",
    "        if match:\n",
    "            variable_name = match.group(1)  # This is the variable name\n",
    "            content = match.group(2)  # This is the content within the quotes\n",
    "            return {variable_name: content}\n",
    "        raise ValueError(f\"Invalid tool call: {e!s}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "707d30b8-6405-4187-a9ed-6146dcc42167",
   "metadata": {},
   "source": [
    "## Defining the OpenAI Agent with Tool Call Parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38ab3938-1138-43ea-b085-f430b42f5377",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.agent.openai import OpenAIAgent\n",
    "from llama_index.llms.openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d852ece7-e5a1-4368-9d59-c7014e0b5b4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(model=\"gpt-3.5-turbo-0613\")\n",
    "agent = OpenAIAgent.from_tools(\n",
    "    [multiply_tool, add_tool],\n",
    "    llm=llm,\n",
    "    verbose=True,\n",
    "    tool_call_parser=custom_tool_call_parser,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "500cbee4",
   "metadata": {},
   "source": [
    "### Chat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9fd1cad5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What is (121 * 3) + 42?\n",
      "=== Calling Function ===\n",
      "Calling function: multiply with args: {\n",
      "  \"a\": 121,\n",
      "  \"b\": 3\n",
      "}\n",
      "Got output: 363\n",
      "========================\n",
      "\n",
      "=== Calling Function ===\n",
      "Calling function: add with args: {\n",
      "  \"a\": 363,\n",
      "  \"b\": 42\n",
      "}\n",
      "Got output: 405\n",
      "========================\n",
      "\n",
      "(121 * 3) + 42 is equal to 405.\n"
     ]
    }
   ],
   "source": [
    "response = agent.chat(\"What is (121 * 3) + 42?\")\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "538bf32f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ToolOutput(content='363', tool_name='multiply', raw_input={'args': (), 'kwargs': {'a': 121, 'b': 3}}, raw_output=363), ToolOutput(content='405', tool_name='add', raw_input={'args': (), 'kwargs': {'a': 363, 'b': 42}}, raw_output=405)]\n"
     ]
    }
   ],
   "source": [
    "# inspect sources\n",
    "print(response.sources)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}