tutorials/tracing/langchain_agent_tracing_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<center>\n",
    "    <p style=\"text-align:center\">\n",
    "        <img alt=\"phoenix logo\" src=\"https://storage.googleapis.com/arize-phoenix-assets/assets/phoenix-logo-light.svg\" width=\"200\"/>\n",
    "        <br>\n",
    "        <a href=\"https://docs.arize.com/phoenix/\">Docs</a>\n",
    "        |\n",
    "        <a href=\"https://github.com/Arize-ai/phoenix\">GitHub</a>\n",
    "        |\n",
    "        <a href=\"https://join.slack.com/t/arize-ai/shared_invite/zt-1px8dcmlf-fmThhDFD_V_48oU7ALan4Q\">Community</a>\n",
    "    </p>\n",
    "</center>\n",
    "<h1 align=\"center\">Tracing and Evaluating a LangChain OpenAI Agent Application</h1>\n",
    "\n",
    "With the new OpenAI API that supports function calling, it’s never been easier to build your own agent.\n",
    "\n",
    "In this notebook tutorial, we showcase how to write your own OpenAI agent in under 50 lines of code and use Phoenix to inspect the internals of the Agent. It is minimal, yet feature complete (with ability to carry on a conversation and use tools)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Install Dependencies and Import Libraries\n",
    "\n",
    "Install Phoenix, LangChain, and OpenAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install \"langchain>=0.0.334\" \"openai>=1\" numexpr arize-phoenix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "import openai\n",
    "import phoenix as px\n",
    "from langchain.agents import AgentType, Tool, initialize_agent\n",
    "from langchain.chains import LLMMathChain\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
    "from phoenix.trace.langchain import LangChainInstrumentor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 2. Launch Phoenix\n",
    "\n",
    "You can run Phoenix in the background to collect trace data emitted by any LangChain application that has been instrumented with the `OpenInferenceTracer`.\n",
    "\n",
    "Launch Phoenix and follow the instructions in the cell output to open the Phoenix UI (the UI should be empty because we have yet to run a LangChain application)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = px.launch_app()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Configure Your OpenAI Credentials\n",
    "\n",
    "Let's make sure we have openAI credentials set up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not (openai_api_key := os.getenv(\"OPENAI_API_KEY\")):\n",
    "    openai_api_key = getpass(\"🔑 Enter your OpenAI API key: \")\n",
    "openai.api_key = openai_api_key\n",
    "os.environ[\"OPENAI_API_KEY\"] = openai_api_key"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Build and Instrument Your Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now instrument LangChain to send trace data to Phoenix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "LangChainInstrumentor().instrument()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now define the LLM model we will use for our agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's define the tools the LLM will have at its disposal. We will use the following tools:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)\n",
    "# Let's give the LLM access to math tools\n",
    "tools = [\n",
    "    Tool(\n",
    "        name=\"Calculator\",\n",
    "        func=llm_math_chain.run,\n",
    "        description=\"useful for when you need to answer questions about math\",\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's setup the Prompt Template. This will inform how the agent will respond to queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", \"You are a helpful assistant\"),\n",
    "        (\"human\", \"{input}\"),\n",
    "        MessagesPlaceholder(variable_name=\"agent_scratchpad\"),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now, we define our agent that’s capable answering questions and calling tools.\n",
    "\n",
    "The meat of the agent logic breaks down into three steps:\n",
    "\n",
    "- Call OpenAI to decide which tool (if any) to call and with what arguments.\n",
    "\n",
    "- Call the tool with the arguments to obtain an output\n",
    "\n",
    "- Call OpenAI to synthesize a response from the conversation context and the tool output.\n",
    "\n",
    "Let's initialize the agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\")\n",
    "agent_executor = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 5. Chat With Your Agent\n",
    "\n",
    "Let's now chat with our agent! Note that the `OpenInferenceTracer` is logging traces of the execution to phoenix as you invoke the agent!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = agent_executor.invoke({\"input\": \"What is 47 raised to the 5th power?\"})\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's chat with our agent a few more times. This time with some follow-up questions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "queries = [\n",
    "    \"What is (121 * 3) + 42?\",\n",
    "    \"what is 3 * 3?\",\n",
    "    \"what is 4 * 4?\",\n",
    "    \"what is 75 * (3 + 4)?\",\n",
    "    \"what is 23 times 87\",\n",
    "]\n",
    "\n",
    "for query in queries:\n",
    "    print(f\"> {query}\")\n",
    "    response = agent_executor.invoke({\"input\": query})\n",
    "    print(response)\n",
    "    print(\"---\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Open the `session.url` in your browser to take a look at the traces in Phoenix. Note that LLM spans contain the OpenAI function calls, and that we can inspect what tool the LLM picked based on the queries.\n",
    "\n",
    "To learn more about function calling, check out the [OpenAI API docs](https://openai.com/blog/function-calling-and-other-api-updates).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"View the traces in phoenix: {session.url}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Export Your Trace Data\n",
    "\n",
    "You can export your trace data as a pandas dataframe for further analysis and evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trace_df = px.Client().get_spans_dataframe()\n",
    "trace_df"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}