versions/2023/llm/python/langchain_quick_demo.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "129d7c07",
   "metadata": {},
   "source": [
    "### LangChain and OpenAI LLM Quick Demo\n",
    "\n",
    "This demo showcases how to use LangChain to build useful large language model applications. LangChain is a framework that allows us to interact with Large Language Models (LLMs) like OpenAI GPTs.\n",
    "\n",
    "First, install the required modules.\n",
    "\n",
    "Always best to upgrade since `langchain` is rapidliy developing.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc9a593b",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install langchain --upgrade\n",
    "! pip install openai --upgrade"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "834aba87",
   "metadata": {},
   "source": [
    "Then, import the packages to use.\n",
    "\n",
    "Please create your [OpenAI](https://openai.com/) key before proceeding. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b88d9429",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import textwrap\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "query = input(\"OpenAI key: \")\n",
    "os.environ[\"OPENAI_API_KEY\"] = query"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0923bdaa",
   "metadata": {},
   "source": [
    "#### Demo 1: A Simple Question and Answering Example\n",
    "\n",
    "The simplest example is a question and answering application. To build the demo, execute the following: \n",
    "\n",
    "- Load OpenAI Large Language Model (`llm`). Temperature is a measure of uncertainty or randomness of answers. Higher temperature means more randomness. In other words, temp = 0 means the model will always give the same answer. Temp = 1 means the model will give answers with more variance. Let's set it to 0.5.\n",
    "- Provide a text query\n",
    "- Request the Large Language Model to process the query\n",
    "- Print the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fccf101",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(temperature=0.5)\n",
    "text = \"What are the 10 best places to visit?\"\n",
    "print(llm(text))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "002a8718",
   "metadata": {},
   "source": [
    "#### Demo 2: Using a Prompt Template\n",
    "\n",
    "A meaningful interaction with a LLM requires a good prompt to elicit a good answer. In this example a prompt template is provided to elicit a good answer from the LLM. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d49eaed3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"to_do_what\"],\n",
    "    template=\"What are the 10 best places to {to_do_what}?\",\n",
    ")\n",
    "formatted_prompt = prompt.format(to_do_what=\"eat\")\n",
    "print(f\"Formatted prompt: {formatted_prompt}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2dfe7850",
   "metadata": {},
   "source": [
    "#### Demo 3: A Simple Chain\n",
    "\n",
    "LLM is not very useful unless we can interact with it using prompt. In the same way, a prompt is not valuable unless we can use it to interact with a LLM. A chain puts them together. Below is a simple chain.\n",
    "\n",
    "Chain: Prompt &rarr; LLM \n",
    "\n",
    "Then, the chain can now be queried.\n",
    "\n",
    "`chain.run(<to_do_what>`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b1bcffe",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain\n",
    "\n",
    "chain = LLMChain(llm=llm, prompt=prompt)\n",
    "formatted_prompt = prompt.format(to_do_what=\"...\")\n",
    "query = input(formatted_prompt)\n",
    "formatted_prompt = prompt.format(to_do_what=query)\n",
    "print(f\"Human: {formatted_prompt}\")\n",
    "results = chain.run(query).split('\\n')\n",
    "print(f\"AI: \")\n",
    "for result in results:\n",
    "    print(result)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "172cb57a",
   "metadata": {},
   "source": [
    "#### Demo 4: Enabling Long-Term Memory \n",
    "\n",
    "A conversation has context. Sometimes, the context has a reference to a discussion long time ago. Without the ability to remember something said a while back, the conversation can easily become frustrating. Thus, memory is important. In this example, a memory is incorporated in the chain.\n",
    "\n",
    "Below is a conversation with context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0d58984",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import OpenAI, ConversationChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "\n",
    "llm = OpenAI(temperature=0.5)\n",
    "# we use a memory to store the conversation history\n",
    "memory = ConversationBufferMemory() \n",
    "conversation = ConversationChain(\n",
    "        llm=llm, \n",
    "        verbose=True, \n",
    "        memory=memory\n",
    "        )\n",
    "\n",
    "conversation.predict(input=\"Hi! I want to talk to you. What is your name?\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "cfc05376",
   "metadata": {},
   "source": [
    "##### Let's Build a Conversation Chain\n",
    "\n",
    "The human can ask questions all about a topic. Assume that there is context in the conversation. To end the conversation, the human can say \"bye\". "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33219880",
   "metadata": {},
   "outputs": [],
   "source": [
    "while True:\n",
    "    input_prompt = \"Human: \"\n",
    "    query = input(input_prompt)\n",
    "\n",
    "    if query.lower() == \"bye\":\n",
    "        print(\"AI: Bye!\")\n",
    "        # print the conversation history\n",
    "        print(memory.chat_memory)\n",
    "        break\n",
    "\n",
    "    text = f\"{input_prompt}: {query}\"\n",
    "    print(textwrap.fill(text, width=80))\n",
    "    text = f\"AI: {conversation.predict(input=query)}\"\n",
    "    print(textwrap.fill(text, width=80))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3d50d4f4",
   "metadata": {},
   "source": [
    "#### Demo 5: Using Agent and Tools to Determine what Actions to Take to Solve a Problem\n",
    "\n",
    "Sometimes, a problem can not be solved in one step. Additional tools may also needed in order to solve the problem. In this demo, the LLM uses an agent to determine what actions to take and what tools to use.\n",
    "\n",
    "The focus is on math problems. `llm-math` tool is used. More info about [tools](https://langchain.readthedocs.io/en/latest/modules/agents/tools.html) and [agents](https://langchain.readthedocs.io/en/latest/modules/agents/agents.html) are available in the LangChain documentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e62e892c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents import load_tools\n",
    "from langchain.agents import initialize_agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5dea1639",
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, let's load the language model we're going to use to control the agent.\n",
    "llm = OpenAI(temperature=0.0)\n",
    "\n",
    "# Next, let's load some tools to use. \n",
    "# Note that the `llm-math` tool uses an LLM, so we need to pass that in.\n",
    "tools = load_tools([\"llm-math\"], llm=llm)\n",
    "\n",
    "# Finally, let's initialize an agent with the tools, the language model, \n",
    "# and the type of agent we want to use.\n",
    "agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", verbose=True)\n",
    "\n",
    "# Now let's test it out!\n",
    "\n",
    "query = input(\"Human: \")\n",
    "text = f\"Human: {query}\"\n",
    "agent.run(query)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1af836b3",
   "metadata": {},
   "source": [
    "### Demo 6: Evaluation\n",
    "\n",
    "Evaluation of chains is difficult to perform because of: 1) Lack of data and 2) Lack of metrics.\n",
    "\n",
    "- Lack of data\n",
    "\n",
    "Unlike machine learning problems, there is not enough data to evaluate chains. As such, the evaluation is performed on specific examples only. For example, Q&A on documents with known answers. LangChain provides a small number of datasets for evaluation.\n",
    "\n",
    "- Lack of metrics\n",
    "\n",
    "Unlike machine learning problems where there are more or less precise answers (e.g. accuracy on classification, mAP on detection, etc.), there are no precise answers in language. As such, the evaluation is more complicated. How do you evaluate the quality of a conversation or essay?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4120c014",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Comment this out if you are NOT using tracing\n",
    "import os\n",
    "\n",
    "from langchain.evaluation.loading import load_dataset\n",
    "\n",
    "os.environ[\"LANGCHAIN_HANDLER\"] = \"langchain\"\n",
    "dataset = load_dataset(\"question-answering-state-of-the-union\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llm",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "82ad6d7bc5c4d1338129f53582d686f501ccbc16afd0f50ed25b05b303292daa"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}