jovyan/natural-language-interface-llama2-textanayzer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Interaction with Data\n",
    "This notebook shows an example of Natural Language Interaction with Data.\n",
    "\n",
    "- First a keyspace and table are set up for Credit Card Transactions\n",
    "- Secondly a prompt will be defined that is based on the [SQL-PaLM](https://arxiv.org/abs/2306.00739) paper.\n",
    "- This prompt instructs to create CQL code that allows for queries on the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-ASFNIL6IKiB"
   },
   "source": [
    "## Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "S_h_Ah_Bb_Qo",
    "outputId": "8e7e200b-8412-4a9f-bfcc-5bfd41dfc777",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install ragstack-ai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jveF3xKMZaDE"
   },
   "source": [
    "## Visualize Ragstack dependencies\n",
    "RAGStack is a curated stack of the best open-source software for easing implementation of the RAG pattern in production-ready applications using DataStax Enterprise, Astra Vector DB or Apache Cassandra as a vector store.\n",
    "\n",
    "A single command (pip install ragstack-ai) unlocks all the open-source packages required to build production-ready RAG applications with LangChain and DataStax Enterprise, Astra Vector DB or Apache Cassandra.\n",
    "\n",
    "For each open-source project included in RAGStack, we select a version lineup and then test the combination for compatibility, performance, and security. Our extensive test suite ensures that RAGStack components work well together so you can confidently deploy them in production. We also run security scans on all components using industry-standard tools to ensure that you are not exposed to known vulnerabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FjOp-3oMZbXn",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install pipdeptree\n",
    "!pipdeptree -p ragstack-ai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tjBB04JPIPC_"
   },
   "source": [
    "## Keeping it all locally and within the enterprise firewall\n",
    "In this notebook we'll keep all services local to ensure maximum safety:\n",
    "\n",
    "- For the Vector Database, [DataStax Enterprise 7](https://www.datastax.com/blog/get-started-with-the-datastax-enterprise-7-0-developer-vector-search-preview) will be used.\n",
    "- For the Foundational Model we'll be using [llama2](https://).\n",
    "\n",
    "Read more about LLama2 and how it stacks up to GPT-3.5 [here](https://)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get an inference engine with Llama2 started\n",
    "There are a multitude of inference engines. You can go for [LM Studio](https://lmstudio.ai/) which has a nice UI. In this notebook, we'll use [Ollama](https://ollama.com/).\n",
    "\n",
    "1. Get started by [downloading](https://ollama.com/download)\n",
    "2. Install it to your machine\n",
    "3. Start the inference engine, while downloading Llama2 (~4GB) with the command `ollama run llama2` in a terminal\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calling Mistral directly (not ideal because of API and implementation specifics and no support through RAGStack)\n",
    "# This step is optional and for informational purposes only\n",
    "\n",
    "!pip install ollama\n",
    "\n",
    "import ollama\n",
    "\n",
    "from ollama import Client\n",
    "client = Client(host='http://host.docker.internal:11434')\n",
    "response = client.chat(\n",
    "    model='llama2:latest', \n",
    "    messages=[\n",
    "        {'role': 'system', 'content': \"You're a helpful assistant\"},\n",
    "        {'role': 'user', 'content': 'Why is the sky blue?'}\n",
    "    ]\n",
    ")\n",
    "\n",
    "print(response['message']['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calling LLama2 through RAGStack (THIS IS THE DEFAULT IN THIS NOTEBOOK)\n",
    "# Way better because RAGStack transparantly takes away implementation specifics and provides a clean interface through LangChain\n",
    "\n",
    "from langchain_community.chat_models.ollama import ChatOllama\n",
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "\n",
    "client = ChatOllama(\n",
    "    model=\"llama2:latest\", \n",
    "    num_ctx=4096,\n",
    "    base_url=\"http://host.docker.internal:11434\",\n",
    "    temperature=0\n",
    ")\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(content=\"You're a helpful assistant\"),\n",
    "    HumanMessage(content=\"Why is the sky blue?\"),\n",
    "]\n",
    "\n",
    "response = client.invoke(messages)\n",
    "\n",
    "print (response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now let's run DSE 7 Vector Store\n",
    "Make sure you have [Docker](https://www.docker.com/) installed.\n",
    "\n",
    "Run DSE 7 in any of these two ways from a terminal window:\n",
    "1. `docker-compose up` (using the docker-compose.yml file in the root of this repository)\n",
    "2. `docker run -e DS_LICENSE=accept -p 9042:9042 datastax/dse-server:7.0.0-alpha.4`\n",
    "\n",
    "And then create a default keyspace as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cassandra.cluster import Cluster\n",
    "\n",
    "# Connect to DSE7\n",
    "cluster = Cluster([\"host.docker.internal\"])\n",
    "session = cluster.connect()\n",
    "\n",
    "def execute_statement(statement: str):\n",
    "    # This is a simple wrapper around executing CQL statements in our\n",
    "    # Cassandra cluster, and either raising an error or returning the results\n",
    "    try:\n",
    "        rows = session.execute(statement)\n",
    "        return rows.all()\n",
    "    except:\n",
    "        print(f\"Query Failed: {statement}\")\n",
    "        raise\n",
    "\n",
    "# Create the default keyspace\n",
    "execute_statement(\"CREATE KEYSPACE IF NOT EXISTS default_keyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}\")\n",
    "\n",
    "# Create the CC keyspace\n",
    "execute_statement(\"CREATE KEYSPACE IF NOT EXISTS creditcardtransactions WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the schema\n",
    "This schema allows for storing credit card transactions in a simple way. It also creates an SAI Index on the information column that enables the new Text Analyzer for keyword search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_schema_cql = f\"\"\"\n",
    "CREATE TABLE IF NOT EXISTS creditcardtransactions.transactions (\n",
    "    credit_card_number text,\n",
    "    transaction_date date,\n",
    "    transaction_time time,\n",
    "    transaction_id uuid,\n",
    "    merchant text,\n",
    "    information text,\n",
    "    amount decimal,\n",
    "    currency text,\n",
    "    status text,\n",
    "    PRIMARY KEY ((credit_card_number), transaction_date, transaction_time, transaction_id)\n",
    ") WITH CLUSTERING ORDER BY (transaction_date DESC, transaction_time DESC);\n",
    "\n",
    "CREATE CUSTOM INDEX IF NOT EXISTS transactions_information_idx ON creditcardtransactions.transactions (information) USING 'org.apache.cassandra.index.sai.StorageAttachedIndex' WITH OPTIONS = {{ 'index_analyzer': 'standard'}};\n",
    "\"\"\"\n",
    "\n",
    "# This parses the text above into executable strings by the driver\n",
    "for statement in create_schema_cql.split(\";\"):\n",
    "    if len(statement.strip()):\n",
    "        execute_statement(statement.strip())\n",
    "\n",
    "print(\"Done\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load some data\n",
    "The following step loads some sample data into the database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_data_cql = f\"\"\"\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-01', '08:30:00', uuid(), 'Coffee Shop', 'Latte and croissant', 5.75, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-02', '12:45:00', uuid(), 'Restaurant', 'Lunch with colleagues', 20.50, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-03', '18:30:00', uuid(), 'Cinema', 'Movie night - 2 tickets', 15.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-05', '09:00:00', uuid(), 'Gas Station', 'Fuel for the week', 45.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-08', '15:00:00', uuid(), 'Online Store', 'Books and electronics', 120.95, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-10', '20:00:00', uuid(), 'Italian Restaurant', 'Dinner date', 60.30, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-15', '14:00:00', uuid(), 'Bookstore', 'Educational books', 35.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-18', '11:00:00', uuid(), 'Grocery Store', 'Weekly groceries', 90.75, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-20', '07:45:00', uuid(), 'Coffee Shop', 'Morning coffee', 3.50, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-23', '19:00:00', uuid(), 'Electronics Store', 'New headphones', 75.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-25', '10:30:00', uuid(), 'Local Bakery', 'Weekend breakfast treats', 8.50, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-26', '15:20:00', uuid(), 'Sporting Goods Store', 'Running shoes', 110.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-27', '20:00:00', uuid(), 'Online Subscription Service', 'Monthly streaming service fee', 12.99, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-29', '13:00:00', uuid(), 'Pharmacy', 'Medication', 23.45, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-30', '08:15:00', uuid(), 'Public Transit', 'Monthly bus pass', 45.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-03-31', '18:30:00', uuid(), 'Concert Venue', 'Concert tickets', 90.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-04-01', '12:00:00', uuid(), 'Pet Store', 'Pet food and toys', 60.75, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-04-02', '14:30:00', uuid(), 'Home Improvement Store', 'Gardening supplies', 35.20, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-04-03', '19:45:00', uuid(), 'Diner', 'Dinner with friends', 50.00, 'USD', 'Completed');\n",
    "INSERT INTO creditcardtransactions.transactions (credit_card_number, transaction_date, transaction_time, transaction_id, merchant, information, amount, currency, status) VALUES ('1234567890123456', '2023-04-04', '17:00:00', uuid(), 'Electronics Store', 'Smartphone case', 25.00, 'USD', 'Completed');\n",
    "\"\"\"\n",
    "\n",
    "# This parses the text above into executable strings by the driver\n",
    "for statement in create_data_cql.split(\";\"):\n",
    "    if len(statement.strip()):\n",
    "        execute_statement(statement.strip())\n",
    "\n",
    "print(\"Done\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's add some metadata to the table\n",
    "This metadata helps the LLM to understand which tables to use for querying"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_comments_cql = f\"\"\"\n",
    "ALTER TABLE creditcardtransactions.transactions WITH comment = 'Credit card transactions. Notable columns are: transaction_date, merchant and amount. Column information holds additional information about the purchase.';\n",
    "\"\"\"\n",
    "\n",
    "# This parses the text above into executable strings by the driver\n",
    "for statement in add_comments_cql.split(\";\"):\n",
    "    if len(statement.strip()):\n",
    "        execute_statement(statement.strip())\n",
    "\n",
    "print(\"Done\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the Natural Language to CQL conversion through the LLM\n",
    "The code below runs the following steps:\n",
    "1. It first creates a prompt that tells LLama2 how to handle user queries and convert them to CQL\n",
    "2. It then takes the resulting CQL and executes this on DSE 7\n",
    "3. Finally, the results are returned to the user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TEXT2CQL_PROMPT = \"\"\"Convert the question to CQL (Cassandra Query Language) that can retrieve an appropriate answer, \n",
    "or answer saying that the data model does not support answering such a question in a performant way. \n",
    "\n",
    "Use the following information about the schema to construct CQL queries:\n",
    "\n",
    "[Schema : values (type)]\n",
    "{schema}\n",
    "\n",
    "[Partition Keys]\n",
    "{partition_keys}\n",
    "\n",
    "[Clustering Keys]\n",
    "{clustering_keys}\n",
    "\n",
    "[SAI Indexes]\n",
    "creditcardtransactions.transactions : information (text)\n",
    "\n",
    "In case of full text search ONLY, use the following example to construct CQL:\n",
    "SELECT * FROM creditcardtransactions.transactions WHERE information : 'word'\n",
    "Make sure to use the colon as stated! Do this instead of using CONTAINS or LIKE.\n",
    "\n",
    "Only respond with the actual CQL and don't explain or provide notes!\n",
    "\n",
    "[Question]\n",
    "{question}\n",
    "\n",
    "[CQL]\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "def generate_schema_partition_clustering_keys(keyspace: str = \"default_keyspace\") -> (str, str):\n",
    "    \"\"\"Generates a TEXT2CQL_PROMPT compatible schema for a keyspace\"\"\"\n",
    "    # Get all table names in our keyspace\n",
    "    table_names = execute_statement(\n",
    "        f\"SELECT table_name, comment FROM system_schema.tables WHERE keyspace_name = '{keyspace}'\"\n",
    "    )\n",
    "    tn_str = \", \".join([\"'\" + tn.table_name + \"'\" for tn in table_names])\n",
    "\n",
    "    # Now get all the column names corresponding to those tables\n",
    "    columns = execute_statement(\n",
    "        f\"SELECT * FROM system_schema.columns WHERE table_name IN ({tn_str}) AND keyspace_name = '{keyspace}' ALLOW FILTERING\"\n",
    "    )\n",
    "\n",
    "    # Now, we construct our prompt template formatted schema, partition_keys, and clustering keys\n",
    "    # from the table and column objects returned from the DB\n",
    "    schema = \" | \".join([\n",
    "        f\"{keyspace}.{table.table_name} '{table.comment}' : \" + \" , \".join([\n",
    "            f\"{col.column_name} ({col.type})\"\n",
    "            for col in columns\n",
    "            if col.table_name == table.table_name\n",
    "        ])\n",
    "        for table in table_names\n",
    "    ])\n",
    "    partition_keys = \" | \".join([\n",
    "        f\"{keyspace}.{table.table_name} : \" + \" , \".join([\n",
    "            col.column_name for col in columns\n",
    "            if col.table_name == table.table_name\n",
    "            and col.kind == \"partition_key\"\n",
    "        ])\n",
    "        for table in table_names\n",
    "    ])\n",
    "    clustering_keys = \" | \".join([\n",
    "        f\"{keyspace}.{table.table_name} : \" + \" , \".join([\n",
    "            f\"{col.column_name} ({col.clustering_order})\" for col in columns\n",
    "            if col.table_name == table.table_name\n",
    "            and col.kind == \"clustering\"\n",
    "        ])\n",
    "        for table in table_names\n",
    "    ])\n",
    "    return schema, partition_keys, clustering_keys\n",
    "\n",
    "\n",
    "def execute_query_from_question(question: str, debug_cql: bool = True, debug_prompt: bool = False, return_cql: bool = False):\n",
    "    \"\"\"Generates and executes CQL from a user question based on LLM output\"\"\"\n",
    "    # Get all of the variables necessary to fill out the prompt\n",
    "    schema, partition_keys, clustering_keys = generate_schema_partition_clustering_keys(keyspace=\"creditcardtransactions\")\n",
    "    prompt = TEXT2CQL_PROMPT.format(\n",
    "        schema=schema,\n",
    "        partition_keys=partition_keys,\n",
    "        clustering_keys=clustering_keys,\n",
    "        question=question,\n",
    "    )\n",
    "\n",
    "    if debug_prompt:\n",
    "        print(f\"Prompting model with:\\n{prompt}\")\n",
    "\n",
    "    # Get generated CQL from the LLM\n",
    "    messages = [\n",
    "        HumanMessage(content=prompt),\n",
    "    ]\n",
    "    completion = client.invoke(messages).content\n",
    "\n",
    "    if debug_cql:\n",
    "        print(f\"Question: {question}\\nGenerated Query: {completion}\\n\")\n",
    "\n",
    "    # Need to trim trailing ';' if present to work with cassandra-driver\n",
    "    if completion.find(\";\") > -1:\n",
    "        completion = completion[:completion.find(\";\")]\n",
    "    # Need to drop backticks\n",
    "    completion = completion.replace('`', '')\n",
    "\n",
    "    results = execute_statement(completion)\n",
    "\n",
    "    if return_cql:\n",
    "        return (results, completion)\n",
    "    else:\n",
    "        return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show full prompting trace\n",
    "execute_query_from_question(\"What did I purchase on 2023-04-04? My credit card number is 1234567890123456\", debug_prompt=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show full prompting trace\n",
    "execute_query_from_question(\"Where did I purchase coffee? My credit card number is 1234567890123456\", debug_prompt=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now we get a full question-answer chain going!\n",
    "In this step we'll use all of teh above, but will ask the LLM now to construct a friendly answer for an end-user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ANSWER_PROMPT = \"\"\"Query:\n",
    "```\n",
    "{cql}\n",
    "```\n",
    "\n",
    "Output:\n",
    "```\n",
    "{results_repr}\n",
    "```\n",
    "===\n",
    "\n",
    "Given the above results from querying the DB, answer the following user question.\n",
    "When answering the question don't answer in a technical way, hence answer as if you're talking to a layman.\n",
    "\n",
    "{question}\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "def answer_question(question: str, debug_cql: bool = False, debug_prompt: bool = False) -> str:\n",
    "    \"\"\"Conducts a full RAG pipeline where the LLM retrieves relevant information\n",
    "    and references it to answer the question in natural language.\n",
    "    \"\"\"\n",
    "    # Get necessary fields to fill out prompt\n",
    "    query_results, cql = execute_query_from_question(\n",
    "        question=question,\n",
    "        debug_cql=debug_cql,\n",
    "        debug_prompt=debug_prompt,\n",
    "        return_cql=True,\n",
    "    )\n",
    "    prompt = ANSWER_PROMPT.format(\n",
    "        question=question,\n",
    "        results_repr=str(query_results),\n",
    "        cql=cql,\n",
    "    )\n",
    "\n",
    "    if debug_prompt:\n",
    "        print(f\"Prompting model with:\\n{prompt}\")\n",
    "\n",
    "    # Get generated answer from the LLM\n",
    "    messages = [\n",
    "        HumanMessage(content=prompt),\n",
    "    ]\n",
    "    return client.invoke(messages).content\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show full prompting trace\n",
    "print(\n",
    "    answer_question(\"Where did I purchase coffee? My credit card number is 1234567890123456\", debug_prompt=True)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show full prompting trace\n",
    "print(\n",
    "    answer_question(\"How many coffees did I purchase in March 2023? My credit card number is 1234567890123456\", debug_prompt=True)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show full prompting trace\n",
    "print(\n",
    "    answer_question(\"What's the total amount of money I spent in March 2023? My credit card number is 1234567890123456\", debug_prompt=True)\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "04f92315dc0543a099514c406e62c1a5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3b41855991134da2a274def356ede94c",
       "IPY_MODEL_da4e53df4a364f0db19258c2afe96abb",
       "IPY_MODEL_08e3ecaedca8462386e99eef3eba3dc0"
      ],
      "layout": "IPY_MODEL_5598a649cb4a45928a0f8836e13cd9fc"
     }
    },
    "08e3ecaedca8462386e99eef3eba3dc0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e42c1f51deee4f96bb01123e664fb1f5",
      "placeholder": "​",
      "style": "IPY_MODEL_375c60f1f4ff44d684b822b1b7a62464",
      "value": " 15.6k/15.6k [00:00&lt;00:00, 230kB/s]"
     }
    },
    "15cc610694344258ba5c0a765ce73002": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "375c60f1f4ff44d684b822b1b7a62464": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3b41855991134da2a274def356ede94c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e9305cb306a64ffe825922010b537673",
      "placeholder": "​",
      "style": "IPY_MODEL_80b092aa458e4d3684a7ca32d018d592",
      "value": "Downloading readme: 100%"
     }
    },
    "4acac7af66bd4a7daca02a095f34dda3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5598a649cb4a45928a0f8836e13cd9fc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "80b092aa458e4d3684a7ca32d018d592": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "da4e53df4a364f0db19258c2afe96abb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_15cc610694344258ba5c0a765ce73002",
      "max": 15586,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4acac7af66bd4a7daca02a095f34dda3",
      "value": 15586
     }
    },
    "e42c1f51deee4f96bb01123e664fb1f5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e9305cb306a64ffe825922010b537673": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}