bootcamp/RAG/zilliz_pipeline_rag.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Build RAG using Zilliz Cloud Pipelines\n",
    "\n",
    "[Zilliz Cloud Pipelines](https://docs.zilliz.com/docs/pipelines) is AI-powered retrieval service. It simplifies the maintenance of information retrieval system by providing ingestion and search pipelines as easy-to-use API service. As an AI application developer, with quality optimization and devops taken care of, you can focus on building AI applications tailored to your specific use case.\n",
    "\n",
    "In this notebook, we show how to use [Zilliz Cloud Pipelines](https://zilliz.com/zilliz-cloud-pipelines) to build a simple yet scalable [Retrieval Augmented Generation (RAG)](https://zilliz.com/use-cases/llm-retrieval-augmented-generation) application. Retrieval is at the heart of RAG solution, which typically involves maintaining a knowledge base with document parsing and chunking, hosting an embedding model and using vector database as retrieval engine. With Zilliz Cloud Pipelines, you don't need to deal with such a complex tech stack. Everything can be done with an API call.\n",
    "\n",
    "We first create the an Ingestion pipeline for document indexing and a Search pipeline for knowledge retrieval. Then we run Ingestion pipeline by API call to import documents to establish the knowledge base. Finally, we build an RAG application that runs Search pipeline to conduct Retrieval Augmented Generation.\n",
    "\n",
    "![](../../images/rag_and_pipeline.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Setup\n",
    "### Prerequisites\n",
    "Please make sure you have a Serverless cluster in Zilliz Cloud. If not already, you can [sign up for free](https://cloud.zilliz.com/signup?utm_source=referral&utm_medium=partner&utm_campaign=2023-12-21_github-docs_zilliz-pipeline-rag_github).\n",
    "\n",
    "To learn how to create a Serverless cluster and get your CLOUD_REGION, CLUSTER_ID, API_KEY and PROJECT_ID, please refer to this [page](https://docs.zilliz.com/docs/create-cluster) for more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the Serverless Cluster created, please get the cluster id, API key and project id as shown and fill in the following code:\n",
    "\n",
    "![](../../images/zilliz_api_key_cluster_id.jpeg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "CLOUD_REGION = 'gcp-us-west1'\n",
    "CLUSTER_ID = 'your CLUSTER_ID'\n",
    "API_KEY = 'your API_KEY'\n",
    "PROJECT_ID = 'your PROJECT_ID'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Create an ingestion pipeline\n",
    "[Ingestion pipelines](https://docs.zilliz.com/docs/understanding-pipelines#ingestion-pipelines) can transform unstructured data into searchable vector embeddings and store them in Zilliz Cloud Vector Database.\n",
    "\n",
    "In the Ingestion pipeline, you can specify functions to customize its behavior. The input data that Ingestion pipeline expects also depends on the specified functions. Currently, Ingestion pipeline allows two types of functions:\n",
    "\n",
    "- The `INDEX_DOC` function expects a document as input. It splits the input text document into chunks and generates a vector embedding for each chunk. This function maps an input field (doc_url) to four output fields (doc_name, chunk_id, chunk_text, and embedding) in the auto-generated collection.\n",
    "- The `PRESERVE` function stores a user-defined input as additional [scalar](https://milvus.io/docs/scalar_index.md) field in the auto-generated collection. This is typically used to store meta information of the document, such as publisher info, tags that describes the property of the file.\n",
    "\n",
    "In the following example we create an Ingestion pipeline with an `INDEX_DOC` function and a `PRESERVE` function. As part of creating the Ingestion pipeline, a vector database collection named `my_rag_collection` will be created in the cluster. It contains five fields:\n",
    "- `doc_name`, `chunk_id`, `chunk_text`, `embedding` as defined by `INDEX_DOC` function\n",
    "- `version` as defined by `PRESERVE` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'code': 200, 'data': {'pipelineId': 'pipe-d5a147fbdbdb7a13cae98a', 'name': 'my_ingestion_pipeline', 'type': 'INGESTION', 'description': 'A pipeline that splits a text file into chunks and generates embeddings. It also stores the doc version with each chunk.', 'status': 'SERVING', 'functions': [{'action': 'INDEX_DOC', 'name': 'index_my_doc', 'inputField': 'doc_url', 'language': 'ENGLISH', 'chunkSize': 500}, {'action': 'PRESERVE', 'name': 'keep_doc_info', 'inputField': 'version', 'outputField': 'version', 'fieldType': 'VarChar'}], 'clusterId': 'in03-423dca989cc7410', 'newCollectionName': 'my_rag_collection', 'totalTokenUsage': 0}}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "headers = {\n",
    "    \"Content-Type\": \"application/json\",\n",
    "    \"Accept\": \"application/json\",\n",
    "    \"Authorization\": f\"Bearer {API_KEY}\"\n",
    "}\n",
    "\n",
    "create_pipeline_url = f\"https://controller.api.{CLOUD_REGION}.zillizcloud.com/v1/pipelines\"\n",
    "\n",
    "collection_name = 'my_rag_collection'\n",
    "\n",
    "data = {\n",
    "    \"projectId\": PROJECT_ID,\n",
    "    \"name\": \"my_ingestion_pipeline\",\n",
    "    \"description\": \"A pipeline that splits a text file into chunks and generates embeddings. It also stores the doc version with each chunk.\",\n",
    "    \"type\": \"INGESTION\",\n",
    "    \"functions\": [\n",
    "        {\n",
    "            \"name\": \"index_my_doc\",\n",
    "            \"action\": \"INDEX_DOC\",\n",
    "            \"inputField\": \"doc_url\",\n",
    "            \"language\": \"ENGLISH\",\n",
    "            \"chunkSize\": 500,\n",
    "        },\n",
    "        {\n",
    "            \"name\": \"keep_doc_info\",\n",
    "            \"action\": \"PRESERVE\",\n",
    "            \"inputField\": \"version\",\n",
    "            \"outputField\": \"version\",\n",
    "            \"fieldType\": \"VarChar\"\n",
    "        }\n",
    "    ],\n",
    "    \"clusterId\": f\"{CLUSTER_ID}\",\n",
    "    \"newCollectionName\": f\"{collection_name}\"\n",
    "}\n",
    "\n",
    "response = requests.post(create_pipeline_url, headers=headers, json=data)\n",
    "print(response.json())\n",
    "ingestion_pipe_id = response.json()[\"data\"][\"pipelineId\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "After successful creation, it will return a pipeline ID. We will run this pipeline later with pipeline ID to ingest a document."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Create a search pipeline\n",
    "[Search pipelines](https://docs.zilliz.com/docs/understanding-pipelines#search-pipelines) enables semantic search by converting a query string into a vector embedding and then retrieving top-K nearest neighbour vectors, each vector represents a chunk of ingested document and carries other associated information such as file name and preserved properties.\n",
    "\n",
    "A Search pipeline contains one type of function `SEARCH_DOC_CHUNK`, in which you need to set the the cluster and collection to search from.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'code': 200, 'data': {'pipelineId': 'pipe-9d84aa1ed59d6b641c51f6', 'name': 'my_search_pipeline', 'type': 'SEARCH', 'description': 'A pipeline that receives text and search for semantically similar doc chunks', 'status': 'SERVING', 'functions': [{'action': 'SEARCH_DOC_CHUNK', 'name': 'search_chunk_text_and_title', 'inputField': 'query_text', 'clusterId': 'in03-423dca989cc7410', 'collectionName': 'my_rag_collection'}], 'totalTokenUsage': 0}}\n"
     ]
    }
   ],
   "source": [
    "data = {\n",
    "    \"projectId\": PROJECT_ID,\n",
    "    \"name\": \"my_search_pipeline\",\n",
    "    \"description\": \"A pipeline that receives text and search for semantically similar doc chunks\",\n",
    "    \"type\": \"SEARCH\",\n",
    "    \"functions\": [\n",
    "        {\n",
    "            \"name\": \"search_chunk_text_and_title\",\n",
    "            \"action\": \"SEARCH_DOC_CHUNK\",\n",
    "            \"inputField\": \"query_text\",\n",
    "            \"clusterId\": f\"{CLUSTER_ID}\",\n",
    "            \"collectionName\": f\"{collection_name}\"\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "response = requests.post(create_pipeline_url, headers=headers, json=data)\n",
    "\n",
    "print(response.json())\n",
    "search_pipe_id = response.json()[\"data\"][\"pipelineId\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Similarly, after successful creation, it will return a pipeline ID. We will run this pipeline later and will use this pipeline ID."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the creating pipelines through [RESTful API](https://docs.zilliz.com/docs/create-piplines-rest) as introduced in this notebook, you can also create pipelines through [Web UI](https://docs.zilliz.com/docs/create-piplines-gui) with a few clicks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Run ingestion pipeline\n",
    "\n",
    "Ingestion pipeline accepts files from Object Storage Service such as [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) or [Google Cloud Storage (GCS)](https://cloud.google.com/storage/docs/uploads-downloads). Supported file types include `.txt`, `.pdf`, `.md`, `.html`, `.epub`, `.csv`, `.doc`, `.docx`, `.xls`, `.xlsx`, `.ppt`, `.pptx`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "In the following demo, we run ingestion pipeline with the file [milvus_doc.md](https://publicdataset.zillizcloud.com/milvus_doc.md) stored on Google Cloud Storage and attach its version info with each indexed doc chunk. The file explains how to use Milvus (an open-source vector database)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'code': 200, 'data': {'token_usage': 1247, 'doc_name': 'milvus_doc.md', 'num_chunks': 10}}\n"
     ]
    }
   ],
   "source": [
    "gcs_url = 'https://publicdataset.zillizcloud.com/milvus_doc.md'  # public or pre-signed url of a file stored on AWS S3 or Google Cloud Storage\n",
    "\n",
    "\n",
    "run_pipeline_url = f\"https://controller.api.{CLOUD_REGION}.zillizcloud.com/v1/pipelines/{ingestion_pipe_id}/run\"\n",
    "\n",
    "data = {\n",
    "    \"data\":\n",
    "        {\n",
    "            \"doc_url\": f\"{gcs_url}\",\n",
    "            \"version\": '2.3'\n",
    "        }\n",
    "}\n",
    "\n",
    "response = requests.post(run_pipeline_url, headers=headers, json=data)\n",
    "\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now we have successfully ingested the document by splitting it into doc chunks and uploading the generated embedding into the vector database collection. If you want to inspect the data in the collection, you can use the Data Preview tool in [Zilliz Cloud web UI](https://cloud.zilliz.com)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Build RAG application with Search pipeline\n",
    "\n",
    "### Run search pipeline\n",
    "The first step in building an RAG app is to retrieve information pieces (doc chunks) most relevant to the question from a knowledge base (typically a vector database collection).\n",
    "\n",
    "This is as simple as running a Search pipeline that we just created above. Following is how to run a Search pipeline with query text and specifications, and we wrap this run with a function that can be used in the RAG app we will show shortly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'code': 200,\n",
      " 'data': {'result': [{'chunk_id': 3,\n",
      "                      'chunk_text': '# Delete Entities\\n'\n",
      "                                    '## Prepare boolean expression\\n'\n",
      "                                    '### Complex boolean expression\\n'\n",
      "                                    'To filter entities that meet specific '\n",
      "                                    'conditions, define complex boolean '\n",
      "                                    'expressions.  \\n'\n",
      "                                    'Filter entities whose word_count is '\n",
      "                                    'greater than or equal to 11000:  \\n'\n",
      "                                    '```python\\n'\n",
      "                                    'expr = \"word_count >= 11000\"\\n'\n",
      "                                    '```  \\n'\n",
      "                                    'Filter entities whose book_name is not '\n",
      "                                    'Unknown:  \\n'\n",
      "                                    '```python\\n'\n",
      "                                    'expr = \"book_name != Unknown\"\\n'\n",
      "                                    '```  \\n'\n",
      "                                    'Filter entities whose primary key values '\n",
      "                                    'are greater than 5 and word_count is '\n",
      "                                    'smaller than or equal to 9999:  \\n'\n",
      "                                    '```python\\n'\n",
      "                                    'expr = \"book_id > 5 && word_count <= '\n",
      "                                    '9999\"\\n'\n",
      "                                    '```',\n",
      "                      'distance': 0.7901315689086914,\n",
      "                      'doc_name': 'milvus_doc.md',\n",
      "                      'id': 447197973648703557,\n",
      "                      'version': '2.3'},\n",
      "                     {'chunk_id': 0,\n",
      "                      'chunk_text': '# Delete Entities\\n'\n",
      "                                    'This topic describes how to delete '\n",
      "                                    'entities in Milvus.  \\n'\n",
      "                                    'Milvus supports deleting entities by '\n",
      "                                    'primary key or complex boolean '\n",
      "                                    'expressions. Deleting entities by primary '\n",
      "                                    'key is much faster and lighter than '\n",
      "                                    'deleting them by complex boolean '\n",
      "                                    'expressions. This is because Milvus '\n",
      "                                    'executes queries first when deleting data '\n",
      "                                    'by complex boolean expressions.  \\n'\n",
      "                                    'Deleted entities can still be retrieved '\n",
      "                                    'immediately after the deletion if the '\n",
      "                                    'consistency level is set lower than '\n",
      "                                    'Strong.\\n'\n",
      "                                    'Entities deleted beyond the pre-specified '\n",
      "                                    'span of time for Time Travel cannot be '\n",
      "                                    'retrieved again.\\n'\n",
      "                                    'Frequent deletion operations will impact '\n",
      "                                    'the system performance.  \\n'\n",
      "                                    'Before deleting entities by comlpex '\n",
      "                                    'boolean expressions, make sure the '\n",
      "                                    'collection has been loaded.\\n'\n",
      "                                    'Deleting entities by complex boolean '\n",
      "                                    'expressions is not an atomic operation. '\n",
      "                                    'Therefore, if it fails halfway through, '\n",
      "                                    'some data may still be deleted.\\n'\n",
      "                                    'Deleting entities by complex boolean '\n",
      "                                    'expressions is supported only when the '\n",
      "                                    'consistency is set to Bounded. For '\n",
      "                                    'details, see Consistency.',\n",
      "                      'distance': 0.7727086544036865,\n",
      "                      'doc_name': 'milvus_doc.md',\n",
      "                      'id': 447197973648703554,\n",
      "                      'version': '2.3'}],\n",
      "          'token_usage': 21}}\n"
     ]
    },
    {
     "data": {
      "text/plain": "[{'chunk_text': '# Delete Entities\\n## Prepare boolean expression\\n### Complex boolean expression\\nTo filter entities that meet specific conditions, define complex boolean expressions.  \\nFilter entities whose word_count is greater than or equal to 11000:  \\n```python\\nexpr = \"word_count >= 11000\"\\n```  \\nFilter entities whose book_name is not Unknown:  \\n```python\\nexpr = \"book_name != Unknown\"\\n```  \\nFilter entities whose primary key values are greater than 5 and word_count is smaller than or equal to 9999:  \\n```python\\nexpr = \"book_id > 5 && word_count <= 9999\"\\n```',\n  'version': '2.3'},\n {'chunk_text': '# Delete Entities\\nThis topic describes how to delete entities in Milvus.  \\nMilvus supports deleting entities by primary key or complex boolean expressions. Deleting entities by primary key is much faster and lighter than deleting them by complex boolean expressions. This is because Milvus executes queries first when deleting data by complex boolean expressions.  \\nDeleted entities can still be retrieved immediately after the deletion if the consistency level is set lower than Strong.\\nEntities deleted beyond the pre-specified span of time for Time Travel cannot be retrieved again.\\nFrequent deletion operations will impact the system performance.  \\nBefore deleting entities by comlpex boolean expressions, make sure the collection has been loaded.\\nDeleting entities by complex boolean expressions is not an atomic operation. Therefore, if it fails halfway through, some data may still be deleted.\\nDeleting entities by complex boolean expressions is supported only when the consistency is set to Bounded. For details, see Consistency.',\n  'version': '2.3'}]"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pprint\n",
    "\n",
    "\n",
    "def retrieval_with_pipeline(question, search_pipe_id, top_k=2, verbose=False):\n",
    "    run_pipeline_url = f\"https://controller.api.{CLOUD_REGION}.zillizcloud.com/v1/pipelines/{search_pipe_id}/run\"\n",
    "\n",
    "    data = {\n",
    "        \"data\": {\n",
    "            \"query_text\": question\n",
    "        },\n",
    "        \"params\": {\n",
    "            \"limit\": top_k,\n",
    "            \"offset\": 0,\n",
    "            \"outputFields\": [\n",
    "                \"chunk_text\",\n",
    "                \"chunk_id\",\n",
    "                \"doc_name\",\n",
    "                \"version\"\n",
    "            ],\n",
    "        }\n",
    "    }\n",
    "    response = requests.post(run_pipeline_url, headers=headers, json=data)\n",
    "    if verbose:\n",
    "        pprint.pprint(response.json())\n",
    "    results = response.json()[\"data\"][\"result\"]\n",
    "    retrieved_texts = [{'chunk_text': result['chunk_text'], 'version': result['version']} for result in results]\n",
    "    return retrieved_texts\n",
    "\n",
    "\n",
    "question = 'Can users delete entities by complex boolean expressions?'\n",
    "retrieval_with_pipeline(question, search_pipe_id, top_k=2, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We can see that when we ask a question, this search run can return the top k knowledge fragments we need. This is also a basis for forming RAG."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Build a chatbot powered by RAG \n",
    "With the above convenient helper function `retrieval_with_pipeline`, we can retrieve the knowledge ingested into the vector database.\n",
    "Below, we show a simple RAG app that can answer based on the knowledge we have ingested previously. It uses OpenAI `gpt-3.5-turbo` as LLM and a simple prompt. To test it, you can replace with your own OpenAI API Key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI()\n",
    "client.api_key = os.getenv('OPENAI_API_KEY')  # your OpenAI API key\n",
    "\n",
    "\n",
    "class Chatbot:\n",
    "    def __init__(self, search_pipe_id):\n",
    "        self._search_pipe_id = search_pipe_id\n",
    "\n",
    "    def retrieve(self, query: str) -> list:\n",
    "        \"\"\"\n",
    "        Retrieve relevant text with Zilliz Cloud Pipelines.\n",
    "        \"\"\"\n",
    "        results = retrieval_with_pipeline(query, self._search_pipe_id, top_k=2)\n",
    "        return results\n",
    "\n",
    "    def generate_answer(self, query: str, context_str: list) -> str:\n",
    "        \"\"\"\n",
    "        Generate answer based on context, which is from the result of Search pipeline run.\n",
    "        \"\"\"\n",
    "        completion = client.chat.completions.create(\n",
    "            model=\"gpt-3.5-turbo\",\n",
    "            temperature=0,\n",
    "            messages=\n",
    "            [\n",
    "                {\"role\": \"user\",\n",
    "                 \"content\":\n",
    "                     f\"We have provided context information below. \\n\"\n",
    "                     f\"---------------------\\n\"\n",
    "                     f\"{context_str}\"\n",
    "                     f\"\\n---------------------\\n\"\n",
    "                     f\"Given this information, please answer the question: {query}\"\n",
    "                 }\n",
    "            ]\n",
    "        ).choices[0].message.content\n",
    "        return completion\n",
    "\n",
    "    def chat_with_rag(self, query: str) -> str:\n",
    "        context_str = self.retrieve(query)\n",
    "        completion = self.generate_answer(query, context_str)\n",
    "        return completion\n",
    "\n",
    "    def chat_without_rag(self, query: str) -> str:\n",
    "        return client.chat.completions.create(\n",
    "            model=\"gpt-3.5-turbo\",\n",
    "            temperature=0,\n",
    "            messages=\n",
    "            [\n",
    "                {\"role\": \"user\",\n",
    "                 \"content\": query\n",
    "                 }\n",
    "            ]\n",
    "        ).choices[0].message.content\n",
    "\n",
    "chatbot = Chatbot(search_pipe_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "This implements an RAG chatbot, it will use Search pipeline to retrieve the most relevant chunks from ingested documents, and enhance the answer quality with it. Let's see how it works in action!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Chat with RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "'Yes, in Milvus 2.3, users can delete entities by complex boolean expressions.'"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "question = 'In Milvus 2.3, can users delete entities by complex boolean expressions?'\n",
    "chatbot.chat_with_rag(question)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The ground truth content in the original knowledge text is:\n",
    "> **Milvus supports deleting entities by primary key or complex boolean expressions**. Deleting entities by primary key is much faster and lighter than deleting them by complex boolean expressions. This is because Milvus executes queries first when deleting data by complex boolean expressions.\n",
    "\n",
    "\n",
    "\n",
    "We can tell that the RAG we built successfully answers this question that requires deep domain knowledge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "'No, in Milvus 2.3, users cannot delete entities by complex boolean expressions. The delete operation in Milvus 2.3 is performed based on the entity IDs. Users need to provide the specific entity IDs that they want to delete.'"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chatbot.chat_without_rag(question)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "In opposite, the LLM without RAG doesn't have domain knowledge required for this question, even worse, it outputs incorrect answer. This is a typical example of the so called [hallucination](https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)) problem of LLM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's how to use Zilliz Cloud Pipelines to build RAG applications. To learn more, you can refer to https://docs.zilliz.com/docs/pipelines for detailed information.\n",
    "\n",
    "If you have any question, feel free to contact us at support@zilliz.com"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}