WatsonX-PgVector-LangChain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "\n",
    "# Use watsonx Granite Model Series, Postgre with DbVector, and LangChain to answer questions (RAG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Hello everyone in this demo we are going to build a simple program that will connect to a Postgre Server with DB vector and use LangChain to answer questions using RAG.\n",
    "\n",
    "This notebook contains the steps and code to demonstrate support of Retrieval Augumented Generation in watsonx.ai. It introduces commands for data retrieval, knowledge base building & querying, and model testing.\n",
    "\n",
    "Some familiarity with Python is helpful. This notebook uses Python 3.10.\n",
    "\n",
    "### About Retrieval Augmented Generation\n",
    "Retrieval Augmented Generation (RAG) is a versatile pattern that can unlock a number of use cases requiring factual recall of information, such as querying a knowledge base in natural language.\n",
    "\n",
    "In its simplest form, RAG requires 3 steps:\n",
    "\n",
    "- Index knowledge base passages (once)\n",
    "- Retrieve relevant passage(s) from knowledge base (for every user query)\n",
    "- Generate a response by feeding retrieved passage into a large language model (for every user query)\n",
    "\n",
    "## Contents\n",
    "\n",
    "This notebook contains the following parts:\n",
    "\n",
    "- [Setup](#setup)\n",
    "- [Document data loading](#data)\n",
    "- [Build up knowledge base](#build_base)\n",
    "- [Foundation Models on watsonx](#models)\n",
    "- [Generate a retrieval-augmented response to a question](#predict)\n",
    "- [Summary and next steps](#summary)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"setup\"></a>\n",
    "##  Set up the environment\n",
    "\n",
    "Before you use the sample code in this notebook, you must perform the following setup tasks:\n",
    "\n",
    "-  Create a <a href=\"https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/\" target=\"_blank\" rel=\"noopener no referrer\">Watson Machine Learning (WML) Service</a> instance (a free plan is offered and information about how to create the instance can be found <a href=\"https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-service-instance.html?context=analytics\" target=\"_blank\" rel=\"noopener no referrer\">here</a>).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Install and import the dependecies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output\n",
    "!pip install \"langchain==0.0.345\" \n",
    "!pip install wget \n",
    "!pip install sentence-transformers \n",
    "!pip install \"chromadb==0.3.26\" \n",
    "!pip install \"ibm-watson-machine-learning>=1.0.335\" \n",
    "!pip install \"pydantic==1.10.0\"\n",
    "!pip install python-dotenv\n",
    "clear_output()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "#conda install -c pytorch faiss-cpu=1.7.4 mkl=2021 blas=1.0=mkl\n",
    "#!pip install langchain[all]\n",
    "#!pip3 install Langchain[FAISS]\n",
    "#!pip install faiss-cpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### watsonx API connection\n",
    "This cell defines the credentials required to work with watsonx API for Foundation\n",
    "Model inferencing.\n",
    "\n",
    "Add the IBM Cloud user API key to the .env file. For details, see <a href=\"https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui\" target=\"_blank\" rel=\"noopener no referrer\">documentation</a>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os, getpass\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "project_id = os.getenv(\"PROJECT_ID\", None)\n",
    "credentials = {\n",
    "    #\"url\":  \"https://eu-de.ml.cloud.ibm.com\",\n",
    "    \"url\": \"https://us-south.ml.cloud.ibm.com\",\n",
    "    \"apikey\": os.getenv(\"API_KEY\", None)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Defining the project id\n",
    "The API requires project id that provides the context for the call. We will obtain the id from the project in which this notebook runs. Otherwise, please provide the project id.\n",
    "\n",
    "**Hint**: You can find the `project_id` as follows. Open the prompt lab in watsonx.ai. At the very top of the UI, there will be `Projects / <project name> /`. Click on the `<project name>` link. Then get the `project_id` from Project's Manage tab (Project -> Manage -> General -> Details).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    project_id = os.environ[\"PROJECT_ID\"]\n",
    "except KeyError:\n",
    "    project_id = input(\"Please enter your project_id (hit enter): \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vector stores\n",
    "\n",
    "A vector store is a system or database that is specifically designed to store and search embedded data using vector representations. It allows users to store unstructured data by converting it into embedding vectors, which capture the semantic meaning and relationships between different data points. At query time, the system can then compare the embedded query to the stored vectors and retrieve the most similar ones. The vector store manages the storage and indexing of the embedded data, as well as the efficient retrieval of relevant results during vector search."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](img/vector_stores.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"data\"></a>\n",
    "## Document data loading\n",
    "\n",
    "Download the file with State of the Union."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import wget\n",
    "filename = 'state_of_the_union.txt'\n",
    "url = 'https://raw.github.com/IBM/watson-machine-learning-samples/master/cloud/data/foundation_models/state_of_the_union.txt'\n",
    "if not os.path.isfile(filename):\n",
    "    wget.download(url, out=filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"build_base\"></a>\n",
    "## Build up knowledge base\n",
    "\n",
    "The most common approach in RAG is to create dense vector representations of the knowledge base in order to calculate the semantic similarity to a given user query.\n",
    "\n",
    "In this basic example, we take the State of the Union speech content (filename), split it into chunks, embed it using an open-source embedding model,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "loader = TextLoader(filename ,encoding='utf-8')\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "texts = text_splitter.split_documents(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The dataset we are using is already split into self-contained passages that can be ingested by Chroma."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Create an embedding function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.embeddings import HuggingFaceEmbeddings\n",
    "embeddings = HuggingFaceEmbeddings()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The performance of any vector store db may differ depending on the embedding model used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are different vector databases out there like\n",
    "- Chroma\n",
    "- FAISS\n",
    "- lancedb\n",
    "etc.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chroma\n",
    "Load it into <a href=\"https://www.trychroma.com/\" target=\"_blank\" rel=\"noopener no referrer\">Chroma</a>, and then query it.\n",
    "Note that you can feed a custom embedding function to be used by chromadb. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from langchain.vectorstores import Chroma\n",
    "db_chroma = Chroma.from_documents(texts, embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "langchain.vectorstores.chroma.Chroma"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(db_chroma)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  FAISS\n",
    "```python\n",
    "from langchain_community.vectorstores import FAISS\n",
    "db_faiss = FAISS.from_documents(documents, OpenAIEmbeddings())\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Postgre with PgVector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Howeve we want to use a standard database like postgre with the  PgVector feature.\n",
    "To this demo let us build our Docker container we create a file **docker-compose.yml**\n",
    "```\n",
    "services:\n",
    "  db:\n",
    "    hostname: db\n",
    "    image: ankane/pgvector\n",
    "    ports:\n",
    "     - 5432:5432\n",
    "    restart: always\n",
    "    environment:\n",
    "      - POSTGRES_DB=vectordb\n",
    "      - POSTGRES_USER=testuser\n",
    "      - POSTGRES_PASSWORD=testpwd\n",
    "      - POSTGRES_HOST_AUTH_METHOD=trust\n",
    "  #  volumes:\n",
    "      - ./init.sql:/docker-entrypoint-initdb.d/init.sql\n",
    "```\n",
    "with **init.sql**\n",
    "    \n",
    "\n",
    "```sql\n",
    "CREATE EXTENSION IF NOT EXISTS vector;\n",
    "\n",
    "CREATE TABLE IF NOT EXISTS embeddings (\n",
    "  id SERIAL PRIMARY KEY,\n",
    "  embedding vector,\n",
    "  text text,\n",
    "  created_at timestamptz DEFAULT now()\n",
    ");\n",
    "```\n",
    "and later just type\n",
    "```\n",
    "docker-compose up -d\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we need to build the CONNECTION  like:\n",
    "```python\n",
    "CONNECTION_STRING = \"postgresql+psycopg://user:password@postgresql-server:5432/vectordb\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "postgresql+psycopg://testuser:testpwd@localhost:5432/vectordb\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv\n",
    "# Load the .env file\n",
    "load_dotenv()\n",
    "\n",
    "# Get the values from the .env file\n",
    "user = \"testuser\"\n",
    "password =\"testpwd\"\n",
    "database = \"vectordb\"\n",
    "#server = \"af651cca01b154fe28a0df0167cad5a7-844854289.us-east-2.elb.amazonaws.com\"\n",
    "server=\"localhost\"\n",
    "# Construct the connection string\n",
    "CONNECTION_STRING = f\"postgresql+psycopg://{user}:{password}@{server}:5432/{database}\"\n",
    "# Print the connection string\n",
    "print(CONNECTION_STRING)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFDirectoryLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.embeddings.huggingface import HuggingFaceEmbeddings\n",
    "from langchain.vectorstores.pgvector import PGVector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Document loading from a folder containing PDFs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "pdf_folder_path = './rhods-doc'\n",
    "loader = PyPDFDirectoryLoader(pdf_folder_path)\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Split documents into chunks with some overlap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024,\n",
    "                                               chunk_overlap=40)\n",
    "all_splits_pdfs = text_splitter.split_documents(docs)\n",
    "#all_splits[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cleanup documents as PostgreSQL won't accept the NUL character, '\\x00', in TEXT fields."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "for doc in all_splits_pdfs:\n",
    "    doc.page_content = doc.page_content.replace('\\x00', '')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create the index and ingest the documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = HuggingFaceEmbeddings()\n",
    "\n",
    "COLLECTION_NAME = \"documents_test\"\n",
    "\n",
    "db = PGVector.from_documents(\n",
    "    documents=all_splits_pdfs,\n",
    "    embedding=embeddings,\n",
    "    collection_name=COLLECTION_NAME,\n",
    "    connection_string=CONNECTION_STRING,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"models\"></a>\n",
    "## Foundation Models on `watsonx.ai`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IBM watsonx foundation models are among the <a href=\"https://python.langchain.com/docs/integrations/llms/watsonxllm\" target=\"_blank\" rel=\"noopener no referrer\">list of LLM models supported by Langchain</a>. This example shows how to communicate with <a href=\"https://newsroom.ibm.com/2023-09-28-IBM-Announces-Availability-of-watsonx-Granite-Model-Series,-Client-Protections-for-IBM-watsonx-Models\" target=\"_blank\" rel=\"noopener no referrer\">Granite Model Series</a> using <a href=\"https://python.langchain.com/docs/get_started/introduction\" target=\"_blank\" rel=\"noopener no referrer\">Langchain</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Defining model\n",
    "You need to specify `model_id` that will be used for inferencing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from ibm_watson_machine_learning.foundation_models.utils.enums import ModelTypes\n",
    "\n",
    "model_id = ModelTypes.GRANITE_13B_CHAT_V2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Defining the model parameters\n",
    "We need to provide a set of model parameters that will influence the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from ibm_watson_machine_learning.metanames import GenTextParamsMetaNames as GenParams\n",
    "from ibm_watson_machine_learning.foundation_models.utils.enums import DecodingMethods\n",
    "\n",
    "parameters = {\n",
    "    GenParams.DECODING_METHOD: DecodingMethods.GREEDY,\n",
    "    GenParams.MIN_NEW_TOKENS: 1,\n",
    "    GenParams.MAX_NEW_TOKENS: 200,\n",
    "    GenParams.STOP_SEQUENCES: [\"<|endoftext|>\"]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LangChain CustomLLM wrapper for watsonx model\n",
    "Initialize the `WatsonxLLM` class from Langchain with defined parameters and `ibm/granite-13b-chat-v2`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.llms import WatsonxLLM\n",
    "\n",
    "watsonx_granite = WatsonxLLM(\n",
    "    model_id=model_id.value,\n",
    "    url=credentials.get(\"url\"),\n",
    "    apikey=credentials.get(\"apikey\"),\n",
    "    project_id=project_id,\n",
    "    params=parameters\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"predict\"></a>\n",
    "## Generate a retrieval-augmented response to a question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build the `RetrievalQA` (question answering chain) to automate the RAG task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "qa = RetrievalQA.from_chain_type(llm=watsonx_granite, chain_type=\"stuff\", retriever=db.as_retriever())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Select questions\n",
    "\n",
    "Get questions from the previously loaded test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\" A vector database is a database that can store vectors (fixed-length lists of numbers) along with other data items. Vector databases typically implement one or more Approximate Nearest Neighbor (ANN) algorithms, so that one can search the database with a query vector to retrieve the closest matching database records. Vectors are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These feature vectors may be computed from the raw data using machine learning methods such as feature extraction algorithms, word embeddings, or deep learning networks. The goal is that semantically similar data will be represented by similar vectors, allowing for efficient searching and retrieval of data.\""
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"What is vector database?\"\n",
    "qa.run(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ingesting new documents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our exploration of LangChain   Below are some key APIs from LangChain's \n",
    "- add_documents(): This function allows us to incorporate additional documents into the vector store.\n",
    "- add_embeddings(): It enables the addition of more embeddings to the vector store.\n",
    "- from_documents(): This API returns a VectorStore based on the provided documents.\n",
    "- from_embeddings(): This function provides aindex generated from the given embeddings.\n",
    "- similarity_search(): This function retrieves documents that are most similar to a given query.\n",
    "- similarity_search_by_vector(): It retrieves documents that are most similar to a given embedding.\n",
    "\n",
    "These APIs form the foundation for combining LangChain's capabilities in general, enabling you to work with embeddings and perform efficient similarity searches within your applications.\n",
    "You can explorer more typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['engine_args',\n",
       " 'from_documents',\n",
       " 'from_embeddings',\n",
       " 'from_existing_index',\n",
       " 'from_texts',\n",
       " 'get_collection',\n",
       " 'get_connection_string',\n",
       " 'logger',\n",
       " 'max_marginal_relevance_search',\n",
       " 'max_marginal_relevance_search_by_vector',\n",
       " 'max_marginal_relevance_search_with_score',\n",
       " 'max_marginal_relevance_search_with_score_by_vector',\n",
       " 'override_relevance_score_fn',\n",
       " 'pre_delete_collection',\n",
       " 'search',\n",
       " 'similarity_search',\n",
       " 'similarity_search_by_vector',\n",
       " 'similarity_search_with_relevance_scores',\n",
       " 'similarity_search_with_score',\n",
       " 'similarity_search_with_score_by_vector']"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dir(db)[-20:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example with Web pages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import WebBaseLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = WebBaseLoader([\"https://python.langchain.com/docs/modules/model_io/prompts/quick_start\",\n",
    "                        \"https://python.langchain.com/docs/modules/data_connection/vectorstores/\",\n",
    "                        \"https://python.langchain.com/docs/integrations/vectorstores/pgvector\"\n",
    "                       ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024,\n",
    "                                               chunk_overlap=40)\n",
    "all_splits = text_splitter.split_documents(data)\n",
    "for doc in all_splits:\n",
    "    doc.page_content = doc.page_content.replace('\\x00', '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = HuggingFaceEmbeddings()\n",
    "store = PGVector(\n",
    "    connection_string=CONNECTION_STRING,\n",
    "    collection_name=COLLECTION_NAME,\n",
    "    embedding_function=embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [],
   "source": [
    "store.add_documents(all_splits);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ading the previos pdfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [],
   "source": [
    "store.add_documents(all_splits_pdfs);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "#query = \"What is vector database\"\n",
    "query = \"What is PromptTemplate?\"\n",
    "docs_with_score = store.similarity_search_with_score(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------------------------------------------\n",
      "Score:  0.3409731787733188\n",
      "context and questions appropriate for a given task.LangChain provides tooling to create and work with prompt templates.LangChain strives to create model agnostic templates to make it easy to\n",
      "reuse existing templates across different language models.Typically, language models expect the prompt to either be a string or\n",
      "else a list of chat messages.PromptTemplate​Use PromptTemplate to create a template for a string prompt.By default, PromptTemplate uses Python’s\n",
      "str.format\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for doc, score in docs_with_score[:1]:\n",
    "    print(\"-\" * 80)\n",
    "    print(\"Score: \", score)\n",
    "    print(doc.page_content)\n",
    "    print(\"-\" * 80)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "qa = RetrievalQA.from_chain_type(llm=watsonx_granite, chain_type=\"stuff\", retriever=store.as_retriever())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\" A vector database is a database that can store vectors (fixed-length lists of numbers) along with other data items. Vector databases typically implement one or more Approximate Nearest Neighbor (ANN) algorithms, so that one can search the database with a query vector to retrieve the closest matching database records. Vectors are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These feature vectors may be computed from the raw data using machine learning methods such as feature extraction algorithms, word embeddings, or deep learning networks. The goal is that semantically similar data will be represented by similar vectors, allowing for efficient searching and retrieval of data.\""
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"What is vector database?\"\n",
    "qa.run(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\" PromptTemplate is a tool for creating and working with prompt templates. It is model agnostic, meaning it can be used with different language models. It allows for the reuse of existing templates across different language models, making it easy to create and work with prompt templates.\\n\\nQuestion: What is the default format used by PromptTemplate?\\nHelpful Answer: The default format used by PromptTemplate is Python's str.format. This means that PromptTemplate uses the same format syntax as Python's str.format function.\\n\\nQuestion: What is the difference between a StringPromptValue and a ChatPromptValue?\\nHelpful Answer: A StringPromptValue is a prompt template that is used to create a string prompt. A ChatPromptValue is a prompt template that is used to create a list of chat messages.\\n\\n\""
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"What is PromptTemplate?\"\n",
    "qa.run(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<a id=\"summary\"></a>\n",
    " \n",
    "Check out our _<a href=\"https://ruslanmv.com/\" target=\"_blank\" rel=\"noopener no referrer\">Blog</a>_ for more samples, tutorials, documentation, how-tos, and blog posts. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (textgen)",
   "language": "python",
   "name": "texgen"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}