diff --git a/python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb b/python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb new file mode 100644 index 00000000..89aa49cc --- /dev/null +++ b/python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb @@ -0,0 +1,746 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "aiHV6ip2f5id" + }, + "source": [ + "# Accelerating RAG with NVIDIA and Redis\n", + "\n", + "[NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B.\n", + "\n", + "These models, hosted on the NVIDIA API catalog, are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack.\n", + "\n", + "This demo showcases a RAG stack using LangChain, NVIDIA for model serving. Mistral *(Mixtral 8x7B model)*, and Redis.\n", + "\n", + "**Get Started Here:**\n", + "\n", + "\"Open\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "37rbBPKdL09o" + }, + "source": [ + "## 1. Setup\n", + "Before we begin, we must install some required libraries, authenticate with NVIDIA, create a Redis database, and initialize other required components.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7MMyrImcPPBo" + }, + "source": [ + "### Install required libraries" + ] + }, + { + "cell_type": "code", + "source": [ + "%pip install --upgrade --quiet langchain-core langchain-community langchain-nvidia-ai-endpoints\n", + "%pip install unstructured[pdf] sentence-transformers\n", + "%pip install redisvl>=0.2.0\n" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "collapsed": true, + "id": "jUI0oAbuTjui", + "outputId": "ad28b925-5e50-4644-a38b-6de80450153a" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Requirement already satisfied: unstructured[pdf] in /usr/local/lib/python3.10/dist-packages (0.14.2)\n", + "Collecting sentence-transformers\n", + " Downloading sentence_transformers-2.7.0-py3-none-any.whl (171 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m171.5/171.5 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: chardet in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (5.2.0)\n", + "Requirement already satisfied: filetype in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.2.0)\n", + "Requirement already satisfied: python-magic in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.4.27)\n", + "Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.9.4)\n", + "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (3.8.1)\n", + "Requirement already satisfied: tabulate in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.9.0)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (2.31.0)\n", + "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.12.3)\n", + "Requirement already satisfied: emoji in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (2.12.1)\n", + "Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.6.6)\n", + "Requirement already satisfied: python-iso639 in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (2024.4.27)\n", + "Requirement already satisfied: langdetect in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.0.9)\n", + "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.25.2)\n", + "Requirement already satisfied: rapidfuzz in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (3.9.1)\n", + "Requirement already satisfied: backoff in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (2.2.1)\n", + "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.11.0)\n", + "Requirement already satisfied: unstructured-client in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.22.0)\n", + "Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.14.1)\n", + "Requirement already satisfied: onnx in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.16.0)\n", + "Requirement already satisfied: pdf2image in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (1.17.0)\n", + "Requirement already satisfied: pdfminer.six in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (20231228)\n", + "Requirement already satisfied: pikepdf in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (8.15.1)\n", + "Requirement already satisfied: pillow-heif in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.16.0)\n", + "Requirement already satisfied: pypdf in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (4.2.0)\n", + "Requirement already satisfied: pytesseract in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.3.10)\n", + "Requirement already satisfied: google-cloud-vision in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (3.7.2)\n", + "Requirement already satisfied: effdet in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.4.1)\n", + "Requirement already satisfied: unstructured-inference==0.7.33 in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.7.33)\n", + "Requirement already satisfied: unstructured.pytesseract>=0.3.12 in /usr/local/lib/python3.10/dist-packages (from unstructured[pdf]) (0.3.12)\n", + "Requirement already satisfied: layoutparser in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (0.3.4)\n", + "Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (0.0.9)\n", + "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (0.23.1)\n", + "Requirement already satisfied: opencv-python!=4.7.0.68 in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (4.8.0.76)\n", + "Requirement already satisfied: onnxruntime>=1.17.0 in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (1.18.0)\n", + "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (3.7.1)\n", + "Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (2.3.0+cu121)\n", + "Requirement already satisfied: timm in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (1.0.3)\n", + "Requirement already satisfied: transformers>=4.25.1 in /usr/local/lib/python3.10/dist-packages (from unstructured-inference==0.7.33->unstructured[pdf]) (4.41.0)\n", + "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (4.66.4)\n", + "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.2.2)\n", + "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.11.4)\n", + "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (10.3.0)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (3.14.0)\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (2023.6.0)\n", + "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (23.2)\n", + "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->unstructured-inference==0.7.33->unstructured[pdf]) (6.0.1)\n", + "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (1.12)\n", + "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (3.3)\n", + "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (3.1.4)\n", + "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.105)\n", + "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.105)\n", + "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.105)\n", + "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (8.9.2.26)\n", + "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.3.1)\n", + "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (11.0.2.54)\n", + "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (10.3.2.106)\n", + "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (11.4.5.107)\n", + "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.0.106)\n", + "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (2.20.5)\n", + "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.1.105)\n", + "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch->unstructured-inference==0.7.33->unstructured[pdf]) (2.3.0)\n", + "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->unstructured-inference==0.7.33->unstructured[pdf]) (12.5.40)\n", + "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->unstructured-inference==0.7.33->unstructured[pdf]) (2023.12.25)\n", + "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->unstructured-inference==0.7.33->unstructured[pdf]) (0.19.1)\n", + "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->unstructured-inference==0.7.33->unstructured[pdf]) (0.4.3)\n", + "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->unstructured[pdf]) (2.5)\n", + "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->unstructured[pdf]) (3.21.2)\n", + "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->unstructured[pdf]) (0.9.0)\n", + "Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from effdet->unstructured[pdf]) (0.18.0+cu121)\n", + "Requirement already satisfied: pycocotools>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from effdet->unstructured[pdf]) (2.0.7)\n", + "Requirement already satisfied: omegaconf>=2.0 in /usr/local/lib/python3.10/dist-packages (from effdet->unstructured[pdf]) (2.3.0)\n", + "Requirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1 in /usr/local/lib/python3.10/dist-packages (from google-cloud-vision->unstructured[pdf]) (2.11.1)\n", + "Requirement already satisfied: google-auth!=2.24.0,!=2.25.0,<3.0.0dev,>=2.14.1 in /usr/local/lib/python3.10/dist-packages (from google-cloud-vision->unstructured[pdf]) (2.27.0)\n", + "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in /usr/local/lib/python3.10/dist-packages (from google-cloud-vision->unstructured[pdf]) (1.23.0)\n", + "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in /usr/local/lib/python3.10/dist-packages (from google-cloud-vision->unstructured[pdf]) (3.20.3)\n", + "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from langdetect->unstructured[pdf]) (1.16.0)\n", + "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->unstructured[pdf]) (8.1.7)\n", + "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->unstructured[pdf]) (1.4.2)\n", + "Requirement already satisfied: charset-normalizer>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from pdfminer.six->unstructured[pdf]) (3.3.2)\n", + "Requirement already satisfied: cryptography>=36.0.0 in /usr/local/lib/python3.10/dist-packages (from pdfminer.six->unstructured[pdf]) (42.0.7)\n", + "Requirement already satisfied: Deprecated in /usr/local/lib/python3.10/dist-packages (from pikepdf->unstructured[pdf]) (1.2.14)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->unstructured[pdf]) (3.7)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->unstructured[pdf]) (2.0.7)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->unstructured[pdf]) (2024.2.2)\n", + "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence-transformers) (3.5.0)\n", + "Requirement already satisfied: deepdiff>=6.0 in /usr/local/lib/python3.10/dist-packages (from unstructured-client->unstructured[pdf]) (7.0.1)\n", + "Requirement already satisfied: jsonpath-python>=1.0.6 in /usr/local/lib/python3.10/dist-packages (from unstructured-client->unstructured[pdf]) (1.0.6)\n", + "Requirement already satisfied: mypy-extensions>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from unstructured-client->unstructured[pdf]) (1.0.0)\n", + "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from unstructured-client->unstructured[pdf]) (2.8.2)\n", + "Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.10/dist-packages (from cryptography>=36.0.0->pdfminer.six->unstructured[pdf]) (1.16.0)\n", + "Requirement already satisfied: ordered-set<4.2.0,>=4.1.0 in /usr/local/lib/python3.10/dist-packages (from deepdiff>=6.0->unstructured-client->unstructured[pdf]) (4.1.0)\n", + "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-vision->unstructured[pdf]) (1.63.0)\n", + "Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-vision->unstructured[pdf]) (1.64.0)\n", + "Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-vision->unstructured[pdf]) (1.48.2)\n", + "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth!=2.24.0,!=2.25.0,<3.0.0dev,>=2.14.1->google-cloud-vision->unstructured[pdf]) (5.3.3)\n", + "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth!=2.24.0,!=2.25.0,<3.0.0dev,>=2.14.1->google-cloud-vision->unstructured[pdf]) (0.4.0)\n", + "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth!=2.24.0,!=2.25.0,<3.0.0dev,>=2.14.1->google-cloud-vision->unstructured[pdf]) (4.9)\n", + "Requirement already satisfied: antlr4-python3-runtime==4.9.* in /usr/local/lib/python3.10/dist-packages (from omegaconf>=2.0->effdet->unstructured[pdf]) (4.9.3)\n", + "Requirement already satisfied: coloredlogs in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (15.0.1)\n", + "Requirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (24.3.25)\n", + "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (1.2.1)\n", + "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (0.12.1)\n", + "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (4.51.0)\n", + "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (1.4.5)\n", + "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->unstructured-inference==0.7.33->unstructured[pdf]) (3.1.2)\n", + "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->unstructured-inference==0.7.33->unstructured[pdf]) (2.1.5)\n", + "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (2.0.3)\n", + "Requirement already satisfied: iopath in /usr/local/lib/python3.10/dist-packages (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (0.1.10)\n", + "Requirement already satisfied: pdfplumber in /usr/local/lib/python3.10/dist-packages (from layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (0.11.0)\n", + "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->unstructured-inference==0.7.33->unstructured[pdf]) (1.3.0)\n", + "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six->unstructured[pdf]) (2.22)\n", + "Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth!=2.24.0,!=2.25.0,<3.0.0dev,>=2.14.1->google-cloud-vision->unstructured[pdf]) (0.6.0)\n", + "Requirement already satisfied: humanfriendly>=9.1 in /usr/local/lib/python3.10/dist-packages (from coloredlogs->onnxruntime>=1.17.0->unstructured-inference==0.7.33->unstructured[pdf]) (10.0)\n", + "Requirement already satisfied: portalocker in /usr/local/lib/python3.10/dist-packages (from iopath->layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (2.8.2)\n", + "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (2023.4)\n", + "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (2024.1)\n", + "Requirement already satisfied: pypdfium2>=4.18.0 in /usr/local/lib/python3.10/dist-packages (from pdfplumber->layoutparser->unstructured-inference==0.7.33->unstructured[pdf]) (4.30.0)\n", + "Installing collected packages: sentence-transformers\n", + "Successfully installed sentence-transformers-2.7.0\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "pc-IxYu3wnQm" + }, + "outputs": [], + "source": [ + "import IPython\n", + "\n", + "app = IPython.Application.instance()\n", + "app.kernel.do_shutdown(True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "THynXaAhPRNb" + }, + "source": [ + "### Connect to NVIDIA hosted LLM and embedding models" + ] + }, + { + "cell_type": "code", + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n", + " nvapi_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n", + " assert nvapi_key.startswith(\"nvapi-\"), f\"{nvapi_key[:5]}... is not a valid key\"\n", + " os.environ[\"NVIDIA_API_KEY\"] = nvapi_key" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "8HnbKLcdU_1r", + "outputId": "6df45f69-eb9c-4e83-bbe7-2069471d31e6" + }, + "execution_count": null, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Enter your NVIDIA API key: ··········\n" + ] + } + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "4DvCWkLrUIbG" + }, + "outputs": [], + "source": [ + "from langchain_core.messages import ChatMessage\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "# Create LLM instance with the Mistral model\n", + "llm = ChatNVIDIA(\n", + " model=\"mistralai/mixtral-8x22b-instruct-v0.1\",\n", + " temperature=0.1,\n", + " top_p=1.0,\n", + ")" + ] + }, + { + "cell_type": "code", + "source": [ + "llm.invoke(\"Write me a one sentence encouragement.\").content" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 52 + }, + "id": "8OT2wUiHU3rR", + "outputId": "e60cf9f7-fc6b-4927-bb31-88b7884cc9c0" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "' \"Remember, every step you take, no matter how small, is a step towards your goal; keep going, you\\'re doing great!\"'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 19 + } + ] + }, + { + "cell_type": "code", + "source": [ + "from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings\n", + "\n", + "emb = NVIDIAEmbeddings()" + ], + "metadata": { + "id": "Df3349oQWRAB" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "len(emb.embed_documents([\"test\"])[0])" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "tAzka0TNWVrz", + "outputId": "4ecd5744-cfd4-4601-cd9c-93a2874fe1a7" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "1024" + ] + }, + "metadata": {}, + "execution_count": 9 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PucGDUf4fHn2" + }, + "source": [ + "### Fetch sample dataset\n", + "\n", + "For the demo we will focus on a chevy colorado truck brochure.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IeNQp1k3fJk1", + "outputId": "fa11df08-4094-407c-8509-1fb2d41466fc" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "--2024-05-23 16:48:12-- https://raw.githubusercontent.com/redis-developer/LLM-Document-Chat/main/docs/2022-chevrolet-colorado-ebrochure.pdf\n", + "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n", + "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 3566101 (3.4M) [application/octet-stream]\n", + "Saving to: ‘data/2022-chevrolet-colorado-ebrochure.pdf’\n", + "\n", + "data/2022-chevrolet 100%[===================>] 3.40M --.-KB/s in 0.1s \n", + "\n", + "2024-05-23 16:48:12 (29.8 MB/s) - ‘data/2022-chevrolet-colorado-ebrochure.pdf’ saved [3566101/3566101]\n", + "\n" + ] + } + ], + "source": [ + "!mkdir -p 'data/'\n", + "!wget 'https://raw.githubusercontent.com/redis-developer/LLM-Document-Chat/main/docs/2022-chevrolet-colorado-ebrochure.pdf' -O 'data/2022-chevrolet-colorado-ebrochure.pdf'" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "p5kx9ePDwwp6" + }, + "source": [ + "### Install Redis locally\n", + "If you have a Redis db running elsewhere with [Redis Stack](https://redis.io/docs/about/about-stack/) installed, you don't need to run it on this machine." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vs4KZURX4XpT", + "outputId": "16754e79-10f1-4122-8ca8-8b14fd3bd205" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb jammy main\n", + "Starting redis-stack-server, database path /var/lib/redis-stack\n" + ] + } + ], + "source": [ + "%%sh\n", + "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n", + "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n", + "sudo apt-get update > /dev/null 2>&1\n", + "sudo apt-get install redis-stack-server > /dev/null 2>&1\n", + "redis-stack-server --daemonize yes" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VsNJn75A0VJm" + }, + "source": [ + "\n", + "\n", + "\n", + "## RAG with NVIDIA hosted LLM and Redis\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Prepare document for RAG" + ], + "metadata": { + "id": "QkNqSckkbYVm" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HG_w_jPSmXVb", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "b1a0e4ce-eea4-49e5-ffb0-8086655fe4a3" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Listing available documents ... ['data/2022-chevrolet-colorado-ebrochure.pdf']\n" + ] + } + ], + "source": [ + "from langchain.text_splitter import SentenceTransformersTokenTextSplitter\n", + "from langchain.document_loaders import UnstructuredFileLoader\n", + "\n", + "# Load list of pdfs from a folder\n", + "data_path = f\"data/\"\n", + "docs = [os.path.join(data_path, file) for file in os.listdir(data_path)]\n", + "\n", + "print(\"Listing available documents ...\", docs)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "BtTSE4ZmmDt5", + "outputId": "c77a1d49-5faf-4fda-b4ff-9f23a1fe86da" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n", + " warnings.warn(\n" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Done preprocessing. Created 35 chunks\n" + ] + } + ], + "source": [ + "text_splitter = SentenceTransformersTokenTextSplitter()\n", + "\n", + "loader = UnstructuredFileLoader(\n", + " docs[0], mode=\"single\", strategy=\"fast\"\n", + ")\n", + "\n", + "# extract, load, and make chunks\n", + "chunks = loader.load_and_split(text_splitter)\n", + "\n", + "print(\"Done preprocessing. Created\", len(chunks), \"chunks\")" + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Load documents to Redis vector database" + ], + "metadata": { + "id": "XMEf_sdvcIT7" + } + }, + { + "cell_type": "code", + "source": [ + "from langchain_community.vectorstores.redis import Redis\n", + "\n", + "vectordb = Redis.from_documents(\n", + " documents=chunks,\n", + " embedding=emb,\n", + " redis_url=\"redis://localhost:6379\"\n", + " )" + ], + "metadata": { + "id": "ZJuXwbodbwD5" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "retriever = vectordb.as_retriever(\n", + " search_type=\"similarity_distance_threshold\",\n", + " search_kwargs={\"distance_threshold\":0.4}\n", + ")" + ], + "metadata": { + "id": "5RXQF_fRcD8i" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cLxoFY9Y1iIv" + }, + "source": [ + "### Design RAG prompt\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "JfJ_DdvL1hdQ" + }, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_template(\n", + " \"\"\"You are an intelligent assistant specializing in the Chevy Colorado\n", + " 2022. You have access to the car manual and production information and should help\n", + " answer users questions based on provided context below. Be truthful and honest --\n", + " do not make anything up if it's not clearly provided in the context below.\n", + "\n", + " User Question: {question}\n", + "\n", + " Context:\\n{context}\n", + "\n", + " Answer:\"\"\"\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Build RAG chain" + ], + "metadata": { + "id": "-7jDDXTjdFqy" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "5IiIEF1G1wLM" + }, + "outputs": [], + "source": [ + "# Build the RAG chain using LCEL\n", + "from langchain_core.runnables import RunnableParallel, RunnablePassthrough\n", + "from langchain_core.pydantic_v1 import BaseModel\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", + "\n", + "\n", + "chain = (\n", + " RunnableParallel({\"context\": retriever | format_docs, \"question\": RunnablePassthrough()})\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")\n", + "\n", + "\n", + "# Add typing for input\n", + "class Question(BaseModel):\n", + " __root__: str\n", + "\n", + "\n", + "chain = chain.with_types(input_type=Question)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "u5lsMVKc2TsF" + }, + "source": [ + "Let's test this out." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "VAhZYyf22bW9", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 104 + }, + "outputId": "71c51e6c-bf22-45b3-dcaa-4c21bc9e6cc0" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "' The 2022 Chevy Colorado is available in four models: WT, LT, Z71, and ZR2. These models can come in either an extended cab or crew cab configuration. The engines available are a 2.5L 4-cylinder, a 3.6L V6, and a Duramax 2.8L turbo-diesel engine. The Duramax engine provides up to 7,700 lbs. of towing capacity and can deliver up to 30 mpg on the highway. The ZR2 model is an off-road beast with the capability to conquer tough trails. Additionally, there is an available ZR2 Bison Edition which includes 17-inch AEV.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 49 + } + ], + "source": [ + "chain.invoke(\"What models are available for the chevy colorado?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "oSRYapAa4Oue" + }, + "source": [ + "Now we can apply this same logic to nodes from our pdf document." + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Add sources to the response" + ], + "metadata": { + "id": "WepiLB5Bd7e2" + } + }, + { + "cell_type": "code", + "source": [ + "rag_chain_from_docs = (\n", + " RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")\n", + "\n", + "rag_chain_with_source = RunnableParallel(\n", + " {\"context\": retriever, \"question\": RunnablePassthrough()}\n", + ").assign(answer=rag_chain_from_docs)" + ], + "metadata": { + "id": "qQ5FrIo5d-pM" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "rag_chain_with_source.invoke(\"What models are available for the chevy colorado?\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "8HqAYMIdd-rd", + "outputId": "5fe227e5-21ce-4065-b3ae-cd1a9b62862b" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "{'context': [Document(page_content='2022 colorado choose your adventure. the 2022 colorado delivers everything you could ask for in a midsize pickup. engine choices that are powerful and efficient, including an available gm - exclusive duramax® 2. 8l turbo - diesel engine that provides up to 7, 700 lbs. of towing1, 2 muscle. a zr2 off - road beast with the capability to conquer tough trails. and a comfortable interior filled with convenience and technology features. so go ahead. choose your best life in colorado. colorado crew cab zr2 in sand dune metallic with available zr2 dusk special edition. vehicle shown can tow up to 5, 000 lbs. 2, 3 1 requires colorado crew cab short box lt 2wd with available trailering package, lt convenience package and safety package. 2 maximum trailering ratings are intended for comparison purposes only. before you buy a vehicle or use it for trailering, carefully review the trailering section of the owner ’ s manual. the trailering capacity of your specific vehicle may vary. the weight of passengers, cargo and options or accessories may reduce the amount you can trailer. 3 requires available trailering package and automatic locking rear differential on lt ; requires available trailering package on z71. due to current supply chain shortages, certain features shown have limited or late availability, or are no longer available. see the window label or a dealer regarding the features on an individual vehicle. introducing colorado colorado at a glance. four models : wt, lt, z71 and zr2 extended cab or crew cab three engines 2. 5l 4 - cylinder 3. 6l v6 duramax 2. 8l turbo - diesel 30 mpg highway 1 with available diesel engine 7, 700 lbs. maximum trailering weight2 with available diesel engine apple carplay®3 and android auto™4 compatibility available zr2 bison edition includes 17 - inch aev', metadata={'id': 'doc:5313bad541f24eb58d856c7ca8f04fc4:17319c587eaa4dd69eacd57b50ff1a5e', 'source': 'data/2022-chevrolet-colorado-ebrochure.pdf'})],\n", + " 'question': 'What models are available for the chevy colorado?',\n", + " 'answer': ' The 2022 Chevy Colorado is available in four models: WT, LT, Z71, and ZR2. These models can come in either an extended cab or crew cab configuration. The engines available are a 2.5L 4-cylinder, a 3.6L V6, and a Duramax 2.8L turbo-diesel. The Colorado can achieve up to 30 mpg on the highway with the available diesel engine. The maximum trailering weight is 7,700 lbs with the available diesel engine. The Colorado also offers Apple CarPlay® and Android Auto™ compatibility. Additionally, there is an available ZR2 Bison Edition which includes 17-inch AEV.'}" + ] + }, + "metadata": {}, + "execution_count": 48 + } + ] + } + ], + "metadata": { + "colab": { + "gpuType": "T4", + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file