langchain_tutorial.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## This is a tutorial on how to perform QA and query on your own pdf file (it can be extended to a collection of pdfs).\n",
        "\n",
        "Acknowledgement: Sophia Yang's article on the functionality of LangChain => https://towardsdatascience.com/4-ways-of-question-answering-in-langchain-188c6707cc5a"
      ],
      "metadata": {
        "id": "2rWmds1KSVuJ"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "mrZ2fY3iBmA7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "34861f91-d500-4276-eb0f-fbba446fff3f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/gdrive\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "from google.colab import drive\n",
        "import colab_env"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install langchain openai chromadb tiktoken pypdf colab-env"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Iuzl5rFlQ33B",
        "outputId": "bec005ff-e60c-45ce-9c32-a65b38ac4746"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: langchain in /usr/local/lib/python3.10/dist-packages (0.0.209)\n",
            "Requirement already satisfied: openai in /usr/local/lib/python3.10/dist-packages (0.27.8)\n",
            "Requirement already satisfied: chromadb in /usr/local/lib/python3.10/dist-packages (0.3.26)\n",
            "Requirement already satisfied: tiktoken in /usr/local/lib/python3.10/dist-packages (0.4.0)\n",
            "Requirement already satisfied: pypdf in /usr/local/lib/python3.10/dist-packages (3.10.0)\n",
            "Requirement already satisfied: colab-env in /usr/local/lib/python3.10/dist-packages (0.2.0)\n",
            "Requirement already satisfied: PyYAML>=5.4.1 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.10)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.8.4)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.2)\n",
            "Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.5.8)\n",
            "Requirement already satisfied: langchainplus-sdk>=0.0.13 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.0.16)\n",
            "Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.8.4)\n",
            "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.22.4)\n",
            "Requirement already satisfied: openapi-schema-pydantic<2.0,>=1.2 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.2.4)\n",
            "Requirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.10.7)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.2)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai) (4.65.0)\n",
            "Requirement already satisfied: pandas>=1.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.5.3)\n",
            "Requirement already satisfied: hnswlib>=0.7 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.7.0)\n",
            "Requirement already satisfied: clickhouse-connect>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.6.4)\n",
            "Requirement already satisfied: duckdb>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.8.1)\n",
            "Requirement already satisfied: fastapi>=0.85.1 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.98.0)\n",
            "Requirement already satisfied: uvicorn[standard]>=0.18.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.22.0)\n",
            "Requirement already satisfied: posthog>=2.4.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (3.0.1)\n",
            "Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (4.5.0)\n",
            "Requirement already satisfied: pulsar-client>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (3.2.0)\n",
            "Requirement already satisfied: onnxruntime>=1.14.1 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.15.1)\n",
            "Requirement already satisfied: tokenizers>=0.13.2 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.13.3)\n",
            "Requirement already satisfied: overrides>=7.3.1 in /usr/local/lib/python3.10/dist-packages (from chromadb) (7.3.1)\n",
            "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken) (2022.10.31)\n",
            "Requirement already satisfied: python-dotenv<1.0,>=0.10.0 in /usr/local/lib/python3.10/dist-packages (from colab-env) (0.21.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (2.0.12)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (2022.12.7)\n",
            "Requirement already satisfied: urllib3>=1.26 in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (1.26.15)\n",
            "Requirement already satisfied: pytz in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (2022.7.1)\n",
            "Requirement already satisfied: zstandard in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (0.21.0)\n",
            "Requirement already satisfied: lz4 in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (4.3.2)\n",
            "Requirement already satisfied: marshmallow<4.0.0,>=3.3.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (3.19.0)\n",
            "Requirement already satisfied: marshmallow-enum<2.0.0,>=1.5.1 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (1.5.1)\n",
            "Requirement already satisfied: typing-inspect>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (0.9.0)\n",
            "Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /usr/local/lib/python3.10/dist-packages (from fastapi>=0.85.1->chromadb) (0.27.0)\n",
            "Requirement already satisfied: coloredlogs in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (15.0.1)\n",
            "Requirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (23.3.3)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (23.1)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (3.20.3)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (1.11.1)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.3->chromadb) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from posthog>=2.4.0->chromadb) (1.16.0)\n",
            "Requirement already satisfied: monotonic>=1.5 in /usr/local/lib/python3.10/dist-packages (from posthog>=2.4.0->chromadb) (1.6)\n",
            "Requirement already satisfied: backoff>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from posthog>=2.4.0->chromadb) (2.2.1)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.4)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (8.1.3)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.14.0)\n",
            "Requirement already satisfied: httptools>=0.5.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.5.0)\n",
            "Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.17.0)\n",
            "Requirement already satisfied: watchfiles>=0.13 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.19.0)\n",
            "Requirement already satisfied: websockets>=10.4 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (11.0.3)\n",
            "Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.28.0,>=0.27.0->fastapi>=0.85.1->chromadb) (3.6.2)\n",
            "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (1.0.0)\n",
            "Requirement already satisfied: humanfriendly>=9.1 in /usr/local/lib/python3.10/dist-packages (from coloredlogs->onnxruntime>=1.14.1->chromadb) (10.0)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime>=1.14.1->chromadb) (1.3.0)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.28.0,>=0.27.0->fastapi>=0.85.1->chromadb) (1.3.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Before starting, use colab-env to store openai_api_key in vars.env file\n",
        "see example: https://colab.research.google.com/github/apolitical/colab-env/blob/master/colab_env_testbed.ipynb#scrollTo=4LMqPJ9i5OZo"
      ],
      "metadata": {
        "id": "iXvrrMMpG34j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Short little langchain tutorial on QA task for PDF files"
      ],
      "metadata": {
        "id": "TAEXYYDjB1sU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### instantiate the LLM model\n",
        "\n",
        "\n",
        "*   OpenAI defaults to `text-davinci-003`\n",
        "*   ChatGPT defaults to `gpt-3.5-turbo`\n",
        "\n"
      ],
      "metadata": {
        "id": "XDzZrVAAXmTS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.chains import RetrievalQA\n",
        "from langchain.llms import OpenAI\n",
        "from langchain.document_loaders import TextLoader\n",
        "from langchain.document_loaders import PyPDFLoader\n",
        "from langchain.indexes import VectorstoreIndexCreator\n",
        "from langchain.text_splitter import CharacterTextSplitter\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.vectorstores import Chroma\n",
        "from langchain.chat_models import ChatOpenAI\n",
        "from langchain.chains.question_answering import load_qa_chain\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "\n",
        "from langchain.schema import (\n",
        "    AIMessage,\n",
        "    HumanMessage,\n",
        "    SystemMessage\n",
        ")"
      ],
      "metadata": {
        "id": "DBiW0uohBxHe"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CUuoiPZEN0GA",
        "outputId": "e531879f-b603-45b4-e243-82a698a6efe8"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "os.chdir('/content/drive/My Drive/langchain')"
      ],
      "metadata": {
        "id": "wJKGmne9NnSE"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "llm = OpenAI(openai_api_key=os.environ[\"OPENAI_API_KEY\"], temperature=1)\n"
      ],
      "metadata": {
        "id": "feWvY7hbTcB8"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "chatgpt = ChatOpenAI(model_name='gpt-3.5-turbo', openai_api_key=os.environ[\"OPENAI_API_KEY\"], temperature=1)\n"
      ],
      "metadata": {
        "id": "oJb_aRRmKVHM"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# try a simple prompt\n",
        "text = \"does pineapple belong on a thin crust pizza?\""
      ],
      "metadata": {
        "id": "ncZvVE-jV-GW"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "chatgpt([HumanMessage(content=text)])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cTBEDMekK_KD",
        "outputId": "4bb7e43e-04bf-461a-b360-3092b3fccb3c"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "AIMessage(content='As an AI language model, I do not have personal preferences. However, it is a matter of personal taste whether one prefers pineapple on a thin crust pizza. Some people enjoy the combination of sweet and savory flavors, while others do not like fruit on their pizza. Ultimately, it is up to individual taste buds to decide what toppings they prefer on their pizza.', additional_kwargs={}, example=False)"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "llm(text)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "zOkKW6R8VokM",
        "outputId": "e40c673a-8026-4a37-9b0d-989580eb89b0"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\n\\nNo, pineapple does not belong on a thin crust pizza. It is much more common to see pineapple on a thicker, deep dish crust pizza.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## load the pdf file\n",
        "### in loader.load(), it returns the pages in the pdf as list of document and feeds the whole document or set of documents to the LLM."
      ],
      "metadata": {
        "id": "qJFmqLDZPnns"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "loader = PyPDFLoader(\"text_files/automated speech-based screening of depression.pdf\")\n",
        "documents = loader.load()"
      ],
      "metadata": {
        "id": "dWWuYeJZPPxx"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "documents[8]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B--z32sEQ25C",
        "outputId": "7d22edfc-94dc-475f-9299-a3b618b92ffb"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Document(page_content=' Karol Chlasta et al. / Procedia Com puter Science  00 (2019) 000 –000  9 \\nTable 2. Summar y of other classification results for different CNN architectures . \\nModel  Type of Input  Hyperparameters  (LR; EP) Accuracy  F1 Score  Precision  Recall  \\nResNet 18  Image (224x224px)  0.01; 3 78% 0.0000  - 0.0000  \\nResNet 34 \\nResNet 50  \\nResNet 50  Image (224x224px)  \\nImage ( 224x224px)  \\nImage (512x512px)  0.001 ; 3 \\n0.01; 3 \\n0.01; 3  81% \\n67% \\n78% 0.6154  \\n0.3077  \\n0.5714  0.5714  \\n0.3333  \\n0.5714  0.6667  \\n0.2857  \\n0.5714  \\nResNet 101  \\nResNet 101  \\nResNet 152  \\nResNet 152  Image (512x512px)  \\nImage  (1024x1024 px) \\nImage (512x512px)  \\nImage (1024x1024px)  0.001 ; 4 \\n0.0044; 4  \\n0.0001 ; 3 \\n0.003 ; 3 81% \\n78% \\n63% \\n74% 0.2857  \\n0.4000  \\n0.3750  \\n0.5333  0.2500  \\n0.2500  \\n0.3750  \\n0.5000  0.3333  \\n1.0000  \\n0.3750  \\n0.5714  \\n \\nThe ResNet -34 system using a smaller Set A and TTA classified 21 voice sam ples correctly, with only six \\nsamp les classified incorrectly.  The ResNet -50 system using a larger Set B classified  443 voice samples correctly, \\nwith only 199 samples classified incorrectly . The confusion matrix es for both configurations are  presented in Fig. 8. \\n \\n \\n \\nFig. 8. Confusion Mat rix for our classi fication system  on (a) Set A  and (b) Set B  using  image s of 224x224px . \\n5. Conclusions and future work  \\nThis paper proposed a method that uses deep convolutional neural networks for depression detection in speech.  \\nWe tested five network  architectures and select ed ResNet -34 and ResNet -50 as presenting  the best classification \\nresults.  The results suggest a promising new direction in using audio spectrograms for preliminary screening of \\ndepressive subjects by using short  samples of their vo ice. The spectrograms proved to have a potential for \\ngenerating CNN learnable features . The algorithm attained accuracies of 70% and 77% using the TTA method.  This \\nwas despite  the challenging nature of voice as a predictor of depression.  The solution use d small sample sizes  (15 \\nseconds) , which diminish ed the effect of noise , in our view .  \\nOur system can be used independently, or as an element of  a more complex, hybrid , or multimodal solution . The \\nmain advantage s of this method are its relative simplicity, c oupled with its state -of-the art a ccuracy.  The solution \\nachieved  accuracy similar to Afshan  et al.  [12] on short speech utterances . The classification sensitivity ( recall ) of \\n41% is 3% higher than a context -free model  [14] and  achieves the same p recision  as a weighted (audio) model  [14]. ', metadata={'source': 'text_files/automated speech-based screening of depression.pdf', 'page': 8})"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "chain = load_qa_chain(llm=ChatOpenAI(model_name='gpt-3.5-turbo'), chain_type=\"map_reduce\")\n",
        "query = \"what is the finding of this experiment?\"\n",
        "chain.run(input_documents=documents, question=query)"
      ],
      "metadata": {
        "id": "oviHWeaNTcQT",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "b74c003c-9ed6-4de4-b311-d6edc96c02b7"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'The experiment proposed a method that uses deep convolutional neural networks for depression detection in speech. The proposed method produced a promising classification accuracy of around 70% for a ResNet-34 model, and 71% for a ResNet-50 model, both trained on spectrograms of 224x224 px. This result can be improved to 77% with Test Time Augmentation (TTA). The full summary of the results is presented in Table 2. Therefore, the finding of the experiment is that the proposed method achieved a promising accuracy of depression detection in speech using deep convolutional neural networks.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## RetrievalQA\n",
        "#### retrieve the most relevant chunck of text and feed those to the language model.\n",
        "\n",
        "#### Note that `chain_type=\"stuff\"` uses ALL of the text, and it can exceed the token limit and trigger API errors."
      ],
      "metadata": {
        "id": "lsC9DbCUQrY_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# split the documents into chunks\n",
        "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
        "texts = text_splitter.split_documents(documents)\n",
        "# select which embeddings we want to use\n",
        "embeddings = OpenAIEmbeddings()\n",
        "# create the vectorestore to use as the index\n",
        "db = Chroma.from_documents(texts, embeddings)\n",
        "# expose this index in a retriever interface\n",
        "retriever = db.as_retriever(search_type=\"similarity\", search_kwargs={\"k\":2})\n",
        "# create a chain to answer questions\n",
        "qa = RetrievalQA.from_chain_type(\n",
        "    llm=ChatOpenAI(model_name='gpt-3.5-turbo'), chain_type=\"stuff\", retriever=retriever, return_source_documents=True)\n",
        "query = \"what is the finding of this experiment?\"\n",
        "result = qa({\"query\": query})\n"
      ],
      "metadata": {
        "id": "KgG2NNs0TcVw"
      },
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "result"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wEzS0jB8TrfE",
        "outputId": "6815f15b-1f3d-4176-be40-4b2d46fd5b93"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'query': 'what is the finding of this experiment?',\n",
              " 'result': 'The experiment presented in this paper proposes a novel method for automated speech-based screening of depression using deep convolutional neural networks. The proposed method produced a promising classification accuracy of around 70% for a ResNet-34 model, and 71% for a ResNet-50 model, both trained on spectrograms of 224x224 px. This result can be improved to 77% with Test Time Augmentation (TTA). The overall finding suggests that deep convolutional neural networks have potential for automated speech-based screening of depression.',\n",
              " 'source_documents': [Document(page_content='Karol Chlasta et al. / Procedia Com puter Science  00 (2019) 000 –000  3 \\n3.980, RMSE 4.653). Multimodal approaches based on different neural network  architectures  appear to be more \\neffective than those based on a single modality, and they are an interesting a rea of furth er research.  \\nAnother set of interesting results was presented during 2018 Interspeech Conference**. Specifically , Afshan et al. \\n[12] focused on the effectiveness of voice quality features in detecting depression. They proposed the use of voice \\nquality features in combination with Mel-frequency cepstral coefficients (MFCCs)  [13], which showed improved \\nperformance in detecting depression. As a result, they were able to achieve an accuracy of 77% even when the test \\nutterances were as short as 10 s. The accuracy  was as high as 95% when the test utterances were 1.8 min utes long.  \\nHanai et al.  [14] presented a depression detection model based on sequences of audio and text transcriptions. \\nThey evaluated a regularized logistic regression model (with an d without conditi oning on the type of questions \\nasked), and a long short -term memory (LSTM)  model [28] (using the sequences of responses, and without \\nknowledge of the type of questions that prompted the responses). They also evaluated a multi -modal LSTM model \\nthat combine d the audio and text features. The interesting findings they announced were that context -free modelling \\nof the interviews based on text features performed better than audio features when classifying for a binary outcome \\n(depressed vs . non-depressed). On th e other hand, audio features were more accurate in determining the multi -class \\ndepression score (MAE 5.01 vs. 7.02). When weighting the model according to the questions asked, audio features \\nperformed better than text features (F1 of 0.67  vs. 0.44) with pe rfect rates of precision (1.00). The overall \\nperformance of audio improved when conditioning on the question being asked (F1 0.67 vs. 0.50). According to \\nthese researchers the multi -modal model yielded the best performance (F1 0.77 and re call 0.83). Sequen ce models \\nalso displayed the best multi -class classification performance.  \\nThis paper  proposes  a novel method for automated speech -based screening of depression using deep \\nconvolutional neural networks. We present comprehensive experiments  on distress analy sis interview corpus \\n(DAIC) [ 15] to show the potential of our classification method and evaluate  the results  obtained . \\n2. Data  \\n2.1. Data source and data description  \\nThe DAIC  database contains clinical interviews designed to support the diagnosis o f psychological di stress \\nconditions such as anxiety , depression, and post‐traumatic stress  [15]. These interviews were collected as part of a \\nlarger effort to evaluate  a computer agent that interviews people with mental illness [ 16]. The data were publishe d \\nfree of charge fo r scientific use by the University of Southern California††. The archive  contains 92 GB of data \\nstored as  a package of 189 folders  compressed in zip forma t. They contain 300 to 492 session recordings. Each file \\nin the archive represents a single session , and it contains a text transcript ion of the recording , participant audio files, \\nand facial features . The audio files were recorded with a head mounted microphone (Sennheiser HSP 4 -EW-3) at 16  \\nkHz. Interviews  were conducted by an animated  virtual interviewe r called Ellie, controlled by a human interviewer \\nin another room.  Figure 1 presents Ellie, the virtual interviewer.  \\n \\n \\nFig. 1. Ellie, the virtual interviewer  [11]. \\n \\n \\n** Website of Interspeech 2018 Conference https://interspeech2018.org/  \\n†† Website of DAIC -WOZ Database http://dcapswoz.ict.usc.edu/', metadata={'source': 'text_files/automated speech-based screening of depression.pdf', 'page': 2}),\n",
              "  Document(page_content='8 Karol Chlasta et al. / Procedia Computer Science  00 (2019) 000–000 \\nFig. 7. Cyclic stochastic gradient descent with restart (SGDR) : (a) cycle_len = 1 ; (b) cycle_len = 3 [ 26]. \\nThe thir d aspect is fine-tuning and le arning rate annealing , which is used in step s 7, 8, and 9. It is used when all \\nlayers of our neural network  are unlocked . Having a well -trained final layer  in step s 4 and 5 , we refine the remaining \\nlayers. The remaining layers  had already been trained to r ecognize  images, so the approach does not destroy the \\naccurately pre-adjusted scales  of our neural network . At that point we set a 3 –10x smaller learning step for the \\nprevious layer, than for the next layer. In this model , earlier layers have more general -purpo se func tions, and they \\nare fine -tuned to the new datasets. As a result , different learning rate levels are used  for different layers: the first few \\nlayers were  trained at level 1e -4 (0.0001) , the middle layers at 1e -3 (0.001) , and the final layers (see FC layers in \\nFig. 5) lowered to 1e -2 (0.01).  \\n4. Experiments  and results  \\nWe performed all our experiment s using  Google Collaboratory  platform , which  enable d us to use NVidia  K100 \\ngraphic cards for computations . It also uses Jupyter N otebook standard, facilitatin g the exchange of code and results \\nwith other researchers. Our Python code and results can be accessed easily by cloning  a public Jupyter notebook‡‡‡‡. \\nWe use d PyTorch  and fast.ai§§§§ for neural network training ; in our opinion , this  accelerated the development \\nprocess and facilitated application of best practice  (e.g. this approach allowed us to evaluate multiple pre -trained \\nResNet architectures by changing a  single  parameter value ). Our classification results were evaluated using four \\nclassification assessment  metrics  [24]: Accuracy , F1 Score  (F-measure ), Precision,  and Recall (Sensitivity) . \\nThe experiments included the data  and CNN preparation , as well as fine -tuning procedure required to obtain the \\nstate-of-the-art results for our automated speech -based depression scr eening system. Apart from evaluating five \\nCNN architectures (ResNet -18, 34, 50, 101, 152) , we also checked if generating higher resolution spectrogram s \\nwould impact  the classification results.  We confirmed that gene rating  a set of  larger  input spectrograms  (e.g. \\n1024x1024) would not significantly  improve the results.  We also perform ed Test Time Augmentation (TTA) [ 25]. \\nThe TTA created predictions based on the original spectrogram from our dataset and four augmentat ions of it. The \\nmean prognosis from all the images was equal to 77%.  \\nThe proposed method produced  a promising classification accuracy  of around 70% for a  ResNet -34 model , and \\n71% for a ResNet -50 model, both trained on spectrograms of 224x224  px. This result can be improved to 77% with \\nTTA.  A full s ummary of the results is presented in Table 2 . The variations in  the results for each system \\nconfiguration is attributable to the fact that both the training and test datasets were randomly selected at each system \\ninitialization  stage . \\n \\n \\n‡‡‡‡Link to Google Colab notebook: https://co lab.research.google.com/drive/1MHcM2uBgY_MgjwAJbz_TSjlLXBdaJ gpm \\n§§§§ Fast.ai uses PyTorch and provides a single consistent API to the most important deep le arning applications and data types. More \\ninformation: https://www.fast.ai/', metadata={'source': 'text_files/automated speech-based screening of depression.pdf', 'page': 7})]}"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# try a different doc"
      ],
      "metadata": {
        "id": "KHvGuoclJB_u"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "loader_HOA_doc = PyPDFLoader(\"text_files/HOA_rules_exterior.pdf\")\n",
        "hoa_documents = loader_HOA_doc.load()"
      ],
      "metadata": {
        "id": "bM-XSQxtR7QX"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# split the documents into chunks\n",
        "text_splitter = CharacterTextSplitter(chunk_size=200, chunk_overlap=0)\n",
        "texts = text_splitter.split_documents(hoa_documents)\n",
        "# select which embeddings we want to use\n",
        "embeddings = OpenAIEmbeddings()\n",
        "# create the vectorestore to use as the index\n",
        "db = Chroma.from_documents(texts, embeddings)\n",
        "# expose this index in a retriever interface\n",
        "retriever = db.as_retriever(search_type=\"similarity\", search_kwargs={\"k\":2})\n",
        "# create a chain to answer questions\n",
        "qa = RetrievalQA.from_chain_type(\n",
        "    llm=ChatOpenAI(model_name='gpt-3.5-turbo'), chain_type=\"stuff\", retriever=retriever, return_source_documents=True)\n",
        "query_1 = \"Can people have Wall Mounted Basketball Hoop on the porch or balcony?\"\n",
        "query_2 = \"Can people have satellite dishes on the front porch\"\n",
        "\n",
        "result_1 = qa({\"query\": query_1})\n",
        "result_2 = qa({\"query\": query_2})"
      ],
      "metadata": {
        "id": "P0UX0kppJVyS"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "result_1"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WWc4s5PhJkOF",
        "outputId": "855f0b03-e295-4e61-b806-89338d21008b"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'query': 'Can people have Wall Mounted Basketball Hoop on the porch or balcony?',\n",
              " 'result': \"The Village West at Centennial's Rules and Regulations do not specifically mention whether or not wall-mounted basketball hoops are allowed on porches or balconies. However, it does state that all improvements made to a lot, including outdoor structures, even temporary, require design review and approval from the Design Review Committee. Therefore, if a resident wishes to install a wall-mounted basketball hoop on their porch or balcony, they will need to submit complete plans and specifications to the DRC for approval.\",\n",
              " 'source_documents': [Document(page_content='VILLAGE WEST at CENTENNIAL\\nRULES and REGULATIONS\\n11/15/2013 Page 4 of 6In order to ensure continued property value and community appeal, please consider the\\nhigh standard of the Community when selecting décor for your Front Porches, and\\nAttached Decks. For example, upholstered furniture or camping equipment is not\\nappropriate Front Porch or Attached Deck décor in a community like Village West at\\nCentennial. Outdoor furniture in a state of disrepair (i.e. torn cushions, covers, rusting\\nframes) is not allowed. Barbeques and patio furniture are permitted on Front Porches\\nor Attached Decks so long as they are maintained in a well-kept manner. Baby gates, pet\\ngates, pet cages or pet carriers may not be left unattended on or attached to Front\\nPorches or Decks.\\nExercise equipment, bicycles and coolers (except for dairy delivery) may not be stored\\non Front Porches and Attached Decks or Common Elements at any time. Wheeled toys,\\nballs, or other toys should not be left in public view overnight. Garden hoses and\\ngardening equipment must be detached and stored out of view from October 1 to April\\n30. For snow removal purposes, decorations including all types of landscape lighting\\nshould not be kept on Common Elements, walkways or stairways from October 1 to April\\n30. No storage of any kind should be on the Common Elements.\\nIn every case, having given proper notice to an Owner and an opportunity to be heard,\\nthe Association reserves the right, at Owner’s expense, to remove any décor from any\\nCommon Element or Lot that interferes with the irrigation or drainage systems or\\nmaintenance of Common Elements, or creates any unsafe or unsightly condition.\\nEXTERIOR BUILDING SURFACES\\nAn Owner is liable for any amounts incurred by the Association for repair, maintenance,\\nreplacement, and/or reconstruction that arise due to holes of any size that were made\\nin any exterior surface of an Owner’s Home for any purpose by any Owner, or any\\nmember for an Owner’s family or a tenant, guest or invitee of any Owner. A\\ndetermination of the act of omission, which caused holes in the exterior surface, by any\\nOwner, or any member of family or a tenant, guest or invitee of the Owner’s liability\\ntherefore, shall be made by the Board of Directors at a hearing after notice to the\\nOwner. If it is determined that the holes in the exterior surface were caused by the\\nact or omission of any Owner, or any member of an Owner’s family or a tenant, guest or\\ninvitee of any Owner, the Association my undertake repair, maintenance, replacement,\\nand/or reconstruction of the exterior surface of the Home damaged by the hole(s) to\\nrestore the surface to its original design and levy an assessment against the Owner for\\nthe cost thereof.', metadata={'source': 'text_files/HOA_rules_exterior.pdf', 'page': 3}),\n",
              "  Document(page_content='VILLAGE WEST at CENTENNIAL\\nRULES and REGULATIONS\\n11/15/2013 Page 6 of 6pickup must be placed in suitable, tightly-covered containers to avoid being blown out of\\nthe containers.\\nGarbage containers may not be placed outside, before 5:00 a.m. on the day the garbage\\nis scheduled to be collected by the Association’s waste contractor. Garbage containers\\nmust be brought inside, no later than 10:00 p.m. on the day the garbage is scheduled to\\nbe collected by the Association’s waste contractor.\\nDESIGN REVIEW\\nBy way of reminder, Article 5 of the Association’s Declaration is very clear that no\\nimprovements shall be constructed, erected, placed, planted, applied or installed upon\\nany Lot unless complete plans and specifications therefor have been first submitted to\\nand approved by the DRC. See the Design Review Committee Guidelines information\\nsheet.For example, changes that require design review and approval include but are\\nnot limited to:\\n\\uf0a7\\uf020the building of a deck or patio, even in an enclosed yard\\n\\uf0a7\\uf020the planting of anything in the ground, except in an enclosed backyard\\n\\uf0a7\\uf020the installation of any outdoor structure, even temporary, even in an enclosed\\nyard\\n\\uf0a7\\uf020the installation of a storm door or screen door\\n\\uf0a7\\uf020the installation of extension pipes to sump pump drains\\n\\uf0a7\\uf020the attachment of anything to a roof, eave or gutter\\nDesign Review request forms are available on the VWC website or by calling the VWC\\nmanagement company.', metadata={'source': 'text_files/HOA_rules_exterior.pdf', 'page': 5})]}"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "result_2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9ad8JI-bJnGl",
        "outputId": "7c14ad00-313f-4cd8-91ec-f637fcce4660"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'query': 'Can people have satellite dishes on the front porch',\n",
              " 'result': \"The rules and regulations of Village West at Centennial state that satellite dishes must be installed in the preferred manner and a DRC review form showing the location must be on file for each satellite dish. While it does not specifically say whether or not satellite dishes are allowed on the front porch, it is recommended that owners consider the high standard of the community when selecting décor for their front porches and attached decks. Therefore, it is best to consult the guidelines for satellite installation and the DRC form available on the Association's website or from the property management company to find out more information about the preferred locations for satellite dishes.\",\n",
              " 'source_documents': [Document(page_content='VILLAGE WEST at CENTENNIAL\\nRULES and REGULATIONS\\n11/15/2013 Page 5 of 6SATELLITE DISHES\\n\\uf0b7\\uf020To ensure all satellite equipment is installed in the preferred manner, a DRC\\nReview form showing the location must be on file for each satellite dish.\\nGuidelines for satellite installation and the DRC form may be obtained from the\\nAssociation’s website or from the property management company.\\n\\uf0b7\\uf020This policy replaces any previous satellite dish policy for the Association, whether\\nstated verbally or in writing.\\n\\uf0b7\\uf020All satellite dish installations must comply with any applicable state or local\\nbuilding codes and/or ordinances.\\n\\uf0b7\\uf020Satellite dishes must be in working order or removed.\\n\\uf0b7\\uf020Any damage resulting from the installation, presence, and/or removal of a\\nsatellite dish on a Home is the responsibility of the Owner, pursuant to Section\\n9.5 of the Declaration.\\n\\uf0b7\\uf020Current owners are responsible for all previous installations.\\n\\uf0b7\\uf020A list of preferred and recommended vendors for repairs is available for owners\\nto use on the VWC website or by calling the VWC management company.\\nPEST CONTROL\\nThe control or removal of all pests within the boundaries of an Owner’s Lot is the sole\\nresponsibility of Owner unless it is a community wide problem where the origin cannot\\nbe determined.\\nEXTERIOR LIGHTING\\nOwners are responsible to keep both front and rear exterior lighting fixtures at their\\nHomes maintained and in working order. Owners must use DRC-recommended light\\nfixture replacements (available on VWC website or from the VWC management\\ncompany). While alley lights are set to come on automatically at dusk, for safety\\npurposes Owners are encouraged to turn front lights on at night and/or put such lights\\non timers.\\nA list of preferred and recommended vendors for repairs is available for owners to use\\non the VWC website or by calling the VWC management company.\\nGARBAGE CONTAINERS\\nOnly containers provided by the Association’s waste contractor may be used for garbage\\ncollection purposes. Additional waste and recycle containers may be purchased by\\nOwner’s from the waste contractor for the Association. All trash/recycling set out for', metadata={'source': 'text_files/HOA_rules_exterior.pdf', 'page': 4}),\n",
              "  Document(page_content='VILLAGE WEST at CENTENNIAL\\nRULES and REGULATIONS\\n11/15/2013 Page 4 of 6In order to ensure continued property value and community appeal, please consider the\\nhigh standard of the Community when selecting décor for your Front Porches, and\\nAttached Decks. For example, upholstered furniture or camping equipment is not\\nappropriate Front Porch or Attached Deck décor in a community like Village West at\\nCentennial. Outdoor furniture in a state of disrepair (i.e. torn cushions, covers, rusting\\nframes) is not allowed. Barbeques and patio furniture are permitted on Front Porches\\nor Attached Decks so long as they are maintained in a well-kept manner. Baby gates, pet\\ngates, pet cages or pet carriers may not be left unattended on or attached to Front\\nPorches or Decks.\\nExercise equipment, bicycles and coolers (except for dairy delivery) may not be stored\\non Front Porches and Attached Decks or Common Elements at any time. Wheeled toys,\\nballs, or other toys should not be left in public view overnight. Garden hoses and\\ngardening equipment must be detached and stored out of view from October 1 to April\\n30. For snow removal purposes, decorations including all types of landscape lighting\\nshould not be kept on Common Elements, walkways or stairways from October 1 to April\\n30. No storage of any kind should be on the Common Elements.\\nIn every case, having given proper notice to an Owner and an opportunity to be heard,\\nthe Association reserves the right, at Owner’s expense, to remove any décor from any\\nCommon Element or Lot that interferes with the irrigation or drainage systems or\\nmaintenance of Common Elements, or creates any unsafe or unsightly condition.\\nEXTERIOR BUILDING SURFACES\\nAn Owner is liable for any amounts incurred by the Association for repair, maintenance,\\nreplacement, and/or reconstruction that arise due to holes of any size that were made\\nin any exterior surface of an Owner’s Home for any purpose by any Owner, or any\\nmember for an Owner’s family or a tenant, guest or invitee of any Owner. A\\ndetermination of the act of omission, which caused holes in the exterior surface, by any\\nOwner, or any member of family or a tenant, guest or invitee of the Owner’s liability\\ntherefore, shall be made by the Board of Directors at a hearing after notice to the\\nOwner. If it is determined that the holes in the exterior surface were caused by the\\nact or omission of any Owner, or any member of an Owner’s family or a tenant, guest or\\ninvitee of any Owner, the Association my undertake repair, maintenance, replacement,\\nand/or reconstruction of the exterior surface of the Home damaged by the hole(s) to\\nrestore the surface to its original design and levy an assessment against the Owner for\\nthe cost thereof.', metadata={'source': 'text_files/HOA_rules_exterior.pdf', 'page': 3})]}"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "DsHmbBvPJ-fl"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}