개인_데이터셋을_통한_llama2_fine_tune.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OvPuN06Y6lcC"
      },
      "source": [
        "# 개인 데이터셋을 통한 llama2 fine-tune"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QnY-RWum6-TB"
      },
      "source": [
        "llama2를 사용한 fine-tune의 간편화에 목적을 두었습니다. \n",
        "2가지 방법 (gradientai, T4 GPU)으로 나눠서 설명했습니다. \n",
        "\n",
        "- Gradientai \n",
        "  - AI 클라우드를 제공하는 회사 \n",
        "  - 사용자만 접근할 수 있는 개인 모델 구축 \n",
        "  - 다양한 언어 지원 (python, java) \n",
        "  - 기본 모델 지원 \n",
        "    - Bloom-560, Llama-2 (7B,13B), Nous-Hermes-Llama-2 \n",
        "  - Fine-Tuning 기능 지원 \n",
        "\n",
        "\n",
        "- T4 GPU 방법 \n",
        "  - T4 GPU에서 llama2 기반 모델 load를 위한 parameter 조정 \n",
        "  - 4bit load 적용 \n",
        "  - SFTTrainer를 사용한 학습 \n",
        "    - SFT는 특정 task에 모델을 조정하는 방법 \n",
        "    - 이번에 SFT를 사용하는 이유는 개인 데이터셋으로 만든 데이터에 최적화하기 위함 (다른 trainer를 사용해도 문제없습니다.) \n",
        "    - 개인 데이터셋으로 만든 데이터로 SFT학습 후 주어진 task에 해당하는 질문에 유사한 답변을 하는 것이 목표여서 해당 Trainer를 사용했습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eEd-lvP57POT"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XKC3aUZBjXEK",
        "outputId": "00a12cd9-510d-46d1-b7c1-2569338f24ec"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting llama-index==0.9.27\n",
            "  Downloading llama_index-0.9.27-py3-none-any.whl (15.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.8/15.8 MB\u001b[0m \u001b[31m49.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (2.0.24)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (3.9.1)\n",
            "Collecting beautifulsoup4<5.0.0,>=4.12.2 (from llama-index==0.9.27)\n",
            "  Downloading beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.0/143.0 kB\u001b[0m \u001b[31m19.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting dataclasses-json (from llama-index==0.9.27)\n",
            "  Downloading dataclasses_json-0.6.3-py3-none-any.whl (28 kB)\n",
            "Collecting deprecated>=1.2.9.3 (from llama-index==0.9.27)\n",
            "  Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (2023.6.0)\n",
            "Collecting httpx (from llama-index==0.9.27)\n",
            "  Downloading httpx-0.26.0-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.9/75.9 kB\u001b[0m \u001b[31m8.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (1.5.8)\n",
            "Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (3.2.1)\n",
            "Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (3.8.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (1.23.5)\n",
            "Collecting openai>=1.1.0 (from llama-index==0.9.27)\n",
            "  Downloading openai-1.7.0-py3-none-any.whl (224 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m224.7/224.7 kB\u001b[0m \u001b[31m17.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (1.5.3)\n",
            "Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (8.2.3)\n",
            "Collecting tiktoken>=0.3.3 (from llama-index==0.9.27)\n",
            "  Downloading tiktoken-0.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m68.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.9.27) (4.5.0)\n",
            "Collecting typing-inspect>=0.8.0 (from llama-index==0.9.27)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index==0.9.27) (4.0.3)\n",
            "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4<5.0.0,>=4.12.2->llama-index==0.9.27) (2.5)\n",
            "Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.10/dist-packages (from deprecated>=1.2.9.3->llama-index==0.9.27) (1.14.1)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index==0.9.27) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index==0.9.27) (1.3.2)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index==0.9.27) (2023.6.3)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index==0.9.27) (4.66.1)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index==0.9.27) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index==0.9.27) (1.7.0)\n",
            "Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index==0.9.27) (1.10.13)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index==0.9.27) (1.3.0)\n",
            "Collecting typing-extensions>=4.5.0 (from llama-index==0.9.27)\n",
            "  Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index==0.9.27) (2023.11.17)\n",
            "Collecting httpcore==1.* (from httpx->llama-index==0.9.27)\n",
            "  Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index==0.9.27) (3.6)\n",
            "Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->llama-index==0.9.27)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index==0.9.27) (3.3.2)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index==0.9.27) (2.0.7)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index==0.9.27) (3.0.3)\n",
            "Collecting mypy-extensions>=0.3.0 (from typing-inspect>=0.8.0->llama-index==0.9.27)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json->llama-index==0.9.27)\n",
            "  Downloading marshmallow-3.20.2-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index==0.9.27) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index==0.9.27) (2023.3.post1)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai>=1.1.0->llama-index==0.9.27) (1.2.0)\n",
            "Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.10/dist-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->llama-index==0.9.27) (23.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->llama-index==0.9.27) (1.16.0)\n",
            "Installing collected packages: typing-extensions, mypy-extensions, marshmallow, h11, deprecated, beautifulsoup4, typing-inspect, tiktoken, httpcore, httpx, dataclasses-json, openai, llama-index\n",
            "  Attempting uninstall: typing-extensions\n",
            "    Found existing installation: typing_extensions 4.5.0\n",
            "    Uninstalling typing_extensions-4.5.0:\n",
            "      Successfully uninstalled typing_extensions-4.5.0\n",
            "  Attempting uninstall: beautifulsoup4\n",
            "    Found existing installation: beautifulsoup4 4.11.2\n",
            "    Uninstalling beautifulsoup4-4.11.2:\n",
            "      Successfully uninstalled beautifulsoup4-4.11.2\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.9.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed beautifulsoup4-4.12.2 dataclasses-json-0.6.3 deprecated-1.2.14 h11-0.14.0 httpcore-1.0.2 httpx-0.26.0 llama-index-0.9.27 marshmallow-3.20.2 mypy-extensions-1.0.0 openai-1.7.0 tiktoken-0.5.2 typing-extensions-4.9.0 typing-inspect-0.9.0\n",
            "Requirement already satisfied: gdown in /usr/local/lib/python3.10/dist-packages (4.6.6)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from gdown) (3.13.1)\n",
            "Requirement already satisfied: requests[socks] in /usr/local/lib/python3.10/dist-packages (from gdown) (2.31.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from gdown) (1.16.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from gdown) (4.66.1)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from gdown) (4.12.2)\n",
            "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->gdown) (2.5)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (2023.11.17)\n",
            "Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.10/dist-packages (from requests[socks]->gdown) (1.7.1)\n",
            "Collecting llama-hub\n",
            "  Downloading llama_hub-0.0.67-py3-none-any.whl (36.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m36.1/36.1 MB\u001b[0m \u001b[31m32.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting html2text (from llama-hub)\n",
            "  Downloading html2text-2020.1.16-py3-none-any.whl (32 kB)\n",
            "Requirement already satisfied: llama-index>=0.9.8 in /usr/local/lib/python3.10/dist-packages (from llama-hub) (0.9.27)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from llama-hub) (5.9.5)\n",
            "Collecting pyaml<24.0.0,>=23.9.7 (from llama-hub)\n",
            "  Downloading pyaml-23.12.0-py3-none-any.whl (23 kB)\n",
            "Collecting retrying (from llama-hub)\n",
            "  Downloading retrying-1.3.4-py3-none-any.whl (11 kB)\n",
            "Requirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (2.0.24)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (3.9.1)\n",
            "Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.2 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (4.12.2)\n",
            "Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (0.6.3)\n",
            "Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (1.2.14)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (2023.6.0)\n",
            "Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (0.26.0)\n",
            "Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (1.5.8)\n",
            "Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (3.2.1)\n",
            "Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (3.8.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (1.23.5)\n",
            "Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (1.7.0)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (1.5.3)\n",
            "Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (8.2.3)\n",
            "Requirement already satisfied: tiktoken>=0.3.3 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (0.5.2)\n",
            "Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (4.9.0)\n",
            "Requirement already satisfied: typing-inspect>=0.8.0 in /usr/local/lib/python3.10/dist-packages (from llama-index>=0.9.8->llama-hub) (0.9.0)\n",
            "Requirement already satisfied: PyYAML in /usr/local/lib/python3.10/dist-packages (from pyaml<24.0.0,>=23.9.7->llama-hub) (6.0.1)\n",
            "Requirement already satisfied: six>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from retrying->llama-hub) (1.16.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index>=0.9.8->llama-hub) (4.0.3)\n",
            "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4<5.0.0,>=4.12.2->llama-index>=0.9.8->llama-hub) (2.5)\n",
            "Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.10/dist-packages (from deprecated>=1.2.9.3->llama-index>=0.9.8->llama-hub) (1.14.1)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index>=0.9.8->llama-hub) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index>=0.9.8->llama-hub) (1.3.2)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index>=0.9.8->llama-hub) (2023.6.3)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index>=0.9.8->llama-hub) (4.66.1)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index>=0.9.8->llama-hub) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index>=0.9.8->llama-hub) (1.7.0)\n",
            "Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index>=0.9.8->llama-hub) (1.10.13)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai>=1.1.0->llama-index>=0.9.8->llama-hub) (1.3.0)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index>=0.9.8->llama-hub) (2023.11.17)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index>=0.9.8->llama-hub) (1.0.2)\n",
            "Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->llama-index>=0.9.8->llama-hub) (3.6)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx->llama-index>=0.9.8->llama-hub) (0.14.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index>=0.9.8->llama-hub) (3.3.2)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index>=0.9.8->llama-hub) (2.0.7)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index>=0.9.8->llama-hub) (3.0.3)\n",
            "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect>=0.8.0->llama-index>=0.9.8->llama-hub) (1.0.0)\n",
            "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->llama-index>=0.9.8->llama-hub) (3.20.2)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index>=0.9.8->llama-hub) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index>=0.9.8->llama-hub) (2023.3.post1)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai>=1.1.0->llama-index>=0.9.8->llama-hub) (1.2.0)\n",
            "Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.10/dist-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->llama-index>=0.9.8->llama-hub) (23.2)\n",
            "Installing collected packages: retrying, pyaml, html2text, llama-hub\n",
            "Successfully installed html2text-2020.1.16 llama-hub-0.0.67 pyaml-23.12.0 retrying-1.3.4\n",
            "Collecting PyMuPDF\n",
            "  Downloading PyMuPDF-1.23.8-cp310-none-manylinux2014_x86_64.whl (4.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.4/4.4 MB\u001b[0m \u001b[31m15.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting PyMuPDFb==1.23.7 (from PyMuPDF)\n",
            "  Downloading PyMuPDFb-1.23.7-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (30.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m30.6/30.6 MB\u001b[0m \u001b[31m12.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: PyMuPDFb, PyMuPDF\n",
            "Successfully installed PyMuPDF-1.23.8 PyMuPDFb-1.23.7\n",
            "Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.10/dist-packages (1.5.8)\n",
            "Collecting jsonlines\n",
            "  Downloading jsonlines-4.0.0-py3-none-any.whl (8.7 kB)\n",
            "Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonlines) (23.2.0)\n",
            "Installing collected packages: jsonlines\n",
            "Successfully installed jsonlines-4.0.0\n",
            "Collecting gradio==3.48.0\n",
            "  Downloading gradio-3.48.0-py3-none-any.whl (20.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m49.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting aiofiles<24.0,>=22.0 (from gradio==3.48.0)\n",
            "  Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (4.2.2)\n",
            "Collecting fastapi (from gradio==3.48.0)\n",
            "  Downloading fastapi-0.108.0-py3-none-any.whl (92 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.0/92.0 kB\u001b[0m \u001b[31m12.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting ffmpy (from gradio==3.48.0)\n",
            "  Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting gradio-client==0.6.1 (from gradio==3.48.0)\n",
            "  Downloading gradio_client-0.6.1-py3-none-any.whl (299 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m299.2/299.2 kB\u001b[0m \u001b[31m29.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (0.26.0)\n",
            "Requirement already satisfied: huggingface-hub>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (0.20.2)\n",
            "Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (6.1.1)\n",
            "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (3.1.2)\n",
            "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (2.1.3)\n",
            "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (3.7.1)\n",
            "Requirement already satisfied: numpy~=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (1.23.5)\n",
            "Collecting orjson~=3.0 (from gradio==3.48.0)\n",
            "  Downloading orjson-3.9.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m17.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (23.2)\n",
            "Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (1.5.3)\n",
            "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (9.4.0)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (1.10.13)\n",
            "Collecting pydub (from gradio==3.48.0)\n",
            "  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
            "Collecting python-multipart (from gradio==3.48.0)\n",
            "  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (6.0.1)\n",
            "Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (2.31.0)\n",
            "Collecting semantic-version~=2.0 (from gradio==3.48.0)\n",
            "  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0) (4.9.0)\n",
            "Collecting uvicorn>=0.14.0 (from gradio==3.48.0)\n",
            "  Downloading uvicorn-0.25.0-py3-none-any.whl (60 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.3/60.3 kB\u001b[0m \u001b[31m7.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting websockets<12.0,>=10.0 (from gradio==3.48.0)\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m18.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.6.1->gradio==3.48.0) (2023.6.0)\n",
            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0) (4.19.2)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0) (0.12.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio==3.48.0) (3.13.1)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio==3.48.0) (4.66.1)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (1.2.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (4.47.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (1.4.5)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (3.1.1)\n",
            "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio==3.48.0) (2023.3.post1)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0) (2023.11.17)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.48.0) (8.1.7)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.48.0) (0.14.0)\n",
            "Collecting starlette<0.33.0,>=0.29.0 (from fastapi->gradio==3.48.0)\n",
            "  Downloading starlette-0.32.0.post1-py3-none-any.whl (70 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.0/70.0 kB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0) (3.7.1)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0) (1.0.2)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0) (1.3.0)\n",
            "Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0) (23.2.0)\n",
            "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0) (2023.12.1)\n",
            "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0) (0.32.0)\n",
            "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0) (0.16.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio==3.48.0) (1.16.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio==3.48.0) (1.2.0)\n",
            "Building wheels for collected packages: ffmpy\n",
            "  Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=c51644c1193992c0ae0918b35e0d750883d7508682a6f0f3ec03f80e7f2f1141\n",
            "  Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n",
            "Successfully built ffmpy\n",
            "Installing collected packages: pydub, ffmpy, websockets, uvicorn, semantic-version, python-multipart, orjson, aiofiles, starlette, gradio-client, fastapi, gradio\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "lida 0.0.10 requires kaleido, which is not installed.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed aiofiles-23.2.1 fastapi-0.108.0 ffmpy-0.3.1 gradio-3.48.0 gradio-client-0.6.1 orjson-3.9.10 pydub-0.25.1 python-multipart-0.0.6 semantic-version-2.10.0 starlette-0.32.0.post1 uvicorn-0.25.0 websockets-11.0.3\n",
            "Collecting trl\n",
            "  Downloading trl-0.7.9-py3-none-any.whl (141 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m141.1/141.1 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: torch>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from trl) (2.1.0+cu121)\n",
            "Requirement already satisfied: transformers>=4.31.0 in /usr/local/lib/python3.10/dist-packages (from trl) (4.35.2)\n",
            "Requirement already satisfied: numpy>=1.18.2 in /usr/local/lib/python3.10/dist-packages (from trl) (1.23.5)\n",
            "Collecting accelerate (from trl)\n",
            "  Downloading accelerate-0.25.0-py3-none-any.whl (265 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m265.7/265.7 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting datasets (from trl)\n",
            "  Downloading datasets-2.16.1-py3-none-any.whl (507 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m507.1/507.1 kB\u001b[0m \u001b[31m12.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tyro>=0.5.11 (from trl)\n",
            "  Downloading tyro-0.6.3-py3-none-any.whl (78 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.9/78.9 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (3.13.1)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (4.9.0)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (1.12)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (3.2.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (3.1.2)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (2023.6.0)\n",
            "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->trl) (2.1.0)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (0.20.2)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (23.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (6.0.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (2023.6.3)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (2.31.0)\n",
            "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (0.15.0)\n",
            "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (0.4.1)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.31.0->trl) (4.66.1)\n",
            "Collecting docstring-parser>=0.14.1 (from tyro>=0.5.11->trl)\n",
            "  Downloading docstring_parser-0.15-py3-none-any.whl (36 kB)\n",
            "Requirement already satisfied: rich>=11.1.0 in /usr/local/lib/python3.10/dist-packages (from tyro>=0.5.11->trl) (13.7.0)\n",
            "Collecting shtab>=1.5.6 (from tyro>=0.5.11->trl)\n",
            "  Downloading shtab-1.6.5-py3-none-any.whl (13 kB)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate->trl) (5.9.5)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->trl) (10.0.1)\n",
            "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets->trl) (0.6)\n",
            "Collecting dill<0.3.8,>=0.3.0 (from datasets->trl)\n",
            "  Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m14.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets->trl) (1.5.3)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets->trl) (3.4.1)\n",
            "Collecting multiprocess (from datasets->trl)\n",
            "  Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m16.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets->trl) (3.9.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl) (4.0.3)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.31.0->trl) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.31.0->trl) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.31.0->trl) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers>=4.31.0->trl) (2023.11.17)\n",
            "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (3.0.0)\n",
            "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (2.16.1)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.4.0->trl) (2.1.3)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->trl) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->trl) (2023.3.post1)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.4.0->trl) (1.3.0)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro>=0.5.11->trl) (0.1.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets->trl) (1.16.0)\n",
            "Installing collected packages: shtab, docstring-parser, dill, multiprocess, tyro, accelerate, datasets, trl\n",
            "Successfully installed accelerate-0.25.0 datasets-2.16.1 dill-0.3.7 docstring-parser-0.15 multiprocess-0.70.15 shtab-1.6.5 trl-0.7.9 tyro-0.6.3\n",
            "Collecting pypdf\n",
            "  Downloading pypdf-3.17.4-py3-none-any.whl (278 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m278.2/278.2 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: pypdf\n",
            "Successfully installed pypdf-3.17.4\n",
            "Collecting langchain\n",
            "  Downloading langchain-0.1.0-py3-none-any.whl (797 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m798.0/798.0 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.24)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.1)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
            "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.6.3)\n",
            "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
            "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
            "Collecting langchain-community<0.1,>=0.0.9 (from langchain)\n",
            "  Downloading langchain_community-0.0.11-py3-none-any.whl (1.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m26.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain-core<0.2,>=0.1.7 (from langchain)\n",
            "  Downloading langchain_core-0.1.9-py3-none-any.whl (216 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m216.5/216.5 kB\u001b[0m \u001b[31m27.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langsmith<0.1.0,>=0.0.77 (from langchain)\n",
            "  Downloading langsmith-0.0.79-py3-none-any.whl (48 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.4/48.4 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.23.5)\n",
            "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.10.13)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
            "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (3.20.2)\n",
            "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (0.9.0)\n",
            "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
            "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
            "Requirement already satisfied: anyio<5,>=3 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (3.7.1)\n",
            "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (23.2)\n",
            "Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (4.9.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2023.11.17)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.7->langchain) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.7->langchain) (1.2.0)\n",
            "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain) (1.0.0)\n",
            "Installing collected packages: jsonpointer, langsmith, jsonpatch, langchain-core, langchain-community, langchain\n",
            "Successfully installed jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.0 langchain-community-0.0.11 langchain-core-0.1.9 langsmith-0.0.79\n",
            "Collecting chromadb\n",
            "  Downloading chromadb-0.4.22-py3-none-any.whl (509 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m509.0/509.0 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: build>=1.0.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.0.3)\n",
            "Requirement already satisfied: requests>=2.28 in /usr/local/lib/python3.10/dist-packages (from chromadb) (2.31.0)\n",
            "Requirement already satisfied: pydantic>=1.9 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.10.13)\n",
            "Collecting chroma-hnswlib==0.7.3 (from chromadb)\n",
            "  Downloading chroma_hnswlib-0.7.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m24.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: fastapi>=0.95.2 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.108.0)\n",
            "Requirement already satisfied: uvicorn[standard]>=0.18.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.25.0)\n",
            "Requirement already satisfied: numpy>=1.22.5 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.23.5)\n",
            "Collecting posthog>=2.4.0 (from chromadb)\n",
            "  Downloading posthog-3.3.0-py2.py3-none-any.whl (40 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40.6/40.6 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (4.9.0)\n",
            "Collecting pulsar-client>=3.1.0 (from chromadb)\n",
            "  Downloading pulsar_client-3.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m69.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting onnxruntime>=1.14.1 (from chromadb)\n",
            "  Downloading onnxruntime-1.16.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.4/6.4 MB\u001b[0m \u001b[31m60.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-api>=1.2.0 (from chromadb)\n",
            "  Downloading opentelemetry_api-1.22.0-py3-none-any.whl (57 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.9/57.9 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-exporter-otlp-proto-grpc>=1.2.0 (from chromadb)\n",
            "  Downloading opentelemetry_exporter_otlp_proto_grpc-1.22.0-py3-none-any.whl (18 kB)\n",
            "Collecting opentelemetry-instrumentation-fastapi>=0.41b0 (from chromadb)\n",
            "  Downloading opentelemetry_instrumentation_fastapi-0.43b0-py3-none-any.whl (11 kB)\n",
            "Collecting opentelemetry-sdk>=1.2.0 (from chromadb)\n",
            "  Downloading opentelemetry_sdk-1.22.0-py3-none-any.whl (105 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m105.6/105.6 kB\u001b[0m \u001b[31m14.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tokenizers>=0.13.2 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.15.0)\n",
            "Collecting pypika>=0.48.9 (from chromadb)\n",
            "  Downloading PyPika-0.48.9.tar.gz (67 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: tqdm>=4.65.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (4.66.1)\n",
            "Collecting overrides>=7.3.1 (from chromadb)\n",
            "  Downloading overrides-7.4.0-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.10/dist-packages (from chromadb) (6.1.1)\n",
            "Requirement already satisfied: grpcio>=1.58.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.60.0)\n",
            "Collecting bcrypt>=4.0.1 (from chromadb)\n",
            "  Downloading bcrypt-4.1.2-cp39-abi3-manylinux_2_28_x86_64.whl (698 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m698.9/698.9 kB\u001b[0m \u001b[31m61.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: typer>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.9.0)\n",
            "Collecting kubernetes>=28.1.0 (from chromadb)\n",
            "  Downloading kubernetes-29.0.0-py2.py3-none-any.whl (1.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m71.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tenacity>=8.2.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (8.2.3)\n",
            "Requirement already satisfied: PyYAML>=6.0.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (6.0.1)\n",
            "Collecting mmh3>=4.0.1 (from chromadb)\n",
            "  Downloading mmh3-4.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (67 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.6/67.6 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging>=19.0 in /usr/local/lib/python3.10/dist-packages (from build>=1.0.3->chromadb) (23.2)\n",
            "Requirement already satisfied: pyproject_hooks in /usr/local/lib/python3.10/dist-packages (from build>=1.0.3->chromadb) (1.0.0)\n",
            "Requirement already satisfied: tomli>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from build>=1.0.3->chromadb) (2.0.1)\n",
            "Requirement already satisfied: starlette<0.33.0,>=0.29.0 in /usr/local/lib/python3.10/dist-packages (from fastapi>=0.95.2->chromadb) (0.32.0.post1)\n",
            "Requirement already satisfied: certifi>=14.05.14 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (2023.11.17)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (1.16.0)\n",
            "Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (2.8.2)\n",
            "Requirement already satisfied: google-auth>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (2.17.3)\n",
            "Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (1.7.0)\n",
            "Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (1.3.1)\n",
            "Requirement already satisfied: oauthlib>=3.2.2 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (3.2.2)\n",
            "Requirement already satisfied: urllib3>=1.24.2 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb) (2.0.7)\n",
            "Collecting coloredlogs (from onnxruntime>=1.14.1->chromadb)\n",
            "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m7.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (23.5.26)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (3.20.3)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (1.12)\n",
            "Requirement already satisfied: deprecated>=1.2.6 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-api>=1.2.0->chromadb) (1.2.14)\n",
            "Collecting importlib-metadata<7.0,>=6.0 (from opentelemetry-api>=1.2.0->chromadb)\n",
            "  Downloading importlib_metadata-6.11.0-py3-none-any.whl (23 kB)\n",
            "Collecting backoff<3.0.0,>=1.10.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)\n",
            "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: googleapis-common-protos~=1.52 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb) (1.62.0)\n",
            "Collecting opentelemetry-exporter-otlp-proto-common==1.22.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)\n",
            "  Downloading opentelemetry_exporter_otlp_proto_common-1.22.0-py3-none-any.whl (17 kB)\n",
            "Collecting opentelemetry-proto==1.22.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)\n",
            "  Downloading opentelemetry_proto-1.22.0-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.8/50.8 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-instrumentation-asgi==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)\n",
            "  Downloading opentelemetry_instrumentation_asgi-0.43b0-py3-none-any.whl (14 kB)\n",
            "Collecting opentelemetry-instrumentation==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)\n",
            "  Downloading opentelemetry_instrumentation-0.43b0-py3-none-any.whl (28 kB)\n",
            "Collecting opentelemetry-semantic-conventions==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)\n",
            "  Downloading opentelemetry_semantic_conventions-0.43b0-py3-none-any.whl (36 kB)\n",
            "Collecting opentelemetry-util-http==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)\n",
            "  Downloading opentelemetry_util_http-0.43b0-py3-none-any.whl (6.9 kB)\n",
            "Requirement already satisfied: setuptools>=16.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-instrumentation==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (67.7.2)\n",
            "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-instrumentation==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (1.14.1)\n",
            "Collecting asgiref~=3.0 (from opentelemetry-instrumentation-asgi==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)\n",
            "  Downloading asgiref-3.7.2-py3-none-any.whl (24 kB)\n",
            "Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb)\n",
            "  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.28->chromadb) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.28->chromadb) (3.6)\n",
            "Requirement already satisfied: huggingface_hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers>=0.13.2->chromadb) (0.20.2)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.10/dist-packages (from typer>=0.9.0->chromadb) (8.1.7)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.14.0)\n",
            "Collecting httptools>=0.5.0 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.4/341.4 kB\u001b[0m \u001b[31m39.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-dotenv>=0.13 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)\n",
            "Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading uvloop-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m50.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading watchfiles-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m78.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: websockets>=10.4 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (11.0.3)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (5.3.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.3.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (4.9)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers>=0.13.2->chromadb) (3.13.1)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers>=0.13.2->chromadb) (2023.6.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api>=1.2.0->chromadb) (3.17.0)\n",
            "Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.33.0,>=0.29.0->fastapi>=0.95.2->chromadb) (3.7.1)\n",
            "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb)\n",
            "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m13.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime>=1.14.1->chromadb) (1.3.0)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.33.0,>=0.29.0->fastapi>=0.95.2->chromadb) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.33.0,>=0.29.0->fastapi>=0.95.2->chromadb) (1.2.0)\n",
            "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.5.1)\n",
            "Building wheels for collected packages: pypika\n",
            "  Building wheel for pypika (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53723 sha256=5c34a42da80a54478777744f9f5cbb28882bd132523435591fce7bbe9a61e967\n",
            "  Stored in directory: /root/.cache/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4\n",
            "Successfully built pypika\n",
            "Installing collected packages: pypika, monotonic, mmh3, uvloop, python-dotenv, pulsar-client, overrides, opentelemetry-util-http, opentelemetry-semantic-conventions, opentelemetry-proto, importlib-metadata, humanfriendly, httptools, chroma-hnswlib, bcrypt, backoff, asgiref, watchfiles, posthog, opentelemetry-exporter-otlp-proto-common, opentelemetry-api, coloredlogs, opentelemetry-sdk, opentelemetry-instrumentation, onnxruntime, kubernetes, opentelemetry-instrumentation-asgi, opentelemetry-exporter-otlp-proto-grpc, opentelemetry-instrumentation-fastapi, chromadb\n",
            "  Attempting uninstall: importlib-metadata\n",
            "    Found existing installation: importlib-metadata 7.0.1\n",
            "    Uninstalling importlib-metadata-7.0.1:\n",
            "      Successfully uninstalled importlib-metadata-7.0.1\n",
            "Successfully installed asgiref-3.7.2 backoff-2.2.1 bcrypt-4.1.2 chroma-hnswlib-0.7.3 chromadb-0.4.22 coloredlogs-15.0.1 httptools-0.6.1 humanfriendly-10.0 importlib-metadata-6.11.0 kubernetes-29.0.0 mmh3-4.1.0 monotonic-1.6 onnxruntime-1.16.3 opentelemetry-api-1.22.0 opentelemetry-exporter-otlp-proto-common-1.22.0 opentelemetry-exporter-otlp-proto-grpc-1.22.0 opentelemetry-instrumentation-0.43b0 opentelemetry-instrumentation-asgi-0.43b0 opentelemetry-instrumentation-fastapi-0.43b0 opentelemetry-proto-1.22.0 opentelemetry-sdk-1.22.0 opentelemetry-semantic-conventions-0.43b0 opentelemetry-util-http-0.43b0 overrides-7.4.0 posthog-3.3.0 pulsar-client-3.4.0 pypika-0.48.9 python-dotenv-1.0.0 uvloop-0.19.0 watchfiles-0.21.0\n",
            "Requirement already satisfied: pydantic==1.10.13 in /usr/local/lib/python3.10/dist-packages (1.10.13)\n",
            "Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic==1.10.13) (4.9.0)\n",
            "Collecting gradientai\n",
            "  Downloading gradientai-1.4.0-py3-none-any.whl (171 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m171.1/171.1 kB\u001b[0m \u001b[31m1.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting aenum>=3.1.11 (from gradientai)\n",
            "  Downloading aenum-3.1.15-py3-none-any.whl (137 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.6/137.6 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pydantic<2.0.0,>=1.10.5 in /usr/local/lib/python3.10/dist-packages (from gradientai) (1.10.13)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from gradientai) (2.8.2)\n",
            "Requirement already satisfied: urllib3>=1.25.3 in /usr/local/lib/python3.10/dist-packages (from gradientai) (2.0.7)\n",
            "Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<2.0.0,>=1.10.5->gradientai) (4.9.0)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->gradientai) (1.16.0)\n",
            "Installing collected packages: aenum, gradientai\n",
            "Successfully installed aenum-3.1.15 gradientai-1.4.0\n",
            "Collecting sentence-transformers\n",
            "  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.0/86.0 kB\u001b[0m \u001b[31m1.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (4.35.2)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (4.66.1)\n",
            "Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (2.1.0+cu121)\n",
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (0.16.0+cu121)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.23.5)\n",
            "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.2.2)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.11.4)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (3.8.1)\n",
            "Collecting sentencepiece (from sentence-transformers)\n",
            "  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: huggingface-hub>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (0.20.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (3.13.1)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2023.6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2.31.0)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (6.0.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (4.9.0)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (23.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers) (1.12)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers) (3.2.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers) (3.1.2)\n",
            "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers) (2.1.0)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (2023.6.3)\n",
            "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.15.0)\n",
            "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.4.1)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers) (1.3.2)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence-transformers) (3.2.0)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision->sentence-transformers) (9.4.0)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.6.0->sentence-transformers) (2.1.3)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2023.11.17)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.6.0->sentence-transformers) (1.3.0)\n",
            "Building wheels for collected packages: sentence-transformers\n",
            "  Building wheel for sentence-transformers (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125923 sha256=9e85a3a1a1a4c5ef24a0cbc5694ee9c541d4976de72f6d1829d2ad808aefbce1\n",
            "  Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f\n",
            "Successfully built sentence-transformers\n",
            "Installing collected packages: sentencepiece, sentence-transformers\n",
            "Successfully installed sentence-transformers-2.2.2 sentencepiece-0.1.99\n",
            "Cloning into 'easy_finetuner'...\n",
            "remote: Enumerating objects: 53, done.\u001b[K\n",
            "remote: Counting objects: 100% (53/53), done.\u001b[K\n",
            "remote: Compressing objects: 100% (45/45), done.\u001b[K\n",
            "remote: Total 53 (delta 18), reused 5 (delta 0), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (53/53), 24.46 KiB | 3.49 MiB/s, done.\n",
            "Resolving deltas: 100% (18/18), done.\n",
            "/content/easy_finetuner\n",
            "Collecting git+https://github.com/huggingface/peft.git (from -r requirements.txt (line 3))\n",
            "  Cloning https://github.com/huggingface/peft.git to /tmp/pip-req-build-u_xjo6_g\n",
            "  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft.git /tmp/pip-req-build-u_xjo6_g\n",
            "  Resolved https://github.com/huggingface/peft.git to commit c6b28a22b86b2547cbf4443a9348a9e232f85c9e\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting loralib (from -r requirements.txt (line 1))\n",
            "  Downloading loralib-0.1.2-py3-none-any.whl (10 kB)\n",
            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (0.1.99)\n",
            "Requirement already satisfied: gradio==3.48.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (3.48.0)\n",
            "Collecting accelerate==0.21.0 (from -r requirements.txt (line 5))\n",
            "  Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting bitsandbytes==0.40.2 (from -r requirements.txt (line 6))\n",
            "  Downloading bitsandbytes-0.40.2-py3-none-any.whl (92.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.5/92.5 MB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting transformers==4.31.0 (from -r requirements.txt (line 7))\n",
            "  Downloading transformers-4.31.0-py3-none-any.whl (7.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m59.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting trl==0.4.7 (from -r requirements.txt (line 8))\n",
            "  Downloading trl-0.4.7-py3-none-any.whl (77 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.4/77.4 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: jsonlines in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (4.0.0)\n",
            "Requirement already satisfied: aiofiles<24.0,>=22.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (23.2.1)\n",
            "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (4.2.2)\n",
            "Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.108.0)\n",
            "Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.3.1)\n",
            "Requirement already satisfied: gradio-client==0.6.1 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.6.1)\n",
            "Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.26.0)\n",
            "Requirement already satisfied: huggingface-hub>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.20.2)\n",
            "Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (6.1.1)\n",
            "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (3.1.2)\n",
            "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (2.1.3)\n",
            "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (3.7.1)\n",
            "Requirement already satisfied: numpy~=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (1.23.5)\n",
            "Requirement already satisfied: orjson~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (3.9.10)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (23.2)\n",
            "Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (1.5.3)\n",
            "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (9.4.0)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (1.10.13)\n",
            "Requirement already satisfied: pydub in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.25.1)\n",
            "Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.0.6)\n",
            "Requirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (6.0.1)\n",
            "Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (2.31.0)\n",
            "Requirement already satisfied: semantic-version~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (2.10.0)\n",
            "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (4.9.0)\n",
            "Requirement already satisfied: uvicorn>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (0.25.0)\n",
            "Requirement already satisfied: websockets<12.0,>=10.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.48.0->-r requirements.txt (line 4)) (11.0.3)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 5)) (5.9.5)\n",
            "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 5)) (2.1.0+cu121)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r requirements.txt (line 7)) (3.13.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r requirements.txt (line 7)) (2023.6.3)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.31.0->-r requirements.txt (line 7))\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m36.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r requirements.txt (line 7)) (0.4.1)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r requirements.txt (line 7)) (4.66.1)\n",
            "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (from trl==0.4.7->-r requirements.txt (line 8)) (2.16.1)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.6.1->gradio==3.48.0->-r requirements.txt (line 4)) (2023.6.0)\n",
            "Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonlines->-r requirements.txt (line 9)) (23.2.0)\n",
            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (4.19.2)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.12.0)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (1.2.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (4.47.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (1.4.5)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (3.1.1)\n",
            "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio==3.48.0->-r requirements.txt (line 4)) (2023.3.post1)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0->-r requirements.txt (line 4)) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0->-r requirements.txt (line 4)) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0->-r requirements.txt (line 4)) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio==3.48.0->-r requirements.txt (line 4)) (2023.11.17)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 5)) (1.12)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 5)) (3.2.1)\n",
            "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 5)) (2.1.0)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.48.0->-r requirements.txt (line 4)) (8.1.7)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.14.0)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (10.0.1)\n",
            "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (0.6)\n",
            "Requirement already satisfied: dill<0.3.8,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (0.3.7)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (3.4.1)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (0.70.15)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets->trl==0.4.7->-r requirements.txt (line 8)) (3.9.1)\n",
            "Requirement already satisfied: starlette<0.33.0,>=0.29.0 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio==3.48.0->-r requirements.txt (line 4)) (0.32.0.post1)\n",
            "Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0->-r requirements.txt (line 4)) (3.7.1)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0->-r requirements.txt (line 4)) (1.0.2)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.48.0->-r requirements.txt (line 4)) (1.3.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl==0.4.7->-r requirements.txt (line 8)) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl==0.4.7->-r requirements.txt (line 8)) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl==0.4.7->-r requirements.txt (line 8)) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl==0.4.7->-r requirements.txt (line 8)) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->trl==0.4.7->-r requirements.txt (line 8)) (4.0.3)\n",
            "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (2023.12.1)\n",
            "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.32.0)\n",
            "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio==3.48.0->-r requirements.txt (line 4)) (0.16.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio==3.48.0->-r requirements.txt (line 4)) (1.16.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio==3.48.0->-r requirements.txt (line 4)) (1.2.0)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 5)) (1.3.0)\n",
            "Building wheels for collected packages: peft\n",
            "  Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for peft: filename=peft-0.7.2.dev0-py3-none-any.whl size=172246 sha256=c73df251bfb4fd08a096c631d213680fe5630ba163b0b1df786238583259d9a1\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-1dndeig8/wheels/d7/c7/de/1368fac8590e1b103ddc2ec2a28ad51d83aded1a3830e8a087\n",
            "Successfully built peft\n",
            "Installing collected packages: tokenizers, bitsandbytes, loralib, transformers, accelerate, peft, trl\n",
            "  Attempting uninstall: tokenizers\n",
            "    Found existing installation: tokenizers 0.15.0\n",
            "    Uninstalling tokenizers-0.15.0:\n",
            "      Successfully uninstalled tokenizers-0.15.0\n",
            "  Attempting uninstall: transformers\n",
            "    Found existing installation: transformers 4.35.2\n",
            "    Uninstalling transformers-4.35.2:\n",
            "      Successfully uninstalled transformers-4.35.2\n",
            "  Attempting uninstall: accelerate\n",
            "    Found existing installation: accelerate 0.25.0\n",
            "    Uninstalling accelerate-0.25.0:\n",
            "      Successfully uninstalled accelerate-0.25.0\n",
            "  Attempting uninstall: trl\n",
            "    Found existing installation: trl 0.7.9\n",
            "    Uninstalling trl-0.7.9:\n",
            "      Successfully uninstalled trl-0.7.9\n",
            "Successfully installed accelerate-0.21.0 bitsandbytes-0.40.2 loralib-0.1.2 peft-0.7.2.dev0 tokenizers-0.13.3 transformers-4.31.0 trl-0.4.7\n",
            "/content\n"
          ]
        }
      ],
      "source": [
        "!pip install llama-index==0.9.27\n",
        "!pip install gdown\n",
        "!pip install llama-hub\n",
        "!pip install PyMuPDF\n",
        "!pip install nest-asyncio\n",
        "!pip install jsonlines\n",
        "!pip install gradio==3.48.0\n",
        "!pip install trl\n",
        "!pip install pypdf\n",
        "!pip install langchain\n",
        "!pip install chromadb\n",
        "!pip install pydantic==1.10.13\n",
        "!pip install gradientai\n",
        "!pip install sentence-transformers\n",
        "\n",
        "!git clone https://github.com/choijhyeok/easy_finetuner.git\n",
        "%cd easy_finetuner\n",
        "!pip install -r requirements.txt\n",
        "%cd .."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uKbsVu0MXjfh"
      },
      "source": [
        "# GPU 없이 llama2 fine-tune"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8JUA9uGiP0U"
      },
      "source": [
        "## llama2 파인튜닝"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nPkEF2KTVCbk"
      },
      "source": [
        "### 필요한 모듈 불러오기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_5yf0d5AVEZ6"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms import GradientModelAdapterLLM\n",
        "import os\n",
        "import gdown\n",
        "from langchain.prompts.chat import (\n",
        "    ChatPromptTemplate,\n",
        "    SystemMessagePromptTemplate,\n",
        "    HumanMessagePromptTemplate,\n",
        ")\n",
        "from langchain.chains import RetrievalQAWithSourcesChain\n",
        "from llama_hub.file.pymu_pdf.base import PyMuPDFReader\n",
        "from langchain.document_loaders import PyPDFLoader\n",
        "from langchain.schema import Document\n",
        "from pathlib import Path\n",
        "from llama_index.llms import GradientBaseModelLLM\n",
        "from llama_index.finetuning.gradient.base import GradientFinetuneEngine\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain.vectorstores import Chroma\n",
        "import re\n",
        "from langchain.llms import GradientLLM\n",
        "import warnings\n",
        "from langchain.embeddings import HuggingFaceEmbeddings\n",
        "import jsonlines\n",
        "from datasets import Dataset\n",
        "warnings.filterwarnings('ignore')\n",
        "\n",
        "os.environ[\"GRADIENT_ACCESS_TOKEN\"] = ''\n",
        "os.environ[\"GRADIENT_WORKSPACE_ID\"] = \"\"\n",
        "os.environ[\"model_adapter_id\"] = \"\"\n",
        "os.environ[\"huggingface_token\"] = \"\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zcoVaPqdX7qY"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T9EUoAO9Uzp8"
      },
      "source": [
        "### 사용할 데이터 준비"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 322
        },
        "id": "ghPYBPk9U26M",
        "outputId": "bf561e40-03b1-44aa-f80d-82af0e265351"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=16hHL4hLer3nWhX18STvr061LcHzfFgN2\n",
            "To: /content/qa_버거킹_train.jsonl\n",
            "100%|██████████| 13.6k/13.6k [00:00<00:00, 15.9MB/s]\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1nB6ERfII2ODEDS_1xY3C5TBZHeOMMcPI\n",
            "To: /content/qa_버거킹_train_ko.jsonl\n",
            "100%|██████████| 16.8k/16.8k [00:00<00:00, 27.7MB/s]\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=11U7let6PY_YCJpgRT0Dpr5DXSO3Ceqep\n",
            "To: /content/버거킹.pdf\n",
            "100%|██████████| 346k/346k [00:00<00:00, 78.1MB/s]\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1BifMUDNX2v_4B7hb4YHv6eDinQqAjTZK\n",
            "To: /content/Burger-King.pdf\n",
            "100%|██████████| 316k/316k [00:00<00:00, 67.0MB/s]\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Burger-King.pdf'"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "gdown.download(url=\"https://drive.google.com/file/d/16hHL4hLer3nWhX18STvr061LcHzfFgN2/view?usp=sharing\", output=\"qa_버거킹_train.jsonl\", quiet=False)\n",
        "gdown.download(url=\"https://drive.google.com/file/d/1nB6ERfII2ODEDS_1xY3C5TBZHeOMMcPI/view?usp=sharing\", output=\"qa_버거킹_train_ko.jsonl\", quiet=False)\n",
        "gdown.download(url=\"https://drive.google.com/file/d/11U7let6PY_YCJpgRT0Dpr5DXSO3Ceqep/view?usp=sharing\", output=\"버거킹.pdf\", quiet=False)\n",
        "gdown.download(url=\"https://drive.google.com/file/d/1BifMUDNX2v_4B7hb4YHv6eDinQqAjTZK/view?usp=sharing\", output=\"Burger-King.pdf\", quiet=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "blAv2rLaX-T3"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A-5akKQ8U32j"
      },
      "source": [
        "### 파인튜닝할 모델 및 데이터 선택"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qrwemTfLVAj9"
      },
      "outputs": [],
      "source": [
        "## base_model 선택\n",
        "base_model_slug = \"llama2-7b-chat\"\n",
        "base_llm = GradientBaseModelLLM(\n",
        "    base_model_slug=base_model_slug, max_tokens=500, is_chat_model=True\n",
        ")\n",
        "\n",
        "finetune_engine = GradientFinetuneEngine(\n",
        "    base_model_slug=base_model_slug,\n",
        "    name=\"bugurking\",\n",
        "    data_path=\"qa_버거킹_train.jsonl\",\n",
        "    verbose=True,\n",
        "    max_steps=200,\n",
        "    batch_size=1,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yunVBzyXYByP"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gd1KBIxeU9xd"
      },
      "source": [
        "### 파인튜닝 finetune"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D_uZFtgmEdAk"
      },
      "outputs": [],
      "source": [
        "# 파인튜닝 2 epoch만\n",
        "for i in range(2):\n",
        "    print(f\"** EPOCH {i} **\")\n",
        "    finetune_engine.finetune()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PncJDw5JAHGU"
      },
      "outputs": [],
      "source": [
        "# 파인튜닝한 모델 사용\n",
        "llm = finetune_engine.get_finetuned_model(\n",
        "  max_tokens=500, is_chat_model=False\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m5RwdCsBYGl2"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "daboP4YJVQYs"
      },
      "source": [
        "### 기존에 이미 파인튜닝한 모델이 있다면 해당 모델 사용"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 161
        },
        "id": "efuMYNn2O9yw",
        "outputId": "aaf877bf-88ce-463f-eda9-2f43b7c98f5c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'\\nBurger King’s menu is quite extensive, with many options to choose from. Here are three items from their menu that I would recommend:\\n1. Whopper: Burger King’s signature burger, the Whopper, is a must-try. It features a juicy beef patty topped with lettuce, tomato, onion, pickles, and a slice of American cheese. The Whopper is a classic burger that is sure to satisfy your cravings.\\n2. Chicken King: Burger King’s Chicken King is a crispy and flavorful chicken sandwich that is sure to please. It features a breaded chicken patty topped with lettuce, tomato, onion, and a slice of American cheese. The Chicken King is a great option for those who want a chicken sandwich that is a little different from the usual.\\n3. Mozza Ball: Burger King’s Mozza Ball is a unique and tasty burger that features a beef patty topped with mozzarella cheese, lettuce, tomato, and a special sauce. The Mozza Ball is a great option for those who want a burger that is a little different from the usual.\\n\\nThese are just a few of the many items on Burger King’s menu, but they are definitely worth trying. Burger King is known for its high-quality ingredients and generous portion sizes, so you can be sure that you’ll get a great meal at a reasonable price.'"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "llm = GradientModelAdapterLLM(\n",
        "    model_adapter_id=os.environ[\"model_adapter_id\"],\n",
        "    max_tokens=500,\n",
        ")\n",
        "\n",
        "# 3가지 버거킹 메뉴를 추천해 달라는 내용으로 모델 테스트\n",
        "llm.complete('Recommend only 3 items from Burger King’s menu.').text"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nxW_isgNJTBc"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8gieR3AaJUw3"
      },
      "source": [
        "## Langchain Rag 연결"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hsjImCyjJquY"
      },
      "source": [
        "### PDF 로드"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z-o-0CviWPhW",
        "outputId": "8843fa18-4c9e-4698-9194-88e615c1a4ed"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Document(page_content='NutritioN al\\niNformatio N\\nCanada november 2008', metadata={'source': 'Burger-King.pdf', 'page': 0})"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "loader = PyPDFLoader(\"Burger-King.pdf\")\n",
        "documents = loader.load()\n",
        "documents[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r03ewXcfJn7P"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hm9GKBnLJlEX"
      },
      "source": [
        "### PDF text 전처리"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4RqldS0vd1di"
      },
      "outputs": [],
      "source": [
        "output = []\n",
        "\n",
        "# text 정제\n",
        "for page in documents:\n",
        "    text = page.page_content\n",
        "    text = re.sub(r\"(\\w+)-\\n(\\w+)\", r\"\\1\\2\", text)   # 안녕-\\n하세요 -> 안녕하세요\n",
        "    text = re.sub(r\"(?<!\\n\\s)\\n(?!\\s\\n)\", \" \", text.strip()) # \"인\\n공\\n\\n지능펙\\n토리 -> 인공지능펙토리\n",
        "    text = re.sub(r\"\\n\\s*\\n\", \"\\n\\n\", text) # \\n버\\n\\n거\\n\\n킹\\n -> 버\\n거\\n킹\n",
        "    text = re.sub(r'®', '',text)\n",
        "    output.append(text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FRvyIiVweft4"
      },
      "outputs": [],
      "source": [
        "doc_chunks = []\n",
        "\n",
        "for idx, line in enumerate(output):\n",
        "    text_splitter = RecursiveCharacterTextSplitter(\n",
        "        chunk_size=2000, # 최대 청크 길이\n",
        "        separators=[\"\\n\\n\", \"\\n\", \".\", \"!\", \"?\", \",\", \" \", \"\"], #  텍스트를 청크로 분할하는 데 사용되는 문자 목록\n",
        "        chunk_overlap=0, # 인접한 청크 간에 중복되는 문자 수\n",
        "    )\n",
        "    chunks = text_splitter.split_text(line)\n",
        "    for chunk in chunks:\n",
        "        doc = Document(\n",
        "            page_content=chunk, metadata={ \"source\": 'Burger-King.pdf', \"page\": idx}\n",
        "        )\n",
        "        doc_chunks.append(doc)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJBCjF0jLyRa"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l7ZbzUhnLJlf"
      },
      "source": [
        "### huggingface의 embed_model 사용"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7LSOAIYhK1U2"
      },
      "outputs": [],
      "source": [
        "embed_model = HuggingFaceEmbeddings(\n",
        "    model_name=\"sentence-transformers/all-mpnet-base-v2\"\n",
        ")\n",
        "index = Chroma.from_documents(doc_chunks, embed_model)\n",
        "retriever = index.as_retriever(search_kwargs={\"k\": 2})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S-mVRl_wL2BD"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_6MiObeWL2_x"
      },
      "source": [
        "### GradientLLM load & langchain"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ltSnvOlIaIRP"
      },
      "outputs": [],
      "source": [
        "llm = GradientLLM(\n",
        "    model=os.environ[\"model_adapter_id\"],\n",
        "    model_kwargs=dict(max_generated_token_count=500),\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UtokEFanfCX9"
      },
      "outputs": [],
      "source": [
        "\n",
        "\n",
        "def llama2_output(out_text):\n",
        "  sep = out_text.split('Bot:')[-1].strip() if 'Bot:' in out_text else out_text\n",
        "  sep = sep.split('\\n\\nHuman')[0].strip() if '\\n\\nHuman' in sep else sep\n",
        "  return sep\n",
        "\n",
        "\n",
        "\n",
        "system_template=\"\"\"To answer the question at the end, use the following context. If you don't know the answer, just say you don't know and don't try to make up an answer.\n",
        "I want you to act as my Burger King menu recommender. It tells you your budget and suggests what to buy. You should only reply to items you recommend. Don't write a description.\n",
        "\n",
        "Below is an example.\n",
        "“My budget is 10,000 won, and it is the best menu combination within the budget.\"\n",
        "\n",
        "please answer in korean.\n",
        "\n",
        "\n",
        "{summaries}\n",
        "\"\"\"\n",
        "messages = [\n",
        "    SystemMessagePromptTemplate.from_template(system_template),\n",
        "    HumanMessagePromptTemplate.from_template(\"{question}\")\n",
        "]\n",
        "prompt = ChatPromptTemplate.from_messages(messages)\n",
        "\n",
        "\n",
        "chain_type_kwargs = {\"prompt\": prompt}\n",
        "bk_chain = RetrievalQAWithSourcesChain.from_chain_type(\n",
        "    llm,\n",
        "    chain_type=\"stuff\",\n",
        "    retriever=retriever,\n",
        "    chain_type_kwargs=chain_type_kwargs,\n",
        "    return_source_documents=True)\n",
        "    #reduce_k_below_max_tokens=True\n",
        "# )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LNwczxF2OmzP",
        "outputId": "addf686b-b829-41d8-aa03-c84867fbb83a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : Recommend only 3 items from Burger King’s menu.\n",
            "\n",
            "답변 : Sure, I can recommend 3 items from Burger King's menu based on your budget of 10,000 won. Here are my recommendations:\n",
            "\n",
            "1. Whopper Sandwich: This is Burger King's signature dish, and it's a great value for money. The Whopper features a 100% beef patty, sesame seed bun, tomato slices, lettuce, pickle slices, onion, mayonnaise, and ketchup. It's a classic burger that's sure to satisfy your hunger.\n",
            "\n",
            "2. Spicy Chicken Sandwich: If you like a little spice in your life, then the Spicy Chicken Sandwich is the way to go. This sandwich features a breaded chicken patty, sesame seed bun, lettuce, mayonnaise, and ketchup. What sets this sandwich apart is the addition of crispy onions and a spicy sauce that will leave you craving more.\n",
            "\n",
            "3. Tendergrill Chicken Sandwich: For a more premium burger experience, try the Tendergrill Chicken Sandwich. This sandwich features a breaded chicken patty, sesame seed bun, lettuce, mayonnaise, and ketchup. What makes this sandwich unique is the addition of bacon bits and a special sauce that will make your taste buds dance with joy.\n",
            "\n",
            "I hope you enjoy these recommendations!\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": 'Recommend only 3 items from Burger King’s menu.'})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {llama2_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vy_kfSOPfHXL",
        "outputId": "9818722f-d61e-4396-9e5e-ff4cf66c08b4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : Please provide the menu combination, price, and menu description without paying a total of 30,000 won.\n",
            "\n",
            "답변 : Burger King: 🍔 Whopper Sandwich, 🥤 Double Croissant, 🥤 English Muffin, 🥤 enormous omelette, 🥤 1% ChoColate milk, 🥤 hot ChoColate, 🥤 daSani water, 🥤 MoCha BK Joe iCed CoFFee. Total price: 28,900 won.\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": 'Please provide the menu combination, price, and menu description without paying a total of 30,000 won.'})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {llama2_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "36oL3TqNPfRK",
        "outputId": "9931ac68-09ee-4948-c1aa-c3333f40da9b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : What is included in the Crispy King & Mozzarella Ball set with Diablo Sauce?\n",
            "\n",
            "답변 : \n",
            "\n",
            "Burger King Menu Recommender: The Crispy King & Mozzarella Ball set with Diablo Sauce includes the following items:\n",
            "\n",
            "* Crispy King: A crispy chicken sandwich with a juicy chicken patty, topped with lettuce, tomato, and mayonnaise.\n",
            "* Mozzarella Ball: A ball of melted mozzarella cheese, served on the side.\n",
            "* Diablo Sauce: A spicy sauce made with a blend of hot peppers, vinegar, and spices.\n",
            "\n",
            "So, the Crispy King & Mozzarella Ball set with Diablo Sauce is a combination of a crispy chicken sandwich, melted mozzarella cheese, and a spicy sauce.\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": \"What is included in the Crispy King & Mozzarella Ball set with Diablo Sauce?\"})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {llama2_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JrFzh5t4VZms"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dqOsV2UBqbdR"
      },
      "source": [
        "## Gradio를 통한 bugerking chat"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 613
        },
        "id": "xe0tcI4djTL1",
        "outputId": "bcf3f579-7614-4355-a4ee-9bc5a480deb2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Colab notebook detected. To show errors in colab notebook, set debug=True in launch()\n",
            "Running on public URL: https://fe176376fd1abdf691.gradio.live\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<div><iframe src=\"https://fe176376fd1abdf691.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": []
          },
          "execution_count": 68,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "import logging\n",
        "import sys\n",
        "import gradio as gr\n",
        "import torch\n",
        "import gc\n",
        "\n",
        "def reset_state():\n",
        "    return [], [], \"Reset Done\"\n",
        "def reset_textbox():\n",
        "    return gr.update(value=\"\"),\"\"\n",
        "def transfer_input(inputs):\n",
        "    textbox = reset_textbox()\n",
        "    return (\n",
        "        inputs,\n",
        "        gr.update(value=\"\"),\n",
        "        gr.Button.update(visible=True),\n",
        "    )\n",
        "\n",
        "title = \"\"\"<h1 align=\"left\" style=\"min-width:350px; margin-top:0;\"> <img src=\"https://lh3.google.com/u/0/d/1txdmhh6pWjdJBpqGBRMdC0qQX2f7pzxI=w2020-h952-iv1\" width=\"32px\" style=\"display: inline\"> AIF 버거킹 chat </h1>\"\"\"\n",
        "description_top = \"\"\"\\\n",
        "<div align=\"left\">\n",
        "<p></p>\n",
        "<p>\n",
        "</p >\n",
        "</div>\n",
        "\"\"\"\n",
        "\n",
        "CONCURRENT_COUNT = 100\n",
        "\n",
        "ALREADY_CONVERTED_MARK = \"<!-- ALREADY CONVERTED BY PARSER. -->\"\n",
        "\n",
        "small_and_beautiful_theme = gr.themes.Soft(\n",
        "        primary_hue=gr.themes.Color(\n",
        "            c50=\"#02C160\",\n",
        "            c100=\"rgba(2, 193, 96, 0.2)\",\n",
        "            c200=\"#02C160\",\n",
        "            c300=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c400=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c500=\"rgba(2, 193, 96, 1.0)\",\n",
        "            c600=\"rgba(2, 193, 96, 1.0)\",\n",
        "            c700=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c800=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c900=\"#02C160\",\n",
        "            c950=\"#02C160\",\n",
        "        ),\n",
        "        secondary_hue=gr.themes.Color(\n",
        "            c50=\"#576b95\",\n",
        "            c100=\"#576b95\",\n",
        "            c200=\"#576b95\",\n",
        "            c300=\"#576b95\",\n",
        "            c400=\"#576b95\",\n",
        "            c500=\"#576b95\",\n",
        "            c600=\"#576b95\",\n",
        "            c700=\"#576b95\",\n",
        "            c800=\"#576b95\",\n",
        "            c900=\"#576b95\",\n",
        "            c950=\"#576b95\",\n",
        "        ),\n",
        "        neutral_hue=gr.themes.Color(\n",
        "            name=\"gray\",\n",
        "            c50=\"#f9fafb\",\n",
        "            c100=\"#f3f4f6\",\n",
        "            c200=\"#e5e7eb\",\n",
        "            c300=\"#d1d5db\",\n",
        "            c400=\"#B2B2B2\",\n",
        "            c500=\"#808080\",\n",
        "            c600=\"#636363\",\n",
        "            c700=\"#515151\",\n",
        "            c800=\"#393939\",\n",
        "            c900=\"#272727\",\n",
        "            c950=\"#171717\",\n",
        "        ),\n",
        "        radius_size=gr.themes.sizes.radius_sm,\n",
        "    ).set(\n",
        "        button_primary_background_fill=\"#06AE56\",\n",
        "        button_primary_background_fill_dark=\"#06AE56\",\n",
        "        button_primary_background_fill_hover=\"#07C863\",\n",
        "        button_primary_border_color=\"#06AE56\",\n",
        "        button_primary_border_color_dark=\"#06AE56\",\n",
        "        button_primary_text_color=\"#FFFFFF\",\n",
        "        button_primary_text_color_dark=\"#FFFFFF\",\n",
        "        button_secondary_background_fill=\"#F2F2F2\",\n",
        "        button_secondary_background_fill_dark=\"#2B2B2B\",\n",
        "        button_secondary_text_color=\"#393939\",\n",
        "        button_secondary_text_color_dark=\"#FFFFFF\",\n",
        "        # background_fill_primary=\"#F7F7F7\",\n",
        "        # background_fill_primary_dark=\"#1F1F1F\",\n",
        "        block_title_text_color=\"*primary_500\",\n",
        "        block_title_background_fill=\"*primary_100\",\n",
        "        input_background_fill=\"#F6F6F6\",\n",
        "    )\n",
        "\n",
        "with open(\"/content/easy_finetuner/custom.css\", \"r\", encoding=\"utf-8\") as f:\n",
        "    customCSS = f.read()\n",
        "\n",
        "logging.basicConfig(\n",
        "    level=logging.DEBUG,\n",
        "    format=\"%(asctime)s [%(levelname)s] [%(filename)s:%(lineno)d] %(message)s\",\n",
        ")\n",
        "\n",
        "\n",
        "total_count = 0\n",
        "def predict(input_text,\n",
        "            history):\n",
        "    global bk_chain\n",
        "\n",
        "    result = bk_chain({\"question\": input_text})\n",
        "    answer = llama2_output(result['answer'])\n",
        "\n",
        "    # answer = llm.complete(input_text).text\n",
        "    # print(input_text)\n",
        "    # print(answer)\n",
        "    history = history + [((input_text, None))]\n",
        "    history = history + [((None, answer))]\n",
        "    return history, history, \"Generate: Success\"\n",
        "\n",
        "\n",
        "with gr.Blocks(css=customCSS, theme=small_and_beautiful_theme) as demo:\n",
        "    history = gr.State([])\n",
        "    user_question = gr.State(\"\")\n",
        "    with gr.Row():\n",
        "        gr.HTML(title)\n",
        "        status_display = gr.Markdown(\"Success\", elem_id=\"status_display\")\n",
        "    gr.Markdown(description_top)\n",
        "    with gr.Row(scale=1).style(equal_height=True):\n",
        "        with gr.Column(scale=5):\n",
        "            with gr.Row(scale=1):\n",
        "                chatbot = gr.Chatbot(avatar_images=('https://yt3.googleusercontent.com/_JbQDtNPfI8h6RPW_9Og5qlGhSBhpMp5qX3JR7iNeSC9XZL4btbNE3dFB4ec77tauPA-nLGQTQ=s900-c-k-c0x00ffffff-no-rj', 'https://github.com/jmorganca/ollama/assets/3325447/0d0b44e2-8f4a-4e99-9b52-a5c1c741c8f7'),elem_id=\"chuanhu_chatbot\").style(height=\"100%\")\n",
        "            with gr.Row(scale=1):\n",
        "                with gr.Column(scale=12):\n",
        "                    user_input = gr.Textbox(\n",
        "                        show_label=False, placeholder=\"Enter text\"\n",
        "                    ).style(container=False)\n",
        "                with gr.Column(min_width=70, scale=1):\n",
        "                    submitBtn = gr.Button(\"Send\")\n",
        "                with gr.Column(min_width=70, scale=1):\n",
        "                    cancelBtn = gr.Button(\"Stop\")\n",
        "            with gr.Row(scale=1):\n",
        "                emptyBtn = gr.Button(\n",
        "                    \"🧹 New Conversation\",\n",
        "                )\n",
        "\n",
        "\n",
        "    predict_args = dict(\n",
        "        fn=predict,\n",
        "        inputs=[\n",
        "            user_question,\n",
        "            history\n",
        "        ],\n",
        "        outputs=[chatbot, history, status_display],\n",
        "        show_progress=True,\n",
        "    )\n",
        "\n",
        "    reset_args = dict(\n",
        "        fn=reset_textbox, inputs=[], outputs=[user_input, status_display]\n",
        "    )\n",
        "\n",
        "    # Chatbot\n",
        "    transfer_input_args = dict(\n",
        "        fn=transfer_input, inputs=[user_input], outputs=[user_question, user_input, submitBtn], show_progress=True\n",
        "    )\n",
        "\n",
        "\n",
        "\n",
        "    predict_event1 = user_input.submit(**transfer_input_args).then(**predict_args)\n",
        "\n",
        "    predict_event2 = submitBtn.click(**transfer_input_args).then(**predict_args)\n",
        "\n",
        "    gr.Markdown(\"<h2>버거킹 chat 시연 리스트</h2>\")\n",
        "    gr.Examples(\n",
        "        examples=[\n",
        "            \"Recommend only 3 items from Burger King’s menu.\",\n",
        "            \"Please provide the menu combination, price, and menu description without paying a total of 30,000 won.\",\n",
        "            \"What is included in the Crispy King & Mozzarella Ball set with Diablo Sauce?\"\n",
        "                  ],\n",
        "        inputs=user_input\n",
        "    )\n",
        "\n",
        "\n",
        "    emptyBtn.click(\n",
        "        reset_state,\n",
        "        outputs=[chatbot, history, status_display],\n",
        "        show_progress=True,\n",
        "    )\n",
        "    emptyBtn.click(**reset_args)\n",
        "\n",
        "\n",
        "demo.queue(concurrency_count=1).launch(share=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qQzbZVKmUgEq"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQfa15NHXr0Q"
      },
      "source": [
        "# colab 무료 GPU를 통한 fine-tune"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R5KOjTybYbjX"
      },
      "source": [
        "## llama2 fine tune 데이터 생성"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8-LAd_0dZ5ox"
      },
      "source": [
        "### meta llama2 7b fine tune 데이터 생성"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 194,
          "referenced_widgets": [
            "6f4bbdb8a57e4d6d8c660867a47777a3",
            "d2d673e40e0d4ad3ababde7c2953cb5c",
            "afa4aff51df44a09a323c0ca3d31839e",
            "162e27edc0854ac28377669ae89a5504",
            "97eb2c1611034f468dc733367c375256",
            "667b80adfff647e1903aaf06d632a653",
            "b62c4812975b4285b612c6edccfbfc31",
            "18de8b4c297942b9bae68c5b879b944a",
            "6b05461b75fc49c0a192c9102c0bbf5d",
            "849637e9357040dfaef6c5a1ed4c22fe",
            "0de107edb99d48dd8f6211bef888708a"
          ]
        },
        "id": "1Qq3_bxxMrrt",
        "outputId": "7ce77f4e-b5ca-4d39-85d1-05d4268b979e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "데이터셋 확인\n",
            "['<s>### Instruction: \\nWhat makes the Monster X set spicy? \\n\\n### Response: \\nThe Monster X set is made spicy by the intense flavor of the Diablo sauce.</s>', '<s>### Instruction: \\nWhat is the cost of a regular French Fries at Burger King? \\n\\n### Response: \\nThe cost of a regular French Fries at Burger King is 3,000 won.</s>', '<s>### Instruction: \\nHow much does the Whole Shrimp Whopper set cost at Burger King? \\n\\n### Response: \\nThe Whole Shrimp Whopper set at Burger King costs 11,300 won.</s>', '<s>### Instruction: \\nWhat type of sauce is used in the Side Shrimp Burger at Burger King? \\n\\n### Response: \\nThe Side Shrimp Burger at Burger King uses a sweet and sour sauce.</s>', \"<s>### Instruction: \\nWhat is the main ingredient in Burger King's Whopper? \\n\\n### Response: \\nThe main ingredient in Burger King's Whopper is a freshly grilled beef patty.</s>\"]\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6f4bbdb8a57e4d6d8c660867a47777a3",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Saving the dataset (0/1 shards):   0%|          | 0/81 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "데이터셋 info 확인\n",
            "Dataset({\n",
            "    features: ['text'],\n",
            "    num_rows: 81\n",
            "})\n"
          ]
        }
      ],
      "source": [
        "bergerking_dataset = []\n",
        "with jsonlines.open(\"/content/qa_버거킹_train.jsonl\") as f:\n",
        "    for line in f.iter():\n",
        "      # bergerking_dataset.append(f'<s>[INST] {line[\"inputs\"]} [/INST] {line[\"response\"]} </s>')\n",
        "      bergerking_dataset.append(f'<s>### Instruction: \\n{line[\"inputs\"]} \\n\\n### Response: \\n{line[\"response\"]}</s>')\n",
        "\n",
        "# 데이터셋 확인\n",
        "print('데이터셋 확인')\n",
        "print(bergerking_dataset[:5])\n",
        "\n",
        "# 데이터셋 생성 및 저장\n",
        "burgerking_dataset = Dataset.from_dict({\"text\": bergerking_dataset})\n",
        "burgerking_dataset.save_to_disk('/content/easy_finetuner/example-datasets/burgerking_dataset')\n",
        "\n",
        "# 데이터셋 info 확인\n",
        "print('데이터셋 info 확인')\n",
        "print(burgerking_dataset)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z5cF3JhTabeo"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZcnMbpg_amMB"
      },
      "source": [
        "### ko llama2 7b fine tune 데이터 생성"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 194,
          "referenced_widgets": [
            "888756553d154e4faffaf2b03a0d66a4",
            "e13ed38e533e4bad996252e19cd32000",
            "578ec5a307444645a87aa95c3a659fe6",
            "f7027e18b6c64490a0ffd66e29eaaad1",
            "2368be330217479980bbf7f3d81cfaf5",
            "f7a6748ee5a6421fa2f51df72d6c0ecb",
            "76a0b81f731b40de898828616efc8768",
            "6b43f664d6cc4e468af83033761b7ed3",
            "0d5424dba59845f6a1fc503794decf1a",
            "b0b8d246d4f74b599a1c35647697d62a",
            "a48f8add4b0a41f3baab01d19795c249"
          ]
        },
        "id": "Fpane0ZSRZaP",
        "outputId": "cb5d72c6-befd-4eb5-c444-a26cb024183c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "데이터셋 확인\n",
            "['<s>### Instruction: \\n와퍼의 주요 재료는 무엇인가요? \\n\\n### Response: \\n와퍼의 주요 재료는 순 쇠고기 패티와 다양한 야채입니다.</s>', '<s>### Instruction: \\n코카-콜라L의 가격은 얼마인가요? \\n\\n### Response: \\n코카-콜라L의 가격은 3,100원입니다.</s>', '<s>### Instruction: \\n코코넛슈림프 6조각과 스위트칠리소스의 가격은 얼마인가요? \\n\\n### Response: \\n코코넛슈림프 6조각과 스위트칠리소스의 가격은 4,800원입니다.</s>', '<s>### Instruction: \\n크리미모짜볼5조각의 가격은 얼마인가요? \\n\\n### Response: \\n크리미모짜볼5조각의 가격은 3,500원입니다.</s>', '<s>### Instruction: \\n치킨버거 세트에는 어떤 소스로 고소함을 더했나요? \\n\\n### Response: \\n치킨버거 세트에는 풍부한 마요 소스로 고소함을 더했습니다.</s>']\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "888756553d154e4faffaf2b03a0d66a4",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Saving the dataset (0/1 shards):   0%|          | 0/97 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "데이터셋 info 확인\n",
            "Dataset({\n",
            "    features: ['text'],\n",
            "    num_rows: 97\n",
            "})\n"
          ]
        }
      ],
      "source": [
        "bergerking_dataset = []\n",
        "with jsonlines.open(\"/content/qa_버거킹_train_ko.jsonl\") as f:\n",
        "    for line in f.iter():\n",
        "      bergerking_dataset.append(f'<s>### Instruction: \\n{line[\"inputs\"]} \\n\\n### Response: \\n{line[\"response\"]}</s>')\n",
        "\n",
        "# 데이터셋 확인\n",
        "print('데이터셋 확인')\n",
        "print(bergerking_dataset[:5])\n",
        "\n",
        "# 데이터셋 생성 및 저장\n",
        "burgerking_dataset = Dataset.from_dict({\"text\": bergerking_dataset})\n",
        "burgerking_dataset.save_to_disk('/content/easy_finetuner/example-datasets/burgerking_dataset_ko')\n",
        "\n",
        "# 데이터셋 info 확인\n",
        "print('데이터셋 info 확인')\n",
        "print(burgerking_dataset)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3xAuABUKac49"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uy7jiMM4YufO"
      },
      "source": [
        "## Easy Finetuner 실행"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R51gBXuAZ092"
      },
      "source": [
        "- https://github.com/choijhyeok/easy_finetuner\n",
        "- 기존의 여러 gradio를 참고해서 만들었습니다. (T4 GPU에 최적화)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JZINrNH4OkVM"
      },
      "outputs": [],
      "source": [
        "\"버거킹에서 판매하는 스프라이트 제로의 특징은 무엇인가요?\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SJ81RsG4QnFn",
        "outputId": "9a803b9d-9d60-4604-a97b-74d2a6b5eb2b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content/easy_finetuner\n",
            "2024-01-10 08:58:38.973649: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
            "2024-01-10 08:58:38.973707: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
            "2024-01-10 08:58:38.975176: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
            "2024-01-10 08:58:40.060743: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/utils.py:816: UserWarning: Expected at least 12 arguments for function <function UI.training_launch_block.<locals>.train at 0x7daa8fb33a30>, received 10.\n",
            "  warnings.warn(\n",
            "Running on local URL:  http://127.0.0.1:8000\n",
            "Running on public URL: https://4414011af21f35c6b9.gradio.live\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n",
            "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:2193: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1714: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/components/dropdown.py:176: UserWarning: Using the update method is deprecated. Simply return a new object instead, e.g. `return gr.Dropdown(...)` instead of `return gr.Dropdown.update(...)`.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/helpers.py:784: UserWarning: Unexpected argument. Filling with None.\n",
            "  warnings.warn(\"Unexpected argument. Filling with None.\")\n",
            "/usr/local/lib/python3.10/dist-packages/peft/utils/other.py:145: FutureWarning: prepare_model_for_int8_training is deprecated and will be removed in a future version. Use prepare_model_for_kbit_training instead.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/trl/trainer/sft_trainer.py:159: UserWarning: You didn't pass a `max_seq_length` argument to the SFTTrainer, this will default to 1024\n",
            "  warnings.warn(\n",
            "You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n",
            "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
            "  warnings.warn(\n",
            "{'loss': 2.568, 'learning_rate': 0.00018550053929480202, 'epoch': 1.0}\n",
            "{'loss': 0.7543, 'learning_rate': 0.000136764169663272, 'epoch': 2.0}\n",
            "{'loss': 0.5639, 'learning_rate': 7.307467669163655e-05, 'epoch': 3.0}\n",
            "{'loss': 0.4329, 'learning_rate': 2.03365443542764e-05, 'epoch': 4.0}\n",
            "{'loss': 0.3698, 'learning_rate': 0.0, 'epoch': 5.0}\n",
            "{'train_runtime': 151.0097, 'train_samples_per_second': 3.212, 'train_steps_per_second': 0.828, 'train_loss': 0.9377700119018555, 'epoch': 5.0}\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/components/dropdown.py:176: UserWarning: Using the update method is deprecated. Simply return a new object instead, e.g. `return gr.Dropdown(...)` instead of `return gr.Dropdown.update(...)`.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/components/dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: beomi_llama-2-ko-7b_burgerking-ko or set allow_custom_value=True.\n",
            "  warnings.warn(\n",
            "Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers\n",
            "pip install xformers.\n",
            "The model 'PeftModelForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'FalconForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MusicgenForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].\n",
            "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1270: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )\n",
            "  warnings.warn(\n",
            "Keyboard interruption in main thread... closing server.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2361, in block_thread\n",
            "    time.sleep(0.1)\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/easy_finetuner/app.py\", line 483, in <module>\n",
            "    ui.run()\n",
            "  File \"/content/easy_finetuner/app.py\", line 479, in run\n",
            "    self.ui.queue().launch(show_error=True, share=SHARE, server_name=SERVER_HOST, server_port=SERVER_PORT)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2266, in launch\n",
            "    self.block_thread()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2365, in block_thread\n",
            "    self.server.close()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/gradio/networking.py\", line 75, in close\n",
            "    self.thread.join()\n",
            "  File \"/usr/lib/python3.10/threading.py\", line 1096, in join\n",
            "    self._wait_for_tstate_lock()\n",
            "  File \"/usr/lib/python3.10/threading.py\", line 1116, in _wait_for_tstate_lock\n",
            "    if lock.acquire(block, timeout):\n",
            "KeyboardInterrupt\n",
            "Killing tunnel 127.0.0.1:8000 <> https://4414011af21f35c6b9.gradio.live\n"
          ]
        }
      ],
      "source": [
        "%cd /content/easy_finetuner\n",
        "!python app.py --share"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zUVb_30tjPHn"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Gs5IJyoTcoJ"
      },
      "source": [
        "## fine-tune llama2 RAG"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "be4d68bd90d542b089b017f9632e6438",
            "180200658f4246fe9cec26585751d447",
            "5fb07ff1a602452ba5b8d3f16b64b075",
            "73317d860aee479dbc3c196edab8a9e2",
            "6972dfe325fe4503aeef774b52d7df6b",
            "19ece5bb8a9d4260a994d37577fdd873",
            "c5cb50e148fd49bfa58c790d0b51feb9",
            "59cc670c6690466db945f5e87851a71c",
            "bf40c379c91a4f2086a6aed3990e7814",
            "baeaeac4c780493fb8b5b701454df008",
            "9e864f89e1a346b8909b199d6cdb9ea3"
          ]
        },
        "id": "JSCpLfsoENcm",
        "outputId": "e78ac654-0903-4b30-e4a9-cf99013fac38"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "be4d68bd90d542b089b017f9632e6438",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/15 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from transformers import AutoTokenizer, LlamaForCausalLM\n",
        "import torch\n",
        "import argparse\n",
        "import os\n",
        "import torch\n",
        "from datasets import load_dataset\n",
        "from transformers import (\n",
        "    AutoModelForCausalLM,\n",
        "    AutoTokenizer,\n",
        "    BitsAndBytesConfig,\n",
        "    HfArgumentParser,\n",
        "    TrainingArguments,\n",
        "    pipeline,\n",
        "    logging,\n",
        ")\n",
        "from peft import LoraConfig, PeftModel\n",
        "from trl import SFTTrainer\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n",
        "\n",
        "\n",
        "def llama2_prompt(input_text):\n",
        "  return f'### Instruction:\\n{input_text}\\n\\n### Response:'\n",
        "\n",
        "def llama2_output(ouput_text):\n",
        "  sep = ouput_text[0]['generated_text'].split('### Response:')[1].split('### Instruction')[0].split('## Instruction')[0].split('# Instruction')[0].split('Instruction')[0]\n",
        "  sep = sep[1:] if sep[0] == '.' else sep\n",
        "  sep = sep[:sep.find('.')+1] if '.' in sep else sep\n",
        "  return sep\n",
        "\n",
        "adapter_name = 'beomi_llama-2-ko-7b_burgerking-ko'\n",
        "\n",
        "\n",
        "compute_dtype = getattr(torch, 'float16')\n",
        "\n",
        "bnb_config = BitsAndBytesConfig(\n",
        "    load_in_4bit=True,\n",
        "    bnb_4bit_quant_type='nf4',\n",
        "    bnb_4bit_compute_dtype=compute_dtype,\n",
        "    bnb_4bit_use_double_quant=False\n",
        ")\n",
        "\n",
        "model = AutoModelForCausalLM.from_pretrained('beomi/llama-2-ko-7b', quantization_config=bnb_config, device_map={'': 0}, use_auth_token=os.environ[\"huggingface_token\"])\n",
        "model.config.use_cache = False\n",
        "model.config.pretraining_tp = 1\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained('beomi/llama-2-ko-7b', trust_remote_code=True, use_auth_token=os.environ[\"huggingface_token\"])\n",
        "tokenizer.pad_token = tokenizer.eos_token\n",
        "tokenizer.padding_side = \"right\"\n",
        "\n",
        "\n",
        "\n",
        "model = PeftModel.from_pretrained(model, f'/content/easy_finetuner/lora/{adapter_name}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SvXGo6QpxvoJ",
        "outputId": "fd7aeca8-ff90-43b7-c9b0-3b8465e4d6fb"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers\n",
            "pip install xformers.\n",
            "The model 'PeftModelForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'FalconForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MusicgenForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].\n"
          ]
        }
      ],
      "source": [
        "pipe = pipeline(task=\"text-generation\",\n",
        "                model=model,\n",
        "                tokenizer=tokenizer,\n",
        "                max_length=150,\n",
        "                do_sample=True,\n",
        "                temperature=0.1,\n",
        "                num_return_sequences=1,\n",
        "                eos_token_id=tokenizer.eos_token_id,\n",
        "                top_k=3,\n",
        "                # top_p=0.3,\n",
        "                repetition_penalty = 1.3,\n",
        "                framework='pt'\n",
        "                # early_stopping=True\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2q7YhfjzDUTJ",
        "outputId": "ac8ea10c-c837-4293-f119-60e1cb0b288f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "prompt = \"스모키 바비큐 X의 독특한 특징은 무엇인가요?\"\n",
        "result = pipe(llama2_prompt(prompt))\n",
        "print(llama2_output(result))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "74eMNTKWJVPM",
        "outputId": "3207dd68-18b0-4fa5-f616-91b52951169d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "스프라이트 제로는 0kcal이며, 칼로리 걱정 없이 즐길 수 있는 음료입니다.\n"
          ]
        }
      ],
      "source": [
        "prompt = \"버거킹에서 판매하는 스프라이트 제로의 특징은 무엇인가요?\"\n",
        "result = pipe(llama2_prompt(prompt))\n",
        "print(llama2_output(result))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vBwr6YaKKxb6",
        "outputId": "d9965834-1f19-4ffe-dc5d-66248f2994cb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "코코넛슈림프 3조각과 스위트칠리소스의 가격은 5,800원입니다.\n"
          ]
        }
      ],
      "source": [
        "prompt = \"코코넛슈림프 3조각과 스위트칠리소스의 가격은 얼마인가요?\"\n",
        "result = pipe(llama2_prompt(prompt))\n",
        "print(llama2_output(result))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rd8BRRJa7X2K"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JSbBU923yC2A",
        "outputId": "6dfd85c3-1811-40a5-8363-e6db61d30616"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content\n"
          ]
        }
      ],
      "source": [
        "%cd /content/\n",
        "loader = PyPDFLoader('/content/버거킹.pdf')\n",
        "documents = loader.load()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iKrvdIuHyNRw"
      },
      "outputs": [],
      "source": [
        "output = []\n",
        "# text 정제\n",
        "for page in documents:\n",
        "    text = page.page_content\n",
        "    text = re.sub(r\"(\\w+)-\\n(\\w+)\", r\"\\1\\2\", text)   # 안녕-\\n하세요 -> 안녕하세요\n",
        "    text = re.sub(r\"(?<!\\n\\s)\\n(?!\\s\\n)\", \" \", text.strip()) # \"인\\n공\\n\\n지능펙\\n토리 -> 인공지능펙토리\n",
        "    text = re.sub(r\"\\n\\s*\\n\", \"\\n\\n\", text) # \\n버\\n\\n거\\n\\n킹\\n -> 버\\n거\\n킹\n",
        "    output.append(text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nf_5KOICyCzs"
      },
      "outputs": [],
      "source": [
        "doc_chunks = []\n",
        "\n",
        "for line in output:\n",
        "    text_splitter = RecursiveCharacterTextSplitter(\n",
        "        chunk_size=2000, # 최대 청크 길이\n",
        "        separators=[\"\\n\\n\", \"\\n\", \".\", \"!\", \"?\", \",\", \" \", \"\"], #  텍스트를 청크로 분할하는 데 사용되는 문자 목록\n",
        "        chunk_overlap=0, # 인접한 청크 간에 중복되는 문자 수\n",
        "    )\n",
        "    chunks = text_splitter.split_text(line)\n",
        "    for i, chunk in enumerate(chunks):\n",
        "        doc = Document(\n",
        "            page_content=chunk, metadata={ \"source\": '버거킹.pdf', \"page\": i}\n",
        "        )\n",
        "        doc_chunks.append(doc)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NaByc6bcyCxD"
      },
      "outputs": [],
      "source": [
        "embed_model = HuggingFaceEmbeddings(\n",
        "    model_name=\"jhgan/ko-sbert-sts\"\n",
        ")\n",
        "index = Chroma.from_documents(doc_chunks, embed_model)\n",
        "retriever = index.as_retriever(search_kwargs={\"k\": 1})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JjswvJl_kGN",
        "outputId": "51a2a4f6-7015-4c13-a992-b56ac2e298a9"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "The model 'PeftModelForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'FalconForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MusicgenForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].\n"
          ]
        }
      ],
      "source": [
        "from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline\n",
        "\n",
        "\n",
        "pipe = pipeline(task=\"text-generation\",\n",
        "                model=model,\n",
        "                tokenizer=tokenizer,\n",
        "                max_length=780,\n",
        "                do_sample=True,\n",
        "                temperature=0.1,\n",
        "                num_return_sequences=1,\n",
        "                eos_token_id=tokenizer.eos_token_id,\n",
        "                top_k=3,\n",
        "                # top_p=0.3,\n",
        "                repetition_penalty = 1.3,\n",
        "                framework='pt'\n",
        "                # early_stopping=True\n",
        ")\n",
        "\n",
        "\n",
        "llm = HuggingFacePipeline(pipeline=pipe)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h8ba0VhkofDO"
      },
      "outputs": [],
      "source": [
        "\n",
        "def bkchain_output(text):\n",
        "  text = text.split('Machine:')[1] if 'Machine:' in text else text\n",
        "  text = text.split('Human:')[0] if 'Human:' in text else text\n",
        "  return text.strip()\n",
        "\n",
        "\n",
        "\n",
        "system_template=\"\"\"To answer the question at the end, use the following context. If you don't know the answer, just say you don't know and don't try to make up an answer.\n",
        "I want you to act as my Burger King menu recommender. It tells you your budget and suggests what to buy.\n",
        "\n",
        "please answer in korean.\n",
        "\n",
        "\n",
        "{summaries}\n",
        "\"\"\"\n",
        "messages = [\n",
        "    SystemMessagePromptTemplate.from_template(system_template),\n",
        "    HumanMessagePromptTemplate.from_template(\"{question}\")\n",
        "]\n",
        "prompt = ChatPromptTemplate.from_messages(messages)\n",
        "\n",
        "\n",
        "chain_type_kwargs = {\"prompt\": prompt}\n",
        "bk_chain = RetrievalQAWithSourcesChain.from_chain_type(\n",
        "    llm,\n",
        "    chain_type=\"stuff\",\n",
        "    retriever=retriever,\n",
        "    chain_type_kwargs=chain_type_kwargs,\n",
        "    return_source_documents=True,\n",
        "    reduce_k_below_max_tokens=True\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zP6XdkF_oo0b",
        "outputId": "d7eec84c-57e4-420c-efc0-9e649e28c1c9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : 스모키 바비큐 X의 독특한 특징은 무엇인가요?\n",
            "\n",
            "답변 : 스모키 바비큐 X는 강력한 불향을 입혀 더욱 깊은 풍미를 느낄 수 있습니다. 또한 육즙 가득한 쇠고기 패티 위에 아낌없이 뿌려진 모짜렐라 치즈의 환상의 조합을 자랑합니다. ​\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": \"스모키 바비큐 X의 독특한 특징은 무엇인가요?\"})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {bkchain_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8u7Z4nr6pq2s",
        "outputId": "4f62e7f7-1e19-4ca1-b899-10eda90bf165"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : 버거킹에서 판매하는 스프라이트 제로의 특징은 무엇인가요?\n",
            "\n",
            "답변 : 스프라이트 제로는 칼로리가 낮으며, 천연감미료인 에리스리톨을 사용하여 단 맛을 구현하였습니다. 또한 인공색소가 아닌 자연유래 색소를 사용해 더욱 건강한 음료를 즐기실 수 있습니다.​\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": \"버거킹에서 판매하는 스프라이트 제로의 특징은 무엇인가요?\"})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {bkchain_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aOv4kXnpp1Oz",
        "outputId": "135dcf6e-dced-448e-ae8f-a37914f2cc6d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "질문 : 롱치킨버거에는 어떤 소스와 야채가 사용되었나요?\n",
            "\n",
            "답변 : 롱치킨버거는 부드럽고 달콤한 마요네즈 소스에 신선한 양상추를 곁들여 더욱더욱 푸짐합니다. ​\n"
          ]
        }
      ],
      "source": [
        "result = bk_chain({\"question\": \"롱치킨버거에는 어떤 소스와 야채가 사용되었나요?\"})\n",
        "\n",
        "print(f\"질문 : {result['question']}\")\n",
        "print()\n",
        "print(f\"답변 : {bkchain_output(result['answer'])}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sx0tCux_k7i3"
      },
      "source": [
        "&nbsp;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VMeqmpOsk8sB"
      },
      "source": [
        "### Local llm Gradio"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 649
        },
        "id": "-RFCj8OxxxYp",
        "outputId": "03a48019-2ad3-4070-ce54-08f87251fd3b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Setting queue=True in a Colab notebook requires sharing enabled. Setting `share=True` (you can turn this off by setting `share=False` in `launch()` explicitly).\n",
            "\n",
            "Colab notebook detected. To show errors in colab notebook, set debug=True in launch()\n",
            "Running on public URL: https://606a5c6aa6d5f9d745.gradio.live\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<div><iframe src=\"https://606a5c6aa6d5f9d745.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": []
          },
          "execution_count": 153,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "import logging\n",
        "import sys\n",
        "import gradio as gr\n",
        "import torch\n",
        "import gc\n",
        "\n",
        "def reset_state():\n",
        "    return [], [], \"Reset Done\"\n",
        "def reset_textbox():\n",
        "    return gr.update(value=\"\"),\"\"\n",
        "def transfer_input(inputs):\n",
        "    textbox = reset_textbox()\n",
        "    return (\n",
        "        inputs,\n",
        "        gr.update(value=\"\"),\n",
        "        gr.Button.update(visible=True),\n",
        "    )\n",
        "\n",
        "title = \"\"\"<h1 align=\"left\" style=\"min-width:350px; margin-top:0;\"> <img src=\"https://lh3.google.com/u/0/d/1txdmhh6pWjdJBpqGBRMdC0qQX2f7pzxI=w2020-h952-iv1\" width=\"32px\" style=\"display: inline\"> AIF 버거킹 chat </h1>\"\"\"\n",
        "description_top = \"\"\"\\\n",
        "<div align=\"left\">\n",
        "<p></p>\n",
        "<p>\n",
        "</p >\n",
        "</div>\n",
        "\"\"\"\n",
        "\n",
        "CONCURRENT_COUNT = 100\n",
        "\n",
        "\n",
        "ALREADY_CONVERTED_MARK = \"<!-- ALREADY CONVERTED BY PARSER. -->\"\n",
        "\n",
        "small_and_beautiful_theme = gr.themes.Soft(\n",
        "        primary_hue=gr.themes.Color(\n",
        "            c50=\"#02C160\",\n",
        "            c100=\"rgba(2, 193, 96, 0.2)\",\n",
        "            c200=\"#02C160\",\n",
        "            c300=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c400=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c500=\"rgba(2, 193, 96, 1.0)\",\n",
        "            c600=\"rgba(2, 193, 96, 1.0)\",\n",
        "            c700=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c800=\"rgba(2, 193, 96, 0.32)\",\n",
        "            c900=\"#02C160\",\n",
        "            c950=\"#02C160\",\n",
        "        ),\n",
        "        secondary_hue=gr.themes.Color(\n",
        "            c50=\"#576b95\",\n",
        "            c100=\"#576b95\",\n",
        "            c200=\"#576b95\",\n",
        "            c300=\"#576b95\",\n",
        "            c400=\"#576b95\",\n",
        "            c500=\"#576b95\",\n",
        "            c600=\"#576b95\",\n",
        "            c700=\"#576b95\",\n",
        "            c800=\"#576b95\",\n",
        "            c900=\"#576b95\",\n",
        "            c950=\"#576b95\",\n",
        "        ),\n",
        "        neutral_hue=gr.themes.Color(\n",
        "            name=\"gray\",\n",
        "            c50=\"#f9fafb\",\n",
        "            c100=\"#f3f4f6\",\n",
        "            c200=\"#e5e7eb\",\n",
        "            c300=\"#d1d5db\",\n",
        "            c400=\"#B2B2B2\",\n",
        "            c500=\"#808080\",\n",
        "            c600=\"#636363\",\n",
        "            c700=\"#515151\",\n",
        "            c800=\"#393939\",\n",
        "            c900=\"#272727\",\n",
        "            c950=\"#171717\",\n",
        "        ),\n",
        "        radius_size=gr.themes.sizes.radius_sm,\n",
        "    ).set(\n",
        "        button_primary_background_fill=\"#06AE56\",\n",
        "        button_primary_background_fill_dark=\"#06AE56\",\n",
        "        button_primary_background_fill_hover=\"#07C863\",\n",
        "        button_primary_border_color=\"#06AE56\",\n",
        "        button_primary_border_color_dark=\"#06AE56\",\n",
        "        button_primary_text_color=\"#FFFFFF\",\n",
        "        button_primary_text_color_dark=\"#FFFFFF\",\n",
        "        button_secondary_background_fill=\"#F2F2F2\",\n",
        "        button_secondary_background_fill_dark=\"#2B2B2B\",\n",
        "        button_secondary_text_color=\"#393939\",\n",
        "        button_secondary_text_color_dark=\"#FFFFFF\",\n",
        "        # background_fill_primary=\"#F7F7F7\",\n",
        "        # background_fill_primary_dark=\"#1F1F1F\",\n",
        "        block_title_text_color=\"*primary_500\",\n",
        "        block_title_background_fill=\"*primary_100\",\n",
        "        input_background_fill=\"#F6F6F6\",\n",
        "    )\n",
        "\n",
        "with open(\"/content/easy_finetuner/custom.css\", \"r\", encoding=\"utf-8\") as f:\n",
        "    customCSS = f.read()\n",
        "\n",
        "logging.basicConfig(\n",
        "    level=logging.DEBUG,\n",
        "    format=\"%(asctime)s [%(levelname)s] [%(filename)s:%(lineno)d] %(message)s\",\n",
        ")\n",
        "\n",
        "\n",
        "total_count = 0\n",
        "def predict(input_text,\n",
        "            history):\n",
        "    global bk_chain\n",
        "\n",
        "    result = bk_chain({\"question\": input_text})\n",
        "    history = history + [((input_text, None))]\n",
        "    history = history + [((None, bkchain_output(result['answer'])))]\n",
        "    return history, history, \"Generate: Success\"\n",
        "\n",
        "\n",
        "with gr.Blocks(css=customCSS, theme=small_and_beautiful_theme) as demo:\n",
        "    history = gr.State([])\n",
        "    user_question = gr.State(\"\")\n",
        "    with gr.Row():\n",
        "        gr.HTML(title)\n",
        "        status_display = gr.Markdown(\"Success\", elem_id=\"status_display\")\n",
        "    gr.Markdown(description_top)\n",
        "    with gr.Row(scale=1).style(equal_height=True):\n",
        "        with gr.Column(scale=5):\n",
        "            with gr.Row(scale=1):\n",
        "                chatbot = gr.Chatbot(avatar_images=('https://yt3.googleusercontent.com/_JbQDtNPfI8h6RPW_9Og5qlGhSBhpMp5qX3JR7iNeSC9XZL4btbNE3dFB4ec77tauPA-nLGQTQ=s900-c-k-c0x00ffffff-no-rj', 'https://github.com/jmorganca/ollama/assets/3325447/0d0b44e2-8f4a-4e99-9b52-a5c1c741c8f7'),elem_id=\"chuanhu_chatbot\").style(height=\"100%\")\n",
        "            with gr.Row(scale=1):\n",
        "                with gr.Column(scale=12):\n",
        "                    user_input = gr.Textbox(\n",
        "                        show_label=False, placeholder=\"Enter text\"\n",
        "                    ).style(container=False)\n",
        "                with gr.Column(min_width=70, scale=1):\n",
        "                    submitBtn = gr.Button(\"Send\")\n",
        "                with gr.Column(min_width=70, scale=1):\n",
        "                    cancelBtn = gr.Button(\"Stop\")\n",
        "            with gr.Row(scale=1):\n",
        "                emptyBtn = gr.Button(\n",
        "                    \"🧹 New Conversation\",\n",
        "                )\n",
        "\n",
        "\n",
        "    predict_args = dict(\n",
        "        fn=predict,\n",
        "        inputs=[\n",
        "            user_question,\n",
        "            history\n",
        "        ],\n",
        "        outputs=[chatbot, history, status_display],\n",
        "        show_progress=True,\n",
        "    )\n",
        "\n",
        "    reset_args = dict(\n",
        "        fn=reset_textbox, inputs=[], outputs=[user_input, status_display]\n",
        "    )\n",
        "\n",
        "    # Chatbot\n",
        "    transfer_input_args = dict(\n",
        "        fn=transfer_input, inputs=[user_input], outputs=[user_question, user_input, submitBtn], show_progress=True\n",
        "    )\n",
        "\n",
        "\n",
        "\n",
        "    predict_event1 = user_input.submit(**transfer_input_args).then(**predict_args)\n",
        "    predict_event2 = submitBtn.click(**transfer_input_args).then(**predict_args)\n",
        "\n",
        "    gr.Markdown(\"<h2>버거킹 chat 시연 리스트</h2>\")\n",
        "    gr.Examples(\n",
        "        examples=[\n",
        "            \"스모키 바비큐 X의 독특한 특징은 무엇인가요?\",\n",
        "            \"버거킹에서 판매하는 스프라이트 제로의 특징은 무엇인가요?\",\n",
        "            \"롱치킨버거에는 어떤 소스와 야채가 사용되었나요?\"\n",
        "                  ],\n",
        "        inputs=user_input\n",
        "    )\n",
        "\n",
        "\n",
        "    emptyBtn.click(\n",
        "        reset_state,\n",
        "        outputs=[chatbot, history, status_display],\n",
        "        show_progress=True,\n",
        "    )\n",
        "    emptyBtn.click(**reset_args)\n",
        "\n",
        "\n",
        "demo.queue(concurrency_count=1).launch()"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0d5424dba59845f6a1fc503794decf1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0de107edb99d48dd8f6211bef888708a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "162e27edc0854ac28377669ae89a5504": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_849637e9357040dfaef6c5a1ed4c22fe",
            "placeholder": "​",
            "style": "IPY_MODEL_0de107edb99d48dd8f6211bef888708a",
            "value": " 81/81 [00:00&lt;00:00, 2921.65 examples/s]"
          }
        },
        "180200658f4246fe9cec26585751d447": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_19ece5bb8a9d4260a994d37577fdd873",
            "placeholder": "​",
            "style": "IPY_MODEL_c5cb50e148fd49bfa58c790d0b51feb9",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "18de8b4c297942b9bae68c5b879b944a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "19ece5bb8a9d4260a994d37577fdd873": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2368be330217479980bbf7f3d81cfaf5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "578ec5a307444645a87aa95c3a659fe6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b43f664d6cc4e468af83033761b7ed3",
            "max": 97,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0d5424dba59845f6a1fc503794decf1a",
            "value": 97
          }
        },
        "59cc670c6690466db945f5e87851a71c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5fb07ff1a602452ba5b8d3f16b64b075": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59cc670c6690466db945f5e87851a71c",
            "max": 15,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bf40c379c91a4f2086a6aed3990e7814",
            "value": 15
          }
        },
        "667b80adfff647e1903aaf06d632a653": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6972dfe325fe4503aeef774b52d7df6b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6b05461b75fc49c0a192c9102c0bbf5d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6b43f664d6cc4e468af83033761b7ed3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6f4bbdb8a57e4d6d8c660867a47777a3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d2d673e40e0d4ad3ababde7c2953cb5c",
              "IPY_MODEL_afa4aff51df44a09a323c0ca3d31839e",
              "IPY_MODEL_162e27edc0854ac28377669ae89a5504"
            ],
            "layout": "IPY_MODEL_97eb2c1611034f468dc733367c375256"
          }
        },
        "73317d860aee479dbc3c196edab8a9e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_baeaeac4c780493fb8b5b701454df008",
            "placeholder": "​",
            "style": "IPY_MODEL_9e864f89e1a346b8909b199d6cdb9ea3",
            "value": " 15/15 [01:11&lt;00:00,  4.00s/it]"
          }
        },
        "76a0b81f731b40de898828616efc8768": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "849637e9357040dfaef6c5a1ed4c22fe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "888756553d154e4faffaf2b03a0d66a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e13ed38e533e4bad996252e19cd32000",
              "IPY_MODEL_578ec5a307444645a87aa95c3a659fe6",
              "IPY_MODEL_f7027e18b6c64490a0ffd66e29eaaad1"
            ],
            "layout": "IPY_MODEL_2368be330217479980bbf7f3d81cfaf5"
          }
        },
        "97eb2c1611034f468dc733367c375256": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e864f89e1a346b8909b199d6cdb9ea3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a48f8add4b0a41f3baab01d19795c249": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "afa4aff51df44a09a323c0ca3d31839e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_18de8b4c297942b9bae68c5b879b944a",
            "max": 81,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6b05461b75fc49c0a192c9102c0bbf5d",
            "value": 81
          }
        },
        "b0b8d246d4f74b599a1c35647697d62a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b62c4812975b4285b612c6edccfbfc31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "baeaeac4c780493fb8b5b701454df008": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be4d68bd90d542b089b017f9632e6438": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_180200658f4246fe9cec26585751d447",
              "IPY_MODEL_5fb07ff1a602452ba5b8d3f16b64b075",
              "IPY_MODEL_73317d860aee479dbc3c196edab8a9e2"
            ],
            "layout": "IPY_MODEL_6972dfe325fe4503aeef774b52d7df6b"
          }
        },
        "bf40c379c91a4f2086a6aed3990e7814": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c5cb50e148fd49bfa58c790d0b51feb9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d2d673e40e0d4ad3ababde7c2953cb5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_667b80adfff647e1903aaf06d632a653",
            "placeholder": "​",
            "style": "IPY_MODEL_b62c4812975b4285b612c6edccfbfc31",
            "value": "Saving the dataset (1/1 shards): 100%"
          }
        },
        "e13ed38e533e4bad996252e19cd32000": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7a6748ee5a6421fa2f51df72d6c0ecb",
            "placeholder": "​",
            "style": "IPY_MODEL_76a0b81f731b40de898828616efc8768",
            "value": "Saving the dataset (1/1 shards): 100%"
          }
        },
        "f7027e18b6c64490a0ffd66e29eaaad1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b0b8d246d4f74b599a1c35647697d62a",
            "placeholder": "​",
            "style": "IPY_MODEL_a48f8add4b0a41f3baab01d19795c249",
            "value": " 97/97 [00:00&lt;00:00, 3095.55 examples/s]"
          }
        },
        "f7a6748ee5a6421fa2f51df72d6c0ecb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}