01_intro/Homework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Homework: Introduction\n",
    "\n",
    "In this homework, we'll learn more about search and use Elastic Search for practice.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "source": [
    "### Q1. Running Elastic\n",
    "\n",
    "Run Elastic Search 8.4.3, and get the cluster information. If you run it on localhost, this is how you do it:\n",
    "\n",
    "``` curl localhost:9200 ```\n",
    "\n",
    "What's the ```version.build_hash``` value?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"name\" : \"ec1bcb4f6551\",\n",
      "  \"cluster_name\" : \"docker-cluster\",\n",
      "  \"cluster_uuid\" : \"ZxzLghYcRfK4R7r61bmjIA\",\n",
      "  \"version\" : {\n",
      "    \"number\" : \"8.4.3\",\n",
      "    \"build_flavor\" : \"default\",\n",
      "    \"build_type\" : \"docker\",\n",
      "    \"build_hash\" : \"42f05b9372a9a4a470db3b52817899b99a76ee73\",\n",
      "    \"build_date\" : \"2022-10-04T07:17:24.662462378Z\",\n",
      "    \"build_snapshot\" : false,\n",
      "    \"lucene_version\" : \"9.3.0\",\n",
      "    \"minimum_wire_compatibility_version\" : \"7.17.0\",\n",
      "    \"minimum_index_compatibility_version\" : \"7.0.0\"\n",
      "  },\n",
      "  \"tagline\" : \"You Know, for Search\"\n",
      "}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "\n",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n",
      "100   539  100   539    0     0   5861      0 --:--:-- --:--:-- --:--:--  5923\n"
     ]
    }
   ],
   "source": [
    "!curl localhost:9200 #\"42f05b9372a9a4a470db3b52817899b99a76ee73\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "source": [
    "### Getting the data\n",
    "Now let's get the FAQ data. You can run this snippet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests \n",
    "\n",
    "docs_url = 'https://github.com/DataTalksClub/llm-zoomcamp/blob/main/01-intro/documents.json?raw=1'\n",
    "docs_response = requests.get(docs_url)\n",
    "documents_raw = docs_response.json()\n",
    "\n",
    "documents = []\n",
    "\n",
    "for course in documents_raw:\n",
    "    course_name = course['course']\n",
    "\n",
    "    for doc in course['documents']:\n",
    "        doc['course'] = course_name\n",
    "        documents.append(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q2. Indexing the data\n",
    "\n",
    "Index the data in the same way as was shown in the course videos. Make the course field a keyword and the rest should be text.\n",
    "\n",
    "Don't forget to install the ElasticSearch client for Python:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``` pip install elasticsearch ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which function do you use for adding your data to elastic?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from elasticsearch import Elasticsearch\n",
    "\n",
    "es_client = Elasticsearch('http://localhost:9200') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectApiResponse({'acknowledged': True, 'shards_acknowledged': True, 'index': 'course-questions'})"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index_settings = {\n",
    "    \"settings\": {\n",
    "        \"number_of_shards\": 1,\n",
    "        \"number_of_replicas\": 0\n",
    "    },\n",
    "    \"mappings\": {\n",
    "        \"properties\": {\n",
    "            \"text\": {\"type\": \"text\"},\n",
    "            \"section\": {\"type\": \"text\"},\n",
    "            \"question\": {\"type\": \"text\"},\n",
    "            \"course\": {\"type\": \"keyword\"} \n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "index_name = \"course-questions\"\n",
    "\n",
    "es_client.indices.create(index=index_name, body=index_settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q3. Searching\n",
    "\n",
    "Now let's search in our index.\n",
    "\n",
    "We will execute a query \"How do I execute a command in a running docker container?\".\n",
    "\n",
    "Use only question and text fields and give question a boost of 4, and use \"type\": \"best_fields\".\n",
    "\n",
    "What's the score for the top ranking result?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\daniel\\Documents\\Universidad\\Cursos\\LLMZoomCamp\\.venv\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from tqdm.auto import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/948 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 948/948 [01:00<00:00, 15.69it/s]\n"
     ]
    }
   ],
   "source": [
    "for doc in tqdm(documents):\n",
    "    es_client.index(index=index_name, document=doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"How do I execute a command in a running docker container?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "search_query = {\n",
    "    \"size\": 5,\n",
    "    \"query\": {\n",
    "        \"bool\": {\n",
    "            \"must\": {\n",
    "                \"multi_match\": {\n",
    "                    \"query\": query,\n",
    "                    \"fields\": [\"question^4\", \"text\"],\n",
    "                    \"type\": \"best_fields\"\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "response = es_client.search(index=index_name, body=search_query)\n",
    "\n",
    "result_docs = []\n",
    "\n",
    "for hit in response['hits']['hits']:\n",
    "    result_docs.append(hit['_source'])\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_index': 'course-questions',\n",
       " '_id': 'wv78RpABYgsTHHJGNtOa',\n",
       " '_score': 84.050095,\n",
       " '_source': {'text': 'Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\\ndocker run -it --entrypoint bash <image>\\nIf the container is already running, execute a command in the specific container:\\ndocker ps (find the container-id)\\ndocker exec -it <container-id> bash\\n(Marcos MJD)',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I debug a docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'}}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response['hits']['hits'][0] #84.050095"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q4. Filtering\n",
    "\n",
    "Now let's only limit the questions to machine-learning-zoomcamp.\n",
    "\n",
    "Return 3 results. What's the 3rd question returned by the search engine?\n",
    "\n",
    "- How do I debug a docker container?\n",
    "- How do I copy files from a different folder into docker container’s working directory?\n",
    "- How do Lambda container images work?\n",
    "- How can I annotate a graph?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def elastic_search(query):\n",
    "    search_query = {\n",
    "        \"size\": 3,\n",
    "        \"query\": {\n",
    "            \"bool\": {\n",
    "                \"must\": {\n",
    "                    \"multi_match\": {\n",
    "                        \"query\": query,\n",
    "                        \"fields\": [\"question^4\", \"text\", \"section\"],\n",
    "                        \"type\": \"best_fields\"\n",
    "                    }\n",
    "                },\n",
    "                \"filter\": {\n",
    "                    \"term\": {\n",
    "                        \"course\": \"machine-learning-zoomcamp\"\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "\n",
    "    response = es_client.search(index=index_name, body=search_query)\n",
    "    \n",
    "    result_docs = []\n",
    "    \n",
    "    for hit in response['hits']['hits']:\n",
    "        result_docs.append(hit['_source'])\n",
    "    \n",
    "    return result_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'text': 'Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\\ndocker run -it --entrypoint bash <image>\\nIf the container is already running, execute a command in the specific container:\\ndocker ps (find the container-id)\\ndocker exec -it <container-id> bash\\n(Marcos MJD)',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I debug a docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': \"You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\\nTo copy a file or directory from your local machine into a running Docker container, you can use the `docker cp command`. The basic syntax is as follows:\\ndocker cp /path/to/local/file_or_directory container_id:/path/in/container\\nHrithik Kumar Advani\",\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from my local machine to docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': 'You can copy files from your local machine into a Docker container using the docker cp command. Here\\'s how to do it:\\nIn the Dockerfile, you can provide the folder containing the files that you want to copy over. The basic syntax is as follows:\\nCOPY [\"src/predict.py\", \"models/xgb_model.bin\", \"./\"]\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tGopakumar Gopinathan',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from a different folder into docker container’s working directory?',\n",
       "  'course': 'machine-learning-zoomcamp'}]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elastic_search(query) #'How do I copy files from a different folder into docker container’s working directory?'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q5. Building a prompt\n",
    "\n",
    "Now we're ready to build a prompt to send to an LLM.\n",
    "\n",
    "Take the records returned from Elasticsearch in Q4 and use this template to build the context. Separate context entries by two linebreaks (\\n\\n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = elastic_search(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_prompt(query, search_results):\n",
    "    prompt_template = \"\"\"\n",
    "You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
    "Use only the facts from the CONTEXT when answering the QUESTION.\n",
    "\n",
    "QUESTION: {question}\n",
    "\n",
    "CONTEXT:\n",
    "{context}\n",
    "\"\"\".strip()\n",
    "\n",
    "    context_entries = []\n",
    "    for doc in search_results:\n",
    "        context_entry = f\"section: {doc['section']}\\n\\nquestion: {doc['question']}\\n\\nanswer: {doc['text']}\\n\"\n",
    "        context_entries.append(context_entry)\n",
    "    \n",
    "    context = \"\\n\\n\".join(context_entries)\n",
    "\n",
    "    prompt = prompt_template.format(question=query, context=context).strip()\n",
    "    \n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1644"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(build_prompt(\"How do I execute a command in a running docker container?\", results))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q6. Tokens\n",
    "\n",
    "When we use the OpenAI Platform, we're charged by the number of tokens we send in our prompt and receive in the response.\n",
    "\n",
    "The OpenAI python package uses tiktoken for tokenization:\n",
    "\n",
    "``` pip install tiktoken ```\n",
    "\n",
    "Let's calculate the number of tokens in our query:\n",
    "\n",
    "``` encoding = tiktoken.encoding_for_model(\"gpt-4o\") ```\n",
    "\n",
    "Use the encode function. How many tokens does our prompt have?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tiktoken\n",
      "  Downloading tiktoken-0.7.0-cp310-cp310-win_amd64.whl.metadata (6.8 kB)\n",
      "Collecting regex>=2022.1.18 (from tiktoken)\n",
      "  Downloading regex-2024.5.15-cp310-cp310-win_amd64.whl.metadata (41 kB)\n",
      "     ---------------------------------------- 0.0/42.0 kB ? eta -:--:--\n",
      "     ----------------------------- ---------- 30.7/42.0 kB 1.3 MB/s eta 0:00:01\n",
      "     -------------------------------------- 42.0/42.0 kB 503.7 kB/s eta 0:00:00\n",
      "Requirement already satisfied: requests>=2.26.0 in c:\\users\\daniel\\documents\\universidad\\cursos\\llmzoomcamp\\.venv\\lib\\site-packages (from tiktoken) (2.32.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\daniel\\documents\\universidad\\cursos\\llmzoomcamp\\.venv\\lib\\site-packages (from requests>=2.26.0->tiktoken) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\daniel\\documents\\universidad\\cursos\\llmzoomcamp\\.venv\\lib\\site-packages (from requests>=2.26.0->tiktoken) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\daniel\\documents\\universidad\\cursos\\llmzoomcamp\\.venv\\lib\\site-packages (from requests>=2.26.0->tiktoken) (2.2.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\daniel\\documents\\universidad\\cursos\\llmzoomcamp\\.venv\\lib\\site-packages (from requests>=2.26.0->tiktoken) (2024.6.2)\n",
      "Downloading tiktoken-0.7.0-cp310-cp310-win_amd64.whl (798 kB)\n",
      "   ---------------------------------------- 0.0/798.9 kB ? eta -:--:--\n",
      "   --- ------------------------------------ 61.4/798.9 kB 3.2 MB/s eta 0:00:01\n",
      "   ------ --------------------------------- 122.9/798.9 kB 1.8 MB/s eta 0:00:01\n",
      "   ------------ --------------------------- 245.8/798.9 kB 1.9 MB/s eta 0:00:01\n",
      "   ------------------------ --------------- 491.5/798.9 kB 2.6 MB/s eta 0:00:01\n",
      "   ---------------------------------- ----- 696.3/798.9 kB 3.1 MB/s eta 0:00:01\n",
      "   ---------------------------------------- 798.9/798.9 kB 3.0 MB/s eta 0:00:00\n",
      "Downloading regex-2024.5.15-cp310-cp310-win_amd64.whl (268 kB)\n",
      "   ---------------------------------------- 0.0/269.0 kB ? eta -:--:--\n",
      "   ---------------------------------------  266.2/269.0 kB 5.6 MB/s eta 0:00:01\n",
      "   ---------------------------------------- 269.0/269.0 kB 4.2 MB/s eta 0:00:00\n",
      "Installing collected packages: regex, tiktoken\n",
      "Successfully installed regex-2024.5.15 tiktoken-0.7.0\n"
     ]
    }
   ],
   "source": [
    "!pip install tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "355"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(encoding.encode(build_prompt(\"How do I execute a command in a running docker container?\", results)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b\"You're\""
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encoding.decode_single_token_bytes(63842)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}