01-intro/homework_wk1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3dc90173-3453-43d4-836d-211c47cf94fd",
   "metadata": {},
   "source": [
    "# Homework Week 1\n",
    "\n",
    "https://github.com/DataTalksClub/llm-zoomcamp/blob/main/cohorts/2024/01-intro/homework.md"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee510019-e773-4263-b96a-b311e15633ef",
   "metadata": {},
   "source": [
    "## Q1. Running Elastic \n",
    "\n",
    "Run Elastic Search 8.4.3, and get the cluster information. If you run it on localhost, this is how you do it:\n",
    "\n",
    "```bash\n",
    "curl localhost:9200\n",
    "```\n",
    "\n",
    "What's the `version.build_hash` value?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a10efb3-3c88-469b-8542-3b7e0f4e9084",
   "metadata": {},
   "source": [
    "Answer: 42f05b9372a9a4a470db3b52817899b99a76ee73"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fd2ea90-1ef0-4f08-a752-e6e1293ce944",
   "metadata": {},
   "source": [
    "## Getting the data\n",
    "\n",
    "Now let's get the FAQ data. You can run this snippet:\n",
    "\n",
    "Note that you need to have the `requests` library:\n",
    "\n",
    "```bash\n",
    "pip install requests\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "fac68366-0926-4a83-90cc-ac57f0b9de68",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests \n",
    "\n",
    "docs_url = 'https://github.com/DataTalksClub/llm-zoomcamp/blob/main/01-intro/documents.json?raw=1'\n",
    "docs_response = requests.get(docs_url)\n",
    "documents_raw = docs_response.json()\n",
    "\n",
    "documents = []\n",
    "\n",
    "for course in documents_raw:\n",
    "    course_name = course['course']\n",
    "\n",
    "    for doc in course['documents']:\n",
    "        doc['course'] = course_name\n",
    "        documents.append(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8702b9f7-09b4-499d-bfef-64a809f0f55d",
   "metadata": {},
   "source": [
    "## Q2. Indexing the data\n",
    "\n",
    "Index the data in the same way as was shown in the course videos. Make the `course` field a keyword and the rest should be text. \n",
    "\n",
    "Don't forget to install the ElasticSearch client for Python:\n",
    "\n",
    "```bash\n",
    "pip install elasticsearch\n",
    "```\n",
    "\n",
    "Which function do you use for adding your data to elastic?\n",
    "\n",
    "* `insert`\n",
    "* `index`\n",
    "* `put`\n",
    "* `add`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5e328c6-24a7-4e02-a5f9-a97e21128920",
   "metadata": {},
   "source": [
    "Answer: `index`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1127e14c-7945-4dde-b2cc-b8adef7fb3df",
   "metadata": {},
   "source": [
    "## Q3. Searching\n",
    "\n",
    "Now let's search in our index. \n",
    "\n",
    "We will execute a query \"How do I execute a command in a running docker container?\". \n",
    "\n",
    "Use only `question` and `text` fields and give `question` a boost of 4, and use `\"type\": \"best_fields\"`.\n",
    "\n",
    "What's the score for the top ranking result?\n",
    "\n",
    "* 94.05\n",
    "* 84.05\n",
    "* 74.05\n",
    "* 64.05\n",
    "\n",
    "Look at the `_score` field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c5c5c6ad-d60e-427a-b6a4-0c04545c804a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from elasticsearch import Elasticsearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a59c835a-d2f2-4d3c-94e4-035a06fc25dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "es_client = Elasticsearch(\"http://localhost:9200\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ff070ef4-2782-4789-b764-9fa002edecbf",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_settings = {\n",
    "    \"settings\": {\n",
    "        \"number_of_shards\": 1,\n",
    "        \"number_of_replicas\": 0\n",
    "    },\n",
    "    \"mappings\": {\n",
    "        \"properties\": {\n",
    "            \"text\": {\"type\": \"text\"},\n",
    "            \"section\": {\"type\": \"text\"},\n",
    "            \"question\": {\"type\": \"text\"},\n",
    "            \"course\": {\"type\": \"keyword\"} \n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "index_name = \"homework-questions\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "4923e67c-6498-4384-8857-11713e5be0de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index created.\n"
     ]
    }
   ],
   "source": [
    "# Delete the index if it exists\n",
    "# es_client.indices.delete(index=index_name)\n",
    "\n",
    "# Now, create the index anew\n",
    "es_client.indices.create(index=index_name, body=index_settings)\n",
    "print(\"Index created.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d045934c-2587-4c26-9a5f-bc85bca3714b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/python/3.10.13/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████| 948/948 [00:20<00:00, 45.22it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm.auto import tqdm\n",
    "for doc in tqdm(documents):\n",
    "    es_client.index(index=index_name, document=doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "d1092fae-4fa8-4666-8650-8270ebcb7586",
   "metadata": {},
   "outputs": [],
   "source": [
    "def elastic_search_score(query, index_name):\n",
    "\n",
    "    search_query = {\n",
    "        \"size\": 5,\n",
    "        \"query\": {\n",
    "            \"bool\": {\n",
    "                \"must\": [\n",
    "                    {\n",
    "                        \"multi_match\": {\n",
    "                            \"query\": query,\n",
    "                            \"fields\": [\"question^4\", \"text\"],\n",
    "                            \"type\": \"best_fields\"\n",
    "                        }\n",
    "                    }\n",
    "                ]\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    response = es_client.search(index=index_name, body=search_query)\n",
    "\n",
    "    result_docs = {\n",
    "        'result': [],\n",
    "        'score': []\n",
    "    }\n",
    "    \n",
    "    for hit in response['hits']['hits']:\n",
    "        result_docs['result'].append(hit['_source'])\n",
    "        result_docs['score'].append(hit['_score'])\n",
    "\n",
    "    return result_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b1f09e74-8170-4820-8eb6-0558bcf19712",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example of calling the function\n",
    "query = 'How do I execute a command in a running docker container?'\n",
    "response = elastic_search_score(query, index_name)\n",
    "\n",
    "top_score = response['score'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "42a7ce5b-bb4b-4491-9fb6-982d8b72ca48",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: Top record's score: 84.050095\n"
     ]
    }
   ],
   "source": [
    "print(f\"Answer: Top record's score: {top_score}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c60929ce-1643-4f92-9c9d-098d437dcd33",
   "metadata": {},
   "source": [
    "## Q4. Filtering\n",
    "\n",
    "Now let's only limit the questions to `machine-learning-zoomcamp`.\n",
    "\n",
    "Return 3 results. What's the 3rd question returned by the search engine?\n",
    "\n",
    "* How do I debug a docker container?\n",
    "* How do I copy files from a different folder into docker container’s working directory?\n",
    "* How do Lambda container images work?\n",
    "* How can I annotate a graph?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "id": "4390ff7e-b660-4e62-bfbe-fb2d7e725c36",
   "metadata": {},
   "outputs": [],
   "source": [
    "def elastic_search_top3_ml(query, index_name=\"homework-questions\"):\n",
    "\n",
    "    search_query = {\n",
    "        \"size\": 5,\n",
    "        \"query\": {\n",
    "            \"bool\": {\n",
    "                \"must\": [\n",
    "                    {\n",
    "                        \"multi_match\": {\n",
    "                            \"query\": query,\n",
    "                            \"fields\": [\"question^4\", \"text\"],\n",
    "                            \"type\": \"best_fields\"\n",
    "                        }\n",
    "                    }\n",
    "                ],\n",
    "                \"filter\": [\n",
    "                    {\n",
    "                        \"term\": {\n",
    "                            \"course\": \"machine-learning-zoomcamp\"\n",
    "                        }\n",
    "                    }\n",
    "                ]\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    \n",
    "    response = es_client.search(index=index_name, body=search_query)\n",
    "\n",
    "    result_docs = {\n",
    "        'result': [],\n",
    "        'score': [],\n",
    "        'question': []\n",
    "    }\n",
    "\n",
    "    for hit in response['hits']['hits'][:3]:\n",
    "        result_docs['result'].append(hit['_source'])  # Storing the entire source document\n",
    "        result_docs['score'].append(hit['_score'])    # Storing the score\n",
    "        result_docs['question'].append(hit['_source'].get('question', 'No question available'))  # Safely accessing the question field\n",
    "\n",
    "    return result_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "id": "000a66cb-8c76-4e02-b7e5-765751459b5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'How do I execute a command in a running docker container?'\n",
    "q4_response = elastic_search_top3_ml(query)\n",
    "\n",
    "top3_result = q4_response['question'][2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "id": "3fab5034-2659-4601-a4b8-f43eaf3a038e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: How do I copy files from a different folder into docker container’s working directory?\n"
     ]
    }
   ],
   "source": [
    "print(f\"Answer: {top3_result}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "b6a60409-28b4-4097-beff-e045926fb19d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'text': 'Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\\ndocker run -it --entrypoint bash <image>\\nIf the container is already running, execute a command in the specific container:\\ndocker ps (find the container-id)\\ndocker exec -it <container-id> bash\\n(Marcos MJD)',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I debug a docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': \"You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\\nTo copy a file or directory from your local machine into a running Docker container, you can use the `docker cp command`. The basic syntax is as follows:\\ndocker cp /path/to/local/file_or_directory container_id:/path/in/container\\nHrithik Kumar Advani\",\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from my local machine to docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': 'You can copy files from your local machine into a Docker container using the docker cp command. Here\\'s how to do it:\\nIn the Dockerfile, you can provide the folder containing the files that you want to copy over. The basic syntax is as follows:\\nCOPY [\"src/predict.py\", \"models/xgb_model.bin\", \"./\"]\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tGopakumar Gopinathan',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from a different folder into docker container’s working directory?',\n",
       "  'course': 'machine-learning-zoomcamp'}]"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "q4_response['result']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74c4e4d5-6632-4886-8f1d-75450be587e4",
   "metadata": {},
   "source": [
    "## Q5. Building a prompt\n",
    "\n",
    "Now we're ready to build a prompt to send to an LLM. \n",
    "\n",
    "Take the records returned from Elasticsearch in Q4 and use this template to build the context. Separate context entries by two linebreaks (`\\n\\n`)\n",
    "\n",
    "```python\n",
    "context_template = \"\"\"\n",
    "Q: {question}\n",
    "A: {text}\n",
    "\"\"\".strip()\n",
    "```\n",
    "\n",
    "Now use the context you just created along with the \"How do I execute a command in a running docker container?\" question \n",
    "to construct a prompt using the template below:\n",
    "\n",
    "```python\n",
    "prompt_template = \"\"\"\n",
    "You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
    "Use only the facts from the CONTEXT when answering the QUESTION.\n",
    "\n",
    "QUESTION: {question}\n",
    "\n",
    "CONTEXT:\n",
    "{context}\n",
    "\"\"\".strip()\n",
    "```\n",
    "\n",
    "What's the length of the resulting prompt? (use the `len` function)\n",
    "\n",
    "* 962\n",
    "* 1462\n",
    "* 1962\n",
    "* 2462\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "id": "887ee8c0-aa13-43bb-ace1-aaf55a079408",
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_prompt(query, search_results):\n",
    "    \n",
    "    prompt_template = \"\"\"\n",
    "You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
    "Use only the facts from the CONTEXT when answering the QUESTION.\n",
    "\n",
    "QUESTION: {question}\n",
    "\n",
    "CONTEXT:\n",
    "{context}\n",
    "\"\"\".strip()\n",
    "    \n",
    "    context_template = \"\"\"\n",
    "Q: {question}\n",
    "A: {text}\n",
    "\"\"\".strip()\n",
    "\n",
    "    context = \"\"\n",
    "\n",
    "    for doc in search_results['result']:\n",
    "        # Safely get 'question' and 'text' from the document\n",
    "        # question = doc.get('question', 'No question available')\n",
    "        # text = doc.get('text', 'No text available')\n",
    "        question = doc['question']\n",
    "        text = doc['text']\n",
    "        \n",
    "        # Format the context string\n",
    "        context = context + f\"{context_template.format(question=question, text=text).strip()}\\n\\n\"\n",
    "\n",
    "    prompt =  prompt_template.format(question=query, context=context).strip()\n",
    "\n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "id": "0117e21f-6850-4688-9ace-8daff962b48d",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = build_prompt(query, q4_response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "id": "7241a62b-bada-4255-be11-5d13123b01e9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: Top record's score: 1462\n"
     ]
    }
   ],
   "source": [
    "print(f\"Answer: Top record's score: {len(prompt)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a54ca0b-f6a8-4dbc-aa3d-1516cb01b8aa",
   "metadata": {},
   "source": [
    "## Q6. Tokens\n",
    "\n",
    "When we use the OpenAI Platform, we're charged by the number of \n",
    "tokens we send in our prompt and receive in the response.\n",
    "\n",
    "The OpenAI python package uses `tiktoken` for tokenization:\n",
    "\n",
    "```bash\n",
    "pip install tiktoken\n",
    "```\n",
    "\n",
    "Let's calculate the number of tokens in our query: \n",
    "\n",
    "```python\n",
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")\n",
    "```\n",
    "\n",
    "Use the `encode` function. How many tokens does our prompt have?\n",
    "\n",
    "* 122\n",
    "* 222\n",
    "* 322\n",
    "* 422\n",
    "\n",
    "Note: to decode back a token into a word, you can use the `decode_single_token_bytes` function:\n",
    "\n",
    "```python\n",
    "encoding.decode_single_token_bytes(63842)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "78b718d3-9594-4625-ad6a-3d17935fe8ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "d6aaf712-1147-4ed4-a847-eec6725a4562",
   "metadata": {},
   "outputs": [],
   "source": [
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "fab3a4b9-815b-43a1-b5d3-f7b9bad92229",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "322"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Tokenize the text using the optimized tokenizer\n",
    "tokens = encoding.encode(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "11a70396-f79f-470c-9f02-da8dcec5f59d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: How many tokens does our prompt have? -> 322\n"
     ]
    }
   ],
   "source": [
    "print(f\"Answer: How many tokens does our prompt have? -> {len(tokens)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a59a747-6c45-480f-9cd5-e497642ed702",
   "metadata": {},
   "outputs": [],
   "source": [
    "decoded_tokens = encoding.decode(tokens)\n",
    "decoded_tokens_safe = [encoding.decode_single_token_bytes(token) for token in tokens]\n",
    "\n",
    "# print(decoded_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f518b59-53b0-4ee5-9996-7b87fa20221c",
   "metadata": {},
   "source": [
    "## Bonus: generating the answer (ungraded)\n",
    "\n",
    "Let's send the prompt to OpenAI. What's the response?  \n",
    "\n",
    "Note: you can replace OpenAI with Ollama. See module 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "id": "c85e6ad4-6574-40f8-bc4c-bea0cc68ffbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "def llm(prompt):\n",
    "    response = client.chat.completions.create(\n",
    "    model='gpt-4o',\n",
    "    messages=[{\"role\": \"user\", \"content\": prompt}]\n",
    "    )\n",
    "\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "id": "630357c3-db31-49c6-834e-c06d9208a65e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def rag(query):\n",
    "    search_result = elastic_search_top3_ml(query, index_name=\"homework-questions\")\n",
    "    prompt = build_prompt(query, search_result)\n",
    "    answer = llm(prompt)\n",
    "\n",
    "    return answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "id": "4d3bcd4b-3502-4034-b9c0-93c89237e1ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "id": "0bcc61dc-4de8-46ad-ac86-e3a3c426efa5",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "id": "10952c67-12c6-4a9b-a1ad-f1df48613b73",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To execute a command in a running Docker container, you would use the `docker exec` command. First, you need to find the container ID of the running Docker container. Here are the steps you should follow:\n",
      "\n",
      "1. List all running containers to find the container ID using the command:\n",
      "   ```sh\n",
      "   docker ps\n",
      "   ```\n",
      "\n",
      "2. Once you have the container ID, you can execute a command inside the running container. For example, to start a bash session, you would use:\n",
      "   ```sh\n",
      "   docker exec -it <container-id> bash\n",
      "   ```\n",
      "\n",
      "Replace `<container-id>` with the actual ID of the container you want to interact with.\n"
     ]
    }
   ],
   "source": [
    "query = 'How do I execute a command in a running docker container?'\n",
    "\n",
    "print(rag(query))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31d1d1a0-4373-4dd6-ad09-ba1d5c0cee05",
   "metadata": {},
   "source": [
    "## Bonus: calculating the costs (ungraded)\n",
    "\n",
    "Suppose that on average per request we send 150 tokens and receive back 250 tokens.\n",
    "\n",
    "How much will it cost to run 1000 requests?\n",
    "\n",
    "You can see the prices [here](https://openai.com/api/pricing/)\n",
    "\n",
    "On June 17, the prices for gpt4o are:\n",
    "\n",
    "* Input: \\$0.005 / 1K tokens\n",
    "* Output: \\$0.015 / 1K tokens\n",
    "\n",
    "You can redo the calculations with the values you got in Q6 and Q7."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}