01-intro/homework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Homework: Introduction\n",
    "\n",
    "In this homework, we'll learn more about search and use Elastic Search for practice. \n",
    "\n",
    "> It's possible that your answers won't match exactly. If it's the case, select the closest one.\n",
    "\n",
    "## Q1. Running Elastic \n",
    "\n",
    "Run Elastic Search 8.4.3, and get the cluster information. If you run it on localhost, this is how you do it:\n",
    "\n",
    "```bash\n",
    "curl localhost:9200\n",
    "```\n",
    "\n",
    "What's the `version.build_hash` value?\n",
    "\n",
    "\n",
    "## Getting the data\n",
    "\n",
    "Now let's get the FAQ data. You can run this snippet:\n",
    "\n",
    "```python\n",
    "import requests \n",
    "\n",
    "docs_url = 'https://github.com/DataTalksClub/llm-zoomcamp/blob/main/01-intro/documents.json?raw=1'\n",
    "docs_response = requests.get(docs_url)\n",
    "documents_raw = docs_response.json()\n",
    "\n",
    "documents = []\n",
    "\n",
    "for course in documents_raw:\n",
    "    course_name = course['course']\n",
    "\n",
    "    for doc in course['documents']:\n",
    "        doc['course'] = course_name\n",
    "        documents.append(doc)\n",
    "```\n",
    "\n",
    "Note that you need to have the `requests` library:\n",
    "\n",
    "```bash\n",
    "pip install requests\n",
    "```\n",
    "\n",
    "## Q2. Indexing the data\n",
    "\n",
    "Index the data in the same way as was shown in the course videos. Make the `course` field a keyword and the rest should be text. \n",
    "\n",
    "Don't forget to install the ElasticSearch client for Python:\n",
    "\n",
    "```bash\n",
    "pip install elasticsearch\n",
    "```\n",
    "\n",
    "Which function do you use for adding your data to elastic?\n",
    "\n",
    "* `insert`\n",
    "* `index`\n",
    "* `put`\n",
    "* `add`\n",
    "\n",
    "## Q3. Searching\n",
    "\n",
    "Now let's search in our index. \n",
    "\n",
    "We will execute a query \"How do I execute a command in a running docker container?\". \n",
    "\n",
    "Use only `question` and `text` fields and give `question` a boost of 4, and use `\"type\": \"best_fields\"`.\n",
    "\n",
    "What's the score for the top ranking result?\n",
    "\n",
    "* 94.05\n",
    "* 84.05\n",
    "* 74.05\n",
    "* 64.05\n",
    "\n",
    "Look at the `_score` field.\n",
    "\n",
    "## Q4. Filtering\n",
    "\n",
    "Now let's only limit the questions to `machine-learning-zoomcamp`.\n",
    "\n",
    "Return 3 results. What's the 3rd question returned by the search engine?\n",
    "\n",
    "* How do I debug a docker container?\n",
    "* How do I copy files from a different folder into docker container’s working directory?\n",
    "* How do Lambda container images work?\n",
    "* How can I annotate a graph?\n",
    "\n",
    "## Q5. Building a prompt\n",
    "\n",
    "Now we're ready to build a prompt to send to an LLM. \n",
    "\n",
    "Take the records returned from Elasticsearch in Q4 and use this template to build the context. Separate context entries by two linebreaks (`\\n\\n`)\n",
    "```python\n",
    "context_template = \"\"\"\n",
    "Q: {question}\n",
    "A: {text}\n",
    "\"\"\".strip()\n",
    "```\n",
    "\n",
    "Now use the context you just created along with the \"How do I execute a command in a running docker container?\" question \n",
    "to construct a prompt using the template below:\n",
    "\n",
    "```\n",
    "prompt_template = \"\"\"\n",
    "You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
    "Use only the facts from the CONTEXT when answering the QUESTION.\n",
    "\n",
    "QUESTION: {question}\n",
    "\n",
    "CONTEXT:\n",
    "{context}\n",
    "\"\"\".strip()\n",
    "```\n",
    "\n",
    "What's the length of the resulting prompt? (use the `len` function)\n",
    "\n",
    "* 962\n",
    "* 1462\n",
    "* 1962\n",
    "* 2462\n",
    "\n",
    "## Q6. Tokens\n",
    "\n",
    "When we use the OpenAI Platform, we're charged by the number of \n",
    "tokens we send in our prompt and receive in the response.\n",
    "\n",
    "The OpenAI python package uses `tiktoken` for tokenization:\n",
    "\n",
    "```bash\n",
    "pip install tiktoken\n",
    "```\n",
    "\n",
    "Let's calculate the number of tokens in our query: \n",
    "\n",
    "```python\n",
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")\n",
    "```\n",
    "\n",
    "Use the `encode` function. How many tokens does our prompt have?\n",
    "\n",
    "* 122\n",
    "* 222\n",
    "* 322\n",
    "* 422\n",
    "\n",
    "Note: to decode back a token into a word, you can use the `decode_single_token_bytes` function:\n",
    "\n",
    "```python\n",
    "encoding.decode_single_token_bytes(63842)\n",
    "```\n",
    "\n",
    "## Bonus: generating the answer (ungraded)\n",
    "\n",
    "Let's send the prompt to OpenAI. What's the response?  \n",
    "\n",
    "Note: you can replace OpenAI with Ollama. See module 2.\n",
    "\n",
    "## Bonus: calculating the costs (ungraded)\n",
    "\n",
    "Suppose that on average per request we send 150 tokens and receive back 250 tokens.\n",
    "\n",
    "How much will it cost to run 1000 requests?\n",
    "\n",
    "You can see the prices [here](https://openai.com/api/pricing/)\n",
    "\n",
    "On June 17, the prices for gpt4o are:\n",
    "\n",
    "* Input: $0.005 / 1K tokens\n",
    "* Output: $0.015 / 1K tokens\n",
    "\n",
    "You can redo the calculations with the values you got in Q6 and Q7.\n",
    "\n",
    "\n",
    "## Submit the results\n",
    "\n",
    "* Submit your results here: https://courses.datatalks.club/llm-zoomcamp-2024/homework/hw1\n",
    "* It's possible that your answers won't match exactly. If it's the case, select the closest one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests \n",
    "from elasticsearch import Elasticsearch\n",
    "from openai import OpenAI\n",
    "from dotenv import load_dotenv\n",
    "import os\n",
    "import tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv()  # Load environment variables from .env file\n",
    "\n",
    "api_key = os.getenv(\"OPENAI_API_KEY\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "docs_url = 'https://github.com/DataTalksClub/llm-zoomcamp/blob/main/01-intro/documents.json?raw=1'\n",
    "docs_response = requests.get(docs_url)\n",
    "documents_raw = docs_response.json()\n",
    "\n",
    "documents = []\n",
    "\n",
    "for course in documents_raw:\n",
    "    course_name = course['course']\n",
    "\n",
    "    for doc in course['documents']:\n",
    "        doc['course'] = course_name\n",
    "        documents.append(doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'name': '9aa365363338', 'cluster_name': 'docker-cluster', 'cluster_uuid': 'ZHM8oNZPT8eDmS5Rk3NiaQ', 'version': {'number': '8.4.3', 'build_flavor': 'default', 'build_type': 'docker', 'build_hash': '42f05b9372a9a4a470db3b52817899b99a76ee73', 'build_date': '2022-10-04T07:17:24.662462378Z', 'build_snapshot': False, 'lucene_version': '9.3.0', 'minimum_wire_compatibility_version': '7.17.0', 'minimum_index_compatibility_version': '7.0.0'}, 'tagline': 'You Know, for Search'}\n",
      "\n",
      "Build Hash: 42f05b9372a9a4a470db3b52817899b99a76ee73\n"
     ]
    }
   ],
   "source": [
    "# Connect to Elasticsearch\n",
    "es = Elasticsearch(\"http://localhost:9200\")\n",
    "\n",
    "# Get cluster information\n",
    "info = es.info()\n",
    "\n",
    "# Print cluster information\n",
    "print(info)\n",
    "\n",
    "build_hash = info['version']['build_hash']\n",
    "print(\"\\nBuild Hash:\", build_hash)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectApiResponse({'acknowledged': True, 'shards_acknowledged': True, 'index': 'course-questions'})"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index_settings = {\n",
    "    \"settings\": {\n",
    "        \"number_of_shards\": 1,\n",
    "        \"number_of_replicas\": 0\n",
    "    },\n",
    "    \"mappings\": {\n",
    "        \"properties\": {\n",
    "            \"text\": {\"type\": \"text\"},\n",
    "            \"section\": {\"type\": \"text\"},\n",
    "            \"question\": {\"type\": \"text\"},\n",
    "            \"course\": {\"type\": \"keyword\"} \n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "index_name = \"course-questions\"\n",
    "\n",
    "es.indices.create(index=index_name, body=index_settings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'text': \"The purpose of this document is to capture frequently asked technical questions\\nThe exact day and hour of the course will be 15th Jan 2024 at 17h00. The course will start with the first  “Office Hours'' live.1\\nSubscribe to course public Google Calendar (it works from Desktop only).\\nRegister before the course starts using this link.\\nJoin the course Telegram channel with announcements.\\nDon’t forget to register in DataTalks.Club's Slack and join the channel.\",\n",
       " 'section': 'General course-related questions',\n",
       " 'question': 'Course - When will the course start?',\n",
       " 'course': 'data-engineering-zoomcamp'}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "for doc in documents:\n",
    "    es.index(index=index_name, document=doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"How do I execute a command in a running docker container?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def elastic_search(query):\n",
    "    search_query = {\n",
    "        \"size\": 5,\n",
    "        \"query\": {\n",
    "            \"bool\": {\n",
    "                \"must\": {\n",
    "                    \"multi_match\": {\n",
    "                        \"query\": query,\n",
    "                        \"fields\": [\"question^4\", \"text\"],\n",
    "                        \"type\": \"best_fields\"\n",
    "                    }\n",
    "                },\n",
    "                # \"filter\": {\n",
    "                #     \"term\": {\n",
    "                #         \"course\": \"data-engineering-zoomcamp\"\n",
    "                #     }\n",
    "                # }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "\n",
    "    response = es.search(index=index_name, body=search_query)\n",
    "    \n",
    "    result_docs = []\n",
    "    \n",
    "    for hit in response['hits']['hits']:\n",
    "        result_docs.append({\n",
    "            'document': hit['_source'],\n",
    "            'score': hit['_score']\n",
    "        })\n",
    "    \n",
    "    return result_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'document': {'text': 'Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\\ndocker run -it --entrypoint bash <image>\\nIf the container is already running, execute a command in the specific container:\\ndocker ps (find the container-id)\\ndocker exec -it <container-id> bash\\n(Marcos MJD)',\n",
       "   'section': '5. Deploying Machine Learning Models',\n",
       "   'question': 'How do I debug a docker container?',\n",
       "   'course': 'machine-learning-zoomcamp'},\n",
       "  'score': 84.050095},\n",
       " {'document': {'text': 'In case running pgcli  locally causes issues or you do not want to install it locally you can use it running in a Docker container instead.\\nBelow the usage with values used in the videos of the course for:\\nnetwork name (docker network)\\npostgres related variables for pgcli\\nHostname\\nUsername\\nPort\\nDatabase name\\n$ docker run -it --rm --network pg-network ai2ys/dockerized-pgcli:4.0.1\\n175dd47cda07:/# pgcli -h pg-database -U root -p 5432 -d ny_taxi\\nPassword for root:\\nServer: PostgreSQL 16.1 (Debian 16.1-1.pgdg120+1)\\nVersion: 4.0.1\\nHome: http://pgcli.com\\nroot@pg-database:ny_taxi> \\\\dt\\n+--------+------------------+-------+-------+\\n| Schema | Name             | Type  | Owner |\\n|--------+------------------+-------+-------|\\n| public | yellow_taxi_data | table | root  |\\n+--------+------------------+-------+-------+\\nSELECT 1\\nTime: 0.009s\\nroot@pg-database:ny_taxi>',\n",
       "   'section': 'Module 1: Docker and Terraform',\n",
       "   'question': 'PGCLI - running in a Docker container',\n",
       "   'course': 'data-engineering-zoomcamp'},\n",
       "  'score': 75.54128},\n",
       " {'document': {'text': 'If you are trying to run Flask gunicorn & MLFlow server from the same container, defining both in Dockerfile with CMD will only run MLFlow & not Flask.\\nSolution: Create separate shell script with server run commands, for eg:\\n> \\tscript1.sh\\n#!/bin/bash\\ngunicorn --bind=0.0.0.0:9696 predict:app\\nAnother script with e.g. MLFlow server:\\n>\\tscript2.sh\\n#!/bin/bash\\nmlflow server -h 0.0.0.0 -p 5000 --backend-store-uri=sqlite:///mlflow.db --default-artifact-root=g3://zc-bucket/mlruns/\\nCreate a wrapper script to run above 2 scripts:\\n>\\twrapper_script.sh\\n#!/bin/bash\\n# Start the first process\\n./script1.sh &\\n# Start the second process\\n./script2.sh &\\n# Wait for any process to exit\\nwait -n\\n# Exit with status of process that exited first\\nexit $?\\nGive executable permissions to all scripts:\\nchmod +x *.sh\\nNow we can define last line of Dockerfile as:\\n> \\tDockerfile\\nCMD ./wrapper_script.sh\\nDont forget to expose all ports defined by services!',\n",
       "   'section': 'Module 4: Deployment',\n",
       "   'question': 'Running multiple services in a Docker container',\n",
       "   'course': 'mlops-zoomcamp'},\n",
       "  'score': 72.08518},\n",
       " {'document': {'text': \"You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\\nTo copy a file or directory from your local machine into a running Docker container, you can use the `docker cp command`. The basic syntax is as follows:\\ndocker cp /path/to/local/file_or_directory container_id:/path/in/container\\nHrithik Kumar Advani\",\n",
       "   'section': '5. Deploying Machine Learning Models',\n",
       "   'question': 'How do I copy files from my local machine to docker container?',\n",
       "   'course': 'machine-learning-zoomcamp'},\n",
       "  'score': 51.04628},\n",
       " {'document': {'text': 'You can copy files from your local machine into a Docker container using the docker cp command. Here\\'s how to do it:\\nIn the Dockerfile, you can provide the folder containing the files that you want to copy over. The basic syntax is as follows:\\nCOPY [\"src/predict.py\", \"models/xgb_model.bin\", \"./\"]\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tGopakumar Gopinathan',\n",
       "   'section': '5. Deploying Machine Learning Models',\n",
       "   'question': 'How do I copy files from a different folder into docker container’s working directory?',\n",
       "   'course': 'machine-learning-zoomcamp'},\n",
       "  'score': 49.938507}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_results = elastic_search(query)\n",
    "search_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "84.050095"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_results[0]['score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def elastic_search(query):\n",
    "    search_query = {\n",
    "        \"size\": 3,\n",
    "        \"query\": {\n",
    "            \"bool\": {\n",
    "                \"must\": {\n",
    "                    \"multi_match\": {\n",
    "                        \"query\": query,\n",
    "                        \"fields\": [\"question^4\", \"text\"],\n",
    "                        \"type\": \"best_fields\"\n",
    "                    }\n",
    "                },\n",
    "                \"filter\": {\n",
    "                    \"term\": {\n",
    "                        \"course\": \"machine-learning-zoomcamp\"\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "\n",
    "    response = es.search(index=index_name, body=search_query)\n",
    "    \n",
    "    result_docs = []\n",
    "    \n",
    "    for hit in response['hits']['hits']:\n",
    "        result_docs.append(hit['_source'])\n",
    "    \n",
    "    return result_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'text': 'Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\\ndocker run -it --entrypoint bash <image>\\nIf the container is already running, execute a command in the specific container:\\ndocker ps (find the container-id)\\ndocker exec -it <container-id> bash\\n(Marcos MJD)',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I debug a docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': \"You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\\nTo copy a file or directory from your local machine into a running Docker container, you can use the `docker cp command`. The basic syntax is as follows:\\ndocker cp /path/to/local/file_or_directory container_id:/path/in/container\\nHrithik Kumar Advani\",\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from my local machine to docker container?',\n",
       "  'course': 'machine-learning-zoomcamp'},\n",
       " {'text': 'You can copy files from your local machine into a Docker container using the docker cp command. Here\\'s how to do it:\\nIn the Dockerfile, you can provide the folder containing the files that you want to copy over. The basic syntax is as follows:\\nCOPY [\"src/predict.py\", \"models/xgb_model.bin\", \"./\"]\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tGopakumar Gopinathan',\n",
       "  'section': '5. Deploying Machine Learning Models',\n",
       "  'question': 'How do I copy files from a different folder into docker container’s working directory?',\n",
       "  'course': 'machine-learning-zoomcamp'}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_results = elastic_search(query)\n",
    "search_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'How do I copy files from a different folder into docker container’s working directory?'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_results[2]['question']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_prompt(query, search_results):\n",
    "    prompt_template = \"\"\"\n",
    "    You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
    "    Use only the facts from the CONTEXT when answering the QUESTION.\n",
    "\n",
    "    QUESTION: {question}\n",
    "\n",
    "    CONTEXT:\n",
    "    {context}\n",
    "    \"\"\".strip()\n",
    "\n",
    "    context = \"\"\n",
    "    \n",
    "    for doc in search_results:\n",
    "        context = context + f\"Q: {doc['question']}\\nA: {doc['text']}\\n\\n\"\n",
    "    \n",
    "    prompt = prompt_template.format(question=query, context=context).strip()\n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You're a course teaching assistant. Answer the QUESTION based on the CONTEXT from the FAQ database.\n",
      "    Use only the facts from the CONTEXT when answering the QUESTION.\n",
      "\n",
      "    QUESTION: How do I execute a command in a running docker container?\n",
      "\n",
      "    CONTEXT:\n",
      "    Q: How do I debug a docker container?\n",
      "A: Launch the container image in interactive mode and overriding the entrypoint, so that it starts a bash command.\n",
      "docker run -it --entrypoint bash <image>\n",
      "If the container is already running, execute a command in the specific container:\n",
      "docker ps (find the container-id)\n",
      "docker exec -it <container-id> bash\n",
      "(Marcos MJD)\n",
      "\n",
      "Q: How do I copy files from my local machine to docker container?\n",
      "A: You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\n",
      "To copy a file or directory from your local machine into a running Docker container, you can use the `docker cp command`. The basic syntax is as follows:\n",
      "docker cp /path/to/local/file_or_directory container_id:/path/in/container\n",
      "Hrithik Kumar Advani\n",
      "\n",
      "Q: How do I copy files from a different folder into docker container’s working directory?\n",
      "A: You can copy files from your local machine into a Docker container using the docker cp command. Here's how to do it:\n",
      "In the Dockerfile, you can provide the folder containing the files that you want to copy over. The basic syntax is as follows:\n",
      "COPY [\"src/predict.py\", \"models/xgb_model.bin\", \"./\"]\t\t\t\t\t\t\t\t\t\t\tGopakumar Gopinathan\n",
      "\n",
      "Lenght of prompt is 1478\n"
     ]
    }
   ],
   "source": [
    "prompt = build_prompt(query, search_results)\n",
    "print(prompt)\n",
    "print('\\nLenght of prompt is', len(prompt))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of tokens = 326\n"
     ]
    }
   ],
   "source": [
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")\n",
    "num_tokens = len(encoding.encode(prompt))\n",
    "print('Number of tokens =', num_tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def llm(prompt):\n",
    "    response = client.chat.completions.create(\n",
    "        model='gpt-4o',\n",
    "        messages=[{\"role\": \"user\", \"content\": prompt}]\n",
    "    )\n",
    "    \n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rag(query):\n",
    "    search_results = elastic_search(query)\n",
    "    prompt = build_prompt(query, search_results)\n",
    "    answer = llm(prompt)\n",
    "    return answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'To execute a command in a running Docker container, use the following steps:\\n\\n1. First, find the Container ID of the running container by using the `docker ps` command:\\n    ```\\n    docker ps\\n    ```\\n\\n2. Once you have the Container ID, use the `docker exec` command to execute a bash session in the container, replacing `<container-id>` with the actual Container ID:\\n    ```\\n    docker exec -it <container-id> bash\\n    ```\\n\\nThis will open a bash session inside the running container, and you can then execute your desired commands.'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rag(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total cost for average token usage (150 input, 250 output) for 1000 requests: $4.50\n",
      "Total cost for specific values from Q6 and Q7 (1478 input, 326 output) for 1000 requests: $12.28\n"
     ]
    }
   ],
   "source": [
    "def calculate_cost(input_tokens_per_request, output_tokens_per_request, num_requests, input_cost_per_k, output_cost_per_k):\n",
    "    # Calculate total tokens\n",
    "    total_input_tokens = input_tokens_per_request * num_requests\n",
    "    total_output_tokens = output_tokens_per_request * num_requests\n",
    "\n",
    "    # Convert tokens to thousands\n",
    "    total_input_tokens_k = total_input_tokens / 1000\n",
    "    total_output_tokens_k = total_output_tokens / 1000\n",
    "\n",
    "    # Calculate costs\n",
    "    cost_input = total_input_tokens_k * input_cost_per_k\n",
    "    cost_output = total_output_tokens_k * output_cost_per_k\n",
    "\n",
    "    # Total cost\n",
    "    total_cost = cost_input + cost_output\n",
    "\n",
    "    return total_cost\n",
    "\n",
    "# Average tokens per request scenario\n",
    "average_input_tokens_per_request = 150\n",
    "average_output_tokens_per_request = 250\n",
    "num_requests = 1000\n",
    "input_cost_per_k = 0.005  # $0.005 per 1K tokens\n",
    "output_cost_per_k = 0.015  # $0.015 per 1K tokens\n",
    "\n",
    "average_cost = calculate_cost(average_input_tokens_per_request, average_output_tokens_per_request, num_requests, input_cost_per_k, output_cost_per_k)\n",
    "print(f\"Total cost for average token usage (150 input, 250 output) for 1000 requests: ${average_cost:.2f}\")\n",
    "\n",
    "# Specific values from Q6 and Q7 scenario\n",
    "q6_input_tokens_per_request = 1478\n",
    "q7_output_tokens_per_request = 326\n",
    "\n",
    "specific_cost = calculate_cost(q6_input_tokens_per_request, q7_output_tokens_per_request, num_requests, input_cost_per_k, output_cost_per_k)\n",
    "print(f\"Total cost for specific values from Q6 and Q7 (1478 input, 326 output) for 1000 requests: ${specific_cost:.2f}\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}