02-open-source/02-homework/homework_02_Q6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9e9cbe4c-7724-4bd8-ba5f-96e733977423",
   "metadata": {},
   "source": [
    "#### Question 6. Serving the LLM. Number of output tokens "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "081a7edc-f893-406a-89b3-e5ac877b7d39",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID   IMAGE            COMMAND               CREATED         STATUS         PORTS                                           NAMES\n",
      "c16e6a81232c   ollama-gemma2b   \"/bin/ollama serve\"   4 minutes ago   Up 4 minutes   0.0.0.0:11434->11434/tcp, :::11434->11434/tcp   ollama-llmzoomcamp\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "69b75dd3-3106-44a2-9f54-8494b8f0e8e3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25lpulling manifest ⠋ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠙ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠸ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠸ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠼ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest \n",
      "pulling c1864a5eb193... 100% ▕████████████████▏ 1.7 GB                         \n",
      "pulling 097a36493f71... 100% ▕████████████████▏ 8.4 KB                         \n",
      "pulling 109037bec39c... 100% ▕████████████████▏  136 B                         \n",
      "pulling 22a838ceb7fb... 100% ▕████████████████▏   84 B                         \n",
      "pulling 887433b89a90... 100% ▕████████████████▏  483 B                         \n",
      "verifying sha256 digest \n",
      "writing manifest \n",
      "removing any unused layers \n",
      "success \u001b[?25h\n"
     ]
    }
   ],
   "source": [
    "!docker exec ollama-llmzoomcamp ollama pull gemma:2b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "988ece59-951a-4b32-ba3f-cb8efb66a9bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(\n",
    "    base_url='http://localhost:11434/v1/',\n",
    "    api_key='ollama',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f34d69df-9b16-40ad-82c8-bfab907b09db",
   "metadata": {},
   "outputs": [],
   "source": [
    "def llm(prompt):\n",
    "    response = client.chat.completions.create(\n",
    "        model='gemma:2b',\n",
    "        messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "        temperature=0.0\n",
    "    )\n",
    "    \n",
    "    return response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f206ff6d-f8a6-49d5-905c-2ea5dc4d73be",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"What's the formula for energy?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "703c4cc4-80db-4a51-a143-867128816d99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CompletionUsage(completion_tokens=304, prompt_tokens=0, total_tokens=304)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = llm(prompt)\n",
    "response.usage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "efdbf0c3-9913-4337-825b-105dcc8fc6e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Sure, here's the formula for energy:\\n\\n**E = K + U**\\n\\nWhere:\\n\\n* **E** is the energy in joules (J)\\n* **K** is the kinetic energy in joules (J)\\n* **U** is the potential energy in joules (J)\\n\\n**Kinetic energy (K)** is the energy an object possesses when it moves or is in motion. It is calculated as half the product of an object's mass (m) and its velocity (v) squared:\\n\\n**K = 1/2 * m * v^2**\\n\\n**Potential energy (U)** is the energy an object possesses when it is in a position or has a specific configuration. It is calculated as the product of an object's mass and the gravitational constant (g) multiplied by the height or distance of the object from a reference point.\\n\\n**Gravitational potential energy (U)** is given by the formula:\\n\\n**U = mgh**\\n\\nWhere:\\n\\n* **m** is the mass of the object in kilograms (kg)\\n* **g** is the acceleration due to gravity in meters per second squared (m/s^2)\\n* **h** is the height or distance of the object in meters (m)\\n\\nThe formula for energy can be used to calculate the total energy of an object, the energy of a specific part of an object, or the change in energy of an object over time.\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a2deca8b-ede4-4174-9a93-8ad9408114c9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure, here's the formula for energy:\n",
      "\n",
      "**E = K + U**\n",
      "\n",
      "Where:\n",
      "\n",
      "* **E** is the energy in joules (J)\n",
      "* **K** is the kinetic energy in joules (J)\n",
      "* **U** is the potential energy in joules (J)\n",
      "\n",
      "**Kinetic energy (K)** is the energy an object possesses when it moves or is in motion. It is calculated as half the product of an object's mass (m) and its velocity (v) squared:\n",
      "\n",
      "**K = 1/2 * m * v^2**\n",
      "\n",
      "**Potential energy (U)** is the energy an object possesses when it is in a position or has a specific configuration. It is calculated as the product of an object's mass and the gravitational constant (g) multiplied by the height or distance of the object from a reference point.\n",
      "\n",
      "**Gravitational potential energy (U)** is given by the formula:\n",
      "\n",
      "**U = mgh**\n",
      "\n",
      "Where:\n",
      "\n",
      "* **m** is the mass of the object in kilograms (kg)\n",
      "* **g** is the acceleration due to gravity in meters per second squared (m/s^2)\n",
      "* **h** is the height or distance of the object in meters (m)\n",
      "\n",
      "The formula for energy can be used to calculate the total energy of an object, the energy of a specific part of an object, or the change in energy of an object over time.\n"
     ]
    }
   ],
   "source": [
    "print(_)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}