Homework_02/LLM_Zoomcamp_Week_02_Homework.ipynb.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7327118c-c46a-4da8-b958-18745af9d952",
   "metadata": {},
   "source": [
    "# Q1. Running Ollama with Docker\n",
    "What's the version of ollama client?\n",
    "\n",
    "To find out, enter the container and execute ollama with the -v flag."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2481c9d4-1f18-4aa2-9e94-66185d9fc12a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ollama version is 0.1.48\n"
     ]
    }
   ],
   "source": [
    "!ollama -v"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21e7c6dd-c8c2-4768-a8d8-79d431e2ee69",
   "metadata": {},
   "source": [
    "# Q2. Downloading an LLM\n",
    "We're interested in the metadata about this model. You can find it in models/manifests/registry.ollama.ai/library\n",
    "\n",
    "What's the content of the file related to gemma?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "18d6b05b-885c-40ae-b029-aa05aa9aba6e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"schemaVersion\":2,\"mediaType\":\"application/vnd.docker.distribution.manifest.v2+json\",\"config\":{\"mediaType\":\"application/vnd.docker.container.image.v1+json\",\"digest\":\"sha256:887433b89a901c156f7e6944442f3c9e57f3c55d6ed52042cbb7303aea994290\",\"size\":483},\"layers\":[{\"mediaType\":\"application/vnd.ollama.image.model\",\"digest\":\"sha256:c1864a5eb19305c40519da12cc543519e48a0697ecd30e15d5ac228644957d12\",\"size\":1678447520},{\"mediaType\":\"application/vnd.ollama.image.license\",\"digest\":\"sha256:097a36493f718248845233af1d3fefe7a303f864fae13bc31a3a9704229378ca\",\"size\":8433},{\"mediaType\":\"application/vnd.ollama.image.template\",\"digest\":\"sha256:109037bec39c0becc8221222ae23557559bc594290945a2c4221ab4f303b8871\",\"size\":136},{\"mediaType\":\"application/vnd.ollama.image.params\",\"digest\":\"sha256:22a838ceb7fb22755a3b0ae9b4eadde629d19be1f651f73efb8c6b4e2cd0eea0\",\"size\":84}]}"
     ]
    }
   ],
   "source": [
    "!cat ollama_files/models/manifests/registry.ollama.ai/library/gemma/2b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd5c8359-c131-4e3c-a682-5fef5b6de19f",
   "metadata": {},
   "source": [
    "# Q3. Running the LLM\n",
    "Test the following prompt: \"10 * 10\". What's the answer?"
   ]
  },
  {
   "cell_type": "raw",
   "id": "0fd066db-cada-4fc8-b5c2-9299da0f21ef",
   "metadata": {},
   "source": [
    "(llm_zoomcamp) optimistix@Rileens-MacBook-Pro Week_02 % ollama run gemma:2b \n",
    ">>> 10 * 10\n",
    "Sure, here's the answer to your question:\n",
    "\n",
    "10 * 10\n",
    "\n",
    ">>> what is 10*10\n",
    "The answer is 100. 10 * 10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68ff3d67-4003-41bc-b99e-ea652fdc5cfa",
   "metadata": {},
   "source": [
    "# Q4. Donwloading the weights\n",
    "\n",
    "What's the size of the ollama_files/models folder?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1bb9d304-f6c9-44e6-9a4a-db98c177f3e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1639140\tollama_files/models\n"
     ]
    }
   ],
   "source": [
    "!du -sk ollama_files/models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0eabe14a-245a-4e61-98ea-dca97193a3a9",
   "metadata": {},
   "source": [
    "# Q5. Adding the weights\n",
    "Let's now stop the container and add the weights to a new image\n",
    "\n",
    "For that, let's create a Dockerfile:\n",
    "\n",
    "FROM ollama/ollama\n",
    "\n",
    "COPY ...\n",
    "What do you put after COPY?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b776f2c-2807-46dc-beb0-318d5ca3995a",
   "metadata": {},
   "source": [
    "./ollama_files /root/.ollama"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "354d4a2d-0672-48c9-ad15-4c795ab6419b",
   "metadata": {},
   "source": [
    "# Q6. Serving it\n",
    "We can connect to it using the OpenAI client\n",
    "\n",
    "Let's test it with the following prompt:\n",
    "\n",
    "prompt = \"What's the formula for energy?\"\n",
    "Also, to make results reproducible, set the temperature parameter to 0:\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    #...\n",
    "    temperature=0.0\n",
    ")\n",
    "How many completion tokens did you get in response?\n",
    "\n",
    "304\n",
    "604\n",
    "904\n",
    "1204\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3f298bfd-3aed-437e-891c-105a445a983d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure, here's the formula for energy:\n",
      "\n",
      "**E = K + U**\n",
      "\n",
      "Where:\n",
      "\n",
      "* **E** is the energy in joules (J)\n",
      "* **K** is the kinetic energy in joules (J)\n",
      "* **U** is the potential energy in joules (J)\n",
      "\n",
      "**Kinetic energy (K)** is the energy an object possesses when it moves or is in motion. It is calculated as half the product of an object's mass (m) and its velocity (v) squared:\n",
      "\n",
      "**K = 1/2mv^2**\n",
      "\n",
      "**Potential energy (U)** is the energy an object possesses due to its position or configuration. It is calculated as the product of an object's mass, gravitational constant (g), and height or position above a reference point.\n",
      "\n",
      "**U = mgh**\n",
      "\n",
      "Where:\n",
      "\n",
      "* **m** is the mass in kilograms (kg)\n",
      "* **g** is the gravitational constant (9.8 m/s^2)\n",
      "* **h** is the height or position in meters (m)\n",
      "\n",
      "The formula shows that energy can be expressed as the sum of kinetic and potential energy. The kinetic energy is a measure of the object's ability to do work, while the potential energy is a measure of the object's ability to do work against a force.\n",
      "Number of completion tokens: 281\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(\n",
    "    base_url='http://localhost:11434/v1/',\n",
    "    api_key='ollama',\n",
    ")\n",
    "\n",
    "prompt = \"What's the formula for energy?\"\n",
    "response = client.chat.completions.create(\n",
    "    model='gemma:2b',\n",
    "    messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "    temperature=0.0\n",
    ")\n",
    "print(response.choices[0].message.content)\n",
    "\n",
    "completion_tokens = response.usage.completion_tokens\n",
    "print(f\"Number of completion tokens: {completion_tokens}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6710c6e-8808-4be3-9b93-d7d11a4054ce",
   "metadata": {},
   "source": [
    "The answer is 304, since it is the closest to 281."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}