2024_hw2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "54eb4c39-35f5-4e1e-9fc3-516a9d34ee69",
   "metadata": {},
   "source": [
    "# Q1. Running Ollama with Docker"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5126eca-24bf-45fe-8f2b-f9c61eb9afe5",
   "metadata": {},
   "source": [
    "#### [Q1 Answer] \n",
    "ollama version is 0.1.48"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0144ec46-6d8d-40dc-8791-73f9b6cd9e3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# run ollama\n",
    "$ docker run -it \\\n",
    "    --rm \\\n",
    "    -v ollama:/root/.ollama \\\n",
    "    -p 11434:11434 \\\n",
    "    --name ollama \\\n",
    "    ollama/ollama\n",
    "\n",
    "# get container id\n",
    "$ docker ps\n",
    "\n",
    "# enter container\n",
    "$ docker exec -it 7a30839483c3 bash\n",
    "\n",
    "# get ollama version\n",
    "7a30839483c3$ ollama -v\n",
    "ollama version is 0.1.48"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fee0383-d1df-4038-8aec-03cf7d85aa5d",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Q2. Downloading an LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff288967-a19d-4d20-8500-c714b1c3938b",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### [Q2 Answer] \n",
    "{\"schemaVersion\":2,\"mediaType\":\"application/vnd.docker.distribution.manifest.v2+json\",\"config\":{\"mediaType\":\"application/vnd.docker.container.image.v1+json\",\"digest\":\"sha256:887433b89a901c156f7e6944442f3c9e57f3c55d6ed52042cbb7303aea994290\",\"size\":483},\"layers\":[{\"mediaType\":\"application/vnd.ollama.image.model\",\"digest\":\"sha256:c1864a5eb19305c40519da12cc543519e48a0697ecd30e15d5ac228644957d12\",\"size\":1678447520},{\"mediaType\":\"application/vnd.ollama.image.license\",\"digest\":\"sha256:097a36493f718248845233af1d3fefe7a303f864fae13bc31a3a9704229378ca\",\"size\":8433},{\"mediaType\":\"application/vnd.ollama.image.template\",\"digest\":\"sha256:109037bec39c0becc8221222ae23557559bc594290945a2c4221ab4f303b8871\",\"size\":136},{\"mediaType\":\"application/vnd.ollama.image.params\",\"digest\":\"sha256:22a838ceb7fb22755a3b0ae9b4eadde629d19be1f651f73efb8c6b4e2cd0eea0\",\"size\":84}]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d93df94-c5f0-4d5a-ab52-ff9938e20d62",
   "metadata": {},
   "outputs": [],
   "source": [
    "# pull gemma in container\n",
    "7a30839483c3$ ollama pull gemma:2b\n",
    "\n",
    "# get gemma metadata\n",
    "7a30839483c3$ cat /root/.ollama/models/manifests/registry.ollama.ai/library/gemma/2b\n",
    "{\n",
    "\t\"schemaVersion\": 2,\n",
    "\t\"mediaType\": \"application/vnd.docker.distribution.manifest.v2+json\",\n",
    "\t\"config\": {\n",
    "\t\t\"mediaType\": \"application/vnd.docker.container.image.v1+json\",\n",
    "\t\t\"digest\": \"sha256:887433b89a901c156f7e6944442f3c9e57f3c55d6ed52042cbb7303aea994290\",\n",
    "\t\t\"size\": 483\n",
    "\t},\n",
    "\t\"layers\": [\n",
    "\t\t{\n",
    "\t\t\t\"mediaType\": \"application/vnd.ollama.image.model\",\n",
    "\t\t\t\"digest\": \"sha256:c1864a5eb19305c40519da12cc543519e48a0697ecd30e15d5ac228644957d12\",\n",
    "\t\t\t\"size\": 1678447520\n",
    "\t\t},\n",
    "\t\t{\n",
    "\t\t\t\"mediaType\": \"application/vnd.ollama.image.license\",\n",
    "\t\t\t\"digest\": \"sha256:097a36493f718248845233af1d3fefe7a303f864fae13bc31a3a9704229378ca\",\n",
    "\t\t\t\"size\": 8433\n",
    "\t\t},\n",
    "\t\t{\n",
    "\t\t\t\"mediaType\": \"application/vnd.ollama.image.template\",\n",
    "\t\t\t\"digest\": \"sha256:109037bec39c0becc8221222ae23557559bc594290945a2c4221ab4f303b8871\",\n",
    "\t\t\t\"size\": 136\n",
    "\t\t},\n",
    "\t\t{\n",
    "\t\t\t\"mediaType\": \"application/vnd.ollama.image.params\",\n",
    "\t\t\t\"digest\": \"sha256:22a838ceb7fb22755a3b0ae9b4eadde629d19be1f651f73efb8c6b4e2cd0eea0\",\n",
    "\t\t\t\"size\": 84\n",
    "\t\t}\n",
    "\t]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "734cbf26-a825-4aa0-8fdb-558658f64c1e",
   "metadata": {},
   "source": [
    "# Q3. Running the LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "884d2f2b-a2af-40da-9776-e0b1013f72f2",
   "metadata": {},
   "source": [
    "#### [Q3 Answer] \n",
    " To solve this, we simply perform the multiplication of the two numbers provided:\n",
    "\n",
    "\n",
    "```plaintext\n",
    "\n",
    "  10\n",
    "\n",
    "x 10\n",
    "\n",
    "———\n",
    "\n",
    "  100\n",
    "\n",
    "```\n",
    "\n",
    "\n",
    "Therefore, `1 startMultiplication` equals `100`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e1bbee7-44ee-4f6a-91da-dbd449f51d43",
   "metadata": {},
   "outputs": [],
   "source": [
    "# run llm\n",
    "7a30839483c3$ ollama run phi3\n",
    "\n",
    "# given prompt\n",
    ">>> 10 * 10\n",
    " To solve this, we simply perform the multiplication of the two numbers provided:\n",
    "\n",
    "\n",
    "```plaintext\n",
    "\n",
    "  10\n",
    "\n",
    "x 10\n",
    "\n",
    "———\n",
    "\n",
    "  100\n",
    "\n",
    "```\n",
    "\n",
    "\n",
    "Therefore, `1 startMultiplication` equals `100`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d52d087-02c0-47c3-9622-8ab40964f719",
   "metadata": {},
   "source": [
    "# Q4. Donwloading the weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b39a84b-f674-46ab-b611-b6cfb88ea566",
   "metadata": {},
   "source": [
    "#### [Q4 Answer] \n",
    "1.7G"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b1e599a-5abf-4bfe-ac4c-aef94c1c844e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create local ollama folder\n",
    "$ mkdir /home/jupyter/llm/ollama_files\n",
    "\n",
    "# run ollama container again with mapping to local folder\n",
    "$ docker run -it \\\n",
    "    --rm \\\n",
    "    -v /home/jupyter/llm/ollama_files:/root/.ollama \\\n",
    "    -p 11434:11434 \\\n",
    "    --name ollama \\\n",
    "    ollama/ollama\n",
    "\n",
    "# pulla gemma model in new container\n",
    "$ docker exec -it ollama ollama pull gemma:2b \n",
    "\n",
    "# enter container\n",
    "$ docker exec -it ollama bash\n",
    "\n",
    "# get models folder size (from container)\n",
    "da2271fff893$ du -h /root/.ollama/models\n",
    "8.0K    models/manifests/registry.ollama.ai/library/gemma\n",
    "12K     models/manifests/registry.ollama.ai/library\n",
    "16K     models/manifests/registry.ollama.ai\n",
    "20K     models/manifests\n",
    "1.6G    models/blobs\n",
    "1.6G    models\n",
    "\n",
    "# get models folder size (from local)\n",
    "$ du -h /home/jupyter/llm/ollama_files\n",
    "8.0K    /home/jupyter/llm/ollama_files/models/manifests/registry.ollama.ai/library/gemma\n",
    "12K     /home/jupyter/llm/ollama_files/models/manifests/registry.ollama.ai/library\n",
    "16K     /home/jupyter/llm/ollama_files/models/manifests/registry.ollama.ai\n",
    "20K     /home/jupyter/llm/ollama_files/models/manifests\n",
    "1.6G    /home/jupyter/llm/ollama_files/models/blobs\n",
    "1.6G    /home/jupyter/llm/ollama_files/models\n",
    "1.6G    /home/jupyter/llm/ollama_files"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a92c0065-3513-436b-8bca-a053e1741e0d",
   "metadata": {},
   "source": [
    "# Q5. Adding the weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec500049-d89b-4c58-b97a-297d98f96a4d",
   "metadata": {},
   "source": [
    "#### [Q5 Answer] \n",
    "./ollama_files /root/.ollama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d468869-9c39-41ec-a7c0-8a3b9637f21c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Dockerfile\n",
    "'''\n",
    "FROM ollama/ollama\n",
    "COPY ./ollama_files /root/.ollama\n",
    "'''"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5446da51-8b12-415e-b85b-8dd4528ba47c",
   "metadata": {},
   "source": [
    "# Q6. Serving it"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d53d717f-bf6a-48c0-8de6-bd2de5e390e7",
   "metadata": {},
   "source": [
    "#### [Q6 Answer] \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "575ce009-97ae-4ad0-abc6-b2a81eb7f415",
   "metadata": {},
   "outputs": [],
   "source": [
    "# build it\n",
    "$ cd /home/jupyter/llm/\n",
    "$ docker build -t ollama-gemma2b .\n",
    "\n",
    "# run it\n",
    "$ docker run -it --rm -p 11434:11434 ollama-gemma2b\n",
    "\n",
    "# get container id\n",
    "$ docker ps\n",
    "\n",
    "# enter container\n",
    "$ docker exec -it fad2b37737ad bash\n",
    "\n",
    "# check models\n",
    "b7d183c5b0d7$ du -h /root/.ollama/models\n",
    "8.0K    /root/.ollama/models/manifests/registry.ollama.ai/library/gemma\n",
    "12K     /root/.ollama/models/manifests/registry.ollama.ai/library\n",
    "16K     /root/.ollama/models/manifests/registry.ollama.ai\n",
    "20K     /root/.ollama/models/manifests\n",
    "1.6G    /root/.ollama/models/blobs\n",
    "1.6G    /root/.ollama/models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dcd05621-fa7a-4f76-b495-52bcadcec82c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[198, 623, 5985, 20690, 316, 17950, 121749, 5954, 350, 6003, 8, 328, 448, 2817, 382, 1402, 197493, 20666, 314, 220, 200, 78549, 90, 16, 29124, 17, 92, 52251, 61, 17, 2381, 3144, 4522, 46352, 284, 2381, 8, 382, 290, 4842, 328, 290, 2817, 326, 46352, 323, 2381, 8, 382, 1617, 30061, 364, 2653, 6451, 5954, 11, 11884, 152581, 6451, 5954, 350, 38, 3111, 936, 5862, 146677, 9753, 11, 480, 665, 413, 25702, 472, 1402, 197493, 499, 3111, 314, 284, 956, 2381, 3144, 4522, 46352, 329, 2381, 8, 382, 290, 59117, 5192, 316, 44967, 350, 160401, 220, 24, 13, 9989, 284, 5031, 13848, 402, 16464, 8, 326, 46352, 312, 2381, 8, 382, 290, 4679, 328, 290, 2817, 5151, 261, 9682, 2438, 558]\n",
      "121\n"
     ]
    }
   ],
   "source": [
    "import tiktoken\n",
    "\n",
    "response=\"\"\"\n",
    " The general formula to calculate kinetic energy (KE) of an object is:\n",
    "\n",
    "\\[ KE = \\frac{1}{2}mv^2 \\]\n",
    "\n",
    "where \\( m \\) is the mass of the object and \\( v \\) is its velocity.\n",
    "\n",
    "For potential energy, particularly gravitational potential energy (GPE), near Earth's surface, it can be calculated as:\n",
    "\n",
    "\\[ GPE = mgh \\]\n",
    "\n",
    "where \\( g \\) is the acceleration due to gravity (approximately 9.81 m/s² on Earth) and \\( h \\) is the height of the object above a reference point.\n",
    "\"\"\"\n",
    "\n",
    "encoding = tiktoken.encoding_for_model(\"gpt-4o\")\n",
    "encoded_prompt = encoding.encode(response)\n",
    "\n",
    "# Print the encoded tokens\n",
    "print(encoded_prompt)\n",
    "print(len(encoded_prompt))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85dce4ef-5a96-4bda-831c-42921a0113f8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "python3",
   "name": "common-cpu.m113",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/base-cpu:m113"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}