deploy-ml-model/Deploy_your_first_service.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Service deployment using TrueFoundry\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gRhpN7is8PhA"
   },
   "source": [
    "##### This notebook demonstrates a demo of how you can deploy your first service with TrueFoundry.\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZUqbnAknyr57"
   },
   "source": [
    "After you complete the guide, you will have a successfully deployed model. Your deployed API will look like this:\n",
    "\n",
    "![](https://files.readme.io/e1affc0-Screenshot_2022-11-11_at_5.07.48_PM.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gkJQLuPP0Vta"
   },
   "source": [
    "## Project structure\n",
    "\n",
    "To complete this guide, you are going to review the following **files**:\n",
    "\n",
    "- `app.py` : contains our inference and FastAPI code\n",
    "- `iris_classifier.joblib` : the model file\n",
    "- `deploy.py`: contains our deployment code\n",
    "- `requirements.txt` : contains our dependencies\n",
    "\n",
    "Your **final file structure** is going to look like this:\n",
    "\n",
    "```\n",
    ".\n",
    "├── app.py\n",
    "├── iris_classifier.joblib\n",
    "├── deploy.py\n",
    "└── requirements.txt\n",
    "```\n",
    "\n",
    "As you can see, all the following files are created in the same folder/directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tfEqLC9b0cMY"
   },
   "source": [
    "# Setup\n",
    "\n",
    "Let's first setup all the things we need to deploy our service.\n",
    "\n",
    "- Signup or Login on TrueFoundry\n",
    "- Setup Workspace\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B4d_6OnQ2R6e"
   },
   "source": [
    "Let's start with installing truefoundry.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "Y8lgCr9Pwzuw",
    "outputId": "ac0e9102-1649-4357-c059-6cf928110ce0"
   },
   "outputs": [],
   "source": [
    "%pip install -U \"truefoundry>=0.5.9,<0.6.0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Login into TrueFoundry**\n",
    "\n",
    "In order to login run the cell below. Host can be found from the TrueFoundry UI as shown below like https://app.truefoundry.com\n",
    "\n",
    "![image.png](../common/images/host.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "IkXbOJVv8eNE"
   },
   "outputs": [],
   "source": [
    "!tfy login --host \"<Host name of TrueFoundry UI. e.g. https://company.truefoundry.cloud>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wov-JTue8gEF"
   },
   "source": [
    "**Select the `Workspace` in which you want to deploy your application.**\n",
    "\n",
    "Once you run the cell below you will get a prompt to enter your Workspace FQN. Follow the docs to\n",
    "\n",
    "**Create a Workspace**: https://docs.truefoundry.com/docs/key-concepts#creating-a-workspace\n",
    "\n",
    "**Get Existing Workspace FQN**: https://docs.truefoundry.com/docs/key-concepts#getting-workspace-fqn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import click"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XP_ib5m48k7a"
   },
   "outputs": [],
   "source": [
    "WORKSPACE_FQN = click.prompt(\n",
    "    \"Enter the Workspace FQN\",\n",
    "    type=str,\n",
    ")\n",
    "print(f\"\\nWorkspace FQN set to {WORKSPACE_FQN!r}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n41XXaJD25fM",
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Step 1: Fetch model\n",
    "\n",
    "## Model details\n",
    "\n",
    "For this guide, we have already trained a model. The given model has been trained on _iris dataset_ and uploaded to [GitHub](https://github.com/truefoundry/getting-started-examples/raw/b0ec188da047b75b334ea544560b583ab3b97510/deploy-ml-model/iris_classifier.joblib).\n",
    "\n",
    "> **Attributes** :\n",
    "> sepal length in cm, sepal width in cm, petal length in cm, petal width in cm\n",
    ">\n",
    "> **Predicted Attribute** :  \n",
    "> class of iris plant (one of the following - Iris Setosa, Iris Versicolour, Iris Virginica)\n",
    "\n",
    "## Download instructions\n",
    "\n",
    "Download the model from the following [link](https://github.com/truefoundry/getting-started-examples/raw/b0ec188da047b75b334ea544560b583ab3b97510/deploy-ml-model/iris_classifier.joblib).  \n",
    "Then move the model in your dev directory we created.\n",
    "\n",
    "Afterwards, your directory should look like this :\n",
    "\n",
    "```\n",
    ".\n",
    "└── iris_classifier.joblib\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wmy0aNHS3nnQ",
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Step 2: Implement the Inference Service code.\n",
    "\n",
    "The first step is to create a web API and deploy the model.  \n",
    "For this we are going to use [FastAPI](https://fastapi.tiangolo.com/) for this. FastAPI is a modern, intuitive web framework for building web APIs in python.\n",
    "\n",
    "Create the `app.py` and `requirements.txt` files in the same directory where the model is stored.\n",
    "\n",
    "```\n",
    ".\n",
    "├── iris_classifier.joblib\n",
    "├── app.py\n",
    "└── requirements.txt\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2BrljaTI8ugP"
   },
   "source": [
    "### **`app.py`**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iUqqXRVeyAdt",
    "outputId": "fe7b2dc2-a494-4e29-c5ea-65c7d9b88acb"
   },
   "outputs": [],
   "source": [
    "%%writefile app.py\n",
    "import os\n",
    "import joblib\n",
    "import pandas as pd\n",
    "from fastapi import FastAPI\n",
    "\n",
    "model = joblib.load(\"iris_classifier.joblib\")\n",
    "app = FastAPI(docs_url=\"/\", root_path=os.getenv(\"TFY_SERVICE_ROOT_PATH\"))\n",
    "\n",
    "@app.post(\"/predict\")\n",
    "def predict(\n",
    "    sepal_length: float, sepal_width: float, petal_length: float, petal_width: float\n",
    "):\n",
    "    data = dict(\n",
    "        sepal_length=sepal_length,\n",
    "        sepal_width=sepal_width,\n",
    "        petal_length=petal_length,\n",
    "        petal_width=petal_width,\n",
    "    )\n",
    "    prediction = int(model.predict(pd.DataFrame([data]))[0])\n",
    "    return {\"prediction\": prediction}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6zYON72f3ydW"
   },
   "source": [
    "Click on this [link](https://docs.truefoundry.com/recipes/create-a-fastapi-service-code-to-deploy-model) to understand the **`app.py`**:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile requirements.txt\n",
    "fastapi==0.114.0\n",
    "uvicorn==0.30.6\n",
    "scikit-learn==1.5.0\n",
    "joblib==1.3.2\n",
    "pandas==2.2.2\n",
    "numpy==1.26.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R2Q926YI4Yan",
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Step 3: Deploying the Inference App\n",
    "\n",
    "You can deploy services on TrueFoundry programmatically via our **Python SDK**.\n",
    "\n",
    "Create a `deploy.py`, after which our file structure will look like this:\n",
    "\n",
    "**File Structure**\n",
    "\n",
    "```Text Text\n",
    ".\n",
    "├── iris_classifier.joblib\n",
    "├── app.py\n",
    "├── deploy.py\n",
    "└── requirements.txt\n",
    "```\n",
    "\n",
    "### **`deploy.py`**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7pqgBDBmyAX1",
    "outputId": "d59b16ed-4e72-4094-f0c4-d1e802088c8b"
   },
   "outputs": [],
   "source": [
    "%%writefile deploy.py\n",
    "import argparse\n",
    "import logging\n",
    "from truefoundry.deploy import Build, PythonBuild, Service, Resources, Port, LocalSource\n",
    "\n",
    "logging.basicConfig(\n",
    "    level=logging.INFO, format=\"%(asctime)s [%(name)s] %(levelname)-8s %(message)s\"\n",
    ")\n",
    "\n",
    "\n",
    "def str_or_none(value):\n",
    "    return None if not value or value == \"None\" else value\n",
    "\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument(\n",
    "    \"--name\", \n",
    "    required=False, \n",
    "    default=\"iris-classifier-svc\",\n",
    "    type=str, \n",
    "    help=\"Name of the application.\"\n",
    ")\n",
    "parser.add_argument(\n",
    "    \"--workspace_fqn\",\n",
    "    \"--workspace-fqn\",\n",
    "    required=True,\n",
    "    type=str,\n",
    "    help=\"FQN of the workspace where application will be deployed.\",\n",
    ")\n",
    "parser.add_argument(\n",
    "    \"--host\",\n",
    "    required=True,\n",
    "    type=str,\n",
    "    help=\"Host where the application will be available for access. Ex:- my-app.my-org.com\",\n",
    ")\n",
    "parser.add_argument(\n",
    "    \"--path\",\n",
    "    required=False,\n",
    "    default=None,\n",
    "    type=str_or_none,\n",
    "    help=\"Path in addition to the host where the application will be available for access. Eg: my-org.com/my-path\",\n",
    ")\n",
    "args = parser.parse_args()\n",
    "\n",
    "service = Service(\n",
    "    name=args.name,\n",
    "    # Define how to build your code into a Docker image\n",
    "    image=Build(\n",
    "        # `LocalSource` helps specify the details of your local source code.\n",
    "        build_source=LocalSource(local_build=False),\n",
    "        # `PythonBuild` helps specify the details of your Python Code.\n",
    "      \t# These details will be used to templatize a DockerFile to build your Docker Image\n",
    "        build_spec=PythonBuild(\n",
    "            python_version=\"3.11\",\n",
    "            command=\"uvicorn app:app --port 8000 --host 0.0.0.0\",\n",
    "            requirements_path=\"requirements.txt\",\n",
    "        )\n",
    "    ),\n",
    "    # Set the ports your server will listen on\n",
    "    ports=[\n",
    "        # Providing a host and path value depends on the base domain urls configured in the cluster settings.\n",
    "        # You can learn how to find the base domain urls available to you https://docs.truefoundry.com/docs/define-ports-and-domains#identifying-available-domains\n",
    "        Port(port=8000, host=args.host, path=args.path)\n",
    "    ],\n",
    "    # Define the resource constraints.\n",
    "    #\n",
    "    # Requests are the minimum amount of resources that a container needs to run.\n",
    "    # Limits are the maximum amount of resources that a container can use.\n",
    "    resources=Resources(\n",
    "        cpu_request=0.1,\n",
    "        cpu_limit=0.1,\n",
    "        memory_request=500,\n",
    "        memory_limit=500,\n",
    "    ),\n",
    "    # Define environment variables that your Service will have access to\n",
    "    env={\"UVICORN_WEB_CONCURRENCY\": \"1\", \"ENVIRONMENT\": \"dev\"},\n",
    "    labels={\"tfy_openapi_path\": \"openapi.json\"},\n",
    ")\n",
    "service.deploy(workspace_fqn=args.workspace_fqn, wait=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will need a endpoint to access the deployed service. This host should follow the base domain url configured in the cluster.\n",
    "\n",
    "Please refer to following docs to get the base domain url to make your endpoint:\n",
    "\n",
    "https://docs.truefoundry.com/docs/define-ports-and-domains#identifying-available-domains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SERVICE_HOST = click.prompt(\n",
    "    \"Enter the host for the Service endpoint (e.g. my-service.org-domain.com OR org-domain.com)\",\n",
    "    type=str,\n",
    ")\n",
    "SERVICE_PATH = click.prompt(\n",
    "    \"Optionally, enter the path for Service endpoint (e.g. /my-service/). You can leave this blank if your domain supports subdomains: \",\n",
    "    type=str,\n",
    "    default='',\n",
    ")\n",
    "SERVICE_PATH = f\"/{SERVICE_PATH.strip('/')}/\" if SERVICE_PATH else None\n",
    "\n",
    "print(f\"\\nService Host set to {SERVICE_HOST!r}\")\n",
    "print(f\"\\nService Path set to {SERVICE_PATH!r}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6ex8BEPH5OY5"
   },
   "source": [
    "Now to deploy our FastAPI Service run the command below\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jMYgmbFk5Jd0",
    "outputId": "c6a1212b-7096-44f3-ea4e-d75565ca4e0f"
   },
   "outputs": [],
   "source": [
    "!python deploy.py --workspace_fqn $WORKSPACE_FQN --host $SERVICE_HOST --path $SERVICE_PATH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the build is complete, you will see a link to the dashboard after a message like <br>\n",
    "`You can find the application on the dashboard:-`.\n",
    "\n",
    "Click on the link to access the deployment dashboard.\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "jupyter-base",
   "language": "python",
   "name": "jupyter-base"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}