classification/notebooks/21_deployment_on_azure_container_instances.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
    "\n",
    "<i>Licensed under the MIT License.</i>\n",
    "\n",
    "# Deployment of a model to an Azure Container Instance (ACI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/ComputerVision/classification/notebooks/21_deployment_on_azure_container_instances.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of contents <a id=\"table_of_content\"></a>\n",
    "\n",
    "1. [Introduction](#intro)\n",
    "1. [Model retrieval and export](#model)\n",
    "1. [Model deployment on Azure](#deploy)\n",
    "  1. [Workspace retrieval](#workspace)\n",
    "  1. [Model registration](#register)\n",
    "  1. [Scoring script](#scoring)\n",
    "  1. [Environment setup](#env)\n",
    "  1. [Computational resources](#compute)\n",
    "  1. [Web service deployment](#websvc)\n",
    "1. [Notes on web service deployment](#notes)\n",
    "1. [Clean-up](#clean)\n",
    "1. [Next steps](#next-steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction <a id=\"intro\"></a>\n",
    "\n",
    "While building a good performing model is important, for it to be useful, it needs to be accessible. In this notebook, we will learn how to make this possible by deploying our model onto Azure. We will more particularly see how to:\n",
    "- Register a model there\n",
    "- Create a Docker image that contains our model\n",
    "- Deploy a web service on [Azure Container Instances](https://azure.microsoft.com/en-us/services/container-instances/) using this Docker image.\n",
    "\n",
    "<img src=\"media/ACI_diagram_2.jpg\" width=\"500\" style=\"float: left;\" alt=\"Web service deployment workflow\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-requisites <a id=\"pre-reqs\"></a>\n",
    "For this notebook to run properly on our machine, an Azure workspace is required. If we don't have one, we need to first run through the short [20_azure_workspace_setup.ipynb](20_azure_workspace_setup.ipynb) notebook to create it.\n",
    "\n",
    "### Library import <a id=\"libraries\"></a>\n",
    "Throughout this notebook, we will be using a variety of libraries. We are listing them here for better readibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Azure ML SDK Version: 1.0.48\n"
     ]
    }
   ],
   "source": [
    "# For automatic reloading of modified libraries\n",
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# Regular python libraries\n",
    "import os\n",
    "import sys\n",
    "\n",
    "# fast.ai\n",
    "from fastai.vision import models\n",
    "\n",
    "# Azure\n",
    "import azureml.core\n",
    "from azureml.core import Experiment, Workspace\n",
    "from azureml.core.image import ContainerImage\n",
    "from azureml.core.model import Model\n",
    "from azureml.core.webservice import AciWebservice, Webservice\n",
    "from azureml.exceptions import WebserviceException\n",
    "\n",
    "# Computer Vision repository\n",
    "sys.path.extend([\".\", \"../..\"])\n",
    "# This \"sys.path.extend()\" statement allows us to move up the directory hierarchy \n",
    "# and access the utils_cv package\n",
    "from utils_cv.common.deployment import generate_yaml\n",
    "from utils_cv.common.data import root_path \n",
    "from utils_cv.classification.model import IMAGENET_IM_SIZE, model_to_learner\n",
    "\n",
    "# Check core SDK version number\n",
    "print(f\"Azure ML SDK Version: {azureml.core.VERSION}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Model retrieval and export <a id=\"model\"></a>\n",
    "\n",
    "For demonstration purposes, we will use here a ResNet18 model, pretrained on ImageNet. The following steps would be the same if we had trained a model locally (cf. [**01_training_introduction.ipynb**](01_training_introduction.ipynb) notebook for details).\n",
    "\n",
    "Let's first retrieve the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn = model_to_learner(models.resnet18(pretrained=True), IMAGENET_IM_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To be able to use this model, we need to export it to our local machine. We store it in an `outputs/` subfolder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_folder = os.path.join(os.getcwd(), 'outputs')\n",
    "model_name = 'im_classif_resnet18'  # Name we will give our model both locally and on Azure\n",
    "pickled_model_name = f'{model_name}.pkl'\n",
    "os.makedirs(output_folder, exist_ok=True)\n",
    "\n",
    "learn.export(os.path.join(output_folder, pickled_model_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create or access an Azure ML Workspace, you will need the following information. If you are coming from previous notebook you can retreive existing workspace, or create a new one if you are just starting with this notebook.\n",
    "\n",
    "- subscription ID: the ID of the Azure subscription we are using\n",
    "- resource group: the name of the resource group in which our workspace resides\n",
    "- workspace region: the geographical area in which our workspace resides (e.g. \"eastus2\" -- other examples are ---available here -- note the lack of spaces)\n",
    "- workspace name: the name of the workspace we want to create or retrieve.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "subscription_id = \"YOUR_SUBSCRIPTION_ID\"\n",
    "resource_group = \"YOUR_RESOURCE_GROUP_NAME\"  \n",
    "workspace_name = \"YOUR_WORKSPACE_NAME\"  \n",
    "workspace_region = \"YOUR_WORKSPACE_REGION\" #Possible values eastus, eastus2 and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Model deployment on Azure <a id=\"deploy\"></a>\n",
    "\n",
    "### 3.A Workspace retrieval <a id=\"workspace\"></a>\n",
    "\n",
    "In [prior notebook](20_azure_workspace_setup.ipynb) notebook, we created a workspace. This is a critical object from which we will build all the pieces we need to deploy our model as a web service. Let's start by retrieving it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - Warning: Falling back to use azure cli login credentials.\n",
      "If you run your code in unattended mode, i.e., where you can't give a user input, then we recommend to use ServicePrincipalAuthentication or MsiAuthentication.\n",
      "Please refer to aka.ms/aml-notebook-auth for different authentication mechanisms in azureml-sdk.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Workspace name: amlnotebookws\n",
      "Workspace region: eastus\n",
      "Resource group: amlnotebookrg\n"
     ]
    }
   ],
   "source": [
    "# A util method that creates a workspace or retrieves one if it exists, also takes care of Azure Authentication\n",
    "from utils_cv.common.azureml import get_or_create_workspace\n",
    "\n",
    "ws = get_or_create_workspace(\n",
    "        subscription_id,\n",
    "        resource_group,\n",
    "        workspace_name,\n",
    "        workspace_region)\n",
    "\n",
    "# Print the workspace attributes\n",
    "print('Workspace name: ' + ws.name, \n",
    "      'Workspace region: ' + ws.location, \n",
    "      'Subscription id: ' + ws.subscription_id, \n",
    "      'Resource group: ' + ws.resource_group, sep = '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.B Model registration <a id=\"register\"></a>\n",
    "\n",
    "Our final goal is to deploy our model as a web service. To do so, we need to first register it in our workspace, i.e. place it in our workspace's model registry. We can do this in 2 ways:\n",
    "1. register the model directly\n",
    "2. upload the model on Azure and then register it there.\n",
    "\n",
    "The advantage of the first method is that it does not require the setup of an experiment or of any runs. The advantage of the second fashion is that we can keep track of the models that we used or trained in a given experiment, and understand where the ones we ended up registering come from.\n",
    "\n",
    "The cells below show each of the methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Without experiment <a id=\"noexp\"></a>\n",
    "\n",
    "We leverage the `register` method from the Azure ML `Model` object. For that, we just need the location of the model we saved on our local machine, its name and our workspace object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Registering model im_classif_resnet18\n"
     ]
    }
   ],
   "source": [
    "model = Model.register(\n",
    "    model_path = os.path.join('outputs', pickled_model_name),\n",
    "    model_name = model_name,\n",
    "    tags = {\"Model\": \"Pretrained ResNet18\"},\n",
    "    description = \"Image classifier\",\n",
    "    workspace = ws\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### With an experiment <a id=\"exp\"></a>\n",
    "\n",
    "An experiment contains a series of trials called `Runs`. A run typically contains some tasks, such as training a model, etc. Through a run's methods, we can log several metrics such as training and test loss and accuracy, and even tag our run. The full description of the run class is available [here](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.run.run?view=azure-ml-py). In our case, however, we just need the run to attach our model file to our workspace and experiment.\n",
    "\n",
    "We do this by using `run.upload_file()` and `run.register_model()`, which takes:\n",
    "- a `model_name` that represents what our model does\n",
    "- and the `model_path` relative to the run.\n",
    "\n",
    "Using `run.upload_file()` and specifying the `outputs/` folder allows us to check the presence of the uploaded model on the Azure portal. This is especially convenient when we want to try different versions of a model, or even different models entirely, and keep track of them all, even if we end up registering only the best performing one.\n",
    "\n",
    "Let's first create a new experiment. If an experiment with the same name already exists in our workspace, the run we will generate will be recorded under that already existing experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New/Existing experiment:\n",
      "       --> Name: image-classifier-webservice\n",
      "       --> Workspace name: amlnotebookws\n"
     ]
    }
   ],
   "source": [
    "# Create a new/Retrieve an existing experiment\n",
    "experiment_name = 'image-classifier-webservice'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "print(f\"New/Existing experiment:\\n \\\n",
    "      --> Name: {experiment.name}\\n \\\n",
    "      --> Workspace name: {experiment.workspace.name}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the run\n",
    "run = experiment.start_logging(snapshot_directory=None)\n",
    "# \"snapshot_directory=None\" prevents a snapshot from being saved -- this helps keep the amount of storage used low"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have launched our run, we can see our experiment on the Azure portal, under `Experiments` (in the left-hand side list).\n",
    "\n",
    "<img src=\"media/experiment.jpg\" width=\"800\" alt=\"Azure portal view of experiment\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now attach our local model to our workspace and experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<azureml._restclient.models.batch_artifact_content_information_dto.BatchArtifactContentInformationDto at 0x1a2e939c88>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Upload the model (.pkl) file to Azure\n",
    "run.upload_file(\n",
    "    name=os.path.join('outputs', pickled_model_name), \n",
    "    path_or_stream=os.path.join(os.getcwd(), 'outputs', pickled_model_name)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the model with the workspace\n",
    "model = run.register_model(\n",
    "    model_name=model_name,\n",
    "    model_path=os.path.join('outputs', pickled_model_name),\n",
    "    tags = {\"Model\": \"Pretrained ResNet18\"},\n",
    ")\n",
    "# !!! We need to make sure that the model name we use here is the same as in the scoring script below !!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the model is uploaded and registered, we can see it on the Azure platform, under `Outputs` and `Models`\n",
    "\n",
    "<div class=\"inline-block\">\n",
    "    <img src=\"media/uploaded_model.jpg\" width=\"800\" alt=\"Azure portal view of the Outputs/ folder\">\n",
    "</div>\n",
    "\n",
    "<div class=\"inline-block\">\n",
    "    <img src=\"media/models.jpg\" width=\"800\" alt=\"Azure portal view of the Models section\">\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also check that it is programatically accessible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model:\n",
      " --> Name: im_classif_resnet18\n",
      "       --> ID: im_classif_resnet18:8\n",
      "       --> Path:azureml-models/im_classif_resnet18/8/im_classif_resnet18.pkl\n"
     ]
    }
   ],
   "source": [
    "print(f\"Model:\\n --> Name: {model.name}\\n \\\n",
    "      --> ID: {model.id}\\n \\\n",
    "      --> Path:{model._get_model_path_remote(model.name, model.version, ws)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['outputs/im_classif_resnet18.pkl']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run.get_file_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we are also interested in verifying which model we uploaded, we can download it to our local machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'im_classif_resnet18.pkl'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.download(exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i><b>Note:</b> If we ran the cells in both the \"with an experiment\" and \"without experiment\" sections, we got 2 iterations of the same model registered on Azure. This is not a problem as any operation that we perform on the \"model\" object, later on, will be associated with the latest version of the model that we registered. To clean things up, we can go to the portal, select the model we do not want and click the \"Delete\" button. In general, we would register the model using only one of these 2 methods. </i>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are all done with our model registration, so we can close our run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Close the run\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"width:100%\"><tr><th>Experiment</th><th>Id</th><th>Type</th><th>Status</th><th>Details Page</th><th>Docs Page</th></tr><tr><td>image-classifier-webservice</td><td>58a22e83-11da-4062-b8ca-0d140fa176e5</td><td></td><td>Running</td><td><a href=\"https://mlworkspace.azure.ai/portal/subscriptions/2ad17db4-e26d-4c9e-999e-adae9182530c/resourceGroups/amlnotebookrg/providers/Microsoft.MachineLearningServices/workspaces/amlnotebookws/experiments/image-classifier-webservice/runs/58a22e83-11da-4062-b8ca-0d140fa176e5\" target=\"_blank\" rel=\"noopener\">Link to Azure Portal</a></td><td><a href=\"https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.run.Run?view=azure-ml-py\" target=\"_blank\" rel=\"noopener\">Link to Documentation</a></td></tr></table>"
      ],
      "text/plain": [
       "Run(Experiment: image-classifier-webservice,\n",
       "Id: 58a22e83-11da-4062-b8ca-0d140fa176e5,\n",
       "Type: None,\n",
       "Status: Running)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access the portal\n",
    "run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.C Scoring script <a id=\"scoring\"></a>\n",
    "For the web service to return predictions on a given input image, we need to provide it with instructions on how to use the model we just registered. These instructions are stored in the scoring script.\n",
    "\n",
    "This script must contain two required functions, `init()` and `run(input_data)`:\n",
    "- In the `init()` function, we typically load the model into a global object. This function is executed only once when the Docker container is started.\n",
    "- In the `run(input_data)` function, the model is used to predict a value based on the input data. The input and output of `run` typically use JSON as serialization and de-serialization format but we are not limited to that.\n",
    "\n",
    "<i><b>Note:</b> The \"run()\" function here is different from the \"run\" object we created in our experiment</i>\n",
    "\n",
    "This file must also be stored in the current directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "scoring_script = \"score.py\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting score.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile $scoring_script\n",
    "# Copyright (c) Microsoft. All rights reserved.\n",
    "# Licensed under the MIT license.\n",
    "\n",
    "import os\n",
    "import json\n",
    "\n",
    "from base64 import b64decode\n",
    "from io import BytesIO\n",
    "\n",
    "from azureml.core.model import Model\n",
    "from fastai.vision import load_learner, open_image\n",
    "\n",
    "def init():\n",
    "    global model\n",
    "    model_path = Model.get_model_path(model_name='im_classif_resnet18')\n",
    "    # ! We cannot use the *model_name* variable here otherwise the execution on Azure will fail !\n",
    "    \n",
    "    model_dir_path, model_filename = os.path.split(model_path)\n",
    "    model = load_learner(path=model_dir_path, fname=model_filename)\n",
    "\n",
    "\n",
    "def run(raw_data):\n",
    "\n",
    "    # Expects raw_data to be a list within a json file\n",
    "    result = []    \n",
    "    \n",
    "    for im_string in json.loads(raw_data)['data']:\n",
    "        im_bytes = b64decode(im_string)\n",
    "        try:\n",
    "            im = open_image(BytesIO(im_bytes))\n",
    "            pred_class, pred_idx, outputs = model.predict(im)\n",
    "            result.append({\"label\": str(pred_class), \"probability\": str(outputs[pred_idx].item())})\n",
    "        except Exception as e:\n",
    "            result.append({\"label\": str(e), \"probability\": ''})\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.D Environment setup <a id=\"env\"></a>\n",
    "\n",
    "In order to make predictions on the Azure platform, it is important to create an environment as similar as possible to the one in which the model was trained. Here, we use a fast.ai pretrained model that also requires pytorch and a few other libraries. To re-create this environment, we use a [Docker container](https://www.docker.com/resources/what-container). We configure it via a yaml file that will contain all the conda dependencies needed by the model. This yaml file is a subset of  `<repo_root>/classification/environment.yml`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Conda environment specification. The dependencies defined in this file will\r\n",
      "# be automatically provisioned for runs with userManagedDependencies=False.\r\n",
      "\n",
      "# Details about the Conda environment file format:\r\n",
      "# https://conda.io/docs/user-guide/tasks/manage-environments.html#create-env-file-manually\r\n",
      "\n",
      "name: project_environment\n",
      "dependencies:\n",
      "  # The python interpreter version.\r\n",
      "  # Currently Azure ML only supports 3.5.2 and later.\r\n",
      "- python=3.6.2\n",
      "\n",
      "- pip:\n",
      "    # Required packages for AzureML execution, history, and data preparation.\r\n",
      "  - azureml-defaults\n",
      "\n",
      "- pytorch==1.0.0\n",
      "- fastai==1.0.48\n",
      "- spacy\n",
      "- dataclasses\n",
      "channels:\n",
      "- conda-forge\n",
      "- defaults\n",
      "- pytorch\n",
      "- fastai\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Create a deployment-specific yaml file from classification/environment.yml\n",
    "try:\n",
    "    generate_yaml(\n",
    "        directory=os.path.join(root_path()), \n",
    "        ref_filename='environment.yml',\n",
    "        needed_libraries=['pytorch', 'spacy', 'fastai', 'dataclasses'],\n",
    "        conda_filename='myenv.yml'\n",
    "    )\n",
    "    # Note: Take a look at the generate_yaml() function for details on how to create your yaml file from scratch\n",
    "\n",
    "except FileNotFoundError:\n",
    "    raise FileNotFoundError(\"The *environment.yml* file is missing - Please make sure to retrieve it from the github repository\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are different ways of creating a Docker image on Azure. Here, we create it separately from the service it will be used by. This way of proceeding gives us direct access to the Docker image object. Thus, if the service deployment fails, but the Docker image gets deployed successfully, we can try deploying the service again, without having to create a new image all over again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configure the Docker image\n",
    "try:\n",
    "    image_config = ContainerImage.image_configuration(\n",
    "        execution_script = \"score.py\",\n",
    "        runtime = \"python\",\n",
    "        conda_file = \"myenv.yml\",\n",
    "        description = \"Image with fast.ai Resnet18 model (fastai 1.0.48)\",\n",
    "        tags = {'training set': \"ImageNet\", \n",
    "                'architecture': \"CNN ResNet18\",\n",
    "                'type': 'Pretrained'}\n",
    "    )\n",
    "except WebserviceException:\n",
    "    raise FileNotFoundError(\"The files *score.py* and/or *myenv.yaml* could not be found - Please run the cells above again\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating image\n"
     ]
    }
   ],
   "source": [
    "# Create the Docker image\n",
    "try:\n",
    "    docker_image = ContainerImage.create(\n",
    "        name = \"image-classif-resnet18-f48\",\n",
    "        models = [model],\n",
    "        image_config = image_config,\n",
    "        workspace = ws\n",
    "    )\n",
    "    # The image name should not contain more than 32 characters, and should not contain any spaces, dots or underscores\n",
    "except WebserviceException:\n",
    "    raise FileNotFoundError(\"The files *score.py* and/or *myenv.yaml* could not be found - Please run the cells above again\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running.......................................................................................................................................\n",
      "Succeeded\n",
      "Image creation operation finished for image image-classif-resnet18-f48:2, operation \"Succeeded\"\n",
      "CPU times: user 980 ms, sys: 192 ms, total: 1.17 s\n",
      "Wall time: 11min 35s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "docker_image.wait_for_creation(show_output = True)  # This can take up to 12 min"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the image gets successfully created, we expect to see:\n",
    "\n",
    "`Creating image\n",
    "Running .....\n",
    "SucceededImage creation operation finished for image <docker_image_name>, operation \"Succeeded\"\n",
    "Wall time: Xmin`\n",
    "\n",
    "It happens, sometimes, that the deployment of the Docker image fails. Re-running the previous command typically solves the problem. If it doesn't, however, we can run the following one and inspect the deployment logs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://amlnotebstorage914c3d2fd.blob.core.windows.net/azureml/ImageLogs/1a39d9e5-6432-4a4b-95ec-13211a811a60/build.log?sv=2018-03-28&sr=b&sig=kjIXAcI69WUzWofD77LKHotev2f%2BQhXjoHyBEBEkqZU%3D&st=2019-07-18T17%3A58%3A03Z&se=2019-08-17T18%3A03%3A03Z&sp=rl\n"
     ]
    }
   ],
   "source": [
    "print(ws.images[\"image-classif-resnet18-f48\"].image_build_log_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.E Computational resources <a id=\"compute\"></a>\n",
    "\n",
    "In this notebook, we use [Azure Container Instances](https://docs.microsoft.com/en-us/azure/container-instances/container-instances-overview) (ACI) which are good for quick and [cost-effective](https://azure.microsoft.com/en-us/pricing/details/container-instances/) development/test deployment scenarios.\n",
    "\n",
    "To set them up properly, we need to indicate the number of CPU cores and the amount of memory we want to allocate to our web service. Optional tags and descriptions are also available for us to identify the instances in AzureML when looking at the `Compute` tab in the Azure Portal. We also enable monitoring, through the `enable_app_insights` parameter. Once our web app is up and running, this will allow us to measure the amount of traffic it gets, how long it takes to respond, the type of exceptions that get raised, etc. We will do so through [Application Insights](https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview), which is an application performance management service.\n",
    "\n",
    "<i><b>Note:</b> For production workloads, it is better to use [Azure Kubernetes Service](https://docs.microsoft.com/en-us/azure/aks/) (AKS) instead. We will demonstrate how to do this in the [next notebook](22_deployment_on_azure_kubernetes_service.ipynb).<i>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a deployment configuration with 1 CPU and 5 gigabytes of RAM, and add monitoring to it\n",
    "aci_config = AciWebservice.deploy_configuration(\n",
    "    cpu_cores=1,\n",
    "    memory_gb=5,\n",
    "    enable_app_insights=True,\n",
    "    tags={'webservice': 'image classification model (fastai 1.0.48)'},\n",
    "    description='This service classifies images into 1000 different groups.'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.F Web service deployment <a id=\"websvc\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final step to deploying our web service is to call `WebService.deploy_from_image()`. This function uses the Docker image and the deployment configuration we created above to perform the following:\n",
    "\n",
    "- Deploy the docker image to an Azure Container Instance\n",
    "- Call the `init()` function in our scoring file\n",
    "- Provide an HTTP endpoint for scoring calls\n",
    "\n",
    "The `deploy_from_image` method requires the following parameters:\n",
    "\n",
    "- workspace: the workspace containing the service\n",
    "- name: a unique name used to identify the service in the workspace\n",
    "- image: a docker image object that contains the environment needed for scoring/inference\n",
    "- deployment_config: a configuration object describing the compute type\n",
    "\n",
    "Azure Container Instances have no associated ComputeTarget, so we do not specify any here. Remember, we already provided information on the number of CPUs and the amount of memory needed in the service configuration file above.\n",
    "\n",
    "<i><b>Note:</b> The web service creation can take a few minutes</i>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating service\n"
     ]
    }
   ],
   "source": [
    "# Define how to deploy the web service\n",
    "service_name = 'im-classif-websvc'\n",
    "service = Webservice.deploy_from_image(\n",
    "    workspace=ws,\n",
    "    name=service_name,\n",
    "    image=docker_image,\n",
    "    deployment_config=aci_config\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An alternative way of deploying the service is to deploy from the model directly. In that case, we would need to provide the docker image configuration object (image_config), and our list of models (just one of them here).\n",
    "The advantage of `deploy_from_image` over <a href=\"https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice(class)?view=azure-ml-py#deploy-from-model-workspace--name--models--image-config--deployment-config-none--deployment-target-none-\">deploy_from_model</a> is that the former allows us\n",
    "to re-use the same Docker image in case the deployment of this service fails, or even for other\n",
    "types of deployments, as we will see in the next notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running............................................\n",
      "SucceededACI service creation operation finished, operation \"Succeeded\"\n"
     ]
    }
   ],
   "source": [
    "# Deploy the web service\n",
    "service.wait_for_deployment(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When successful, we expect to see the following:\n",
    "\n",
    "`\n",
    "Creating service\n",
    "Running .....\n",
    "SucceededACI service creation operation finished, operation \"Succeeded\"`\n",
    "\n",
    "In the case where the deployment is not successful, we can look at the image and service logs to debug. [These instructions](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-troubleshoot-deployment) can also be helpful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Access the service logs\n",
    "# print(service.get_logs())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Service im-classif-websvc is _Healthy_ and available at http://c7bf18e3-cef1-4179-a524-59f862ffa1d9.eastus.azurecontainer.io/score\n"
     ]
    }
   ],
   "source": [
    "# Retrieve the service status\n",
    "print(f\"Service {service.name} is _{service.state}_ and available at {service.scoring_uri}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also check the presence and status of both our new Docker image and web service on the Azure portal, under the `Images` and `Deployments` tabs, respectively.\n",
    "\n",
    "\n",
    "<img src=\"media/docker_images.jpg\" width=\"800\" alt=\"Azure portal view of the Images section\">\n",
    "<img src=\"media/deployments.jpg\" width=\"800\" alt=\"Azure portal view of the Deployments section\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Notes on web service deployment <a id=\"notes\"></a>\n",
    "\n",
    "As we discussed above, Azure Container Instances tend to be used to develop and test deployments. They are typically configured with CPUs, which usually suffice when the number of requests per second is not too high. When working with several instances, we can configure them further by specifically [allocating CPU resources](https://docs.microsoft.com/en-us/azure/container-instances/container-instances-container-groups#deployment) to each of them.\n",
    "\n",
    "For production requirements, i.e. when &gt; 100 requests per second are expected, we recommend deploying models to Azure Kubernetes Service (AKS). It is a convenient infrastructure as it manages hosted Kubernetes environments, and makes it easy to deploy and manage containerized applications without container orchestration expertise. It also supports deployments with CPU clusters and deployments with GPU clusters.\n",
    "\n",
    "We will see an example of this in the [next notebook](22_deployment_on_azure_kubernetes_service.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Clean up <a id=\"clean\"></a>\n",
    "\n",
    "Throughout the notebook, we used a workspace and Azure container instances. To get a sense of the cost we incurred, we can refer to this [calculator](https://azure.microsoft.com/en-us/pricing/calculator/). We can also navigate to the [Cost Management + Billing](https://ms.portal.azure.com/#blade/Microsoft_Azure_Billing/ModernBillingMenuBlade/Overview) pane on the portal, click on our subscription ID, and click on the Cost Analysis tab to check our credit usage.\n",
    "\n",
    "In order not to incur extra costs, let's delete the resources we no longer need.\n",
    "\n",
    "Once we have verified that our web service works well on ACI (cf. \"Next steps\" section below), we can delete it. This helps reduce [costs](https://azure.microsoft.com/en-us/pricing/details/container-instances/), since the container group we were paying for no longer exists, and allows us to keep our workspace clean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# service.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, the main resource we are paying for is the <b>Standard</b> Azure Container Registry (ACR), which contains our Docker image. Details on pricing are available [here](https://azure.microsoft.com/en-us/pricing/details/container-registry/).\n",
    "\n",
    "We may decide to use our Docker image in a separate ACI or even in an AKS deployment. In that case, we should keep it available in our workspace. However, if we no longer have a use for it, we can delete it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# docker_image.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If our goal is to continue using our workspace, we should keep it available. On the contrary, if we plan on no longer using it and its associated resources, we can delete it.\n",
    "\n",
    "<i><b>Note:</b> Deleting the workspace will delete all the experiments, outputs, models, Docker images, deployments, etc. that we created in that workspace</i>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ws.delete(delete_dependent_resources=True)\n",
    "# This deletes our workspace, the container registry, the account storage, Application Insights and the key vault"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Next steps <a id=\"next-steps\"></a>\n",
    "\n",
    "In the [next tutorial](22_deployment_on_azure_kubernetes_service.ipynb), we will leverage the same Docker image, and deploy our model on AKS. We will then test both of our web services in the [23_aci_aks_web_service_testing.ipynb](23_aci_aks_web_service_testing.ipynb) notebook."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "cv",
   "language": "python",
   "name": "cv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}