how-to-use-azureml/deployment/enable-app-insights-in-production-service/enable-app-insights-in-production-service.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Enabling App Insights for Services in Production\n",
        "With this notebook, you can learn how to enable App Insights for standard service monitoring, plus, we provide examples for doing custom logging within a scoring files in a model. \n",
        "\n",
        "\n",
        "## What does Application Insights monitor?\n",
        "It monitors request rates, response times, failure rates, etc. For more information visit [App Insights docs.](https://docs.microsoft.com/en-us/azure/application-insights/app-insights-overview)\n",
        "\n",
        "\n",
        "## What is different compared to standard production deployment process?\n",
        "If you want to enable generic App Insights for a service run:\n",
        "```python\n",
        "aks_service= Webservice(ws, \"aks-w-dc2\")\n",
        "aks_service.update(enable_app_insights=True)```\n",
        "Where \"aks-w-dc2\" is your service name. You can also do this from the Azure Portal under your Workspace--> deployments--> Select deployment--> Edit--> Advanced Settings--> Select \"Enable AppInsights diagnostics\"\n",
        "\n",
        "If you want to log custom traces, you will follow the standard deplyment process for AKS and you will:\n",
        "1. Update scoring file.\n",
        "2. Update aks configuration.\n",
        "3. Deploy the model with this new configuration. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deployment/enable-app-insights-in-production-service/enable-app-insights-in-production-service.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. Import your dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Workspace\n",
        "from azureml.core.compute import AksCompute, ComputeTarget\n",
        "from azureml.core.webservice import AksWebservice\n",
        "import azureml.core\n",
        "import json\n",
        "print(azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. Set up your configuration and create a workspace\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. Register Model\n",
        "Register an existing trained model, add descirption and tags."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "#Register the model\n",
        "from azureml.core.model import Model\n",
        "model = Model.register(model_path = \"sklearn_regression_model.pkl\", # this points to a local file\n",
        "                       model_name = \"sklearn_regression_model.pkl\", # this is the name the model is registered as\n",
        "                       tags = {'area': \"diabetes\", 'type': \"regression\"},\n",
        "                       description = \"Ridge regression model to predict diabetes\",\n",
        "                       workspace = ws)\n",
        "\n",
        "print(model.name, model.description, model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. *Update your scoring file with custom print statements*\n",
        "Here is an example:\n",
        "### a. In your init function add:\n",
        "```python\n",
        "print (\"model initialized\" + time.strftime(\"%H:%M:%S\"))```\n",
        "\n",
        "### b. In your run function add:\n",
        "```python\n",
        "print (\"Prediction created\" + time.strftime(\"%H:%M:%S\"))```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%writefile score.py\n",
        "import os\n",
        "import pickle\n",
        "import json\n",
        "import numpy \n",
        "from sklearn.externals import joblib\n",
        "from sklearn.linear_model import Ridge\n",
        "import time\n",
        "\n",
        "def init():\n",
        "    global model\n",
        "    #Print statement for appinsights custom traces:\n",
        "    print (\"model initialized\" + time.strftime(\"%H:%M:%S\"))\n",
        "    \n",
        "    # AZUREML_MODEL_DIR is an environment variable created during deployment.\n",
        "    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)\n",
        "    # For multiple models, it points to the folder containing all deployed models (./azureml-models)\n",
        "    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_regression_model.pkl')\n",
        "    \n",
        "    # deserialize the model file back into a sklearn model\n",
        "    model = joblib.load(model_path)\n",
        "  \n",
        "\n",
        "# note you can pass in multiple rows for scoring\n",
        "def run(raw_data):\n",
        "    try:\n",
        "        data = json.loads(raw_data)['data']\n",
        "        data = numpy.array(data)\n",
        "        result = model.predict(data)\n",
        "        print (\"Prediction created\" + time.strftime(\"%H:%M:%S\"))\n",
        "        # you can return any datatype as long as it is JSON-serializable\n",
        "        return result.tolist()\n",
        "    except Exception as e:\n",
        "        error = str(e)\n",
        "        print (error + time.strftime(\"%H:%M:%S\"))\n",
        "        return error"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 5. *Create myenv.yml file*\n",
        "Please note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.conda_dependencies import CondaDependencies \n",
        "\n",
        "myenv = CondaDependencies.create(conda_packages=['numpy','scikit-learn'],\n",
        "                                 pip_packages=['azureml-defaults'])\n",
        "\n",
        "with open(\"myenv.yml\",\"w\") as f:\n",
        "    f.write(myenv.serialize_to_string())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 6. Create Inference Configuration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.model import InferenceConfig\n",
        "from azureml.core.environment import Environment\n",
        "\n",
        "\n",
        "myenv = Environment.from_conda_specification(name=\"myenv\", file_path=\"myenv.yml\")\n",
        "inference_config = InferenceConfig(entry_script=\"score.py\", environment=myenv)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Deploy to ACI (Optional)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.webservice import AciWebservice\n",
        "\n",
        "aci_deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, \n",
        "                                               memory_gb = 1, \n",
        "                                               tags = {'area': \"diabetes\", 'type': \"regression\"}, \n",
        "                                               description = 'Predict diabetes using regression model',\n",
        "                                               enable_app_insights = True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.webservice import Webservice\n",
        "\n",
        "aci_service_name = 'my-aci-service-4'\n",
        "aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aci_deployment_config)\n",
        "aci_service.wait_for_deployment(True)\n",
        "print(aci_service.state)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "test_sample = json.dumps({'data': [\n",
        "    [1,28,13,45,54,6,57,8,8,10], \n",
        "    [101,9,8,37,6,45,4,3,2,41]\n",
        "]})\n",
        "test_sample = bytes(test_sample,encoding='utf8')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "if aci_service.state == \"Healthy\":\n",
        "    prediction = aci_service.run(input_data=test_sample)\n",
        "    print(prediction)\n",
        "else:\n",
        "    raise ValueError(\"Service deployment isn't healthy, can't call the service. Error: \", aci_service.error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 7. Deploy to AKS service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Create AKS compute if you haven't done so."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Use the default configuration (can also provide parameters to customize)\n",
        "prov_config = AksCompute.provisioning_configuration()\n",
        "\n",
        "aks_name = 'my-aks-test3' \n",
        "# Create the cluster\n",
        "aks_target = ComputeTarget.create(workspace = ws, \n",
        "                                  name = aks_name, \n",
        "                                  provisioning_configuration = prov_config)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "aks_target.wait_for_completion(show_output = True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(aks_target.provisioning_state)\n",
        "print(aks_target.provisioning_errors)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "If you already have a cluster you can attach the service to it:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```python \n",
        "%%time\n",
        "resource_id = '/subscriptions/<subscriptionid>/resourcegroups/<resourcegroupname>/providers/Microsoft.ContainerService/managedClusters/<aksservername>'\n",
        "create_name= 'myaks4'\n",
        "attach_config = AksCompute.attach_configuration(resource_id=resource_id)\n",
        "aks_target = ComputeTarget.attach(workspace = ws, \n",
        "                                  name = create_name, \n",
        "                                  attach_configuration=attach_config)\n",
        "## Wait for the operation to complete\n",
        "aks_target.wait_for_provisioning(True)```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### a. *Activate App Insights through updating AKS Webservice configuration*\n",
        "In order to enable App Insights in your service you will need to update your AKS configuration file:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "#Set the web service configuration\n",
        "aks_deployment_config = AksWebservice.deploy_configuration(enable_app_insights=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### b. Deploy your service"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "if aks_target.provisioning_state== \"Succeeded\": \n",
        "    aks_service_name ='aks-w-dc5'\n",
        "    aks_service = Model.deploy(ws,\n",
        "                               aks_service_name, \n",
        "                               [model], \n",
        "                               inference_config, \n",
        "                               aks_deployment_config, \n",
        "                               deployment_target = aks_target)    \n",
        "    aks_service.wait_for_deployment(show_output = True)\n",
        "    print(aks_service.state)\n",
        "else:\n",
        "    raise ValueError(\"AKS provisioning failed. Error: \", aks_service.error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 8. Test your service "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "test_sample = json.dumps({'data': [\n",
        "    [1,28,13,45,54,6,57,8,8,10], \n",
        "    [101,9,8,37,6,45,4,3,2,41]\n",
        "]})\n",
        "test_sample = bytes(test_sample,encoding='utf8')\n",
        "\n",
        "if aks_service.state == \"Healthy\":\n",
        "    prediction = aks_service.run(input_data=test_sample)\n",
        "    print(prediction)\n",
        "else:\n",
        "    raise ValueError(\"Service deployment isn't healthy, can't call the service. Error: \", aks_service.error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 9. See your service telemetry in App Insights\n",
        "1. Go to the [Azure Portal](https://portal.azure.com/)\n",
        "2. All resources--> Select the subscription/resource group where you created your Workspace--> Select the App Insights type\n",
        "3. Click on the AppInsights resource. You'll see a highlevel dashboard with information on Requests, Server response time and availability.\n",
        "4. Click on the top banner \"Analytics\"\n",
        "5. In the \"Schema\" section select \"traces\" and run your query.\n",
        "6. Voila! All your custom traces should be there."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Disable App Insights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "aks_service.update(enable_app_insights=False)\n",
        "aks_service.wait_for_deployment(show_output = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Clean up"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "aks_service.delete()\n",
        "aci_service.delete()\n",
        "model.delete()"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "shipatel"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}