sdk/python/using-mlflow/deploy/track_with_databricks_deploy_aml.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "08921861-db40-4ad7-8ecf-40792023a9ee",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "# Training models in Azure Databricks and deploying them on Azure ML\n",
    "\n",
    "This notebook demostrates how to train models in Azure Databricks (or any Databricks implementation) and deploying those models on Azure ML. Two workflows are demostrated here depending on the level of integration you want to keep and how you want to do tracking:\n",
    "\n",
    "1. **Scenario 1: Training on Azure Databricks while tracking experiments and models in Azure ML:** This example shows how to do training of models in Azure Databricks while doing all the tracking of experiments in Azure ML (instead of in the MLflow instance running on Azure Databricks). This will also allow you to seemessly deploy models to Azure ML deployment targets in the easiest way.\n",
    "2. **Scenario 2: Training and tracking experiments in Azure Databricks with Model Registries in Azure ML:** This example shows how to do training and tracking of models in Azure Databricks. Tracking of experiments happens here in the MLflow instance running on Azure Databricks. However, model registries are kept on Azure ML to allow quick model's deployment from a centralized location and registry of models.\n",
    "\n",
    "Read each scenario to know more about advantages and disadvantages of each approach."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "5354b299-1ba4-4d79-8f1a-6bb7e4d30de7",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## Before starting\n",
    "\n",
    "To run this notebook ensure you have:\n",
    "- A Databricks workspace with a compute with the following libraries:\n",
    "  - xgboost\n",
    "  - scikit-learn==1.1.1\n",
    "  - pandas\n",
    "  - numpy\n",
    "  - mlflow\n",
    "  - azureml-mlflow\n",
    "\n",
    "Also, configure the following variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "faa070f5-4945-44b2-898e-85cc5b113d88",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "workspace = \"<AML_WORKSPACE_NAME>\"\n",
    "adb_user_id = \"<ADB_USER_ID>\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need to connect MLflow to the Azure Machine Learning workspace you want to work on. MLflow uses the tracking URI to indicate the MLflow server you want to connect to. There are multiple ways to get the Azure Machine Learning MLflow Tracking URI. In this tutorial we will use the Azure ML SDK for Python, but you can check [Set up tracking environment - Azure Machine Learning Docs](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-mlflow-cli-runs#set-up-tracking-environment) for more alternatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), subscription_id, resource_group, workspace\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the workspace object to get the tracking URI:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "azureml_tracking_uri = ml_client.workspaces.get(\n",
    "    ml_client.workspace_name\n",
    ").mlflow_tracking_uri"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "6af409d2-1023-435d-861a-b3a0e9cb2c52",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## Scenario 1: Training on Azure Databricks while tracking experiments and models in Azure ML"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this scenario, you want tracking and model registry to happen in Azure Machine Learning, however, you want to keep training models in Azure Databricks. To do that, we need to configure the tracking URI on each instance of Databricks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "mlflow.set_tracking_uri(azureml_tracking_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "fa79621d-3b10-478e-80ea-72fdb5cc986b",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "### Training a heart condition classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e77baa39-de8a-478a-90d0-6fdede1ca0f0",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Configuring the experiment\n",
    "\n",
    "Tracking of experiments will happen in Azure ML and hence we need to use the naming convention we generally use with MLflow. \n",
    "\n",
    ">Note that naming in Azure Databricks is different as you have to use the path to where the experiment will be saved. In Azure ML and in general MLflow this is not the case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "7eeaffc4-44de-4ce6-919a-9463d167d77c",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.set_experiment(experiment_name=\"heart-condition-classifier\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e81f7e6b-e573-4a45-875e-e05050aa7185",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "> **About authentication:** Interactive Authentication or Device Authentication will be triggered when you can `set_experiment`. This is used to authenticate against Azure Machine Learning and be able to call the tracking API. If you are executing the code in the context of a job where interactive authentication is not possible, see the example `notebooks/using-mlflow/train-with-mlflow/xgboost_service_principal.ipynb` for an example about how to use a Service Principal to authenticate against Azure Machine Learning and MLflow.\n",
    "\n",
    "Since all the tracking is happening in Azure ML, you can train and register models in the regular way you do with mlflow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "78130dc3-e9d0-4eb4-9d58-4a2284784e6d",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Exploring the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "3437300b-3620-4b7e-b694-1f42e80fa325",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "file_url = \"https://azuremlexampledata.blob.core.windows.net/data/heart-disease-uci/data/heart.csv\"\n",
    "df = pd.read_csv(file_url)\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "84b9c591-a2db-4711-91e6-fa0024c8b88e",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "As we can see, some of the variables are categorical. To make it simpler for our model to handle these values, let's use their encoded values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "80753b37-b6da-4ca8-931a-b71ac790b21c",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "df[\"thal\"] = df[\"thal\"].astype(\"category\").cat.codes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "eb4ee0a7-d736-4766-a166-c914fc3bed04",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Let's split our dataset in train and test, so we can assess the performance of the model without overfitting the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "140c57a5-b55d-403f-9780-f6e11f2eba38",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df.drop(\"target\", axis=1), df[\"target\"], test_size=0.3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "5bd85ef2-c6c2-40de-ab50-a4cba889505d",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Training a model\n",
    "\n",
    "We are going to use autologging capabilities in MLflow to track parameters and metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "1fec5718-48c0-4386-b225-ff0ea974ebfb",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.xgboost.autolog()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "67ced902-bdcf-4848-8ccf-29f55473e27a",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Let's create a simple classifier and train it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "7f9c318a-72c6-411b-90c0-df0304a0d989",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "from sklearn.metrics import accuracy_score, recall_score\n",
    "\n",
    "model = XGBClassifier(use_label_encoder=False, eval_metric=\"logloss\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "f986077b-3112-4df9-9027-d58011992277",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "with mlflow.start_run() as run:\n",
    "    model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)\n",
    "\n",
    "    y_pred = model.predict(X_test)\n",
    "\n",
    "    accuracy = accuracy_score(y_test, y_pred)\n",
    "    recall = recall_score(y_test, y_pred)\n",
    "\n",
    "    print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))\n",
    "    print(\"Recall: %.2f%%\" % (recall * 100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "1a8973a7-7bf7-4c43-9610-b0b229387c2b",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "### Registering the model in Azure ML\n",
    "\n",
    "Since our experiments are being tracked in Azure ML, we can simply register models in the registry like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "c5ebdf1d-fe07-46e3-b7c6-841f1b1e31ee",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.register_model(\n",
    "    model_uri=f\"runs:/{run.info.run_id}/model\", name=\"databricks-heart-classifier\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "0fb3fea1-58fc-4e09-8383-38b19720702a",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## Scenario 2: Training and tracking experiments in Azure Databricks with Model Registries in Azure ML\n",
    "\n",
    "In some cases you may want to keep doing tracking of experiments in the MLflow instance that comes with Azure Databricks. This is the case for instance of customers that were already using MLflow in Azure Databricks so they want to keep they existing experiments there. However, they may want to take adavantage of the deployment capabilities of Azure ML including managed inference solutions, no-code deployments, etc.\n",
    "\n",
    "In this cases, it is possible to keep tracking of experiments on Azure Databricks while keeping you model's registered and deployed in Azure ML. This example shows you how to achieve this configuration:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "4307ecd8-4813-4b74-b64b-56bf11371bdf",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "### Configuring models' registry\n",
    "\n",
    "MLflow allows you to segregate the instance where experiments are being tracked from the instance where models' are being tracked (or registered). The first one is referred to **Tracking URI** while the second one is referred as **Registry URI**. By default, both of them are set to the same value, and in Azure Databricks, both of them are set to \"databricks\" meaning that tracking and model registries will happen inside of the MLflow instance that Databricks runs for you.\n",
    "\n",
    "We are going to track the experiments in Azure Databricks, but model registries will be held in Azure ML. This will allow us to manage the model's lifecycle - including deployments - in Azure ML."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e8214ec7-c127-4291-96ce-26810cefadcc",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "mlflow.set_registry_uri(azureml_tracking_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "56e04276-eab2-406a-a0a1-d5712a0318f0",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Configuring the experiment\n",
    "\n",
    "Tracking of experiments will happen in Azure Datbricks and hence we need to use the naming we use here.  \n",
    "\n",
    ">Note that naming in Azure Databricks is different as you have to use the path to where the experiment will be saved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e898c1d0-71c8-47da-8e5e-32bb1775a946",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.set_experiment(\n",
    "    experiment_name=f\"/Users/{adb_user_id}/heart-condition-classifier\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "c081edac-92b9-4e5b-ba05-094778436ea1",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Exploring the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "61ab7440-df19-4291-8cce-8287a88d7d59",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "file_url = \"https://azuremlexampledata.blob.core.windows.net/data/heart-disease-uci/data/heart.csv\"\n",
    "df = pd.read_csv(file_url)\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "1f447164-f29e-4942-bb81-f2fcd3d81f20",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "As we can see, some of the variables are categorical. To make it simpler for our model to handle these values, let's use their encoded values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "639803b1-f5fa-4694-9b6d-5be901aa4a2e",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "df[\"thal\"] = df[\"thal\"].astype(\"category\").cat.codes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "01b9ea99-e3db-42ca-92cf-a5ebfe9c3f07",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Let's split our dataset in train and test, so we can assess the performance of the model without overfitting the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e3b187b1-3bb6-4ee8-a252-774df33ad7ea",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df.drop(\"target\", axis=1), df[\"target\"], test_size=0.3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "d17da7da-15a0-4d8d-ad9f-fc9ef33e7c3f",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Training a model\n",
    "\n",
    "We are going to use autologging capabilities in MLflow to track parameters and metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "396ab4ff-98e3-4af0-8e9f-03396f43c4eb",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.xgboost.autolog()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "ec878b29-2ca7-458c-a4b3-cbbd63e02b43",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Let's create a simple classifier and train it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "517c1a55-b89b-41a8-83d8-be312ffe348a",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "from sklearn.metrics import accuracy_score, recall_score\n",
    "\n",
    "model = XGBClassifier(use_label_encoder=False, eval_metric=\"logloss\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "e684009d-88fa-4b08-9848-cd74f8924e49",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "with mlflow.start_run() as run:\n",
    "    model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)\n",
    "\n",
    "    y_pred = model.predict(X_test)\n",
    "\n",
    "    accuracy = accuracy_score(y_test, y_pred)\n",
    "    recall = reca\n",
    "    ll_score(y_test, y_pred)\n",
    "\n",
    "    print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))\n",
    "    print(\"Recall: %.2f%%\" % (recall * 100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "61dc514c-6da7-4e33-b294-dfd18fe97a98",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "### Registering the model in Azure ML\n",
    "\n",
    "So far, our model is trained and tracked inside of the MLflow instance in Azure Databricks. Now we want to register this model in Azure ML to manage the life cicle there. However, if we try to register the model as we usually do using the sintax `mlflow.register_model(model_uri=f\"runs:/{run.info.run_id}/model\").` you will found an error. The reason why this is happening is related to where runs are being stored.\n",
    "\n",
    "Right now runs are being stored in Azure Databricks and models in Azure ML. If you try to create a registered model from a Run, Azure ML don't have any way to guess how to get access to the runs, that are stored in a different service. because of that, you can't use `runs:/` URI for registering models.\n",
    "\n",
    "To overcome this limitation, you have to register the model from the artifacts themselfs, which you can achieve by first downloading them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "14b15341-7465-4004-9acb-b6073543645b",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "client = mlflow.tracking.MlflowClient()\n",
    "model_path = client.download_artifacts(run.info.run_id, path=\"model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "341e38aa-b6f6-4900-9b2f-dbf0ebb292a3",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "`model_path` is a local path to the artifacts representing the MLmodel created. We can use this artifacts to register the model now:\n",
    "\n",
    "> **Important:** Note that doing this has some implications. Since Azure ML knows nothing about the run that generated this model, lineage is lost from this point on. You can, although, store the RUN ID that generated this model in a tag in the registry for your reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "538b04c0-5e2e-4670-ae12-2a69b22a932b",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.register_model(\n",
    "    model_uri=f\"file://{model_path}\", name=\"databricks-heart-classifier\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "23e80561-2471-42d6-b7ca-1a1be44033a0",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Notice in the instruction above how the protocol is now `file://` instead of `runs:/`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "d4a9e5f7-aff7-4568-90c9-f83b88a93763",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## Deploying models registered in Azure ML\n",
    "\n",
    "Once a model is registered in Azure ML, you can deploy them using either the UI interface in Azure ML Studio, the Azure ML CLI v2 from a console, or the azureml-mlflow plugin for MLflow. Use the approach it best suites your needs. Here we will demostrate how to do that using the MLflow deployment plugin.\n",
    "\n",
    "### Deploying models registered in Azure ML to Managed Inference\n",
    "\n",
    "To make the deployment happen, you will need a deployment client. Deployments can be generated using both the Python API for MLflow or MLflow CLI. In both cases, a JSON configuration file needs to be indicated with the details of the deployment you want to achieve. The full specification of this configuration can be found at [Managed online deployment schema (v2)](https://docs.microsoft.com/en-us/azure/machine-learning/reference-yaml-deployment-managed-online)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "8a8e71f9-7521-45cf-8a71-5a977882ff07",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "from mlflow.deployments import get_deploy_client\n",
    "\n",
    "# Create the deployment configuration.\n",
    "deploy_config = {\n",
    "    \"instance_type\": \"Standard_DS3_v2\",\n",
    "    \"instance_count\": 1,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "423d0819-2edb-4408-91cd-c2131dba5cff",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Write the deployment configuration into a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "6d9867ad-b3d6-4d61-9667-83143d91786f",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "deployment_config_path = \"deployment_config.json\"\n",
    "with open(deployment_config_path, \"w\") as outfile:\n",
    "    outfile.write(json.dumps(deploy_config))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "efb8e57b-c312-4bcb-b48b-39a739e38dbe",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Configuring the deployment client"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indicate to MLflow where we want to deploy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "bb2cb7a9-eb0d-4507-b70d-adaf05b320d3",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "client = get_deploy_client(azureml_tracking_uri)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indicate to MLflow from where the models need to be pulled from. Currently, the source and target URLs need to be the same:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "42ff35ff-62d6-4bc9-8d19-269a6d7efa67",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "mlflow.set_tracking_uri(azureml_tracking_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "63359765-c215-4328-8a2c-3687f8338d36",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "#### Deploying the model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MLflow requires the deployment configuration to be passed as a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config = {\"deploy-config-file\": deployment_config_path}\n",
    "model_name = \"databricks-heart-classifier\"\n",
    "model_version = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "inputWidgets": {},
     "nuid": "d2f4770b-b702-46c5-9e68-c737cf93e7ed",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "# define the model path and the name is the service name\n",
    "# if model is not registered, it gets registered automatically and a name is autogenerated using the \"name\" parameter below\n",
    "client.create_deployment(\n",
    "    model_uri=f\"models:/{model_name}/{model_version}\",\n",
    "    config=config,\n",
    "    name=\"mymodel-mir-deployment\",\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "application/vnd.databricks.v1+notebook": {
   "dashboards": [],
   "language": "python",
   "notebookMetadata": {
    "pythonIndentUnit": 2
   },
   "notebookName": "track_with_databrick_deploy_aml",
   "notebookOrigID": 1033315985576863,
   "widgets": {}
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK V2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}