From e96a8159ec0c62c16c661f73c8256384e590a78d Mon Sep 17 00:00:00 2001 From: SeokJin Han <4353157+dem108@users.noreply.github.com> Date: Thu, 30 Mar 2023 18:22:46 -0700 Subject: [PATCH] adjust order to align with CLI/Studio samples --- .../online-endpoints-simple-deployment.ipynb | 212 ++++++++---------- 1 file changed, 99 insertions(+), 113 deletions(-) diff --git a/sdk/python/endpoints/online/managed/online-endpoints-simple-deployment.ipynb b/sdk/python/endpoints/online/managed/online-endpoints-simple-deployment.ipynb index cd2c0a68d6..bef9ad1398 100644 --- a/sdk/python/endpoints/online/managed/online-endpoints-simple-deployment.ipynb +++ b/sdk/python/endpoints/online/managed/online-endpoints-simple-deployment.ipynb @@ -1,6 +1,7 @@ { "cells": [ { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -10,7 +11,7 @@ "\n", "Managed online endpoints help to deploy your ML models in a turnkey manner. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way. Managed online endpoints take care of serving, scaling, securing, and monitoring your models, freeing you from the overhead of setting up and managing the underlying infrastructure. \n", "\n", - "For more information, see [What are Azure Machine Learning endpoints?](https://docs.microsoft.com/azure/machine-learning/concept-endpoints)." + "For more information, see [What are Azure Machine Learning endpoints?](https://learn.microsoft.com/azure/machine-learning/concept-endpoints), and [Deploy an ML model with an online endpoint](https://learn.microsoft.com/azure/machine-learning/how-to-deploy-online-endpoints)." ] }, { @@ -118,16 +119,19 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 2. Create local endpoint and deployment\n", + "# 2. Define endpoint and deployment\n", "\n", - "## 2.1 Create local endpoint\n", + "## 2.1 Define the endpoint\n", "\n", - "The goal of a local endpoint deployment is to validate and debug your code and configuration before you deploy to Azure. Local deployment has the following limitations:\n", - "* Local endpoints *do not support* traffic rules, authentication, or probe settings.\n", - "* Local endpoints support only one deployment per endpoint" + "To define an endpoint, you need to specify:\n", + "\n", + "* Endpoint name: The name of the endpoint. It must be unique in the Azure region. For more information on the naming rules, see [managed online endpoint limits](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints).\n", + "* Authentication mode: The authentication method for the endpoint. Choose between key-based authentication and Azure Machine Learning token-based authentication. A key doesn't expire, but a token does expire. For more information on authenticating, see [Authenticate to an online endpoint](how-to-authenticate-online-endpoint.md).\n", + "* Optionally, you can add a description and tags to your endpoint." ] }, { @@ -136,50 +140,49 @@ "metadata": {}, "outputs": [], "source": [ - "# Creating a local endpoint\n", + "# Define an endpoint name\n", + "endpoint_name = \"my-endpoint\"\n", + "\n", + "# Example way to define a random name\n", "import datetime\n", "\n", - "local_endpoint_name = \"local-\" + datetime.datetime.now().strftime(\"%m%d%H%M%f\")\n", + "endpoint_name = \"endpt-\" + datetime.datetime.now().strftime(\"%m%d%H%M%f\")\n", "\n", "# create an online endpoint\n", "endpoint = ManagedOnlineEndpoint(\n", - " name=local_endpoint_name, description=\"this is a sample local endpoint\"\n", + " name = endpoint_name,\n", + " description = \"this is a sample online endpoint\",\n", + " auth_mode = \"key\",\n", + " tags = {\"foo\": \"bar\"},\n", ")" ] }, { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ml_client.online_endpoints.begin_create_or_update(endpoint, local=True)" - ] - }, - { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.2 Create local deployment\n", + "## 2.2 Define the deployment\n", "\n", - "The example contains all the files needed to deploy a model on an online endpoint. To deploy a model, you must have:\n", + "A deployment is a set of resources required for hosting the model that does the actual inferencing. To deploy a model, you must have:\n", "\n", - "* Model files (or the name and version of a model that's already registered in your workspace). In the example, we have a scikit-learn model that does regression.\n", - "* The code that's required to score the model. In this case, we have a score.py file.\n", - "* An environment in which your model runs. As you'll see, the environment might be a Docker image with Conda dependencies, or it might be a Dockerfile.\n", - "* Settings to specify the instance type and scaling capacity.\n", + "- Model files (or the name and version of a model that's already registered in your workspace). In the example, we have a scikit-learn model that does regression.\n", + "- A scoring script, that is, code that executes the model on a given input request. The scoring script receives data submitted to a deployed web service and passes it to the model. The script then executes the model and returns its response to the client. The scoring script is specific to your model and must understand the data that the model expects as input and returns as output. In this example, we have a *score.py* file.\n", + "- An environment in which your model runs. The environment can be a Docker image with Conda dependencies or a Dockerfile.\n", + "- Settings to specify the instance type and scaling capacity.\n", "\n", - "### Key aspects of deployment \n", + "The following table describes the key attributes of a deployment:\n", "\n", - "- `name` - Name of the deployment.\n", - "- `endpoint_name` - Name of the endpoint to create the deployment under.\n", - "- `model` - The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification.\n", - "- `environment` - The environment to use for the deployment. This value can be either a reference to an existing versioned environment in the workspace or an inline environment specification.\n", - "- `code_configuration` - the configuration for the source code and scoring script\n", - " - `path`- Path to the source code directory for scoring the model\n", - " - `scoring_script` - Relative path to the scoring file in the source code directory\n", - "- `instance_type` - The VM size to use for the deployment. For the list of supported sizes, see [Managed online endpoints SKU list](https://docs.microsoft.com/en-us/azure/machine-learning/reference-managed-online-endpoints-vm-sku-list).\n", - "- `instance_count` - The number of instances to use for the deployment" + "| Attribute | Description |\n", + "|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n", + "| Name | The name of the deployment. |\n", + "| Endpoint name | The name of the endpoint to create the deployment under. |\n", + "| Model | The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification. |\n", + "| Code path | The path to the directory on the local development environment that contains all the Python source code for scoring the model. You can use nested directories and packages. |\n", + "| Scoring script | The relative path to the scoring file in the source code directory. This Python code must have an `init()` function and a `run()` function. The `init()` function will be called after the model is created or updated (you can use it to cache the model in memory, for example). The `run()` function is called at every invocation of the endpoint to do the actual scoring and prediction. |\n", + "| Environment | The environment to host the model and code. This value can be either a reference to an existing versioned environment in the workspace or an inline environment specification. |\n", + "| Instance type | The VM size to use for the deployment. For the list of supported sizes, see [Managed online endpoints SKU list](reference-managed-online-endpoints-vm-sku-list.md). |\n", + "| Instance count | The number of instances to use for the deployment. Base the value on the workload you expect. For high availability, we recommend that you set the value to at least `3`. We reserve an extra 20% for performing upgrades. For more information, see [managed online endpoint quotas](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints). |" ] }, { @@ -196,7 +199,7 @@ "\n", "blue_deployment = ManagedOnlineDeployment(\n", " name=\"blue\",\n", - " endpoint_name=local_endpoint_name,\n", + " endpoint_name=endpoint_name,\n", " model=model,\n", " environment=env,\n", " code_configuration=CodeConfiguration(\n", @@ -208,23 +211,18 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ml_client.online_deployments.begin_create_or_update(\n", - " deployment=blue_deployment, local=True\n", - ")" - ] - }, - { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 3. Verify the local deployment succeeded\n", + "# 3. Create local endpoint and deployment\n", + "\n", + "## 3.1 Create local endpoint\n", "\n", - "## 3.1 Check the status to see whether the model was deployed without error" + "The goal of a local endpoint deployment is to validate and debug your code and configuration before you deploy to Azure. Local deployment has the following limitations:\n", + "* Local endpoints *do not support* traffic rules, authentication, or probe settings.\n", + "* Local endpoints support only one deployment per endpoint.\n", + "* They support local model files only. If you want to test registered models, first download them, then use `path` in the deployment definition to refer to the parent folder." ] }, { @@ -233,14 +231,17 @@ "metadata": {}, "outputs": [], "source": [ - "ml_client.online_endpoints.get(name=local_endpoint_name, local=True)" + "ml_client.online_endpoints.begin_create_or_update(endpoint, local=True)" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 3.2 Get logs" + "## 3.2 Create local deployment\n", + "\n", + "Now, create a deployment named `blue` under the endpoint." ] }, { @@ -249,8 +250,8 @@ "metadata": {}, "outputs": [], "source": [ - "ml_client.online_deployments.get_logs(\n", - " name=\"blue\", endpoint_name=local_endpoint_name, local=True, lines=50\n", + "ml_client.online_deployments.begin_create_or_update(\n", + " deployment=blue_deployment, local=True\n", ")" ] }, @@ -258,8 +259,17 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 3.3 Invoke the local endpoint\n", - "Invoke the endpoint to score the model by using the convenience command invoke and passing query parameters that are stored in a JSON file" + "The `local=True` flag directs the SDK to deploy the endpoint in the Docker environment." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 4. Verify the local deployment succeeded\n", + "\n", + "## 4.1 Check the status to see whether the model was deployed without error" ] }, { @@ -268,26 +278,15 @@ "metadata": {}, "outputs": [], "source": [ - "ml_client.online_endpoints.invoke(\n", - " endpoint_name=local_endpoint_name,\n", - " request_file=\"../model-1/sample-request.json\",\n", - " local=True,\n", - ")" + "ml_client.online_endpoints.get(name=endpoint_name, local=True)" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 4. Deploy your online endpoint to Azure\n", - "Next, deploy your online endpoint to Azure.\n", - "\n", - "## 4.1 Configure online endpoint\n", - "`endpoint_name`: The name of the endpoint. It must be unique in the Azure region. Naming rules are defined under [managed online endpoint limits](https://docs.microsoft.com/azure/machine-learning/how-to-manage-quotas#azure-machine-learning-managed-online-endpoints-preview).\n", - "\n", - "`auth_mode` : Use `key` for key-based authentication. Use `aml_token` for Azure Machine Learning token-based authentication. A `key` does not expire, but `aml_token` does expire. \n", - "\n", - "Optionally, you can add description, tags to your endpoint." + "## 4.2 Get logs" ] }, { @@ -296,27 +295,18 @@ "metadata": {}, "outputs": [], "source": [ - "# Creating a unique endpoint name with current datetime to avoid conflicts\n", - "import datetime\n", - "\n", - "online_endpoint_name = \"endpoint-\" + datetime.datetime.now().strftime(\"%m%d%H%M%f\")\n", - "\n", - "# create an online endpoint\n", - "endpoint = ManagedOnlineEndpoint(\n", - " name=online_endpoint_name,\n", - " description=\"this is a sample online endpoint\",\n", - " auth_mode=\"key\",\n", - " tags={\"foo\": \"bar\"},\n", + "ml_client.online_deployments.get_logs(\n", + " name=\"blue\", endpoint_name=endpoint_name, local=True, lines=50\n", ")" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 4.2 Create the endpoint\n", - "\n", - "Using the `MLClient` created earlier, we will now create the Endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues." + "## 4.3 Invoke the local endpoint\n", + "Invoke the endpoint to score the model by using the convenience command invoke and passing query parameters that are stored in a JSON file" ] }, { @@ -325,16 +315,23 @@ "metadata": {}, "outputs": [], "source": [ - "ml_client.online_endpoints.begin_create_or_update(endpoint).result()" + "ml_client.online_endpoints.invoke(\n", + " endpoint_name=endpoint_name,\n", + " request_file=\"../model-1/sample-request.json\",\n", + " local=True,\n", + ")" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 4.3 Configure online deployment\n", + "# 5. Deploy your online endpoint to Azure\n", + "Next, deploy your online endpoint to Azure.\n", "\n", - "A deployment is a set of resources required for hosting the model that does the actual inferencing. We will create a deployment for our endpoint using the `ManagedOnlineDeployment` class." + "## 5.1 Create the endpoint\n", + "Using the `endpoint` we defined earlier and the `MLClient` created earlier, we'll now create the endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues." ] }, { @@ -343,32 +340,17 @@ "metadata": {}, "outputs": [], "source": [ - "model = Model(path=\"../model-1/model/sklearn_regression_model.pkl\")\n", - "env = Environment(\n", - " conda_file=\"../model-1/environment/conda.yml\",\n", - " image=\"mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest\",\n", - ")\n", - "\n", - "blue_deployment = ManagedOnlineDeployment(\n", - " name=\"blue\",\n", - " endpoint_name=online_endpoint_name,\n", - " model=model,\n", - " environment=env,\n", - " code_configuration=CodeConfiguration(\n", - " code=\"../model-1/onlinescoring\", scoring_script=\"score.py\"\n", - " ),\n", - " instance_type=\"Standard_DS3_v2\",\n", - " instance_count=1,\n", - ")" + "ml_client.online_endpoints.begin_create_or_update(endpoint).result()" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 4.4 Create the deployment\n", + "## 5.2 Create the deployment\n", "\n", - "Using the `MLClient` created earlier, we will now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues." + "Using the `blue_deployment` that we defined earlier and the `MLClient` we created earlier, we'll now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues." ] }, { @@ -392,10 +374,11 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 5. Test the endpoint with sample data\n", + "# 6. Test the endpoint with sample data\n", "Using the `MLClient` created earlier, we will get a handle to the endpoint. The endpoint can be invoked using the `invoke` command with the following parameters:\n", "- `endpoint_name` - Name of the endpoint\n", "- `request_file` - File with request data\n", @@ -412,19 +395,20 @@ "source": [ "# test the blue deployment with some sample data\n", "ml_client.online_endpoints.invoke(\n", - " endpoint_name=online_endpoint_name,\n", + " endpoint_name=endpoint_name,\n", " deployment_name=\"blue\",\n", " request_file=\"../model-1/sample-request.json\",\n", ")" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 6. Managing endpoints and deployments\n", + "# 7. Managing endpoints and deployments\n", "\n", - "## 6.1 Get details of the endpoint" + "## 7.1 Get details of the endpoint" ] }, { @@ -434,7 +418,7 @@ "outputs": [], "source": [ "# Get the details for online endpoint\n", - "endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)\n", + "endpoint = ml_client.online_endpoints.get(name=endpoint_name)\n", "\n", "# existing traffic details\n", "print(endpoint.traffic)\n", @@ -444,10 +428,11 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "## 6.2 Get the logs for the new deployment\n", + "## 7.2 Get the logs for the new deployment\n", "Get the logs for the green deployment and verify as needed" ] }, @@ -458,15 +443,16 @@ "outputs": [], "source": [ "ml_client.online_deployments.get_logs(\n", - " name=\"blue\", endpoint_name=online_endpoint_name, lines=50\n", + " name=\"blue\", endpoint_name=endpoint_name, lines=50\n", ")" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "# 7. Delete the endpoint\n" + "# 8. Delete the endpoint\n" ] }, { @@ -475,7 +461,7 @@ "metadata": {}, "outputs": [], "source": [ - "ml_client.online_endpoints.begin_delete(name=online_endpoint_name)" + "ml_client.online_endpoints.begin_delete(name=endpoint_name)" ] } ],