| title | titleSuffix | description | services | ms.service | ms.subservice | author | ms.author | ms.reviewer | ms.date | ms.topic | ms.custom |
|---|---|---|---|---|---|---|---|---|---|---|---|
Access Azure resources from an online endpoint |
Azure Machine Learning |
Securely access Azure resources for your machine learning model deployment from an online endpoint with a system-assigned or user-assigned managed identity. |
machine-learning |
machine-learning |
inferencing |
msakande |
mopeakande |
sehan |
03/18/2024 |
how-to |
devplatv2, cliv2 |
[!INCLUDE dev v2]
Learn how to access Azure resources from your scoring script with an online endpoint and either a system-assigned managed identity or a user-assigned managed identity.
Both managed endpoints and Kubernetes endpoints allow Azure Machine Learning to manage the burden of provisioning your compute resource and deploying your machine learning model. Typically your model needs to access Azure resources such as the Azure Container Registry or your blob storage for inferencing; with a managed identity, you can access these resources without needing to manage credentials in your code. Learn more about managed identities.
This guide assumes you don't have a managed identity, a storage account, or an online endpoint. If you already have these components, skip to the Give access permission to the managed identity section.
-
To use Azure Machine Learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
Install and configure the Azure CLI and ML (v2) extension. For more information, see Install, set up, and use the 2.0 CLI.
-
An Azure resource group, in which you (or the service principal you use) need to have User Access Administrator and Contributor access. You have such a resource group if you configured your ML extension per the preceding article.
-
An Azure Machine Learning workspace. You already have a workspace if you configured your ML extension per the preceding article.
-
A trained machine learning model ready for scoring and deployment. If you're following along with the sample, a model is provided.
-
If you haven't already set the defaults for the Azure CLI, save your default settings. To avoid passing in the values for your subscription, workspace, and resource group multiple times, run this code:
az account set --subscription <subscription ID> az configure --defaults gitworkspace=<Azure Machine Learning workspace name> group=<resource group> -
To follow along with the sample, clone the samples repository and then change directory to cli.
git clone https://github.com/Azure/azureml-examples --depth 1 cd azureml-examples/cli
-
To use Azure Machine Learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
Install and configure the Azure CLI and ML (v2) extension. For more information, see Install, set up, and use the 2.0 CLI.
-
An Azure Resource group, in which you (or the service principal you use) need to have User Access Administrator and Contributor access. You have such a resource group if you configured your ML extension per the preceding article.
-
An Azure Machine Learning workspace. You have a workspace if you configured your ML extension per the above article.
-
A trained machine learning model ready for scoring and deployment. If you're following along with the sample, a model is provided.
-
If you haven't already set the defaults for the Azure CLI, save your default settings. To avoid passing in the values for your subscription, workspace, and resource group multiple times, run this code:
az account set --subscription <subscription ID> az configure --defaults gitworkspace=<Azure Machine Learning workspace name> group=<resource group> -
To follow along with the sample, clone the samples repository and then change directory to cli.
git clone https://github.com/Azure/azureml-examples --depth 1 cd azureml-examples/cli
-
To use Azure Machine Learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
Install and configure the Azure Machine Learning Python SDK (v2). For more information, see Install and set up SDK (v2).
-
An Azure Resource group, in which you (or the service principal you use) need to have User Access Administrator and Contributor access. You have such a resource group if you configured your ML extension per the preceding article.
-
An Azure Machine Learning workspace. You already have a workspace if you configured your ML extension per the preceding article.
-
A trained machine learning model ready for scoring and deployment. If you're following along with the sample, a model is provided.
-
Clone the samples repository, then change the directory.
git clone https://github.com/Azure/azureml-examples --depth 1 cd azureml-examples/sdk/endpoints/online/managed/managed-identities -
To follow along with this notebook, access the companion example notebook within in the sdk/endpoints/online/managed/managed-identities directory.
-
Other Python packages are required for this example:
- Microsoft Azure Storage Management Client
- Microsoft Azure Authorization Management Client
Install them with the following code:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=install-packages)]
-
To use Azure Machine Learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
Role creation permissions for your subscription or the Azure resources accessed by the user-assigned identity.
-
Install and configure the Azure Machine Learning Python SDK (v2). For more information, see Install and set up SDK (v2).
-
An Azure Resource group, in which you (or the service principal you use) need to have User Access Administrator and Contributor access. You have such a resource group if you configured your ML extension per the preceding article.
-
An Azure Machine Learning workspace. You already have a workspace if you configured your ML extension per the preceding article.
-
A trained machine learning model ready for scoring and deployment. If you're following along with the sample, a model is provided.
-
Clone the samples repository.
git clone https://github.com/Azure/azureml-examples --depth 1 cd azureml-examples/sdk/endpoints/online/managed/managed-identities -
To follow along with this notebook, access the companion example notebook within in the sdk/endpoints/online/managed/managed-identities directory.
-
Other Python packages are required for this example:
- Microsoft Azure MSI Management Client
- Microsoft Azure Storage Client
- Microsoft Azure Authorization Management Client
Install them with the following code:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=install-packages)]
- The identity for an endpoint is immutable. During endpoint creation, you can associate it with a system-assigned identity (default) or a user-assigned identity. You can't change the identity after the endpoint is created.
- If your ARC and blob storage are configured as private, that is, behind a virtual network, then access from the Kubernetes endpoint should be over the private link regardless of whether your workspace is public or private. More details about private link setting, refer to How to secure workspace vnet.
Configure the variable names for the workspace, workspace location, and the endpoint you want to create for use with your deployment.
The following code exports these values as environment variables in your endpoint:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="set_variables" :::
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in az storage account create and az storage container create commands in the next section.
The following code exports those values as environment variables:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="configure_storage_names" :::
After these variables are exported, create a text file locally. When the endpoint is deployed, the scoring script accesses this text file using the system-assigned managed identity that's generated upon endpoint creation.
Decide on the name of your endpoint, workspace, and workspace location, then export that value as an environment variable:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="set_variables" :::
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in az storage account create and az storage container create commands in the next section.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="configure_storage_names" :::
After these variables are exported, create a text file locally. When the endpoint is deployed, the scoring script accesses this text file using the user-assigned managed identity used in the endpoint.
Decide on the name of your user identity name, and export that value as an environment variable:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="set_user_identity_name" :::
Assign values for the workspace and deployment-related variables:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=1-assign-variables)]
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in the storage account and container creation code by the StorageManagementClient and ContainerClient.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=1-specify-storage-details)]
After these variables are assigned, create a text file locally. When the endpoint is deployed, the scoring script accesses this text file using the system-assigned managed identity that's generated upon endpoint creation.
Now, get a handle to the workspace and retrieve its location:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=1-retrieve-workspace-location)]
Use this value to create a storage account.
Assign values for the workspace and deployment-related variables:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=1-assign-variables)]
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in the storage account and container creation code by the StorageManagementClient and ContainerClient.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=1-specify-storage-details)]
After these variables are assigned, create a text file locally. When the endpoint is deployed, the scoring script accesses this text file using the user-assigned managed identity that's generated upon endpoint creation.
Decide on the name of your user identity name: [!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=1-decide-name-user-identity)]
Now, get a handle to the workspace and retrieve its location:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=1-retrieve-workspace-location)]
Use this value to create a storage account.
To deploy an online endpoint with the CLI, you need to define the configuration in a YAML file. For more information on the YAML schema, see online endpoint YAML reference document.
The YAML files in the following examples are used to create online endpoints.
The following YAML example is located at endpoints/online/managed/managed-identities/1-sai-create-endpoint. The file,
- Defines the name by which you want to refer to the endpoint,
my-sai-endpoint. - Specifies the type of authorization to use to access the endpoint,
auth-mode: key.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/1-sai-create-endpoint.yml":::
This YAML example, 2-sai-deployment.yml,
- Specifies that the type of endpoint you want to create is an
onlineendpoint. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as, which model to deploy and which environment and scoring script to use.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/2-sai-deployment.yml":::
To deploy an online endpoint with the CLI, you need to define the configuration in a YAML file. For more information on the YAML schema, see online endpoint YAML reference document.
The YAML files in the following examples are used to create online endpoints.
The following YAML example is located at endpoints/online/managed/managed-identities/1-uai-create-endpoint. The file,
- Defines the name by which you want to refer to the endpoint,
my-uai-endpoint. - Specifies the type of authorization to use to access the endpoint,
auth-mode: key. - Indicates the identity type to use,
type: user_assigned
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/1-uai-create-endpoint.yml":::
This YAML example, 2-sai-deployment.yml,
- Specifies that the type of endpoint you want to create is an
onlineendpoint. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as, which model to deploy and which environment and scoring script to use.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/2-uai-deployment.yml":::
To deploy an online endpoint with the Python SDK (v2), objects can be used to define the following configuration. Alternatively, YAML files can be loaded using the .load method.
The following Python endpoint object:
- Assigns the name by which you want to refer to the endpoint to the variable
endpoint_name. - Specifies the type of authorization to use to access the endpoint
auth-mode="key".
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=2-define-endpoint-configuration)]
This deployment object:
- Specifies that the type of deployment you want to create is a
ManagedOnlineDeploymentvia the class. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as the
nameandinstance_count - Defines extra objects inline and associates them with the deployment for
Model,CodeConfiguration, andEnvironment. - Includes environment variables needed for the system-assigned managed identity to access storage.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=2-define-deployment-configuration)]
To deploy an online endpoint with the Python SDK (v2), objects can be used to define the following configuration. Alternatively, YAML files can be loaded using the .load method.
For a user-assigned identity, you define the endpoint configuration after the user-assigned managed identity is created.
This deployment object:
- Specifies that the type of deployment you want to create is a
ManagedOnlineDeploymentvia the class. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as the
nameandinstance_count - Defines more objects inline and associates them with the deployment for
Model,CodeConfiguration, andEnvironment. - Includes environment variables needed for the user-assigned managed identity to access storage.
- Adds a placeholder environment variable for
UAI_CLIENT_ID, which is added after creating one and before actually deploying this configuration.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=2-define-deployment-configuration)]
To access Azure resources, create a system-assigned or user-assigned managed identity for your online endpoint.
When you create an online endpoint, a system-assigned managed identity is automatically generated for you, so no need to create a separate one.
To create a user-assigned managed identity, use the following command:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_user_identity" :::
When you create an online endpoint, a system-assigned managed identity is automatically generated for you, so no need to create a separate one.
To create a user-assigned managed identity, first get a handle to the ManagedServiceIdentityClient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=3-get-handle)]
Then, create the identity:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=3-create-identity)]
Now, retrieve the identity object, which contains details you use:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=3-retrieve-identity-object)]
For this example, create a blob storage account and blob container, and then upload the previously created text file to the blob container. You give the online endpoint and managed identity access to this storage account and blob container.
First, create a storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_storage_account" :::
Next, create the blob container in the storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_storage_container" :::
Then, upload your text file to the blob container.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="upload_file_to_storage" :::
First, create a storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_storage_account" :::
You can also retrieve an existing storage account ID with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_storage_account_id" :::
Next, create the blob container in the storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_storage_container" :::
Then, upload file in container.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="upload_file_to_storage" :::
First, get a handle to the StorageManagementclient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=4-get-handle)]
Then, create a storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=4-create-storage-account)]
Next, create the blob container in the storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=4-create-blob-container)]
Retrieve the storage account key and create a handle to the container with ContainerClient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=4-create-container-client)]
Then, upload a blob to the container with the ContainerClient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=4-upload-blob)]
First, get a handle to the StorageManagementclient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=4-get-handle)]
Then, create a storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=4-create-storage-account)]
Next, create the blob container in the storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=4-create-blob-container)]
Retrieve the storage account key and create a handle to the container with ContainerClient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=4-create-container-client)]
Then, upload a blob to the container with the ContainerClient:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=4-upload-blob)]
The following code creates an online endpoint without specifying a deployment.
Warning
The identity for an endpoint is immutable. During endpoint creation, you can associate it with a system-assigned identity (default) or a user-assigned identity. You can't change the identity after the endpoint has been created.
When you create an online endpoint, a system-assigned managed identity is created for the endpoint by default.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_endpoint" :::
Check the status of the endpoint with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_endpoint_Status" :::
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_endpoint" :::
Check the status of the endpoint with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_endpoint_Status" :::
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
When you create an online endpoint, a system-assigned managed identity is created for the endpoint by default.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=5-create-online-endpoint)]
Check the status of the endpoint via the details of the deployed endpoint object with the following code:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=5-get-details)]
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
The following Python endpoint object:
- Assigns the name by which you want to refer to the endpoint to the variable
endpoint_name. - Specifies the type of authorization to use to access the endpoint
auth-mode="key". - Defines its identity as a
ManagedServiceIdentityand specifies the Managed Identity as user assigned.
Define and deploy the endpoint:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=5-create-online-endpoint)]
Check the status of the endpoint via the details of the deployed endpoint object with the following code:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=5-get-details)]
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
Important
Online endpoints require Azure Container Registry pull permission, AcrPull permission, to the container registry and Storage Blob Data Reader permission to the default datastore of the workspace.
You can allow the online endpoint permission to access your storage via its system-assigned managed identity or give permission to the user-assigned managed identity to access the storage account created in the previous section.
Retrieve the system-assigned managed identity that was created for your endpoint.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="get_system_identity" :::
From here, you can give the system-assigned managed identity permission to access your storage.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="give_permission_to_user_storage_account" :::
Retrieve user-assigned managed identity client ID.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_user_identity_client_id" :::
Retrieve the user-assigned managed identity ID.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_user_identity_id" :::
Get the container registry associated with workspace.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_container_registry_id" :::
Retrieve the default storage of the workspace.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_workspace_storage_id" :::
Give permission of storage account to the user-assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_user_storage_account" :::
Give permission of container registry to user assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_container_registry" :::
Give permission of default workspace storage to user-assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_workspace_storage_account" :::
First, make an AuthorizationManagementClient to list role definitions:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=6-get-role-definitions-client)]
Now, initialize one to make role assignments:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=6-get-role-assignments-client)]
Then, get the principal ID of the system-assigned managed identity:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=6-get-sai-details)]
Next, assign the Storage Blob Data Reader role to the endpoint. The role definition is retrieved by name and passed along with the Principal ID of the endpoint. The role is applied at the scope of the storage account created above and allows the endpoint to read the file.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=6-give-permission-user-storage-account)]
First, make an AuthorizationManagementClient to list role definitions:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-get-role-definitions-client)]
Now, initialize one to make role assignments:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-get-role-assignments-client)]
Then, get the principal ID and client ID of the user-assigned managed identity. To assign roles, you only need the principal ID. However, you use the client ID to fill the UAI_CLIENT_ID placeholder environment variable before creating the deployment.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-get-uai-details)]
Next, assign the Storage Blob Data Reader role to the endpoint. The role definition is retrieved by name and passed along with the principal ID of the endpoint. The role is applied at the scope of the storage account created above to allow the endpoint to read the file.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-give-permission-user-storage-account)]
For the next two permissions, you need the workspace and container registry objects:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-retrieve-workspace-acr)]
Next, assign the AcrPull role to the user-assigned identity. This role allows images to be pulled from an Azure Container Registry. The scope is applied at the level of the container registry associated with the workspace.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-give-permission-container-registry)]
Finally, assign the Storage Blob Data Reader role to the endpoint at the workspace storage account scope. This role assignment allows the endpoint to read blobs in the workspace storage account as well as the newly created storage account.
The role has the same name and capabilities as the first role assigned above, however it is applied at a different scope and has a different ID.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=6-give-permission-workspace-storage)]
Refer to the following script to understand how to use your identity token to access Azure resources, in this scenario, the storage account created in previous sections.
:::code language="python" source="~/azureml-examples-main/cli/endpoints/online/model-1/onlinescoring/score_managedidentity.py":::
Create a deployment that's associated with the online endpoint. Learn more about deploying to online endpoints.
Warning
This deployment can take approximately 8-14 minutes depending on whether the underlying environment/image is being built for the first time. Subsequent deployments using the same environment will go quicker.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="deploy" :::
Note
The value of the --name argument may override the name key inside the YAML file.
Check the status of the deployment.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_deploy_Status" :::
To refine the above query to only return specific data, see Query Azure CLI command output.
Note
The init method in the scoring script reads the file from your storage account using the system-assigned managed identity token.
To check the init method output, see the deployment log with the following code.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_deployment_log" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_endpoint" :::
Note
The value of the --name argument may override the name key inside the YAML file.
Once the command executes, you can check the status of the deployment.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_endpoint_Status" :::
To refine the above query to only return specific data, see Query Azure CLI command output.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_deployment_log" :::
Note
The init method in the scoring script reads the file from your storage account using the user-assigned managed identity token.
To check the init method output, see the deployment log with the following code.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_deployment_log" :::
First, create the deployment:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=7-create-deployment)]
Once deployment completes, check its status and confirm its identity details:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=7-check-deployment-status)]
Note
The init method in the scoring script reads the file from your storage account using the system-assigned managed identity token.
To check the init method output, see the deployment log with the following code.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=7-get-deployment-logs)]
Now that the deployment is confirmed, set the traffic to 100%:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=7-set-traffic)]
Before you deploy, update the UAI_CLIENT_ID environment variable placeholder.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=7-update-uai-client-id)]
Now, create the deployment:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=7-create-deployment)]
Once deployment completes, check its status and confirm its identity details:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=7-check-deployment-status)]
Note
The init method in the scoring script reads the file from your storage account using the user-assigned managed identity token.
To check the init method output, see the deployment log with the following code.
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=7-get-deployment-logs)]
Now that the deployment is confirmed, set the traffic to 100%:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=7-set-traffic)]
When your deployment completes, the model, the environment, and the endpoint are registered to your Azure Machine Learning workspace.
Once your online endpoint is deployed, test and confirm its operation with a request. Details of inferencing vary from model to model. For this guide, the JSON query parameters look like:
:::code language="json" source="~/azureml-examples-main/cli/endpoints/online/model-1/sample-request.json" :::
To call your endpoint, run:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="test_endpoint" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="test_endpoint" :::
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=8-confirm-endpoint-deployed-successfully)]
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=8-confirm-endpoint-deployed-successfully)]
If you don't plan to continue using the deployed online endpoint and storage, delete them to reduce costs. When you delete the endpoint, all of its associated deployments are deleted as well.
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="delete_endpoint" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="delete_storage_account" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_endpoint" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_storage_account" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_user_identity" :::
Delete the endpoint:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=9-delete-endpoint)]
Delete the storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-sai.ipynb?name=9-delete-storage-account)]
Delete the endpoint:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=9-delete-endpoint)]
Delete the storage account:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=9-delete-storage-account)]
Delete the User-assigned managed identity:
[!notebook-python[] (~/azureml-examples-main/sdk/python/endpoints/online/managed/managed-identities/online-endpoints-managed-identity-uai.ipynb?name=9-delete-uai)]
- Deploy and score a machine learning model by using an online endpoint
- Perform safe rollout of new deployments for real-time inference
- Install and set up the CLI (v2)
- Managed online endpoints SKU list
- View costs for an Azure Machine Learning managed online endpoint
- Monitor online endpoints
- For limitations of managed online endpoint and Kubernetes online endpoint, see limits for online endpoints