Azure ML Online Endpoints model profiling

This repo shows the more extensive capabilities of using GitHub Actions with Azure Machine Learning to profile models for online inferencing using Online Endpoints.

Learn more about how to use online endpoints for online infernecing.

Getting started

1. Prerequisites

The following prerequisites are required to make this repository work:

• Azure subscription
• Owner of the Azure subscription
• Access to GitHub Actions

If you don’t have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.

2. Create repository

Fork this repo.

3. Setting up the required secrets

A service principal needs to be generated for authentication and getting access to your Azure subscription. We suggest adding a service principal with owner rights to a new resource group or to the one where you have deployed your existing Azure Machine Learning workspace. Just go to the Azure Portal to find the details of your resource group or workspace. Then start the Cloud CLI or install the Azure CLI on your computer and execute the following command to generate the required credentials:

# Replace {service-principal-name}, {subscription-id} and {resource-group} with your 
# Azure subscription id and resource group name and any name for your service principle
az ad sp create-for-rbac --name {service-principal-name} \
                         --role owner \
                         --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group}

This will generate the following JSON output:

{
  "clientId": "<GUID>",
  "clientSecret": "<GUID>",
  "subscriptionId": "<GUID>",
  "tenantId": "<GUID>",
  (...)
}

Add this JSON output as a secret with the name AZURE_CREDENTIALS in your GitHub repository:

To do so, click on the Settings tab in your repository, then click on Secrets and finally add the new secret with the name AZURE_CREDENTIALS to your repository.

Please follow this link for more details.

4. Define your workspace parameters

This example uses secrets to store your workspace parameters, please add SUBSCRIPTION_ID, AML_WORKSPACE and RESOURCE_GROUP as secrets in your GitHub repository.

5. Modify the code

Now you can start modifying the code in the code folder, so that your model and not the provided sample model gets deployed and profiled on Azure. Where required, modify the environment yaml so that the environment will have the correct packages installed in the conda environment for your inferencing.

6. Kickoff auto profile with multiple SKUs

Modify the profile GitHub action with desired list of SKUs in the format of ["sku:num_concurrent_requests", "sku:num_concurrent_requests"]. It will auto kick off the model deployment and profiling on the SKU.

Documentation

Code structure

File/folder	Description
code	Sample data science source code that will be submitted to Azure Machine Learning to deploy and profile machine learning models.
code/online-endpoint/model-1	Sample model, including model files, environment definition and scoring script.
code/online-endpoint/model-2	Sample model, including model files, environment definition and scoring script.
code/online-endpoint/blue-deployment.yml	Online deployment YML file to deploy the model.
code/online-endpoint/endpoint.yml	Online endpoint YML file to define the endpoint.
code/online-endpoint/sample-request.json	Sample request to test the online deployment.
code/profiling/create-online-endpoint.sh	Script to deploy the model to an online endpoint.
code/profiling/how-to-profile-online-endpoint.sh	Script to profile an online endpoint.
.github/workflows	Folder for GitHub workflows.
docs	Resources for this README.
CODE_OF_CONDUCT.md	Microsoft Open Source Code of Conduct.
LICENSE	The license for the sample.
README.md	This README file.
SECURITY.md	Microsoft Security README.