Skip to content

Latest commit

 

History

History
143 lines (91 loc) · 6.86 KB

how-to-use-mlflow-configure-tracking.md

File metadata and controls

143 lines (91 loc) · 6.86 KB
Raw
title titleSuffix description services author ms.author ms.reviewer ms.service ms.subservice ms.date ms.topic ms.custom ms.devlang
Configure MLflow for Azure Machine Learning
Azure Machine Learning
Connect MLflow to Azure Machine Learning workspaces to log metrics, artifacts, and deploy models.
machine-learning
santiagxf
fasantia
mopeakande
machine-learning
mlops
01/19/2024
how-to
mlflow, cliv2, devplatv2
azurecli

Configure MLflow for Azure Machine Learning

This article explains how you can configure MLflow to connect to an Azure Machine Learning workspace for tracking, registries, and deployment.

Azure Machine Learning workspaces are MLflow-compatible, which means they can act as an MLflow server without any extra configuration. Each workspace has an MLflow tracking URI that MLflow can use to connect to the workspace. Azure Machine Learning workspaces are already configured to work with MLflow so no extra configuration is required.

However, if you work outside of Azure Machine Learning (like your local machine, Azure Synapse Analytics, or Azure Databricks), you need to configure MLflow to point to the workspace.

Important

When running on Azure Compute (Azure Machine Learning Notebooks, Jupyter notebooks hosted on Azure Machine Learning compute instances, or jobs running on Azure Machine Learning compute clusters), you don't have to configure the tracking URI. It's automatically configured for you.

Prerequisites

You need the following prerequisites to follow this tutorial:

[!INCLUDE mlflow-prereqs]

Configure MLflow tracking URI

To connect MLflow to an Azure Machine Learning workspace, you need the tracking URI for the workspace. Each workspace has its own tracking URI and it has the protocol azureml://.

[!INCLUDE mlflow-configure-tracking]

Configure authentication

Once the tracking is set, you also need to configure the authentication method for the associated workspace. By default, the Azure Machine Learning plugin for MLflow performs interactive authentication by opening the default browser to prompt for credentials.

The Azure Machine Learning plugin for MLflow supports several authentication mechanisms through the package azure-identity, which is installed as a dependency for the plugin azureml-mlflow. The following authentication methods are tried one by one until one of them succeeds:

  1. Environment: Reads account information specified via environment variables and uses it to authenticate.
  2. Managed Identity: If the application is deployed to an Azure host with Managed Identity enabled, it authenticates with it.
  3. Azure CLI: If a user signs in via the Azure CLI az login command, it authenticates as that user.
  4. Azure PowerShell: If a user signs in via Azure PowerShell's Connect-AzAccount command, it authenticates as that user.
  5. Interactive browser: Interactively authenticates a user via the default browser.

[!INCLUDE mlflow-configure-auth]

If you'd rather use a certificate instead of a secret, you can configure the environment variables AZURE_CLIENT_CERTIFICATE_PATH to the path to a PEM or PKCS12 certificate file (including private key) and AZURE_CLIENT_CERTIFICATE_PASSWORD with the password of the certificate file, if any.

Configure authorization and permission levels

Some default roles like AzureML Data Scientist or Contributor are already configured to perform MLflow operations in an Azure Machine Learning workspace. If using a custom role, you need the following permissions:

  • To use MLflow tracking:

    • Microsoft.MachineLearningServices/workspaces/experiments/*
    • Microsoft.MachineLearningServices/workspaces/jobs/*

  • To use MLflow model registry:

    • Microsoft.MachineLearningServices/workspaces/models/*/*

To learn how to grant access for the service principal you created or user account to your workspace, see Grant access.

Troubleshooting authentication

MLflow tries to authenticate to Azure Machine Learning on the first operation that interacts with the service, like mlflow.set_experiment() or mlflow.start_run(). If you find issues or unexpected authentication prompts during the process, you can increase the logging level to get more details about the error:

import logging

logging.getLogger("azure").setLevel(logging.DEBUG)

Set experiment name (optional)

All MLflow runs are logged to the active experiment. By default, runs are logged to an experiment named Default that is automatically created for you. You can configure the experiment where tracking is happening.

Tip

When submitting jobs using Azure Machine Learning CLI v2, you can set the experiment name using the property experiment_name in the YAML definition of the job. You don't have to configure it on your training script. See YAML: display name, experiment name, description, and tags for details.

MLflow SDK

Configure your experiment by using MLflow command mlflow.set_experiment().

experiment_name = 'experiment_with_mlflow'
mlflow.set_experiment(experiment_name)

Using environment variables

You can also set one of the MLflow environment variables MLFLOW_EXPERIMENT_NAME or MLFLOW_EXPERIMENT_ID with the experiment name.

export MLFLOW_EXPERIMENT_NAME="experiment_with_mlflow"

Nonpublic Azure Clouds support

The Azure Machine Learning plugin for MLflow is configured by default to work with the global Azure cloud. However, you can configure the Azure cloud you're using by setting the environment variable AZUREML_CURRENT_CLOUD.

MLflow SDK

import os

os.environ["AZUREML_CURRENT_CLOUD"] = "AzureChinaCloud"

Using environment variables

export AZUREML_CURRENT_CLOUD="AzureChinaCloud"

You can identify the cloud you're using with the following Azure CLI command:

az cloud list

The current cloud has the value IsActive set to True.

Next steps

Now that your environment is connected to your workspace in Azure Machine Learning, you can start to work with it.