Serverless MLflow Azure Terraform Module

Module for deploying serverless MLflow instance on Azure, using Serverless SQL Server, Container App Service and Azure Blob Storage. Deployed instance has built-in authentication mechanism with support for Azure Active Directory or Google providers (see AUTHENTICATION section).

We help companies turn their data into assets

---

USAGE

data "azurerm_resource_group" "this" {
  name = "my-resource-group"
}

module "azure_mlfow" {
  source  = "../.."
  context = module.this.context
  resource_group = data.azurerm_resource_group.this.name
  auth = {
    type          = "azureActiveDirectory"
    client_secret = ""
    client_id     = ""
    azureActiveDirectory = {
      tenant_id = ""
    }
  }
}

AUTHENTICATION

Useful docs:

• https://docs.microsoft.com/en-us/azure/container-apps/authentication
• AD: https://docs.microsoft.com/en-us/azure/container-apps/authentication-azure-active-directory
• Google: https://docs.microsoft.com/en-us/azure/container-apps/authentication-google

As of 2022-08-11 the module only supports AD and Google providers.

Google

As far as I've tested, when using Google provider, Authorization: Bearer <token> header is not supported, so using this with MLflow will require implementing additional mlflow.request_header_provider.

1. Obtain Client ID and Client Secret from GCP OAuth Credentials.
2. Set the auth variable in the module:

  auth = {
    type = "google"
    client_secret = "<client secret>"
    client_id = "<client id>"
  }

3. Run terraform apply
4. Take the mlflow_tracking_uri output from the module and set appropriate callback in Google OAuth 2.0 config: <mlflow_tracking_uri>/.auth/login/google/callback
5. Save the settings.

User login

Users will login with standard redirect to Google page.

Server to server

Needs to use: https://docs.microsoft.com/en-us/azure/container-apps/authentication#client-directed-sign-in

1. Obtain the token

export T=$(curl -X POST -L https://<mlflow url on .azurecontainerapps.io>/.auth/login/google -d "{\"id_token\": \"$(gcloud auth print-identity-token)\"}" -H "Content-Type: application/json" | jq -r .authenticationToken )

2. Use the token

curl -L https://<mlflow url on .azurecontainerapps.io>/api/2.0/mlflow/experiments/list -H "X-Zumo-Auth: $T"

Azure Active Directory

1. Create App Registration in Azure Active Directory
2. Fill the name (for now, Redirect URL will be empty).
3. In the newly created app, open Certificates & secrets section.
4. In Client secrets tab, click New client secret and create one. Save Value (this value will be only displayed once!).
5. Set the auth variable in the module:

auth = {
  type = "azureActiveDirectory"
  client_secret = "<client secret>"
  client_id = "<Application (client) ID>"
  azureActiveDirectory = {
    tenant_id = "<Directory (tenant) ID>"
  }
}

6. Run terraform apply
7. Take the mlflow_tracking_uri output from the module
8. Open Authentication section, in Platform configurations click Add platform, select Web, fill in Redirect URIs with <mlflow_tracking_uri>/.auth/login/aad/callback, select ID tokens (used for implicit and hybrid flows) and click Configure.

User login

Users will login with standard redirect to Microsoft login page.

Server to server

1. Prepare:

• client id
• client secret
• tenant id

2. Obtain the token from Microsoft endpoint

export T=$(curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/token \
-d 'client_id=<client id>' \
-d 'grant_type=client_credentials' \
-d 'scope=<client id>%2f.default' \
-d 'client_secret=<client secret>' | jq -r .access_token) 

Note that scope has both <client id> and a suffix %2f.default which is a url encoded ./default.

3. Use the token

curl -L https://<mlflow url on .azurecontainerapps.io>/api/2.0/mlflow/experiments/list -H "Authorization: Bearer $T"

EXAMPLES

data "azurerm_resource_group" "this" {
  name = "my-resource-group"
}

module "azure_mlfow" {
  source  = "../.."
  context = module.this.context

  resource_group_name = data.azurerm_resource_group.this.name

  #  auth = {
  #    type = "google"
  #    client_secret = ""
  #    client_id = ""
  #  }

  auth = {
    type          = "azureActiveDirectory"
    client_secret = ""
    client_id     = ""
    azureActiveDirectory = {
      tenant_id = ""
    }
  }
}

Inputs

Name	Description	Type	Default	Required
additional_tag_map	Additional key-value pairs to add to each map in tags_as_list_of_maps. Not added to tags or id. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration.	map(string)	{}	no
attributes	ID element. Additional attributes (e.g. workers or cluster) to add to id, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the delimiter and treated as a single ID element.	list(string)	[]	no
auth	OAuth2.0 provider configuration for MLflow app instance. Supported types: azureActiveDirectory, google	object({ type = string client_id = string client_secret = string azureActiveDirectory = optional(object({ tenant_id = string })) })	n/a	yes
context	Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as null to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged.	any	{ "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null }	no
cpu	Container App CPU requirements	number	0.75	no
db_admin_username	Database Server root username	string	"mlflow_root"	no
db_backup_type	Storage account type used to store backups for this database. One of: Geo, GeoZone, Local, Zone	string	"Local"	no
db_max_size_gb	Max database size in GB	number	4	no
delimiter	Delimiter to be used between ID elements. Defaults to - (hyphen). Set to "" to use no delimiter at all.	string	null	no
descriptor_formats	Describe additional descriptors to be output in the descriptors output map. Map of maps. Keys are names of descriptors. Values are maps of the form {<br> format = string<br> labels = list(string)<br>} (Type is any so the map values can later be enhanced to provide additional options.) format is a Terraform format string to be passed to the format() function. labels is a list of labels, in order, to pass to format() function. Label values will be normalized before being passed to format() so they will be identical to how they appear in id. Default is {} (descriptors output will be empty).	any	{}	no
enabled	Set to false to prevent the module from creating any resources	bool	null	no
environment	ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT'	string	null	no
id_length_limit	Limit id to this many characters (minimum 6). Set to 0 for unlimited length. Set to null for keep the existing setting, which defaults to 0. Does not affect id_full.	number	null	no
label_key_case	Controls the letter case of the tags keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the tags input. Possible values: lower, title, upper. Default value: title.	string	null	no
label_order	The order in which the labels (ID elements) appear in the id. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present.	list(string)	null	no
label_value_case	Controls the letter case of ID elements (labels) as included in id, set as tag values, and output by this module individually. Does not affect values of tags passed in via the tags input. Possible values: lower, title, upper and none (no transformation). Set this to title and set delimiter to "" to yield Pascal Case IDs. Default value: lower.	string	null	no
labels_as_tags	Set of labels (ID elements) to include as tags in the tags output. Default is to include all labels. Tags with empty values will not be included in the tags output. Set to [] to suppress all generated tags. Notes: The value of the name tag, if included, will be the id, not the name. Unlike other null-label inputs, the initial setting of labels_as_tags cannot be changed in later chained modules. Attempts to change it will be silently ignored.	set(string)	[ "default" ]	no
location	Location for all of the resources. Falls back to the Resource Group's default.	string	null	no
log_analytics_workspace_id	Existing log analytics workspace id (or leave null to create one)	string	null	no
max_replicas	Max number of Container App replicas	number	1	no
memory	Container App memory requirements	string	"1.5Gi"	no
min_replicas	Min number of Container App replicas	number	0	no
mlflow_docker_image	MLflow docker image. See https://github.com/getindata/mlflow-docker	string	"gcr.io/getindata-images-public/mlflow:azure-1.28.0"	no
name	ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a tag. The "name" tag is set to the full id string. There is no tag with the value of the name input.	string	null	no
namespace	ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique	string	null	no
regex_replace_chars	Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, "/[^a-zA-Z0-9-]/" is used to remove all characters other than hyphens, letters and digits.	string	null	no
resource_group_name	Resource Group name	string	n/a	yes
stage	ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release'	string	null	no
storage_sku	Azure Storage Account SKU. See https://docs.microsoft.com/en-us/rest/api/storagerp/srp_sku_types	string	"Standard_GRS"	no
tags	Additional tags (e.g. {'BusinessUnit': 'XYZ'}). Neither the tag keys nor the tag values will be modified by this module.	map(string)	{}	no
tenant	ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for	string	null	no

Modules

Name	Source	Version
storage_account	getindata/storage-account/azurerm	1.1.0
this	cloudposse/label/null	0.25.0

Outputs

Name	Description
mlflow_tracking_uri	Link to deployed MLflow instance
storage_connection_string	Connection string to MLflow's blob storage account

Providers

Name	Version
azapi	0.4.0
azurerm	>=3.17.0
random	>=3.3.2

Requirements

Name	Version
terraform	>= 1.3.0
azapi	0.4.0
azurerm	>=3.17.0
random	>=3.3.2

Resources

Name	Type
azapi_resource.managed_environment	resource
azapi_resource.mlflow_app	resource
azapi_resource.mlflow_app_auth	resource
azurerm_log_analytics_workspace.log_workspace	resource
azurerm_mssql_database.mlflow_db	resource
azurerm_mssql_firewall_rule.mlflow_db_firewall_allow_internal	resource
azurerm_mssql_server.mlflow_db_server	resource
random_password.db_password	resource
azurerm_log_analytics_workspace.log_workspace	data source
azurerm_resource_group.rg	data source
azurerm_storage_account.storage_account	data source

CONTRIBUTING

Contributions are very welcomed!

Start by reviewing contribution guide and our code of conduct. After that, start coding and ship your changes by creating a new PR.

LICENSE

Apache 2 Licensed. See LICENSE for full details.

AUTHORS

Made with contrib.rocks.