Get started with Prefect on Azure Container Instances

This demo shows how to run Prefect flows on Azure Container Instances. There are a number of components to this:

1. An Azure ARM template with the following resources:

• A VNet with a subnet
• An ACI ContainerGroup set up to run a Prefect agent
• A Key vault for keeping secrets, we add three secrets already on deployment
• A user-assigned managed identity with access to the key vault

2. Prefect blocks for setting up the things you need in order to run a deployment on ACI:

• ACI Credentials block, containing a Service Principal that can create ContainerGroups for jobs to run
• ACI Job block, with details for how the ACI job should be set up - including attaching the user-managed identity
• An Azure storage block to store the deployment code
• A String block that contains the Azure Key Vault URL

3. A sample deployment that shows how a job can access secrets from the key vault

Preparation

Although a lot is automated, there are some things you need to bring your own of.

First, I assume you aren't completely new to Azure, and that you are already logged in to the Azure CLI. I also assume that you either use the Azure CLI, or that you are comfortable enough with PowerShell that you can translate the commands for yourself.

Secondly, you will need to have your Prefect API key and workspace URL handy. I can't provide those for you.

Third, as a part of this you will need to create the Azure Service Principal that will create the ACI ContainerGroups for the jobs (and that you will provide to the ACI Credentials Block). This means you will need fairly elevated rights on your Azure account.

Lastly, you will need to keep your subscription ID handy.

Steps

Create the resource group

All of the azure resources will be deployed in a dedicated resource group. Because that resource group name has to be hardcoded in a few Blocks, I have decided to name the resource group for you. Say hello to aci-rg. Feel free to create it in the portal, but if you want to be like the cool kids, run az group create -n aci-rg -l northeurope. You can change the location to your heart's content though. The ARM template will create all the resources in the location of the resource group.

Deploy the ARM template

The ARM template doesn't have a parameters file, but you can pass some parameters through the command line when creating. Since there are quite a few resources in the ARM template we could have created a lot of parameters to customize names, networks and more, but you would probably just edit those directly in the deployment file anyways. It's not like you are going to spin up many of these without first adjusting the template anyways. The parameters you do need to fill in are the Prefect API Key and Prefect workspace URL, and a unique name for your new key vault.

You are going to use the keyvault-name further down too, so let's define it as an environment variable:

export KV_NAME="<some unique KV name>"

az deployment group create \            
-g "acirg" \
--template-file ./aci-rg/azuredeploy.json \
--parameters \
keyvaultname=$KV_NAME \
prefect-api-key=<prefect api key> \
prefect-api-url=https://api.prefect.cloud/api/accounts/<account-id>/workspaces/<workspace-id>

Create the Service Principal

If you aren't familiar with Service Principals, you can think of them as system users - useful for computers outside of your Azure accoint to authenticate and do whatever it needs to do.

az ad sp create-for-rbac -n "prefect-aci" --role "Contributor" --scopes "/subscriptions/<subscription-id>/resourceGroups/aci-rg"

OK, you might be able to find a less overwhelming role than Contributor, I certainally would try to do that if this was in a corporate environment. I have attached a rough draft at a custom role definition for that purpose, but I'm not certain it would be allowed to assign the user-managed identity that we need.

Once the service principal is created, you will get some output in the form of a JSON with some credentials:

{
  "appId": "<UUID-4-string-appId>",
  "displayName": "prefect-aci",
  "password": "<AUTOGENERATED PASSWORD>",
  "tenant": "<YOUR TENANT ID>"
}

Keep meticulous track of these credentials, they will be needed in the next step.

Register ACI Credentials block

The python script to register the ACI Credentials gets the Service Principal credentials from OS-level env variables. If we start by defining these, you won't have to hardcode credentials in the python file.

export TENANT_ID="<YOUR TENANT ID>"
export CLIENT_ID="<UUID-4-string-appId>"
export CLIENT_SECRET="<AUTOGENERATED PASSWORD>"

Now, just run the python file python ./prefect-blocks/01_aci_credentials.py. This should only take a second, and you should see a credentials block pop up in the Prefect UI.

Register the ACI Job block

You are on the home stretch now. In order to register the ACI Job block, you need your Azure subscription ID. The python file reads it from an OS env variable, so let's start by exporting that one:

export SUBSCRIPTION_ID="<YOUR SUBSCRIPTION ID>"

Now, you can run the ACI job block python file python ./prefect-blocks/02_aci_job.py

Register storage and Keyvault URL

The Keyvault URL block is ready to go, as long as the environment variables are defined the script can be run without problems.

The Azure storage block needs two variables though, a storage account name and a storage account key. Since I'm focusing on ACI here, I won't go into details on creating a storage account and finding the storage account key.

Deploy a flow

Now, it's time to take all of this for a spin. You probably know the drill: make sure to install the requirements in prefect-flows/key_vault_demo/requirements.txt, and then run prefect-flows/key_vault_demo/deployment.py. The new deployment should pop up in the UI.

A note about prefect images

One peculiarity about this setup is that the docker images used need more than just a prefect library installed - it also needs to have the tini entrypoint process running, adlfs and the prefect-azure library installed. The quickest way to do this is to start with a pre-built prefect image and install the extra libraries. The radbrt/orion_aci_base image does just that.