Azure Machine Learning Examples

Welcome to the Azure Machine Learning examples repository!

Prerequisites

1. An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
2. A terminal and Python >=3.6,<3.9.

Setup

Clone this repository and install required packages:

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples
pip install --upgrade -r requirements.txt

To create or setup a workspace with the assets used in these examples, run the setup script.

If you do not have an Azure ML Workspace, run python setup-workspace.py --subscription-id $ID, where $ID is your Azure subscription id. A resource group, Azure ML Workspace, and other necessary resources will be created in the subscription.

If you have an Azure ML Workspace, install the Azure ML CLI and run az ml folder attach -w $WS -g $RG, where $WS and $RG are the workspace and resource group names.

Run python setup-workspace.py -h to see other arguments.

Getting started

To get started, see the introductory tutorial which uses Azure ML to:

• run a "hello world" job on cloud compute, demonstrating the basics
• run a series of PyTorch training jobs on cloud compute, demonstrating mlflow tracking & using cloud data

These concepts are sufficient to understand all examples in this repository, which are listed below.

Contents

A lightweight template repository for automating the ML lifecycle can be found here.

directory	description
.cloud	cloud templates
.github	GitHub specific files like Actions workflow yaml definitions and issue templates
notebooks	interactive jupyter notebooks for iterative ML development
tutorials	self-contained directories of tutorials
workflows	self-contained directories of job to be run, organized by scenario then tool then project

Examples

Tutorials (tutorials)

path	status	notebooks	description
an-introduction		1.hello-world.ipynb 2.pytorch-model.ipynb 3.pytorch-model-cloud-data.ipynb	learn the basics of Azure Machine Learning
automl-with-pycaret		1.classification.ipynb	learn how to automate ML with PyCaret
deploy-edge		ase-gpu.ipynb	learn how to use Edge device for model deployment and scoring
deploy-triton		1.densenet-local.ipynb 2.bidaf-aks-v100.ipynb	learn how to efficiently deploy to GPUs using triton inference server
using-dask		1.intro-to-dask.ipynb 2.dask-cloudprovider.ipynb	learn how to read from cloud data and scale PyData tools (Numpy, Pandas, Scikit-Learn, etc.) with Dask
using-pytorch-lightning		1.train-single-node.ipynb 2.log-with-tensorboard.ipynb 3.log-with-mlflow.ipynb 4.train-multi-node-ddp.ipynb	learn how to train and log metrics with PyTorch Lightning
using-rapids		1.train-and-hpo.ipynb 2.train-multi-gpu.ipynb	learn how to accelerate PyData tools (numpy, pandas, scikit-learn, etc) on NVIDIA GPUs with rapids
using-xgboost		1.local-eda.ipynb 2.distributed-cpu.ipynb	learn how to use XGBoost on Azure

Notebooks (notebooks)

path	status	description
train-lightgbm-local.ipynb		use mlflow for tracking local notebook experimentation in the cloud

Train (workflows/train)

path	status	description
deepspeed/cifar/job.py		train CIFAR-10 using DeepSpeed and PyTorch
fastai/mnist-mlproject/job.py		train fastai resnet18 model on mnist data via mlflow mlproject
fastai/mnist/job.py		train fastai resnet18 model on mnist data
fastai/pets/job.py		train fastai resnet34 model on pets data
lightgbm/iris/job.py		train a lightgbm model on iris data
pytorch/cifar-distributed/job.py		train CNN model on CIFAR-10 dataset with distributed PyTorch
pytorch/mnist-mlproject/job.py		train a pytorch CNN model on mnist data via mlflow mlproject
pytorch/mnist/job.py		train a pytorch CNN model on mnist data
scikit-learn/diabetes-mlproject/job.py		train sklearn ridge model on diabetes data via mlflow mlproject
scikit-learn/diabetes/job.py		train sklearn ridge model on diabetes data
tensorflow/mnist-distributed-horovod/job.py		train tensorflow CNN model on mnist data distributed via horovod
tensorflow/mnist-distributed/job.py		train tensorflow CNN model on mnist data distributed via tensorflow
tensorflow/mnist/job.py		train tensorflow NN model on mnist data
transformers/glue/1-aml-finetune-job.py		Submit GLUE finetuning with Huggingface transformers library on Azure ML
transformers/glue/2-aml-comparison-of-sku-job.py		Experiment comparing training performance of GLUE finetuning task with differing hardware.
transformers/glue/3-aml-hyperdrive-job.py		Automatic hyperparameter optimization with Azure ML HyperDrive library.
xgboost/iris/job.py		train xgboost model on iris data

Deploy (workflows/deploy)

path	status	description
pytorch/mnist/job.py		deploy pytorch cnn model trained on mnist data to aks
scikit-learn/diabetes/job.py		deploy sklearn ridge model trained on diabetes data to AKS

Contributing

We welcome contributions and suggestions! Please see the contributing guidelines for details.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.

Reference

• GitHub Template
• Open Source Azure ML Cheat Sheet
• Azure Machine Learning Documentation