page_type | languages | products | description | |||
---|---|---|---|---|---|---|
sample |
|
|
Top-level directory for official Azure Machine Learning Python SDK sample code and notebooks. |
Welcome to the Azure Machine Learning examples repository!
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
- A terminal and Python >=3.6,<3.9.
Clone this repository and install required packages:
git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/python-sdk
pip install --upgrade -r requirements.txt
To create or setup a workspace with the assets used in these examples, run the setup script.
If you do not have an Azure ML workspace, run python setup-workspace.py --subscription-id $ID
, where $ID
is your Azure subscription id. A resource group, Azure ML workspace, and other necessary resources will be created in the subscription.
If you have an Azure ML Workspace, install the Azure ML CLI and run az ml folder attach -w $WS -g $RG
, where $WS
and $RG
are the workspace and resource group names.
Run python setup-workspace.py -h
to see other arguments.
To get started, see the introductory tutorial which uses Azure ML to:
- run a
"hello world"
job on cloud compute, demonstrating the basics - run a series of PyTorch training jobs on cloud compute, demonstrating mlflow tracking & using cloud data
These concepts are sufficient to understand all examples in this repository, which are listed below.
Tutorials (tutorials)
path | status | notebooks | description |
---|---|---|---|
an-introduction | 1.hello-world.ipynb 2.pytorch-model.ipynb 3.pytorch-model-cloud-data.ipynb |
Run "hello world" and train a simple model on Azure Machine Learning. | |
automl-with-pycaret | 1.classification.ipynb | Learn how to use PyCaret for automated machine learning, with tracking and scaling in Azure ML. | |
deploy-local | 1.deploy-local.ipynb 2.deploy-local-cli.ipynb |
no description | |
using-dask | 1.intro-to-dask.ipynb | Learn how to read from cloud data and scale PyData tools (Numpy, Pandas, Scikit-Learn, etc.) with Dask and Azure ML. | |
using-pipelines | dataset-and-pipelineparameter.ipynb image-classification.ipynb publish-and-run-using-rest-endpoint.ipynb style-transfer-parallel-run.ipynb |
no description | |
using-rapids | 1.train-and-hpo.ipynb 2.train-multi-gpu.ipynb |
Learn how to accelerate PyData tools (Numpy, Pandas, Scikit-Learn, etc.) on NVIDIA GPUs with RAPIDS and Azure ML. |
Notebooks (notebooks)
path | status | description |
---|---|---|
train-lightgbm-local.ipynb | use mlflow for tracking local notebook experimentation in the cloud |
Train (workflows/train)
path | status | description |
---|---|---|
deepspeed/cifar/job.py | train CIFAR-10 using DeepSpeed and PyTorch | |
deepspeed/transformers/job.py | train Huggingface transformer using DeepSpeed | |
fastai/mnist-mlproject/job.py | train fastai resnet18 model on mnist data via mlflow mlproject | |
fastai/mnist/job.py | train fastai resnet18 model on mnist data | |
fastai/pets/job.py | train fastai resnet34 model on pets data | |
lightgbm/iris/job.py | train a lightgbm model on iris data | |
pytorch/cifar-distributed/job.py | train CNN model on CIFAR-10 dataset with distributed PyTorch | |
pytorch/mnist-mlproject/job.py | train a pytorch CNN model on mnist data via mlflow mlproject | |
pytorch/mnist/job.py | train a pytorch CNN model on mnist data | |
scikit-learn/diabetes-mlproject/job.py | train sklearn ridge model on diabetes data via mlflow mlproject | |
scikit-learn/diabetes/job.py | train sklearn ridge model on diabetes data | |
tensorflow/mnist-distributed-horovod/job.py | train tensorflow CNN model on mnist data distributed via horovod | |
tensorflow/mnist-distributed/job.py | train tensorflow CNN model on mnist data distributed via tensorflow | |
tensorflow/mnist/job.py | train tensorflow NN model on mnist data | |
transformers/glue/1-aml-finetune-job.py | Submit GLUE finetuning with Huggingface transformers library on Azure ML | |
transformers/glue/2-aml-comparison-of-sku-job.py | Experiment comparing training performance of GLUE finetuning task with differing hardware. | |
transformers/glue/3-aml-hyperdrive-job.py | Automatic hyperparameter optimization with Azure ML HyperDrive library. | |
xgboost/iris/job.py | train xgboost model on iris data |
Deploy (workflows/deploy)
path | status | description |
---|---|---|
pytorch/mnist/job.py | deploy pytorch cnn model trained on mnist data to aks | |
scikit-learn/diabetes/job.py | deploy sklearn ridge model trained on diabetes data to AKS |
Experimental tutorials (experimental)
path | status | notebooks | description | why experimental? |
---|---|---|---|---|
deploy-edge | ase-gpu.ipynb | Learn how to deploy models to Edge devices using Azure ML. | untested portions of tutorial | |
deploy-triton | 1.bidaf-ncd-local.ipynb | Learn how to efficiently deploy to GPUs with the Triton inference server and Azure ML. | in preview | |
using-pytorch-lightning | 1.train-single-node.ipynb 2.log-with-tensorboard.ipynb 3.log-with-mlflow.ipynb 4.train-multi-node-ddp.ipynb |
Learn how to train and log metrics with PyTorch Lightning and Azure ML. | issues with multinode pytorch lightning | |
using-xgboost | 1.local-eda.ipynb 2.distributed-cpu.ipynb |
Learn how to use XGBoost with Azure ML. | issues with multinode xgboost |
A lightweight template repository for automating the ML lifecycle can be found here. The contents of this repository are described below.
Note: It is not recommended to fork this repository and use it as a template directly. This repository is structured to host a large number of examples and CI for automation and testing.
directory | description |
---|---|
experimental |
self-contained directories of experimental tutorials |
notebooks |
interactive Jupyter notebooks for iterative ML development |
tutorials |
self-contained directories of tutorials |
workflows |
self-contained directories of job to be run, organized by scenario then tool then project |
We welcome contributions and suggestions! Please see the contributing guidelines for details.
This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.