| page_type | languages | products | description | ||
|---|---|---|---|---|---|
sample |
|
|
Top-level directory for official Azure Machine Learning Python SDK v2 sample code. |
We are excited to introduce the public preview of Azure Machine Learning Python SDK v2. Azure ML Python SDK v2 is an updated Python SDK package, which allows users to:
- Submit training jobs
- Manage data, models, environments
- Perform managed inferencing (real time and batch)
- Stitch together multiple tasks and production workflows using Azure ML pipelines
- Run standalone AutoML training for various ml-tasks
Please note that this Private Preview release is subject to the Supplemental Terms of Use for Microsoft Azure Previews.
The SDK v2 comes with many improvements and features:
-
Use Python functions to build a single step or a complex workflow
SDK v2 allows you to build a single command or a chain of commands like python functions - the command has a name, parameters, expects input, and returns output.
-
Move from simple to complex concepts incrementally
SDK v2 allows you to:
- Construct a single command.
- Add a hyperparameter sweep on top of that command,
- Add the command with various others into a pipeline one after the other. This construction is useful, given the iterative nature of machine learning.
-
Reusable components in pipelines
Azure ML introduces components for managing and reusing common logic across pipelines. This functionality is available only via CLI v2 and SDK v2.
-
Managed inferencing
Azure ML offers endpoints to streamline model deployments for both real-time and batch inference deployments. This functionality is available only via CLI v2 and SDK v2.
Python SDK v2 can be used in various ways – in python scripts, Jupyter Notebooks to create, submit / manage jobs, pipelines, and your resources/assets. You can use the Azure ML notebooks, VS Code or other editors of your choice to manage to your code. Checkout an over view of our samples.
If you are facing any issues while using the new feature, please reach out to Azure ML SDK feedback. For general feedback, please submit an GitHub issue.
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
- Install the SDK v2
pip uninstall azure-ai-ml
pip install --pre azure-ai-ml
git clone https://github.com/Azure/azureml-examples
cd azureml-examples/sdk
Test Status is for branch - main
| Area | Sub-Area | Notebook | Description | Status |
|---|---|---|---|---|
| assets | component | component | Create a component asset |  |
| assets | data | data | Read, write and register a data asset |  |
| assets | environment | environment | Create custom environments from docker and/or conda YAML |  |
| assets | model | model | Create model from local files, cloud files, Runs |  |
| endpoints | batch | mnist-nonmlflow | Create and test batch endpoint and deployement |  |
| endpoints | online | online-endpoints-custom-container | Deploy a custom container as an online endpoint. Use web servers other than the default Python Flask server used by Azure ML without losing the benefits of Azure ML's built-in monitoring, scaling, alerting, and authentication. |  |
| endpoints | online | kubernetes-online-endpoints-safe-rollout | Safely rollout a new version of a web service to production by rolling out the change to a small subset of users/requests before rolling it out completely |  |
| endpoints | online | kubernetes-online-endpoints-simple-deployment | Use an online endpoint to deploy your model, so you don't have to create and manage the underlying infrastructure |  |
| endpoints | online | online-endpoints-safe-rollout | Safely rollout a new version of a web service to production by rolling out the change to a small subset of users/requests before rolling it out completely |  |
| endpoints | online | online-endpoints-simple-deployment | Use an online endpoint to deploy your model, so you don't have to create and manage the underlying infrastructure |  |
| endpoints | online | online-endpoints-deploy-mlflow-model | Deploy an mlflow model to an online endpoint. This will be a no-code-deployment. It doesn't require scoring script and environment. |  |
| jobs | automl-standalone-jobs | automl-classification-task-bankmarketing-mlflow | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-classification-task-bankmarketing | no description |  |
| jobs | automl-standalone-jobs | mlflow-model-local-inference-test | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-forecasting-task-energy-demand-advanced-mlflow | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-forecasting-task-energy-demand-advanced | no description |  |
| jobs | automl-standalone-jobs | mlflow-model-local-inference-test | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-image-classification-multiclass-task-fridge-items | no description |  |
| jobs | automl-standalone-jobs | automl-image-classification-multilabel-task-fridge-items | no description |  |
| jobs | automl-standalone-jobs | automl-image-instance-segmentation-task-fridge-items | no description |  |
| jobs | automl-standalone-jobs | automl-image-object-detection-task-fridge-items-mlflow | no description |  |
| jobs | automl-standalone-jobs | automl-image-object-detection-task-fridge-items | no description |  |
| jobs | automl-standalone-jobs | mlflow-model-local-inference-test | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-nlp-text-classification-multiclass-task-sentiment-mlflow | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-nlp-text-classification-multiclass-task-sentiment | no description |  |
| jobs | automl-standalone-jobs | mlflow-model-local-inference-test | no description - This sample is excluded from automated tests |  |
| jobs | automl-standalone-jobs | automl-nlp-text-classification-multilabel-task-paper-cat | no description |  |
| jobs | automl-standalone-jobs | automl-nlp-text-ner-task | no description |  |
| jobs | automl-standalone-jobs | automl-regression-task-hardware-performance | no description |  |
| jobs | configuration.ipynb | configuration | Setting up your Azure Machine Learning services workspace and configuring needed resources |  |
| jobs | pipelines | pipeline_with_components_from_yaml | Create pipeline with CommandComponents from local YAML file |  |
| jobs | pipelines | pipeline_with_python_function_components | Create pipeline with command_component decorator |  |
| jobs | pipelines | pipeline_with_hyperparameter_sweep | Use sweep (hyperdrive) in pipeline to train mnist model using tensorflow |  |
| jobs | pipelines | pipeline_with_non_python_components | Create a pipeline with command function |  |
| jobs | pipelines | pipeline_with_registered_components | Register component and then use these components to build pipeline |  |
| jobs | pipelines | train_mnist_with_tensorflow | Create pipeline using components to run a distributed job with tensorflow |  |
| jobs | pipelines | train_cifar_10_with_pytorch | Get data, train and evaluate a model in pipeline with Components |  |
| jobs | pipelines | nyc_taxi_data_regression | Build pipeline with components for 5 jobs - prep data, transform data, train model, predict results and evaluate model performance |  |
| jobs | pipelines | image_classification_with_densenet | Create pipeline to train cnn image classification model |  |
| jobs | pipelines | image_classification_keras_minist_convnet | Create pipeline to train cnn image classification model with keras |  |
| jobs | pipelines | rai_pipeline_sample | Create sample RAI pipeline |  |
| jobs | single-step | lightgbm-iris-sweep | Run hyperparameter sweep on a Command or CommandComponent |  |
| jobs | single-step | pytorch-iris | Run Command to train a neural network with PyTorch on Iris dataset |  |
| jobs | single-step | accident-prediction | Run R in a Command to train a prediction model |  |
| jobs | single-step | sklearn-diabetes | Run Command to train a scikit-learn LinearRegression model on the Diabetes dataset |  |
| jobs | single-step | iris-scikit-learn | Run Command to train a scikit-learn SVM on the Iris dataset |  |
| jobs | single-step | sklearn-mnist | Run a Command to train a scikit-learn SVM on the mnist dataset. |  |
| jobs | single-step | tensorflow-mnist-distributed-horovod | Run a Distributed Command to train a basic neural network with distributed MPI on the MNIST dataset using Horovod |  |
| jobs | single-step | tensorflow-mnist-distributed | Run a Distributed Command to train a basic neural network with TensorFlow on the MNIST dataset |  |
| jobs | single-step | tensorflow-mnist | Run a Command to train a basic neural network with TensorFlow on the MNIST dataset |  |
| resources | compute | compute | Create compute in Azure ML workspace - This sample is only tested on demand |  |
| resources | datastores | datastore | Create datastores and use in a Command - This sample is excluded from automated tests |  |
| resources | workspace | workspace | Create Azure ML workspace - This sample is only tested on demand |  |
Azure Machine Learning Python SDK v2 comes with many new features like standalone local jobs, reusable components for pipelines and managed online/batch inferencing. The SDK v2 brings consistency and ease of use across all assets of the platform. The Python SDK v2 offers the following capabilities:
- Run Standalone Jobs - run a discrete ML activity as Job. This job can be run locally or on the cloud. We currently support the following types of jobs:
- Command - run a command (Python, R, Windows Command, Linux Shell etc.)
- Sweep - run a hyperparameter sweep on your Command
- Run multiple jobs using our improved Pipelines
- Run a series of commands stitched into a pipeline (New)
- Components - run pipelines using reusable components (New)
- Use your models for Managed Online inferencing (New)
- Use your models for Managed batch inferencing
- Manage AML resources – workspace, compute, datastores
- Manage AML assets - Datasets, environments, models
- AutoML - run standalone AutoML training for various ml-tasks:
- Classification (Tabular data)
- Regression (Tabular data)
- Time Series Forecasting (Tabular data)
- Image Classification (Multi*class) (New)
- Image Classification (Multi*label) (New)
- Image Object Detection (New)
- Image Instance Segmentation (New)
- NLP Text Classification (Multi*class) (New)
- NLP Text Classification (Multi*label) (New)
- NLP Text Named Entity Recognition (NER) (New)
We welcome contributions and suggestions! Please see the contributing guidelines for details.
This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.