| title | description | services | author | ms.service | ms.author | manager | ms.custom | ms.topic | ms.date |
|---|---|---|---|---|---|---|---|---|---|
include file |
include file |
machine-learning |
sdgilley |
machine-learning |
sgilley |
cgronlund |
include file |
include |
06/25/2020 |
The compute target you use to host your model will affect the cost and availability of your deployed endpoint. Use this table to choose an appropriate compute target.
| Compute target | Used for | GPU support | FPGA support | Description |
|---|---|---|---|---|
| Local web service | Testing/debugging | Use for limited testing and troubleshooting. Hardware acceleration depends on use of libraries in the local system. | ||
| Azure Machine Learning compute instance web service | Testing/debugging | Use for limited testing and troubleshooting. | ||
| Azure Kubernetes Service (AKS) | Real-time inference | Yes (web service deployment) | Yes | Use for high-scale production deployments. Provides fast response time and autoscaling of the deployed service. Cluster autoscaling isn't supported through the Azure Machine Learning SDK. To change the nodes in the AKS cluster, use the UI for your AKS cluster in the Azure portal. AKS is the only option available for the designer. |
| Azure Container Instances | Testing or development | Use for low-scale CPU-based workloads that require less than 48 GB of RAM. | ||
| Azure Machine Learning compute clusters | Batch inference | Yes (machine learning pipeline) | Run batch scoring on serverless compute. Supports normal and low-priority VMs. | |
| Azure Functions | (Preview) Real-time inference | |||
| Azure IoT Edge | (Preview) IoT module | Deploy and serve machine learning models on IoT devices. | ||
| Azure Data Box Edge | Via IoT Edge | Yes | Deploy and serve machine learning models on IoT devices. |
Note
Although compute targets like local, Azure Machine Learning compute, and Azure Machine Learning compute clusters support GPU for training and experimentation, using GPU for inference when deployed as a web service is supported only on AKS.
Using a GPU for inference when scoring with a machine learning pipeline is supported only on Azure Machine Learning compute.
Note
- Container instances are suitable only for small models less than 1 GB in size.
- Use single-node AKS clusters for dev/test of larger models.