The customized VPC configuration at the instance group level that overrides the default VPC configuration of the SageMaker HyperPod cluster.
" + "documentation":"The customized Amazon VPC configuration at the instance group level that overrides the default Amazon VPC configuration of the SageMaker HyperPod cluster.
" } }, "documentation":"Details of an instance group in a SageMaker HyperPod cluster.
" @@ -7720,7 +7720,7 @@ }, "OverrideVpcConfig":{ "shape":"VpcConfig", - "documentation":"To configure multi-AZ deployments, customize the VPC configuration at the instance group level. You can specify different subnets and security groups across different AZs in the instance group specification to override a SageMaker HyperPod cluster's default VPC configuration. For more information about deploying a cluster in multiple AZs, see Setting up SageMaker HyperPod clusters across multiple AZs.
If you configure your VPC with IPv6 support and specify subnets with IPv6 addressing enabled in your instance group VPC configuration, the nodes automatically use IPv6 addressing for network communication.
For information about adding IPv6 support for your VPC, see IPv6 support for your VPC.
For information about creating a new VPC for use with IPv6, see Create a VPC.
To configure multi-AZ deployments, customize the Amazon VPC configuration at the instance group level. You can specify different subnets and security groups across different AZs in the instance group specification to override a SageMaker HyperPod cluster's default Amazon VPC configuration. For more information about deploying a cluster in multiple AZs, see Setting up SageMaker HyperPod clusters across multiple AZs.
When your Amazon VPC and subnets support IPv6, network communications differ based on the cluster orchestration platform:
Slurm-orchestrated clusters automatically configure nodes with dual IPv6 and IPv4 addresses, allowing immediate IPv6 network communications.
In Amazon EKS-orchestrated clusters, nodes receive dual-stack addressing, but pods can only use IPv6 when the Amazon EKS cluster is explicitly IPv6-enabled. For information about deploying an IPv6 Amazon EKS cluster, see Amazon EKS IPv6 Cluster Deployment.
Additional resources for IPv6 configuration:
For information about adding IPv6 support to your VPC, see to IPv6 Support for VPC.
For information about creating a new IPv6-compatible VPC, see Amazon VPC Creation Guide.
To configure SageMaker HyperPod with a custom Amazon VPC, see Custom Amazon VPC Setup for SageMaker HyperPod.
The specifications of an instance group that you need to define.
" @@ -7942,7 +7942,7 @@ }, "OverrideVpcConfig":{ "shape":"VpcConfig", - "documentation":"The customized VPC configuration at the instance group level that overrides the default VPC configuration of the SageMaker HyperPod cluster.
" + "documentation":"The customized Amazon VPC configuration at the instance group level that overrides the default Amazon VPC configuration of the SageMaker HyperPod cluster.
" }, "ThreadsPerCore":{ "shape":"ClusterThreadsPerCore", @@ -7958,7 +7958,7 @@ }, "PrivatePrimaryIpv6":{ "shape":"ClusterPrivatePrimaryIpv6", - "documentation":"The private primary IPv6 address of the SageMaker HyperPod cluster node when configured with an Amazon VPC that supports IPv6 and includes subnets with IPv6 addressing enabled in either the cluster VPC configuration or the instance group VPC configuration.
" + "documentation":"The private primary IPv6 address of the SageMaker HyperPod cluster node when configured with an Amazon VPC that supports IPv6 and includes subnets with IPv6 addressing enabled in either the cluster Amazon VPC configuration or the instance group Amazon VPC configuration.
" }, "PrivateDnsHostname":{ "shape":"ClusterPrivateDnsHostname", @@ -9322,7 +9322,7 @@ }, "VpcConfig":{ "shape":"VpcConfig", - "documentation":"Specifies the Amazon Virtual Private Cloud (VPC) that is associated with the Amazon SageMaker HyperPod cluster. You can control access to and from your resources by configuring your VPC. For more information, see Give SageMaker access to resources in your Amazon VPC.
If you configure your VPC with IPv6 support and specify subnets with IPv6 addressing enabled in your VPC configuration, the cluster automatically uses IPv6 addressing for network communication.
For information about adding IPv6 support for your VPC, see IPv6 support for your VPC.
For information about creating a new VPC for use with IPv6, see Create a VPC.
Specifies the Amazon Virtual Private Cloud (VPC) that is associated with the Amazon SageMaker HyperPod cluster. You can control access to and from your resources by configuring your VPC. For more information, see Give SageMaker access to resources in your Amazon VPC.
When your Amazon VPC and subnets support IPv6, network communications differ based on the cluster orchestration platform:
Slurm-orchestrated clusters automatically configure nodes with dual IPv6 and IPv4 addresses, allowing immediate IPv6 network communications.
In Amazon EKS-orchestrated clusters, nodes receive dual-stack addressing, but pods can only use IPv6 when the Amazon EKS cluster is explicitly IPv6-enabled. For information about deploying an IPv6 Amazon EKS cluster, see Amazon EKS IPv6 Cluster Deployment.
Additional resources for IPv6 configuration:
For information about adding IPv6 support to your VPC, see to IPv6 Support for VPC.
For information about creating a new IPv6-compatible VPC, see Amazon VPC Creation Guide.
To configure SageMaker HyperPod with a custom Amazon VPC, see Custom Amazon VPC Setup for SageMaker HyperPod.
Specifies an option from a collection of preconfigured Amazon Machine Image (AMI) images. Each image is configured by Amazon Web Services with a set of software and driver versions. Amazon Web Services optimizes these configurations for different machine learning workloads.
By selecting an AMI version, you can ensure that your inference environment is compatible with specific software requirements, such as CUDA driver versions, Linux kernel versions, or Amazon Web Services Neuron driver versions.
The AMI version names, and their configurations, are the following:
Accelerator: GPU
NVIDIA driver version: 535.54.03
CUDA version: 12.2
Accelerator: GPU
NVIDIA driver version: 535.54.03
CUDA driver version: 12.2
CUDA Container Toolkit with disabled CUDA-compat mounting
Accelerator: GPU
NVIDIA driver version: 550.144.01
CUDA version: 12.4
Container Toolkit with disabled CUDA-compat mounting
Specifies an option from a collection of preconfigured Amazon Machine Image (AMI) images. Each image is configured by Amazon Web Services with a set of software and driver versions. Amazon Web Services optimizes these configurations for different machine learning workloads.
By selecting an AMI version, you can ensure that your inference environment is compatible with specific software requirements, such as CUDA driver versions, Linux kernel versions, or Amazon Web Services Neuron driver versions.
The AMI version names, and their configurations, are the following:
Accelerator: GPU
NVIDIA driver version: 535
CUDA version: 12.2
Accelerator: GPU
NVIDIA driver version: 535
CUDA version: 12.2
NVIDIA Container Toolkit with disabled CUDA-compat mounting
Accelerator: GPU
NVIDIA driver version: 550
CUDA version: 12.4
NVIDIA Container Toolkit with disabled CUDA-compat mounting
Identifies a model that you want to host and the resources chosen to deploy for hosting it. If you are deploying multiple models, tell SageMaker how to distribute traffic among the models by specifying variant weights. For more information on production variants, check Production variants.
" @@ -33764,6 +33764,16 @@ "ml.g6.16xlarge", "ml.g6.24xlarge", "ml.g6.48xlarge", + "ml.r8g.medium", + "ml.r8g.large", + "ml.r8g.xlarge", + "ml.r8g.2xlarge", + "ml.r8g.4xlarge", + "ml.r8g.8xlarge", + "ml.r8g.12xlarge", + "ml.r8g.16xlarge", + "ml.r8g.24xlarge", + "ml.r8g.48xlarge", "ml.g6e.xlarge", "ml.g6e.2xlarge", "ml.g6e.4xlarge", diff --git a/src/sagemaker_core/main/resources.py b/src/sagemaker_core/main/resources.py index dd5f1173..8e7a0b33 100644 --- a/src/sagemaker_core/main/resources.py +++ b/src/sagemaker_core/main/resources.py @@ -3280,7 +3280,7 @@ def create( Parameters: cluster_name: The name for the new SageMaker HyperPod cluster. instance_groups: The instance groups to be created in the SageMaker HyperPod cluster. - vpc_config: Specifies the Amazon Virtual Private Cloud (VPC) that is associated with the Amazon SageMaker HyperPod cluster. You can control access to and from your resources by configuring your VPC. For more information, see Give SageMaker access to resources in your Amazon VPC. If you configure your VPC with IPv6 support and specify subnets with IPv6 addressing enabled in your VPC configuration, the cluster automatically uses IPv6 addressing for network communication. For information about adding IPv6 support for your VPC, see IPv6 support for your VPC. For information about creating a new VPC for use with IPv6, see Create a VPC. + vpc_config: Specifies the Amazon Virtual Private Cloud (VPC) that is associated with the Amazon SageMaker HyperPod cluster. You can control access to and from your resources by configuring your VPC. For more information, see Give SageMaker access to resources in your Amazon VPC. When your Amazon VPC and subnets support IPv6, network communications differ based on the cluster orchestration platform: Slurm-orchestrated clusters automatically configure nodes with dual IPv6 and IPv4 addresses, allowing immediate IPv6 network communications. In Amazon EKS-orchestrated clusters, nodes receive dual-stack addressing, but pods can only use IPv6 when the Amazon EKS cluster is explicitly IPv6-enabled. For information about deploying an IPv6 Amazon EKS cluster, see Amazon EKS IPv6 Cluster Deployment. Additional resources for IPv6 configuration: For information about adding IPv6 support to your VPC, see to IPv6 Support for VPC. For information about creating a new IPv6-compatible VPC, see Amazon VPC Creation Guide. To configure SageMaker HyperPod with a custom Amazon VPC, see Custom Amazon VPC Setup for SageMaker HyperPod. tags: Custom tags for managing the SageMaker HyperPod cluster as an Amazon Web Services resource. You can add tags to your cluster in the same way you add them in other Amazon Web Services services that support tagging. To learn more about tagging Amazon Web Services resources in general, see Tagging Amazon Web Services Resources User Guide. orchestrator: The type of orchestrator to use for the SageMaker HyperPod cluster. Currently, the only supported value is "eks", which is to use an Amazon Elastic Kubernetes Service (EKS) cluster as the orchestrator. node_recovery: The node recovery mode for the SageMaker HyperPod cluster. When set to Automatic, SageMaker HyperPod will automatically reboot or replace faulty nodes when issues are detected. When set to None, cluster administrators will need to manually manage any faulty cluster instances. diff --git a/src/sagemaker_core/main/shapes.py b/src/sagemaker_core/main/shapes.py index 6ba2fe1c..1b356859 100644 --- a/src/sagemaker_core/main/shapes.py +++ b/src/sagemaker_core/main/shapes.py @@ -3153,7 +3153,7 @@ class ClusterInstanceGroupDetails(Base): status: The current status of the cluster instance group. InService: The instance group is active and healthy. Creating: The instance group is being provisioned. Updating: The instance group is being updated. Failed: The instance group has failed to provision or is no longer healthy. Degraded: The instance group is degraded, meaning that some instances have failed to provision or are no longer healthy. Deleting: The instance group is being deleted. training_plan_arn: The Amazon Resource Name (ARN); of the training plan associated with this cluster instance group. For more information about how to reserve GPU capacity for your SageMaker HyperPod clusters using Amazon SageMaker Training Plan, see CreateTrainingPlan . training_plan_status: The current status of the training plan associated with this cluster instance group. - override_vpc_config: The customized VPC configuration at the instance group level that overrides the default VPC configuration of the SageMaker HyperPod cluster. + override_vpc_config: The customized Amazon VPC configuration at the instance group level that overrides the default Amazon VPC configuration of the SageMaker HyperPod cluster. """ current_count: Optional[int] = Unassigned() @@ -3187,7 +3187,7 @@ class ClusterInstanceGroupSpecification(Base): instance_storage_configs: Specifies the additional storage configurations for the instances in the SageMaker HyperPod cluster instance group. on_start_deep_health_checks: A flag indicating whether deep health checks should be performed when the cluster instance group is created or updated. training_plan_arn: The Amazon Resource Name (ARN); of the training plan to use for this cluster instance group. For more information about how to reserve GPU capacity for your SageMaker HyperPod clusters using Amazon SageMaker Training Plan, see CreateTrainingPlan . - override_vpc_config: To configure multi-AZ deployments, customize the VPC configuration at the instance group level. You can specify different subnets and security groups across different AZs in the instance group specification to override a SageMaker HyperPod cluster's default VPC configuration. For more information about deploying a cluster in multiple AZs, see Setting up SageMaker HyperPod clusters across multiple AZs. If you configure your VPC with IPv6 support and specify subnets with IPv6 addressing enabled in your instance group VPC configuration, the nodes automatically use IPv6 addressing for network communication. For information about adding IPv6 support for your VPC, see IPv6 support for your VPC. For information about creating a new VPC for use with IPv6, see Create a VPC. + override_vpc_config: To configure multi-AZ deployments, customize the Amazon VPC configuration at the instance group level. You can specify different subnets and security groups across different AZs in the instance group specification to override a SageMaker HyperPod cluster's default Amazon VPC configuration. For more information about deploying a cluster in multiple AZs, see Setting up SageMaker HyperPod clusters across multiple AZs. When your Amazon VPC and subnets support IPv6, network communications differ based on the cluster orchestration platform: Slurm-orchestrated clusters automatically configure nodes with dual IPv6 and IPv4 addresses, allowing immediate IPv6 network communications. In Amazon EKS-orchestrated clusters, nodes receive dual-stack addressing, but pods can only use IPv6 when the Amazon EKS cluster is explicitly IPv6-enabled. For information about deploying an IPv6 Amazon EKS cluster, see Amazon EKS IPv6 Cluster Deployment. Additional resources for IPv6 configuration: For information about adding IPv6 support to your VPC, see to IPv6 Support for VPC. For information about creating a new IPv6-compatible VPC, see Amazon VPC Creation Guide. To configure SageMaker HyperPod with a custom Amazon VPC, see Custom Amazon VPC Setup for SageMaker HyperPod. """ instance_count: int @@ -3245,11 +3245,11 @@ class ClusterNodeDetails(Base): instance_type: The type of the instance. launch_time: The time when the instance is launched. life_cycle_config: The LifeCycle configuration applied to the instance. - override_vpc_config: The customized VPC configuration at the instance group level that overrides the default VPC configuration of the SageMaker HyperPod cluster. + override_vpc_config: The customized Amazon VPC configuration at the instance group level that overrides the default Amazon VPC configuration of the SageMaker HyperPod cluster. threads_per_core: The number of threads per CPU core you specified under CreateCluster. instance_storage_configs: The configurations of additional storage specified to the instance group where the instance (node) is launched. private_primary_ip: The private primary IP address of the SageMaker HyperPod cluster node. - private_primary_ipv6: The private primary IPv6 address of the SageMaker HyperPod cluster node when configured with an Amazon VPC that supports IPv6 and includes subnets with IPv6 addressing enabled in either the cluster VPC configuration or the instance group VPC configuration. + private_primary_ipv6: The private primary IPv6 address of the SageMaker HyperPod cluster node when configured with an Amazon VPC that supports IPv6 and includes subnets with IPv6 addressing enabled in either the cluster Amazon VPC configuration or the instance group Amazon VPC configuration. private_dns_hostname: The private DNS hostname of the SageMaker HyperPod cluster node. placement: The placement details of the SageMaker HyperPod cluster node. """ @@ -4858,7 +4858,7 @@ class ProductionVariant(Base): enable_ssm_access: You can use this parameter to turn on native Amazon Web Services Systems Manager (SSM) access for a production variant behind an endpoint. By default, SSM access is disabled for all production variants behind an endpoint. You can turn on or turn off SSM access for a production variant behind an existing endpoint by creating a new endpoint configuration and calling UpdateEndpoint. managed_instance_scaling: Settings that control the range in the number of instances that the endpoint provisions as it scales up or down to accommodate traffic. routing_config: Settings that control how the endpoint routes incoming traffic to the instances that the endpoint hosts. - inference_ami_version: Specifies an option from a collection of preconfigured Amazon Machine Image (AMI) images. Each image is configured by Amazon Web Services with a set of software and driver versions. Amazon Web Services optimizes these configurations for different machine learning workloads. By selecting an AMI version, you can ensure that your inference environment is compatible with specific software requirements, such as CUDA driver versions, Linux kernel versions, or Amazon Web Services Neuron driver versions. The AMI version names, and their configurations, are the following: al2-ami-sagemaker-inference-gpu-2 Accelerator: GPU NVIDIA driver version: 535.54.03 CUDA version: 12.2 al2-ami-sagemaker-inference-gpu-2-1 Accelerator: GPU NVIDIA driver version: 535.54.03 CUDA driver version: 12.2 CUDA Container Toolkit with disabled CUDA-compat mounting al2-ami-sagemaker-inference-gpu-3-1 Accelerator: GPU NVIDIA driver version: 550.144.01 CUDA version: 12.4 Container Toolkit with disabled CUDA-compat mounting + inference_ami_version: Specifies an option from a collection of preconfigured Amazon Machine Image (AMI) images. Each image is configured by Amazon Web Services with a set of software and driver versions. Amazon Web Services optimizes these configurations for different machine learning workloads. By selecting an AMI version, you can ensure that your inference environment is compatible with specific software requirements, such as CUDA driver versions, Linux kernel versions, or Amazon Web Services Neuron driver versions. The AMI version names, and their configurations, are the following: al2-ami-sagemaker-inference-gpu-2 Accelerator: GPU NVIDIA driver version: 535 CUDA version: 12.2 al2-ami-sagemaker-inference-gpu-2-1 Accelerator: GPU NVIDIA driver version: 535 CUDA version: 12.2 NVIDIA Container Toolkit with disabled CUDA-compat mounting al2-ami-sagemaker-inference-gpu-3-1 Accelerator: GPU NVIDIA driver version: 550 CUDA version: 12.4 NVIDIA Container Toolkit with disabled CUDA-compat mounting """ variant_name: str