diff --git a/installing/installing_aws/installing-aws-localzone.adoc b/installing/installing_aws/installing-aws-localzone.adoc index 608c9373dd12..2e6120ddaeeb 100644 --- a/installing/installing_aws/installing-aws-localzone.adoc +++ b/installing/installing_aws/installing-aws-localzone.adoc @@ -41,6 +41,24 @@ If you have an AWS profile stored on your computer, it must not use a temporary Be sure to also review this site list if you are configuring a proxy. ==== * If the cloud identity and access management (IAM) APIs are not accessible in your environment, or if you do not want to store an administrator-level credential secret in the `kube-system` namespace, you can xref:../../installing/installing_aws/manually-creating-iam.adoc#manually-creating-iam-aws[manually create and maintain IAM credentials]. +* Add permission for the user who creates the cluster to modify the Local Zone group with `ec2:ModifyAvailabilityZoneGroup`. For example: ++ +.An example of a permissive IAM policy to attach to a user or role +[source,yaml] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Action": [ + "ec2:ModifyAvailabilityZoneGroup" + ], + "Effect": "Allow", + "Resource": "*" + } + ] +} +---- include::modules/cluster-limitations-local-zone.adoc[leveloffset=+1] @@ -51,14 +69,14 @@ include::modules/cluster-limitations-local-zone.adoc[leveloffset=+1] include::modules/cluster-entitlements.adoc[leveloffset=+1] -include::modules/installation-aws-add-local-zone-locations.adoc[leveloffset=+1] - include::modules/installation-aws-marketplace-subscribe.adoc[leveloffset=+1] include::modules/installation-creating-aws-vpc-localzone.adoc[leveloffset=+1] include::modules/installation-cloudformation-vpc-localzone.adoc[leveloffset=+2] +include::modules/installation-aws-add-local-zone-locations.adoc[leveloffset=+1] + include::modules/installation-creating-aws-subnet-localzone.adoc[leveloffset=+1] include::modules/installation-cloudformation-subnet-localzone.adoc[leveloffset=+2] @@ -82,6 +100,17 @@ include::modules/installation-aws-tested-machine-types.adoc[leveloffset=+2] * See link:https://aws.amazon.com/about-aws/global-infrastructure/localzones/features/[AWS Local Zones features] in the AWS documentation for more information about AWS Local Zones and the supported instances types and services. include::modules/installation-generate-aws-user-infra-install-config.adoc[leveloffset=+2] +// Suggest to standarize edge-pool's specific files with same prefixes, like: machine-edge-pool-[...] or compute-edge-pool-[...] (which is more compatible with install-config.yaml/compute) +include::modules/machines-edge-machine-pool.adoc[leveloffset=+2] +include::modules/edge-machine-pools-aws-local-zones.adoc[leveloffset=+3] + +[role="_additional-resources"] +.Additional resources + +* xref:../../networking/changing-cluster-network-mtu.adoc#mtu-value-selection_changing-cluster-network-mtu[Changing the MTU for the cluster network] +* xref:../../networking/changing-cluster-network-mtu.adoc#nw-ovn-ipsec-enable_configuring-ipsec-ovn[Enabling IPsec encryption] + +include::modules/install-creating-install-config-aws-local-zones.adoc[leveloffset=+2] [role="_additional-resources"] .Additional resources @@ -91,13 +120,9 @@ include::modules/installation-generate-aws-user-infra-install-config.adoc[levelo //include::modules/installation-configure-proxy.adoc[leveloffset=+2] //Put this back if QE validates it. -include::modules/installation-localzone-generate-k8s-manifest.adoc[leveloffset=+2] +// Verify removal due to automation. +// include::modules/installation-localzone-generate-k8s-manifest.adoc[leveloffset=+2] -[role="_additional-resources"] -.Additional resources - -* xref:../../networking/changing-cluster-network-mtu.adoc#mtu-value-selection_changing-cluster-network-mtu[Changing the MTU for the cluster network] -* xref:../../networking/changing-cluster-network-mtu.adoc#nw-ovn-ipsec-enable_configuring-ipsec-ovn[Enabling IPsec encryption] include::modules/installation-launching-installer.adoc[leveloffset=+1] @@ -115,6 +140,8 @@ include::modules/logging-in-by-using-the-web-console.adoc[leveloffset=+1] * See xref:../../web_console/web-console.adoc#web-console[Accessing the web console] for more details about accessing and understanding the {product-title} web console. +include::modules/machine-edge-pool-review-nodes.adoc[leveloffset=+1] + include::modules/cluster-telemetry.adoc[leveloffset=+1] [role="_additional-resources"] @@ -125,6 +152,7 @@ include::modules/cluster-telemetry.adoc[leveloffset=+1] [id="installing-aws-localzone-next-steps"] == Next steps +* xref:../../post_installation_configuration/cluster-tasks.adoc#installation-extend-edge-nodes-aws-local-zones_post-install-cluster-tasks[Creating user workloads in AWS Local Zones]. * xref:../../installing/validating-an-installation.adoc#validating-an-installation[Validating an installation]. * xref:../../post_installation_configuration/cluster-tasks.adoc#available_cluster_customizations[Customize your cluster]. * If necessary, you can xref:../../support/remote_health_monitoring/opting-out-of-remote-health-reporting.adoc#opting-out-remote-health-reporting_opting-out-remote-health-reporting[opt out of remote health reporting]. diff --git a/modules/edge-machine-pools-aws-local-zones.adoc b/modules/edge-machine-pools-aws-local-zones.adoc new file mode 100644 index 000000000000..302d29b2b059 --- /dev/null +++ b/modules/edge-machine-pools-aws-local-zones.adoc @@ -0,0 +1,25 @@ +// Module included in the following assemblies: +// * installing/installing_aws/installing-aws-localzone.adoc + +:_content-type: CONCEPT +[id="edge-machine-pools-aws-local-zones_{context}"] += Edge compute pools and AWS Local Zones + +Edge worker nodes are tainted worker nodes that run in AWS Local Zones locations. + +When deploying a cluster that uses Local Zones: + +* Amazon EC2 instances in the Local Zones are more expensive than Amazon EC2 instances in the Availability Zones. +* Latency between applications and end users is lower in Local Zones, and it may vary by location. There is a latency impact for some workloads if, for example, routers are mixed between Local Zones and Availability Zones. +* The cluster-network Maximum Transmission Unit (MTU) is adjusted automatically to the lower restricted by AWS when Local Zone subnets are detected on the `install-config.yaml`, according to the network plugin. For example, the adjusted values are 1200 for OVN-Kubernetes and 1250 for OpenShift SDN. If additional features are enabled, manual MTU adjustment can be necessary. + +[IMPORTANT] +==== +Generally, the Maximum Transmission Unit (MTU) between an Amazon EC2 instance in a Local Zone and an Amazon EC2 instance in the Region is 1300. For more information, see link:https://docs.aws.amazon.com/local-zones/latest/ug/how-local-zones-work.html[How Local Zones work] in the AWS documentation. +The cluster network MTU must be always less than the EC2 MTU to account for the overhead. The specific overhead is determined by the network plugin, for example: + +- OVN-Kubernetes: `100 bytes` +- OpenShift SDN: `50 bytes` + +The network plugin can provide additional features, like IPsec, that also must be decreased the MTU. For additional information, see the documentation. +==== diff --git a/modules/install-creating-install-config-aws-local-zones.adoc b/modules/install-creating-install-config-aws-local-zones.adoc new file mode 100644 index 000000000000..4ea327ea350c --- /dev/null +++ b/modules/install-creating-install-config-aws-local-zones.adoc @@ -0,0 +1,35 @@ +// Module included in the following assemblies: +// * installing/installing_aws/installing-aws-localzone.adoc + +:_content-type: PROCEDURE +[id="install-creating-install-config-aws-local-zones_{context}"] += Modifying an installation configuration file to use AWS Local Zones subnets + +Modify an `install-config.yaml` file to include AWS Local Zones subnets. + +.Prerequisites + +* You created subnets by using the procedure "Creating a subnet in AWS Local Zones". +* You created an `install-config.yaml` file by using the procedure "Creating the installation configuration file". + +.Procedure + +* Add the VPC and Local Zone subnets as the values of the `platform.aws.subnets` property. As an example: ++ +[source,yaml] +---- +... +platform: + aws: + region: us-west-2 + subnets: <1> + - publicSubnetId-1 + - publicSubnetId-2 + - publicSubnetId-3 + - privateSubnetId-1 + - privateSubnetId-2 + - privateSubnetId-3 + - publicSubnetId-LocalZone-1 +... +---- +<1> List of subnets created in the Availability and Local Zones. \ No newline at end of file diff --git a/modules/installation-aws-add-local-zone-locations.adoc b/modules/installation-aws-add-local-zone-locations.adoc index 41f323342852..691e858b509c 100644 --- a/modules/installation-aws-add-local-zone-locations.adoc +++ b/modules/installation-aws-add-local-zone-locations.adoc @@ -23,29 +23,32 @@ $ export CLUSTER_REGION="" <1> ---- <1> For ``, specify a valid AWS region name, such as `us-east-1`. -. Review the list of zones that your region contains by running the following command: +. List the zones that are available in your region by running the following command: + [source,terminal] ---- -$ aws ec2 describe-availability-zones \ - --filters Name=region-name,Values=${CLUSTER_REGION} \ - --query 'AvailabilityZones[].ZoneName' \ +$ aws --region ${CLUSTER_REGION} ec2 describe-availability-zones \ + --query 'AvailabilityZones[].[{ZoneName: ZoneName, GroupName: GroupName, Status: OptInStatus}]' \ + --filters Name=zone-type,Values=local-zone \ --all-availability-zones ---- + -Depending on the region, the list of available zones can be long. The different zones use the following naming conventions: +Depending on the region, the list of available zones can be long. The command will return the following fields: + -`${REGION}[a-z]`:: Availability zones available in the region. -`${REGION}-LID-N[a-z]`:: Available AWS Local Zones. `${REGION}LID-N` is the zone group identifier, and `[a-z]` is the zone identifier. -`${REGION}-wl1-LID-wlz-[1-9]`:: Available Wavelength zones. +`ZoneName`:: The name of the Local Zone. +`GroupName`:: The group that the zone is part of. You need to save this name to opt in. +`Status`:: The status of the Local Zone group. If the status is `not-opted-in`, you must opt in the `GroupName` by running the commands that follow. . Export a variable to contain the name of the Local Zone to host your VPC by running the following command: + [source,terminal] ---- -$ export ZONE_GROUP_NAME="${CLUSTER_REGION}--" <1> +$ export ZONE_GROUP_NAME="" <1> ---- -<1> For `-`, specify the location identifier and zone identifier for the Local Zone that you selected for your region. For example, specify `nyc-1a` to use the US East (New York) Local Zone. ++ +where: + +:: Specifies the name of the group of the Local Zone you want to create subnets on. For example, specify `us-east-1-nyc-1` to use the zone `us-east-1-nyc-1a`, US East (New York). . Opt in to the zone group on your AWS account by running the following command: + diff --git a/modules/installation-cloudformation-subnet-localzone.adoc b/modules/installation-cloudformation-subnet-localzone.adoc index f37134bc2257..47503db35a56 100644 --- a/modules/installation-cloudformation-subnet-localzone.adoc +++ b/modules/installation-cloudformation-subnet-localzone.adoc @@ -16,20 +16,17 @@ you need for your {product-title} cluster that uses AWS Local Zones. ---- # CloudFormation template used to create Local Zone subnets and dependencies AWSTemplateFormatVersion: 2010-09-09 -Description: Template for Best Practice VPC with 1-3 AZs +Description: Template for create Public Local Zone subnets Parameters: - ClusterName: - Description: ClusterName used to prefix resource names - Type: String VpcId: Description: VPC Id Type: String - LocalZoneName: - Description: Local Zone Name (Example us-east-1-bos-1) + ZoneName: + Description: Local Zone Name (Example us-east-1-nyc-1a) Type: String - LocalZoneNameShort: - Description: Short name for Local Zone used on tag Name (Example bos1) + SubnetName: + Description: Local Zone Name (Example cluster-public-us-east-1-nyc-1a) Type: String PublicRouteTableId: Description: Public Route Table ID to associate the Local Zone subnet @@ -47,12 +44,10 @@ Resources: Properties: VpcId: !Ref VpcId CidrBlock: !Ref PublicSubnetCidr - AvailabilityZone: !Ref LocalZoneName + AvailabilityZone: !Ref ZoneName Tags: - Key: Name - Value: !Join - - "" - - [ !Ref ClusterName, "-public-", !Ref LocalZoneNameShort, "-1" ] + Value: !Ref SubnetName - Key: kubernetes.io/cluster/unmanaged Value: "true" @@ -66,9 +61,6 @@ Outputs: PublicSubnetIds: Description: Subnet IDs of the public subnets. Value: - !Join [ - "", - [!Ref PublicSubnet] - ] + !Join ["", [!Ref PublicSubnet]] ---- ==== diff --git a/modules/installation-cloudformation-vpc-localzone.adoc b/modules/installation-cloudformation-vpc-localzone.adoc index b0c3b8d8ef58..e242a8f0f46d 100644 --- a/modules/installation-cloudformation-vpc-localzone.adoc +++ b/modules/installation-cloudformation-vpc-localzone.adoc @@ -4,10 +4,10 @@ :_content-type: REFERENCE [id="installation-cloudformation-vpc-localzone_{context}"] -= CloudFormation template for the VPC that uses AWS Local Zones += CloudFormation template for the VPC You can use the following CloudFormation template to deploy the VPC that -you need for your {product-title} cluster that uses AWS Local Zones. +you need for your {product-title} cluster. .CloudFormation template for the VPC [%collapsible] @@ -18,9 +18,6 @@ AWSTemplateFormatVersion: 2010-09-09 Description: Template for Best Practice VPC with 1-3 AZs Parameters: - ClusterName: - Type: String - Description: ClusterName used to prefix resource names VpcCidr: AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-4]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-24. @@ -55,8 +52,6 @@ Metadata: Parameters: - AvailabilityZoneCount ParameterLabels: - ClusterName: - default: "" AvailabilityZoneCount: default: "Availability Zone Count" VpcCidr: @@ -75,12 +70,6 @@ Resources: EnableDnsSupport: "true" EnableDnsHostnames: "true" CidrBlock: !Ref VpcCidr - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-vpc" ] ] - - Key: !Join [ "", [ "kubernetes.io/cluster/unmanaged" ] ] - Value: "shared" - PublicSubnet: Type: "AWS::EC2::Subnet" Properties: @@ -89,9 +78,6 @@ Resources: AvailabilityZone: !Select - 0 - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-public-1" ] ] PublicSubnet2: Type: "AWS::EC2::Subnet" Condition: DoAz2 @@ -101,9 +87,6 @@ Resources: AvailabilityZone: !Select - 1 - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-public-2" ] ] PublicSubnet3: Type: "AWS::EC2::Subnet" Condition: DoAz3 @@ -111,31 +94,19 @@ Resources: VpcId: !Ref VPC CidrBlock: !Select [2, !Cidr [!Ref VpcCidr, 6, !Ref SubnetBits]] AvailabilityZone: !Select - - 2 - - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-public-3" ] ] - + - 2 + - Fn::GetAZs: !Ref "AWS::Region" InternetGateway: Type: "AWS::EC2::InternetGateway" - Properties: - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-igw" ] ] GatewayToInternet: Type: "AWS::EC2::VPCGatewayAttachment" Properties: VpcId: !Ref VPC InternetGatewayId: !Ref InternetGateway - PublicRouteTable: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-rtb-public" ] ] PublicRoute: Type: "AWS::EC2::Route" DependsOn: GatewayToInternet @@ -150,15 +121,16 @@ Resources: RouteTableId: !Ref PublicRouteTable PublicSubnetRouteTableAssociation2: Type: "AWS::EC2::SubnetRouteTableAssociation" + Condition: DoAz2 Properties: SubnetId: !Ref PublicSubnet2 RouteTableId: !Ref PublicRouteTable PublicSubnetRouteTableAssociation3: + Condition: DoAz3 Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: SubnetId: !Ref PublicSubnet3 RouteTableId: !Ref PublicRouteTable - PrivateSubnet: Type: "AWS::EC2::Subnet" Properties: @@ -167,16 +139,10 @@ Resources: AvailabilityZone: !Select - 0 - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-private-1" ] ] PrivateRouteTable: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-rtb-private-1" ] ] PrivateSubnetRouteTableAssociation: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: @@ -192,9 +158,6 @@ Resources: - EIP - AllocationId SubnetId: !Ref PublicSubnet - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-natgw-private-1" ] ] EIP: Type: "AWS::EC2::EIP" Properties: @@ -207,7 +170,6 @@ Resources: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: Ref: NAT - PrivateSubnet2: Type: "AWS::EC2::Subnet" Condition: DoAz2 @@ -217,17 +179,11 @@ Resources: AvailabilityZone: !Select - 1 - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-private-2" ] ] PrivateRouteTable2: Type: "AWS::EC2::RouteTable" Condition: DoAz2 Properties: VpcId: !Ref VPC - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-rtb-private-2" ] ] PrivateSubnetRouteTableAssociation2: Type: "AWS::EC2::SubnetRouteTableAssociation" Condition: DoAz2 @@ -245,17 +201,11 @@ Resources: - EIP2 - AllocationId SubnetId: !Ref PublicSubnet2 - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-natgw-private-2" ] ] EIP2: Type: "AWS::EC2::EIP" Condition: DoAz2 Properties: Domain: vpc - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-eip-private-2" ] ] Route2: Type: "AWS::EC2::Route" Condition: DoAz2 @@ -265,7 +215,6 @@ Resources: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: Ref: NAT2 - PrivateSubnet3: Type: "AWS::EC2::Subnet" Condition: DoAz3 @@ -275,17 +224,11 @@ Resources: AvailabilityZone: !Select - 2 - Fn::GetAZs: !Ref "AWS::Region" - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-private-3" ] ] PrivateRouteTable3: Type: "AWS::EC2::RouteTable" Condition: DoAz3 Properties: VpcId: !Ref VPC - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-rtb-private-3" ] ] PrivateSubnetRouteTableAssociation3: Type: "AWS::EC2::SubnetRouteTableAssociation" Condition: DoAz3 @@ -303,17 +246,11 @@ Resources: - EIP3 - AllocationId SubnetId: !Ref PublicSubnet3 - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-natgw-private-3" ] ] EIP3: Type: "AWS::EC2::EIP" Condition: DoAz3 Properties: Domain: vpc - Tags: - - Key: Name - Value: !Join [ "", [ !Ref ClusterName, "-eip-private-3" ] ] Route3: Type: "AWS::EC2::Route" Condition: DoAz3 @@ -323,7 +260,6 @@ Resources: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: Ref: NAT3 - S3Endpoint: Type: AWS::EC2::VPCEndpoint Properties: @@ -369,8 +305,5 @@ Outputs: PublicRouteTableId: Description: Public Route table ID Value: !Ref PublicRouteTable - PrivateRouteTableId: - Description: Private Route table ID - Value: !Ref PrivateRouteTable ---- ==== diff --git a/modules/installation-configuration-parameters.adoc b/modules/installation-configuration-parameters.adoc index 64fc084ef684..d03ae43a408d 100644 --- a/modules/installation-configuration-parameters.adoc +++ b/modules/installation-configuration-parameters.adoc @@ -897,7 +897,13 @@ You can add up to 25 user defined tags during installation. The remaining 25 tag |`platform.aws.subnets` -|If you provide the VPC instead of allowing the installation program to create the VPC for you, specify the subnet for the cluster to use. The subnet must be part of the same `machineNetwork[].cidr` ranges that you specify. For a standard cluster, specify a public and a private subnet for each availability zone. For a private cluster, specify a private subnet for each availability zone. +|If you provide the VPC instead of allowing the installation program to create the VPC for you, specify the subnet for the cluster to use. The subnet must be part of the same `machineNetwork[].cidr` ranges that you specify. + +For a standard cluster, specify a public and a private subnet for each availability zone. + +For a private cluster, specify a private subnet for each availability zone. + +For clusters that use AWS Local Zones, you must add AWS Local Zone subnets to this list to ensure edge machine pool creation. |Valid subnet IDs. |==== diff --git a/modules/installation-creating-aws-subnet-localzone.adoc b/modules/installation-creating-aws-subnet-localzone.adoc index c0726b1ac892..9546ed8e3d8d 100644 --- a/modules/installation-creating-aws-subnet-localzone.adoc +++ b/modules/installation-creating-aws-subnet-localzone.adoc @@ -35,47 +35,33 @@ requires: ---- [ { - "ParameterKey": "ClusterName", <1> - "ParameterValue": "mycluster" <2> + "ParameterKey": "VpcId", + "ParameterValue": "" <1> }, { - "ParameterKey": "VpcId", <3> - "ParameterValue": "vpc-" <4> + "ParameterKey": "PublicRouteTableId", + "ParameterValue": "" <2> }, { - "ParameterKey": "PublicRouteTableId", <5> - "ParameterValue": "" <6> + "ParameterKey": "ZoneName", + "ParameterValue": "" <3> }, { - "ParameterKey": "LocalZoneName", <7> - "ParameterValue": "--" <8> + "ParameterKey": "SubnetName", + "ParameterValue": "" }, { - "ParameterKey": "LocalZoneNameShort", <9> - "ParameterValue": "" <10> - }, - { - "ParameterKey": "PublicSubnetCidr", <11> - "ParameterValue": "10.0.128.0/20" <12> + "ParameterKey": "PublicSubnetCidr", + "ParameterValue": "10.0.192.0/20" <4> } ] ---- -<1> A short, representative cluster name to use for hostnames, etc. -<2> Specify the cluster name that you used when you generated the `install-config.yaml` file for the cluster. -<3> The VPC ID in which the Local Zone's subnet will be created. -<4> Specify the `VpcId` value from the output of the CloudFormation template +<1> Specify the VPC ID, which is the value `VpcID` in the output of the CloudFormation template. for the VPC. -<5> The Public Route Table ID for the VPC. -<6> Specify the `PublicRouteTableId` value from the output of the CloudFormation template for the VPC. -<7> The Local Zone name that the VPC belongs to. -<8> Specify the Local Zone that you opted your AWS account into, such as `us-east-1-nyc-1a`. -<9> The shortname of the AWS Local Zone that the VPC belongs to. -<10> Specify a short name for the AWS Local Zone that you opted your AWS account into, such as ``. For example, `us-east-1-nyc-1a` is shortened to `nyc-1a`. -//How do we determine this shortname? -<11> The CIDR block to allow access to the Local Zone. -<12> Specify a CIDR block in the format `x.x.x.x/16-24`. -//How do we know what this CIDR is? - +<2> Specify the Route Table ID, which is the value of the `PublicRouteTableId` in the CloudFormation stack +for the VPC. +<3> Specify the AWS Local Zone name, which is the value of the `ZoneName` field in the `AvailabilityZones` object that you retrieve in the section "Opting into AWS Local Zones". +<4> Specify a CIDR block that is used to create the Local Zone subnet. This block must be part of the VPC CIDR block `VpcCidr`. . Copy the template from the *CloudFormation template for the subnet* section of this topic and save it as a YAML file on your computer. This template @@ -104,7 +90,7 @@ parameters JSON file. .Example output [source,terminal] ---- -arn:aws:cloudformation:us-east-1:123456789012:stack/cluster-lz-nyc1/dbedae40-2fd3-11eb-820e-12a48460849f +arn:aws:cloudformation:us-east-1:123456789012:stack//dbedae40-2fd3-11eb-820e-12a48460849f ---- . Confirm that the template components exist by running the following command: diff --git a/modules/installation-creating-aws-vpc-localzone.adoc b/modules/installation-creating-aws-vpc-localzone.adoc index 3b4e12689de0..1cca351d8450 100644 --- a/modules/installation-creating-aws-vpc-localzone.adoc +++ b/modules/installation-creating-aws-vpc-localzone.adoc @@ -35,32 +35,25 @@ requires: ---- [ { - "ParameterKey": "ClusterName", <1> - "ParameterValue": "mycluster" <2> + "ParameterKey": "VpcCidr", <1> + "ParameterValue": "10.0.0.0/16" <2> }, { - "ParameterKey": "VpcCidr", <3> - "ParameterValue": "10.0.0.0/16" <4> + "ParameterKey": "AvailabilityZoneCount", <3> + "ParameterValue": "3" <4> }, { - "ParameterKey": "AvailabilityZoneCount", <5> - "ParameterValue": "3" <6> - }, - { - "ParameterKey": "SubnetBits", <7> - "ParameterValue": "12" <8> + "ParameterKey": "SubnetBits", <5> + "ParameterValue": "12" <6> } ] ---- -<1> A short, representative cluster name to use for hostnames, etc. -<2> Specify the cluster name that you used when you generated the -`install-config.yaml` file for the cluster. -<3> The CIDR block for the VPC. -<4> Specify a CIDR block in the format `x.x.x.x/16-24`. -<5> The number of availability zones to deploy the VPC in. -<6> Specify an integer between `1` and `3`. -<7> The size of each subnet in each availability zone. -<8> Specify an integer between `5` and `13`, where `5` is `/27` and `13` is `/19`. +<1> The CIDR block for the VPC. +<2> Specify a CIDR block in the format `x.x.x.x/16-24`. +<3> The number of availability zones to deploy the VPC in. +<4> Specify an integer between `1` and `3`. +<5> The size of each subnet in each availability zone. +<6> Specify an integer between `5` and `13`, where `5` is `/27` and `13` is `/19`. . Copy the template from the *CloudFormation template for the VPC* section of this topic and save it as a YAML file on your computer. This template diff --git a/modules/installation-generate-aws-user-infra-install-config.adoc b/modules/installation-generate-aws-user-infra-install-config.adoc index 9501617b1345..b81cf578cecc 100644 --- a/modules/installation-generate-aws-user-infra-install-config.adoc +++ b/modules/installation-generate-aws-user-infra-install-config.adoc @@ -130,24 +130,6 @@ publish: Internal By setting this option, you create an internal Ingress Controller and a private load balancer. endif::restricted[] -ifdef::localzone[] -. Edit the `install-config.yaml` file to provide the subnets for the availability zones that your VPC uses: -+ -[source,yaml] ----- -platform: - aws: - subnets: <1> - - publicSubnetId-1 - - publicSubnetId-2 - - publicSubnetId-3 - - privateSubnetId-1 - - privateSubnetId-2 - - privateSubnetId-3 ----- -<1> Add the `subnets` section and specify the `PrivateSubnetIds` and `PublicSubnetIds` values from the outputs of the CloudFormation template for the VPC. Do not include the Local Zone subnets here. -endif::localzone[] - ifdef::three-node-cluster[] . If you are installing a three-node cluster, modify the `install-config.yaml` file by setting the `compute.replicas` parameter to `0`. This ensures that the cluster's control planes are schedulable. For more information, see "Installing a three-node cluster on AWS". endif::three-node-cluster[] diff --git a/modules/installation-user-infra-generate.adoc b/modules/installation-user-infra-generate.adoc index d27f72edec72..de6a6362cb8f 100644 --- a/modules/installation-user-infra-generate.adoc +++ b/modules/installation-user-infra-generate.adoc @@ -84,7 +84,7 @@ ifdef::aws,gcp[] To install {product-title} on {cp-first} ({cp}) using user-provisioned infrastructure, you must generate the files that the installation program needs to deploy your cluster and modify them so that the cluster creates only the machines that it will use. You generate and customize the `install-config.yaml` file, Kubernetes manifests, and Ignition config files. You also have the option to first set up a separate `var` partition during the preparation phases of installation. endif::aws,gcp[] ifdef::localzone[] -To install {product-title} on {cp-first} ({cp}) and use AWS Local Zones, you must generate the files that the installation program needs to deploy your cluster and modify them so that the cluster creates only the machines that it will use. You generate and customize the `install-config.yaml` file and Kubernetes manifests. +To install {product-title} on {cp-first} ({cp}) and use AWS Local Zones, you must generate the files that the installation program needs to deploy your cluster and modify them so that the cluster creates only the machines that it will use. You generate and customize the `install-config.yaml` file and configure add Local Zone subnets to it. endif::localzone[] ifdef::gcp-shared[] To install {product-title} on {cp-first} ({cp}) into a shared VPC, you must generate the `install-config.yaml` file and modify it so that the cluster uses the correct VPC networks, DNS zones, and project names. diff --git a/modules/machine-edge-pool-review-nodes.adoc b/modules/machine-edge-pool-review-nodes.adoc new file mode 100644 index 000000000000..de108e5aaa10 --- /dev/null +++ b/modules/machine-edge-pool-review-nodes.adoc @@ -0,0 +1,58 @@ +// Module included in the following assemblies +// * installing/installing_aws/installing-aws-localzone.adoc + +:_content-type: PROCEDURE +[id="machine-edge-pool-review-nodes_{context}"] += Verifying nodes that were created with edge compute pool + +After you install a cluster that uses AWS Local Zones, check the status of the machine that was created by the machine set manifests created at install time. + +. To check the machine sets created from the subnet you added to the `install-config.yaml` file, run the following command: ++ +[source,terminal] +---- +$ oc get machineset -n openshift-machine-api +---- ++ +.Example output +[source,terminal] +---- +NAME DESIRED CURRENT READY AVAILABLE AGE +cluster-7xw5g-edge-us-east-1-nyc-1a 1 1 1 1 3h4m +cluster-7xw5g-worker-us-east-1a 1 1 1 1 3h4m +cluster-7xw5g-worker-us-east-1b 1 1 1 1 3h4m +cluster-7xw5g-worker-us-east-1c 1 1 1 1 3h4m +---- + +. To check the machines that were created from the machine sets, run the following command: ++ +[source,terminal] +---- +$ oc get machines -n openshift-machine-api +---- ++ +.Example output +---- +NAME PHASE TYPE REGION ZONE AGE +cluster-7xw5g-edge-us-east-1-nyc-1a-wbclh Running c5d.2xlarge us-east-1 us-east-1-nyc-1a 3h +cluster-7xw5g-master-0 Running m6i.xlarge us-east-1 us-east-1a 3h4m +cluster-7xw5g-master-1 Running m6i.xlarge us-east-1 us-east-1b 3h4m +cluster-7xw5g-master-2 Running m6i.xlarge us-east-1 us-east-1c 3h4m +cluster-7xw5g-worker-us-east-1a-rtp45 Running m6i.xlarge us-east-1 us-east-1a 3h +cluster-7xw5g-worker-us-east-1b-glm7c Running m6i.xlarge us-east-1 us-east-1b 3h +cluster-7xw5g-worker-us-east-1c-qfvz4 Running m6i.xlarge us-east-1 us-east-1c 3h +---- + +. To check nodes with edge roles, run the following command: ++ +[source,terminal] +---- +$ oc get nodes -l node-role.kubernetes.io/edge +---- ++ +.Example output +[source,terminal] +---- +NAME STATUS ROLES AGE VERSION +ip-10-0-207-188.ec2.internal Ready edge,worker 172m v1.25.2+d2e245f +---- \ No newline at end of file diff --git a/modules/machines-edge-machine-pool.adoc b/modules/machines-edge-machine-pool.adoc new file mode 100644 index 000000000000..5054d3529c72 --- /dev/null +++ b/modules/machines-edge-machine-pool.adoc @@ -0,0 +1,105 @@ +// Module included in the following assemblies +// * installing/installing_aws/installing-aws-localzone.adoc + +:_content-type: CONCEPT +[id="machines-edge-machine-pool_{context}"] += The edge compute pool for AWS Local Zones + +{product-title} 4.12 introduced a new compute pool, _edge_, that is designed for use in remote zones. The edge compute pool configuration is common between AWS Local Zone locations. However, due to the type and size limitations of resources like EC2 and EBS on Local Zone resources, the default instance type that is created can vary from the traditional worker pool. + +The default Elastic Block Store (EBS) for Local Zone locations is `gp2`, which differs from the regular worker pool. The instance type used for each Local Zone on edge compute pool also might differ from worker pools, depending on the instance offerings on the zone. + +The edge compute pool creates new labels that developers can use to deploy applications onto AWS Local Zone nodes. The new labels are: + +* `node-role.kubernetes.io/edge=''` +* `machine.openshift.io/zone-type=local-zone` +* `machine.openshift.io/zone-group=$ZONE_GROUP_NAME` + + +By default, the system creates the edge compute pool manifests only if users add AWS Local Zone subnet IDs to the list `platform.aws.subnets`. + +The edge compute pool's machine sets have a `NoSchedule taint` by default to prevent regular workloads from being spread out on those machines. Users can only run user workloads if the tolerations are defined on the pod spec. + +The following examples show `install-config.yaml` files that use the edge machine pool. + +.Configuration that uses an edge pool with default settings +[source,yaml] +---- +apiVersion: v1 +baseDomain: devcluster.openshift.com +metadata: + name: ipi-localzone +platform: + aws: + region: us-west-2 + subnets: + - publicSubnetId-1 + - publicSubnetId-2 + - publicSubnetId-3 + - privateSubnetId-1 + - privateSubnetId-2 + - privateSubnetId-3 + - publicSubnetId-LocalZone-1 + pullSecret: '{"auths": ...}' +sshKey: ssh-ed25519 AAAA... +---- + +.Configuration that uses an edge pool with a custom instance type +[source,yaml] +---- +apiVersion: v1 +baseDomain: devcluster.openshift.com +metadata: + name: ipi-localzone +compute: +- name: edge + platform: + aws: + type: m5.4xlarge +platform: + aws: + region: us-west-2 + subnets: + - publicSubnetId-1 + - publicSubnetId-2 + - publicSubnetId-3 + - privateSubnetId-1 + - privateSubnetId-2 + - privateSubnetId-3 + - publicSubnetId-LocalZone-1 +pullSecret: '{"auths": ...}' +sshKey: ssh-ed25519 AAAA... +---- + +Instance types differ between locations. To verify availability in the Local Zone in which the cluster will run, see the AWS documentation. + +.Configuration that uses an edge pool with a custom EBS type +[source,yaml] +---- +apiVersion: v1 +baseDomain: devcluster.openshift.com +metadata: + name: ipi-localzone +compute: +- name: edge + platform: + aws: + rootVolume: + type: gp3 + size: 120 +platform: + aws: + region: us-west-2 + subnets: + - publicSubnetId-1 + - publicSubnetId-2 + - publicSubnetId-3 + - privateSubnetId-1 + - privateSubnetId-2 + - privateSubnetId-3 + - publicSubnetId-LocalZone-1 +pullSecret: '{"auths": ...}' +sshKey: ssh-ed25519 AAAA... +---- + +EBS types differ between locations. Check the AWS documentation to verify availability in the Local Zone in which the cluster will run. \ No newline at end of file