Skip to content

Commit

Permalink
Merge pull request #2311 from jparrill/doc/HOSTEDCP-736
Browse files Browse the repository at this point in the history 
HOSTEDCP-736: Docs on how we handle aws permissions
  • Loading branch information
@openshift-merge-robot
openshift-merge-robot committed Mar 23, 2023
2 parents 4d1fa83 + 986e259 commit a0377ae
Showing 1 changed file with 360 additions and 4 deletions.
364 changes: 360 additions & 4 deletions docs/content/reference/infrastructure/aws.md
View file
@@ -1,5 +1,5 @@

## Infrastructure - AWS
## Infrastructure

In this section we want to dissect who creates what and what not. It contains 4 stages:

Expand Down Expand Up @@ -92,6 +92,362 @@ In this section we want to dissect who creates what and what not. It contains 4
- S3 bucket for registry

!!! note
For the Private Link networking to work, we've observed that the Endpoint zone in the hosted cluster AWS account, must match the zone of the instance resolved by the Service Endpoint in the management cluster AWS account.
In AWS the Zone names are just alias e.g. "us-east-2b" which do not necessarily map to the same zone in different accounts.
Because of this for Private link to be guaranteed to work, the management cluster must have subnets/workers in all zones of its region.

For the Private Link networking to work, we've observed that the Endpoint zone in the hosted cluster AWS account, must match the zone of the instance resolved by the Service Endpoint in the management cluster AWS account.

In AWS the Zone names are just alias e.g. "us-east-2b" which do not necessarily map to the same zone in different accounts.

Because of this for Private link to be guaranteed to work, the management cluster must have subnets/workers in all zones of its region.

## IAM

In this section we want to clarify how the AWS IAM works in Hypershift context.

Hypershift expects to have the ARN roles already created in AWS, so the entity responsible to create them lies in the consumer (CLI/OCM). Hypershift tries to enable granularity to honor principle of least privilege components, which means that every component will use their own role to operate or create AWS objects and the roles are limited to what is required for the normal functioning of the product.

As an example, our CLI can handle the creation of these roles, you can follow [this article](../../how-to/aws/create-infra-iam-separately.md) to make this happen.

The HostedCluster receives the ARN roles as input, then the CLI/OCM creates an AWS permission config for each component. That lets a component to auth via [STS](https://github.com/openshift/managed-cluster-config/tree/master/resources/sts) and preconfigured oidc idp.

The roles created are consumed by some components from Hypershift, running on Control Plane and operating at Data Plane side:

These are the different roles

- **controlPlaneOperatorARN**
- **imageRegistryARN**
- **ingressARN**
- **kubeCloudControllerARN**
- **nodePoolManagementARN**
- **storageARN**
- **networkARN**

This is a sample of how this looks like the reference to the IAM Roles from the HostedCluster perspective

```yaml
...
endpointAccess: Public
region: us-east-2
resourceTags:
- key: kubernetes.io/cluster/cewong-dev-bz4j5
value: owned
rolesRef:
controlPlaneOperatorARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-control-plane-operator
imageRegistryARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-openshift-image-registry
ingressARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-openshift-ingress
kubeCloudControllerARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-cloud-controller
networkARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-cloud-network-config-controller
nodePoolManagementARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-node-pool
storageARN: arn:aws:iam::820196288204:role/cewong-dev-bz4j5-aws-ebs-csi-driver-controller
type: AWS
...
```

And these are samples for each one of the roles Hypershift uses:


=== "**IngressARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticloadbalancing:DescribeLoadBalancers",
"tag:GetResources",
"route53:ListHostedZones"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::PUBLIC_ZONE_ID",
"arn:aws:route53:::PRIVATE_ZONE_ID"
]
}
]
}
```

=== "**ImageRegistryARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:PutBucketTagging",
"s3:GetBucketTagging",
"s3:PutBucketPublicAccessBlock",
"s3:GetBucketPublicAccessBlock",
"s3:PutEncryptionConfiguration",
"s3:GetEncryptionConfiguration",
"s3:PutLifecycleConfiguration",
"s3:GetLifecycleConfiguration",
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucketMultipartUploads",
"s3:AbortMultipartUpload",
"s3:ListMultipartUploadParts"
],
"Resource": "*"
}
]
}
```
=== "**StorageARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteSnapshot",
"ec2:DeleteTags",
"ec2:DeleteVolume",
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications",
"ec2:DetachVolume",
"ec2:ModifyVolume"
],
"Resource": "*"
}
]
}
```

=== "**NetworkARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstanceTypes",
"ec2:UnassignPrivateIpAddresses",
"ec2:AssignPrivateIpAddresses",
"ec2:UnassignIpv6Addresses",
"ec2:AssignIpv6Addresses",
"ec2:DescribeSubnets",
"ec2:DescribeNetworkInterfaces"
],
"Resource": "*"
}
]
}
```

=== "**KubeCloudControllerARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeImages",
"ec2:DescribeRegions",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVolumes",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume",
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateRoute",
"ec2:DeleteRoute",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVolume",
"ec2:DetachVolume",
"ec2:RevokeSecurityGroupIngress",
"ec2:DescribeVpcs",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"iam:CreateServiceLinkedRole",
"kms:DescribeKey"
],
"Resource": [
"*"
],
"Effect": "Allow"
}
]
}
```

=== "**NodePoolManagementARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:AllocateAddress",
"ec2:AssociateRouteTable",
"ec2:AttachInternetGateway",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateInternetGateway",
"ec2:CreateNatGateway",
"ec2:CreateRoute",
"ec2:CreateRouteTable",
"ec2:CreateSecurityGroup",
"ec2:CreateSubnet",
"ec2:CreateTags",
"ec2:DeleteInternetGateway",
"ec2:DeleteNatGateway",
"ec2:DeleteRouteTable",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSubnet",
"ec2:DeleteTags",
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeInternetGateways",
"ec2:DescribeNatGateways",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeNetworkInterfaceAttribute",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"ec2:DescribeVpcAttribute",
"ec2:DescribeVolumes",
"ec2:DetachInternetGateway",
"ec2:DisassociateRouteTable",
"ec2:DisassociateAddress",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:ModifySubnetAttribute",
"ec2:ReleaseAddress",
"ec2:RevokeSecurityGroupIngress",
"ec2:RunInstances",
"ec2:TerminateInstances",
"tag:GetResources",
"ec2:CreateLaunchTemplate",
"ec2:CreateLaunchTemplateVersion",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeLaunchTemplateVersions",
"ec2:DeleteLaunchTemplate",
"ec2:DeleteLaunchTemplateVersions"
],
"Resource": [
"*"
],
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
}
},
"Action": [
"iam:CreateServiceLinkedRole"
],
"Resource": [
"arn:*:iam::*:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing"
],
"Effect": "Allow"
},
{
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:*:iam::*:role/*-worker-role"
],
"Effect": "Allow"
}
]
}
```

=== "**ControlPlaneOperatorARN**"

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateVpcEndpoint",
"ec2:DescribeVpcEndpoints",
"ec2:ModifyVpcEndpoint",
"ec2:DeleteVpcEndpoints",
"ec2:CreateTags",
"route53:ListHostedZones"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets"
],
"Resource": "arn:aws:route53:::%s"
}
]
}
```

0 comments on commit a0377ae

Please sign in to comment.