Merge pull request #56409 from lpettyjo/OSDOCS-5199

OSDOCS-5199:AWS EFS cross account support

modules/persistent-storage-csi-efs-cross-account.adoc

@@ -0,0 +1,288 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi-aws-efs.adoc
+// 
+
+:_content-type: PROCEDURE
+[id="persistent-storage-csi-efs-cross-account_{context}"]
+= AWS EFS CSI cross account support
+
+Cross account support allows you to have an {product-title} cluster in one AWS account and mount your file system in another AWS account using the AWS Elastic File System (EFS) Container Storage Interface (CSI) driver.
+
+[NOTE]
+====
+Both the {product-title} cluster and EFS file system must be in the same region.
+====
+
+.Prerequisites
+
+* Access to an {product-title} cluster with administrator rights
+
+* Two valid AWS accounts
+
+.Procedure
+
+The following procedure demonstrates how to set up:
+
+* {product-title} cluster in AWS account A
+
+* Mount an AWS EFS file system in account B
+
+To use AWS EFS across accounts:
+
+. Install {product-title} cluster with AWS account A and install the EFS CSI Driver Operator.
+
+. Create an EFS volume in AWS account B:
+
+.. Create a virtual private cloud (VPC) called, for example, "my-efs-vpc” with CIDR, for example, “172.20.0.0/16” and subnet for the AWS EFS volume.
+
+.. On the AWS console, go to https://console.aws.amazon.com/efs.
+
+.. Click *Create new filesystem*:
+
+... Create a filesystem named, for example, "my-filesystem”.
+
+... Select the VPC created earlier (“my-efs-vpc”). 
+
+... Accept the default for the remaining settings.
+
+.. Ensure that the volume and Mount Targets have been created:
+
+... Check https://console.aws.amazon.com/efs#/file-systems.
+
+... Click your volume, and on the *Network* tab wait for all Mount Targets to be available (approximately 1-2 minutes).
+
+.. On the *Network* tab, copy the Security Group ID. You will need it for the next step.
+
+. Configure networking access to the AWS EFS volume on AWS account B:
+
+.. Go to https://console.aws.amazon.com/ec2/v2/home#SecurityGroups.
+
+.. Find the Security Group used by the AWS EFS volume by filtering for the group ID copied earlier.
+
+.. On the *Inbound rules* tab, click  *Edit inbound rules*, and then add a new rule to allow {product-title} nodes to access the AWS EFS volumes (that is, use NFS ports from the cluster):
++
+* *Type*: NFS
+* *Protocol*: TCP
+* *Port range*: 2049
+* *Source*: Custom/IP address range of your {product-title} cluster nodes (for example, “10.0.0.0/16”)
+
+.. Save the rule.
++
+[NOTE]
+====
+If you encounter mounting issues, re-check the port number, IP address range, and verify that the AWS EFS volume uses the expected security group.
+====
+
+. Create VPC peering between the {product-title} cluster VPC in AWS account A and the AWS EFS VPC in AWS account B:
++
+Ensure the two VPCs are using different network CIDRs, and after creating the VPC peering, add routes in each VPC to connect the two VPC networks.
+
+.. Create a peering connection called, for example, “my-efs-crossaccount-peering-connection” in account B. For the local VPC ID, use the EFS-located VPC. To peer with the VPC for account A, for the VPC ID use the {product-title} cluster VPC ID.
+
+.. Accept the peer connection in AWS account A.
+
+.. Modify the route table of each subnet (EFS-volume used subnets) in AWS account B:
+
+... On the left pane, under *Virtual private cloud*, click the down arrow to expand the available options.
+
+... Under *Virtual private cloud*, click *Route tables"*.
+
+... Click the *Routes* tab.
+
+... Under *Destination*, enter 10.0.0.0/16. 
+
+... Under *Target*, use the peer connection type point from the created peer connection.
+
+.. Modify the route table of each subnet ({product-title} cluster nodes used subnets) in AWS account A:
+
+... On the left pane, under *Virtual private cloud*, click the down arrow to expand the available options.
+
+... Under *Virtual private cloud*, click *Route tables"*.
+
+... Click the *Routes* tab.
+
+... Under *Destination*, enter the CIDR for the VPC in account B, which for this example is 172.20.0.0/16. 
+
+... Under *Target*, use the peer connection type point from the created peer connection.
+
+. Create an IAM role, for example, “my-efs-acrossaccount-role” in AWS account B, which has a trust relationship with AWS account A, and add an inline AWS EFS policy with permissions to call “my-efs-acrossaccount-driver-policy”. 
++
+This role is used by the CSI driver's controller service running on the {product-title} cluster in AWS account A to determine the mount targets for your file system in AWS account B.
++
+[source, json]
+----
+# Trust relationships trusted entity trusted account A configuration on my-efs-acrossaccount-role in account B
+
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Principal": {
+                "AWS": "arn:aws:iam::301721915996:root"
+            },
+            "Action": "sts:AssumeRole",
+            "Condition": {}
+        }
+    ]
+}
+
+# my-cross-account-assume-policy policy attached to my-efs-acrossaccount-role in account B
+
+{
+    "Version": "2012-10-17",
+    "Statement": {
+        "Effect": "Allow",
+        "Action": "sts:AssumeRole",
+        "Resource": "arn:aws:iam::589722580343:role/my-efs-acrossaccount-role"
+    }
+}
+
+# my-efs-acrossaccount-driver-policy attached to my-efs-acrossaccount-role in account B
+
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": [
+                "ec2:DescribeNetworkInterfaces",
+                "ec2:DescribeSubnets"
+            ],
+            "Resource": "*"
+        },
+        {
+            "Sid": "VisualEditor1",
+            "Effect": "Allow",
+            "Action": [
+                "elasticfilesystem:DescribeMountTargets",
+                "elasticfilesystem:DeleteAccessPoint",
+                "elasticfilesystem:ClientMount",
+                "elasticfilesystem:DescribeAccessPoints",
+                "elasticfilesystem:ClientWrite",
+                "elasticfilesystem:ClientRootAccess",
+                "elasticfilesystem:DescribeFileSystems",
+                "elasticfilesystem:CreateAccessPoint"
+            ],
+            "Resource": [
+                "arn:aws:elasticfilesystem:*:589722580343:access-point/*",
+                "arn:aws:elasticfilesystem:*:589722580343:file-system/*"
+            ]
+        }
+    ]
+}
+----
+
+. In AWS account A, attach an inline policy to the IAM role of the AWS EFS CSI driver's controller service account with the necessary permissions to perform Security Token Service (STS) assume role on the IAM role created earlier.
++
+[source, json]
+----
+# my-cross-account-assume-policy policy attached to Openshift cluster efs csi driver user in account A
+
+{
+    "Version": "2012-10-17",
+    "Statement": {
+        "Effect": "Allow",
+        "Action": "sts:AssumeRole",
+        "Resource": "arn:aws:iam::589722580343:role/my-efs-acrossaccount-role"
+    }
+}
+----
+
+. In AWS account A, attach the AWS-managed policy “AmazonElasticFileSystemClientFullAccess” to {product-title} cluster master role. The role name is in the form `<clusterID>-master-role` (for example, `my-0120ef-czjrl-master-role`).
+
+. Create a Kubernetes secret with `awsRoleArn` as the key and the role created earlier as the value:
++
+[source, cli]
+----
+$ oc -n openshift-cluster-csi-drivers create secret generic my-efs-cross-account --from-literal=awsRoleArn='arn:aws:iam::589722580343:role/my-efs-acrossaccount-role'
+----
++
+Since the driver controller needs to get the cross account role information from the secret, you need to add the secret role binding to the AWS EFS CSI driver controller ServiceAccount (SA):
++
+[source, cli]
+----
+$ oc -n openshift-cluster-csi-drivers create role access-secrets --verb=get,list,watch --resource=secrets
+
+$ oc -n openshift-cluster-csi-drivers create rolebinding --role=access-secrets default-to-secrets --serviceaccount=openshift-cluster-csi-drivers:aws-efs-csi-driver-controller-sa
+----
+
+. Create a `filesystem` policy for the file system (AWS EFS volume) in account B, which allows AWS account A to perform a mount on it. 
++
+[NOTE]
+----
+This step is not mandatory, but can be safer for AWS EFS volume usage.
+----
++
+[source, json]
+----
+# EFS volume filesystem policy in account B
+{
+    "Version": "2012-10-17",
+    "Id": "efs-policy-wizard-8089bf4a-9787-40f0-958e-bc2363012ace",
+    "Statement": [
+        {
+            "Sid": "efs-statement-bd285549-cfa2-4f8b-861e-c372399fd238",
+            "Effect": "Allow",
+            "Principal": {
+                "AWS": "*"
+            },
+            "Action": [
+                "elasticfilesystem:ClientRootAccess",
+                "elasticfilesystem:ClientWrite",
+                "elasticfilesystem:ClientMount"
+            ],
+            "Resource": "arn:aws:elasticfilesystem:us-east-2:589722580343:file-system/fs-091066a9bf9becbd5",
+            "Condition": {
+                "Bool": {
+                    "elasticfilesystem:AccessedViaMountTarget": "true"
+                }
+            }
+        },
+        {
+            "Sid": "efs-statement-03646e39-d80f-4daf-b396-281be1e43bab",
+            "Effect": "Allow",
+            "Principal": {
+                "AWS": "arn:aws:iam::589722580343:role/my-efs-acrossaccount-role"
+            },
+            "Action": [
+                "elasticfilesystem:ClientRootAccess",
+                "elasticfilesystem:ClientWrite",
+                "elasticfilesystem:ClientMount"
+            ],
+            "Resource": "arn:aws:elasticfilesystem:us-east-2:589722580343:file-system/fs-091066a9bf9becbd5"
+        }
+    ]
+}
+----
+
+. Create an AWS EFS volume storage class using a similar configuration to the following:
++
+[source, yaml]
+----
+# The cross account efs volume storageClass
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+  name: efs-cross-account-mount-sc
+provisioner: efs.csi.aws.com
+mountOptions:
+  - tls
+parameters:
+  provisioningMode: efs-ap
+  fileSystemId: fs-00f6c3ae6f06388bb
+  directoryPerms: "700"
+  gidRangeStart: "1000"
+  gidRangeEnd: "2000"
+  basePath: "/account-a-data"
+  csi.storage.k8s.io/provisioner-secret-name: my-efs-cross-account
+  csi.storage.k8s.io/provisioner-secret-namespace: openshift-cluster-csi-drivers
+volumeBindingMode: Immediate
+----
+
+
+
+
+

storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc

@@ -51,6 +51,8 @@ include::modules/storage-create-storage-class.adoc[leveloffset=+1]
 include::modules/storage-create-storage-class-console.adoc[leveloffset=+2]
 include::modules/storage-create-storage-class-cli.adoc[leveloffset=+2]
 
+include::modules/persistent-storage-csi-efs-cross-account.adoc[leveloffset=+1]
+
 include::modules/persistent-storage-csi-efs-create-volume.adoc[leveloffset=+1]
 
 include::modules/persistent-storage-csi-dynamic-provisioning-aws-efs.adoc[leveloffset=+1]