Allow master to touch volumes tagged with kubernetes.io/cluster/<clusterName>:owned

[wongma7]

This fixes a specific situation where CSIMigrationAWS gets enabled then disabled, step by step details are below.

Basically, kops KCM in-tree EBS plugin and CSI EBS plugin need to agree on what tags to add to provisioned volumes and what IAM policies to ship with such that those provisioned volumes can be managed by either KCM or CSI. Otherwise when migration gets toggled, KCM/CSI won't be allowed to Attach/Delete volumes that were provisioned by CSI/KCM before the toggle. For more details, see: kubernetes-sigs/aws-ebs-csi-driver#927 (comment)

1. Enable CSIMigrationAWS
2. Deploy CSI EBS plugin and provide the clusterName via the k8s-tag-cluster-id flag. This tells it to create volumes with the kubernetes.io/cluster/:owned tag, to partially mimic KCM volume creation behavior. Partially because KCM creates volumes with the legacy KubernetesCluster: tag too.
3. With the default StorageClass with provisioner kubernetes.io/aws-ebs installed, create a PVC.
4. Since migration is enabled, CSI EBS plugin acts as provisioner kubernetes.io/aws-ebs. It creates a volume with tag kubernetes.io/cluster/:owned and corresponding PV. This volume can be attached, detached, and deleted by CSI.
5. Disable CSIMigrationAWS (let's say I decide I don't like CSI...I expect my existing volumes with spec awsElasticBlockStore to go back to being managed by the in-tree plugin instead of CSI)
6. The volume created in step 3 cannot be attached or deleted by KCM because it is only allowed to attach or delete volumes with the legacy KubernetesCluster: tag. (This goes against my expectation.)

So to fix step 6, this PR grants master instance on which KCM is running permission to touch volumes tagged with kubernetes.io/cluster/:owned in addition to the legacy KubernetesCluster: tag.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johngmyers

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [johngmyers]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rifelpet]

These aren't related to volumes and the EBS CSI driver so I'm curious why they're being included here.

[olemarkus]

Looks like a mistake to me. But I am planning on moving the EBS specific stuff into a dedicated function to support IRSA. Can remove it as part of that PR.

[wongma7]

yes I just copied the existing KubernetesCluster policy without questioning it

[olemarkus]

I believe we add legacy tags to volumes provisioned by the CSI driver as well.

[johngmyers]

We should be moving away from depending on the legacy tags. In a release we should probably drop the old policy statement.

[wongma7]

@olemarkus , the CSI driver needs this patch to add legacy tags kubernetes-sigs/aws-ebs-csi-driver#932. The CSI driver installed by kops will need to have the --k8s-tag-cluster-id argument set on it if not already, I'll check

[olemarkus]

We pass it in through --extra-tags. It is a generic list of tags that users can configure on all resources kOps/k8s provision. By default it contains the KubernetesCluster tag.