IAM policy changes required for kube-router

[drobinson123]

2. What kops version are you running? use kops version

Version 1.8.0-beta.1 (git-9b71713)

3. What Kubernetes version are you running?

1.8.2 -- I bumped kubernetesVersion

4. What cloud provider are you using?

AWS

5. What commands did you execute and what happened after commands executed?

kops create cluster \
  --zones=${cluster} \
  --name=${cluster}.${domain} \
  --vpc=${vpc} \
  --authorization=RBAC \
  --dns private \
  --node-size=m4.xlarge \
  --master-size=m3.medium \
  --node-count=${node_count} \
  --master-zones=${cluster} \
  --target=terraform \
  --topology private \
  --networking kube-router \
  --ssh-public-key ~/.ssh/kube_aws_rsa.pub

kops edit cluster --name=${CLUSTER_NAME}
# changed
kubernetesVersion: 1.8.2

kops edit instancegroup...
# changed tme image for nodes and masters:
image: 595879546273/CoreOS-stable-1520.8.0-hvm

6. What you expected to happen:

A working cluster.

7. How can we to reproduce it (as minimally and precisely as possible):

8. Anything else do we need to know:

The IAM node policy doesn't grant access to the kube-router path in S3 (eg s3://mycluster.example.com-state-store/mycluster.example.com/pki/private/kube-router/blah.key). S3 returns a 403. Node logs show nodeup failing w/ access denied.

I was able to work around the problem by manually editing the IAM policy (adding "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/private/kube-router/*")

        {
            "Sid": "kopsK8sS3NodeBucketSelectiveGet",
            "Effect": "Allow",
            "Action": [
                "s3:Get*"
            ],
            "Resource": [
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/addons/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/cluster.spec",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/config",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/instancegroup/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/issued/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/private/kube-proxy/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/private/kube-router/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/private/kubelet/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/pki/ssh/*",
                "arn:aws:s3:::us-west-2a.token.io-state-store/us-west-2a.token.io/secrets/dockerconfig"
            ]
        },

The commit that tightened access is here.

[chrislovecnm]

You can add a policy as well to the cluster as a work around.

[chrislovecnm]

More details. Kops supports adding another policy to your cluster as well