Better support for aws-iam-authenticator

[joshmyers]

what

This commit adds a command kopsctl login to build a kubecfg that optionally can use
aws-iam-authenticator, given we have the KOPS_AWS_IAM_AUTHENTICATOR_ENABLED set to true

why

We want to enable RBAC using aws-iam-authenticator to auth users against the Kops cluster via IAM roles.

Note that we have to fetch the CA data using openssl s_client as we don’t have a kubecfg to pull it from yet and we don't want to have to access the Kops state store S3 bucket where this information is stored.

Note this is a non breaking change and while we can use the templated kubecfg, as admin operators, we can still run kops export kubectl.

kopsctl login should be run after assuming an AWS role within Geodesic.

There will be further PRs to create specific IAM roles for KubernetesAdmin and KubernetesRead See: cloudposse/terraform-aws-components#111

Testing

Given an aws-iam-authenticator configMap of:

clusterID: us-west-2.testing.cloudposse.co
    server:
      mapRoles:
      - roleARN: arn:aws:iam::126450723953:role/KubernetesAdmin
        username: kubernetes-admin
        groups: ["system:masters"]
      - roleARN: arn:aws:iam::126450723953:role/KubernetesReadOnly
        username: kubernetes-readonly
        groups: ["system:authenticated"]

and ~/.aws/config containing:

[profile cpco-testing-admin]
region         = us-west-2
role_arn       = arn:aws:iam::126450723953:role/OrganizationAccountAccessRole
mfa_serial     = arn:aws:iam::323330167063:mfa/josh
source_profile = cpco

[profile cpco-testing-admin-kubernetes]
region         = us-west-2
role_arn       = arn:aws:iam::126450723953:role/KubernetesAdmin
mfa_serial     = arn:aws:iam::323330167063:mfa/josh
source_profile = cpco

[profile cpco-testing-readonly-kubernetes]
region         = us-west-2
role_arn       = arn:aws:iam::126450723953:role/KubernetesReadOnly
mfa_serial     = arn:aws:iam::323330167063:mfa/josh
source_profile = cpco

[profile cpco-root-admin]
region         = us-west-2
role_arn       = arn:aws:iam::323330167063:role/cpco-root-admin
mfa_serial     = arn:aws:iam::323330167063:mfa/josh
source_profile = cpco

[profile cpco]

---

login to testing.cloudposse.co geodesic

Try and kubectl get nodes:

 ✗   (none) ~ ⨠  kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the right host or port?

Build our kubecfg:

 ✗   (none) ~ ⨠  kopsctl cluster kubecfg build
kopsctl ≫ starting task cluster.kubecfg.build
Wrote configuration to /dev/shm/kubecfg...
 ⧉  testing (⎈ |us-west-2.testing.cloudposse.co:default)

Check our kubecfg:

 ✗   (none) ~ ⨠  cat /dev/shm/kubecfg
apiVersion: v1
kind: Config
preferences: {}

clusters:
- cluster:
    server: https://api.us-west-2.testing.cloudposse.co
    certificate-authority-data: <<REDACTED>>
  name: us-west-2.testing.cloudposse.co

contexts:
- context:
    cluster: us-west-2.testing.cloudposse.co
    user: us-west-2.testing.cloudposse.co
  name: us-west-2.testing.cloudposse.co

current-context: us-west-2.testing.cloudposse.co
users:
- name: us-west-2.testing.cloudposse.co
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      args:
        - token
        - -i
        - us-west-2.testing.cloudposse.co

Try and get nodes without having assumed our IAM role:

 ✗   (none) ~ ⨠  kubectl get nodes
could not get token: could not create session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::126450723953:role/OrganizationAccountAccessRole, source profile has no shared credentials
could not get token: could not create session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::126450723953:role/OrganizationAccountAccessRole, source profile has no shared credentials
could not get token: could not create session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::126450723953:role/OrganizationAccountAccessRole, source profile has no shared credentials
could not get token: could not create session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::126450723953:role/OrganizationAccountAccessRole, source profile has no shared credentials
could not get token: could not create session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::126450723953:role/OrganizationAccountAccessRole, source profile has no shared credentials
Unable to connect to the server: getting credentials: exec: exit status 1

Assume IAM role so we can successfully run some kubectl commands:

 ⧉  testing (⎈ |us-west-2.testing.cloudposse.co:default)
 ✗   (none) ~ ⨠  assume-role
Enter passphrase to unlock /conf/.awsvault/keys/:
* Assumed role arn:aws:iam::126450723953:role/OrganizationAccountAccessRole
 ✓   (cpco-testing-admin) ~ ⨠  kubectl get nodes
NAME                                           STATUS     ROLES    AGE   VERSION
ip-172-20-100-72.us-west-2.compute.internal    Ready      master   2d    v1.10.11
ip-172-20-111-167.us-west-2.compute.internal   Ready      node     25d   v1.10.11
ip-172-20-35-199.us-west-2.compute.internal    Ready      node     15d   v1.10.11
ip-172-20-41-111.us-west-2.compute.internal    Ready      master   2d    v1.10.11
ip-172-20-45-164.us-west-2.compute.internal    Ready      node     58d   v1.10.11
ip-172-20-92-206.us-west-2.compute.internal    Ready      node     3d    v1.10.11
ip-172-20-94-81.us-west-2.compute.internal     NotReady   master   2d    v1.10.11

[osterman]

Run make bash/fmt/check

[osterman]

please add links to your research in the PR description
also, if there were any other influences/examples for taking this approach.

[osterman]

Something about this should tie it to being necessary for usage with aws-iam-authenticator (like a comment) =)

[joshmyers]

The thought was that this would be the way to generate kubecfg for all usage (where folks don't want to kops export kubecfg) not necessarily just for the aws-iam-authenticator use case

[osterman]

See comments

[osterman]

🥇