feat!: Create least privilege default service account

[abhikaddy]

Attempting to fix #1416

Fixes #1416

[apeabody]

/gcbrun

[abhikaddy]

@Jberlinsky @ericyz

[abhikaddy]

@apeabody Can I get what failed in the cloud build check?

[apeabody]

/gcbrun

[apeabody]

@apeabody Can I get what failed in the cloud build check?

re-triggered the check

[abhikaddy]

Hi Good Day!
Can I get a merge? @apeabody @ericyz @Jberlinsky

[apeabody]

Thanks for the contribution @abhikaddy!

As this permissions reduction could be a breaking change for some users, can you please add a quick update note regarding the change to docs/upgrading_to_v29.0.md, similar to those in https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/master/docs

[abhikaddy]

Hi @apeabody,

As requested, added docs/upgrading_to_v29.0.md.
Requesting a review.

Thanks.

[apeabody]

/gcbrun

[apeabody]

/gcbrun

[apeabody]

/gcbrun

[shinebayar-g]

Are we supposed to create roles/container.nodeServiceAccount by ourselves? This role doesn't exist.

[legal90]

That role exists, but it's not visible in GCP Console. It seems that it was deprecated (and hidden?).
gcloud CLI output confirms that: https://cloud.google.com/iam/docs/understanding-roles#container.nodeServiceAccount

$ gcloud iam roles describe roles/container.nodeServiceAccount
description: Least privilege role to use as the service account for GKE Nodes.
etag: AA==
includedPermissions:
- autoscaling.sites.writeMetrics
- logging.logEntries.create
- monitoring.metricDescriptors.create
- monitoring.metricDescriptors.list
- monitoring.timeSeries.create
- monitoring.timeSeries.list
- resourcemanager.projects.get
- resourcemanager.projects.list
- serviceusage.services.use
- storage.objects.get
- storage.objects.list
name: roles/container.nodeServiceAccount
stage: DEPRECATED                                   # <===
title: Kubernetes Engine Node Service Account

This role is also not mentioned here:
cc @apeabody . Should that PR be reverted?

P.s.
In my case I had to disable the SA creation (create_service_account = false) and created it outside of the module instead, as well as create bindings to roles/logging.logWriter, roles/monitoring.metricWriter`.

[apeabody]

Opened #1827

[gorge511]

There is a new role named roles/container.nodeServiceAgent which should be used instead of deprecated roles/container.nodeServiceAccount. Should we migrate to this role instead of reverting the change? Or is there some other reason for revert which I'm not aware of?

$ gcloud iam roles describe roles/container.nodeServiceAgent  
description: Minimal set of permission required by a GKE node to support standard
  capabilities such as logging and monitoring export, and image pulls.
etag: AA==
includedPermissions:
- autoscaling.sites.writeMetrics
- logging.logEntries.create
- monitoring.metricDescriptors.create
- monitoring.metricDescriptors.list
- monitoring.timeSeries.create
- monitoring.timeSeries.list
- resourcemanager.projects.get
- resourcemanager.projects.list
- serviceusage.services.use
- storage.objects.get
- storage.objects.list
name: roles/container.nodeServiceAgent
stage: GA
title: Kubernetes Engine Node Service Agent

[shinebayar-g]

There is a new role named roles/container.nodeServiceAgent which should be used

Is there any source that I can refer to? I thought service agent type of iam service accounts are created by GCP used internally only.

[legal90]

@shinebayar-g You are right. Service Agent roles should not be granted to any principal except service agents:

Warning: Do not grant service agent roles to any principals except service agents. Some service agent roles contain very powerful permissions, and the permissions within these roles can change without notice. Instead, choose a different predefined role, or create a custom role with the permissions you need.

https://cloud.google.com/iam/docs/service-agents

[gorge511]

Ok, you are both right. Agent roles should be used only for Agents, which in this case is probably just Autopilot node agent account.

I just got a message from Google representative that there is currently running effort updating this recommendation and there is new roles/container.defaultNodeServiceAccount currently in ALPHA stage which should replace the old one.

The list of permissions is shorter than in the old role, so I'm little bit scared of using it 🙂

$ gcloud iam roles describe roles/container.defaultNodeServiceAccount
description: Least privilege role to use as the default service account for GKE Nodes.
etag: AA==
includedPermissions:
- autoscaling.sites.writeMetrics
- logging.logEntries.create
- monitoring.metricDescriptors.create
- monitoring.metricDescriptors.list
- monitoring.timeSeries.create
- monitoring.timeSeries.list
- stackdriver.resourceMetadata.write
name: roles/container.defaultNodeServiceAccount
stage: ALPHA
title: Kubernetes Engine Default Node Service Account

Also I came across another docs page which is suggesting usage of 5 different roles.

So I'm confused now 😕

[gorge511]

I received confirmation from Google representative that the new role should be ok to use, so I prepared #1844.

 I've asked Engineering for clarification and recommendation about new role - it is good to go and use

[NeckBeardPrince]

Maybe I'm an outlier here, but anybody else feel a little uneasy switching to a new SA? That's currently, in ALPHA, and there is zero documentation on the new role? Perhaps, we shouldn't switch to using new roles that are still in ALPHA?

[hSATAC]

Is there any downtime for the SA recreation for existing clusters? I simply want to upgrade the google provider...

[NeckBeardPrince]

Is there any downtime for the SA recreation for existing clusters? I simply want to upgrade the google provider...

No downtime.