Impersonation of destination ServiceAccounts, Users, Groups

[jessesuen]

Summary

Kubernetes has the ability to impersonate ServiceAccounts, users, and groups as part of API requests:
https://kubernetes.io/docs/reference/access-authn-authz/authentication/

Argo CD should provide a way for operators to configure service accounts, users, or groups in the destination clusters, and impersonate these identities when performing mutative Argo CD operations (sync, resource actions, etc...) to this cluster. Using this mechanism, operators could use kubernetes RBAC as an alternative means to ensure control (as opposed to project-level allow/deny lists).

Motivation

Argo CD already has project-level features to allow/deny resources. While this is convenient, this mechanism does not use idiomatic Kubernetes RBAC and operators may instead prefer to use normal K8s RBAC roles (provided they set this up in the destination clusters). For operators who are willing to do the extra work of configuring destination clusters with the appropriate service accounts, users, and groups, they could improve the security of the clusters by configuring Argo CD to leverage these roles.

Using this technique, I believe Argo CD could be configured only with get/watch/list/impersonate privileges (and not create/update/patch) but then use impersonation to perform the mutations on resources.

Proposal

This will need to go through the proposal process:

https://github.com/argoproj/argo-cd/blob/master/docs/proposals/001-proposal-template.md

[jannfis]

Thanks for taking this forward, @jessesuen.

My personal opinion is, that implementing such a feature - along with other changes in how permissions are managed - would be a real benefit to users of Argo CD and help adopting Argo CD in organisations where there is a strong requirement for implementing least privilege concepts, along with a strong need for still being able to self-servicing everything. From organisations evaluating the usage of Argo CD, one major concern many of them bring up is about Argo CD being the super-user on all of the managed clusters. This often leads to either Argo CD being dropped out of the choice of available tools, having to go through long and tedious organisational exception processes or for some work-arounds, e.g. using namespace scoped Argo CD installations.

As I'm faced with such requirements constantly, I was thinking about possible ways to mitigate those (imho perfectly valid) concerns, which I'm happy to share for discussion. I think we may need (and want) to discuss and re-design some of the current multi-tenancy mechanisms along with this requirement, possibly turning it into quite a big and fundamental change, tho.

For one, I think we should extend and improve the current self-service capabilities of Argo CD, moving more and more away from using AppProject as the central governance instance within Argo CD and driving it more towards how Kubernetes (and its various distributions) conceptualizes mulit-tenancy. The main benefit of this in my opinion is, that people already know how to configure permissions in Kubernetes (or their distribution), and they also trust these mechanisms. I know this is out of scope for this proposal, but I just want to bring up for considering going forward.

Also, when we consider as much self-service as possible also in the declarative way, we should imho move as much of the control as possible to the application teams without any need for interference or interaction by an Argo CD administrative team.

That all being said (sorry for circling around), I do like the idea of having the impersonation setting on the Application level (which is typically managed by the application teams) instead of at the AppProject level (which is typically managed by an Argo CD admin). Currently, this would - as already pointed out - impose a severe security issue, by allowing application teams to possibly elevate their privileges by specifying a ServiceAccount that they should not have access to. However, there is a possible mitigation when the application controller would allow impersonation only to a ServiceAccount in the namespace where the Application resource lives in. I have recently submitted a proposal #6409 that would allow Application resources to be created in any (or a set of configured) namespace(s), so the flow would be:

• An application team is granted access to a namespace to self-manage their Applications in a declarative way (possibly using Argo CD to do it in a GitOps-way), without having to revert to Argo CD API to create these Applications (for implications, please refer to the proposal in proposal: Applications outside argocd namespace #6409 and the accompanying PoC PR feat: (PoC) Applications outside argocd namespace #6537). One namespace per team, or even one namespace per application.

• These namespaces are pre-populated with one or more ServiceAccounts that define the permissions for each application team (or application).

• Applications may only impersonate to ServiceAccounts that live in the same namespace as the application itself. Application teams can then chose the appropriate ServiceAccount to be used for syncing each application themselves, without being able to elevate their privileges.

• One thing to consider is, how a missing value for the impersonation should be handled. I could imagine either a default setting specified in the AppProject, or using the namespace's default ServiceAccount.

I'm happy to discuss any of the above further.

[sbose78]

impose a severe security issue, by allowing application teams to possibly elevate their privileges by specifying a ServiceAccount that they should not have access to.

I would plug that hole by doing a SubjectAccessReview to find out "if user John minimally has 'read' on Service Account foo"

[scalen]

As an alternative to this, could we not use k8s impersonation as ArgoCD, but still manage the permissions for it in k8s directly? This would alleviate concerns about permissions escalation, as ArgoCD would not, itself, be able to elevate anyone's privilege. (Beyond the basic superuser permissions of the ArgoCD service account, anyway 🙈)

I.e. the cluster managers can give the ArgoCD ServiceAccount permissions to impersonate specific users and groups (and service accounts, though I don't personally see the value of that). Then, if a user attempts an operation in the UI or CLI, and that user's name matches the name of a k8s user that ArgoCD is allowed to impersonate and that user is configured to impersonate a k8s user, then all k8s API calls for that operation can be made --as that user. (The same mechanism would work for ArgoCD groups and roles, and configuration to impersonate k8s groups and service accounts.)

I would argue for this not applying to ArgoCD custom resources, for which ArgoCD provides a much more fine-grained RBAC already.

This would let a user sync and prune an Application, but only directly delete pods (no other resources), for example.

[ronaknnathani]

Having this feature would be very valuable to our use case as we prefer managing RBACs within k8s. For an env with multiple clusters and multiple tenants, this already comes with an overhead and then dealing with another set of RBAC permissions that are ArgoCD specific make this quite painful.

A couple implementations that support this in other platforms:

• Another thing that many terraform providers for k8s support is passing the bearer token during the execution. If Argo can support that, then the RBACs defined in the cluster would take care of the authorization and the resources would be applied as the user who triggered an update.
• Flux v2 allows specifying a service account for the release objects and the service account is used to do write operations.

Whichever route argo tales, having this capability would be a game changer for multi-tenant environments.

[scalen]

I have added detail to my proposal for deferring to k8s RBAC here.

[crenshaw-dev]

@scalen I'll bring up this issue/your proposal in the next Security SIG meeting. Feel free to drop into that meeting if you'd be interested in discussing it with the team!

[alexef]

Hi @crenshaw-dev / @scalen ! did the meeting take place? Is there any update on the state of this proposal?

[crenshaw-dev]

@alexef the outcome was sort of "yes, this important, we should do it, but we don't have time right now."

[tossmilestone]

@jannfis , a ServiceAccount maybe not enough to specify an user, a subject like the subjects in RoleBinding will be more adequate. Our platform have been deeply bound to k8s RBAC, every user in our platform has it's particular RBAC configuration. And the user is not authorized by ServiceAccount but username, such as:

- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: alice

[scalen]

a subject like the subjects in RoleBinding will be more adequate. Our platform have been deeply bound to k8s RBAC, every user in our platform has it's particular RBAC configuration. And the user is not authorized by ServiceAccount but username

This is how k8s impersonation (using --as on the command line) works 🙂 It should be very possible (though a massive refactor) to define a mapping between ArgoCD users/groups/roles and k8s users/groups/service accounts, then use k8s impersonation to carry out any k8s commands from ArgoCD as the appropriate k8s user/group/service account.

See my previous comment for more detail: #7689 (comment)

[tossmilestone]

This is how k8s impersonation (using --as on the command line) works 🙂 It should be very possible (though a massive refactor) to define a mapping between ArgoCD users/groups/roles and k8s users/groups/service accounts, then use k8s impersonation to carry out any k8s commands from ArgoCD as the appropriate k8s user/group/service account.

👍🏼 Yes, that's exactly what we want. Hope the feature is done quickly!