Confusing, possibly erroneous RBAC warnings ("Failed to get pod") #2228
Labels
Comments
|
If this shows what I think it does, the role and binding above don't look right, yet: |
|
@kevincantu Any update on this? even i am getting the same error |
|
@maryala9, if you look closer at my RoleBinding there, the name and namespace of that service account were flipped! Instead of: That last bit should be: When I fixed that it sets the permissions I wanted: And then workflows have started work just fine! |
|
So, in full: # give our webhook ap (as default:default) permissions to create workflows
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: argo-invocation
namespace: argo
rules:
- apiGroups:
- "argoproj.io"
resources:
- "workflows"
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-default-invocation
namespace: argo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: argo-invocation
subjects:
- kind: ServiceAccount
name: default
namespace: default
# give workflows (as argo:default) permissions to run things
# see https://github.com/argoproj/argo/blob/master/docs/workflow-rbac.md
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: argo-workflow
namespace: argo
rules:
# pod get/watch is used to identify the container IDs of the current pod
# pod patch is used to annotate the step's outputs back to controller (e.g. artifact location)
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- watch
- patch
# logs get/watch are used to get the pods logs for script outputs, and for log archival
- apiGroups:
- ""
resources:
- pods/log
verbs:
- get
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: argo-default-workflow
namespace: argo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: argo-workflow
subjects:
- kind: ServiceAccount
name: default
namespace: argo |
|
Thank you that is great! |
|
Thanks, it works. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm setting up a new cluster with Argo v2.4.3 (to run some stuff we've been happily doing on v2.2.0) on EKS v1.14.9-eks-c0eccc and v1.14.8-eks-b8860f AMIs, and while trying to drill down into some problems with getting workflow pods to work I've fallen down an RBAC rabbit hole.
What I saw
For some steps of a workflow, the
maincontainer of the pod shows that step doing its job OK and logging sensibly about it, but then thewaitcontainer contains something like the following:The UI ends up displaying this MESSAGE:
And the most actionable warning seemed to be this:
I believe the code running that is here:
The fix I tried
Anyways, I attempted to fix that by adding permissions to the
argo:defaultservice account, so I began with these instructions, and created the following role and role binding:Results
I expected adding this role (to the argo namespace) and this binding would fix that warning, at least!
Actually, however, with that and more permissive attempts I'm still seeing that warning every time.
Questions
What kind of service account, roles, and binding should I define to give my workflows permission to run?
Alternatively (in case I've got that right 😅 ), is this "forbidden" warning an erroneous red herring occurring because of something like a non-existent pod?
Thanks for taking a look!!
The text was updated successfully, but these errors were encountered: