Kubernetes: Problems with RBAC & multiple Namespaces

[ghost]

Do you want to request a feature or report a bug?

bug

What did you do?

I tried to use Traefik as an Ingress Controller in an RBAC restricted Kubernetes Cluster with multiple namespaces.

What did you expect to see?

I expected that it would be enough if Traefik has the rights to access Ingresses from the respective namespaces.

What did you see instead?

RBAC permission denied errors.

Output of traefik version: (What version of Traefik are you using?)

traefik:1.3.0-alpine

I configured Traefik to watch multiple Namespaces:

[kubernetes]
namespaces = ["default", "`test"]

Traefik is deployed with a ServiceAccount which has the respective rights, but only for the default and test namespace (not cluster-wide). Traefik is implemented to retrieve Ingresses, Services, Endpoints and Secrets cluster-wide and than only filters the results. This means that Traefik requires cluster-wide access to Ingresses, Services, Endpoints and Secrets (https://github.com/containous/traefik/blob/master/provider/kubernetes/client.go).

It would be nice if Traefik would only requires the rights that are absolutely necessary.

[timoreimann]

/cc @dtomcej @errm

[luxas]

Very much agree here, PTAL at this thread: kubernetes/ingress-nginx#816
Traefik should preferably write its own RBAC Roles dynamically depending on what it needs to have access to

[timoreimann]

I was also bitten by the problem very recently when we enabled RBAC in our cluster.

Will likely tackle this soonish.

[timoreimann]

@luxas IIUC, you're suggesting to let Traefik update its RBAC role during runtime by the concrete secret resource names it needs. Wouldn't that require privileges to extend roles arbitrarily with regards to the affected resources?

I'm wondering if that could be considered as an undesirable form of privilege escalation. WDYT?

[luxas]

@timoreimann Agree, I thought about that as well without proposing something concrete to fix it.
The other (better way) would be to just build a Traefik RBAC controller that would adjust these rules.
Then it wouldn't necessarily be that bad if someone unauthorized got to run arbitrary commands from the Traefik container, it just wouldn't have unnecessary privs.

This RBAC controller can probably be done in a more generic Ingress-y way, let's talk to the kubernetes/ingress folks as well to see what they think.

cc @aledbf @ericchiang @liggitt @nicksardo

[kachkaev]

I'm wondering if this list of the namespaces can be made dynamic in some form, so that it was easy to use traefik with CI/CD and without frequent restarts. In this scenario the workload can be deployed into tens of namespaces, the list of which is not known in advance. E.g. the names of namespaces will be something like app-preview-[repo-name]-[branch-name].

[timoreimann]

@kachkaev that definitely seems feasible, though with a bit of effort and design considerations.

Either way, it seems to warrant its own separate issue. Feel free to open one.

[liggitt]

with fuzzy matching of namespaces, you run into cardinality issues. every request must be evaluated against all relevant RBAC bindings. namespaced bindings are only evaluated for requests to that namespace. a fuzzy binding like you describe would have to be evaluated against a request to any namespace, to see if the namespace matched the pattern.

[timoreimann]

It might be more easily feasible to do the filtering on Traefik's end: Have it watch over all namespaces additions/removals, apply the configured namespace pattern, and add/remove informers accordingly.