New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurations with Kubernetes RBAC authorization enforced aren't working #31

Closed
briansmith opened this Issue Dec 12, 2017 · 6 comments

Comments

Projects
None yet
5 participants
@briansmith
Copy link
Contributor

briansmith commented Dec 12, 2017

For 0.1.0 we punted on RBAC support because minikube doesn't yet enable the RBAC authorization mode and we didn't want to ask people to enable RBAC to try out Conduit. However, now we have people trying to use Conduit with RBAC authorization enabled and it's not working. At a minimum we need to make sure that Conduit works regardless of whether RBAC authorization is enforced or not. In particular, some of the controller services need to be running under service accounts that have roles that enable them to make the Kubernetes API server queries for the specific data sets they access.

Ideally we'd do this 100% automatically and it would work regardless of whether RBAC enforcement is enabled. If we can't then we can create a switch for conduit install to disable RBAC. (Because of the "Secure by Default" design goal, the default should be to enable RBAC.)

@briansmith briansmith added the bug label Dec 12, 2017

@briansmith

This comment has been minimized.

Copy link
Contributor Author

briansmith commented Dec 12, 2017

@briansmith

This comment has been minimized.

Copy link
Contributor Author

briansmith commented Dec 12, 2017

Here's one of the multiple user requests for this: https://discourse.linkerd.io/t/demo-support-for-rbac/443

@wmorgan

This comment has been minimized.

Copy link
Member

wmorgan commented Dec 12, 2017

Log messages from the discourse thread:

If I shell into the conduit-proxy container, I’m not able to actually connect to the controller:

curl hxxp://proxy-api.conduit.svc.cluster.local:8086/conduit.proxy.telemetry.Telemetry/Report
curl: (56) Recv failure: Connection reset by peer

Also I see a lot of this in the logs for the destination container in the controller:

E1211 23:45:01.296991 1 reflector.go:199] github.com/runconduit/conduit/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: the server does not allow access to the requested resource (get services)
E1211 23:45:02.302709 1 reflector.go:199] github.com/runconduit/conduit/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: the server does not allow access to the requested resource (get Endpoints)
@rootsongjc

This comment has been minimized.

Copy link

rootsongjc commented Dec 13, 2017

I see these logs on controller's destination container.

E1213 06:07:54.890236       1 reflector.go:199] github.com/runconduit/conduit/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:conduit:default" cannot list endpoints at the cluster scope

The default ServiceAccount has no enough permission to communicate with API server, we need to create a ServiceAccount.

@FaKod

This comment has been minimized.

Copy link
Contributor

FaKod commented Dec 13, 2017

If someone needs RBAC, this may be a good starting point: https://github.com/container-bootcamp/infrastruktur/tree/conduit/conduit

Unfortunately, I cannot test any further, because there are other issues I ran into...
Just look at the logs and add resources resp. apiGroups to the ClusterRole resource and apply the rbac.yaml again

@rootsongjc

This comment has been minimized.

Copy link

rootsongjc commented Dec 13, 2017

@FaKod There need more API permissions for the Deployments and attach ServiceAccount for them. I changed your yaml, and use this yaml to deploy.
conduit-dashboard

@adleong adleong referenced this issue Dec 13, 2017

Merged

Add RBAC support #40

@adleong adleong closed this in #40 Dec 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment