Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RBAC Error when running helm deploy on GKE #27

Closed
robmorgan opened this issue Feb 7, 2019 · 3 comments
Closed

RBAC Error when running helm deploy on GKE #27

robmorgan opened this issue Feb 7, 2019 · 3 comments

Comments

@robmorgan
Copy link
Member

Hi Yori,

I'm trying to deploy Helm to a GKE cluster and it looks like I'm running into some RBAC issues.

Any ideas?

It could be a GKE security feature or limitation?

Here's the full trace:

❯ ./cmd helm deploy \
    --tiller-namespace tiller-world \
    --resource-namespace dev \
    --service-account tiller \
    --tls-common-name tiller \
    --tls-org Gruntwork \
    --tls-org-unit IT \
    --tls-city Phoenix \
    --tls-state AZ \
    --tls-country US \
    --rbac-group admin \
    --client-tls-common-name admin \
    --client-tls-org Gruntwork \
    --kubectl-context-name $(kubectl config current-context)

INFO[2019-02-07T14:12:05+01:00] No kube config path provided. Using default (/Users/robbym/.kube/config)  name=kubergrunt
INFO[2019-02-07T14:12:05+01:00] Validating required resources exist.          name=kubergrunt
INFO[2019-02-07T14:12:05+01:00] Validating the Namespace tiller-world exists  name=kubergrunt
INFO[2019-02-07T14:12:05+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Found Namespace tiller-world                  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Validating the ServiceAccount tiller exists in the Namespace tiller-world  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Found ServiceAccount tiller                   name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] All required resources exist.                 name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Generating certificate key pairs              name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Using /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/671821986 as temp path for storing certificates  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Generating CA TLS certificate key pair        name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Done generating CA TLS certificate key pair   name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Generating Tiller TLS certificate key pair (used to identify server)  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Generating signed TLS certificate key pair    name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Done generating signed TLS Certificate key pair  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Successfully generated Tiller TLS certificate key pair (used to identify server)  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Done generating certificate key pairs         name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Uploading CA certificate key pair as a secret  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Successfully uploaded CA certificate key pair as a secret  name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Deploying Helm Server (Tiller)                name=kubergrunt
INFO[2019-02-07T14:12:06+01:00] Running command: helm --kube-context gke_dev-sandbox-228703_europe-west3_example-cluster --kubeconfig /Users/robbym/.kube/config init --home /Users/robbym/.helm --override spec.template.spec.containers[0].command={/tiller,--storage=secret} --tiller-tls --tiller-tls-verify --tiller-tls-cert /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/671821986/tiller_tiller-world.crt --tiller-tls-key /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/671821986/tiller_tiller-world.pem --tls-ca-cert /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/671821986/tiller_tiller-world_ca.crt --tiller-namespace tiller-world --service-account tiller --wait
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/repository
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/repository/cache
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/repository/local
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/plugins
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/starters
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/cache/archive
INFO[2019-02-07T14:12:06+01:00] Creating /Users/robbym/.helm/repository/repositories.yaml
INFO[2019-02-07T14:12:06+01:00] Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
INFO[2019-02-07T14:12:09+01:00] Adding local repo with URL: http://127.0.0.1:8879/charts
INFO[2019-02-07T14:12:09+01:00] $HELM_HOME has been configured at /Users/robbym/.helm.
INFO[2019-02-07T14:12:09+01:00]
INFO[2019-02-07T14:12:09+01:00] Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
INFO[2019-02-07T14:12:21+01:00] Happy Helming!
INFO[2019-02-07T14:12:21+01:00] Successfully deployed Tiller in namespace tiller-world with service account tiller  name=kubergrunt
INFO[2019-02-07T14:12:21+01:00] Granting access Tiller server deployed in namespace tiller-world to:  name=kubergrunt
INFO[2019-02-07T14:12:21+01:00]         - the RBAC groups [admin]                    name=kubergrunt
INFO[2019-02-07T14:12:21+01:00] Checking if Tiller is deployed in the namespace.  name=kubergrunt
INFO[2019-02-07T14:12:21+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Found a valid Tiller instance in the namespace tiller-world.  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Downloading the CA TLS certificates for Tiller deployed in namespace tiller-world.  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Using /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/050160537 as temp path for storing certificates  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Successfully downloaded CA TLS certificates for Tiller deployed in namespace tiller-world.  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Granting access to deployed Tiller in namespace tiller-world to RBAC groups  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Generating and storing certificate key pair for admin (1 of 1)  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Using /var/folders/4y/w8n6q5x525v9rkl7pt5pwwl40000gn/T/986521124 as temp path for storing client certificates  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Generating client certificates for entity admin  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Generating signed TLS certificate key pair    name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Done generating signed TLS Certificate key pair  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Successfully generated client certificates for entity admin  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Uploading client certificate key pair as secret in namespace tiller-world with name tiller-client-21232f297a57a5a743894a0e4a801fc3-certs  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Successfully uploaded client certificate key pair as a secret  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Successfully generated and stored certificate key pair for admin  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Creating and binding RBAC roles to admin      name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Loading Kubernetes Client with config /Users/robbym/.kube/config and context gke_dev-sandbox-228703_europe-west3_example-cluster  name=kubergrunt
INFO[2019-02-07T14:12:22+01:00] Creating RBAC role to grant access to Tiller in namespace tiller-world to admin  name=kubergrunt
ERRO[2019-02-07T14:12:22+01:00] Error creating RBAC role to grant access to Tiller: roles.rbac.authorization.k8s.io "admin-tiller-world-tiller-access" is forbidden: attempt togrant extra privileges: [{[get] [] [pods] [] []} {[list] [] [pods] [] []} {[get] [] [secrets] [tiller-client-21232f297a57a5a743894a0e4a801fc3-certs] []} {[create] [] [pods/portforward] [] []}] user=&{rob@gruntwork.io  [system:authenticated] map[user-assertion.cloud.google.com:[APTNk9SdKe8IhVYtn/TR9Ua8Qrnlwvkx5D1eORaixPPyFtQD9/9ggHhR0oa7C2fUpHJUPrHB30HeKcLEOS7rrLix66cYGPDr9+tMdUdK3ofoWMPAXryiFEQA9KaJsKBL8jx7D+nyrmuqzXv5DJEQwW/fhYaglXEJUMx8kpP18TcEybphCg9YexGZ4l1iqVAnn4iREHHV63jSXpvkTRpuoB9nLNJMr/UUDIk=]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]  name=kubergrunt
ERRO[2019-02-07T14:12:22+01:00] Error creating and binding RBAC roles to admin  name=kubergrunt
ERRO[2019-02-07T14:12:22+01:00] Error granting access to deployed Tiller in namespace tiller-world to RBAC groups: roles.rbac.authorization.k8s.io "admin-tiller-world-tiller-access" is forbidden: attempt to grant extra privileges: [{[get] [] [pods] [] []} {[list] [] [pods] [] []} {[get] [] [secrets] [tiller-client-21232f297a57a5a743894a0e4a801fc3-certs] []} {[create] [] [pods/portforward] [] []}] user=&{rob@gruntwork.io  [system:authenticated] map[user-assertion.cloud.google.com:[APTNk9SdKe8IhVYtn/TR9Ua8Qrnlwvkx5D1eORaixPPyFtQD9/9ggHhR0oa7C2fUpHJUPrHB30HeKcLEOS7rrLix66cYGPDr9+tMdUdK3ofoWMPAXryiFEQA9KaJsKBL8jx7D+nyrmuqzXv5DJEQwW/fhYaglXEJUMx8kpP18TcEybphCg9YexGZ4l1iqVAnn4iREHHV63jSXpvkTRpuoB9nLNJMr/UUDIk=]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]  name=kubergrunt
ERROR: roles.rbac.authorization.k8s.io "admin-tiller-world-tiller-access" is forbidden: attempt to grant extra privileges: [{[get] [] [pods] [] []} {[list] [] [pods] [] []} {[get] [] [secrets] [tiller-client-21232f297a57a5a743894a0e4a801fc3-certs] []} {[create] [] [pods/portforward] [] []}] user=&{rob@gruntwork.io  [system:authenticated] map[user-assertion.cloud.google.com:[APTNk9SdKe8IhVYtn/TR9Ua8Qrnlwvkx5D1eORaixPPyFtQD9/9ggHhR0oa7C2fUpHJUPrHB30HeKcLEOS7rrLix66cYGPDr9+tMdUdK3ofoWMPAXryiFEQA9KaJsKBL8jx7D+nyrmuqzXv5DJEQwW/fhYaglXEJUMx8kpP18TcEybphCg9YexGZ4l1iqVAnn4iREHHV63jSXpvkTRpuoB9nLNJMr/UUDIk=]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
@robmorgan
Copy link
Member Author

Might be related: cert-manager/cert-manager#256

@rileykarson
Copy link

rileykarson commented Feb 7, 2019 •
edited

This is probably the issue I highlighted in gruntwork-io/terraform-google-gke#14? Your identity needs to have at least the permissions of the roles it creates.

@yorinasub17
Copy link
Contributor

Closing as won't fix, as this is out of scope of kubergrunt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@robmorgan @yorinasub17 @rileykarson