Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ClusterRoleBinding #73

Merged
merged 42 commits into from Oct 17, 2018

Conversation

micahhausler
Copy link
Contributor

Re-create of #1, Fixes hashicorp/terraform#15194

@radeksimko I'll rebase once 1.8 is in /vendor

@munnerz
Copy link

munnerz commented Feb 16, 2018

Any update on this? This is a missing piece for a number of projects of mine!

I'd be happy to take on the rebasing work etc.

@marksost
Copy link

Hey @micahhausler and @radeksimko! I'm in a similar position as @munnerz above. Anything I can do to help?

@Mistobaan
Copy link

need this as well on gke. How are you guys solving in the mean time, @munnerz @marksost ?

@marksost
Copy link

@Mistobaan nothing great to report currently. Been playing around with the Kubernetes Ansible module to apply a declaration we wrote, but it's far from bulletproof and means we need multiple tools and then glue code to tie them together in the right order.

This feature would be super helpful.

@jhoblitt
Copy link
Contributor

jhoblitt commented Mar 13, 2018

It looks like this resource is a blocker to being able to setup helm / use the tf helm provider with any k8s cluster that has RBAC enabled, which includes gke from at least 1.7.x-gke on.

@jhoblitt
Copy link
Contributor

Is the travis setup broken?

@marksost
Copy link

@jhoblitt I believe it's throwing an error from a non-vendored package.

kubernetes/resource_kubernetes_cluster_role_binding.go:11:2: cannot find package "k8s.io/kubernetes/pkg/apis/rbac/v1" in any of:
  /home/travis/gopath/src/github.com/terraform-providers/terraform-provider-kubernetes/vendor/k8s.io/kubernetes/pkg/apis/rbac/v1 (vendor tree)
  /home/travis/.gimme/versions/go1.9.linux.amd64/src/k8s.io/kubernetes/pkg/apis/rbac/v1 (from $GOROOT)
  /home/travis/gopath/src/k8s.io/kubernetes/pkg/apis/rbac/v1 (from $GOPATH)
kubernetes/resource_kubernetes_cluster_role_binding.go:12:2: cannot find package "k8s.io/kubernetes/pkg/client/clientset_generated/clientset" in any of:
  /home/travis/gopath/src/github.com/terraform-providers/terraform-provider-kubernetes/vendor/k8s.io/kubernetes/pkg/client/clientset_generated/clientset (vendor tree)
  /home/travis/.gimme/versions/go1.9.linux.amd64/src/k8s.io/kubernetes/pkg/client/clientset_generated/clientset (from $GOROOT)
  /home/travis/gopath/src/k8s.io/kubernetes/pkg/client/clientset_generated/clientset (from $GOPATH)

Checking the tree, that package doesn't appear to be present: https://github.com/micahhausler/terraform-provider-kubernetes/tree/f-k8s-clusterrolebinding/vendor/k8s.io

@jhoblitt
Copy link
Contributor

Good catch. It looks like #117 will pull that in.

@micahhausler
Copy link
Contributor Author

Yea the Travis error is a package problem. I didn't want to bloat this PR with upgrading the whole k8s client.

@marksost
Copy link

Hey @micahhausler and @radeksimko, wanted to see if support for this might be coming any time soon? Any workaround you'd suggest in the meantime?

@obeyler
Copy link

obeyler commented May 16, 2018

@micahhausler and @radeksimko, Do you think that this functionnality will arrive soon ? RBAC is need to be able to deploy Kubernetes dashboard for example.
As @marksost asked before, is there any workaround or help we can do ?

@brandonjbjelland
Copy link

brandonjbjelland commented Jun 14, 2018

Working with AWS EKS through Terraform and finding the kubernetes provider is currently unusable in this set up (k8s cluster is 1.10) as the user grabbed from the generated kubeconfig has no permissions. I gather it's due to recent k8s RBAC changes which renders the user system:anonymous unable to do anything.

<resource_type> is forbidden: User "system:anonymous"...

I would guess there's a separate issue (or user error) of being able to authenticate using the provided user specified in the kubeconfig and leveraging heptio-authenticator but also, the first steps to creating a useful demo requires creating an admin user that can access the dashboard which requires calls to both ClusterRoleBinding and ServiceAccount.

@costimuraru
Copy link

Hey guys! What's the status on adding support for ClusterRoleBinding in the terraform k8s provider?

@whyman
Copy link

whyman commented Aug 3, 2018

Another month has passed!

Clearly there is a lot of interest in getting this merged - can we at least have the steps required to get this included listed out so we can start working through them?

@micahhausler
Copy link
Contributor Author

The only thing blocking this is an updated K8s client library.

#117 - 1.9
#162 - 1.10

Once one of those are merged, I can update the import paths to make this work

@adamdecaf
Copy link

kubernetes/resource_kubernetes_cluster_role_binding.go:11:2: cannot find package "k8s.io/kubernetes/pkg/apis/rbac/v1" in any of:

It looks like the dependencies need to be updated. Could someone with access to the PR bump those?

@Starefossen
Copy link

Starefossen commented Oct 3, 2018

Yes, we need ClusterRole and ClusterRoleBinding in order to set up dynamic Persistent Volume Claims using Azure File 😄

@alexsomesan
Copy link
Member

alexsomesan commented Oct 4, 2018

@micahhausler Many apologies for the amount of time this PR has been sitting still. I will be taking care of the K8S provider moving forward. I would like to help you get this merged.

The PR to update K8S client packages to 1.10 has been merge, so from that POV you're unblocked.
I've had a quick go at rebasing this branch to current master is the following changes were needed to get it to build.

diff --git a/kubernetes/resource_kubernetes_cluster_role_binding.go b/kubernetes/resource_kubernetes_cluster_role_binding.go
index c43c181a6..f7d915784 100644
--- a/kubernetes/resource_kubernetes_cluster_role_binding.go
+++ b/kubernetes/resource_kubernetes_cluster_role_binding.go
@@ -5,11 +5,11 @@ import (
        "log"

        "github.com/hashicorp/terraform/helper/schema"
+       api "k8s.io/api/rbac/v1"
        "k8s.io/apimachinery/pkg/api/errors"
        meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
        pkgApi "k8s.io/apimachinery/pkg/types"
-       api "k8s.io/kubernetes/pkg/apis/rbac/v1"
-       kubernetes "k8s.io/kubernetes/pkg/client/clientset_generated/clientset"
+       kubernetes "k8s.io/client-go/kubernetes"
 )

 func resourceKubernetesClusterRoleBinding() *schema.Resource {
diff --git a/kubernetes/resource_kubernetes_cluster_role_binding_test.go b/kubernetes/resource_kubernetes_cluster_role_binding_test.go
index 401573072..16467eff6 100644
--- a/kubernetes/resource_kubernetes_cluster_role_binding_test.go
+++ b/kubernetes/resource_kubernetes_cluster_role_binding_test.go
@@ -7,9 +7,9 @@ import (
        "github.com/hashicorp/terraform/helper/acctest"
        "github.com/hashicorp/terraform/helper/resource"
        "github.com/hashicorp/terraform/terraform"
+       api "k8s.io/api/rbac/v1"
        meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-       api "k8s.io/kubernetes/pkg/apis/rbac/v1"
-       kubernetes "k8s.io/kubernetes/pkg/client/clientset_generated/clientset"
+       kubernetes "k8s.io/client-go/kubernetes"
 )

 func TestAccKubernetesClusterRoleBinding(t *testing.T) {
diff --git a/kubernetes/structures_rbac.go b/kubernetes/structures_rbac.go
index fabb29e59..dd7f54e54 100644
--- a/kubernetes/structures_rbac.go
+++ b/kubernetes/structures_rbac.go
@@ -4,7 +4,7 @@ import (
        "strconv"

        "github.com/hashicorp/terraform/helper/schema"
-       api "k8s.io/kubernetes/pkg/apis/rbac/v1"
+       api "k8s.io/api/rbac/v1"
 )

 func expandRBACRoleRef(in interface{}) api.RoleRef {

@alexsomesan
Copy link
Member

Also please have a look at the acceptance tests specific to this new resouce. On a quick run on my machine they fail.
Here's the output:

TESTARGS="-run ^TestAccKubernetesClusterRoleBinding.*$" make testacc                      alex@alexs-macbook
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test $(go list ./... |grep -v 'vendor') -v -run ^TestAccKubernetesClusterRoleBinding.* -timeout 120m
?   	github.com/terraform-providers/terraform-provider-kubernetes	[no test files]
=== RUN   TestAccKubernetesClusterRoleBinding
--- FAIL: TestAccKubernetesClusterRoleBinding (0.08s)
	testing.go:518: Step 0 error: Check failed: 3 error(s) occurred:

		* Check 12/14 error: kubernetes_cluster_role_binding.test: Attribute 'subject.0.api_group' not found
		* Check 13/14 error: kubernetes_cluster_role_binding.test: Attribute 'subject.0.name' not found
		* Check 14/14 error: kubernetes_cluster_role_binding.test: Attribute 'subject.0.kind' not found
=== RUN   TestAccKubernetesClusterRoleBinding_importBasic
--- FAIL: TestAccKubernetesClusterRoleBinding_importBasic (0.07s)
	testing.go:518: Step 0 error: After applying this step, the plan was not empty:

		DIFF:

		UPDATE: kubernetes_cluster_role_binding.test
		  subject.0.api_group: "" => "rbac.authorization.k8s.io"
		  subject.0.kind:      "" => "User"
		  subject.0.name:      "" => "notauser"
		  subject.0.namespace: "" => "default"

		STATE:

		kubernetes_cluster_role_binding.test:
		  ID = tf-acc-test-c0z0mr2ci3
		  provider = provider.kubernetes
		  metadata.# = 1
		  metadata.0.annotations.% = 0
		  metadata.0.generation = 0
		  metadata.0.labels.% = 0
		  metadata.0.name = tf-acc-test-c0z0mr2ci3
		  metadata.0.resource_version = 31130
		  metadata.0.self_link = /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/tf-acc-test-c0z0mr2ci3
		  metadata.0.uid = 20af46f4-c830-11e8-ac4d-feb8ddfbfc24
		  role_ref.% = 3
		  role_ref.api_group = rbac.authorization.k8s.io
		  role_ref.kind = ClusterRole
		  role_ref.name = cluster-admin
		  subject.# = 1
FAIL
FAIL	github.com/terraform-providers/terraform-provider-kubernetes/kubernetes	0.760s
make: *** [testacc] Error 1

@ghost ghost added the size/XXL label Oct 17, 2018
@alexsomesan
Copy link
Member

LGTM
Acceptance tests are passing.

TESTARGS="-run ^TestAccKubernetesClusterRoleBinding.*" make testacc                                                                                                                                   alex@alexs-macbook
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test $(go list ./... |grep -v 'vendor') -v -run ^TestAccKubernetesClusterRoleBinding.* -timeout 120m
?   	github.com/terraform-providers/terraform-provider-kubernetes	[no test files]
=== RUN   TestAccKubernetesClusterRoleBinding
--- PASS: TestAccKubernetesClusterRoleBinding (0.15s)
=== RUN   TestAccKubernetesClusterRoleBinding_importBasic
--- PASS: TestAccKubernetesClusterRoleBinding_importBasic (0.08s)
PASS
ok  	github.com/terraform-providers/terraform-provider-kubernetes/kubernetes	0.847s

@alexsomesan alexsomesan merged commit f4f3726 into hashicorp:master Oct 17, 2018
@jhoblitt
Copy link
Contributor

Woot! Thanks folks!

@marksost
Copy link

Yes, thanks everyone for the work to push this through!

@jmjava
Copy link

jmjava commented Oct 17, 2018

i cloned master and tried using this:

resource "kubernetes_cluster_role_binding" "helm" {
metadata {
name = "helm"
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "ClusterRole"
name = "cluster-admin"
}
subject {
kind = "ServiceAccount"
name = "helm"
api_group = "rbac.authorization.k8s.io"
namespace = "kube-system"
}
}

I am getting this:

Error: kubernetes_cluster_role_binding.helm: Provider doesn't support resource: kubernetes_cluster_role_binding

@alexsomesan
Copy link
Member

@jmjava Did you install the provider after build? (Usually under ~/.terraform.d/plugins/)

Sent with GitHawk

@jmjava
Copy link

jmjava commented Oct 17, 2018

@jmjava Did you install the provider after build? (Usually under ~/.terraform.d/plugins/)

Sent with GitHawk

Thanks... YES --that was the issue. The version downloaded automatically from the web was being used. I did something similar to fix it. Will try your way next time :)

Thanks for this work.

-john

@cdaguerre
Copy link

cdaguerre commented Oct 18, 2018

Hi, I built the provider and moved it to ~/.terraform/plugins and get the following error when trying to create a ClusterRoleBinding:

module.platform-cluster.kubernetes_cluster_role_binding.tiller: Creating...
  metadata.#:                  "" => "1"
  metadata.0.generation:       "" => "<computed>"
  metadata.0.name:             "" => "tiller-clusterrolebinding"
  metadata.0.resource_version: "" => "<computed>"
  metadata.0.self_link:        "" => "<computed>"
  metadata.0.uid:              "" => "<computed>"
  role_ref.%:                  "" => "3"
  role_ref.api_group:          "" => "rbac.authorization.k8s.io"
  role_ref.kind:               "" => "ClusterRole"
  role_ref.name:               "" => "cluster-admin"
  subject.#:                   "" => "1"
  subject.0.api_group:         "" => "rbac.authorization.k8s.io"
  subject.0.kind:              "" => "ServiceAccount"
  subject.0.name:              "" => "tiller"
  subject.0.namespace:         "" => "kube-system"

Error: Error applying plan:

1 error(s) occurred:

* module.platform-cluster.kubernetes_cluster_role_binding.tiller: 1 error(s) occurred:

* kubernetes_cluster_role_binding.tiller: ClusterRoleBinding.rbac.authorization.k8s.io "tiller-clusterrolebinding" is invalid: subjects[0].apiGroup: Unsupported value: "rbac.authorization.k8s.io": supported values: ""

Recource declaration is:

resource "kubernetes_cluster_role_binding" "tiller" {
  metadata {
    name = "tiller-clusterrolebinding"
  }

  role_ref {
    kind = "ClusterRole"
    name = "cluster-admin"
    api_group = "rbac.authorization.k8s.io"
  }

  subject {
    kind = "ServiceAccount"
    name = "tiller"
    api_group = "rbac.authorization.k8s.io"
    namespace = "kube-system"
  }
}

Any idea?

@olib963
Copy link
Contributor

olib963 commented Oct 18, 2018

@cdaguerre I believe the api_group for the subject must be an empty string but must be "rbac.authorization.k8s.io" for role_ref

@alexsomesan
Copy link
Member

@cdaguerre ServiceAccount are actually a core ApiGroup resource, not an RBAC one, so they need api_group set to "" explicitly when used as subjects.

This is because this resource right now defaults to api_group = "rbac.authorization.k8s.io" when none is actually specified. I'm on the fence whether this is a viable approach, but I want to collect more user feedback before making a decision about changing that.

@ntrp
Copy link

ntrp commented Nov 12, 2018

I think a better approach would be to set default empty and allow users to configure it for the role_ref. Like it's now it is confusing in my opinion..

On top of that here is written that it applies only to User and Group but it's not written that for ServiceAccount it must be empty.

@RobMaskell
Copy link

As part of this PR was a resource added for "kubernetes_cluster_role"?
I'm trying to do the equiv of this

kubectl create clusterrole nm-secret-writer --verb=get,list,create,update --resource=secrets

and I can't see it in the docs

@mogopz
Copy link

mogopz commented Nov 27, 2018

I would like to know this too, it looks like the Kubernetes provider can do most of the things I need, but it's missing a few core features like roles and daemonsets. Is there a rough timeline for how long it takes before they are added in?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Kubernetes Cluster Role Binding Provider