Cannot install rbac-manager using Helm and RBACDefinition with Terraform

[markandersontrocme]

Is your feature request related to a problem? Please describe.
Do to Terraform not handling CRDs and CRs very well yet:

• kubernetes_manifest: Error: Failed to determine GroupVersionResource for manifest hashicorp/terraform-provider-kubernetes#1583
• Cannot apply CRD and a CR using it in the same plan/apply due to SSA hashicorp/terraform-provider-kubernetes#1367

when you try to install the helm release with Terraform for rbac-manager and also apply a CR RBACDefinition you get:

│ Error: Failed to determine GroupVersionResource for manifest
│
│   with module.addons.module.rbac_manager.module.helm_chart.kubernetes_manifest.this,
│   on addons/rbac_manager/helm_chart/main.tf line 15, in resource "kubernetes_manifest" "this":
│   15: resource "kubernetes_manifest" "this" {
│
│ no matches for kind "RBACDefinition" in group "rbacmanager.reactiveops.io"

using this Terraform code:

resource "kubernetes_namespace" "this" {
  metadata {
    name = local.namespace
  }
}

resource "helm_release" "this" {
  name       = local.name
  repository = local.chart
  chart      = local.name
  version    = local.chart_version
  namespace  = kubernetes_namespace.this.metadata.0.name
}

resource "kubernetes_manifest" "this" {
  manifest = yamldecode(templatefile("${path.module}/rbac.template.yaml", {
    NAMESPACE = kubernetes_namespace.this.metadata.0.name
  }))

  depends_on = [
    helm_release.this
  ]
}

Describe the solution you'd like
It would be great if we could define a full RBACDefinition in the values.yaml of the helm chart for rbac-manager.

Describe alternatives you've considered
An alternative is to deploy my own helm chart with the RBACDefinition and install that using Terraform, however it is quit a bit of overhead and not super clean IMO.

Additional context
Add any other context or screenshots about the feature request here.

[sudermanjr]

I believe we have decided against specifying rbacDefinitions in the chart, due to various reasons. The precedent exists in a some other repos that we only manage the controller, not how the end-user wants to manage the Custom resources of that controller. That, and historical poor management of custom resources by helm (which may or not be better now)

On a more personal note, I have had terrible experiences with the helm terraform provider, and am not surprised it has issues with this sort of thing.

It seems there is a valid workflow that involves the kubectl provider and static manifest in this comment Is that something you could try instead? That's much cleaner than maintaining a chart just for an rbacdefintion. Out of curiosity, if you're using cert-manager, how do you manage clusterIssuers and Issuers? We typically just use a static manifest for those sorts of things, and it's much nicer than trying to shoehorn them into helm.

[markandersontrocme]

Ah yeah the kubectl provider seems to work perfectly. I was using the official k8s one with the kubernetes_manifest resource and it was failing but the kubectl provider seems 👌

Thanks!

[sudermanjr]

Awesome! Thanks for updating!