Does ValuesFrom support secret in gzip format

[zerunhu]

Sometimes because the data of the secret is too large, it will use gzip for compression. Does the ValuesFrom of helm-controller support obtaining the data of the gzip type? For example, you can judge this Annotations: encoding: gzip . If it is yes, then you can The data is decompressed, and then the value is obtained according to its key.

My case:
I create my infrastructure through terrafrom, and set its backend to k8s, then the state of its backend is stored through secret, because the data of the state may be too large, so it is compressed by a gzip, and I create it through terraform The load-balance needs to get the load-balance-id to fill in helm's service.yaml, so I assign values through ValuesFrom, so I need to get this value from a gzip secret.

If this feature is not available, I'd be happy to submit a PR

this is secret:

This is how I get his data through kubctl:

This is a normal secret:

[hiddeco]

This is not something we support.

I also do not think that we are open to supporting it as proposed, as the encoding: gzip annotation is a Terraform specific annotation and not conform Kubernetes standards, for which a custom Secret Type would have been more fitting.

This is for example done by Helm, which also stores releases in a compressed format:

Name:         sh.helm.release.v1.podinfo-1648201816.v1
Namespace:    source-system
Labels:       modifiedAt=1648201817
              name=podinfo-1648201816
              owner=helm
              status=deployed
              version=1
Annotations:  <none>

Type:  helm.sh/release.v1

Data
====
release:  28176 bytes

[zerunhu]

@hiddeco
thanks a lot for your answer
If this is not a good choice, can I ask a question? I saw that your weave blog mentioned that the infrastructure in gitops is implemented by terraform. I use flux's CRD(such as helm controller) in the application layer. I am very confused about how we connect the facilities created by terraform with the components of flux, such as I gave an example of the transfer of clb-id, how to use the components of the infrastructure I created in terraform in helm-controller, because it seems that the components in flux are all at the application level.
I am very confused about this question, thank you very much for your answer

[chanwit]

@huzerun0306 a maintainer of the Terraform controller here.

Here's my assumption of your setup. Please correct me if I'm wrong.

• You have your infra provisioned by Terraform.
• You're already using Kubernetes backend for it, so that's secret is your TFSTATE managed by Terrform

What would you like is to pick out something from TFSTATE file to feed into a Helm Release. Is that right?
If so, it seems to be a good Terraform/Helm use case.

I'll check if we could have a simple example / tutorial to connect them for you. Please ping me again next week.
Is that OK for you?

[zerunhu]

@chanwit Yes, my case is what you said, we created an infrastructure through terraform-controller, and helm-controller references the information of this infrastructure from its state file. We are looking for a suitable gitops solution to do infrastructure and application level correlation. thank you very much

[chanwit]

@huzerun0306
Glad to know you're already using TF-controller.

Normally, we would not touch the TFSTATE secret directly, but use .spec.writeOutputsToSecret to selectively pick output(s) from TFSTATE to a separate secret, which could be consumed directly by a Helm Release. For example,

---
apiVersion: infra.contrib.fluxcd.io/v1alpha1
kind: Terraform
metadata:
  name: tfc-helloworld
  namespace: flux-system
spec:
  interval: 30m
  path: ./_artifacts/30-zz-terraform
  approvePlan: auto
  sourceRef:
    kind: GitRepository
    name: flux-system
    namespace: flux-system
  # here's the section you would need
  writeOutputsToSecret:
    name: tf-alb-address-secret # your secret name to write outputs to
    outputs:
    - my_alb_address            # choose to write only the Terraform output named "my_alb_address"

After this object got reconciled, you could let your HelmRelease object consume the written secret, in this example tf-alb-address-secret.

[zerunhu]

@chanwit The valuesFrom field of helm-controller only supports secrets under the same namespace, and the writeOutputsToSecret of tf-controller does not support the field of namespace, so it can only follow tf-controller in one namespace, When I use helm-controller to deploy applications with multiple namespaces, Their secrets cannot be managed reasonably. I think the writeOutputsToSecret of tf-controller may support the namespace field?

helm-controller https://github.com/fluxcd/helm-controller/blob/main/api/v2beta1/reference_types.go#L53

[stefanprodan]

@huzerun0306 you may want to consider Kyverno and create a policy for replicating the secret create by tf-controller in other namespaces, see https://kyverno.io/policies/other/sync_secrets/

If tf-controller breaks the namespace boundary that it can't be used on multi-tenant clusters, and I think @chanwit doesn't want that.

[chanwit]

Like @stefanprodan suggested, please use other tools to sync secrets across the namespace boundary.

Right, for multi-tenancy & security reasons - TF-controller will not support writing secrets into the other namespaces.