Secret generation

[knelasevero]

Discussed in #195

^{Originally posted by MatteoJoliveau June 17, 2021}
Hello community!
We're using the old kubernetes-external-secrets controller and are in the verge of migrating to this new implementation that we already like very much.

One feature that I think would be very useful to reduce manual intervention is secrets generation, that is the ability for external-secrets to generate a secret string (e.g. a password) based on some parameters, store it in the backing SecretStore if the key does not exist yet or read it if it already does. There are a lot of secrets that can be generated on the fly and configured, some examples:

• key bases used to derive secret keys for cookie encryption, token signing, etc (e.g. Ruby on Rails SECRET_KEY_BASE variable)
• self-configured user credentials. Some Kubernetes operators, like the MongoDB Community Operator require to inject a password from the outside instead of generating one themselves, which reduces the ability to automatically and declaratively manage deployment of those components if we need to generate and store the password in Vault ourselves.
• shared secrets used for symmetric encryption/signing. Although inferior compared to public key encryption, there are cases where we need to support such encyprtion, and having the secret automatically generated for us on first deployment would be awesome.

Bonus round: external-secrets could also rotate the secrets it generates following some provided configuration to increase security.

Example API implementation:

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: example-secret
spec:
  secretStoreRef:
    name: example-secret-store
    kind: SecretStore
  refreshInterval: "1h"
  target:
    name: example-secret
    creationPolicy: 'Owner'
  data:
    - secretKey: secret-key-base
      remoteRef:
        key: path/to/my/secret
        # This property does not exist in the SecretStore until first deployment of this ExternalSecret
        property: secret-key-base
        generator:
          # Generate a random string
          string:
            length: 16
            useSpecialCharacters: false

Feedbacks are most welcome on this proposal!

[joggeli34]

As this proposal involves writing to the secret stores, I could see just a little bit different use-case:

When using wildcard-certificates with let's encrypt and cert-manager, the certificate sometimes must be copied to other namespaces or clusters.

So when not only allowing to generate a new secret, but additionally to watch on an existing secret and write this to the stores whenever it changes, the secret can be used in other namespaces by just creating a "normal" external secret.

[MatteoJoliveau]

@joggeli34 while I do agree that the sync use-case is valid, I don't think external-secret is the right tool to do it. By synchronizing with the external store and then back into the cluster, we risk exposing secrets unnecessarily, because they could be accessed by anyone with read permissions on the external store key.

However, this approach could be interesting to share secrets generated by other systems (like cert-manager in your example) with non-kubernetes system that can access the same store.

I think it's a good idea but the security implications of this feature should be carefully evaluated

[moolen]

Just as a sidenote: some people use crossplane to generate/write secrets to AWS Secrets Manager and use ESO solely for the read-path.

[iAnomaly]

I see the value in this but I think it should be broken up into two separate features:

1. The ability to create dynamic Secret data using a generator (this issue)
2. The ability to eject/export K8s Secrets back to external secret providers/stores (essentially reverse of this controller's current functionality). I've created a separate feature request/spec for this here: Feature: Reverse Injection/External Sink #347

[MatteoJoliveau]

This is a very good idea @iAnomaly . I see how we could use this separation to distinguish between generating secrets that developers might want to retrieve manually from the store (e.g. an initial admin password) and secrets that are only for the application to use (e.g. a secret key to sign cookies or something).

[knelasevero]

We thought about it and multi-provider secret sync could be very awesome, with some failover capabilities, and some stuff working out of the box. We want to pursue that.

For our secret generation to throw them at external providers, we think that Crossplane (or any other tool made for that) would be better. There are some efforts on implementing that for gcp (crossplane-contrib/provider-gcp#305) and the AWS already provider supports it. Efforts there seem to make more sense.

Please let me know if you want to re-open this issue or if you want to further discuss, feel free to do so! :)