Reference Secrets from ConfigMap

[collimarco]

What would you like to be added:

I would like to have a way to reference Secrets from ConfigMaps.

For example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  EXAMPLEVAR1: foobar1
  EXAMPLEVAR2: foobar2
  EXAMPLESECRET:
    valueFrom:
      secretKeyRef:
        name: my-secrets
        key: my-secret-key

Why is this needed:

ConfigMaps are useful to describe in one place all the env variables: the you can easily reuse the same env for multiple deployments. Basically if I have multiple Deployments or Pods, I can easily include the same ConfigMap and have all the env variables... except the secret env variables :( Indeed Kubernetes doesn't allow me to reference secrets from the ConfigMap, so I have to reference the secrets separately.

I think that it would be useful to have a way to define all the env variables in one place. My proposed feature would achieve that, while also preserving security.

[collimarco]

@kubernetes/sig-apps-feature-requests

[k8s-ci-robot]

@collimarco: Reiterating the mentions to trigger a notification:
@kubernetes/sig-apps-feature-requests

In response to this:

@kubernetes/sig-apps-feature-requests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[almiche]

👋 Hi there I'm just curious if you could expand on some more reasons why this feature should be introduced. Since in order to use the secret inside a deployment for example I would then have to reference the ConfigMap which references the secret resource adding an additional layer I'd need to go through.

[collimarco]

@almiche It would be useful to have a single source of truth: if you have different deployments that share the same env, then you can use a ConfigMap and define all the variables there... but currently it cannot contain secrets, which represent a substancial portion of the env variables.
Also consider that we usually store the ConfigMap in the Git repo, while the Secrets are only defined in Kubernetes: having all the variables names listed in the ConfigMap would be useful for future reference.

[liggitt]

Best practice is to not use secret values in envvars, only as mounted files. if you want to keep all config values in a single object, you can place all the values in a secret object and reference them that way.

Referencing secrets via configmaps is a non-goal... it confuses whether things mounting or injecting the config map are mounting confidential values.

/close

[k8s-ci-robot]

@liggitt: Closing this issue.

In response to this:

Best practice is to not use secret values in envvars, only as mounted files. if you want to keep all config values in a single object, you can place all the values in a secret object and reference them that way.

Referencing secrets via configmaps is a non-goal... it confuses whether things mounting or injecting the config map are mounting confidential values.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[collimarco]

@liggitt

Best practice is to not use secret values in env vars

Indeed I am not doing that in my proposed solution. I am just referencing secrets from env variables, not including secrets in env variables. You already allow to reference secrets from env, in Deployments for example: the problem is that you don't allow to do the same inside ConfigMaps.

if you want to keep all config values in a single object, you can place all the values in a secret object and reference them that way.

1. IMHO this is not the right solution, otherwise why do you offer config and secrets as two separate things?
2. Usually I save the config inside the Git repo and I don't save the secrets: if I follow your suggestion, I cannot save the env variable names inside the Git repo... With my proposed solution we can have a list of all the required env variables inside a config and, at the same time, store the secret values in a safe way using secrets.
3. I think that with your proposed solution I would have to define the env variable / secret mapping inside each Deployment... and that is the kind of duplication that we would like to avoid using ConfigMaps.

[luispabon]

There's no reason to copy the value of the secret in the open into the config map value. Kubernetes is capable of using a pointer or reference to the secret.

Either that, or allow defining templates for environment variables from multiple sources (configMap and secrets) that can also be referenced from envFrom.

This is essential to keep service definitions concise - 10 env vars rapidly multiply when the same container image is used to run an app's deployments (eg multiple queue consumers or the like), its init containers for migrations, cronjobs, etc. Real life apps.

[0marq]

I would also be interested in such a feature. Please consider reopening it.

[yustoris]

So, how do I set common parameters using reference.
Is it boilerplate container setting such following ones ugly, isn't it ?

initContainers:
  - name: init
    env:
    - name: HUGA
      valueFrom:
      fieldRef:
        fieldPath: metadata.namespace
    - name: PIYO
      valueFrom:
        secretKeyRef:
          name: a-secret
          key: a-secret-key
containers:
  - name: container1
    env:
    - name: HUGA
      valueFrom:
      fieldRef:
        fieldPath: metadata.namespace
    - name: PIYO
      valueFrom:
        secretKeyRef:
          name: a-secret
          key: a-secret-key
  - name: container2
    env:
    - name: HUGA
      valueFrom:
      fieldRef:
        fieldPath: metadata.namespace
    - name: PIYO
      valueFrom:
        secretKeyRef:
          name: a-secret
          key: a-secret-key    

[gabriel-samfira]

Adding my 2 cents here. I have an app that uses a config. Inside that config, my app requires a database password. I have that password in a secret. I would like to use it in my config.

Without the ability to reference a secret in a config map, that gets resolved to the actual value of that secret when mounting the configmap as a file, I am obliged to store a secret directly in a config map, or use a secret to store my config and mount the secret as my apps config.

Either way, it's ugly. I can't use a config map, which would make sense, I need to use a secret to store a config, which apart from a couple of lines of config, there is nothing secret about the rest of it.

Or even worse, I need to create a custom entrypoint that generates a config on container start from a config map and a few secrets. Just feels wrong.

[fazil1987]

I think it makes lot of sense to have option that links secret from config map.

[RiadVargas]

As previously mentioned by @gabriel-samfira, there's a bunch of possible useful scenarios where having access to Secrets inside a ConfigMap would avoid having configuration file interpolation during runtime or exposing the sensitive information wide-open directly into ConfigMap.

[knewter]

Right now I think I need this to wire in a kubedb-managed postgresql instance into a helm release crd using flux helm operator, but it's possible i'll just hack around it awfully for now

[gtskaushik]

It is very impractical to not have this feature. The easy way around these are

1. Store the entire config file as secrets (Though only 1 or 2 properties are secrets)
2. Or store the secret directly in a configMap
   Both the above approaches are not best practices

Other not easy approaches

1. My application code running inside my container should be changed so that it can read the sensitive data (values in secrets) directly from env variable rather than the config file
2. Use a script inside my container as the entrypoint to take the values from env variables and replace the placeholders in config file

Please reopen this bud

[rahuldivgan]

Please re open this. This is specially very useful when dealing with opensource helm charts which allow to load custom configurations through labelled config maps. And it is common that such configurations have some kind of secrets like db credentials, smtp credentials etc.

For example:
In prometheus operator regarding alertmanager configurations, it is possible to have custom alert manager configuration which usually needs values like smtp username and smtp password in the receiver section. It would be better if these values can be referred from a secret rather than store these credentials directly.

[vany0114]

+1 reopen it please.

[heckad]

+1 reopen it.

[amit-uc]

/reopen

[k8s-ci-robot]

@amit-uc: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[avnertzur]

The request makes so much sense to me.
Please consider reopening it

[imWildCat]

+1 for reopen

[gmorse-godaddy]

+1

[megarocks]

+1

[arielvalentin]

+1 reopen

[ToReCi]

+1 reopen

[ziadmoubayed]

+1 reopen

[felinae98]

+1 reopen

[ssainz]

+1 reopen

[joodlehammond]

+1 reopen

[samirtahir91]

+1 reopen

[barbarosh]

+1 reopen

[ViliusS]

Instead of commenting +1, please vote on the issue itself by clicking Like button.

[Pitta]

The issue is from 2019 and has a mob of people commenting. Also has over 100 likes. What does it take?

[dims]

@Pitta - One person willing to do the job by writing a KEP [1]

[1] https://github.com/kubernetes/enhancements#is-my-thing-an-enhancement

[dgsardina]

Probably better to create a different resource type like ConfigMapTemplate or similar

[alexstrai]

+1 reopen

[alexstrai]

Hi, I also like to have a way to reference Secrets from ConfigMaps. I have below prometheus config map where , I am defining target with basic auth which I want to refer from my secrets

• job_name: 'test'
  metrics_path: '/actuator/prometheus'
  static_configs:
  • targets: ['test.server.de:8888']
    basic_auth:
    username: "username"
    password: "seceret"

I tried below things 1) replace variable to refer in deployment file and tried to replace this value with environment variable when pod start - job_name: 'test' metrics_path: '/actuator/prometheus' static_configs: - targets: ['test.server.de:8888'] basic_auth: username: useror(user) password: passwordor(password)

in deployment file creates environment variable env: - name: user valueFrom: secretKeyRef: name: monitoring-credentials key: user - name: password valueFrom: secretKeyRef: name: monitoring-credentials key: password

also I tried

envFrom:

• secretRef:
  name: monitoring-credentials

2) I also tried external_labels

global: scrape_interval: 15s # By default, scrape targets every 15 seconds. external_labels: username: $user

passed argument in deployment yaml

• '--enable-feature=expand-external-labels'

3) than I try to use sed to replace variables in config file it does not allow me as config files are read only now. than I create new volume with empty directory container does not start fail with (Back-off restarting failed container) error

-- 'sed /etc/prometheus/prometheus-tmp.yml -e "s/@@user@@/$(user)/g">/etc/prometheus/prometheus.yml'

Unfortunately nothing works .

Have you found a solution to this?

[ghost]

+1 reopen

[nagasaiii]

+1 re-open
It is useful to reference secrets fro configmap

My case here is: I deployed an nginx-ingres-controller which should use config map to read the location snippet and all the config of the nginx server. Inside the configMap > location snippet of the nginx config, I have a very crucial secret key.
I want to hide it. The best solution for it is to reference this key(which is in configMap) from a secret

[SuperCoolAlan]

The workarounds provided in this thread break suggested patterns for use of Secrets and ConfigMap. I don't seek to use any ENV variables, but instead to point to a Secret from a ConfigMap.

+1 to re-open... as we approach year 4 of this issue

[jessebot]

...One person willing to do the job by writing a KEP [1]
[1] https://github.com/kubernetes/enhancements#is-my-thing-an-enhancement

@dims, I was looking at the KEP docs, and wanted to know about:

Socialize an idea with a sponsoring SIG. You can send your idea to their mailing list, or add it to the agenda for one of their upcoming meetings.

Which sig would this feature fall under if we were to write a KEP? Would this be sig-apps? Do we email the mailing list first before working on the KEP template, or should we email the initial completed KEP template to the mailing list?

Update: I never had time to do this, but if someone else would like to, please be my guest and tag me when you do 💙

[dims]

@jessebot sig-apps / sig-storage possibly. KEP's are filed as PR(s) in kubernetes/enhancements repository. you can email them / go to their meetings / talk to them on their slack channels etc..

[h2nguyen]

I like the idea of this suggested feature. However, I may have overlooked another possibility already proposed here. If that's not the case, you might be able to achieve a similar concept using configMapRef and secretRef in the pod specification (I still understand you want to merge the configMapsand secrets to keep the configuration simpler). For example:

apiVersion: v1
kind: Pod
metadata:
  name: test-service-pod
spec:
  containers:
    - name: test-service-container
      image: <a-testable-image>
      envFrom:
        - configMapRef:
            name: test-service-configmap1
        - configMapRef:
            name: test-service-configmap2
        - secretRef:
            name: test-service-secret-set1
        - secretRef:
            name: test-service-secret-set2

By doing so, I think one can still keep the configuration redundancy-free and flexible.

[glerchundi]

I guess people already mentioned it but it should be somewhat easy to create a container which is run as a init-container that just does a template expansion using the configmap as template and one or multiple secrets as data input. The result will be the materialised configuration file with secrets included that will be kept in a tmpfs volume shared with the application of interest. This would seamlessly integrate with resources like SealedSecrets or VaultStaticSecrets as they automatically place the secrets in plain Secret resources. In combination with Reloader the whole thing would be an unbeatable!

Another bonus point for this approach is that it should work for any application that works with config files. Not matter it's your own application or a third-party with their own custom file format.

ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-template
  namespace: myapp
data:
  my-app.config.template: |
    db-dsn: my-sql-user:{{my-sql-password}}@127.0.0.1:1234/my-instance

Secret:

apiVersion: v1
kind: Secret
metadata:
  name: secret
  namespace: myapp
type: Opaque
stringData:
  my-sql-password: <redacted>

Deployment:

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      volumes:
      - name: config
        emptyDir:
          medium: "Memory"
      - name: config-template
        configMap:
          name: config-template
      - name: secret-volume
        secret:
          secretName: secret
      initContainers:
      - name: template-renderer
        image: template-renderer
        command:
        - "render-template"
        - "-template-file"
        - "/config-template/myapp.config.template"
        - "-data-file"
        - "/data/myapp.secrets"
        - "-output-file"
        - "/config/myapp.config"
        volumeMounts:
        - name: config
          mountPath: "/config"
        - name: config-template
          mountPath: "/config-template"
        - name: secret
          readOnly: true
          mountPath: "/data"
      containers:
      - name: myapp
        image: myapp:latest
        command:
        - "myapp"
        - "-c"
        - "/config/myapp.config"
        volumeMounts:
        - name: config
          mountPath: "/config"

I didn't tested it and I'm quite sure it's not gonna work as is, but you get the idea :)

Perhaps I will give to my tmpl-renderer a second life.

[mixolapmati]

• 1 more case to have it done https://docs.openshift.com/container-platform/4.13/security/certificates/updating-ca-bundle.html#ca-bundle-understanding_updating-ca-bundle.

[Alestrix]

• 1 more case to have it done https://docs.openshift.com/container-platform/4.13/security/certificates/updating-ca-bundle.html#ca-bundle-understanding_updating-ca-bundle.

@mixolapmati Maybe it's too late in the evening for me, but how does this relate to secret references in configmaps? (I think I got it now - even though it's not a secret in the literal sense, the public certificate is stored in a K8s Secret resource)

Regarding updating ca bundles, maybe have a look at trust-manager from the cert-manager folks.

[illthizam-healthhelper]

please re open this

[avinode-amagdy]

I'm facing this issue now with managed prometheus on Azure, I cannot configure it to read the service catalog of consul without adding the token in ama-metrics-prometheus-config configmap .. I don't have any control on the deployment sepc to add secret reference because it's deployed and managed by Azure Monitor Workspace.

this feature would be very helpful in such cases.

[RichardDally]

There is no real alternative / solution to this real world issue. Could you reopen and consider this issue ?

[haiwu]

Would be great to have a solution for this one!

[jessebot]

For those still commenting that they want this feature

I believe you need to write a KEP for it to be considered as an official feature request:
https://github.com/kubernetes/enhancements#is-my-thing-an-enhancement

I don't have time to write one right now, but if someone else would like to, please then link it to this issue.

If no one can write the KEP, then I don't think it will be considered, as per #79224 (comment).

I don't believe commenting that you want the feature or that you want this reopened has any effect. You need to write the KEP.

[wtr-kushal]

+1 to reopen

This will provides a unified solution for retrieving SSM parameters into a Kubernetes cluster.