Support using kubernetes volume mounts for configuration settings

[sboardwell]

Would there be interest in discussing using kubernetes volume mounts for configuration in addition to environment variables and flags? Please tell if this is too far fetched but...

• although envFrom: {} exists, I do not generally like to have credentials as environment variables
• using the --password-file allows pointing to a file
  • in kubernetes, this would involved mounting a secret
• if we are mounting anyway, why not have:
  • (less work) a single flag --config-file pointing to a file containing the environment variables GIT_SYNC_USERNAME=my-user á la https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-config-file/#specify-unencoded-data-when-creating-a-secret
  • (more work) point to a directory --config-dir where all the configuration values are files /secret/mount/GIT_SYNC_USERNAME containing the username, etc (kubernetes allows key mapping if necessary with https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#project-secret-keys-to-specific-file-paths)

The advantages I see are:

• the values can be kept in a single place (separation of concerns - config vs app)
• the config file could be a secret, meaning it would be "slightly" more secure - one could choose which bits are shown as env vars, and which are read
• mounts are updated automatically, potentially opening the door on-demand changes without restarting the pod
  • https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automatically
  • imagine being able to change the log level temporarily on a live instance
  • this would need some config reload logic in the app, but still an exciting prospect IMO

Originally posted by @sboardwell in #752 (comment)

[thockin]

I don't object to this but I have some thoughts (shocking, I know):

• most of the config is not "secret", so does that imply 2 different configs?
• do we need a generic "KEY=value" mechanism or is it actually a schema?
• If the config file changes, do we feel obligated to restart? Most of those configs can't be live-updated (without massive changes to the app)

[sboardwell]

Hi, sorry for the radio silence. A few things on at the moment. Will definitely get onto this and #752 in the next few days.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.