Unable to set numerous Vault configuration options

[jjustinic]

Currently, the chart only supports configuring a few of the Hashicorp Vault annotations listed at https://www.vaultproject.io/docs/platform/k8s/injector/annotations on a workload.

In some enterprise environments, the default values will not work and it is necessary to customize the configuration in order to use the chart with Vault. Examples include when multiple clusters are configured at different auth-paths and using a custom tls-secret when Vault instances are only available internally and use an enterprise CA.

[jjustinic]

I've opened a PR that handles additional vault.hashicorp.com/ annotations in a generic way.

I started with this method to avoid tight coupling to the implementation details of Vault and not require additional updates if annotations are added/changed/remove. However, I am happy to enumerate out the current annotations similar to what was done for a number of the required annotations if this is more desired.

This issue is more impactful to us than other issues I've brought up, and would prefer this one has priority over any of my others.

[tsigle]

@jjustinic You bring up some good points and a valid reason to support any number of annotations for vault.hashicorp.com in a generic way.

I'm about to create another enhancement request to handle any generic annotations to be put at the top level of the k8s resource, similar to how we can put labels there. This is specific to the template annotations and related.

[jjustinic]

Thanks @tsigle. Support for generic annotations could solve this. However, support for the generic annotations at the top level would not help for this particular case. For Vault, the annotation needs to be on the pod spec, not just the top level controller. So, the current contents of the rel0.4.7 branch don't resolve this yet.

[tsigle]

@jjustinic I agree on the generic annotations, just was mentioning that, as it was another thread I was working on.

There is always a tug-a-war between making everything open-ended for the learned folks on vault -vs- hand feeding them with specific value name/values.

I'm thinking of proposing the following:

    enabled: false
    hashicorp:
      role: k8s-default
      log-level: info
      preserve-secret-case: true
      secret-volume-path: /run/secrets
      pre-populate-only: true
      serviceAccountName: vault-auth
      secretPrefix: path/to/secrets
      secrets:
      - name: secret-name
        secret: secret-name
    annotations:
      vault.hashicorp.com/tls-secret: "....."
      vault.hashicorp.com/log-level: "debug"

This would basically emit:

      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-inject-secret-secret-name.json: path/to/secrets/secret-name
        vault.hashicorp.com/agent-inject-template-secret-name.json: |
          {{ with secret "path/to/secrets/secret-name/secret-name" -}}
          {{ .Data.data | toJSONPretty }}
          {{- end }}
        vault.hashicorp.com/agent-pre-populate-only: "true"
        vault.hashicorp.com/log-level: debug
        vault.hashicorp.com/preserve-secret-case: "true"
        vault.hashicorp.com/role: k8s-default
        vault.hashicorp.com/secret-volume-path: /run/secrets
        vault.hashicorp.com/tls-secret: "....."

Where the log-level would basically override the setting above.

I think that if we allow for annotations to get set, they would just override/add settings.

[jjustinic]

I'm definitely happy with that for what we need right now @tsigle. I'm happy to update #96 to match what you have above, if that's in any way helpful to get this out.

[tsigle]

Hi @jjustinic . I think the collab on this is working out well. I see how you addressed sending these annotations in as their name appending to the default vault.hashicorp.com/. I will update the default values to the following defaults, which can be overridden by the deployers values. The older name/values will be deprecated, but still supported for some time.

############################################################
  # Fields used to annotate secret hashicorp vault information
  #
  # Theese annotations names will be automatically be
  # appended to the 'vault.hashicorp.com/'
  #
  # https://www.vaultproject.io/docs/platform/k8s/injector/annotations
  ############################################################
  vault:
    enabled: false
    hashicorp:
      annotations:
        agent-inject: true
        agent-init-first: true
        agent-pre-populate-only: true
        log-level: info
        preserve-secret-case: true
        role: k8s-default
        secret-volume-path: /run/secrets
        serviceAccountName: vault-auth
      # secretPrefix: path/to/secrets
      # secrets:
      # - name: secret-name
      #   secret: secret-name

With an injected value of:

pingfederate-admin:
  vault:
    enabled: true
    hashicorp:
      # secretPrefix: path/to/secrets
      # secrets:
      # - name: secret-name
      #   secret: secret-name
      annotations:
        log-level: debug
        tls-secret: vault-tls

resulting annotations would look like (notice overridden log-level and new tls-secret)

  annotations:
    vault.hashicorp.com/agent-init-first: "true"
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/agent-inject-status: injected
    vault.hashicorp.com/agent-pre-populate-only: "true"
    vault.hashicorp.com/log-level: debug
    vault.hashicorp.com/preserve-secret-case: "true"
    vault.hashicorp.com/role: k8s-default
    vault.hashicorp.com/secret-volume-path: /run/secrets
    vault.hashicorp.com/serviceAccountName: vault-auth
    vault.hashicorp.com/tls-secret: vault-tls