add Vault path-prefixes config

[davidkuster]

Changes proposed by this PR

Closing #8712 in favor of this approach. Copying the bulk of the description here because the intent is the same.

Support for Vault kv2 was added in #6115. That supports both kv1 and kv2, but as mentioned in the docs, only a single prefix path is supported:

Concourse is currently limited to looking under a single path, meaning enabling only one secrets engine is supported: kv, or kv_v2.

This PR enables multiple prefix paths and then does the lookup of the secrets in each, for each path template configured.

The idea in our org is to enable a smooth transition of teams from kv1 -> kv2. Without this we'd need to do a big bang release of ensuring all teams have migrated their secrets to the new engine, then update the Concourse config to point at kv2. Instead with this change we can support both, and let someone else herd the cats. 🙂

Note this would potentially double the number of API calls to Vault. I don't see that's a reason to not do it, but something to call out in the docs at a minimum.

• support multiple Vault prefix paths
• limited manual testing
• unit or other tests
• update docs

Notes to reviewer

Believe the approach in #8712 isn't viable due to var_sources. Instead trying the creation of a separate --path-prefixes config option. I tested locally with the following config:

      CONCOURSE_VAULT_PATH_PREFIX: ""
      CONCOURSE_VAULT_PATH_PREFIXES: "/kv2,/kv1"

Quick example of local testing:

jobs:
- name: hiya
  plan:
  - task: simple-task
    config:
      platform: linux
      image_resource:
        type: registry-image
        source: { repository: busybox }
      run:
        path: echo
        args: ["Hello world!", ((kv2-test.hello)), ((kv1-test.hello))]

Where kv2-test secret only exists in the kv2 path, and kv1-test secret only exists in the kv1 path. This results in Hello world! Hello from kv2! Hello from kv1!

Release Note

• Add support for multiple Vault path prefixes. This allows for simultaneous support of multiple secret engines, allowing teams to transition between Vault locations at their own pace.

[xtremerui]

Why it is not allow for Vault to set both, compared to Credhub?