New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(RF)RFCs: next iteration of credential manager support #5

vito opened this Issue Jun 8, 2018 · 2 comments


None yet
3 participants

vito commented Jun 8, 2018

I'm opening this issue as a call to action for anyone familiar with any of the existing credential managers:

  • Vault
  • CredHub
  • AWS SecretsManager
  • Kubernetes

I think it makes sense to discuss each of these as separate RFCs as I don't expect anyone is intimately familiar with all of them. If you're willing to throw your hat in the ring, go ahead and submit a PR with your own RFC! If there's only one specific aspect you feel that you're knowledgeable enough to propose, go ahead and start there. We can work together towards a consistent solution over time. We just need to get the ball rolling first.


We've already received a fair amount of feedback for some credential managers. Mostly Vault, as that's the first one we shipped and seems to have had the most traction (or at least the most opinionated people giving feedback!)

Using arbitrary paths

CredHub/Vault (probably all) users want to be able to use credentials at arbitrary paths, not just the Concourse-enforced schema (/concourse/TEAM/PIPELINE/CRED). This would allow secrets to be reused rather than copied.

Being able to use arbitrary paths would also allow Vault users to use different secret backends for each credential. The current implementation forces users to use the "generic" backend, if they're using recent releases of Vault.

This would be problematic at the moment because credential managers are globally configured, with a single auth scheme (not per-team). So permitting arbitrary path usage would result in team A being able to read team B's credentials, which is Very Bad. Which leads us to...

Per-team credential manager configuration

This would allow teams to use their own private credential managers, and allow one team to use CredHub while another uses Vault. This would allow teams to have different tokens with e.g. Vault, and delegate credential access to Vault's access controls rather than enforcing the path scheme.

We should be careful to encrypt the credential manager config now that it's sitting in the database.

Credential caching

Right now Concourse resolves credentials every time they're used, which can hit Vault and CredHub pretty hard. @rfliam submitted a PR to cache Vault credentials, but in principle this is something we may be able to do generically. (Vault makes it easier to do correctly by having lease durations, but other backends could have some static configuration to determine this at least.)

Other feedback?

I may be missing some stuff here, so feel free to drop in more feedback in the comments. Let's use the text of this first issue as the "source of truth" though - I'll edit it as more feedback comes in.


This comment has been minimized.

sammy commented Jul 19, 2018

Hi there,

I d like to drop a suggestion for Vault.
The idea is that a task should be able to load all available parameters under its pipeline's Vault path and Teams root path, either by default or by a flag, or from an external file.
So our params section for a task could look like:

Assuming team name main and pipeline the snippet below would load by default values stored in vault under /councourse/main/ and /concourse/main/

load_all: true

With the snippet below we could maintain a params file in the code repo, and reuse the specific task for multiple repos
from_file: path/to/params/file

This way we can make tasks generic enough to be easily reusable.

I also described it here:



This comment has been minimized.

goldstar611 commented Nov 1, 2018

I think secrets management engine errors should be prioritized and promoted to admin users in some fashion on the web interface.

For instance, I logged into Concourse CI (v4.2.1) and inspected my failing pipeline only to see an error related to Vault "missing client token". Things go wrong in a dev environment all the time, so first step is to just restart everything. But destroying/setting the pipeline, restarting vault, restarting concourse-web and concourse-workers did do anything for me because the actual error got buried in the syslog:

"Nov  1 13:48:53 concourse-testing concourse[3260]: {"timestamp":"1541098133.032745123","source":
"atc","message":"atc.credential-manager.login.failed","log_level":2,"data":{"error":"Error ma
king API request.\n\nURL: PUT\nCode: 500. Errors:\n
\n* failed to verify client's certificate: x509: certificate has expired or is not yet valid"

It would have been great to know up front that an expired Vault cert was the root issue to the "missing client token" issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment