Add native support for secret managers to Backstage config

[taras]

Context

One of our client is a financial institution that has very rigid security requirements. They do not allow storing credentials in environment variables or anywhere in memory. They recommend that the application calls the secrets manager anytime it needs credentials or a token. Implementing this requirement in the project is difficult because most Backstage plugins assume that the configuration provides sensitive credentials.

Feature Suggestion

Add support for reading secrets directly from secret managers like AWS Secret Manager to Backstage config.

Possible Implementation

I do not see an easy implementation because the secret must be fetched when needed. This means that reading secrets will have to be asynchronous. All the types provided by config are sync, so we'd need a new interface, and all plugins that read secrets synchronously need to change to read the same values async.

I'm creating this issue to keep track of this use case, but I'm unsure how we could do this.

Alternative Solution

Considering that there probably is no easy way to add this functionality without introducing major breaking changes, the alternative approach could be to make the proxy plugin smarter by making it secret store aware. We could allow passing secret ID as configuration. The proxy plugin would look up the secret and append it to the request before passing it on to the destination.

[Rugvip]

Yep, seems tricky to do due to how widespread the config reading is. I do think that once we've migrated more integrations to use the GithubCredentialProvider pattern we might be in a better place to support this for the core integrations though. Regarding the proxy plugin I'd try to avoid injecting secrets in upstream requests altogether tbh, if there are strict security requirements.

Another approach can be to move more things to read-only and overall reduce the scope of the backend secrets, then rely more on client-provided credentials for write actions.

[taras]

Regarding the proxy plugin I'd try to avoid injecting secrets in upstream requests altogether tbh, if there are strict security requirements.

Can you say more about this? Here is what I had in mind

[taras]

@Rugvip would you be open to a PR to add this to the Backstage Proxy Plugin? Alternatively, we'll need to create a separate plugin and that seems like a waste.

[Rugvip]

@taras see the proxy section in the threat model for why proxy plugin isn't a great fit.

For this particular use-case I think a BitbucketCloudCredentialsProvider or something like that is the way to go. That's the way we really want to be injecting custom logic for auth towards external integrations. Especially as the new backend system is being rolled out, that'll be a lot easier to add, but we could start the work before that too.

[taras]

@Rugvip got it, that's a good point. Thank you

[awanlin]

Just wanted to mention that this has come up often on Discord where people want to be able to pull this from Vault without using Environment Variables

[shailendra-ahir-cbre]

Just wanted to mention that this has come up often on Discord where people want to be able to pull this from Vault without using Environment Variables

Yes. We also want same solution.

[freben]

I just want to mention that there's some related discussion in #14084 as linked above. This will probably be best achieved by contributing an update to the intermediate config loading interfaces so that we can get easily pluggable config sources into place.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.