Use a more reliable flow for using secrets in CI

[sherifnada]

Tell us about the problem you're trying to solve

• Github has a max of 200/300 secrets per repo (effectively it is 200 because accessing 300 requires making anyone who edits secrets an admin of the entire Airbyte github org which is not feasible for security reasons unfortunately)
• We are quickly closing in on our max allowed GH secret limit, probably at ~150 now
• GH secrets are write-only so if there is an issue I can't find out what data is in the secret

Describe the solution you’d like

The output of this ticket should be split into two parts:

1. A document which evaluates some secret management solutions/options with a recommendation on which one is the best one. This does not need to be incredibly comprehensive, no need to spend more than 1-2 days on it.
2. Implementing the solution into our CI, so secrets are injected from the secret management solution instead of from the (you will probably need to split this into a few tickets as well)

When evaluating solutions, the following dimensions should be considered:

1. Ease of integrating into Github CI
2. Ease of updating for the developer. It is most important that we can update secrets (for example for new connectors) and see which secrets are being used in CI for debugging.
3. Pricing
4. Max allowed size of secrets (some secret managers have a max allowed size limit on secrets which may prevent us from using big config.json files in CI)
5. Any other dimension you think is important, I probably haven't thought of everything :)

Some ideas:

1. Integrate with an external secret store like Hashicorp Vault. Hashicorp Vault offers a Github integration which can be used to load data from HCV directly into a GH workflow. This would eliminate the issues above.
2. Check-in encrypted secrets into the repo like described here.
3. Google Secrets Manager

Describe the alternative you’ve considered or used

stop creating new connectors
retire

Additional context

Pricing chart for HCV

┆Issue is synchronized with this Asana task by Unito

[sherifnada]

This just got bumped into high priority because we reached the max secrets limit on github

[Phlair]

short-term solution here: #5085