Skip to content

Commit

Permalink
馃摎 Document CI secrets workflow for connector development (#3733)
Browse files Browse the repository at this point in the history
  • Loading branch information
sherifnada committed Jun 1, 2021
1 parent bb4dcb1 commit 691f8b1
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/contributing-to-airbyte/building-new-connector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,11 @@ Once you've finished iterating on the changes to a connector as specified in its

6. The new version of the connector is now available for everyone who uses it. Thank you!

## Using credentials in CI
In order to run integration tests in CI, you'll often need to inject credentials into CI. There are a few steps for doing this:

1. **Place the credentials into Lastpass**: Airbyte uses a shared Lastpass account as the source of truth for all secrets. Place the credentials **exactly as they should be used by the connector** into a secure note i.e: it should basically be a copy paste of the `config.json` passed into a connector via the `--config` flag. We use the following naming pattern: `<source OR destination> <name> creds` e.g: `source google adwords creds` or `destination snowflake creds`.
2. **Add the credentials to Github Secrets**: To inject credentials into a CI workflow, the first step is to add it to Github Secrets. Admin access to the Airbyte repo is required to do this. All Airbyte engineers have admin access and should be able to do this themselves. External contributors or contractors will need to request this from their team lead or project manager who should have admin access. Follow the same naming pattern as all the other secrets e.g: if you are placing credentials for source google adwords, name the secret `SOURCE_GOOGLE_ADWORDS_CREDS`. After doing this step, the secret will be available in the Github run using the workflow secrets syntax.
3. **Inject the credentials into test and publish CI workflows**: edit the files `.github/workflows/publish-command.yml` and `.github/workflows/test-command.yml` to inject the secret into the CI run. This will make these secrets available to the `/test` and `/publish` commands.
4. **During CI, write the secret from env variables to the connector directory**: edit `tools/bin/ci_credentials.sh` to write the secret into the `secrets/` directory of the relevant connector.
5. That should be it.

0 comments on commit 691f8b1

Please sign in to comment.