Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connectors-ci: mask secrets in GHA logs with ::add-mask:: #27087

Merged
merged 4 commits into from
Jun 6, 2023

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Jun 6, 2023

What

Closes #27088

On our connectors testing pipeline our internal ci_credentials library is used to download GSM secrets to file system.
In the context of dagger pipeline the interaction with this library is containerized.
This library uses a VERSION env var to modulate it's behavior:
If VERSION=dev:

  • Write secrets to current working directory: it's the behavior we want for dagger pipelines
  • Do not print out ::add-mask:: logs that are used to mask secrets in GHA logs

If VERSION != dev:

  • Write secrets to an absolute path relative to the GHA action runners file system
  • Print ::add-mask:: to stdout to make GHA redact the secrets from the logs.

We've originally set the env var to dev in our dagger pipeline to get secret written at the right path but we miss the secret masking capabilities while doing it.

How

This PR changes ci_credentials and its usage in our dagger pipeline to:

  • Write secrets to a correct path while not using VERSION=dev
  • Write secrets to mask to a /tmp file that dagger pipeline can read and print out to stdout without any logger prefix.
  • Increase the list of keys considered as secrets by consuming our spec mask.

@alafanechere alafanechere requested review from perangel, cpdeethree and a team June 6, 2023 20:25
@alafanechere
Copy link
Contributor Author

alafanechere commented Jun 6, 2023

🎉 I confirm it work. In a test commit I printed secrets on purpose to the logs and artificially modified a connector.
The secrets are redacted from the GHA logs.
Screen Shot 2023-06-06 at 22 52 04

@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Jun 6, 2023
@alafanechere alafanechere enabled auto-merge (squash) June 6, 2023 21:15
@alafanechere alafanechere merged commit ee1be35 into master Jun 6, 2023
19 checks passed
@alafanechere alafanechere deleted the augustin/connectors-ci/mask-secrets branch June 6, 2023 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

connectors-ci: mask connector secrets in logs
3 participants