(cdk-assets): docker buildkit secrets from environment

[veimox]

Describe the feature

#23778 Introduced the ability to pass secrets within the build process thanks to BuildKit. Such feature supports secrets from two souces:

1. File: docker will read the file and pass the content as the secret
2. Env: docker will read the secret from the Environment and pass it

The PR aboved implemented (1) but not (2)

Use Case

As a developer,

I want to be able to use the Environment to pass secrets to the Docker build step of CDK,

so I don't have to create ephimeral files from secrets that I already have in the environment. Specifically in Github Action pipelines.

Proposed Solution

In line with what is proposed in here, I suggest to

/**
 * buildArgs?: {
 *    [key: string]: string | undefined;
 * };
 */

// Such
const asset = new DockerImageAsset(stack, 'blah', { directory, buildSecrets: { mySecret: undefined } });

This way we keep the same interface and we take the undefined semantic to fetch the secret from the environment.

Specifically we can modify this line such

// Before
 ...flatten(Object.entries(options.buildSecrets || {}).map(([k, v]) => ['--secret', `id=${k},${v}`])),

// After
...flatten(Object.entries(options.buildSecrets || {}).map(([k, v]) => ['--secret', v===undefined ? `id=${k}`:`id=${k},${v}`])),

Otherwise we can create a new method such

DockerBuildSecret.fromEnvironment('MY_SECRET')

and parse that down the line.

This last method would be more scalable to include other methods such the one mentioned in #14395 (pass secrets from Secrets Manager or SSM)

Other Information

Relates to #14395 but is a different user need

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.69.0

Environment details (OS name and version, etc.)

MacOS Ventura 13.2.1

[pahud]

Thank you for your feature request. Any PR would be highly welcome and appreciated.

[veimox]

@pahud do you have any recommendation of one of the two proposed solutions? Or perhaps a third one?

[danwiltshire]

Hey, I'm not able to continue development on this since the repo refactor. The build takes too much ram on my machine.

> ../../scripts/buildup
...
<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

The crux of the solution is here: https://github.com/danwiltshire/aws-cdk/tree/feat/docker-secret-from-env

[dang312]

Dockerbuildsecret is a very good feature, and so is its .fromEnvironment(). Hope anyone else can took @danwiltshire PR and continue on that...

[miikka]

The buildSecrets values are directly passed to Docker CLI's --secret flag, so as a workaround, you can set their value to type=env. This will read MY_SECRET from the environmental variable MY_SECRET:

const asset = new DockerImageAsset(stack, 'blah', { directory, buildSecrets: { 'MY_SECRET': 'type=env' } });

[dang312]

The buildSecrets values are directly passed to Docker CLI's --secret flag, so as a workaround, you can set their value to type=env. This will read MY_SECRET from the environmental variable MY_SECRET:

const asset = new DockerImageAsset(stack, 'blah', { directory, buildSecrets: { 'MY_SECRET': 'type=env' } });

Can confirm it work in my environment now.
Now I can use CodeArtifact repository and also ultilize docker build secret for caching. Many thanks !