Support forwarding env vars from miniwdl's env to tasks #516
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
We're running miniwdl as part of our AWS batch pipelines, and some of the tasks in our pipelines need to be able to fetch secrets from AWS Secrets Manager. In order for our tasks to be able to assume the correct role (the role associated with the Batch task), we need the
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
environment variable forwarded from miniwdl's environment into the task's environment.Since we have no control over the value of this environment variable, there's no way for us to specify it as an input to miniwdl. Many other compute environments pass important context via env vars to the containers they run, so I suspect this functionality will be useful beyond just AWS Batch.
Approach
This PR allows users to specify a list of env vars that will be forwarded from miniwdl's execution environment into the environment of all containers launched via miniwdl.
Checklist
make pretty
to reformat the code with blackmake check
to statically check the code using Pyre and Pylint