copilot variables (secrets: under manifest.yml) are not available in Dockerfile

[sojanthomas-st]

We are trying to deploy our java/spring application using copilot.

The variable passing from manifest.yml file is NOT accessible from .dockerfile. Please see below.

Manifest.yml:

secrets: # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
IWE_ENV: "<arn…IWE_ENV>'"

Dockerfile:

RUN echo $IWE_ENV

If the variable is available in .dockerfile, application will work fine.

Application structure as follows:

application
copilot
manifest.yml
src
Dockerfile
pom.xml
README.md

Please let me know if I need to add more details.

[bvtujo]

Hey @sojanthomas-st, this is working as intended--secrets are only accessible as environment variables from inside the container running on ECS, not at build time. If you need a value to be accessible during Docker build, you can pass it as an argument in the build section of the manifest:

manifest.yml

image:
  build:
    dockerfile: ./Dockerfile
    args:
      MYVAR: myvalue

We don't currently support secrets or SSM parameters during docker build; if you need those, your best bet may be to specify an image location image.location and build your image outside of Copilot with the necessary environment variables specified locally, then push manually.

Does that make sense?

[sojanthomas-st]

Hi Austin, Thanks for your response. Just checking to see if copilot team can add this feature to pass SSM parameters at docker build time? Currently it is a requirement for our application. Thanks Sojan From: Austin Ely ***@***.***> Sent: Monday, August 30, 2021 5:18 PM To: aws/copilot-cli ***@***.***> Cc: Thomas, Sojan ***@***.***>; Mention ***@***.***> Subject: Re: [aws/copilot-cli] copilot variables (secrets: under manifest.yml) are not available in Dockerfile (#2794) Hey @sojanthomas-st<https://github.com/sojanthomas-st>, this is working as intended--secrets are only accessible as environment variables from inside the container running on ECS, not at build time. If you need a value to be accessible during Docker build, you can pass it as an argument in the build section of the manifest: manifest.yml image: build: dockerfile: ./Dockerfile args: MYVAR: myvalue We don't currently support secrets or SSM parameters during docker build; if you need those, your best bet may be to specify an image location image.location and build your image outside of Copilot with the necessary environment variables specified locally, then push manually. Does that make sense? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#2794 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQQCCJEKB3EPXAHA7N6UA5DT7QNUHANCNFSM5DC2X5OQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[bvtujo]

Our current workaround we suggest is to pull the secrets into your environment build and push the image outside of Copilot using the --secret and --ssh flags as appropriate, and specify it in the manifest with the image.location field. Would that work for your use case? Otherwise could you give a thumbs up to #2769? It will help us prioritize.

[sojanthomas-st]

HI Austin, Thanks for your comments. #2769<#2769> Option 2 steps works for us. But that is not enough and need to have secrets available at build time. Something like below. image: build: dockerfile: Dockerfile args: IWE_ENV_VALUE: "'arn:aws:secretsmanager:us-west-2:…..env::'" secrets: # Pass secrets from AWS Systems Manager (SSM) Parameter Store. IWE_ENV_VALUE: "'arn:aws:secretsmanager:us-west-2:…..env::'" Can you please add this request your list and make it available soon? Thanks Sojan From: Austin Ely ***@***.***> Sent: Thursday, September 2, 2021 2:44 PM To: aws/copilot-cli ***@***.***> Cc: Thomas, Sojan ***@***.***>; Mention ***@***.***> Subject: Re: [aws/copilot-cli] copilot variables (secrets: under manifest.yml) are not available in Dockerfile (#2794) Our current workaround we suggest is to pull the secrets into your environment build and push the image outside of Copilot using the --secret and --ssh flags as appropriate, and specify it in the manifest with the image.location field. Would that work for your use case? Otherwise could you give a thumbs up to #2769<#2769>? It will help us prioritize. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#2794 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQQCCJHEAGJW5PRDCEGXTDDT77VY5ANCNFSM5DC2X5OQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[kowsheek]

I spent a tonne of time trying to find a way to inject variables and secrets at build time as well. For compiled applications like Elixir, this would be extremely helpful to read from both Secrets Manager and SSM.

[dereksdev]

This is a necessary feature - we need to pull in an NPM token during build in order to use a private NPM repo

[efekarakus]

Hi @dereksdev !

If you have the secrets available as environment variables, then would this feature help solve your problem? https://aws.github.io/copilot-cli/docs/developing/manifest-env-var/

image:
  build:
    dockerfile: ./Dockerfile
    args:
      MY_SECRET: ${MY_LOCAL_SECRET_FROM_SSM}

[ryanc-scalegrowth]

Would very much prefer to reference secrets directly from our parameter store or secrets manager than to store them in our local environment. Injecting secrets from these services during build time would be a very nice feature to have.