[ecs] [json-secrets]: ContainerDefinition support for JSON secrets

[jmkgreen]

Tell us about your request
Secure deployment of a Docker container with database credentials

Which service(s) is this request for?
ECS (EC2 and Fargate)

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Containers often connect to databases. With ContainerDefinitions we can pass values in from SSM Secrets as environment values through the {{resolve:secretsmanager:/foo:SecretString:FieldName}} mechanism.

However, the documentation for ContainerDefinitions now has Secrets which sounds like an improvement. But it has a ValueFrom that does not appear to support this form of referencing. I tried supplying a ValueFrom of {{resolve:secretsmanager:/foo:SecretString:FieldName}} which results in an error that the ValueFrom value is not understood. I can try to reference my CloudFormation-created secret but it's ARN would then have to be recorded in Parameter Store for referencing in a ContainerDefiintion - and even then I wouldn't be able to pull out the field I want, just the JSON string.

Are you currently working around this issue?
We cannot use Secrets for this purpose. We must remain using environment values.

Additional context
I am under the impression that all the above are intended to be used together..!

[clareliguori]

The resolve:secretsmanager feature in CloudFormation is not intended to work with ValueFrom. "resolve:secretsmanager" resolves the secret value within CloudFormation, and passes the secret value in plain-text to the downstream service API. It's more intended for things like RDS, where the database password is passed to an RDS API in plaintext when creating the database. ECS ValueFrom is when you want to resolve the secret value as close as possible to each container, so that the value is not shown in places like the ECS console.

To confirm I understand the ask correctly, you have a secret "foo" in Secrets Manager with a value like:

{
  "username": "hello",
  "password": "world"
}

And you're looking to assign the value of a single field in that JSON secret to a secret environment variable in your ECS service?

i.e. something like ValueFrom: "arn:aws:secretsmanager:us-west-1:111111111111:secret:foo password" results in an environment variable with value "world". Or in CloudFormation something like this:

ValueFrom: !Join
  - ' '
  - - !Ref MySecret
    - "password"

[jmkgreen]

The resolve:secretsmanager feature in CloudFormation is not intended to work with ValueFrom. "resolve:secretsmanager" resolves the secret value within CloudFormation, and passes the secret value in plain-text to the downstream service API. It's more intended for things like RDS, where the database password is passed to an RDS API in plaintext when creating the database. ECS ValueFrom is when you want to resolve the secret value as close as possible to each container, so that the value is not shown in places like the ECS console.

OK, this is as I originally expected.

To confirm I understand the ask correctly, you have a secret "foo" in Secrets Manager with a value like:

{
  "username": "hello",
  "password": "world"
}

And you're looking to assign the value of a single field in that JSON secret to a secret environment variable in your ECS service?

Yes.

i.e. something like ValueFrom: "arn:aws:secretsmanager:us-west-1:111111111111:secret:foo password" results in an environment variable with value "world". Or in CloudFormation something like this:

ValueFrom: !Join
  - ' '
  - - !Ref MySecret
    - "password"

Yes. The documentation has examples with random characters at the end. It's hard to tell if those random characters are meant to represent a field within JSON (why would it?) or perhaps the random characters CloudFormation adds when creating the resource (why would it?) or something else.

Either way the ValueFrom expects an ARN. If it expected the Name of the Secret then it's easy to plug in, albeit without knowing the sugar to append to pick up the field within. But the ARN is harder as this has random characters at the end which I didn't put there so do not know what they are; the workaround of course is to record the ARN of the secret as a parameter when CFN creates it and load this value when retrieving it - a royal PITA.

I'm hoping I'm simply not reading the documentation as expected and it just needs clarifications perhaps.

[clareliguori]

Ok, this is not something that's supported today, thanks for the feature request

The random characters are part of the Secrets Manager ARN. It adds the random characters to the end of the secret name in the ARN when you create the secret. I believe it's to differentiate it from secrets that may have been previously created with the same name and then deleted.

For example, from the Secrets Manager docs:

$ aws secretsmanager create-secret --name production/MyAwesomeAppSecret --secret-string file://mycreds.json
{
    "SecretARN": "arn:aws:secretsmanager:region:accountid:secret:production/MyAwesomeAppSecret-AbCdEf",
    "SecretName": "production/MyAwesomeAppSecret",
    "SecretVersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE"
}

[mhsdef]

Plus one to this request. I'm an AWS newbie and just spend two hours trying what apparently cannot be done. To work around this I'm going to have to go to every injection point in my app and decode JSON to get at the value I want.

Painful.

[thugec]

+1
Really awful to have so many secret values
it should be per secret - per service and not per secret- per value

[tarciosaraiva]

+1 for that

[dabarrell]

We have a few of use-cases for this, including database credentials. It'd be great if support for this could be added

[ricardolsmendes]

+1

[ns-cweber]

Am I mistaken in thinking that this is a dupe of (or maybe just related to) #385? Also, that issue suggests that this is now supported in CloudFormation, but only for ECS using EC2 launch type, but not for Fargate?

[srrengar]

#385 seems to be the same as this. As ns-cweber mentioned, it is supported for EC2 launch type and we are working on support for Fargate.

[srrengar]

Closing since this is a duplicate of #385