Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support logConfiguration.secretOptions via ecs-params #777

Merged
merged 4 commits into from
May 23, 2019
Merged

Support logConfiguration.secretOptions via ecs-params #777

merged 4 commits into from
May 23, 2019

Conversation

ahayworth
Copy link
Contributor

@ahayworth ahayworth commented May 22, 2019

Description of changes:

This PR enables ecs-cli to support the logConfiguration.secretOptions array as described in the ECS docs.

These parameters allow one to inject secrets into log config options, much the same as one may inject secrets into environment variables via the secrets array - eg, a splunk token. This PR configures these options under the container defs, like so:

version: 1
task_definition:
  services:
    foo:
      logging:
        secret_options:
          - name: super-secret
             value_from: 'arn::aws::blah'
...

To do so, this PR upgrades the aws sdk to 1.19.22 and re-generates mocks. Tests seem to pass, and I can confirm that I'm able to launch an ECS service with secrets parameters successfully injected into logging configuration.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing

  • Unit tests passed
  • Integration tests passed
  • Unit tests added for new functionality
  • Listed manual checks and their outputs in the comments (example)
  • Link to issue or PR for the integration tests:

Documentation

  • Contacted our doc writer
  • Updated our README

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This version of the AWS SDK is the oldest that supports the ECS
SecretOptions container definition option.

At time of writing, it's not terribly far behind the latest release.
This is an ECS-only extension to the docker logging parameters
that allow you to inject secrets values into logging configuration
options: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html#secrets-logconfig

They are almost identical to secrets, and in fact seem to share some
of the same struct code in the aws sdk.
Tests passed after this...
@efekarakus efekarakus changed the base branch from master to dev May 22, 2019 22:28
@PettitWesley PettitWesley self-requested a review May 22, 2019 22:37
@efekarakus
Copy link
Contributor

Awesome 🎉!
Thanks @ahayworth, we'll take a look at this PR soon and get back to you :)

README.md Outdated
@@ -539,6 +543,9 @@ Fields listed under `task_definition` correspond to fields that will be included
* `secrets` allows you to specify secrets which will be retrieved from SSM Parameter Store. See the [ECS Docs](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) for more information, including how reference AWS Secrets Managers secrets from SSM Parameter Store.
* `value_from` is the SSM Parameter ARN or name (if the parameter is in the same region as your ECS Task).
* `name` is the name of the environment variable in which the secret will be stored.
* If you need to inject secrets into your logging configuration, you may set `secret_options` under `logging`. For more information, See the [logging secrets section](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html#secrets-logconfig) of the ECS docs.
* `value_from` is the SSM Parameter ARN or name (if the parameter is in the same region as your ECS Task).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets manager ARN is also supported

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed! 😄
f30c51f

@efekarakus efekarakus self-requested a review May 22, 2019 22:40
@ahayworth
Copy link
Contributor Author

ahayworth commented May 22, 2019

Thanks for fixing the base branch @efekarakus :)

Manual Testing

New Functionality

Configuration

I tested with an in-house deployment, so I've redacted some parameters...

Parameters
# docker-compose.yaml
services:
  redacted-app:
    command: redacted-app
    environment:
      BOT_ENV: staging-ecs
      STATS_ADDR: localhost:8125
    healthcheck:
      interval: 5s
      retries: 5
      test: curl -f http://localhost:3000/_ping
      timeout: 2s
    image: redacted-ecr-repo-url:${SHA:?Deployment SHA is undefined!}
    logging:
      driver: splunk
      options:
        splunk-index: prod-sre
        splunk-sourcetype: sre
        splunk-url: redacted-splunk-url
        tag: '{{.ImageName}}/{{.Name}}/{{.ID}}'
# ecs-params.yaml
version: 1
run_params:
  network_configuration:
    awsvpc_configuration:
      assign_public_ip: DISABLED
      security_groups:
      - sg-12345
      subnets:
      - subnet-12345
task_definition:
  ecs_network_mode: awsvpc
  services:
    redacted-app:
      cpu_shares: 512
      essential: true
      healthcheck:
        start_period: 60s
      logging:
        secret_options:
        - name: splunk-token
          value_from: 'arn:aws:secretsmanager:12345...'
      mem_limit: 512
      secrets:
      - name: BOT_ID
        value_from: 'arn:aws:secretsmanager:123457849...'
  task_execution_role: arn:aws:blah
  task_size:
    cpu_limit: 2048
    mem_limit: 4096

Tests

Create and run a service with secret log params
INFO[0001] Using ECS task definition                     TaskDefinition="ecs-redacted-app:11"
INFO[0001] Updated the ECS service with a new task definition. Old containers will be stopped automatically, and replaced with new ones  desiredCount=1 force-deployment=true service=ecs-redacted-app
INFO[0002] Service redacted-app                                desiredCount=1 runningCount=1 serviceName=redacted-app
INFO[0002] ECS Service has reached a stable state        desiredCount=1 runningCount=1 serviceName=redacted-app

The friendly neighborhood splunk admin assures me that logs are flowing in (whereas before they most definitely were not), so I presume it to be working. 😄

Copy link
Contributor

@efekarakus efekarakus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! I also ran our integration tests against the change and it passes 😄

@efekarakus efekarakus merged commit a3946a4 into aws:dev May 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants