aws-ecs-lb-athena-table

Checks each load balancer used by an ECS service to ensure logging is enabled and creates an Athena table for these logs

Purpose

Ensures access logging in enabled for all ECS services that use a load balancer and creates a corresponding AWS Athena table for each service

Workings

The tool will iterate over all services in an ECS cluster. For each service that uses a load balancer (either application or classic), it wil enable access logging to the specified S3 bucket. It will then create an Athena table to enable easy query of load balancer access logs.

Prerequisites

• Docker must be installed
• Either an AWS role (if running on EC2) or an access key/secret key

Usage

The easiest way to run the tool is from docker (because docker rocks). Arguments are:

docker pull signiant/monitor-ecs-service

docker run \
  signiant/aws-ecs-lb-athena-table \
       -c My-ECS-Cluster \
       -d elb_logs \
       -r us-east-1 \
       -b my-s3-bucket-for-logs

In this example, the arguments after the image name are

• -c ECS cluster name
• -d Athena database to create tables in
• -r region
• -b S3 bucket to place access logs in

NOTE: The S3 bucket must be pre-created and configured to accept logs for load balancers

To use an AWS access key/secret key rather than a role:

docker run \
  -e AWS_ACCESS_KEY_ID=XXXXXX \
  -e AWS_SECRET_ACCESS_KEY=XXXXXX \
  signiant/aws-ecs-lb-athena-table \
       -c My-ECS-Cluster \
       -d elb_logs \
       -r us-east-1 \
       -b my-s3-bucket-for-logs \
       -f

This example also specifies the -f flag which will force re-create the Athena tables.