How to configure AWS for the class

Introduction

This page documents the one-time configuration steps we took to setup an AWS account for the class. While students might be curious about this page, the material herein is intended for the instructors setting up the infrastructure for future iterations of the workshop.

AWS account

We chose to create a dedicated AWS account for the class (and AWS was kind enough to give us educational credits to defray the costs incurred in this account).

IAM

We practiced some degree of least privilege, within what we could achieve using AWS-managed (vs custom) IAM policies.

To that end, we created a group Students and attached the following AWS-managed policies: AmazonEC2ContainerRegistryFullAccess, CloudWatchFullAccess, AmazonEC2ContainerServiceFullAccess, IAMReadOnlyAccess. Each student was given an IAM account (with Console and API access) and was made a member of this group.

For instructors, we created two IAM accounts, an "admin" account with full access to the account, and a "student" account which was a member of the Students group. Since the AWS keys are visible in the ECS configuration state of a running Chatbot (see issue #22 and also #11), it's particular important that instructors use their student account to deploy Chatbots. (We also created a dummy student account (tooler1) which we used to test documentation.)

We created a user Headless-travisci in order to get access keys for our CI system. This user has a slightly more restricted set of permissions than students (specifically: AmazonEC2ContainerRegistryPowerUser, CloudWatchFullAccess, AmazonEC2ContainerServiceFullAccess).

Creating, scaling, and destroying the ECS cluster

I've tried creating a cluster from both the ECS Web UI in the AWS console, and from the ecs-cli command-line tool. The former didn't work: in particular, while EC2 instances were created, for some reason they were somehow not "registered" into the cluster, causing the system to complain that there wasn't any capacity to deploy services.

The following worked (with the right AWS keys installed in my environment):

ecs-cli up --region us-east-1 -c limbo --capability-iam --image-id ami-fad25980 --size 2 --instance-type t2.medium

Note the explicit reference to the image id (ami-fad25980). This is the latest ECS-optimized image for us-east-1 published by AWS as of 12/18/2017 (see http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html). (Interestingly, the instances created by the Web UI used this latest image -- but those instances were left out of the cluster.)

Scaling the cluster:

ecs-cli scale --region us-east-1 -c limbo --capability-iam --size 4

Destroying the cluster is straightforward:

ecs-cli down -c limbo --force

CloudWatch

We created a Log Group called limbo shared by all Chatbots:

aws logs create-log-group --log-group-name limbo --region us-east-1

Each Chatbot had its own stream within that group. We used each Chatbot's "botname" as the awslogs-stream-prefix parameter to the awslogs drive to name these streams.

We created a Dashboard for the class that contained vital metrics of the ECS cluster.