Skip to content
Public documentation for boltopspro/ecs-spot
Ruby Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.meta add public docs Nov 6, 2019
app add public docs Nov 6, 2019
docs add public docs Nov 6, 2019
seed add public docs Nov 6, 2019
.gitignore add public docs Nov 6, 2019
CHANGELOG.md add public docs Nov 6, 2019
Gemfile add public docs Nov 6, 2019
LICENSE.txt add public docs Nov 6, 2019
README.md add public docs Nov 6, 2019
Rakefile add public docs Nov 6, 2019
ecs-spot.gemspec add public docs Nov 6, 2019

README.md

NOTE: This repo contains only the documentation for the private BoltsOps Pro repo code. Original file: https://github.com/boltopspro/ecs-spot/blob/master/README.md The docs are publish so they are available for interested customers. For access to the source code, you must be a paying BoltOps Pro subscriber. If are interested, you can contact us at contact@boltops.com or https://www.boltops.com

ECS Spot Blueprint

Watch the video

Prerequisite

Usage

  1. Add blueprint to Gemfile
  2. Configure: configs/ecs-spot values
  3. Deploy blueprint

Add

Add the blueprint to your lono project's Gemfile.

gem "ecs-spot", git: "git@github.com:boltopspro/ecs-spot.git"

Configure

Use the lono seed command to generate a starter config params files.

LONO_ENV=development lono seed ecs-spot
LONO_ENV=production  lono seed ecs-spot

The files in config/ecs-spot folder will look something like this:

configs/ecs-spot/
├── params
│   ├── development.txt
│   └── production.txt
└── variables
    ├── development.rb
    └── production.rb

Configure the configs/ecs-spot/params and configs/ecs-spot/variables files. There are 2 parameters required: Subnets and Vpc.

configs/ecs-spot/params/development.txt:

# Required parameters:
Subnets=subnet-111,subnet-222 # Find at vpc CloudFormation Outputs
Vpc=vpc-111 # Find at vpc CloudFormation Outputs
# Optional parameters:
# EcsCluster=development
# KeyName=...
# MaxCapacity=100
# MinCapacity=16
# SnsTopicArn=...
# SshLocation=...
# TagName=ecs-spot-development
# TargetCapacity=16
# ExistingSecurityGroup=...

configs/ecs-spot/variables/development.rb:

@instance_types = {
  "m5.large" => {ram: 8.0, cpu: 2}, # supports ENI trunking
#  "r3.large" => {ram: 15.25, cpu: 2},
}

A quick way to get the VPC and subnet values is from the vpc CloudFormation Outputs. Here's an example of development.

It is recommended to run the ECS containers on the PrivateAppSubnets.

Repeat the same process and configure params and variables files for the production environment also.

Deploy

Use the lono cfn deploy command to deploy.

LONO_ENV=development lono cfn deploy ecs-spot --sure --no-wait
LONO_ENV=production  lono cfn deploy ecs-spot --sure --no-wait

If you are using One AWS Account, use these commands instead: One Account.

Instance Access

To access the ec2 instances for debugging we strongly recommend using SSM Session Manager. The instances in this blueprint have SSM manager installed. You can quickly setup Session Manager on your AWS account with the Session Manager Pro script. There are also other ways to access your instance, more info: Instance Access

IAM Permissions

The IAM permissions required for this stack are described below.

Service Description
application-autoscaling Spot Fleet autoscaling
cloudformation To launch the CloudFormation stack.
cloudwatch CloudWatch alarms for AutoScaling.
ec2 Spot Fleet and Security groups.
iam IAM roles for Instance, Spot Fleet, and Auto Scaling
s3 Lono managed s3 bucket

Considerations

This template listens to the spot two-minute warning signal and calls ECS Draining when it is detected. ECS Draining essentially moves the Docker containers to another available EC2 Container Instances.

It takes some time for the new ECS task to reach steady state. If your Docker container bootup process takes too long, then this can result in unavailability. The availability detection is also dependent on the ELB Health Check settings. If the ELB Target Group Health Check takes more than 2 minutes, then the new ECS task will not register in time.

Summary:

  • Your application bootup needs to be fast enough.
  • The ELB health check setting needs to be fast enough.
  • Scaling up and down spot fleets will result in the two-minute warning being fired.
  • Updating the ECS spot fleet CloudFormation stack does not fire the two-minute warning.
  • Follow a best practice and always run at least 2 containers.

Back to Reference Architecture

That's it. Go back to the main boltopspro/reference-architecture

You can’t perform that action at this time.