Skip to content
This repository was archived by the owner on Sep 28, 2022. It is now read-only.

Commit 8ae96ee

Browse files
committed
Rewrote README instructions to make it easier to launch
1 parent c5d374f commit 8ae96ee

File tree

4 files changed

+72
-71
lines changed

4 files changed

+72
-71
lines changed

README.md

Lines changed: 69 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -21,31 +21,56 @@ This sample code is made available under a modified MIT license. See the LICENSE
2121

2222
## Requirements
2323

24-
- Python 2.7
25-
- Docker CLI
24+
- Python 2.7+
25+
- Docker
2626
- Access to an AWS account
27-
- A DockerHub account
27+
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html)
2828

2929
## Examples
3030

3131
- [Load testing a basic HTTP endpoint](https://github.com/aws-samples/distributed-load-testing-using-aws-fargate/tree/master/examples/http)
32-
- [Load testing an Elasticsearch cluster behind a VPC](https://github.com/aws-samples/distributed-load-testing-using-aws-fargate/tree/master/examples/elasticsearch)
33-
- [Coming next] Simulating IoT devices for load testing.
34-
- [Coming next] Simulating Kinesis Streams producers.
32+
- [Load testing an Elasticsearch cluster behind a VPC](https://github.com/aws-samples/distributed-load-testing-using-aws-fargate/tree/master/examples/elasticsearch)
3533

3634

3735
## Getting Started
3836

39-
### 1. Clone this repository
37+
### 1. Launch Solution
38+
39+
In this step you will launch the `Master` CloudFormation stack that will create a Fargate Cluster, an ECR Docker registry, an IAM
40+
Execution Role, a Task Definition, a CloudWatch Log Group, a Security Group and a new VPC with Public Subnets.
41+
42+
Region Name | Region Code | Launch
43+
------|-----|-----
44+
US East (N. Virginia) | us-east-1 | [![Launch in us-east-1](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/master.yaml)
45+
US East (Ohio) | us-east-2 | [![Launch in us-east-2](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/master.yaml)
46+
US West (Oregon) | us-west-2 | [![Launch in us-west-2](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/master.yaml)
47+
48+
You will notice that the CloudFormation stack will expand itself into 2 additional nested stacks: The master stack takes
49+
care of creating global resources like an ECR Registry and the IAM Execution Role. One nested stack takes care of
50+
creating the network (VPC) and the other creates the Fargate cluster.
51+
52+
Once the three stacks have completed deployment, you need to make a note of the Output values in the master stack
53+
(see the following Screenshot) as you will need these values later. Go to the Outputs section of the
54+
master stack and save the DockerRegistryName and DockerRegistryURL values somewhere in your notes.
55+
56+
![CloudFormation Screenshot](docs/cfn-outputs.png)
57+
58+
### 2. Clone this repository
4059

4160
```bash
4261
git clone https://github.com/aws-samples/distributed-load-testing-using-aws-fargate.git
4362
```
4463

45-
### 2. Modify the Taurus test scenario
64+
Switch to the latest release so you have an stable code base.
65+
66+
```bash
67+
git checkout tags/v0.2-beta
68+
```
69+
70+
### 3. Modify the load test scenario
4671

4772
Configure your test scenario by editing the `examples/http/taurus.yml` file.
48-
To can learn more about the syntax of this file, check the Taurus docs: https://gettaurus.org/kb/Index .
73+
To learn more about the syntax of this file, check the Taurus docs: https://gettaurus.org/kb/Index .
4974

5075
```yaml
5176
execution:
@@ -60,66 +85,43 @@ scenarios:
6085
- http://aws.amazon.com
6186
```
6287
63-
### 3. Build and publish the docker image
64-
65-
Once you have completed your test scenario. You need to package it in a Docker image and publish it
66-
in the Docker Hub or in a private registry of your choice.
88+
### 4. Build and push the docker image
6789
68-
For simplicity, I'm going to use the public Docker Hub. If you don't have an account, go ahead and create one in
69-
https://hub.docker.com. Then login from the terminal:
90+
Once you have completed your test scenario, you need to package it in a Docker image and push it to your ECR Docker registry.
91+
To do this, you first need to authenticate against ECR. The following command assumes you have the [AWS CLI installed and configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html).
7092
7193
```bash
72-
docker login
94+
$(aws ecr get-login --no-include-email --region us-west-2)
7395
```
7496

75-
Once logged in, you can build the image running docker build from the root folder of this project.
97+
Then, build the Docker image by running the following command from the root directory of this project. Replace
98+
_{DockerRegistryName}_ with the Output value you saved in Step 1.
7699

77100
```bash
78-
docker build -t your_docker_hub_user/dlt-fargate .
101+
docker build -t {DockerRegistryName} .
79102
```
80103

81-
Test your docker container locally to verify it runs correctly:
104+
Tag the image as `:latest`, you can choose another tag name, but for simplicity I'm doing _latest_.
82105

83106
```bash
84-
docker run -it dlt-fargate taurus.yml
107+
docker tag {DockerRegistryName}:latest {DockerRegistryURL}:latest
85108
```
86109

87-
Now you can push the image to the registry
110+
Test your docker container locally to verify it runs correctly.
88111

89112
```bash
90-
docker push your_docker_hub_user/dlt-fargate
113+
docker run -it {DockerRegistryName} taurus.yml
91114
```
92115

93-
### 4. Create the Fargate Clusters
116+
If the docker container ran as expected, you can proceed to push the image to the registry. Unfortunately the Docker
117+
image for Taurus is quite heavy (~900 MB), so make sure you have good internet connectivity as this push can take a
118+
little while.
94119

95-
Create the Fargate clusters in your AWS account by running the CloudFormation template in `cloudformation/main.yml` on
96-
every region where you want to run tests from. This example works for `us-east-1`, `us-east-2` and `us-west-2`
97-
but its easy to extend the template to work on other regions.
98-
99-
**Note**: Make sure Fargate is available in the regions you want to run this from.
100-
Here is a list of products by region: https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services.
101-
102-
**[Optional]** An alternative way of running this CloudFormation template on multiple regions is to use
103-
[CloudFormation StackSets](https://aws.amazon.com/blogs/aws/use-cloudformation-stacksets-to-provision-resources-across-multiple-aws-accounts-and-regions),
104-
which allow you to create resources on multiple accounts and regions at once. There is some IAM work that needs to be
105-
done in your account to use StackSets, but it's worth the effort.
106-
107-
The CloudFormation template will ask for a few basic parameters and will create everything needed to run Fargate on AWS;
108-
including a VPC, 3 public subnets in different AZs, a security group, an internet gateway, a route table, a CloudWatch
109-
group, a CloudWatch Log Filter, an IAM role for the tasks, an ECS cluster and the Task Definition for Fargate.
110-
111-
![CloudFormation](docs/cloudformation.png)
112-
113-
- **VpcCidrBlock**. CIDR block of the VPC that will be created for the cluster to run on.
114-
- **SubnetACidrBlock**. CIDR block of Subnet A in the VPC.
115-
- **SubnetBCidrBlock**. CIDR block of Subnet B in the VPC.
116-
- **SubnetCCidrBlock**. CIDR block of Subnet C in the VPC.
117-
- **DockerImage**. Specify the docker image that you published to the DockerHub.
118-
- **DockerTaskCpu**. Number of CPU units to assign to the Fargate tasks ([Task Size Reference](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size)).
119-
- **DockerTaskMemory**. Memory in MB to assign to the Fargate tasks ([Task Size Reference](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size))
120-
- **FargateClusterName**. Name of your cluster so you can identify it in the ECS Console.
120+
```bash
121+
docker push {DockerRegistryURL}:latest
122+
```
121123

122-
### 5. Run the tests
124+
### 4. Run the tests in Fargate
123125

124126
Finally, edit the `bin/runner.py` python file to add the list of regions with its CloudFormation stack names that
125127
you launched on the previous step. This python file will read the CloudFormation outputs and based on those parameters
@@ -152,7 +154,7 @@ And finally, when you are ready to run the Distributed Load Test, run the script
152154
python runner.py
153155
```
154156

155-
### 6. Monitor the tests in CloudWatch
157+
### 5. Monitor the test execution in CloudWatch
156158

157159
The CloudFormation template should have created a [CloudWatch Metric Filter](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html)
158160
that will capture the average response time for each HTTP request that was issued to your system under test. You should
@@ -181,3 +183,19 @@ The Dashboard will look something like this:
181183

182184
![CloudWatch](docs/cloudwatch.jpg)
183185

186+
## Launch in Additional Regions (Optional)
187+
188+
It may be likely that running this solution from a single AWS region is enough to load test your application. However,
189+
if you want to take it a step further, you can do so by deploying Fargate clusters in multiple regions and make this a
190+
real distributed load test simulation. For this, I have created a separate CloudFormation template for you to launch the
191+
solution in additional regions. The difference between this template and the Master one, is that this one doesn't
192+
create the ECR Docker Registry and the IAM Execution Role, as you can share these resources across all Fargate deployments.
193+
194+
Use the following buttons to launch the solution in the desired additional regions:
195+
196+
Additional Region | Region Code | Launch
197+
------|-----|-----
198+
US East (N. Virginia) | us-east-1 | [![Launch in us-east-1](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/additional-region.yaml)
199+
US East (Ohio) | us-east-2 | [![Launch in us-east-2](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/additional-region.yaml)
200+
US West (Oregon) | us-west-2 | [![Launch in us-west-2](https://camo.githubusercontent.com/210bb3bfeebe0dd2b4db57ef83837273e1a51891/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f636c6f7564666f726d6174696f6e2d6578616d706c65732f636c6f7564666f726d6174696f6e2d6c61756e63682d737461636b2e706e67)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=WildRydes-Cloud9&templateURL=https://s3-us-west-2.amazonaws.com/load-testing-using-aws-fargate/artifacts/templates/additional-region.yaml)
201+

cloudformation/fargate-cluster.yaml

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -27,26 +27,9 @@ Parameters:
2727
DockerTaskMemory:
2828
Type: Number
2929
Default: 2048
30-
AllowedValues:
31-
- 512
32-
- 1024
33-
- 2048
34-
- 3072
35-
- 4096
36-
- 5120
37-
- 6144
38-
- 7168
39-
- 8192
40-
- 16384
4130
DockerTaskCpu:
4231
Type: Number
4332
Default: 512
44-
AllowedValues:
45-
- 256
46-
- 512
47-
- 1024
48-
- 2048
49-
- 4096
5033

5134
Resources:
5235
Cluster:

cloudformation/master.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,9 +59,9 @@ Resources:
5959
- ':latest'
6060

6161
Outputs:
62-
TaskExecutionRoleArn:
63-
Value: !GetAtt TaskExecutionRole.Arn
64-
DockerRegistry:
62+
DockerRegistryName:
63+
Value: !Ref DockerRegistry
64+
DockerRegistryURL:
6565
Value: !Join
6666
- ''
6767
- - !Ref 'AWS::AccountId'

docs/cfn-outputs.png

258 KB
Loading

0 commit comments

Comments
 (0)