An AWS CloudFormation template that builds and load tests two multi-tenant Amazon Aurora MySQL-Compatible Edition clusters, one with and one without Amazon RDS Proxy. This is a repository that is referenced in the accompanying blog post: Build and load test a multi-tenant SaaS database proxy solution with Amazon RDS Proxy.
Reference architecture:
To deploy the solution, you will require an AWS account. If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Your access to the AWS account must have AWS IAM permissions to launch AWS CloudFormation templates that create IAM roles.
The application is deployed as an AWS CloudFormation template.
Note You are responsible for the cost of the AWS services used while running this sample deployment. There is no additional cost for using this sample. For full details, see the pricing section of this readme, as well as the pricing pages for each AWS service you will be using in this sample. Prices are subject to change.
- Deploy the latest CloudFormation template by following the link below for your preferred AWS region:
| Region | Launch Template |
|---|---|
| US East (N. Virginia) (us-east-1) |  |
| US East (Ohio) (us-east-2) | |
| US West (N. California) (us-west-1) | |
| US West (Oregon) (us-west-2) | |
| Asia Pacific (Mumbai) (ap-south-1) | |
| Asia Pacific (Seoul) (ap-northeast-2) | |
| Asia Pacific (Singapore) (ap-southeast-1) | |
| Asia Pacific (Sydney) (ap-southeast-2) | |
| Asia Pacific (Tokyo) (ap-northeast-1) | |
| Canada (Central) (ca-central-1) | |
| Europe (Frankfurt) (eu-central-1) | |
| Europe (Ireland) (eu-west-1) | |
| Europe (London) (eu-west-2) | |
- If prompted, login using your AWS account credentials.
- You should see a screen titled "Create Stack" at the "Specify template" step. The fields specifying the CloudFormation template are pre-populated. Click the Next button at the bottom of the page.
- On the "Specify stack details" screen you may customize the following parameters of the CloudFormation stack:
| Parameter label | Default | Description |
|---|---|---|
| Create Load Test Stack | true | If False, this creates a Proxy VPC and accompanying reosurces. If True, this additionally creates a Load Test VPC, No Proxy VPC, and accompanying reosurces, in order to run a load test and compare metrics between the Proxy and No Proxy VPCs. |
| Availability Zones | Requires input | The list of Availability Zones to use for the subnets in the VPCs. Select two Availability Zones from the list. |
| Database Reader Instance Class | db.r5.large | The database instance class for the Proxy and No Proxy VPC Amazon Aurora Replicas, for example db.m5.large. |
| Database Writer Instance Class | db.t3.medium | The database instance class for the Proxy and No Proxy VPC Amazon Aurora Writer, for example db.m5.large. |
| Performance Insights Retention Period | 7 | The amount of time, in days, to retain RDS Performance Insights data. Valid values are 7 and 731 (2 years). |
| Lambda Runtime Environment | Node.js | The runtime for the Lambda access function. Valid values are (Python, Nodejs). |
| Infrastructure Environment | DEV | The type of environment with which to tag your infrastructure. Valid values are DEV (development), TEST (test), or PROD (production). |
| Flow Logs | false | Creates an optional CloudWatch Logs group to send the VPC flow logs to. Flow Logs incur additional costs. Set to "false" to disable. |
| Number of Tenants to Create | 200 | The number of tenants to create in the Proxy and No Proxy VPC Aurora clusters. Each tenant has their own dedicated database containing dummy data. Allowed values are 1-200. |
| Latest Amazon Linux AMI | /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 | The latest Amazon Linux AMI from AWS Systems Manager Parameter Store. |
| Locust Instance Type | c5.large | The Amazon EC2 instance type used in the Load Test cluster that runs Locust. |
| Locust App Version | latest | The Locust version to deploy. |
| Locust Worker Instances | 2 | The number of secondary Amazon EC2s for the Load Test Cluster. Allowed values are 2-20. |
| API Endpoint Type | PRIVATE | The Amazon API Gateway endpoint type. Valid values are (EDGE, REGIONAL, PRIVATE). |
| ISP/Public IPv4 | Requires input | The CIDR block or your IP address that you will use to connect to the Locust Dashboard. This limits the CIDR range from which the Locust dashboard can be accessed. |
Note Whilst you can modify the name of the stack, do not increase the length of its name to more than 21 characters. Doing so will lead to a 'CREATE_FAILED' for the stack, with an 'Invalid principal in policy' error message for either the ProxyAccessStack or the NoProxyAccessStack. The reason for this is an AWS Lambda function name exceeding the maximum number of allowed characters. More information on this can be found here.
When completed, click Next.
-
Configure stack options if desired, then click Next.
-
On the review you screen, you must check the boxes for:
- "I acknowledge that AWS CloudFormation might create IAM resources"
- "I acknowledge that AWS CloudFormation might create IAM resources with custom names"
- "I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND"
These are required to allow CloudFormation to create a Role to allow access to resources needed by the stack and name the resources in a dynamic way.
-
Click Create Stack.
-
Wait for the CloudFormation stack to launch. This will take around 30 minutes. Completion is indicated when the "Stack status" becomes "CREATE_COMPLETE".
- You can monitor the stack creation progress in the "Events" tab.
-
Note the LocustAddress and APIGatewayURL displayed in the Outputs tab of the main stack. These can be used to access the Locust dashboard and direct the load test towards the created API Gateway.
See the Local Development guide to get a copy of the project up and running on your local machine for development and testing purposes.
To remove the stack:
- Open the AWS CloudFormation Console.
- Click the rds-proxy-load-test project, right-click and select "Delete Stack".
- Your stack will take some time to be deleted. You can track its progress in the "Events" tab.
- When it is done, the status will change from "DELETE_IN_PROGRESS" to "DELETE_COMPLETE". It will then disappear from the list.
This sample is intended to be deployed only for as long as is strictly necessary, to avoid incurring additional costs. As soon as the load test is completed, the stack should be deleted (see 'Clean up' above). Note that all pricing is estimated based on the us-east-1 region, and without the Free Tier.
Assuming a 150 request per second load test for 30 minutes, the price breakdown is estimated as follows:
| Service | Cost | Description |
|---|---|---|
| Amazon Aurora MySQL | $0.78 | 4x db.r5.large and 2x db.t3.medium database instances running continuously, 150 RPS. |
| Amazon RDS Proxy | $0.02 | Based on 2 vCPUs of db.r5.large, running continuously. |
| Amazon EC2 | $0.14 | 3x c5.large instances running continuously. |
| Amazon CloudWatch | $0.01 | CloudWatch dashboard + 30 metrics. |
| AWS X-Ray | $0.07 | 150 Traces Per Second, with a 5% sampling rate. |
| AWS Secrets Manager | $0.73 | 200 secrets, and 75 API calls per second. |
| AWS Lambda | $1.22 | 150 RPS, assuming 2000ms duration per request. |
| Amazon API Gateway | $0.93 | 150 RPS to the REST API type. |
| AWS Private Link | $0.01 | Interface VPC endpoint for API Gateway, 2AZs, 170GB in total processed per month. |
| Data Transfer | $0.00 | 170Gb transfer between AZs per month. |
| Total | $3.90 | Total. |
Assuming a 150 request per second load test for 4 hours, the price breakdown is estimated as follows:
| Service | Cost | Description |
|---|---|---|
| Amazon Aurora MySQL | $6.25 | 4x db.r5.large and 2x db.t3.medium database instances running continuously, 150 RPS. |
| Amazon RDS Proxy | $0.12 | Based on 2 vCPUs of db.r5.large, running continuously. |
| Amazon EC2 | $1.11 | 3x c5.large instances running continuously. |
| Amazon CloudWatch | $0.05 | CloudWatch dashboard + 30 metrics. |
| AWS X-Ray | $0.56 | 150 Traces Per Second, with a 5% sampling rate. |
| AWS Secrets Manager | $5.84 | 200 secrets, and 75 API calls per second. |
| AWS Lambda | $9.75 | 150 RPS, assuming 2000ms duration per request. |
| Amazon API Gateway | $7.43 | 150 RPS to the REST API type. |
| AWS Private Link | $0.09 | Interface VPC endpoint for API Gateway, 2AZs, 170GB in total processed per month. |
| Data Transfer | $0.02 | 170Gb transfer between AZs per month. |
| Total | $31.21 | Total. |
The above pricing examples (excluding the RDS Proxy cost) are based off the AWS Pricing Calculator, and derived from the monthly cost taken here.
Assuming the resources are deployed for a month, without a load test (all of the resources are running, except for the load test using Locust), the price breakdown is estimated as follows:
| Service | Cost | Description |
|---|---|---|
| Amazon Aurora MySQL | $968.94 | 4x db.r5.large and 2x db.t3.medium database instances running continuously. |
| Amazon RDS Proxy | $21.60 | Based on 2 vCPUs of db.r5.large, running continuously. |
| Amazon EC2 | $195.65 | 3x c5.large instances running continuously. |
| Amazon CloudWatch | $12.00 | CloudWatch dashboard + 30 metrics. |
| AWS Secrets Manager | $80.00 | 200 secrets. |
| AWS Private Link | $14.60 | Interface VPC endpoint for API Gateway, 2AZs. |
| Total | $1293.49 | Total. |
The above pricing example (excluding the RDS Proxy cost) is based off the AWS Pricing Calculator, and derived from the monthly cost taken here.
Contributions are more than welcome. Please read the code of conduct and the contributing guidelines.
This library is licensed under the MIT-0 License. See the LICENSE file. See also the THIRD-PARTY file for third-party notices.