Deployment Automation with Terraform, AWS and Docker

Local Development

This project is designed to support a scalable, secure, and efficient deployment of an API service using AWS services such as ECS, ALB, RDS, and various networking and security components.

Follow the below steps to run a local development environment.

1. Ensure you have the following installed:

• Docker Desktop

2. Clone the project, cd to it in Terminal/Command Prompt and run the following:

docker compose up

3. Browse the project at http://127.0.0.1:8000/api/health-check/

Creating Superuser

To create a superuser to access the Django admin follow these steps.

1. Run the below command and follow the in terminal instructions:

docker compose run --rm app sh -c "python manage.py createsuperuser"

2. Browse the Django admin at [http://127.0.0.1:8000/admin] and login.

Clearing Storage

To clear all storage (including the database) and start fresh:

docker compose down --volumes
docker compose up

Architecture Diagram

Architecture Diagram

Data Flow Summary

• Users send HTTPS requests to the ALB.

• ALB forwards requests to the ECS cluster.

• API services process requests, interact with the database, and store files in EFS if necessary.

• Logs and metrics are collected in CloudWatch.

• Admins use SSM to securely manage infrastructure.

• CI/CD pipelines deploy and update services via GitLab/GitHub.

AWS CLI Authentication

This project uses aws-vault to authenticate with the AWS CLI in the terminal.

To authenticate:

aws-vault exec PROFILE --duration=1h

Replace PROFILE with the name of the profile.

To list profiles, run:

aws-vault list

Task Exec

ECS Exec is used for manually running commands directly on the running containers.

To get shell access to the ecs task:

aws ecs execute-command --region REGION --cluster CLUSTER_NAME --task TASK_ID --container CONTAINER_NAME --interactive --command "/bin/sh"

Replace the following values in the above command:

• REGION: The AWS region where the ECS cluster is setup.
• CLUSTER_NAME: The name of the ECS cluster.
• TASK_ID: The ID of the running ECS task which you want to connect to.
• CONTAINER_NAME: The name of the container to run the command on.

Terraform Commands

Below is a list of how to run the common commands via Docker Compose.

Note: The below commands should be run from ther infra/ directory of the project, and after authenticating with aws-vault.

To run any Terraform command through Docker, use the syntax below:

docker compose run --rm terraform -chdir=TF_DIR COMMAND

Where TF_DIR is the directory containing the Terraform (setup or deploy) and COMMAND is the Terraform command (e.g. plan).

Get outputs from the setup Terraform

docker compose run --rm terraform -chdir=setup output

The output name must be specified if sensitive = true in the output definition, like this:

docker compose run --rm terraform -chdir=setup output cd_user_access_key_secret

GitHub Actions Variables

This section lists the GitHub Actions variables which need to be configured on the GitHub project.

Note: This is only applicable if using GitHub Actions, if you're using GitLab, see GitLab CI/CD Variables below.

If using GitHub Actions, variables are set as either Variables (clear text and readable) or Secrets (values hidden in logs).

Variables:

• AWS_ACCESS_KEY_ID: Access key for the CD AWS IAM user that is created by Terraform and output as cd_user_access_key_id.
• AWS_ACCOUNT_ID: AWS Account ID taken from AWS directly.
• DOCKERHUB_USER: Username for Docker Hub for avoiding Docker Pull rate limit issues.
• ECR_REPO_APP: URL for the Docker repo containing the app image output by Terraform as ecr_repo_app.
• ECR_REPO_PROXY: URL for the Docker repo containing the proxy image output by Terraform as ecr_repo_proxy.

Secrets:

• AWS_SECRET_ACCESS_KEY: Secret key for AWS_ACCESS_KEY_ID set in variables, output by Terraform as cd_user_access_key_secret.
• DOCKERHUB_TOKEN: Token created in DOCKERHUB_USER in Docker Hub.
• TF_VAR_DB_PASSWORD: Password for the RDS database (make something up).
• TF_VAR_DJANGO_SECRET_KEY: Secret key for the Django app (make something up).

Section Notes and Resources

Software Requirements

Checking Each Dependency

Check docker is running:

docker --version

Check aws-vault installed:

aws-vault --version

Check AWS CLI:

aws --version

Check AWS CLI Systems Manager:

session-manager-plugin

Check docker compose:

docker compose --version

Configure Git:

git config --global user.email email@example.com
git config --global user.name "User Name" 
git config --global push.autoSetupRemote true