diff --git a/README.md b/README.md index d5988eb89..0e0253e36 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ jobs: 1. [Certificates](#certificate-inputs) 1. [Load Balancer](#load-balancer-inputs) 1. [EFS](#efs-inputs) -1. [RDS](#rds-inputs) +1. [Amazon Aurora Inputs](#aurora-inputs) 1. [Docker](#docker-inputs) The following inputs can be used as `step.with` keys @@ -192,24 +192,24 @@ The following inputs can be used as `step.with` keys

-#### **RDS Inputs** +#### **Aurora Inputs** | Name | Type | Description | |------------------|---------|------------------------------------| -| `aws_postgres_enable` | Boolean | Set to `true` to enable a postgres database. | -| `aws_postgres_engine` | String | Which Database engine to use. Default is `aurora-postgresql`.| -| `aws_postgres_engine_version` | String | Specify Postgres version. More information [here](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Updates.20180305.html). Default is `11.17`. | -| `aws_postgres_database_group_family` | String | Specify aws database group family. Default is `aurora-postgresql11`. See [this](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/rds/create-db-parameter-group.html).| -| `aws_postgres_instance_class` | String | Define the size of the instances in the DB cluster. Default is `db.t3.medium`. | -| `aws_postgres_security_group_name` | String | The name of the Postgres security group. Defaults to `SG for ${aws_resource_identifier} - PG`. | -| `aws_postgres_subnets` | String | Specify which subnets to use as a list of strings. Example: `i-1234,i-5678,i-9101`. | -| `aws_postgres_cluster_name` | String | Specify a cluster name. Will be created if it does not exist. Defaults to `aws_resource_identifier`. | -| `aws_postgres_database_name` | String | Specify a database name. Will be created if it does not exist. Defaults to `aws_resource_identifier`. | -| `aws_postgres_database_port` | String | Specify a listening port for the database. Default is `5432`.| -| `aws_postgres_restore_snapshot` | String | Restore a snapshot to the DB. Should be set only once. Changes in this value will destroy and recreate the database completely. | -| `aws_postgres_snapshot_name` | String | Specify a database name. Will be created if it does not exist. Won't overwrite. | -| `aws_postgres_snapshot_overwrite` | Boolean | Set to true to overwrite the snapshot. | -| `aws_postgres_database_protection` | Boolean | Protects the database from deletion. Default is `false`.| -| `aws_postgres_database_final_snapshot` | Boolean | Creates a snapshot before deletion. If a string is passed, it will be used as snapsthot name. Defaults to `false`.| +| `aws_aurora_enable` | Boolean | Set to `true` to enable an [Aurora database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_AuroraOverview.html). (Postgres or MySQL). | +| `aws_aurora_engine` | String | Which Database engine to use. Default is `aurora-postgresql`.| +| `aws_aurora_engine_version` | String | Specify database version. More information [Postgres](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Updates.20180305.html) or [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/Welcome.html). Default is `11.17`. (Postgres) | +| `aws_aurora_database_group_family` | String | Specify aws database group family. Default is `aurora-postgresql11`. See [this](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/rds/create-db-parameter-group.html).| +| `aws_aurora_instance_class` | String | Define the size of the instances in the DB cluster. Default is `db.t3.medium`. | +| `aws_aurora_security_group_name` | String | The name of the database security group. Defaults to `SG for ${aws_resource_identifier} - Aurora`. | +| `aws_aurora_subnets` | String | Specify which subnets to use as a list of strings. Example: `i-1234,i-5678,i-9101`. | +| `aws_aurora_cluster_name` | String | Specify a cluster name. Will be created if it does not exist. Defaults to `aws_resource_identifier`. | +| `aws_aurora_database_name` | String | Specify a database name. Will be created if it does not exist. Defaults to `aws_resource_identifier`. | +| `aws_aurora_database_port` | String | Specify a listening port for the database. Default is `5432`.| +| `aws_aurora_restore_snapshot` | String | Restore a snapshot to the DB. Should be set only once. Changes in this value will destroy and recreate the database completely. | +| `aws_aurora_snapshot_name` | String | Specify a database name. Will be created if it does not exist. Won't overwrite. | +| `aws_aurora_snapshot_overwrite` | Boolean | Set to true to overwrite the snapshot. | +| `aws_aurora_database_protection` | Boolean | Protects the database from deletion. Default is `false`.| +| `aws_aurora_database_final_snapshot` | Boolean | Creates a snapshot before deletion. If a string is passed, it will be used as snapsthot name. Defaults to `false`.|

@@ -284,9 +284,11 @@ An example EFS Zone mapping; } ``` -## Adding external Postgres database (AWS RDS) +## Adding external Aurora database (AWS RDS) -If `aws_postgres_enable` is set to `true`, this action will deploy an RDS cluster for Postgres. +If `aws_aurora_enable` is set to `true`, this action will deploy a RDS Aurora cluster using Postgres as a default. + +See [this Terraform provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster) and [this other AWS doc](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/rds/create-db-parameter-group.html) to set up most of the variables. ### Environment variables The following environment variables are added to the `.env` file in your app's `docker-compose.yaml` file. @@ -304,17 +306,18 @@ services: The available environment variables are: | Variable | Description | |----------|-------------| -| `POSTGRES_CLUSTER_ENDPOINT` (and `PGHOST`) | Writer endpoint for the cluster | -| `POSTGRES_CLUSTER_PORT` (and `PGPORT`) | The database port | -| `POSTGRES_CLUSTER_MASTER_PASSWORD` (and `PG_PASSWORD`) | database root password | -| `POSTGRES_CLUSTER_MASTER_USERNAME` (and `PG_USER`) | The database master username | -| `POSTGRES_CLUSTER_DATABASE_NAME` (and `PGDATABASE`) | Name for an automatically created database on cluster creation | -| `POSTGRES_CLUSTER_ARN` | Amazon Resource Name (ARN) of cluster | -| `POSTGRES_CLUSTER_ID` | The RDS Cluster Identifier | -| `POSTGRES_CLUSTER_RESOURCE_ID` | The RDS Cluster Resource ID | -| `POSTGRES_CLUSTER_READER_ENDPOINT` | A read-only endpoint for the cluster, automatically load-balanced across replicas | -| `POSTGRES_CLUSTER_ENGINE_VERSION_ACTUAL` | The running version of the cluster database | -| `POSTGRES_CLUSTER_HOSTED_ZONE_ID`| The Route53 Hosted Zone ID of the endpoint | +| `AURORA_CLUSTER_ENGINE` (and `DBA_ENGINE`) | Engine name - ( mysql/postgres ) | +| `AURORA_CLUSTER_ENDPOINT` (and `DBA_HOST`) | Writer endpoint for the cluster | +| `AURORA_CLUSTER_PORT` (and `DBA_PORT`) | The database port | +| `AURORA_CLUSTER_MASTER_PASSWORD` (and `DBA_PASSWORD`) | database root password | +| `AURORA_CLUSTER_MASTER_USERNAME` (and `DBA_USER`) | The database master username | +| `AURORA_CLUSTER_DATABASE_NAME` (and `DBA_NAME`) | Name for an automatically created database on cluster creation | +| `AURORA_CLUSTER_ARN` | Amazon Resource Name (ARN) of cluster | +| `AURORA_CLUSTER_ID` | The RDS Cluster Identifier | +| `AURORA_CLUSTER_RESOURCE_ID` | The RDS Cluster Resource ID | +| `AURORA_CLUSTER_READER_ENDPOINT` | A read-only endpoint for the cluster, automatically load-balanced across replicas | +| `AURORA_CLUSTER_ENGINE_VERSION_ACTUAL` | The running version of the cluster database | +| `AURORA_CLUSTER_HOSTED_ZONE_ID`| The Route53 Hosted Zone ID of the endpoint | ### AWS Root Certs The AWS root certificate is downloaded and accessible via the `rds-combined-ca-bundle.pem` file in root of your app repo/directory. @@ -327,11 +330,11 @@ const { Client } = require('pg') // set up client const client = new Client({ - host: process.env.PGHOST, - port: process.env.PGPORT, - user: process.env.PG_USER, - password: process.env.PG_PASSWORD, - database: process.env.PGDATABASE, + host: process.env.DBA_HOST, + port: process.env.DBA_PORT, + user: process.env.DBA_USER, + password: process.env.DBA_PASSWORD, + database: process.env.DBA_NAME, ssl: { ca: fs.readFileSync('rds-combined-ca-bundle.pem').toString() } @@ -345,12 +348,12 @@ await client.end(); console.log(`Hello SQL timestamp: ${result.rows[0].now}`); ``` -### Postgres Infrastructure and Cluster Details +### Aurora Infrastructure and Cluster Details Specifically, the following resources will be created: - AWS Security Group - AWS Security Group Rule - Allows access to the cluster's db port: `5432` -- AWS RDS Aurora Postgres - - Includes a single database (set by the input: `aws_postgres_database_name`. defaults to `root`) +- AWS RDS Aurora + - Includes a single database (set by the input: `aws_aurora_database_name`. defaults to `root`) Additional details about the cluster that's created: - Automated backups (7 Days) @@ -359,7 +362,7 @@ Additional details about the cluster that's created: - Monitoring enabled - Sends logs to AWS Cloudwatch -> _**For more details**, see [operations/deployment/terraform/postgres.tf](operations/deployment/terraform/postgres.tf)_ +> _**For more details**, see [link-to-be-updated](operations/deployment/terraform/postgres.tf)_ ## Made with BitOps [BitOps](https://bitops.sh) allows you to define Infrastructure-as-Code for multiple tools in a central place. This action uses a BitOps [Operations Repository](https://bitops.sh/operations-repo-structure/) to set up the necessary Terraform and Ansible to create infrastructure and deploy to it. diff --git a/action.yaml b/action.yaml index 99375eb70..c8f8fab86 100644 --- a/action.yaml +++ b/action.yaml @@ -253,49 +253,49 @@ inputs: required: false # AWS RDS - aws_postgres_enable: + aws_aurora_enable: description: 'Set to "true" to enable a postgres database' required: false - aws_postgres_engine: + aws_aurora_engine: description: 'Which Database engine to use' required: false - aws_postgres_engine_version: + aws_aurora_engine_version: description: 'Specify Postgres version' required: false - aws_postgres_database_group_family: + aws_aurora_database_group_family: description: 'Postgres database group family' required: false - aws_postgres_instance_class: + aws_aurora_instance_class: description: 'Define the size of the instances in the DB cluster' required: false - aws_postgres_security_group_name: + aws_aurora_security_group_name: description: 'The name of the Postgres security group' required: false - aws_postgres_subnets: + aws_aurora_subnets: description: 'Specify which subnets to use as a list of strings. Example: `i-1234,i-5678,i-9101`' required: false - aws_postgres_cluster_name: + aws_aurora_cluster_name: description: 'Specify a cluster name. Will be created if it does not exist' required: false - aws_postgres_database_name: + aws_aurora_database_name: description: 'Specify a database name. Will be created if it does not exist' required: false - aws_postgres_database_port: + aws_aurora_database_port: description: 'Postgres database port' required: false - aws_postgres_restore_snapshot: + aws_aurora_restore_snapshot: description: 'Restore a snapshot to the DB. Should be used only once. Changes in this value will destroy and recreate the database completely.' required: false - aws_postgres_snapshot_name: + aws_aurora_snapshot_name: description: 'Takes a snapshot of the cluster using that name. If none definded, no snapshot will be made. If snap already exists, no new one will be created.' required: false - aws_postgres_snapshot_overwrite: + aws_aurora_snapshot_overwrite: description: 'If the snapshot name is the same as an existing one, will destroy and create a new one.' required: false - aws_postgres_database_protection: + aws_aurora_database_protection: description: 'Protects the database from deletion.' required: false - aws_postgres_database_final_snapshot: + aws_aurora_database_final_snapshot: description: 'Generates a snapshot of the database before deletion.' required: false @@ -500,21 +500,21 @@ runs: AWS_EFS_EC2_MOUNT_POINT: ${{ inputs.aws_efs_ec2_mount_point }} # AWS RDS - AWS_POSTGRES_ENABLE: ${{ inputs.aws_postgres_enable }} - AWS_POSTGRES_ENGINE: ${{ inputs.aws_postgres_engine }} - AWS_POSTGRES_ENGINE_VERSION: ${{ inputs.aws_postgres_engine_version }} - AWS_POSTGRES_DATABASE_GROUP_FAMILY: ${{ inputs.aws_postgres_database_group_family }} - AWS_POSTGRES_INSTANCE_CLASS: ${{ inputs.aws_postgres_instance_class }} - AWS_POSTGRES_SECURITY_GROUP_NAME: ${{ inputs.aws_postgres_security_group_name }} - AWS_POSTGRES_SUBNETS: ${{ inputs.aws_postgres_subnets }} - AWS_POSTGRES_CLUSTER_NAME: ${{ inputs.aws_postgres_cluster_name }} - AWS_POSTGRES_DATABASE_NAME: ${{ inputs.aws_postgres_database_name }} - AWS_POSTGRES_DATABASE_PORT: ${{ inputs.aws_postgres_database_port}} - AWS_POSTGRES_RESTORE_SNAPSHOT: ${{ inputs.aws_postgres_restore_snapshot }} - AWS_POSTGRES_SNAPSHOT_NAME: ${{ inputs.aws_postgres_snapshot_name }} - AWS_POSTGRES_SNAPSHOT_OVERWRITE: ${{ inputs.aws_postgres_snapshot_overwrite }} - AWS_POSTGRES_DATABASE_PROTECTION: ${{ inputs.aws_postgres_database_protection }} - AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT: ${{ inputs.aws_postgres_database_final_snapshot }} + AWS_AURORA_ENABLE: ${{ inputs.aws_aurora_enable }} + AWS_AURORA_ENGINE: ${{ inputs.aws_aurora_engine }} + AWS_AURORA_ENGINE_VERSION: ${{ inputs.aws_aurora_engine_version }} + AWS_AURORA_DATABASE_GROUP_FAMILY: ${{ inputs.aws_aurora_database_group_family }} + AWS_AURORA_INSTANCE_CLASS: ${{ inputs.aws_aurora_instance_class }} + AWS_AURORA_SECURITY_GROUP_NAME: ${{ inputs.aws_aurora_security_group_name }} + AWS_AURORA_SUBNETS: ${{ inputs.aws_aurora_subnets }} + AWS_AURORA_CLUSTER_NAME: ${{ inputs.aws_aurora_cluster_name }} + AWS_AURORA_DATABASE_NAME: ${{ inputs.aws_aurora_database_name }} + AWS_AURORA_DATABASE_PORT: ${{ inputs.aws_aurora_database_port}} + AWS_AURORA_RESTORE_SNAPSHOT: ${{ inputs.aws_aurora_restore_snapshot }} + AWS_AURORA_SNAPSHOT_NAME: ${{ inputs.aws_aurora_snapshot_name }} + AWS_AURORA_SNAPSHOT_OVERWRITE: ${{ inputs.aws_aurora_snapshot_overwrite }} + AWS_AURORA_DATABASE_PROTECTION: ${{ inputs.aws_aurora_database_protection }} + AWS_AURORA_DATABASE_FINAL_SNAPSHOT: ${{ inputs.aws_aurora_database_final_snapshot }} # Docker DOCKER_INSTALL: ${{ inputs.docker_install }} diff --git a/operations/_scripts/deploy/deploy.sh b/operations/_scripts/deploy/deploy.sh index 1e721e0fc..dc94d3571 100755 --- a/operations/_scripts/deploy/deploy.sh +++ b/operations/_scripts/deploy/deploy.sh @@ -76,7 +76,7 @@ cat $GITHUB_ACTION_PATH/operations/deployment/bitops.config.yaml ## Ensuring bucket get's destroyed only if everything is set to be destroyed if [[ $(alpha_only "$TF_STATE_BUCKET_DESTROY") == true ]] && ! [[ $(alpha_only "$TF_STACK_DESTROY") == true ]] ; then - if [[ $(alpha_only "$AWS_POSTGRES_ENABLE") == true ]] || + if [[ $(alpha_only "$AWS_AURORA_ENABLE") == true ]] || [[ $(alpha_only "$AWS_EFS_ENABLE") == true ]] || [[ $(alpha_only "$AWS_EC2_INSTANCE_CREATE") == true ]] || [[ $(alpha_only "$AWS_EKS_CREATE") == true ]]; then diff --git a/operations/_scripts/generate/generate_bitops_config.sh b/operations/_scripts/generate/generate_bitops_config.sh index 04ba65d42..0f299738f 100755 --- a/operations/_scripts/generate/generate_bitops_config.sh +++ b/operations/_scripts/generate/generate_bitops_config.sh @@ -120,7 +120,7 @@ bitops: create_bitops_terraform_config aws false targets fi else - if [[ $(alpha_only "$AWS_EC2_INSTANCE_CREATE") == true ]] || [[ $(alpha_only "$AWS_EFS_CREATE") == true ]] || [[ "$AWS_POSTGRES_ENABLE" != "" ]]; then + if [[ $(alpha_only "$AWS_EC2_INSTANCE_CREATE") == true ]] || [[ $(alpha_only "$AWS_EFS_CREATE") == true ]] || [[ "$AWS_AURORA_ENABLE" != "" ]]; then add_terraform_module aws create_bitops_terraform_config aws true targets fi diff --git a/operations/_scripts/generate/generate_vars_terraform.sh b/operations/_scripts/generate/generate_vars_terraform.sh index 069acbb67..16f7d3815 100644 --- a/operations/_scripts/generate/generate_vars_terraform.sh +++ b/operations/_scripts/generate/generate_vars_terraform.sh @@ -79,18 +79,18 @@ else aws_r53_sub_domain_name="aws_r53_sub_domain_name = \"${GITHUB_IDENTIFIER}\"" fi -aws_postgres_subnets= -if [ -n "${AWS_POSTGRES_SUBNETS}" ]; then - aws_postgres_subnets="aws_postgres_subnets = \"$(comma_str_to_tf_array $AWS_POSTGRES_SUBNETS)\"" +aws_aurora_subnets= +if [ -n "${AWS_AURORA_SUBNETS}" ]; then + aws_aurora_subnets="aws_aurora_subnets = \"$(comma_str_to_tf_array $AWS_AURORA_SUBNETS)\"" fi # If the name is true, set it up to be the GH ID - If not, if it's not false, it's the snap name. -if [ -n "$AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT" ];then - if [[ $(alpha_only "$AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT") == "true" ]]; then - aws_postgres_database_final_snapshot="aws_postgres_database_final_snapshot = \"${GITHUB_IDENTIFIER}\"" +if [ -n "$AWS_AURORA_DATABASE_FINAL_SNAPSHOT" ];then + if [[ $(alpha_only "$AWS_AURORA_DATABASE_FINAL_SNAPSHOT") == "true" ]]; then + aws_aurora_database_final_snapshot="aws_aurora_database_final_snapshot = \"${GITHUB_IDENTIFIER}\"" else - if [[ $(alpha_only "$AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT") != "false" ]]; then - aws_postgres_database_final_snapshot="aws_postgres_database_final_snapshot = \"${AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT}\"" + if [[ $(alpha_only "$AWS_AURORA_DATABASE_FINAL_SNAPSHOT") != "false" ]]; then + aws_aurora_database_final_snapshot="aws_aurora_database_final_snapshot = \"${AWS_AURORA_DATABASE_FINAL_SNAPSHOT}\"" fi fi fi @@ -174,22 +174,22 @@ if [[ $(alpha_only "$AWS_EFS_ENABLE") == true ]]; then fi #-- RDS --# -if [[ $(alpha_only "$AWS_POSTGRES_ENABLE") == true ]]; then - aws_postgres_enable=$(generate_var aws_postgres_enable $AWS_POSTGRES_ENABLE) - aws_postgres_engine=$(generate_var aws_postgres_engine $AWS_POSTGRES_ENGINE) - aws_postgres_engine_version=$(generate_var aws_postgres_engine_version $AWS_POSTGRES_ENGINE_VERSION) - aws_postgres_database_group_family=$(generate_var aws_postgres_database_group_family $AWS_POSTGRES_DATABASE_GROUP_FAMILY) - aws_postgres_instance_class=$(generate_var aws_postgres_instance_class $AWS_POSTGRES_INSTANCE_CLASS) - aws_postgres_security_group_name=$(generate_var aws_postgres_security_group_name $AWS_POSTGRES_SECURITY_GROUP_NAME ) - # aws_postgres_subnets=$(generate_var aws_postgres_subnets $AWS_POSTGRES_SUBNETS) - Special case - aws_postgres_cluster_name=$(generate_var aws_postgres_cluster_name $AWS_POSTGRES_CLUSTER_NAME) - aws_postgres_database_name=$(generate_var aws_postgres_database_name $AWS_POSTGRES_DATABASE_NAME) - aws_postgres_database_port=$(generate_var aws_postgres_database_port $AWS_POSTGRES_DATABASE_PORT) - aws_postgres_restore_snapshot=$(generate_var aws_postgres_restore_snapshot $AWS_POSTGRES_RESTORE_SNAPSHOT) - aws_postgres_snapshot_name=$(generate_var aws_postgres_snapshot_name $AWS_POSTGRES_SNAPSHOT_NAME) - aws_postgres_snapshot_overwrite=$(generate_var aws_postgres_snapshot_overwrite $AWS_POSTGRES_SNAPSHOT_OVERWRITE) - aws_postgres_database_protection=$(generate_var aws_postgres_database_protection $AWS_POSTGRES_DATABASE_PROTECTION ) - # aws_postgres_database_final_snapshot=$(generate_var aws_postgres_database_final_snapshot $AWS_POSTGRES_DATABASE_FINAL_SNAPSHOT ) - Special case +if [[ $(alpha_only "$AWS_AURORA_ENABLE") == true ]]; then + aws_aurora_enable=$(generate_var aws_aurora_enable $AWS_AURORA_ENABLE) + aws_aurora_engine=$(generate_var aws_aurora_engine $AWS_AURORA_ENGINE) + aws_aurora_engine_version=$(generate_var aws_aurora_engine_version $AWS_AURORA_ENGINE_VERSION) + aws_aurora_database_group_family=$(generate_var aws_aurora_database_group_family $AWS_AURORA_DATABASE_GROUP_FAMILY) + aws_aurora_instance_class=$(generate_var aws_aurora_instance_class $AWS_AURORA_INSTANCE_CLASS) + aws_aurora_security_group_name=$(generate_var aws_aurora_security_group_name $AWS_AURORA_SECURITY_GROUP_NAME ) + # aws_aurora_subnets=$(generate_var aws_aurora_subnets $AWS_AURORA_SUBNETS) - Special case + aws_aurora_cluster_name=$(generate_var aws_aurora_cluster_name $AWS_AURORA_CLUSTER_NAME) + aws_aurora_database_name=$(generate_var aws_aurora_database_name $AWS_AURORA_DATABASE_NAME) + aws_aurora_database_port=$(generate_var aws_aurora_database_port $AWS_AURORA_DATABASE_PORT) + aws_aurora_restore_snapshot=$(generate_var aws_aurora_restore_snapshot $AWS_AURORA_RESTORE_SNAPSHOT) + aws_aurora_snapshot_name=$(generate_var aws_aurora_snapshot_name $AWS_AURORA_SNAPSHOT_NAME) + aws_aurora_snapshot_overwrite=$(generate_var aws_aurora_snapshot_overwrite $AWS_AURORA_SNAPSHOT_OVERWRITE) + aws_aurora_database_protection=$(generate_var aws_aurora_database_protection $AWS_AURORA_DATABASE_PROTECTION ) + # aws_aurora_database_final_snapshot=$(generate_var aws_aurora_database_final_snapshot $AWS_AURORA_DATABASE_FINAL_SNAPSHOT ) - Special case fi #-- EKS Cluster --# @@ -300,21 +300,21 @@ $aws_efs_mount_target $aws_efs_ec2_mount_point #-- RDS --# -$aws_postgres_enable -$aws_postgres_engine -$aws_postgres_engine_version -$aws_postgres_database_group_family -$aws_postgres_instance_class -$aws_postgres_security_group_name -$aws_postgres_subnets -$aws_postgres_cluster_name -$aws_postgres_database_name -$aws_postgres_database_port -$aws_postgres_restore_snapshot -$aws_postgres_snapshot_name -$aws_postgres_snapshot_overwrite -$aws_postgres_database_protection -$aws_postgres_database_final_snapshot +$aws_aurora_enable +$aws_aurora_engine +$aws_aurora_engine_version +$aws_aurora_database_group_family +$aws_aurora_instance_class +$aws_aurora_security_group_name +$aws_aurora_subnets +$aws_aurora_cluster_name +$aws_aurora_database_name +$aws_aurora_database_port +$aws_aurora_restore_snapshot +$aws_aurora_snapshot_name +$aws_aurora_snapshot_overwrite +$aws_aurora_database_protection +$aws_aurora_database_final_snapshot #-- EKS --# $aws_eks_create diff --git a/operations/deployment/terraform/aws/aws_variables.tf b/operations/deployment/terraform/aws/aws_variables.tf index 69bc0b987..5eb966f2e 100644 --- a/operations/deployment/terraform/aws/aws_variables.tf +++ b/operations/deployment/terraform/aws/aws_variables.tf @@ -284,77 +284,77 @@ variable "aws_efs_ec2_mount_point" { # AWS RDS -variable "aws_postgres_enable" { +variable "aws_aurora_enable" { type = bool description = "deploy a postgres database" default = false } -variable "aws_postgres_engine" { +variable "aws_aurora_engine" { type = string description = "The engine to use for postgres. Defaults to `aurora-postgresql`. For more details, see: https://aws.amazon.com/rds/, https://registry.terraform.io/modules/terraform-aws-modules/rds-aurora/aws/latest?tab=inputs" default = "aurora-postgresql" } -variable "aws_postgres_engine_version" { +variable "aws_aurora_engine_version" { type = string description = "The version of the engine to use for postgres. Defaults to `11.17`." default = "11.17" } -variable "aws_postgres_database_group_family" { +variable "aws_aurora_database_group_family" { type = string default = "aurora-postgresql11" description = "postgres group family" } -variable "aws_postgres_instance_class" { +variable "aws_aurora_instance_class" { type = string description = "The size of the db instances. For more details, see: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.DBInstanceClass.html, https://registry.terraform.io/modules/terraform-aws-modules/rds-aurora/aws/latest?tab=inputs" default = "db.t3.medium" } -variable "aws_postgres_security_group_name" { +variable "aws_aurora_security_group_name" { type = string default = "" description = "Name of the security group to use for postgres" } -variable "aws_postgres_subnets" { +variable "aws_aurora_subnets" { type = list(string) description = "The list of subnet ids to use for postgres. For more details, see: https://registry.terraform.io/modules/terraform-aws-modules/rds-aurora/aws/latest?tab=inputs" default = [] } -variable "aws_postgres_cluster_name" { +variable "aws_aurora_cluster_name" { type = string description = "The name of the cluster. will be created if it does not exist." default = "" } -variable "aws_postgres_database_name" { +variable "aws_aurora_database_name" { type = string description = "The name of the database. will be created if it does not exist." default = "root" } -variable "aws_postgres_database_port" { +variable "aws_aurora_database_port" { type = string default = "5432" description = "database port" } -variable "aws_postgres_restore_snapshot" { +variable "aws_aurora_restore_snapshot" { type = string default = "" description = "Restore an initial snapshot of the DB." } -variable "aws_postgres_snapshot_name" { +variable "aws_aurora_snapshot_name" { type = string default = "" description = "Takes a snapshot of the DB." } -variable "aws_postgres_snapshot_overwrite" { +variable "aws_aurora_snapshot_overwrite" { type = bool default = false description = "Overwrites snapshot." } -variable "aws_postgres_database_protection" { +variable "aws_aurora_database_protection" { type = bool default = false description = "Protects the database from deletion." } -variable "aws_postgres_database_final_snapshot" { +variable "aws_aurora_database_final_snapshot" { type = string default = "" description = "Generates a snapshot of the database before deletion." diff --git a/operations/deployment/terraform/aws/bitops.after-deploy.d/merge-tf-env.sh b/operations/deployment/terraform/aws/bitops.after-deploy.d/merge-tf-env.sh index 0f58220ac..216415bf3 100755 --- a/operations/deployment/terraform/aws/bitops.after-deploy.d/merge-tf-env.sh +++ b/operations/deployment/terraform/aws/bitops.after-deploy.d/merge-tf-env.sh @@ -9,7 +9,7 @@ ANSIBLE_DIR=ansible/clone_repo TERRAFORM_PATH=terraform/aws # Merging order -order=ec2,efs,postgres,repo,ghv,ghs,aws +order=ec2,efs,aurora,repo,ghv,ghs,aws # Ansible dotenv file -> The final destination of all ENV_OUT_FILE="${BITOPS_ENVROOT}/${ANSIBLE_DIR}/app.env" @@ -20,8 +20,8 @@ ENV_EC2_FILE="${BITOPS_ENVROOT}/${TERRAFORM_PATH}/ec2.env" # EFS dotenv file ENV_EFS_FILE="${BITOPS_ENVROOT}/${TERRAFORM_PATH}/efs.env" -# Postgres dotenv file -ENV_POSTGRES_FILE="${BITOPS_ENVROOT}/${TERRAFORM_PATH}/postgres.env" +# Aurora dotenv file +ENV_AURORA_FILE="${BITOPS_ENVROOT}/${TERRAFORM_PATH}/aurora.env" # Repo env file ENV_REPO_FILE="${BITOPS_ENVROOT}/env-files/repo.env" @@ -80,9 +80,9 @@ function process { # Code to be executed for option6 merge $ENV_EFS_FILE "EFS" ;; - postgres) + aurora) # Code to be executed for option6 - merge $ENV_POSTGRES_FILE "Postgres" + merge $ENV_AURORA_FILE "Aurora" ;; *) # Code to be executed if no matching option is found diff --git a/operations/deployment/terraform/aws/bitovi_main.tf b/operations/deployment/terraform/aws/bitovi_main.tf index e1715b967..b1ed8311d 100644 --- a/operations/deployment/terraform/aws/bitovi_main.tf +++ b/operations/deployment/terraform/aws/bitovi_main.tf @@ -112,7 +112,7 @@ module "ec2_efs" { aws_efs_ec2_mount_point = var.aws_efs_ec2_mount_point # Other ha_zone_mapping = local.ha_zone_mapping - ec2_zone_mapping = try(local.ec2_zone_mapping,"") + ec2_zone_mapping = local.ec2_zone_mapping # Docker docker_efs_mount_target = var.docker_efs_mount_target # Data inputs @@ -131,22 +131,22 @@ module "ec2_efs" { module "aurora_rds" { source = "../modules/aws/aurora" - count = var.aws_postgres_enable ? 1 : 0 + count = var.aws_aurora_enable ? 1 : 0 # RDS - aws_postgres_engine = var.aws_postgres_engine - aws_postgres_engine_version = var.aws_postgres_engine_version - aws_postgres_database_group_family = var.aws_postgres_database_group_family - aws_postgres_instance_class = var.aws_postgres_instance_class - aws_postgres_security_group_name = var.aws_postgres_security_group_name - aws_postgres_subnets = var.aws_postgres_subnets - aws_postgres_cluster_name = var.aws_postgres_cluster_name - aws_postgres_database_name = var.aws_postgres_database_name - aws_postgres_database_port = var.aws_postgres_database_port - aws_postgres_restore_snapshot = var.aws_postgres_restore_snapshot - aws_postgres_snapshot_name = var.aws_postgres_snapshot_name - aws_postgres_snapshot_overwrite = var.aws_postgres_snapshot_overwrite - aws_postgres_database_protection = var.aws_postgres_database_protection - aws_postgres_database_final_snapshot = var.aws_postgres_database_final_snapshot + aws_aurora_engine = var.aws_aurora_engine + aws_aurora_engine_version = var.aws_aurora_engine_version + aws_aurora_database_group_family = var.aws_aurora_database_group_family + aws_aurora_instance_class = var.aws_aurora_instance_class + aws_aurora_security_group_name = var.aws_aurora_security_group_name + aws_aurora_subnets = var.aws_aurora_subnets + aws_aurora_cluster_name = var.aws_aurora_cluster_name + aws_aurora_database_name = var.aws_aurora_database_name + aws_aurora_database_port = var.aws_aurora_database_port + aws_aurora_restore_snapshot = var.aws_aurora_restore_snapshot + aws_aurora_snapshot_name = var.aws_aurora_snapshot_name + aws_aurora_snapshot_overwrite = var.aws_aurora_snapshot_overwrite + aws_aurora_database_protection = var.aws_aurora_database_protection + aws_aurora_database_final_snapshot = var.aws_aurora_database_final_snapshot # Data inputs aws_vpc_default_id = data.aws_vpc.default.id aws_subnets_vpc_subnets_ids = data.aws_subnets.vpc_subnets.ids diff --git a/operations/deployment/terraform/modules/aws/aurora/aws_aurora.tf b/operations/deployment/terraform/modules/aws/aurora/aws_aurora.tf index 85fa2c312..302d947d2 100644 --- a/operations/deployment/terraform/modules/aws/aurora/aws_aurora.tf +++ b/operations/deployment/terraform/modules/aws/aurora/aws_aurora.tf @@ -1,6 +1,6 @@ -resource "aws_security_group" "pg_security_group" { - name = var.aws_postgres_security_group_name != "" ? var.aws_postgres_security_group_name : "SG for ${var.aws_resource_identifier} - PG" - description = "SG for ${var.aws_resource_identifier} - PG" +resource "aws_security_group" "aurora_security_group" { + name = var.aws_aurora_security_group_name != "" ? var.aws_aurora_security_group_name : "SG for ${var.aws_resource_identifier} - Aurora" + description = "SG for ${var.aws_resource_identifier} - Aurora" egress { from_port = 0 to_port = 0 @@ -8,63 +8,63 @@ resource "aws_security_group" "pg_security_group" { cidr_blocks = ["0.0.0.0/0"] } tags = { - Name = "${var.aws_resource_identifier}-pg" + Name = "${var.aws_resource_identifier}-aurora" } } -resource "aws_security_group_rule" "ingress_postgres" { +resource "aws_security_group_rule" "ingress_aurora" { type = "ingress" - description = "${var.aws_resource_identifier} - pgPort" - from_port = tonumber(var.aws_postgres_database_port) - to_port = tonumber(var.aws_postgres_database_port) + description = "${var.aws_resource_identifier} - Aurora Port" + from_port = tonumber(var.aws_aurora_database_port) + to_port = tonumber(var.aws_aurora_database_port) protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] - security_group_id = aws_security_group.pg_security_group.id + security_group_id = aws_security_group.aurora_security_group.id } -module "rds_cluster" { +module "aurora_cluster" { source = "terraform-aws-modules/rds-aurora/aws" version = "v7.7.1" - name = var.aws_postgres_cluster_name != "" ? var.aws_postgres_cluster_name : var.aws_resource_identifier + name = var.aws_aurora_cluster_name != "" ? var.aws_aurora_cluster_name : var.aws_resource_identifier - engine = var.aws_postgres_engine - engine_version = var.aws_postgres_engine_version - instance_class = var.aws_postgres_instance_class + engine = var.aws_aurora_engine + engine_version = var.aws_aurora_engine_version + instance_class = var.aws_aurora_instance_class instances = { 1 = { - instance_class = var.aws_postgres_instance_class + instance_class = var.aws_aurora_instance_class } } # Todo: handle vpc/networking explicitly # vpc_id = var.vpc_id # allowed_cidr_blocks = [var.vpc_cidr] - subnets = var.aws_postgres_subnets == null || length(var.aws_postgres_subnets) == 0 ? var.aws_subnets_vpc_subnets_ids : var.aws_postgres_subnets + subnets = var.aws_aurora_subnets == null || length(var.aws_aurora_subnets) == 0 ? var.aws_subnets_vpc_subnets_ids : var.aws_aurora_subnets - database_name = var.aws_postgres_database_name - port = var.aws_postgres_database_port - deletion_protection = var.aws_postgres_database_protection + database_name = var.aws_aurora_database_name + port = var.aws_aurora_database_port + deletion_protection = var.aws_aurora_database_protection storage_encrypted = true monitoring_interval = 60 create_db_subnet_group = true - db_subnet_group_name = "${var.aws_resource_identifier}-pg" + db_subnet_group_name = "${var.aws_resource_identifier}-aurora" create_security_group = false - vpc_security_group_ids = [aws_security_group.pg_security_group.id] + vpc_security_group_ids = [aws_security_group.aurora_security_group.id] # TODO: take advantage of iam database auth iam_database_authentication_enabled = true master_password = random_password.rds.result create_random_password = false apply_immediately = true - skip_final_snapshot = var.aws_postgres_database_final_snapshot == "" ? true : false - final_snapshot_identifier_prefix = var.aws_postgres_database_final_snapshot - snapshot_identifier = var.aws_postgres_restore_snapshot + skip_final_snapshot = var.aws_aurora_database_final_snapshot == "" ? true : false + final_snapshot_identifier_prefix = var.aws_aurora_database_final_snapshot + snapshot_identifier = var.aws_aurora_restore_snapshot create_db_cluster_parameter_group = true db_cluster_parameter_group_name = var.aws_resource_identifier - db_cluster_parameter_group_family = var.aws_postgres_database_group_family + db_cluster_parameter_group_family = var.aws_aurora_database_group_family db_cluster_parameter_group_description = "${var.aws_resource_identifier} cluster parameter group" - db_cluster_parameter_group_parameters = var.aws_postgres_engine == "aurora-postgresql" ? [ + db_cluster_parameter_group_parameters = var.aws_aurora_engine == "aurora-postgresql" ? [ { name = "log_min_duration_statement" value = 4000 @@ -84,18 +84,18 @@ module "rds_cluster" { create_db_parameter_group = true db_parameter_group_name = var.aws_resource_identifier - db_parameter_group_family = var.aws_postgres_database_group_family + db_parameter_group_family = var.aws_aurora_database_group_family db_parameter_group_description = "${var.aws_resource_identifier} example DB parameter group" - db_parameter_group_parameters = var.aws_postgres_engine == "aurora-postgresql" ? [ + db_parameter_group_parameters = var.aws_aurora_engine == "aurora-postgresql" ? [ { name = "log_min_duration_statement" value = 4000 apply_method = "immediate" } ] : [] - enabled_cloudwatch_logs_exports = var.aws_postgres_engine == "aurora-postgresql" ? ["postgresql"] : ["audit","error","general","slowquery"] + enabled_cloudwatch_logs_exports = var.aws_aurora_engine == "aurora-postgresql" ? ["postgresql"] : ["audit","error","general","slowquery"] tags = { - Name = "${var.aws_resource_identifier}-RDS" + Name = "${var.aws_resource_identifier} - Aurora" } } @@ -128,18 +128,18 @@ resource "random_string" "random_sm" { ### All of this added to handle snapshots resource "aws_db_cluster_snapshot" "db_snapshot" { - count = var.aws_postgres_snapshot_name != "" ? ( var.aws_postgres_snapshot_overwrite ? 0 : 1 ) : 0 - db_cluster_identifier = var.aws_postgres_cluster_name != "" ? var.aws_postgres_cluster_name : var.aws_resource_identifier - db_cluster_snapshot_identifier = var.aws_postgres_snapshot_name + count = var.aws_aurora_snapshot_name != "" ? ( var.aws_aurora_snapshot_overwrite ? 0 : 1 ) : 0 + db_cluster_identifier = var.aws_aurora_cluster_name != "" ? var.aws_aurora_cluster_name : var.aws_resource_identifier + db_cluster_snapshot_identifier = var.aws_aurora_snapshot_name lifecycle { ignore_changes = all } } resource "aws_db_cluster_snapshot" "overwrite_db_snapshot" { - count = var.aws_postgres_snapshot_name != "" ? ( var.aws_postgres_snapshot_overwrite ? 1 : 0 ) : 0 - db_cluster_identifier = var.aws_postgres_cluster_name != "" ? var.aws_postgres_cluster_name : var.aws_resource_identifier - db_cluster_snapshot_identifier = var.aws_postgres_snapshot_name + count = var.aws_aurora_snapshot_name != "" ? ( var.aws_aurora_snapshot_overwrite ? 1 : 0 ) : 0 + db_cluster_identifier = var.aws_aurora_cluster_name != "" ? var.aws_aurora_cluster_name : var.aws_resource_identifier + db_cluster_snapshot_identifier = var.aws_aurora_snapshot_name lifecycle { create_before_destroy = true } diff --git a/operations/deployment/terraform/modules/aws/aurora/aws_aurora_vars.tf b/operations/deployment/terraform/modules/aws/aurora/aws_aurora_vars.tf index 8996af94a..7bf76b78d 100644 --- a/operations/deployment/terraform/modules/aws/aurora/aws_aurora_vars.tf +++ b/operations/deployment/terraform/modules/aws/aurora/aws_aurora_vars.tf @@ -1,18 +1,18 @@ -variable "aws_postgres_engine" {} -variable "aws_postgres_engine_version" {} -variable "aws_postgres_database_group_family" {} -variable "aws_postgres_instance_class" {} -variable "aws_postgres_security_group_name" {} -variable "aws_postgres_subnets" {} -variable "aws_postgres_cluster_name" {} -variable "aws_postgres_database_name" {} -variable "aws_postgres_database_port" {} -variable "aws_postgres_restore_snapshot" {} -variable "aws_postgres_snapshot_name" {} -variable "aws_postgres_snapshot_overwrite" {} -variable "aws_postgres_database_protection" {} -variable "aws_postgres_database_final_snapshot" {} +variable "aws_aurora_engine" {} +variable "aws_aurora_engine_version" {} +variable "aws_aurora_database_group_family" {} +variable "aws_aurora_instance_class" {} +variable "aws_aurora_security_group_name" {} +variable "aws_aurora_subnets" {} +variable "aws_aurora_cluster_name" {} +variable "aws_aurora_database_name" {} +variable "aws_aurora_database_port" {} +variable "aws_aurora_restore_snapshot" {} +variable "aws_aurora_snapshot_name" {} +variable "aws_aurora_snapshot_overwrite" {} +variable "aws_aurora_database_protection" {} +variable "aws_aurora_database_final_snapshot" {} variable "aws_subnets_vpc_subnets_ids" {} variable "aws_resource_identifier" {} variable "aws_resource_identifier_supershort" {} diff --git a/operations/deployment/terraform/modules/aws/aurora/aws_dotenv_aurora_rds.tf b/operations/deployment/terraform/modules/aws/aurora/aws_dotenv_aurora_rds.tf index e97392f9c..57866148e 100644 --- a/operations/deployment/terraform/modules/aws/aurora/aws_dotenv_aurora_rds.tf +++ b/operations/deployment/terraform/modules/aws/aurora/aws_dotenv_aurora_rds.tf @@ -1,39 +1,44 @@ # Additional postgres configuration in postgres.tf -resource "local_file" "postgres-dotenv" { - filename = format("%s/%s", abspath(path.root), "postgres.env") +locals { + dba_engine = var.aws_aurora_engine == "aurora-postgresql" ? "postgres" : "mysql" +} + +resource "local_file" "aurora-dotenv" { + filename = format("%s/%s", abspath(path.root), "aurora.env") content = <<-EOT -#### Postgres values +#### Aurora values # Amazon Resource Name (ARN) of cluster -POSTGRES_CLUSTER_ARN=${module.rds_cluster.cluster_arn} +AURORA_CLUSTER_ARN=${module.aurora_cluster.cluster_arn} # The RDS Cluster Identifier -POSTGRES_CLUSTER_ID=${module.rds_cluster.cluster_id} +AURORA_CLUSTER_ID=${module.aurora_cluster.cluster_id} # The RDS Cluster Resource ID -POSTGRES_CLUSTER_RESOURCE_ID=${module.rds_cluster.cluster_resource_id} +AURORA_CLUSTER_RESOURCE_ID=${module.aurora_cluster.cluster_resource_id} # Writer endpoint for the cluster -POSTGRES_CLUSTER_ENDPOINT=${module.rds_cluster.cluster_endpoint} +AURORA_CLUSTER_ENDPOINT=${module.aurora_cluster.cluster_endpoint} # A read-only endpoint for the cluster, automatically load-balanced across replicas -POSTGRES_CLUSTER_READER_ENDPOINT=${module.rds_cluster.cluster_reader_endpoint} +AURORA_CLUSTER_READER_ENDPOINT=${module.aurora_cluster.cluster_reader_endpoint} # The running version of the cluster database -POSTGRES_CLUSTER_ENGINE_VERSION_ACTUAL=${module.rds_cluster.cluster_engine_version_actual} +AURORA_CLUSTER_ENGINE_VERSION_ACTUAL=${module.aurora_cluster.cluster_engine_version_actual} # Name for an automatically created database on cluster creation -# database_name is not set on `aws_rds_cluster[0]` resource if it was not specified, so can't be used in output -POSTGRES_CLUSTER_DATABASE_NAME=${module.rds_cluster.cluster_database_name == null ? "" : module.rds_cluster.cluster_database_name} +# database_name is not set on `aws_aurora_cluster[0]` resource if it was not specified, so can't be used in output +AURORA_CLUSTER_DATABASE_NAME=${module.aurora_cluster.cluster_database_name == null ? "" : module.aurora_cluster.cluster_database_name} # The database port -POSTGRES_CLUSTER_PORT="${module.rds_cluster.cluster_port}" +AURORA_CLUSTER_PORT="${module.aurora_cluster.cluster_port}" # TODO: use IAM (give ec2 instance(s) access to the DB via a role) # The database master password -POSTGRES_CLUSTER_MASTER_PASSWORD=${module.rds_cluster.cluster_master_password} +AURORA_CLUSTER_MASTER_PASSWORD=${module.aurora_cluster.cluster_master_password} # The database master username -POSTGRES_CLUSTER_MASTER_USERNAME=${module.rds_cluster.cluster_master_username} +AURORA_CLUSTER_MASTER_USERNAME=${module.aurora_cluster.cluster_master_username} # The Route53 Hosted Zone ID of the endpoint -POSTGRES_CLUSTER_HOSTED_ZONE_ID=${module.rds_cluster.cluster_hosted_zone_id} -# POSTGRES specific env vars -PG_USER="${module.rds_cluster.cluster_master_username}" -PG_PASSWORD="${module.rds_cluster.cluster_master_password}" -PGDATABASE=${module.rds_cluster.cluster_database_name == null ? "" : module.rds_cluster.cluster_database_name} -PGPORT=${module.rds_cluster.cluster_port} -PGHOST="${module.rds_cluster.cluster_endpoint}" +AURORA_CLUSTER_HOSTED_ZONE_ID=${module.aurora_cluster.cluster_hosted_zone_id} +# AURORA specific env vars +DBA_ENGINE="${local.dba_engine} +DBA_USER="${module.aurora_cluster.cluster_master_username}" +DBA_PASSWORD="${module.aurora_cluster.cluster_master_password}" +DBA_NAME=${module.aurora_cluster.cluster_database_name == null ? "" : module.aurora_cluster.cluster_database_name} +DBA_PORT=${module.aurora_cluster.cluster_port} +DBA_HOST="${module.aurora_cluster.cluster_endpoint}" EOT } \ No newline at end of file