cloud-platform-terraform-rds-aurora

This Terraform module will create an Amazon Aurora database for use on the Cloud Platform.

Usage

module "rds_aurora" {
  source = "github.com/ministryofjustice/cloud-platform-terraform-rds-aurora?ref=version" # use the latest release

  # VPC configuration
  vpc_name = var.vpc_name

  # Database configuration
  engine         = "aurora-postgresql"
  engine_version = "14.7"
  engine_mode    = "provisioned"
  instance_type  = "db.t4g.medium"
  replica_count  = 1

  # Tags
  business_unit          = var.business_unit
  application            = var.application
  is_production          = var.is_production
  team_name              = var.team_name
  namespace              = var.namespace
  environment_name       = var.environment
  infrastructure_support = var.infrastructure_support
}

Accessing the database

Database Hostname/Credentials

The hostname and credentials for accessing your database will be in a kubernetes secret inside your namespace. You can retrieve them as follows (the decode.rb script is available here):

$ kubectl -n [your namespace] get secret [secret name] -o yaml | ./decode.rb
---
apiVersion: v1
data:
  database_name: ...
  database_password: ...
  database_username: ...
  access_key_id: ...
  secret_access_key: ...
  rds_instance_address: cloud-platform-xxxxx.yyyyy.eu-west-2.rds.amazonaws.com
  rds_instance_endpoint: cloud-platform-xxxxx.yyyyy.eu-west-2.rds.amazonaws.com:5432
  rds_instance_port: '5432'
kind: Secret
metadata:
  creationTimestamp: '2019-05-08T16:14:23Z'
  name: secret-name
  namespace: your-namespace
  resourceVersion: '11111111'
  selfLink: "/api/v1/namespaces/your-namespace/secrets/secret-name"
  uid: 11111111-1111-1111-1111-111111111111
type: Opaque

If you are exporting a database URL from your RDS kubernetes secret, it might have a value like this:

postgres://cpDvquXO5B:R1eDN0xEUnaH6Aqr@cloud-platform-df3589e0e7acba37.cdwm328dlye6.eu-west-2.rds.amazonaws.com:5432/dbdf3589e0e7acba37

The database hostname is part between @ and : In the example above, the database hostname is:

cloud-platform-df3589e0e7acba37.cdwm328dlye6.eu-west-2.rds.amazonaws.com

NB: You should always get the database credentials from this kubernetes secret. Do not be tempted to copy the into another location (such as a ConfigMap). This is because the value of the secret can be updated when this module is updated. As long as you always get your database credentials from the kubernetes secret created by terraform, this is fine. But if you copy the value elsewhere, it will not be automatically updated in the new location, and your application will no longer be able to connect to your database.

Launching psql in the cluster

A Docker image containing the psql utility is available from Bitnami (you cannot use the official postgres image, because it runs as root) and can be launched like this:

$ kubectl -n [your namespace] run --generator=run-pod/v1 shell --rm -i --tty --image bitnami/postgresql -- bash
If you don't see a command prompt, try pressing enter.
postgres@shell:/$

You can then connect to your database like this

postgres@shell:/$ psql -h [rds_instance_address] -U [database_username] [database_name]
Password for username: [...enter database_password here...]
psql (10.7, server 10.6)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
[database_name]=>

Accessing your RDS database from your laptop

Instructions on how to do this are available here

Managing RDS Cluster snapshots - backups and restores

An IAM user account is created which allows management of RDS Cluster snapshots - allowing snapshot create, delete, copy, restore.

Example usage via AWS CLI:

List RDS Cluster snapshots

aws rds describe-db-cluster-snapshots --db-cluster-identifier [db_cluster_identifier]

Create snapshot

aws rds create-db-cluster-snapshot
--db-cluster-identifier [db_cluster_identifier] --db-cluster-snapshot-identifier [your-snapshot-name]

See the examples/ folder for more information.

Requirements

Name	Version
terraform	>= 1.2.5
aws	>= 4.0.0
random	>= 3.0.0

Providers

Name	Version
aws	>= 4.0.0
random	>= 3.0.0

Modules

No modules.

Resources

Name	Type
aws_db_subnet_group.db_subnet	resource
aws_iam_policy.irsa	resource
aws_kms_alias.alias	resource
aws_kms_key.kms	resource
aws_rds_cluster.aurora	resource
aws_rds_cluster_instance.aurora_instances	resource
aws_security_group.rds-sg	resource
random_id.id	resource
random_password.password	resource
random_string.username	resource
aws_caller_identity.current	data source
aws_iam_policy_document.irsa	data source
aws_region.current	data source
aws_subnet.eks_private	data source
aws_subnet.private	data source
aws_subnets.eks_private	data source
aws_subnets.private	data source
aws_vpc.this	data source

Inputs

Name	Description	Type	Default	Required
allow_major_version_upgrade	Determines whether major engine upgrades will be performed automatically in the maintenance window	bool	false	no
application	Application name	string	n/a	yes
auto_minor_version_upgrade	Determines whether minor engine upgrades will be performed automatically in the maintenance window	bool	true	no
backtrack_window	The target backtrack window, in seconds. Only available for aurora engine currently. To disable backtracking, set this value to 0. Defaults to 0. Must be between 0 and 259200 (72 hours)	number	0	no
backup_retention_period	How long to keep backups for (in days)	number	7	no
business_unit	Area of the MOJ responsible for the service	string	n/a	yes
ca_cert_identifier	Specifies the identifier of the CA certificate for the DB instance	string	"rds-ca-rsa2048-g1"	no
db_cluster_parameter_group_name	The name of a DB Cluster parameter group to use	string	null	no
db_name	The name of the database to be created on the instance (if empty, it will be the generated random identifier)	string	""	no
db_parameter_group_name	The name of a DB parameter group to use	string	null	no
deletion_protection	If the DB instance should have deletion protection enabled	bool	false	no
engine	Aurora database engine type, currently aurora, aurora-mysql or aurora-postgresql	string	"aurora"	no
engine_mode	The database engine mode. Valid values: global, parallelquery, provisioned, serverless, multimaster.	string	"provisioned"	no
engine_version	Aurora database engine version.	string	""	no
environment_name	Environment name	string	n/a	yes
infrastructure_support	The team responsible for managing the infrastructure. Should be of the form ()	string	n/a	yes
instance_type	Instance type to use at master instance. If instance_type_replica is not set it will use the same type for replica instances	string	"db.t3.medium"	no
instances_parameters	Customized instance settings. Supported keys: instance_name, instance_type, instance_promotion_tier, publicly_accessible	list(map(string))	[]	no
is_production	Whether this is used for production or not	string	n/a	yes
namespace	Namespace name	string	n/a	yes
performance_insights_enabled	Specifies whether Performance Insights is enabled or not.	bool	false	no
port	The port on which to accept connections	string	""	no
preferred_backup_window	When to perform DB backups	string	"02:00-03:00"	no
preferred_maintenance_window	When to perform DB maintenance	string	"sun:05:00-sun:06:00"	no
publicly_accessible	Whether the DB should have a public IP address	bool	false	no
replica_count	Number of reader nodes to create. If replica_scale_enable is true, the value of replica_scale_min is used instead.	number	1	no
replica_scale_enabled	Whether to enable autoscaling for RDS Aurora read replicas	bool	false	no
replica_scale_min	Minimum number of replicas to allow scaling for	number	2	no
replication_source_identifier	ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica.	string	""	no
scaling_configuration	Map of nested attributes with scaling properties. Only valid when engine_mode is set to serverless	map(string)	{}	no
serverlessv2_scaling_configuration	Map of nested attributes with scaling properties (max_capacity, min_capacity) for Aurora Serverless v2. Only valid when engine mode is set to provisioned. The instance_type variable must also be set to db.serverless	map(string)	{}	no
skip_final_snapshot	Should a final snapshot be created on cluster destroy	bool	false	no
skip_setting_when_migrated	Setting a database name on a replica creates a loop because Aurora tries to change terraform's random string to the source name + some other random string	bool	false	no
snapshot_identifier	DB snapshot to create this database from	string	""	no
team_name	Team name	string	n/a	yes
vpc_name	The name of the vpc (eg.: live-1)	string	n/a	yes

Outputs

Name	Description
database_name	Name of the database
database_password	Database Password
database_username	Database Username
db_cluster_identifier	The RDS DB Cluster Indentifer
irsa_policy_arn	IAM policy ARN for access to create database snapshots
rds_cluster_endpoint	A cluster endpoint (or writer endpoint) for the aurora DB cluster.
rds_cluster_port	The database port
rds_cluster_reader_endpoint	A reader endpoint for the aurora DB cluster. Use the reader endpoint for read operations, such as queries
rds_instance_endpoint	An instance endpoint connecting the DB instance within an Aurora cluster
resource_id	RDS Resource ID - used for performance insights (metrics)

Tags

Some of the inputs for this module are tags. All infrastructure resources must be tagged to meet the MOJ Technical Guidance on Documenting owners of infrastructure.

You should use your namespace variables to populate these. See the Usage section for more information.

Reading Material

• Cloud Platform user guide
• Amazon Aurora user guide