Automate the creation/lifecycle management of RDS database instances

[MikeTheCanuck]

Summary

Integrate the manually-created 2019 RDS database instances into our CloudFormation infrastructure-as-code.

Tasks

• Figure out how to deploy RDS with CloudFormation
• Test management of RDS with CloudFormation to ensure that it keeps existing instances when we upgrade instance classes or assigned storage (so that we don't destroy existing databases)
• Determine if in fact we will be duplicating per-project databases for both Staging and Production purposes
• Generate and deploy the CF templates for each project's Staging databases - migrating databases as required if naming changes
• Generate and deploy the CF templates for each project's new Production databases (with more restrictive requirements)

Definition of Done

• All projects' required databases are live and managed by CloudFormation
• All projects' databases' connectivity have been validated in required environments (internet or container VPC)

[MikeTheCanuck]

Transportation-2019-staging database instance has been manually configured according to #229

[MikeTheCanuck]

Housing-2019-staging database instance has been manually configured according to #230

[MikeTheCanuck]

Some clues where to find compatible CF YAML templates - I noticed a reference in the old README.md in our hackoregon-aws-infrastructure repo to /infrastructure/rds.yaml, which strongly implies that at least as of March 5, 2017 (when we originally forked from the AWS Labs repo), there was a (semi?) working RDS deployment template that fit into this CF architecture.

So while that file is now gone from the originating repo, I notice 800+ forks from that repo, and found a few promising leads in the forked projects:
https://github.com/BrianHGrant/ecs-refarch-cloudformation/blob/master/infrastructure/rds.yaml
https://github.com/jnorback/ecs-refarch-cloudformation/blob/ECSandUpv1/db/postgresdb.yaml

[MikeTheCanuck]

Nearly all the CloudFormation code has been integrated, up to the need for SSM parameters to be integrated at runtime.

This article appears to have the key to constructing SSM parameter references:
https://aws.amazon.com/blogs/mt/using-aws-systems-manager-parameter-store-secure-string-parameters-in-aws-cloudformation-templates/

The only thing we require in addition is to perform a Join operation or the like to insert a specific Project Name into the parameter name in that {{resolve:... statement.

[MikeTheCanuck]

Validation testing done today:

• created new database with RDS template - 20 GiB storage, db.t2.micro instance
• manually created table, column and row in the database (to verify that the data doesn't get blown away by subsequent upgrade operations)
• updated database to db.t2.small
• verified the table and row still exists
• updated storage to 50 GiB
• verified the table and row still exists

All clear. Happy to see that RDS upgrades like this doesn't brick the underlying data.