Add script to copy/anonymize the production database onto staging #1948
Conversation
tools/copy_production_db_to_staging
Outdated
# Constants | ||
STAGING_ID="openapparelregistry-enc-stg" | ||
PRODUCTION_ID="openapparelregistry-enc-prd" | ||
ALLOWED_EMAILS="person@example.com other@example.com" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jwalgran do we have the actual list we want here? Or do we not want to list the default like with the password fields?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are 3 API partners who's accounts we want to preserve but we do not want to name them here in the open source repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This worked delightfully well. Nice job.
I ran a successful test of this after one typo fixup. I made some other small recommendations that don't need another full round of review.
tools/copy_production_db_to_staging
Outdated
# Constants | ||
STAGING_ID="openapparelregistry-enc-stg" | ||
PRODUCTION_ID="openapparelregistry-enc-prd" | ||
ALLOWED_EMAILS="person@example.com other@example.com" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are 3 API partners who's accounts we want to preserve but we do not want to name them here in the open source repo.
72a4511
to
b7fecdc
Compare
Overview
This adds an interactive script meant to be run on the staging bastion, which automates the process of restoring a production database snapshot, anonymizing it, creating a dump, and restoring that dump on top of staging.
Connects #1813
Demo
Notes
I ran this from the bastion.
To run it, I first had to manually install
git
andpython3
:sudo yum install git python37
. Then I cloneopen-apparel-registry
and checked out this branch and ran./tools/copy_production_db_to_staging
.For testing purposes, I used a copy of the staging database I spun up from a recent snapshot.
Testing Instructions
open-apparel-registry/
and ensure the latest version of this branch is checked out../tools/copy_production_db_to_staging
. You will be asked to supply staging/production database passwords, as well as confirm data determined using the AWS CLI (I substituted the staging database copy for the staging database endpoint determined by the script).psql
on either the temporary RDS database or the database you will be copying onto (staging by default)Checklist
fixup!
commits have been squasheddevelop
vs.ogr/develop
)