database_sanitizer

Sample MySQL Database Sanitization Process

Quick Start Guide

1. Clone the repo on the utility server where production data can be accessed, as well as on the local development server.
2. Modify the credentials at the top of scripts/aws_sanitizer.sh.
3. Run the script on a cron or from the command line on a utility server where you are able to access your RDS or standard MySQL databases.
4. Run scripts/aws_sanitizer_utility.sh on the server where you need to work with the data. This script pulls the compressed file from S3 to a defined directory.

FAQ

Dependencies

• AWS CLI
• MySQL 5.6+
• Bash

Application Structure

• scripts/aws_sanitizer.sh relies on the existance of a routines directory containing sanitization routines or queries.
• Applicable database user grants can be stored in databases/grants.sql if your repository is in a secure environment. If you choose not to store user grants here, uncomment line 40 in scripts/aws_sanitizer.sh.

Database Connections

Note that the scripts rely heavily on the use of a defined login path. This makes managing database credentials easy, and keeps sensitive data out of source control. However, if you are not using a MySQL login path, you can simply replace --login-path=$login_path_ with -h hostname -u username -p.