Backup and restore the database of your rails app to amazon S3, encrypting and compressing it on the fly
Pull request Compare This branch is 10 commits ahead, 91 commits behind mmarschall:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


s3db-backup for Rails

Its good practice to have a backup of your database ready at some other place than your production setup in case anything goes badly wrong there. And its a good idea to test your backups regularily. This gem will help you do both in a snap.

Backup Your MySQL Database To Amazon S3

To get your data out of your own data center and store it somewhere save, amazon S3 is a great candidate. But, of course, you do not want to let anyone be able to read your application database. That's why s3db-backup is encrypting your dump before sending it to amazon S3.


As s3db-backup is using standard Unix tools for the heavy lifting, please make sure you have the following installed on your machine:

  • mysqldump (comes with the mysql client, should be there)

  • gzip (never saw a system without it)

  • ccrypt (you might need to install it: apt-get install ccrypt (Debian, Ubuntu), yum install ccrypt (Redhat, CentOS), or brew install ccrypt (Mac OS X using homebrew))

And, of course, you'll need to sign up for an Amazon S3 account and create a bucket using the AWS Management Console (or any other way)

Getting Started

s3db-backup only supports MySQL databases and works from within a Rails application. Here are the steps to get you started:

  • Install the s3db-backup gem

sudo gem install s3db-backup

  • Run the s3db_backup generator to generate the necessary config files and rake tasks

$ script/generate s3db_backup This will generate the following files: config/s3_config.yml, db/secret.txt, and lib/tasks/db.rake

  • Store your AWS access credentials and your desired S3 buckets in the new config/s3_config.yml:

aws_access_key_id: “<your aws access key here>” secret_access_key: “<your aws secret key here>”

production: bucket: “my_app-production”

development: bucket: “my_app-development”

s3db-backup expects the buckets you name there to exist. Please make sure they do (using the Amazon AWS Management Console or any other Amazon Management Tool)

  • Put a long and random string as your encryption/decryption key for ccrypt into db/secret.txt

$ echo “my secret key” > db/secret.txt

Running a Backup

Dumping, encrypting, and uploading your current database is now as easy as typing one command: $ rake s3db:backup When its done, go to your amazon S3 bucket and see your shiny new backup file there!

Restoring a Backup Using rake

To be able to fetch your backups from S3 and load them into your environment (development or production) you can use a set of rake tasks defined in lib/tasks/s3db_backup.rake:

  • Fetch the latest backup from S3 to your local disk as db/latest_prod_dump.sql:

$ rake s3db:latest:fetch Now you can either load that SQL dump with any tool you like or use a second rake task:

  • Load that dump into your environment:

ATTENTION: This step will DROP your database and re-create it from your backup. Be careful when applying to your production environment. You have been warned!

$ rake s3db:latest:load

Restoring a Backup Manually

I do not trust in backup systems, which do everything on their own. In emergency I want to be able to restore my data manually. As s3db-backup only uses standard Unix tools for the job, its pretty easy. Here are the steps to manually restore your database

  • Download the latest backup from your Amazon S3 bucket e.g. mysql-my_app-production-22-09-2010-11h24m10s.sql.gz.cpt

  • Decrypt the file using your key from db/secret.txt

ccdecrypt mysql-my_app-production-22-09-2010-11h24m10s.sql.gz.cpt

  • Unzip the SQL file

gunzip mysql-my_app-production-22-09-2010-11h24m10s.sql.gz

  • Re-create your database

rake db:create

  • Load the dump into mysql

mysql -u <your_db_user> -p <your_db> < mysql-my_app-production-22-09-2010-11h24m10s.sql Now you should have your DB restored.

Note on Patches/Pull Requests

  • Fork the project.

  • Make your feature addition or bug fix.

  • Add tests for it. This is important so I don't break it in a future version unintentionally.

  • Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)

  • Send me a pull request. Bonus points for topic branches.


Copyright © 2010 Matthias Marschall. See LICENSE for details.