Skip to content
Backup database, static files and config to AWS S3 with Cronjob
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.env
LICENSE
README.md
backup-db.sh
backup-files.sh

README.md

Amazon S3 Backup Bot

Relax, it's backed up. Off site, on line, in case. 🤖

Features

  • All configuration is done in the .env file, rather than in the scripts themselves
  • Uploads the backups to Amason AWS S3
  • After upload deletes the tarred files from the local server
  • Dumps each backup folder into its own file
  • Checks, repairs, and optimizes each MySQL database
  • Compress files before upload
  • Detailed information and comments
  • S3 storage class options to cut costs
  • Clear folder structure in S3

Summary

You’ve setup your server (for example with Serverpilot or Laravel Forge on Digital Ocean), but have you thought about backups? This script backups important data, such as database and file dumps, uploads it securely to S3, than deletes the local backups for security reasons. Amazon S3 can be an interestingly safe and cheap way to store your important data.

Folder structure in S3

Backups are stored in the following directory structure separated by client and week number (10) and type

Files: S3://yourbucket/client/10/files/backup_name_2017-03-01_10h18m_Wednesday-folder.tar.gz

Database: S3://yourbucket/client/10/db/backup_name_2017-03-01_10h18m_Wednesday-db.tar.gz

Getting started

1. Install AWS CLI

Logon your server with SSH We'll install the AWS CLI tool using Python PIP, a Python package manager.

# Install Pip
sudo apt-get install -y python-pip

# Install AWS CLI tool globally
sudo pip install awscli

# Check to ensure it exists
which aws

Tip: Do NOT configure AWS CLI with your root AWS credentials - yes it will work, but would you store your root server password in a plaintext file? No, and your AWS credentials give the holder access to unlimited resources, your billing details, your machine images, everything. Just create a new user/group that only has access to S3 and use those credentials to configure S3.

# Configure AWS - adding the KEY, SECRET KEY, default zone and default output format

aws configure
> YOUR KEY HERE
> YOUR SECRET KEY HERE
> eu-west-2
> json

2. Setup backup location and configuration

Now we create the backup folder and the environment file that holds our credentials.

mkdir ~/backup
cd ~/backup

All configuration is done in the .env file, rather than in the scripts themselves. So don’t forget to add your credentials in the appropriate variables, as well as to modify the paths based on your server and AWS setup. Open .env with vim and paste the .env credentials.

vim .env

Now close vim with ESC followed by :wq

3. Setup backup scripts

Now we can create our backup shell scripts.

  • backup-db.sh: This will export the database, compress it, and upload it to our S3 bucket.
  • backup-files.sh: This will export the files, tarball's it, and upload it to our S3 bucket.
vim backup-db.sh
vim backup-files.sh

Paste your scripts and close vim with ESC followed by :wq

4. Test your scripts

Make the scripts executable with chmod

chmod +x backup-db.sh
chmod +x backup-files.sh

And test the scripts

bash backup-db.sh
bash backup-files.sh

5. Add cronjob

Add these commands to the scheduled job.

Backup database
/usr/bin/env bash /home/forge/backup/backup-db.sh &>> /home/forge/backup/backup-db.log
Backup files
/usr/bin/env bash /home/forge/backup/backup-files.sh &>> /home/forge/backup/backup-files.log

Cleanup S3 Backups

Within S3 locate your bucket properties and find the “Lifecycle” option. Then just create a rule and you can delete old backups any number of days after they were created. I created a rule called “Delete Old Backups” that runs on the whole bucket and permanently deletes items created 30 days ago.

Alternative DB backup with automysqlbackup

Backup your datasbase with automysqlbackup. Installation is pretty simple: sudo apt-get install automysqlbackup You’re done! Backups will be made daily. All your databases will be stored in /var/lib/automysqlbackup. If you like to make manual backup just run sudo automysqlbackup.

For more information on installing automysqlbackup go to: Install automysqlbackup on Ubuntu

Sources

You can’t perform that action at this time.