Daily Backup

Overview

This package contains a set of scripts I ended up writting over time with the goal of having an automated tool that would backup my most important files once a day, usually during my first boot on that day.

The functionality is splitted into three main tasks:

1. Create a password encrypted backup file using tar and gpg.
2. Upload the created backup file to some remote destination (using either scp, backblaze) or gdrive).
3. Send an email to the user to let him know the script finished.

We use systemd services and timers to run the script daily.

Here are the programs/dependencies used:

Name	Description	Notes
tar	Used to created compressed backup file.
gpg	Used to encrypt the backup file with a password.
scp	Used to upload the backup file to a remote server.	optional
python3+b2sdk	Used to upload the backup file to a backblaze bucket.	optional
curl + mailgun	Used to send an email after the backup is completed.
systemd	Uses timers and services to run the backup daily.
gdrive	Used to upload the backup to a Google Drive account	optional

Installing

If you use archlinux

git clone https://github.com/vitorqb/daily-backup-pkgbuild.git && cd daily-backup-pkgbuild && makepkg -si

Full Instalation

There is a install script that will install all files for a unix-like system.

./install

Configuring

• Copy the example env configuration to your user home .config

install -D /usr/share/doc/daily-backup/env.example ~/.config/daily-backup/env

• Edit it with your favorite text editor and customize it for your needs. See Options below.
• Since this file contains raw passwords, I highly recommend setting restrictive permissions

sudo chmod 700 ~/.config/daily-backup/env

• Now create a file with a list of files to be backed up. See tar --files-from option. The file must be on the path set with DAILY_BACKUP_FILES_FROM. For example:

# file:{USER_HOME}/.config/daily-backup/files
{USER_HOME}/git
{USER_HOME}/projects
{USER_HOME}/.config
# ...

• Do the same for the files to ignore (see tar --exclude-from). The path is determined by DAILY_BACKUP_EXCLUDE_FROM.

# file:{USER_HOME}/.config/daily-backup/excludes
{USER_HOME}/.config/google-chrome/*

• If you are using scp, make sure you can connect with ssh to the server which will receive the backup. I have something like this:

# file:{USER_HOME}/.ssh/config
HOST <HOST_IP_ADDRESS>
  IdentityFile {USER_HOME}/.ssh/id_rsa
  UserKnownHostsFile {USER_HOME}/.ssh/known_hosts

Enabling systemd service and timer

We rely on systemd service and timer to run the backup daily.

• Copy the example systemd timer and service to the systemd services location:

install -D /usr/share/doc/daily-backup/systemd-templates/* /etc/systemd/system/

• Edit each file substituting USER_NAME for the name of your user and USER_HOME for the full path to the user home.

[Unit]
Description=Daily Backup For ${USER_NAME}

[Service]
EnvironmentFile=${USER_HOME}/.config/daily-backup/env
ExecStart=/usr/bin/daily-backup launch

[Unit]
Description=Runs Daily Backup For ${USER_NAME}

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

• Enable and start the timer:

sudo systemctl enable daily-backup.timer && sudo systemctl start daily-backup.timer

• Now you can try to run and see the logs:

sudo systemctl start daily-backup && sudo journalctl --unit daily-backup --follow

Options

Those are configuration variables from the config file:

• DAILY_BACKUP_PASSWORD

  Password used to encrypt the backup with gpg.

• DAILY_BACKUP_FILES_FROM

  A file passed to tar --files-from for running the backup. It decides which folders/files will be included in the backup.

• DAILY_BACKUP_EXCLUDE_FROM

  A file passed to tar --exclude-from when running the backup. It allows you to ignore specific files or directories when running the backup.

• DAILY_BACKUP_LOCAL_OUTPUT_DIR

  A (local) path where the backups will be stored in your computer.

• DAILY_BACKUP_LOCAL_LOG_DIR

  A (local) path where the backups logs will be stored in your computer.

• DAILY_BACKUP_SHOULD_CLEANUP_LOCAL_FOLDER

  If set to “1”, cleans old backups from the local output folder. The number of backup files to keep is controlled by DAILY_BACKUP_LOCAL_NUM_OF_FILES_TO_KEEP. Defaults to “”, which means no cleanup.

• DAILY_BACKUP_LOCAL_NUM_OF_FILES_TO_KEEP

  The number of local backups to keep. Only has effect if DAILY_BACKUP_SHOULD_CLEANUP_LOCAL_FOLDER=1. Defaults to 10. Example: if set to 10, we keep the 10 most recent backup files every time the cleanup is run.

• DAILY_BACKUP_SHOULD_CLEANUP_REMOTE

  If set to “1”, cleans old backups from the remote storage. The number of backup files to keep is controlled by DAILY_BACKUP_REMOTE_NUM_OF_FILES_TO_KEEP.

• DAILY_BACKUP_REMOTE_NUM_OF_FILES_TO_KEEP

  The number of local backups to keep for the remote backup storage.. Only has effect if DAILY_BACKUP_SHOULD_CLEANUP_REMOTE=1. A cleanup job is run after each successfull upload.

• DAILY_BACKUP_EMAIL_TO

  An email address that will receive an email everytime the backup is run.

• DAILY_BACKUP_MAILGUN_API_KEY

  Mailgun api key used to send emails.

• DAILY_BACKUP_MAILGUN_DOMAIN

  Mailgun domain used to send emails.

• DAILY_BAKCUP_UPLOAD_TYPE

  Either scp or b2. Defines how to upload the backup to a remote server.

• DAILY_BACKUP_REMOTE_OUTPUT_DIR (DAILY_BAKCUP_UPLOAD_TYPE=”scp”)

  An ssh-like path destination for the backups. Example:

<REMOTE_USER>@<REMOTE_IP>:daily-backup/output

• DAILY_BACKUP_SSH_CONFIG_FILE (DAILY_BAKCUP_UPLOAD_TYPE=”scp”)

  The ssh config file used to access the server.

• DAILY_BACKUP_B2_API_KEY (DAILY_BAKCUP_UPLOAD_TYPE=”b2”)

  Secret api key for Backblaze.

• DAILY_BACKUP_B2_API_KEY_ID (DAILY_BAKCUP_UPLOAD_TYPE=”b2”)

  Secret api key id for Backblaze.

• DAILY_BACKUP_B2_BUCKET_NAME (DAILY_BAKCUP_UPLOAD_TYPE=”b2”)

  Name of the bucket where the generated backup will be stored.

• DAILY_BACKUP_GDRIVE_DEST_ID (DAILY_BAKCUP_UPLOAD_TYPE=”gdrive”)

  The ID of the directory where to upload backups. See `gdrive list –absolute`.

• DAILY_BACKUP_GDRIVE_CONFIG (DAILY_BAKCUP_UPLOAD_TYPE=”gdrive”)

  The configuration file to give to gdrive (`-c` option in `gdrive help upload`)