PreBackup

This project aims to provide simple conventions and support for writing {pre,post}-backup scripts with minimal boilerplate, and also provide some common scripts (e.g. for Btrfs snapshots, PostgreSQL…).

Note this project is not a backup solution. So what it actually is? Many backup solutions, like Bacula or just simple rsync, work only with files. The problem is that you often can’t just copy bunch of files from a running system. You need to dump an SQL database, create a snapshot on Btrfs/ZFS/LVM, … to prepare files that will be actually backed up. Once the backup is done, you usually want to remove dumps, destroy snapshots etc. That’s where PreBackup can help.

PreBackup is a supplement to a file-oriented backup software. Its task is to prepare files for back up and do a cleanup when it’s done.

Originally it was written for Bacula, but it can be used with any backup solution. The only requirement is that your backup software can execute some command before and after a backup.

Installation

Gentoo

1. Add CTU’s Gentoo Overlay to your repos.conf or Layman (read howto).

2. emerge app-backup/prebackup

Manually

1. Clone this repository and jump in:

   git clone https://github.com/cvut/prebackup.git
   cd prebackup

2. Install prebackup (you may need to run this with sudo)…

   1. into /usr/local (i.e. /usr/local/bin and /usr/local/share/prebackup):

      ./install

   2. or into specified prefix, e.g. /usr:

      PREFIX=/usr ./install

Provided Scripts

btrfs-snapshot

Creates snapshot of Btrfs subvolume.

source_dev

The device with Btrfs filesystem that contains the subvolume you want to backup. Format is the same as in mount (e.g. /dev/sda2, LABEL=btrfsvol). This variable is required.

source_subvol

Name of the Btrfs subvolume that should be backed up. This variable is required.

mariadb

Dumps MariaDB (formerly MySQL) databases.

exclude_dbs

A list of database names to exclude from backup (optional).

my_cnf_file

Path of mysql/mariadb options file that will be passed to --defaults-extra-file.This is a way how you can provide credentials for DB server. Default is ~/.my.cnf.

mongodb

Dumps MongoDB databases.

exclude_dbs

A list of database names to exclude from backup (optional).

mongo_password

Password for authentication (optional).

mongo_user

Username for authentication (optional).

noop

Does nothing…

postgresql

Dumps PostgreSQL databases.

exclude_dbs

A list of database names to exclude from backup (optional).

Script Format

PreBackup script is a Plain Old Shell Script (POSIX-compliant with local and readonly builtins) with few simple conventions (just like OpenRC runscripts, ebuilds etc.)

You should use shebang with path of the master prebackup script. Then you must define functions pre_backup, post_backup and optionally setup.

setup()

If defined, this function is executed before each pre_backup() and post_backup(), in the same environment. You may use it to check for required variables (using required_var()), to define some common variables or anything else that should be run before pre_backup() and post_backup().

pre_backup()

This function is executed when you run the script as a file named pre-backup (e.g. using symlink) or with argument -c pre-backup. As the name suggests, it should be run before actual backup, e.g. to dump database, create filesystem snapshot, …

post_backup()

This function is executed when you run the script as a file named post-backup (e.g. using symlink) or with argument -c post-backup. It should be run after actual backup, e.g. to clean database dumps, destroy filesystem snapshot, …

Prebackup Script Template

#!/usr/local/bin/prebackup
# vim: set ft=sh ts=4:

setup() {
    echo 'Run before pre_backup and post_backup function.'
}

pre_backup() {
    echo 'Run before actual backup.'
}

post_backup() {
    echo 'Run after actual backup.'
}

Configuration

There are two places where you can define variables for a prebackup script:

• /etc/bacula/prebackup.conf – global configuration file (you can modify this path in bin/prebackup),

• ./config – local configuration file located in the same directory as the pair of pre-backup and post-backup symlinks (or file and symlink) used to run the script for a particular job.

These files are just another shell scripts, but you should define only variables here. The files are sourced in the defined order before an actual prebackup script. If both files exist and define the same variable, the local config overrides the global.

Predefined Variables

This section describes predefined variables available in config files and prebackup scripts.

All variables have sensible default value that may be overriden in global prebackup.conf or job’s config file. Uppercase variables are read-only for prebackup script, but can be modified in config files (except SCRIPT_DIR).

NAME

Name of the “job”. Default is basename of $SCRIPT_DIR, i.e. name of directory where the sourced script resides.

RUN_AS

User to run the script. If you execute the script as a different user than the one specified by $RUN_AS, it will switch to the specified user using sudo. Default is empty, i.e. do not switch user.

SYSLOG

Redirect stdout and stderr to syslog (yes/no)? Default is yes.

SYSLOG_TAG

Tag to mark every line sent to syslog. This is called a program name in syslog-ng. Default is backup:$NAME.

BASE_TARGET_DIR

Base path to a directory where the files should be stored to be backed up by a backup software. Default is /var/tmp/backup.

BASE_TEMP_DIR

Base path to a directory where temporary files for prebackup scripts should be stored. Default is /tmp/prebackup.

VERBOSE

Log even debug messages (yes/no)? Default is no.

target_dir

Path of directory where files for a particular “job” should be stored (i.e. files generated by the sourced script) to be backed up by a backup software. Default is $BASE_TARGET_DIR/$NAME.

temp_dir

Path of directory where temporary files for the sourced script should be stored. Default is $BASE_TEMP_DIR/$NAME.

Predefined Functions

The following functions (defined in utils.sh) are available in prebackup scripts.

fail()

Log error message and exit.

• $1: message

• $2: exit code (default: 1)

info()

Log info message.

• $1: message

debug()

Log debug message if $VERBOSE=yes.

• $1: message

check_dir_exists()

Check if the specified path is a directory and fail if it’s not.

• $1: path to check

check_file_nonempty()

Check if the specified path is a non-empty file and fail if it’s not.

• $1: path to check

check_gzip_nonempty()

Check if the specified path is a non-empty gzip file, fail if it’s not.

• $1: path to check

required_var()

If the specified variable is empty, then log error message and exit.

• $1: variable name

list_has()

Test if the first argument is equal to one of the subsequent arguments, i.e. if list ${@:2} includes $1.

• $1: needle

• $@: elements

Recommended Directory Structure

/
|-- etc
|   `-- bacula
|       |-- job1
|       |   |-- config
|       |   |-- post-backup ---------┐
|       |   `-- pre-backup ----------┤
|       `-- job2                     | s
|       |   |-- pre-backup -------┐  | y
|       |   `-- post-backup <-----┘  | m
|       `-- jobN                     | l
|       |   |-- config               | i
|       |   |-- post-backup ------┐  | n
|       |   `-- pre-backup -------┤  | k
|       `-- prebackup.conf        |  |
`-- usr(/local)                   |  |
    |-- bin                       |  |
    |   `-- prebackup             |  |
    `-- share                     |  |
        `-- prebackup             |  |
            |-- btrfs-snapshot <--┘  |
            |-- postgresql <---------┘
            `-- ...

License

This project is licensed under MIT License.