Skip to content
Permalink
master
Go to file
6 contributors

Users who have contributed to this file

@gozora @petroniusniger @jsmeix @rowens275 @flyinggreenfrog @dagwieers
1437 lines (1177 sloc) 53.6 KB

Scenarios

Bootable ISO

If you simply want a bootable ISO on a central server, you would do:

OUTPUT=ISO
OUTPUT_URL=http://server/path-to-push/

Bootable ISO with an external (commercial) backup software

If you rely on your backup software to do the full restore of a system then you could define:

OUTPUT=ISO
BACKUP=[TSM|NSR|DP|NBU|GALAXY10|SEP|DUPLICITY|BACULA|BAREOS|RBME|FDRUPSTREAM]

When using one of the above backup solution (commercial or open source) then there is no need to use rear mkbackup as the backup workflow would be empty. Just use rear mkrescue

ReaR will incorporate the needed executables and libraries of your chosen backup solution into the rescue image of ReaR.

Bootable ISO together with backup archive stored on NFS/NAS

To create an ISO rescue image and using a central NFS/NAS server to store it together with the backup archive, you could define:

OUTPUT=ISO
BACKUP=NETFS
# BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://your.NFS.server.IP/path/to/your/rear/backup
# BACKUP_PROG_CRYPT_ENABLED="yes"
# { BACKUP_PROG_CRYPT_KEY='my_secret_passphrase' ; } 2>/dev/null

The above example shows that it is also possible to encrypt the backup archive. Currently only 'tar' is supported for backup archive encryption and decryption. When BACKUP_PROG_CRYPT_ENABLED is set to a true value, BACKUP_PROG_CRYPT_KEY must be also set. There is no BACKUP_PROG_CRYPT_KEY value in the /etc/rear/local.conf file in the rescue image. It gets removed because the ReaR rescue/recovery system must be free of secrets. Otherwise the rescue system ISO image and any recovery medium that is made from it would have to be carefully protected against any unwanted access. Therefore BACKUP_PROG_CRYPT_KEY must be manually set before running "rear recover". For example via export BACKUP_PROG_CRYPT_KEY='my_secret_passphrase' before calling "rear recover" and/or also before calling "rear mkbackup" so that there is no need to store it ever in a ReaR config file. On the other hand it is crucial to remember the BACKUP_PROG_CRYPT_KEY value that was used during "rear mkbackup" so that possibly a long time later that rescue image can be used (possibly by someone else) to recover from a disaster. If the BACKUP_PROG_CRYPT_KEY value should be set in a ReaR config file you should avoid that the BACKUP_PROG_CRYPT_KEY value is shown in a log file when 'rear' is run in debugscript mode (where 'set -x' is set) by redirecting STDERR to /dev/null via { command confidential_argument ; } 2>/dev/null where the redirection must be done via a compound group command even for a single confidential command to let the redirection also apply for 'set -x'. See the comment of the UserInput function in lib/_input-output-functions.sh how to keep things confidential when 'rear' is run in debugscript mode.

Bootable USB device with backup to USB

If you want a bootable USB device with a (tar) backup to USB as well, you would use:

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

Bootable tape drive (OBDR) with backup to tape

If you want an OBDR image and backup on tape, and use GNU tar for backup/restore, you would use:

BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Bootable tape drive (OBDR) and Bacula restore

If you want an OBDR image on tape, and the Bacula tools to recover your backup, use:

BACKUP=BACULA
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

ReaR with Borg back end

Important
We strongly recommend to use Borg standalone binary (https://github.com/borgbackup/borg/releases) as it includes all necessities for Borg operations. If you decide to go for different type of Borg installation types, make sure you include all needed files for Borg runtime into ReaR rescue/recovery system. E.g. by using COPY_AS_IS_BORG=( '/usr/lib64/python3.4*' '/usr/bin/python3*' '/usr/bin/pyvenv*' '/usr/lib/python3.4*' '/usr/lib64/libpython3*' )

Borg → SSH

  • Setup ssh key infrastructure for user that will be running backup. Issuing following command must work without any password prompts or remote host identity confirmation:

ssh <BORGBACKUP_USERNAME>@<BORGBACKUP_HOST>

  • Example local.conf:

OUTPUT=ISO
OUTPUT_URL=nfs://foo.bar.xy/mnt/backup/iso

BACKUP=BORG
BORGBACKUP_HOST="foo.bar.xy"
BORGBACKUP_USERNAME="borg_user"
BORGBACKUP_REPO="/mnt/backup/client"
BORGBACKUP_REMOTE_PATH="/usr/local/bin/borg"

# Automatic archive pruning
# (https://borgbackup.readthedocs.io/en/stable/usage/prune.html)
BORGBACKUP_PRUNE_KEEP_WEEKLY=2

# Archive compression
# (https://borgbackup.readthedocs.io/en/stable/usage/create.html)
BORGBACKUP_COMPRESSION="lzma,6"     # Slowest backup, best compression

# Repository encryption
# (https://borgbackup.readthedocs.io/en/stable/usage/init.html)
BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE='S3cr37_P455w0rD'
COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" )

# Borg environment variables
# (https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables)
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

Borg → USB

  • Example local.conf:

OUTPUT=USB
BACKUP=BORG

USB_DEVICE=/dev/disk/by-label/REAR-000

BORGBACKUP_REPO="/my_borg_backup"
BORGBACKUP_UMASK="0002"

BORGBACKUP_PRUNE_KEEP_WEEKLY=2

BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE='S3cr37_P455w0rD'

export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" )
COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" )

SSH_UNPROTECTED_PRIVATE_KEYS="yes"
SSH_FILES="yes"
Important
If using BORGBACKUP_ENC_TYPE="keyfile", don’t forget to make your encryption key available for case of restore! (using COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" ) is a option to consider). Be sure to read https://borgbackup.readthedocs.io/en/stable/usage/init.html, and make your self familiar how encryption in Borg works.
  • Executing rear mkbackup will create Relax-and-Recover rescue/recovery system and start Borg backup process. Once backup finishes, it will also prune old archives from repository, if at least one of BORGBACKUP_PRUNE_KEEP_* variables is set.

  • To recover your system, boot Relax-and-Recover rescue/recovery system and trigger rear recover. You will be prompted which archive to recover from Borg repository, once ReaR finished with layout configuration.

...
Disk layout created.
Starting Borg restore

=== Borg archives list ===
Host:       foo.bar.xy
Repository: /mnt/backup/client

[1] rear_1 	Sun, 2016-10-16 14:08:16
[2] rear_2 	Sun, 2016-10-16 14:32:11

[3] Exit

Choose archive to recover from:

Backup/restore alien file system using BLOCKCLONE and dd

Configuration

  • First we need to set some global options to local.conf

# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://beta.virtual.sk/mnt/rear
  • Now we can define variables that will apply only for targeted block device

# cat alien.conf
BACKUP=BLOCKCLONE                                        # Define BLOCKCLONE as backup method
BACKUP_PROG_ARCHIVE="alien"                              # Name of image file
BACKUP_PROG_SUFFIX=".dd.img"                             # Suffix of image file
BACKUP_PROG_COMPRESS_SUFFIX=""                           # Clear additional suffixes

BLOCKCLONE_PROG=dd                                       # Use dd for image creation
BLOCKCLONE_PROG_OPTS="bs=4k"                             # Additional options that will be passed to dd
BLOCKCLONE_SOURCE_DEV="/dev/sdc1"                        # Device that should be backed up

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdc"                       # Device where partitioning information is stored (optional)
BLOCKCLONE_MBR_FILE="alien_boot_strap.img"               # Output filename for boot strap code
BLOCKCLONE_PARTITIONS_CONF_FILE="alien_partitions.conf"  # Output filename for partition configuration
BLOCKCLONE_ALLOW_MOUNTED="yes"                           # Device can be mounted during backup (default NO)

Running backup

  • Save partitions configuration, bootstrap code and create actual backup of /dev/sdc1

# rear -C alien mkbackuponly
  • Running restore from ReaR restore/recovery system

# rear -C alien restoreonly

Restore alien.dd.img to device: [/dev/sdc1]                 # User is always prompted for restore destination
Device /dev/sdc1 was not found.                             # If destination does not exist ReaR will try to create it (or fail if BLOCKCLONE_SAVE_MBR_DEV was not set during backup)
Restore partition layout to (^c to abort): [/dev/sdc]       # Prompt user for device where partition configuration should be restored
Checking that no-one is using this disk right now ... OK

Disk /dev/sdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x10efb7a9.
Created a new partition 1 of type 'HPFS/NTFS/exFAT' and of size 120 MiB.

/dev/sdc2:
New situation:

Device     Boot Start    End Sectors  Size Id Type
/dev/sdc1        4096 249855  245760  120M  7 HPFS/NTFS/exFAT

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Using Relax-and-Recover with USB storage devices

Using USB devices with Relax-and-Recover can be appealing for several reasons:

  • If you only need to have a bootable rescue environment, a USB device is a cheap device for storing only 25 to 60MB to boot from

  • You can leave the USB device inserted in the system and opt-in booting from it only when disaster hits (although we do recommend storing rescue environments off-site)

  • You can store multiple systems and multiple snapshots on a single device

  • In case you have plenty of space, it might be a simple solution to store complete Disaster Recovery images (rescue + backup) on a single device for a set of systems

  • For migrating a bunch of servers having a single device to boot from might be very appealing

  • We have implemented a specific workflow: inserting a REAR-000 labeled USB stick will invoke rear udev and adds a rescue environment to the USB stick (updating the bootloader if needed)

However USB devices may be slow for backup purposes, especially on older systems or with unreliable/cheap devices.

Configuring Relax-and-Recover for USB storage devices

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.

BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
Important
On RHEL4 or older there are no /dev/disk/by-label/ udev aliases, which means we cannot use device by label. However it is possible to use by-path references, however this makes it very specific to the USB port used. We opted to use the complete device-name, which can be dangerous if you may have other /dev/sdX devices (luckily we have CCISS block devices in /dev/cciss/).

Preparing your USB storage device

To prepare your USB device for use with Relax-and-Recover, do: rear format /dev/sdX

This will create a single partition, make it bootable, format it with ext3, label it REAR-000 and disable warnings related filesystem check for the device.

USB storage as rescue media

Configuring Relax-and-Recover to have Bacula tools

If the rescue environment needs additional tools and workflow, this can be specified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:

BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

Making the rescue USB storage device

To create a rescue USB device, run rear -v mkrescue as shown below after you have inserted a REAR-000 labeled USB device. Make sure the device name for the USB device is what is configured for USB_DEVICE.

[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Finished in 72 seconds.
Warning
Doing the above may replace the existing MBR of the USB device. However any other content on the device is retained.

Booting from USB storage device

Before you can recover our DR backup, it is important to configure the BIOS to boot from the USB device. In some cases it is required to go into the BIOS setup (F9 during boot) to change the boot-order of devices. (In BIOS setup select Standard Boot Order (IPL))

Once booted from the USB device, select the system you like to recover from the list. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.

+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Recovery images"                          |
|    "system.localdomain"                   > |
|    "other.localdomain"                    > |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."
Warning
Booting from a local disk may fail when booting from a USB device. This is caused by the fact that the GRUB bootloader on the local disk is configured as if it is being the first drive (hd0) but it is in fact the second disk (hd1). If you do find menu entries not working from GRUB, please remove the root (hd0,0) line from the entry.

Then select the image you would like to recover.

+---------------------------------------------+
|           "system.localdomain"              |
+---------------------------------------------+
|  "2011-03-26 02:16 backup"                  |
|  "2011-03-25 18:39 backup"                  |
|  "2011-03-05 16:12 rescue image"            |
|---------------------------------------------|
|  "Back"                                     |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"


"Backup using kernel 2.6.32-122.el6.x86_64"
"BACKUP=NETFS OUTPUT=USB OUTPUT_URL=usb:///dev/disk/by-label/REAR-000"
Tip
When browsing through the images you get more information about the image at the bottom of the screen.

Restoring from USB rescue media

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>

Restoring from Bacula tape

Now you need to continue with restoring the actual Bacula backup, for this you have multiple options of which bextract is the most easy and straightforward, but also the slowest and unsafest.

Using a bootstrap file

If you know the JobId of the latest successful full backup, and differential backups the most efficient way to restore is by creating a bootstrap file with this information and using it to restore from tape.

A bootstrap file looks like this:

Volume = VOL-1234
JobId = 914
Job = Bkp_Daily

or

Volume = VOL-1234
VolSessionId = 1
VolSessionTime = 108927638

Using a bootstrap file with bextract is easy, simply do: bextract -b bootstrap.txt Ultrium-1 /mnt/local

Tip
It helps to know exactly how many files you need to restore, and using the FileIndex and Count keywords so bextract does not require to read the whole tape. Use the commands in your shell history to access an example Bacula bootstrap file.
Using bextract

To use bextract to restore everything from a single tape, you can do: bextract -V VOLUME-NAME Ultrium-1 /mnt/local

rear> bextract -V VOL-1234 Ultrium-1 /mnt/local
bextract: match.c:249-0 add_fname_to_include prefix=0 gzip=0 fname=/
bextract: butil.c:282 Using device: "Ultrium-1" for reading.
30-Mar 16:00 bextract JobId 0: Ready to read from volume "VOL-1234" on device "Ultrium-1" (/dev/st0).
bextract JobId 0: -rw-r-----   1 252      bacula     3623795 2011-03-30 11:02:18  /mnt/local/var/lib/bacula/bacula.sql
bextract JobId 0: drwxr-xr-x   2 root     root          4096 2011-02-02 11:48:28  *none*
bextract JobId 0: drwxr-xr-x   4 root     root          1024 2011-02-23 13:09:53  *none*
bextract JobId 0: drwxr-xr-x  12 root     root          4096 2011-02-02 11:50:00  *none*
bextract JobId 0: -rwx------   1 root     root             0 2011-02-02 11:48:24  /mnt/local/.hpshm_keyfile
bextract JobId 0: -rw-r--r--   1 root     root             0 2011-02-22 12:38:03  /mnt/local/.autofsck
...
30-Mar 16:06 bextract JobId 0: End of Volume at file 7 on device "Ultrium-1" (/dev/st0), Volume "VOL-1234"
30-Mar 16:06 bextract JobId 0: End of all volumes.
30-Mar 16:07 bextract JobId 0: Alert: smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
30-Mar 16:07 bextract JobId 0: Alert: Home page is http://smartmontools.sourceforge.net/
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: TapeAlert: OK
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: Error counter log:
30-Mar 16:07 bextract JobId 0: Alert:            Errors Corrected by           Total   Correction     Gigabytes    Total
30-Mar 16:07 bextract JobId 0: Alert:                ECC          rereads/    errors   algorithm      processed    uncorrected
30-Mar 16:07 bextract JobId 0: Alert:            fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
30-Mar 16:07 bextract JobId 0: Alert: read:       1546        0         0         0       1546          0.000           0
30-Mar 16:07 bextract JobId 0: Alert: write:         0        0         0         0          0          0.000           0
165719 files restored.
Warning
In this case bextract will restore all the Bacula jobs on the provided tapes, start from the oldest, down to the latest. As a consequence, deleted files may re-appear and the process may take a very long time.

Finish recovery process

Once finished, continue Relax-and-Recover by typing exit.

rear> exit
Did you restore the backup to /mnt/local ? Ready to continue ? y
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 4424 seconds.
Important
If you neglect to perform this last crucial step, your new system will not boot and you have to install a boot-loader yourself manually, or re-execute this procedure.

USB storage as backup media

Configuring Relax-and-Recover for backup to USB storage device

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

### Exclude certain items
ONLY_INCLUDE_VG=( vg00 )
EXCLUDE_MOUNTPOINTS=( /data )

Making the DR backup to USB storage device

Creating a combined rescue device that integrates the backup on USB, it is sufficient to run rear -v mkbackup as shown below after you have inserted the USB device. Make sure the device name for the USB device is what is configured.

[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Creating archive 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Total bytes written: 3644416000 (3.4GiB, 5.5MiB/s) in 637 seconds.
Writing MBR to /dev/sda
Modifying local GRUB configuration
Copying resulting files to usb location
Finished in 747 seconds.
Important
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory)

Booting from USB storage device

See the section Booting from USB storage device for more information about how to enable your BIOS to boot from a USB storage device.

Restoring a backup from USB storage device

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Backup archive size is 1.2G (compressed)
To recreate HP SmartArray controller 1, type exactly YES: YES
To recreate HP SmartArray controller 7, type exactly YES: YES
Clearing HP SmartArray controller 1
Clearing HP SmartArray controller 7
Recreating HP SmartArray controller 1|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 7|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c1d0 (msdos)
Creating software RAID /dev/md126
Creating software RAID /dev/md127
Creating LVM PV /dev/md127
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /boot on /dev/md126
Mounting filesystem /boot
Creating ext3-filesystem /data on /dev/mapper/vg00-lvdata
Mounting filesystem /data
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating swap on /dev/mapper/vg00-lvswap
Disk layout created.
Restoring from 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Restored 3478 MiB in 134 seconds [avg 26584 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 278 seconds.

If all is well, you can now remove the USB device, restore the BIOS boot order and reboot the system into the recovered OS.

Using Relax-and-Recover with OBDR tapes

Using One-Button-Disaster-Recovery (OBDR) tapes has a few benefits.

  • Within large organisations tape media is already part of a workflow for offsite storage and is a known and trusted technology

  • Tapes can store large amounts of data reliably and restoring large amounts of data is predictable in time and effort

  • OBDR offers booting from tapes, which is very convenient

  • A single tape can hold both the rescue image as well as a complete snapshot of the system (up to 1.6TB with LTO4)

However, you need one tape per system as an OBDR tape can only store one single rescue environment.

Configuring Relax-and-Recover for OBDR rescue tapes

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape only for storing the rescue image, the backup is expected to be handled by Bacula and so the Bacula tools are included in the rescue environment to enable a Bacula restore.

OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Preparing your OBDR rescue tape

To protect normal backup tapes (in case tape drives are also used by another backup solution) Relax-and-Recover expects that the tape to use is labeled REAR-000. To achieve this is to insert a blank tape to use for Relax-and-Recover and run the rear format /dev/stX command.

OBDR tapes as rescue media

Configuring Relax-and-Recover to have Bacula tools

If the rescue environment needs additional tools and workflow, this can be spcified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:

BACKUP=BACULA
OUTPUT=OBDR
BEXTRACT_DEVICE=Ultrium-1
BEXTRACT_VOLUME=VOL-*

Using the BEXTRACT_DEVICE allows you to use the tape device that is referenced from the Bacula configuration. This helps in those cases where the discovery of the various tape drives has already been done and configured in Bacula.

The BEXTRACT_VOLUME variable is optional and is only displayed in the restore instructions on screen as an aid during recovery.

Making the OBDR rescue tape

To create a rescue environment that can boot from an OBDR tape, simply run rear -v mkrescue with a REAR-000 -labeled tape inserted.

[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-dag-ops.iso (48M)
Writing ISO image to tape
Modifying local GRUB configuration
Finished in 119 seconds.
Warning
The message above about /dev/cciss/c1d0 not being used makes sense as this is not a real disk but simply an entry for manipulating the controller. This is specific to CCISS controllers with only a tape device attached.

Booting from OBDR rescue tape

The One Button Disaster Recovery (OBDR) functionality in HP LTO Ultrium drives enables them to emulate CD-ROM devices in specific circumstances (also known as being in ''Disaster Recovery'' mode). The drive can then act as a boot device for PCs that support booting off CD-ROM.

Tip
An OBDR capable drive can be switched into CD-ROM mode by powering on with the eject button held down. Make sure you keep it pressed when the tape drive regains power, and then release the button. If the drive is in OBDR mode, the light will blink regularly. This might be easier in some cases than the below procedure, hence the name One Button Disaster Recovery !
Using a HP Smart Array controller

To boot from OBDR, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the HP Smart Array controller with the tape attached by pressing F8 (or Escape-8 on serial console).

iLO 2 v1.78 Jun 10 2009 10.5.20.171

Slot 0 HP Smart Array P410i Controller       (512MB, v2.00)   1 Logical Drive
Slot 3 HP Smart Array P401 Controller        (512MB, v2.00)   1 Logical Drive
Slot 4 HP Smart Array P212 Controller          (0MB, v2.00)   0 Logical Drives
     Tape or CD-ROM Drive(s) Detected:
         Port 1I: Box 0: Bay 4
1785-Slot 4 Drive Array Not Configured
     No Drives Detected


  Press <F8> to run the Option ROM Configuration for Arrays Utility
  Press <ESC> to skip configuration and continue

Then select Configure OBDR in the menu and select the Tape drive by marking it with X (default is on) and press ENTER and F8 to activate this change so it displays ''Configuration saved''.

Then press ENTER and Escape to leave the Smart Array controller BIOS.

**** System will boot from Tape/CD/OBDR device attached to Smart Array.
Using an LSI controller

To boot from OBDR when using an LSI controller, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the LSI controller BIOS that has the tape attached by pressing F8 (or Escape-8 on serial console).

LSI Logic Corp. MPT BIOS
Copyright 1995-2006 LSI Logic Corp.
MPTBIOS-5.05.21.00
HP Build

<<<Press F8 for configuration options>>>

Then select the option 1. Tape-based One Button Disaster Recovery (OBDR).:

Select a configuration option:
1. Tape-based One Button Disaster Recovery (OBDR).
2. Multi Initiator Configuration.                                 <F9 = Setup>
3. Exit.

And then select the correct tape drive to boot from:

   compatible tape drives found       ->
   NUM   HBA   SCSI ID   Drive information
    0     0       A       - HP       Ultrium 2-SCSI

   Please choose the NUM of the tape drive to place into OBDR mode.

If all goes well, the system will reboot with OBDR-mode enabled:

    The PC will now reboot to begin Tape Recovery....

During the next boot, OBDR-mode will be indicate by:

*** Bootable media located, Using non-Emulation mode ***
Booting the OBDR tape

Once booted from the OBDR tape, select the 'Relax-and-Recover' menu entry from the menu. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.

+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Relax-and-Recover"                        |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."

Restoring the OBDR rescue tape

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>

Restoring from Bacula tape

See the section Restoring from Bacula tape for more information about how to restore a Bacula tape.

OBDR tapes as backup media

An OBDR backup tape is similar to an OBDR rescue tape, but next to the rescue environment, it also consists of a complete backup of the system. This is very convenient in that a single tape can be use for disaster recovery, and recovery is much more simple and completely automated.

Caution
Please make sure that the system fits onto a single tape uncompressed. For an LTO4 Ultrium that would mean no more than 1.6TB.

Configuring Relax-and-Recover for OBDR backup tapes

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape for storing both the rescue image and the backup.

BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Making the OBDR backup tape

To create a bootable backup tape that can boot from OBDR, simply run rear -v mkbackup with a REAR-000 -labeled tape inserted.

[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-system.iso (45M)
Writing ISO image to tape
Creating archive '/dev/nst0'
Total bytes written: 7834132480 (7.3GiB, 24MiB/s) in 317 seconds.
Rewinding tape
Modifying local GRUB configuration
Finished in 389 seconds.
Important
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory)

Booting from OBDR backup tape

See the section Booting from OBDR rescue tape for more information about how to enable OBDR and boot from OBDR tapes.

Restoring from OBDR backup tape

RESCUE SYSTEM:~ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Restoring LVM VG vgrem
Restoring LVM VG vgqry
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Restoring LVM VG vgbck
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Restoring LVM VG vgops
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.
Restoring from 'tape:///dev/nst0/system/backup.tar'
Restored 7460 MiB in 180 seconds [avg 42444 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 361 seconds.

Using ReaR to mount and repair your system

Instead of using your ReaR image to completely recover your system from bare metal (as illustrated in most of the above scenarios), you can also use it as a live media to boot a broken but hopefully repairable system.

Once booted on your recovery image, the mountonly workflow will:

  • activate all Volume Groups

  • offer to decrypt any LUKS-encrypted filesystem that may be present

  • mount all the target filesystems (including the most important virtual ones) below /mnt/local

thereby making it possible for you to explore your system at will, correcting any configuration mistake that may have prevented its startup, or allowing you to simply chroot into it and further repair it using its own administrative tools.

One important point to remember is that the mountonly workflow on its own won’t modify the target system in any way. Of course, once the target filesystems are mounted you, as the administrator, may decide to do so manually.

Beware: The mountonly workflow can only be used on the system where the rescue image was generated, as it bases its logic on the filesystem layout description file generated during the run of the mkrescue or mkbackup workflows.

Here are the steps you would typically follow:

Create your recovery image

Using any of the techniques described in the other scenarios, create a ReaR recovery image for your system (through rear mkrescue or rear mkbackup). If you only take the mountonly workflow into consideration, it doesn’t matter whether you also make a backup of your system or not (obviously, you’d better cover all your bases and make sure you’d be able to perform a full recover as well should the need occur).

Please note, that by default ReaR only includes in the recovery image the tools it will need to recover the system. If you anticipate the need for some extra tools in the context of a repair operation (e.g. tools that you might need in the event chroot`ing into the target system doesn’t work), you should make sure to include them in your recovery image by adding to the `PROGS or REQUIRED_PROGS configuration variables (please refer to the comments in default.conf for the exact meaning of each).

Booting on the recovery image

Arrange for the target system to boot on your recovery image as you would in any of the other scenarios.

Launching the "mount only" workflow

Issue the rear mountonly command to launch the workflow (that one is always verbose):

RESCUE pc-pan:~ # rear mountonly
Relax-and-Recover 2.5 / Git
Running rear mountonly (PID 625)
Using log file: /var/log/rear/rear-pc-pan.log
Running workflow mountonly within the ReaR rescue/recovery system
Comparing disks
Device sda has expected (same) size 34359738368 (will be used for 'mountonly')
Disk configuration looks identical
Proceed with 'mountonly' (yes) otherwise manual disk layout configuration is enforced
(default 'yes' timeout 30 seconds)
yes
User confirmed to proceed with 'mountonly'
Start target system mount.
Mounting filesystem /
Mounting filesystem /home
Mounting filesystem /boot/efi
Please enter the password for LUKS device cr_vg00-lvol4 (/dev/mapper/vg00-lvol4):
Enter passphrase for /dev/mapper/vg00-lvol4:
Mounting filesystem /products
Disk layout processed.
Finished 'mountonly'. The target system is mounted at '/mnt/local'.
Exiting rear mountonly (PID 625) and its descendant processes ...
Running exit tasks

As you can see in the output above, you will first be asked to confirm running the workflow (Proceed with 'mountonly') — simply press return. All the target filesystems should now be mounted below /mnt/local (including LUKS-encrypted ones if present and all needed virtual ones). In case any of them fails to mount, you will be offered to review the mount script and to re-execute it if needed.

Once the system is in the desired state, you can start exploring it, correcting any configuration mistake or filesystem corruption that prevented it from booting properly. In this state, the only tools at your disposal are those included by default in ReaR recovery image, or those you saw fit to add yourself (see above).

If this is not enough and you need to run the native administrative tools hosted inside your target system (such as YaST in the case of SUSE distributions), you are now in a position where you can chroot inside your system to reach them (chroot /mnt/local).

Closing the session

Once done, don’t forget to leave the chroot environment if applicable (Ctrl-D), then issue the shutdown command. This will ensure that all the target filesystems will be cleanly unmounted before the system is restarted.

You can’t perform that action at this time.