Permalink
Fetching contributors…
Cannot retrieve contributors at this time
1312 lines (1069 sloc) 47.5 KB

Scenarios

Bootable ISO

If you simply want a bootable ISO on a central server, you would do:

OUTPUT=ISO
OUTPUT_URL=http://server/path-to-push/

Bootable ISO with an external (commercial) backup software

If you rely on your backup software to do the full restore of a system then you could define:

OUTPUT=ISO
BACKUP=[TSM|NSR|DP|NBU|GALAXY10|SEP|DUPLICITY|BACULA|BAREOS|RBME|FDRUPSTREAM]

When using one of the above backup solution (commercial or open source) then there is no need to use rear mkbackup as the backup workflow would be empty. Just use rear mkrescue

ReaR will incorporate the needed executables and libraries of your chosen backup solution into the rescue image of ReaR.

Bootable ISO and storing archive on NFS/NAS

To create an ISO rescue image and using a central NFS/NAS server to store the archive, you could define:

OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL=nfs://remote-nfs-server/exports
#BACKUP_PROG_CRYPT_ENABLED=1
#BACKUP_PROG_CRYPT_KEY="my_Secret_pw"

The above example shows that it is even possible to protect the archives with a password. Be aware, that the password entry BACKUP_PROG_CRYPT_KEY="my_Secret_pw" will be deleted in the /etc/rear/local.conf file within the rescue image. So, it is important to remember the password (if you use the rescue image months later could cause problems I guess).

Bootable USB device with backup to USB

If you want a bootable USB device with a (tar) backup to USB as well, you would use:

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

Bootable tape drive (OBDR) with backup to tape

If you want an OBDR image and backup on tape, and use GNU tar for backup/restore, you would use:

BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Bootable tape drive (OBDR) and Bacula restore

If you want an OBDR image on tape, and the Bacula tools to recover your backup, use:

BACKUP=BACULA
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

ReaR with Borg back end

Important
We strongly recommend to use Borg standalone binary (https://github.com/borgbackup/borg/releases) as it includes all necessities for Borg operations. If you decide to go for different type of Borg installation types, make sure you include all needed files for Borg runtime into ReaR rescue/recovery system. E.g. by using COPY_AS_IS_BORG=( '/usr/lib64/python3.4*' '/usr/bin/python3*' '/usr/bin/pyvenv*' '/usr/lib/python3.4*' '/usr/lib64/libpython3*' )

Borg → SSH

  • Setup ssh key infrastructure for user that will be running backup. Issuing following command must work without any password prompts or remote host identity confirmation:

ssh <BORGBACKUP_USERNAME>@<BORGBACKUP_HOST>

  • Example local.conf:

OUTPUT=ISO
OUTPUT_URL=nfs://foo.bar.xy/mnt/backup/iso

BACKUP=BORG
BORGBACKUP_HOST="foo.bar.xy"
BORGBACKUP_USERNAME="borg_user"
BORGBACKUP_REPO="/mnt/backup/client"
BORGBACKUP_REMOTE_PATH="/usr/local/bin/borg"

# Automatic archive pruning
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-prune)
BORGBACKUP_PRUNE_WEEKLY=2

# Archive compression
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create)
BORGBACKUP_COMPRESSION="lzma,9"     # Slowest backup, best compression

# Repository encryption
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-init)
BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE="S3cr37_P455w0rD"
COPY_AS_IS_BORG=( "/root/.config/borg/keys/" )

# Borg environment variables
# (https://borgbackup.readthedocs.io/en/stable/usage.html#environment-variables)
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

Borg → USB

  • Example local.conf:

OUTPUT=USB
BACKUP=BORG

USB_DEVICE=/dev/disk/by-label/REAR-000

BORGBACKUP_REPO="/my_borg_backup"
BORGBACKUP_UMASK="0002"

BORGBACKUP_PRUNE_WEEKLY=2

BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE="S3cr37_P455w0rD"

export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" )
COPY_AS_IS_BORG=( '/root/.config/borg/keys/' )

SSH_UNPROTECTED_PRIVATE_KEYS="yes"
SSH_FILES="yes"
Important
If using BORGBACKUP_ENC_TYPE="keyfile", don’t forget to make your encryption key available for case of restore! (using COPY_AS_IS_BORG=( "/root/.config/borg/keys/" ) is a option to consider). Be sure to read https://borgbackup.readthedocs.io/en/stable/usage.html#borg-init, and make your self familiar how encryption in Borg works.
  • Executing rear mkbackup will create Relax-and-Recover rescue/recovery system and start Borg backup process. Once backup finishes, it will also prune old archives from repository, if at least one of BORGBACKUP_PRUNE_* variables is set.

  • To recover your system, boot Relax-and-Recover rescue/recovery system and trigger rear recover. You will be prompted which archive to recover from Borg repository, once ReaR finished with layout configuration.

...
Disk layout created.
Starting Borg restore

=== Borg archives list ===
Host:       foo.bar.xy
Repository: /mnt/backup/client

[1] rear_1 	Sun, 2016-10-16 14:08:16
[2] rear_2 	Sun, 2016-10-16 14:32:11

[3] Exit

Choose archive to recover from:

Backup/restore alien file system using BLOCKCLONE and dd

Configuration

  • First we need to set some global options to local.conf

# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://beta.virtual.sk/mnt/rear
  • Now we can define variables that will apply only for targeted block device

# cat alien.conf
BACKUP=BLOCKCLONE                                        # Define BLOCKCLONE as backup method
BACKUP_PROG_ARCHIVE="alien"                              # Name of image file
BACKUP_PROG_SUFFIX=".dd.img"                             # Suffix of image file
BACKUP_PROG_COMPRESS_SUFFIX=""                           # Clear additional suffixes

BLOCKCLONE_PROG=dd                                       # Use dd for image creation
BLOCKCLONE_PROG_OPTS="bs=4k"                             # Additional options that will be passed to dd
BLOCKCLONE_SOURCE_DEV="/dev/sdc1"                        # Device that should be backed up

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdc"                       # Device where partitioning information is stored (optional)
BLOCKCLONE_MBR_FILE="alien_boot_strap.img"               # Output filename for boot strap code
BLOCKCLONE_PARTITIONS_CONF_FILE="alien_partitions.conf"  # Output filename for partition configuration
BLOCKCLONE_ALLOW_MOUNTED="yes"                           # Device can be mounted during backup (default NO)

Running backup

  • Save partitions configuration, bootstrap code and create actual backup of /dev/sdc1

# rear -C alien mkbackuponly
  • Running restore from ReaR restore/recovery system

# rear -C alien restoreonly

Restore alien.dd.img to device: [/dev/sdc1]                 # User is always prompted for restore destination
Device /dev/sdc1 was not found.                             # If destination does not exist ReaR will try to create it (or fail if BLOCKCLONE_SAVE_MBR_DEV was not set during backup)
Restore partition layout to (^c to abort): [/dev/sdc]       # Prompt user for device where partition configuration should be restored
Checking that no-one is using this disk right now ... OK

Disk /dev/sdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x10efb7a9.
Created a new partition 1 of type 'HPFS/NTFS/exFAT' and of size 120 MiB.

/dev/sdc2:
New situation:

Device     Boot Start    End Sectors  Size Id Type
/dev/sdc1        4096 249855  245760  120M  7 HPFS/NTFS/exFAT

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Using Relax-and-Recover with USB storage devices

Using USB devices with Relax-and-Recover can be appealing for several reasons:

  • If you only need to have a bootable rescue environment, a USB device is a cheap device for storing only 25 to 60MB to boot from

  • You can leave the USB device inserted in the system and opt-in booting from it only when disaster hits (although we do recommend storing rescue environments off-site)

  • You can store multiple systems and multiple snapshots on a single device

  • In case you have plenty of space, it might be a simple solution to store complete Disaster Recovery images (rescue + backup) on a single device for a set of systems

  • For migrating a bunch of servers having a single device to boot from might be very appealing

  • We have implemented a specific workflow: inserting a REAR-000 labeled USB stick will invoke rear udev and adds a rescue environment to the USB stick (updating the bootloader if needed)

However USB devices may be slow for backup purposes, especially on older systems or with unreliable/cheap devices.

Configuring Relax-and-Recover for USB storage devices

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.

BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
Important
On RHEL4 or older there are no /dev/disk/by-label/ udev aliases, which means we cannot use device by label. However it is possible to use by-path references, however this makes it very specific to the USB port used. We opted to use the complete device-name, which can be dangerous if you may have other /dev/sdX devices (luckily we have CCISS block devices in /dev/cciss/).

Preparing your USB storage device

To prepare your USB device for use with Relax-and-Recover, do: rear format /dev/sdX

This will create a single partition, make it bootable, format it with ext3, label it REAR-000 and disable warnings related filesystem check for the device.

USB storage as rescue media

Configuring Relax-and-Recover to have Bacula tools

If the rescue environment needs additional tools and workflow, this can be specified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:

BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

Making the rescue USB storage device

To create a rescue USB device, run rear -v mkrescue as shown below after you have inserted a REAR-000 labeled USB device. Make sure the device name for the USB device is what is configured for USB_DEVICE.

[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Finished in 72 seconds.
Warning
Doing the above may replace the existing MBR of the USB device. However any other content on the device is retained.

Booting from USB storage device

Before you can recover our DR backup, it is important to configure the BIOS to boot from the USB device. In some cases it is required to go into the BIOS setup (F9 during boot) to change the boot-order of devices. (In BIOS setup select Standard Boot Order (IPL))

Once booted from the USB device, select the system you like to recover from the list. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.

+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Recovery images"                          |
|    "system.localdomain"                   > |
|    "other.localdomain"                    > |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."
Warning
Booting from a local disk may fail when booting from a USB device. This is caused by the fact that the GRUB bootloader on the local disk is configured as if it is being the first drive (hd0) but it is in fact the second disk (hd1). If you do find menu entries not working from GRUB, please remove the root (hd0,0) line from the entry.

Then select the image you would like to recover.

+---------------------------------------------+
|           "system.localdomain"              |
+---------------------------------------------+
|  "2011-03-26 02:16 backup"                  |
|  "2011-03-25 18:39 backup"                  |
|  "2011-03-05 16:12 rescue image"            |
|---------------------------------------------|
|  "Back"                                     |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"


"Backup using kernel 2.6.32-122.el6.x86_64"
"BACKUP=NETFS OUTPUT=USB OUTPUT_URL=usb:///dev/disk/by-label/REAR-000"
Tip
When browsing through the images you get more information about the image at the bottom of the screen.

Restoring from USB rescue media

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>

Restoring from Bacula tape

Now you need to continue with restoring the actual Bacula backup, for this you have multiple options of which bextract is the most easy and straightforward, but also the slowest and unsafest.

Using a bootstrap file

If you know the JobId of the latest successful full backup, and differential backups the most efficient way to restore is by creating a bootstrap file with this information and using it to restore from tape.

A bootstrap file looks like this:

Volume = VOL-1234
JobId = 914
Job = Bkp_Daily

or

Volume = VOL-1234
VolSessionId = 1
VolSessionTime = 108927638

Using a bootstrap file with bextract is easy, simply do: bextract -b bootstrap.txt Ultrium-1 /mnt/local

Tip
It helps to know exactly how many files you need to restore, and using the FileIndex and Count keywords so bextract does not require to read the whole tape. Use the commands in your shell history to access an example Bacula bootstrap file.
Using bextract

To use bextract to restore everything from a single tape, you can do: bextract -V VOLUME-NAME Ultrium-1 /mnt/local

rear> bextract -V VOL-1234 Ultrium-1 /mnt/local
bextract: match.c:249-0 add_fname_to_include prefix=0 gzip=0 fname=/
bextract: butil.c:282 Using device: "Ultrium-1" for reading.
30-Mar 16:00 bextract JobId 0: Ready to read from volume "VOL-1234" on device "Ultrium-1" (/dev/st0).
bextract JobId 0: -rw-r-----   1 252      bacula     3623795 2011-03-30 11:02:18  /mnt/local/var/lib/bacula/bacula.sql
bextract JobId 0: drwxr-xr-x   2 root     root          4096 2011-02-02 11:48:28  *none*
bextract JobId 0: drwxr-xr-x   4 root     root          1024 2011-02-23 13:09:53  *none*
bextract JobId 0: drwxr-xr-x  12 root     root          4096 2011-02-02 11:50:00  *none*
bextract JobId 0: -rwx------   1 root     root             0 2011-02-02 11:48:24  /mnt/local/.hpshm_keyfile
bextract JobId 0: -rw-r--r--   1 root     root             0 2011-02-22 12:38:03  /mnt/local/.autofsck
...
30-Mar 16:06 bextract JobId 0: End of Volume at file 7 on device "Ultrium-1" (/dev/st0), Volume "VOL-1234"
30-Mar 16:06 bextract JobId 0: End of all volumes.
30-Mar 16:07 bextract JobId 0: Alert: smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
30-Mar 16:07 bextract JobId 0: Alert: Home page is http://smartmontools.sourceforge.net/
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: TapeAlert: OK
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: Error counter log:
30-Mar 16:07 bextract JobId 0: Alert:            Errors Corrected by           Total   Correction     Gigabytes    Total
30-Mar 16:07 bextract JobId 0: Alert:                ECC          rereads/    errors   algorithm      processed    uncorrected
30-Mar 16:07 bextract JobId 0: Alert:            fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
30-Mar 16:07 bextract JobId 0: Alert: read:       1546        0         0         0       1546          0.000           0
30-Mar 16:07 bextract JobId 0: Alert: write:         0        0         0         0          0          0.000           0
165719 files restored.
Warning
In this case bextract will restore all the Bacula jobs on the provided tapes, start from the oldest, down to the latest. As a consequence, deleted files may re-appear and the process may take a very long time.

Finish recovery process

Once finished, continue Relax-and-Recover by typing exit.

rear> exit
Did you restore the backup to /mnt/local ? Ready to continue ? y
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 4424 seconds.
Important
If you neglect to perform this last crucial step, your new system will not boot and you have to install a boot-loader yourself manually, or re-execute this procedure.

USB storage as backup media

Configuring Relax-and-Recover for backup to USB storage device

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

### Exclude certain items
ONLY_INCLUDE_VG=( vg00 )
EXCLUDE_MOUNTPOINTS=( /data )

Making the DR backup to USB storage device

Creating a combined rescue device that integrates the backup on USB, it is sufficient to run rear -v mkbackup as shown below after you have inserted the USB device. Make sure the device name for the USB device is what is configured.

[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Creating archive 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Total bytes written: 3644416000 (3.4GiB, 5.5MiB/s) in 637 seconds.
Writing MBR to /dev/sda
Modifying local GRUB configuration
Copying resulting files to usb location
Finished in 747 seconds.
Important
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory)

Booting from USB storage device

See the section Booting from USB storage device for more information about how to enable your BIOS to boot from a USB storage device.

Restoring a backup from USB storage device

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Backup archive size is 1.2G (compressed)
To recreate HP SmartArray controller 1, type exactly YES: YES
To recreate HP SmartArray controller 7, type exactly YES: YES
Clearing HP SmartArray controller 1
Clearing HP SmartArray controller 7
Recreating HP SmartArray controller 1|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 7|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c1d0 (msdos)
Creating software RAID /dev/md126
Creating software RAID /dev/md127
Creating LVM PV /dev/md127
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /boot on /dev/md126
Mounting filesystem /boot
Creating ext3-filesystem /data on /dev/mapper/vg00-lvdata
Mounting filesystem /data
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating swap on /dev/mapper/vg00-lvswap
Disk layout created.
Restoring from 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Restored 3478 MiB in 134 seconds [avg 26584 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 278 seconds.

If all is well, you can now remove the USB device, restore the BIOS boot order and reboot the system into the recovered OS.

Using Relax-and-Recover with OBDR tapes

Using One-Button-Disaster-Recovery (OBDR) tapes has a few benefits.

  • Within large organisations tape media is already part of a workflow for offsite storage and is a known and trusted technology

  • Tapes can store large amounts of data reliably and restoring large amounts of data is predictable in time and effort

  • OBDR offers booting from tapes, which is very convenient

  • A single tape can hold both the rescue image as well as a complete snapshot of the system (up to 1.6TB with LTO4)

However, you need one tape per system as an OBDR tape can only store one single rescue environment.

Configuring Relax-and-Recover for OBDR rescue tapes

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape only for storing the rescue image, the backup is expected to be handled by Bacula and so the Bacula tools are included in the rescue environment to enable a Bacula restore.

OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Preparing your OBDR rescue tape

To protect normal backup tapes (in case tape drives are also used by another backup solution) Relax-and-Recover expects that the tape to use is labeled REAR-000. To achieve this is to insert a blank tape to use for Relax-and-Recover and run the rear format /dev/stX command.

OBDR tapes as rescue media

Configuring Relax-and-Recover to have Bacula tools

If the rescue environment needs additional tools and workflow, this can be spcified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:

BACKUP=BACULA
OUTPUT=OBDR
BEXTRACT_DEVICE=Ultrium-1
BEXTRACT_VOLUME=VOL-*

Using the BEXTRACT_DEVICE allows you to use the tape device that is referenced from the Bacula configuration. This helps in those cases where the discovery of the various tape drives has already been done and configured in Bacula.

The BEXTRACT_VOLUME variable is optional and is only displayed in the restore instructions on screen as an aid during recovery.

Making the OBDR rescue tape

To create a rescue environment that can boot from an OBDR tape, simply run rear -v mkrescue with a REAR-000 -labeled tape inserted.

[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-dag-ops.iso (48M)
Writing ISO image to tape
Modifying local GRUB configuration
Finished in 119 seconds.
Warning
The message above about /dev/cciss/c1d0 not being used makes sense as this is not a real disk but simply an entry for manipulating the controller. This is specific to CCISS controllers with only a tape device attached.

Booting from OBDR rescue tape

The One Button Disaster Recovery (OBDR) functionality in HP LTO Ultrium drives enables them to emulate CD-ROM devices in specific circumstances (also known as being in ''Disaster Recovery'' mode). The drive can then act as a boot device for PCs that support booting off CD-ROM.

Tip
An OBDR capable drive can be switched into CD-ROM mode by powering on with the eject button held down. Make sure you keep it pressed when the tape drive regains power, and then release the button. If the drive is in OBDR mode, the light will blink regularly. This might be easier in some cases than the below procedure, hence the name One Button Disaster Recovery !
Using a HP Smart Array controller

To boot from OBDR, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the HP Smart Array controller with the tape attached by pressing F8 (or Escape-8 on serial console).

iLO 2 v1.78 Jun 10 2009 10.5.20.171

Slot 0 HP Smart Array P410i Controller       (512MB, v2.00)   1 Logical Drive
Slot 3 HP Smart Array P401 Controller        (512MB, v2.00)   1 Logical Drive
Slot 4 HP Smart Array P212 Controller          (0MB, v2.00)   0 Logical Drives
     Tape or CD-ROM Drive(s) Detected:
         Port 1I: Box 0: Bay 4
1785-Slot 4 Drive Array Not Configured
     No Drives Detected


  Press <F8> to run the Option ROM Configuration for Arrays Utility
  Press <ESC> to skip configuration and continue

Then select Configure OBDR in the menu and select the Tape drive by marking it with X (default is on) and press ENTER and F8 to activate this change so it displays ''Configuration saved''.

Then press ENTER and Escape to leave the Smart Array controller BIOS.

**** System will boot from Tape/CD/OBDR device attached to Smart Array.
Using an LSI controller

To boot from OBDR when using an LSI controller, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the LSI controller BIOS that has the tape attached by pressing F8 (or Escape-8 on serial console).

LSI Logic Corp. MPT BIOS
Copyright 1995-2006 LSI Logic Corp.
MPTBIOS-5.05.21.00
HP Build

<<<Press F8 for configuration options>>>

Then select the option 1. Tape-based One Button Disaster Recovery (OBDR).:

Select a configuration option:
1. Tape-based One Button Disaster Recovery (OBDR).
2. Multi Initiator Configuration.                                 <F9 = Setup>
3. Exit.

And then select the correct tape drive to boot from:

   compatible tape drives found       ->
   NUM   HBA   SCSI ID   Drive information
    0     0       A       - HP       Ultrium 2-SCSI

   Please choose the NUM of the tape drive to place into OBDR mode.

If all goes well, the system will reboot with OBDR-mode enabled:

    The PC will now reboot to begin Tape Recovery....

During the next boot, OBDR-mode will be indicate by:

*** Bootable media located, Using non-Emulation mode ***
Booting the OBDR tape

Once booted from the OBDR tape, select the 'Relax-and-Recover' menu entry from the menu. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.

+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Relax-and-Recover"                        |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."

Restoring the OBDR rescue tape

Then wait for the system to boot until you get the prompt.

On the shell prompt, type rear recover.

You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.

RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>

Restoring from Bacula tape

See the section Restoring from Bacula tape for more information about how to restore a Bacula tape.

OBDR tapes as backup media

An OBDR backup tape is similar to an OBDR rescue tape, but next to the rescue environment, it also consists of a complete backup of the system. This is very convenient in that a single tape can be use for disaster recovery, and recovery is much more simple and completely automated.

Caution
Please make sure that the system fits onto a single tape uncompressed. For an LTO4 Ultrium that would mean no more than 1.6TB.

Configuring Relax-and-Recover for OBDR backup tapes

The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape for storing both the rescue image and the backup.

BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0

Making the OBDR backup tape

To create a bootable backup tape that can boot from OBDR, simply run rear -v mkbackup with a REAR-000 -labeled tape inserted.

[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-system.iso (45M)
Writing ISO image to tape
Creating archive '/dev/nst0'
Total bytes written: 7834132480 (7.3GiB, 24MiB/s) in 317 seconds.
Rewinding tape
Modifying local GRUB configuration
Finished in 389 seconds.
Important
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory)

Booting from OBDR backup tape

See the section Booting from OBDR rescue tape for more information about how to enable OBDR and boot from OBDR tapes.

Restoring from OBDR backup tape

RESCUE SYSTEM:~ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Restoring LVM VG vgrem
Restoring LVM VG vgqry
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Restoring LVM VG vgbck
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Restoring LVM VG vgops
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.
Restoring from 'tape:///dev/nst0/system/backup.tar'
Restored 7460 MiB in 180 seconds [avg 42444 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 361 seconds.