If you simply want a bootable ISO on a central server, you would do:
OUTPUT=ISO
OUTPUT_URL=http://server/path-to-push/
If you rely on your backup software to do the full restore of a system then you could define:
OUTPUT=ISO
BACKUP=[TSM|NSR|DP|NBU|GALAXY10|SEP|DUPLICITY|BACULA|BAREOS|RBME|FDRUPSTREAM]
When using one of the above backup solution (commercial or open source) then there is no need to use rear mkbackup as the backup workflow would be empty. Just use rear mkrescue
ReaR will incorporate the needed executables and libraries of your chosen backup solution into the rescue image of ReaR.
To create an ISO rescue image and using a central NFS/NAS server to store it together with the backup archive, you could define:
OUTPUT=ISO
BACKUP=NETFS
# BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://your.NFS.server.IP/path/to/your/rear/backup
# BACKUP_PROG_CRYPT_ENABLED="yes"
# { BACKUP_PROG_CRYPT_KEY='my_secret_passphrase' ; } 2>/dev/null
The above example shows that it is also possible to encrypt the backup archive.
Currently only 'tar' is supported for backup archive encryption and decryption.
When BACKUP_PROG_CRYPT_ENABLED is set to a true value, BACKUP_PROG_CRYPT_KEY must be also set.
There is no BACKUP_PROG_CRYPT_KEY value in the /etc/rear/local.conf file in the rescue image.
It gets removed because the ReaR rescue/recovery system must be free of secrets.
Otherwise the rescue system ISO image and any recovery medium that is made from it
would have to be carefully protected against any unwanted access.
Therefore BACKUP_PROG_CRYPT_KEY must be manually set before running "rear recover".
For example in the running ReaR rescue system via
export BACKUP_PROG_CRYPT_KEY='my_secret_passphrase'
before calling "rear recover".
You may also do this on the original system before calling "rear mkbackup"
so that there is no need to store it ever in a ReaR config file
but then ensure to run commands that contain 'my_secret_passphrase'
without keeping the command in a history file (e.g. ~/.bash_history).
On the other hand it is crucial to remember the BACKUP_PROG_CRYPT_KEY value
that was used during "rear mkbackup" so that possibly a long time later that
rescue image can be used (possibly by someone else) to recover from a disaster.
If the BACKUP_PROG_CRYPT_KEY value should be set in a ReaR config file
you should avoid that the BACKUP_PROG_CRYPT_KEY value is shown in a log file
when 'rear' is run in debugscript mode (where 'set -x' is set) by redirecting
STDERR to /dev/null via { command confidential_argument ; } 2>/dev/null
where the redirection must be done via a compound group command even for
a single confidential command to let the redirection also apply for 'set -x'.
See the description in /usr/share/rear/conf/default.conf
how to set variables for secret values in a confidential way.
If you want to create the rescue initramfs with a (tar) backup stored on NFS
server, add the following settings to /etc/rear/local.conf
:
OUTPUT=RAMDISK
OUTPUT_URL=file:///path/to/initramfs
BACKUP=NETFS
BACKUP_URL=nfs://NFS.server.IP/path/to/backup
Make sure that the path in OUTPUT_URL
exists and create the rescue initramfs
and backup using rear mkbackup
.
There are two options how to boot the rescue environment on IBM Z (s390x) under
z/VM created using the OUTPUT=RAMDISK
setting.
-
Using the
kexec
tool as described in RESULT_usage_RAMDISK.txt, if you have access to a running Linux instance. -
Using the
zipl
tool for management of boot devices on s390/s390x.
First, we have to select a partition on the disk that we want to make bootable.
Assume that the disk is called /dev/dasda
and the partition with the filesystem
that will contain the kernel and initramfs is /dev/dasda1
.
We mount this partition to /mnt/bootdisk
and copy there the kernel and initramfs
created in the previous section:
# mount /dev/dasda1 /mnt/bootdisk
# cp /path/to/initramfs/HOSTNAME/* /mnt/bootdisk/
Note
|
See zipl(8) for all device types supported by the --target option
(e.g. DASD, SCSI, …). If you want to use a tape device, see zipl(8)
to use the --tape option instead. Note that the --tape option with
ReaR is untested so contributions to this guide are welcome! Also, it is
not possible to use zipl with the z/VM reader device.
|
Important
|
The following method will overwrite the boot configuration on a device
which contains the given --target directory, in our case
/dev/dasda !
|
Use the following command to make the device mounted at /mnt/bootdisk
bootable
with zipl
:
# zipl --target=/mnt/bootdisk \
--parameters='root=/dev/ram0 ro' \
--image=/mnt/bootdisk/kernel-HOSTNAME \
--ramdisk=/mnt/bootdisk/initramfs-HOSTNAME.img
Run /lib/s390-tools//zipl_helper.device-mapper /mnt/bootdisk/
Building bootmap in '/mnt/bootdisk/'
Adding IPL section
initial ramdisk...: /mnt/bootdisk/initramfs-HOSTNAME.img
kernel image......: /mnt/bootdisk/kernel-HOSTNAME
kernel parmline...: 'root=/dev/ram0 ro'
component address:
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x00009fff
kernel image....: 0x00010000-0x0076afff
parmline........: 0x0076c000-0x0076cfff
initial ramdisk.: 0x00780000-0x28708fff
environment blk.: 0x0077c000-0x0077cfff
Preparing boot device for CCW- and LD-IPL: dasda.
Done.
If you want to boot the device with ReaR recovery from a running Linux
instance, you can use the chreipl
tool to select a new default boot device
and reboot:
# chreipl /mnt/bootdisk
Re-IPL type: ccw
Device: 0.0.0120
Loadparm: ""
Bootparms: ""
clear: 0
# reboot
...
If you do not have a running Linux instance at hand, you can force the boot from the given device using the z/VM Control Program (CP) directly, e.g. through a terminal connection.
The following example only considers DASD disk devices. Consult the official z/VM CP Manual for booting from other devices (e.g. FCP attached SCSI) or for additional details in general.
Use the following command to boot from a DASD boot device through z/VM CP,
where the last argument corresponds to the virtual device number (vdev
).
CP IPL 0120
The virtual device number can be either obtained from the Bus-ID
column using
the lsdasd -s
command from a running Linux instance:
# lsdasd -s
Bus-ID Status Name Device Type BlkSz Size Blocks
================================================================================
0120 active dasda 94:0 ECKD 4096 39128MB 10017000
or directly from z/VM CP with the following query:
CP QUERY VIRTUAL DASD
00:
00: CP QUERY VIRTUAL DASD
00: DASD 0120 ON DASD 2213 R/W 0X0120 SUBCHANNEL = 0004
00: DASD 0121 ON DASD 201A R/W 0X0121 SUBCHANNEL = 0005
...
If you want a bootable USB device with a (tar) backup to USB as well, you would use:
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
If you want an OBDR image and backup on tape, and use GNU tar for backup/restore, you would use:
BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
If you want an OBDR image on tape, and the Bacula tools to recover your backup, use:
BACKUP=BACULA
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
-
Install Borg backup (https://borgbackup.readthedocs.io/en/stable/installation.html).
Important
|
We strongly recommend to use Borg standalone binary (https://github.com/borgbackup/borg/releases) as it includes all necessities for Borg operations.
If you decide to go for different type of Borg installation types, make sure you include all needed files for Borg runtime into ReaR rescue/recovery system.
E.g. by using COPY_AS_IS_BORG=( '/usr/lib64/python3.4*' '/usr/bin/python3*' '/usr/bin/pyvenv*' '/usr/lib/python3.4*' '/usr/lib64/libpython3*' )
|
-
Setup ssh key infrastructure for user that will be running backup. Issuing following command must work without any password prompts or remote host identity confirmation:
ssh <BORGBACKUP_USERNAME>@<BORGBACKUP_HOST>
-
Example local.conf:
OUTPUT=ISO OUTPUT_URL=nfs://foo.bar.xy/mnt/backup/iso BACKUP=BORG BORGBACKUP_HOST="foo.bar.xy" BORGBACKUP_USERNAME="borg_user" BORGBACKUP_REPO="/mnt/backup/client" BORGBACKUP_REMOTE_PATH="/usr/local/bin/borg" # Automatic archive pruning # (https://borgbackup.readthedocs.io/en/stable/usage/prune.html) BORGBACKUP_PRUNE_KEEP_WEEKLY=2 # Archive compression # (https://borgbackup.readthedocs.io/en/stable/usage/create.html) BORGBACKUP_COMPRESSION="lzma,6" # Slowest backup, best compression # Repository encryption # (https://borgbackup.readthedocs.io/en/stable/usage/init.html) BORGBACKUP_ENC_TYPE="keyfile" export BORG_PASSPHRASE='S3cr37_P455w0rD' COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" ) # Borg environment variables # (https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables) export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes" export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"
-
Example local.conf:
OUTPUT=USB BACKUP=BORG USB_DEVICE=/dev/disk/by-label/REAR-000 BORGBACKUP_REPO="/my_borg_backup" BORGBACKUP_UMASK="0002" BORGBACKUP_PRUNE_KEEP_WEEKLY=2 BORGBACKUP_ENC_TYPE="keyfile" export BORG_PASSPHRASE='S3cr37_P455w0rD' export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes" export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes" COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" ) COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" ) SSH_UNPROTECTED_PRIVATE_KEYS="yes" SSH_FILES="yes"
Important
|
If using BORGBACKUP_ENC_TYPE="keyfile", don’t forget to make your
encryption key available for case of restore!
(using COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" ) is a option to consider).
Be sure to read https://borgbackup.readthedocs.io/en/stable/usage/init.html,
and make your self familiar how encryption in Borg works.
|
-
Executing
rear mkbackup
will create Relax-and-Recover rescue/recovery system and start Borg backup process. Once backup finishes, it will also prune old archives from repository, if at least one ofBORGBACKUP_PRUNE_KEEP_*
variables is set. -
To recover your system, boot Relax-and-Recover rescue/recovery system and trigger
rear recover
. You will be prompted which archive to recover from Borg repository, once ReaR finished with layout configuration.
...
Disk layout created.
Starting Borg restore
=== Borg archives list ===
Host: foo.bar.xy
Repository: /mnt/backup/client
[1] rear_1 Sun, 2016-10-16 14:08:16
[2] rear_2 Sun, 2016-10-16 14:32:11
[3] Exit
Choose archive to recover from:
-
First we need to set some global options to local.conf
# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://beta.virtual.sk/mnt/rear
-
Now we can define variables that will apply only for targeted block device
# cat alien.conf
BACKUP=BLOCKCLONE # Define BLOCKCLONE as backup method
BACKUP_PROG_ARCHIVE="alien" # Name of image file
BACKUP_PROG_SUFFIX=".dd.img" # Suffix of image file
BACKUP_PROG_COMPRESS_SUFFIX="" # Clear additional suffixes
BLOCKCLONE_PROG=dd # Use dd for image creation
BLOCKCLONE_PROG_OPTS="bs=4k" # Additional options that will be passed to dd
BLOCKCLONE_SOURCE_DEV="/dev/sdc1" # Device that should be backed up
BLOCKCLONE_SAVE_MBR_DEV="/dev/sdc" # Device where partitioning information is stored (optional)
BLOCKCLONE_MBR_FILE="alien_boot_strap.img" # Output filename for boot strap code
BLOCKCLONE_PARTITIONS_CONF_FILE="alien_partitions.conf" # Output filename for partition configuration
BLOCKCLONE_ALLOW_MOUNTED="yes" # Device can be mounted during backup (default NO)
-
Save partitions configuration, bootstrap code and create actual backup of /dev/sdc1
# rear -C alien mkbackuponly
-
Running restore from ReaR restore/recovery system
# rear -C alien restoreonly
Restore alien.dd.img to device: [/dev/sdc1] # User is always prompted for restore destination
Device /dev/sdc1 was not found. # If destination does not exist ReaR will try to create it (or fail if BLOCKCLONE_SAVE_MBR_DEV was not set during backup)
Restore partition layout to (^c to abort): [/dev/sdc] # Prompt user for device where partition configuration should be restored
Checking that no-one is using this disk right now ... OK
Disk /dev/sdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x10efb7a9.
Created a new partition 1 of type 'HPFS/NTFS/exFAT' and of size 120 MiB.
/dev/sdc2:
New situation:
Device Boot Start End Sectors Size Id Type
/dev/sdc1 4096 249855 245760 120M 7 HPFS/NTFS/exFAT
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Using USB devices with Relax-and-Recover can be appealing for several reasons:
-
If you only need to have a bootable rescue environment, a USB device is a cheap device to store it (needs up to about 1 GB)
-
When you have plenty of space, it could be a simple solution to store complete disaster recovery images (rescue environment + backup)
However USB devices may be slow for backup purposes, especially on older systems and/or with cheap devices.
The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.
BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
To prepare your USB device for use with Relax-and-Recover, do: rear format /dev/sdX
This will will destroy all existing data on that device, create new partitions and filesystems, make it bootable, and label it REAR-000
If the rescue environment needs additional tools and workflow, this can be specified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:
BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
To create a rescue USB device, run rear -v mkrescue as shown below after you have inserted a REAR-000 labeled USB device.
[root@system ~]# rear -v mkrescue Relax-and-Recover 1.12.0svn497 / 2011-07-11 Creating disk layout. Creating root filesystem layout Copying files and directories Copying program files and libraries Copying kernel modules Creating initramfs Finished in 72 seconds.
Before you can recover our DR backup, it is important to configure the BIOS to boot from the USB device. In some cases it is required to go into the BIOS setup (F9 during boot) to change the boot-order of devices. (In BIOS setup select Standard Boot Order (IPL))
Once booted from the USB device, select the system you like to recover from the list. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.
+---------------------------------------------+ | "Relax-and-Recover v1.12.0svn497" | +---------------------------------------------+ | "Recovery images" | | "system.localdomain" > | | "other.localdomain" > | |---------------------------------------------| | "Other actions" | | "Help for Relax-and-Recover" | | "Boot Local disk (hd1)" | | "Boot BIOS disk (0x81)" | | "Boot Next BIOS device" | | "Hardware Detection tool" | | "Memory test" | | "Reboot system" | | "Power off system" | +---------------------------------------------+ "Press [Tab] to edit options or [F1] for help" "Automatic boot in 30 seconds..."
Warning
|
Booting from a local disk may fail when booting from a USB device. This is caused by the fact that the GRUB bootloader on the local disk is configured as if it is being the first drive (hd0) but it is in fact the second disk (hd1). |
Then select the image you would like to recover.
+---------------------------------------------+ | "system.localdomain" | +---------------------------------------------+ | "2011-03-26 02:16 backup" | | "2011-03-25 18:39 backup" | | "2011-03-05 16:12 rescue image" | |---------------------------------------------| | "Back" | | | | | | | | | | | | | | | | | +---------------------------------------------+ "Press [Tab] to edit options or [F1] for help" "Backup using kernel 2.6.32-122.el6.x86_64" "BACKUP=NETFS OUTPUT=USB OUTPUT_URL=usb:///dev/disk/by-label/REAR-000"
Tip
|
When browsing through the images you get more information about the image at the bottom of the screen. |
Then wait for the system to boot until you get the prompt.
On the shell prompt, type rear recover.
You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.
RESCUE SYSTEM:/ # rear recover Relax-and-Recover 1.12.0svn497 / 2011-07-11 NOTICE: Will do driver migration To recreate HP SmartArray controller 3, type exactly YES: YES To recreate HP SmartArray controller 0, type exactly YES: YES Clearing HP SmartArray controller 3 Clearing HP SmartArray controller 0 Recreating HP SmartArray controller 3|A Configuration restored successfully, reloading CCISS driver... OK Recreating HP SmartArray controller 0|A Configuration restored successfully, reloading CCISS driver... OK Comparing disks. Disk configuration is identical, proceeding with restore. Type "Yes" if you want DRBD resource rBCK to become primary: Yes Type "Yes" if you want DRBD resource rOPS to become primary: Yes Start system layout restoration. Creating partitions for disk /dev/cciss/c0d0 (msdos) Creating partitions for disk /dev/cciss/c2d0 (msdos) Creating software RAID /dev/md2 Creating software RAID /dev/md6 Creating software RAID /dev/md3 Creating software RAID /dev/md4 Creating software RAID /dev/md5 Creating software RAID /dev/md1 Creating software RAID /dev/md0 Creating LVM PV /dev/md6 Creating LVM PV /dev/md5 Creating LVM PV /dev/md2 Creating LVM VG vgrem Creating LVM VG vgqry Creating LVM VG vg00 Creating LVM volume vg00/lv00 Creating LVM volume vg00/lvdstpol Creating LVM volume vg00/lvsys Creating LVM volume vg00/lvusr Creating LVM volume vg00/lvtmp Creating LVM volume vg00/lvvar Creating LVM volume vg00/lvopt Creating ext3-filesystem / on /dev/mapper/vg00-lv00 Mounting filesystem / Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol Mounting filesystem /dstpol Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys Mounting filesystem /dstpol/sys Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr Mounting filesystem /usr Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp Mounting filesystem /tmp Creating ext3-filesystem /boot on /dev/md0 Mounting filesystem /boot Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar Mounting filesystem /var Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt Mounting filesystem /opt Creating swap on /dev/md1 Creating DRBD resource rBCK Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd2 Creating LVM VG vgbck Creating LVM volume vgbck/lvetc Creating LVM volume vgbck/lvvar Creating LVM volume vgbck/lvmysql Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc Mounting filesystem /etc/bacula/cluster Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar Mounting filesystem /var/bacula Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql Mounting filesystem /var/lib/mysql/bacula Creating DRBD resource rOPS Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd1 Creating LVM VG vgops Creating LVM volume vgops/lvcachemgr Creating LVM volume vgops/lvbackup Creating LVM volume vgops/lvdata Creating LVM volume vgops/lvdb Creating LVM volume vgops/lvswl Creating LVM volume vgops/lvcluster Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr Mounting filesystem /opt/cache Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup Mounting filesystem /dstpol/backup Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata Mounting filesystem /dstpol/data Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb Mounting filesystem /dstpol/databases Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl Mounting filesystem /dstpol/swl Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster Mounting filesystem /dstpol/sys/cluster Disk layout created. The system is now ready to restore from Bacula. You can use the 'bls' command to get information from your Volume, and 'bextract' to restore jobs from your Volume. It is assumed that you know what is necessary to restore - typically it will be a full backup. You can find useful Bacula commands in the shell history. When finished, type 'exit' in the shell to continue recovery. WARNING: The new root is mounted under '/mnt/local'. rear>
Now you need to continue with restoring the actual Bacula backup, for this you have multiple options of which bextract is the most easy and straightforward, but also the slowest and unsafest.
If you know the JobId of the latest successful full backup, and differential backups the most efficient way to restore is by creating a bootstrap file with this information and using it to restore from tape.
A bootstrap file looks like this:
Volume = VOL-1234 JobId = 914 Job = Bkp_Daily
or
Volume = VOL-1234 VolSessionId = 1 VolSessionTime = 108927638
Using a bootstrap file with bextract is easy, simply do: bextract -b bootstrap.txt Ultrium-1 /mnt/local
Tip
|
It helps to know exactly how many files you need to restore, and using the FileIndex and Count keywords so bextract does not require to read the whole tape. Use the commands in your shell history to access an example Bacula bootstrap file. |
To use bextract to restore everything from a single tape, you can do: bextract -V VOLUME-NAME Ultrium-1 /mnt/local
rear> bextract -V VOL-1234 Ultrium-1 /mnt/local bextract: match.c:249-0 add_fname_to_include prefix=0 gzip=0 fname=/ bextract: butil.c:282 Using device: "Ultrium-1" for reading. 30-Mar 16:00 bextract JobId 0: Ready to read from volume "VOL-1234" on device "Ultrium-1" (/dev/st0). bextract JobId 0: -rw-r----- 1 252 bacula 3623795 2011-03-30 11:02:18 /mnt/local/var/lib/bacula/bacula.sql bextract JobId 0: drwxr-xr-x 2 root root 4096 2011-02-02 11:48:28 *none* bextract JobId 0: drwxr-xr-x 4 root root 1024 2011-02-23 13:09:53 *none* bextract JobId 0: drwxr-xr-x 12 root root 4096 2011-02-02 11:50:00 *none* bextract JobId 0: -rwx------ 1 root root 0 2011-02-02 11:48:24 /mnt/local/.hpshm_keyfile bextract JobId 0: -rw-r--r-- 1 root root 0 2011-02-22 12:38:03 /mnt/local/.autofsck ... 30-Mar 16:06 bextract JobId 0: End of Volume at file 7 on device "Ultrium-1" (/dev/st0), Volume "VOL-1234" 30-Mar 16:06 bextract JobId 0: End of all volumes. 30-Mar 16:07 bextract JobId 0: Alert: smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen 30-Mar 16:07 bextract JobId 0: Alert: Home page is http://smartmontools.sourceforge.net/ 30-Mar 16:07 bextract JobId 0: Alert: 30-Mar 16:07 bextract JobId 0: Alert: TapeAlert: OK 30-Mar 16:07 bextract JobId 0: Alert: 30-Mar 16:07 bextract JobId 0: Alert: Error counter log: 30-Mar 16:07 bextract JobId 0: Alert: Errors Corrected by Total Correction Gigabytes Total 30-Mar 16:07 bextract JobId 0: Alert: ECC rereads/ errors algorithm processed uncorrected 30-Mar 16:07 bextract JobId 0: Alert: fast | delayed rewrites corrected invocations [10^9 bytes] errors 30-Mar 16:07 bextract JobId 0: Alert: read: 1546 0 0 0 1546 0.000 0 30-Mar 16:07 bextract JobId 0: Alert: write: 0 0 0 0 0 0.000 0 165719 files restored.
Warning
|
In this case bextract will restore all the Bacula jobs on the provided tapes, start from the oldest, down to the latest. As a consequence, deleted files may re-appear and the process may take a very long time. |
Once finished, continue Relax-and-Recover by typing exit.
rear> exit Did you restore the backup to /mnt/local ? Ready to continue ? y Installing GRUB boot loader Finished recovering your system. You can explore it under '/mnt/local'. Finished in 4424 seconds.
Important
|
If you neglect to perform this last crucial step, your new system will not boot and you have to install a boot-loader yourself manually, or re-execute this procedure. |
The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with USB storage.
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
### Exclude certain items
ONLY_INCLUDE_VG=( vg00 )
EXCLUDE_MOUNTPOINTS=( /data )
Creating a combined rescue device that integrates the backup on USB, it is sufficient to run rear -v mkbackup as shown below after you have inserted the USB device. Make sure the device name for the USB device is what is configured.
[root@system ~]# rear -v mkbackup Relax-and-Recover 1.12.0svn497 / 2011-07-11 Creating disk layout. Creating root filesystem layout Copying files and directories Copying program files and libraries Copying kernel modules Creating initramfs Creating archive 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz' Total bytes written: 3644416000 (3.4GiB, 5.5MiB/s) in 637 seconds. Writing MBR to /dev/sda Modifying local GRUB configuration Copying resulting files to usb location Finished in 747 seconds.
Important
|
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory) |
See the section Booting from USB storage device for more information about how to enable your BIOS to boot from a USB storage device.
Then wait for the system to boot until you get the prompt.
On the shell prompt, type rear recover.
You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.
RESCUE SYSTEM:/ # rear recover Relax-and-Recover 1.12.0svn497 / 2011-07-11 Backup archive size is 1.2G (compressed) To recreate HP SmartArray controller 1, type exactly YES: YES To recreate HP SmartArray controller 7, type exactly YES: YES Clearing HP SmartArray controller 1 Clearing HP SmartArray controller 7 Recreating HP SmartArray controller 1|A Configuration restored successfully, reloading CCISS driver... OK Recreating HP SmartArray controller 7|A Configuration restored successfully, reloading CCISS driver... OK Comparing disks. Disk configuration is identical, proceeding with restore. Start system layout restoration. Creating partitions for disk /dev/cciss/c0d0 (msdos) Creating partitions for disk /dev/cciss/c1d0 (msdos) Creating software RAID /dev/md126 Creating software RAID /dev/md127 Creating LVM PV /dev/md127 Restoring LVM VG vg00 Creating ext3-filesystem / on /dev/mapper/vg00-lv00 Mounting filesystem / Creating ext3-filesystem /boot on /dev/md126 Mounting filesystem /boot Creating ext3-filesystem /data on /dev/mapper/vg00-lvdata Mounting filesystem /data Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt Mounting filesystem /opt Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp Mounting filesystem /tmp Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr Mounting filesystem /usr Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar Mounting filesystem /var Creating swap on /dev/mapper/vg00-lvswap Disk layout created. Restoring from 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz' Restored 3478 MiB in 134 seconds [avg 26584 KiB/sec] Installing GRUB boot loader Finished recovering your system. You can explore it under '/mnt/local'. Finished in 278 seconds.
If all is well, you can now remove the USB device, restore the BIOS boot order and reboot the system into the recovered OS.
Using One-Button-Disaster-Recovery (OBDR) tapes has a few benefits.
-
Within large organisations tape media is already part of a workflow for offsite storage and is a known and trusted technology
-
Tapes can store large amounts of data reliably and restoring large amounts of data is predictable in time and effort
-
OBDR offers booting from tapes, which is very convenient
-
A single tape can hold both the rescue image as well as a complete snapshot of the system (up to 1.6TB with LTO4)
However, you need one tape per system as an OBDR tape can only store one single rescue environment.
The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape only for storing the rescue image, the backup is expected to be handled by Bacula and so the Bacula tools are included in the rescue environment to enable a Bacula restore.
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
To protect normal backup tapes (in case tape drives are also used by another backup solution) Relax-and-Recover expects that the tape to use is labeled REAR-000. To achieve this is to insert a blank tape to use for Relax-and-Recover and run the rear format /dev/stX command.
If the rescue environment needs additional tools and workflow, this can be specified by using BACKUP=BACULA in the configuration file /etc/rear/local.conf:
BACKUP=BACULA
OUTPUT=OBDR
BEXTRACT_DEVICE=Ultrium-1
BEXTRACT_VOLUME=VOL-*
Using the BEXTRACT_DEVICE allows you to use the tape device that is referenced from the Bacula configuration. This helps in those cases where the discovery of the various tape drives has already been done and configured in Bacula.
The BEXTRACT_VOLUME variable is optional and is only displayed in the restore instructions on screen as an aid during recovery.
To create a rescue environment that can boot from an OBDR tape, simply run rear -v mkrescue with a REAR-000 -labeled tape inserted.
[root@system ~]# rear -v mkrescue Relax-and-Recover 1.12.0svn497 / 2011-07-11 Rewinding tape Writing OBDR header to tape in drive '/dev/nst0' Creating disk layout. Creating root filesystem layout Copying files and directories Copying program files and libraries Copying kernel modules Creating initramfs Making ISO image Wrote ISO image: /var/lib/rear/output/rear-dag-ops.iso (48M) Writing ISO image to tape Modifying local GRUB configuration Finished in 119 seconds.
Warning
|
The message above about /dev/cciss/c1d0 not being used makes sense as this is not a real disk but simply an entry for manipulating the controller. This is specific to CCISS controllers with only a tape device attached. |
The One Button Disaster Recovery (OBDR) functionality in HP LTO Ultrium drives enables them to emulate CD-ROM devices in specific circumstances (also known as being in ''Disaster Recovery'' mode). The drive can then act as a boot device for PCs that support booting off CD-ROM.
Tip
|
An OBDR capable drive can be switched into CD-ROM mode by powering on with the eject button held down. Make sure you keep it pressed when the tape drive regains power, and then release the button. If the drive is in OBDR mode, the light will blink regularly. This might be easier in some cases than the below procedure, hence the name One Button Disaster Recovery ! |
To boot from OBDR, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the HP Smart Array controller with the tape attached by pressing F8 (or Escape-8 on serial console).
iLO 2 v1.78 Jun 10 2009 10.5.20.171 Slot 0 HP Smart Array P410i Controller (512MB, v2.00) 1 Logical Drive Slot 3 HP Smart Array P401 Controller (512MB, v2.00) 1 Logical Drive Slot 4 HP Smart Array P212 Controller (0MB, v2.00) 0 Logical Drives Tape or CD-ROM Drive(s) Detected: Port 1I: Box 0: Bay 4 1785-Slot 4 Drive Array Not Configured No Drives Detected Press <F8> to run the Option ROM Configuration for Arrays Utility Press <ESC> to skip configuration and continue
Then select Configure OBDR in the menu and select the Tape drive by marking it with X (default is on) and press ENTER and F8 to activate this change so it displays ''Configuration saved''.
Then press ENTER and Escape to leave the Smart Array controller BIOS.
**** System will boot from Tape/CD/OBDR device attached to Smart Array.
To boot from OBDR when using an LSI controller, boot your system with the Relax-and-Recover tape inserted. During the boot sequence, interrupt the LSI controller BIOS that has the tape attached by pressing F8 (or Escape-8 on serial console).
LSI Logic Corp. MPT BIOS Copyright 1995-2006 LSI Logic Corp. MPTBIOS-5.05.21.00 HP Build <<<Press F8 for configuration options>>>
Then select the option 1. Tape-based One Button Disaster Recovery (OBDR).:
Select a configuration option: 1. Tape-based One Button Disaster Recovery (OBDR). 2. Multi Initiator Configuration. <F9 = Setup> 3. Exit.
And then select the correct tape drive to boot from:
compatible tape drives found -> NUM HBA SCSI ID Drive information 0 0 A - HP Ultrium 2-SCSI Please choose the NUM of the tape drive to place into OBDR mode.
If all goes well, the system will reboot with OBDR-mode enabled:
The PC will now reboot to begin Tape Recovery....
During the next boot, OBDR-mode will be indicate by:
*** Bootable media located, Using non-Emulation mode ***
Once booted from the OBDR tape, select the 'Relax-and-Recover' menu entry from the menu. If you don’t press a key within 30 seconds, the system will try to boot from the local disk.
+---------------------------------------------+ | "Relax-and-Recover v1.12.0svn497" | +---------------------------------------------+ | "Relax-and-Recover" | |---------------------------------------------| | "Other actions" | | "Help for Relax-and-Recover" | | "Boot Local disk (hd1)" | | "Boot BIOS disk (0x81)" | | "Boot Next BIOS device" | | "Hardware Detection tool" | | "Memory test" | | "Reboot system" | | "Power off system" | | | | | +---------------------------------------------+ "Press [Tab] to edit options or [F1] for help" "Automatic boot in 30 seconds..."
Then wait for the system to boot until you get the prompt.
On the shell prompt, type rear recover.
You may need to answer a few questions depending on your hardware configuration and whether you are restoring to a (slightly) different system.
RESCUE SYSTEM:/ # rear recover Relax-and-Recover 1.12.0svn497 / 2011-07-11 NOTICE: Will do driver migration Rewinding tape To recreate HP SmartArray controller 3, type exactly YES: YES To recreate HP SmartArray controller 0, type exactly YES: YES Clearing HP SmartArray controller 3 Clearing HP SmartArray controller 0 Recreating HP SmartArray controller 3|A Configuration restored successfully, reloading CCISS driver... OK Recreating HP SmartArray controller 0|A Configuration restored successfully, reloading CCISS driver... OK Comparing disks. Disk configuration is identical, proceeding with restore. Type "Yes" if you want DRBD resource rBCK to become primary: Yes Type "Yes" if you want DRBD resource rOPS to become primary: Yes Start system layout restoration. Creating partitions for disk /dev/cciss/c0d0 (msdos) Creating partitions for disk /dev/cciss/c2d0 (msdos) Creating software RAID /dev/md2 Creating software RAID /dev/md6 Creating software RAID /dev/md3 Creating software RAID /dev/md4 Creating software RAID /dev/md5 Creating software RAID /dev/md1 Creating software RAID /dev/md0 Creating LVM PV /dev/md6 Creating LVM PV /dev/md5 Creating LVM PV /dev/md2 Creating LVM VG vgrem Creating LVM VG vgqry Creating LVM VG vg00 Creating LVM volume vg00/lv00 Creating LVM volume vg00/lvdstpol Creating LVM volume vg00/lvsys Creating LVM volume vg00/lvusr Creating LVM volume vg00/lvtmp Creating LVM volume vg00/lvvar Creating LVM volume vg00/lvopt Creating ext3-filesystem / on /dev/mapper/vg00-lv00 Mounting filesystem / Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol Mounting filesystem /dstpol Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys Mounting filesystem /dstpol/sys Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr Mounting filesystem /usr Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp Mounting filesystem /tmp Creating ext3-filesystem /boot on /dev/md0 Mounting filesystem /boot Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar Mounting filesystem /var Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt Mounting filesystem /opt Creating swap on /dev/md1 Creating DRBD resource rBCK Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd2 Creating LVM VG vgbck Creating LVM volume vgbck/lvetc Creating LVM volume vgbck/lvvar Creating LVM volume vgbck/lvmysql Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc Mounting filesystem /etc/bacula/cluster Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar Mounting filesystem /var/bacula Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql Mounting filesystem /var/lib/mysql/bacula Creating DRBD resource rOPS Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd1 Creating LVM VG vgops Creating LVM volume vgops/lvcachemgr Creating LVM volume vgops/lvbackup Creating LVM volume vgops/lvdata Creating LVM volume vgops/lvdb Creating LVM volume vgops/lvswl Creating LVM volume vgops/lvcluster Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr Mounting filesystem /opt/cache Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup Mounting filesystem /dstpol/backup Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata Mounting filesystem /dstpol/data Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb Mounting filesystem /dstpol/databases Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl Mounting filesystem /dstpol/swl Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster Mounting filesystem /dstpol/sys/cluster Disk layout created. The system is now ready to restore from Bacula. You can use the 'bls' command to get information from your Volume, and 'bextract' to restore jobs from your Volume. It is assumed that you know what is necessary to restore - typically it will be a full backup. You can find useful Bacula commands in the shell history. When finished, type 'exit' in the shell to continue recovery. WARNING: The new root is mounted under '/mnt/local'. rear>
See the section Restoring from Bacula tape for more information about how to restore a Bacula tape.
An OBDR backup tape is similar to an OBDR rescue tape, but next to the rescue environment, it also consists of a complete backup of the system. This is very convenient in that a single tape can be use for disaster recovery, and recovery is much more simple and completely automated.
Caution
|
Please make sure that the system fits onto a single tape uncompressed. For an LTO4 Ultrium that would mean no more than 1.6TB. |
The below configuration (/etc/rear/local.conf) gives a list of possible options when you want to run Relax-and-Recover with a tape drive. This example shows how to use the tape for storing both the rescue image and the backup.
BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
To create a bootable backup tape that can boot from OBDR, simply run rear -v mkbackup with a REAR-000 -labeled tape inserted.
[root@system ~]# rear -v mkbackup Relax-and-Recover 1.12.0svn497 / 2011-07-11 Rewinding tape Writing OBDR header to tape in drive '/dev/nst0' Creating disk layout Creating root filesystem layout Copying files and directories Copying program files and libraries Copying kernel modules Creating initramfs Making ISO image Wrote ISO image: /var/lib/rear/output/rear-system.iso (45M) Writing ISO image to tape Creating archive '/dev/nst0' Total bytes written: 7834132480 (7.3GiB, 24MiB/s) in 317 seconds. Rewinding tape Modifying local GRUB configuration Finished in 389 seconds.
Important
|
It is advised to go into single user mode (init 1) before creating a backup to ensure all active data is consistent on disk (and no important processes are active in memory) |
See the section Booting from OBDR rescue tape for more information about how to enable OBDR and boot from OBDR tapes.
RESCUE SYSTEM:~ # rear recover Relax-and-Recover 1.12.0svn497 / 2011-07-11 NOTICE: Will do driver migration Rewinding tape To recreate HP SmartArray controller 3, type exactly YES: YES To recreate HP SmartArray controller 0, type exactly YES: YES Clearing HP SmartArray controller 3 Clearing HP SmartArray controller 0 Recreating HP SmartArray controller 3|A Configuration restored successfully, reloading CCISS driver... OK Recreating HP SmartArray controller 0|A Configuration restored successfully, reloading CCISS driver... OK Comparing disks. Disk configuration is identical, proceeding with restore. Type "Yes" if you want DRBD resource rBCK to become primary: Yes Type "Yes" if you want DRBD resource rOPS to become primary: Yes Start system layout restoration. Creating partitions for disk /dev/cciss/c0d0 (msdos) Creating partitions for disk /dev/cciss/c2d0 (msdos) Creating software RAID /dev/md2 Creating software RAID /dev/md6 Creating software RAID /dev/md3 Creating software RAID /dev/md4 Creating software RAID /dev/md5 Creating software RAID /dev/md1 Creating software RAID /dev/md0 Creating LVM PV /dev/md6 Creating LVM PV /dev/md5 Creating LVM PV /dev/md2 Restoring LVM VG vgrem Restoring LVM VG vgqry Restoring LVM VG vg00 Creating ext3-filesystem / on /dev/mapper/vg00-lv00 Mounting filesystem / Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol Mounting filesystem /dstpol Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys Mounting filesystem /dstpol/sys Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr Mounting filesystem /usr Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp Mounting filesystem /tmp Creating ext3-filesystem /boot on /dev/md0 Mounting filesystem /boot Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar Mounting filesystem /var Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt Mounting filesystem /opt Creating swap on /dev/md1 Creating DRBD resource rBCK Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd2 Restoring LVM VG vgbck Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc Mounting filesystem /etc/bacula/cluster Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar Mounting filesystem /var/bacula Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql Mounting filesystem /var/lib/mysql/bacula Creating DRBD resource rOPS Writing meta data... initializing activity log New drbd meta data block successfully created. Creating LVM PV /dev/drbd1 Restoring LVM VG vgops Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr Mounting filesystem /opt/cache Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup Mounting filesystem /dstpol/backup Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata Mounting filesystem /dstpol/data Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb Mounting filesystem /dstpol/databases Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl Mounting filesystem /dstpol/swl Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster Mounting filesystem /dstpol/sys/cluster Disk layout created. Restoring from 'tape:///dev/nst0/system/backup.tar' Restored 7460 MiB in 180 seconds [avg 42444 KiB/sec] Installing GRUB boot loader Finished recovering your system. You can explore it under '/mnt/local'. Finished in 361 seconds.
Instead of using your ReaR image to completely recover your system from bare metal (as illustrated in most of the above scenarios), you can also use it as a live media to boot a broken but hopefully repairable system.
Once booted on your recovery image, the mountonly
workflow will:
-
activate all Volume Groups
-
offer to decrypt any LUKS-encrypted filesystem that may be present
-
mount all the target filesystems (including the most important virtual ones) below
/mnt/local
thereby making it possible for you to explore your system at will,
correcting any configuration mistake that may have prevented its startup,
or allowing you to simply chroot
into it and further repair it using its
own administrative tools.
One important point to remember is that the mountonly
workflow on its own
won’t modify the target system in any way. Of course, once the target
filesystems are mounted you, as the administrator, may decide to do so
manually.
Beware: The mountonly
workflow can only be used on the system where
the rescue image was generated, as it bases its logic on the filesystem
layout description file generated during the run of the mkrescue
or
mkbackup
workflows.
Here are the steps you would typically follow:
Using any of the techniques described in the other scenarios, create a
ReaR recovery image for your system (through rear mkrescue
or rear
mkbackup
). If you only take the mountonly
workflow into consideration, it
doesn’t matter whether you also make a backup of your system or not
(obviously, you’d better cover all your bases and make sure you’d be able to
perform a full recover
as well should the need occur).
Please note, that by default ReaR only includes in the recovery image the
tools it will need to recover the system. If you anticipate the need for
some extra tools in the context of a repair operation (e.g. tools that you
might need in the event chroot`ing into the target system doesn’t work), you
should make sure to include them in your recovery image by adding to the
`PROGS
or REQUIRED_PROGS
configuration variables (please refer to the
comments in default.conf
for the exact meaning of each).
Arrange for the target system to boot on your recovery image as you would in any of the other scenarios.
Issue the rear mountonly
command to launch the workflow (that one is always
verbose):
RESCUE pc-pan:~ # rear mountonly Relax-and-Recover 2.5 / Git Running rear mountonly (PID 625) Using log file: /var/log/rear/rear-pc-pan.log Running workflow mountonly within the ReaR rescue/recovery system Comparing disks Device sda has expected (same) size 34359738368 (will be used for 'mountonly') Disk configuration looks identical Proceed with 'mountonly' (yes) otherwise manual disk layout configuration is enforced (default 'yes' timeout 30 seconds) yes User confirmed to proceed with 'mountonly' Start target system mount. Mounting filesystem / Mounting filesystem /home Mounting filesystem /boot/efi Please enter the password for LUKS device cr_vg00-lvol4 (/dev/mapper/vg00-lvol4): Enter passphrase for /dev/mapper/vg00-lvol4: Mounting filesystem /products Disk layout processed. Finished 'mountonly'. The target system is mounted at '/mnt/local'. Exiting rear mountonly (PID 625) and its descendant processes ... Running exit tasks
As you can see in the output above, you will first be asked to confirm
running the workflow (Proceed with 'mountonly'
) — simply press return.
All the target filesystems should now be mounted below /mnt/local
(including
LUKS-encrypted ones if present and all needed virtual ones). In case any of
them fails to mount, you will be offered to review the mount script and to
re-execute it if needed.
Once the system is in the desired state, you can start exploring it, correcting any configuration mistake or filesystem corruption that prevented it from booting properly. In this state, the only tools at your disposal are those included by default in ReaR recovery image, or those you saw fit to add yourself (see above).
If this is not enough and you need to run the native administrative tools
hosted inside your target system (such as YaST in the case of SUSE
distributions), you are now in a position where you can chroot
inside
your system to reach them (chroot /mnt/local
).