Permalink
Fetching contributors…
Cannot retrieve contributors at this time
642 lines (533 sloc) 21.4 KB

BLOCKCLONE as backup method

BLOCKCLONE backup method is a bit distinct type of backup, which works directly with block devices. It allows running backups of any kind of block device that can be read/write by Linux drivers and save them to e.g. NFS share or USB drive for later restore. It currently integrates Disk Dump (dd) and ntfsclone (from ntfs-3g package). With BLOCKCLONE, user is also able to make full backup and restore of dual boot (Linux / Windows) environments.

Limitations

  • Works only directly with disk partitions

  • GPT not supported (work in progress)

  • No UEFI support (work in progress)

  • Linux family boot loader must be used as primary (Windows bootloader was not tested)

  • Restore should be done to same sized or larger disks

  • Tests were made with Windows 7/10 with NFS and USB as destinations. Other ReaR backup destinations like SMB or FTP might however work as well.

Warning!

ReaR with BLOCKCLONE is capable of doing backup of Linux/Windows dual boot environments. There is however a need for some basic knowledge on how is source OS setup. Things like boot loader device location, Linux/Windows partitioning and file system layout are essential for backup setup.
Always test before you rely!

Examples

1. Backup/restore of arbitrary block device with BLOCKCLONE and dd on NFS server

Configuration

This is very basic and most simple scenario where we will do backup of single partition (/dev/sdc1) located on separate disk (/dev/sdc).
First we need to set some global options in local.conf, like target for backups. In our small example backups will be stored in /mnt/rear directory on BACKUP_URL NFS server.

# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://<hostname>/mnt/rear

Now we will define variables that will apply only for targeted block device

# cat alien.conf
BACKUP=BLOCKCLONE                                        # Define BLOCKCLONE as backup method
BACKUP_PROG_ARCHIVE="alien"                              # Name of image file
BACKUP_PROG_SUFFIX=".dd.img"                             # Suffix of image file
BACKUP_PROG_COMPRESS_SUFFIX=""                           # Don't use additional suffixes

BLOCKCLONE_PROG=dd                                       # Use dd for image creation
BLOCKCLONE_PROG_OPTS="bs=4k"                             # Additional options that will be passed to dd
BLOCKCLONE_SOURCE_DEV="/dev/sdc1"                        # Device that should be backed up

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdc"                       # Device where partitioning information is stored (optional)
BLOCKCLONE_MBR_FILE="alien_boot_strap.img"               # Output filename for boot strap code
BLOCKCLONE_PARTITIONS_CONF_FILE="alien_partitions.conf"  # Output filename for partition configuration

BLOCKCLONE_ALLOW_MOUNTED="yes"                           # Device can be mounted during backup (default NO)

Running backup

Save partitions configuration, bootstrap code and create actual backup of /dev/sdc1

# rear -C alien mkbackuponly

Running restore from ReaR restore/recovery system

# rear -C alien restoreonly
Restore alien.dd.img to device: [/dev/sdc1]                 # User is always prompted for restore destination
Device /dev/sdc1 was not found.                             # If destination does not exist ReaR will try to create it (or fail if BLOCKCLONE_SAVE_MBR_DEV was not set during backup)
Restore partition layout to (^c to abort): [/dev/sdc]       # Prompt user for device where partition configuration should be restored
Checking that no-one is using this disk right now ... OK

Disk /dev/sdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x10efb7a9.
Created a new partition 1 of type 'HPFS/NTFS/exFAT' and of size 120 MiB.
/dev/sdc2:
New situation:

Device     Boot Start    End Sectors  Size Id Type
/dev/sdc1        4096 249855  245760  120M  7 HPFS/NTFS/exFAT

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Summary

In first example we have run backup of /dev/sdc1 partition and stored it on NFS server. Saved image was later restored from ReaR rescue/recovery system. ReaRs BLOCKCLONE will always ask user for restore destination, so it is users responsibility to identify right target disk/partition prior restore. Unlike NETFS backup method, no guesses about target devices will be made!

Tip
One of easiest ways how to identify right disk could be its size.
Running fdisk -l <device_file> could be helpful.

During restore phase ReaR recognized that target partition does not exist and asked if it should be created. If restore destination does not exist and BLOCKCLONE_SAVE_MBR_DEV was set during backup, ReaR will try to deploy partition setup from saved configuration files (BLOCKCLONE_MBR_FILE and BLOCKCLONE_PARTITIONS_CONF_FILE) and continue with restore.

2. Backup/restore of Linux / Windows 10 dual boot setup with each OS on separate disk

Configuration

In next example we will do backup/restore using BLOCKCLONE and ntfsclone of Linux (installed on /dev/sda) and Windows 10 (installed on /dev/sdb).

Tip
You can locate right disk devices using df and os-prober
# df -h /boot
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        10G  4.9G  5.2G  49% /           # Linux is most probably installed on /dev/sda

# os-prober
/dev/sdb1:Windows 10 (loader):Windows:chain       # Windows 10 is most probably installed on /dev/sdb

First we will configure some ReaR backup global options (similar to first example we will do backup/restore with help of NFS server).

# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://<hostname>/mnt/rear
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )

Now we will define backup parameters for Linux.

# cat base_os.conf
this_file_name=$( basename ${BASH_SOURCE[0]} )
LOGFILE="$LOG_DIR/rear-$HOSTNAME-$WORKFLOW-${this_file_name%.*}.log"
BACKUP_PROG_ARCHIVE="backup-${this_file_name%.*}"
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )

Our Windows 10 is by default installed on two separate partitions (partition 1 for boot data and partition 2 for disk C:), so we will create two separate configuration files for each partition.

Windows boot partition:

# cat windows_boot.conf
BACKUP=BLOCKCLONE
BACKUP_PROG_ARCHIVE="windows_boot"
BACKUP_PROG_SUFFIX=".img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_SOURCE_DEV="/dev/sdb1"
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdb"
BLOCKCLONE_MBR_FILE="windows_boot_strap.img"
BLOCKCLONE_PARTITIONS_CONF_FILE="windows_partitions.conf"

Windows data partition (disk C:\):

# cat windows_data.conf
BACKUP=BLOCKCLONE
BACKUP_PROG_ARCHIVE="windows_data"
BACKUP_PROG_SUFFIX=".img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_SOURCE_DEV="/dev/sdb2"
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdb"
BLOCKCLONE_MBR_FILE="windows_boot_strap.img"
BLOCKCLONE_PARTITIONS_CONF_FILE="windows_partitions.conf"

Running backup

First we will create backup of Linux. mkbackup command will create bootable ISO image with ReaR rescue/recovery system that will be later used for booting broken system and consecutive recovery.

# rear -C base_os mkbackup

Now we create backup of Windows 10 boot partition. Command mkbackuponly will ensure that only partition data and partition layout will be saved (ReaR rescue/recovery system will not be created which is exactly what we want).

# rear -C windows_boot mkbackuponly

Similarly, we create backup of Windows 10 data partition (disk C:\)

# rear -C windows_data mkbackuponly

Running restore from ReaR restore/recovery system

As a first step after ReaR rescue/recovery system booted, we will recover Linux. This step will recover all Linux file systems, OS data and bootloader. Windows disk will remain untouched.

# rear -C base_os recover

In second step will recover Windows 10 boot partition. During this step ReaR will detect that destination partition is not present and ask us for device file where partition(s) should be created. It doesn’t really matter whether we decide to recover Windows 10 boot or data partition first. restoreonly command ensures that previously restored Linux data and partition(s) configuration (currently mounted under /mnt/local) will remain untouched. Before starting Windows 10 recovery we should identify right disk for recovery, as mentioned earlier disk size could be a good start.

# fdisk -l /dev/sdb
Disk /dev/sdb: 50 GiB, 53687091200 bytes, 104857600 sectors

/dev/sdb looks to be right destination, so we can proceed with restore.

# rear -C windows_boot restoreonly
Restore windows_boot.img to device: [/dev/sdb1]
Device /dev/sdb1 was not found.
Restore partition layout to (^c to abort): [/dev/sdb]
Checking that no-one is using this disk right now ... OK
...

Last step is to recover Windows 10 OS data (C:\). Partitions on /dev/sdb were already created in previous step, hence ReaR will skip prompt for restoring partition layout.

# rear -C windows_data restoreonly
Restore windows_data.img to device: [/dev/sdb2]
Ntfsclone image version: 10.1
Cluster size           : 4096 bytes
Image volume size      : 33833349120 bytes (33834 MB)
Image device size      : 33833353216 bytes
Space in use           : 9396 MB (27.8%)
Offset to image data   : 56 (0x38) bytes
Restoring NTFS from image ...
...

At this stage Linux together with Windows 10 is successfully restored.

Tip
As Linux part is still mounted under /mnt/local, you can do some final configuration changes. e.g. adapt GRUB configuration, /etc/fstab, reinstall boot loader …​
Tip
ReaR will by default not include tools for mounting NTFS file systems. You can do it manually by adding REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone mount.ntfs-3g ) to your local.conf

3. Backup/restore of Linux / Windows 10 dual boot setup sharing same disk

Configuration

In this example we will do backup/restore using BLOCKCLONE and ntfsclone of Linux and Windows 10 installed on same disk (/dev/sda). Linux is installed on partition /dev/sda3. Windows 10 is again divided into boot partition located on /dev/sda1 and OS data (C:/) located on /dev/sda2. Backups will be stored on NFS server.

First we set global ReaR options

# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://<hostname>/mnt/rear
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )

BLOCKCLONE_STRICT_PARTITIONING="yes"
BLOCKCLONE_SAVE_MBR_DEV="/dev/sda"

BLOCKCLONE_MBR_FILE="boot_strap.img"
BLOCKCLONE_PARTITIONS_CONF_FILE="partitions.conf"
Important
BLOCKCLONE_STRICT_PARTITIONING is mandatory if backing up Linux / Windows that shares one disk. Not using this option might result to unbootable Windows 10 installation.

Linux configuration

# cat base_os.conf
this_file_name=$( basename ${BASH_SOURCE[0]} )
LOGFILE="$LOG_DIR/rear-$HOSTNAME-$WORKFLOW-${this_file_name%.*}.log"
BACKUP_PROG_ARCHIVE="backup-${this_file_name%.*}"
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )

Windows 10 boot partition configuration

# cat windows_boot.conf
BACKUP=BLOCKCLONE

BACKUP_PROG_ARCHIVE="windows_boot"
BACKUP_PROG_SUFFIX=".nc.img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SOURCE_DEV="/dev/sda1"

Windows 10 data partition configuration

# cat windows_data.conf
BACKUP=BLOCKCLONE
BACKUP_PROG_ARCHIVE="windows_data"
BACKUP_PROG_SUFFIX=".nc.img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SOURCE_DEV="/dev/sda2"

Running backup

Backup of Linux

# rear -C base_os mkbackup

Backup of Windows 10 boot partition

# rear -C windows_boot mkbackuponly

Backup of Windows 10 data partition

# rear -C windows_data mkbackuponly

Running restore from ReaR restore/recovery system

Restore Linux

# rear -C base_os recover

During this step ReaR will also create both Windows 10 partitions

Restore Windows 10 data partition

# rear -C windows_data restoreonly

Restore Windows 10 boot partition

# rear -C windows_boot restoreonly

4. Backup/restore of Linux / Windows 10 dual boot setup sharing same disk with USB as destination

Configuration

In this example we will do backup/restore using BLOCKCLONE and ntfsclone of Linux and Windows 10 installed on same disk (/dev/sda). Linux is installed on partition /dev/sda3. Windows 10 is again divided into boot partition located on /dev/sda1 and OS data (C:/) located on /dev/sda2. Backups will be stored on USB disk drive (/dev/sdb in this example).

Global options

# cat local.conf
OUTPUT=USB
BACKUP=NETFS

USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

USB_SUFFIX="USB_backups"

GRUB_RESCUE=n
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )

BLOCKCLONE_STRICT_PARTITIONING="yes"
BLOCKCLONE_SAVE_MBR_DEV="/dev/sda"

BLOCKCLONE_MBR_FILE="boot_strap.img"
BLOCKCLONE_PARTITIONS_CONF_FILE="partitions.conf"

Options used during Linux backup/restore.

# cat local.conf
OUTPUT=USB
BACKUP=NETFS

USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

USB_SUFFIX="USB_backups"

GRUB_RESCUE=n
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )

BLOCKCLONE_STRICT_PARTITIONING="yes"
BLOCKCLONE_SAVE_MBR_DEV="/dev/sda"

BLOCKCLONE_MBR_FILE="boot_strap.img"
BLOCKCLONE_PARTITIONS_CONF_FILE="partitions.conf"
Important
USB_SUFFIX option is mandatory as it avoids ReaR to hold every backup in separate directory, this behavior is essential for BLOCKCLONE backup method to work correctly.

Windows boot partition options

# cat windows_boot.conf
BACKUP=BLOCKCLONE

BACKUP_PROG_ARCHIVE="windows_boot"
BACKUP_PROG_SUFFIX=".nc.img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SOURCE_DEV="/dev/sda1"

Windows data partition options

# cat windows_data.conf
BACKUP=BLOCKCLONE
BACKUP_PROG_ARCHIVE="windows_data"
BACKUP_PROG_SUFFIX=".nc.img"
BACKUP_PROG_COMPRESS_SUFFIX=""

BLOCKCLONE_PROG=ntfsclone
BLOCKCLONE_PROG_OPTS="--quiet"

BLOCKCLONE_SOURCE_DEV="/dev/sda2"

Running backup

First we need to format target USB device, with rear format command

# rear -v format /dev/sdb
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-centosd.log
USB device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem: Yes
Repartitioning '/dev/sdb'
Creating partition table of type 'msdos' on '/dev/sdb'
Creating ReaR data partition up to 100% of '/dev/sdb'
Setting 'boot' flag on /dev/sdb
Creating ext3 filesystem with label 'REAR-000' on '/dev/sdb1'
Adjusting filesystem parameters on '/dev/sdb1'

Backup of Linux

# rear -C base_os mkbackup

Backup of Windows 10 boot partition

# rear -C windows_boot mkbackuponly
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 524283904 bytes (525 MB)
Current device size: 524288000 bytes (525 MB)
Scanning volume ...
Accounting clusters ...
Space in use       : 338 MB (64.4%)
Saving NTFS to image ...
Syncing ...

Backup of Windows 10 data partition

# rear -C windows_data mkbackuponly
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 18104709120 bytes (18105 MB)
Current device size: 18104713216 bytes (18105 MB)
Scanning volume ...
Accounting clusters ...
Space in use       : 9833 MB (54.3%)
Saving NTFS to image ...
Syncing ...

Running restore from ReaR restore/recovery system

For sake of this demonstration I’ve purposely used ReaR’s rescue/recovery media (USB disk that holds our backed up Linux and Windows 10) as /dev/sda and disk that will be used as restore destination as /dev/sdb. This will demonstrate possibility of ReaR to recover backup to arbitrary disk.
As first step Linux will be restored, this will create all the partitions needed, even those used by Windows 10.

RESCUE centosd:~ # rear -C base_os recover
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-centosd.log
Sourcing additional configuration file '/etc/rear/base_os.conf'
Running workflow recover within the ReaR rescue/recovery system
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
Started rpc.statd.
RPC status rpc.statd available.
Using backup archive '/tmp/rear.70zIHqCYsIbtlr6/outputfs/centosd/backup-base_os.tar.gz'
Calculating backup archive size
Backup archive size is 1001M	/tmp/rear.70zIHqCYsIbtlr6/outputfs/centosd/backup-base_os.tar.gz (compressed)
Comparing disks.
Device sda has size 15733161984, 37580963840 expected
Switching to manual disk layout configuration.
Original disk /dev/sda does not exist in the target system. Please choose an appropriate replacement.
1) /dev/sda
2) /dev/sdb
3) Do not map disk.
#?

Now ReaR recover command stops as it detected that disk layout is not identical. As our desired restore target is /dev/sdb we choose right disk and continue recovery. ReaR will ask to check created restore scripts, but this is not needed in our scenario.

#? 2
2017-01-25 20:54:01 Disk /dev/sdb chosen as replacement for /dev/sda.
Disk /dev/sdb chosen as replacement for /dev/sda.
This is the disk mapping table:
    /dev/sda /dev/sdb
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.

1) View disk layout (disklayout.conf)  4) Go to Relax-and-Recover shell
2) Edit disk layout (disklayout.conf)  5) Continue recovery
3) View original disk space usage      6) Abort Relax-and-Recover
#? 5
Partition primary on /dev/sdb: size reduced to fit on disk.
Please confirm that '/var/lib/rear/layout/diskrestore.sh' is as you expect.

1) View restore script (diskrestore.sh)
2) Edit restore script (diskrestore.sh)
3) View original disk space usage
4) Go to Relax-and-Recover shell
5) Continue recovery
6) Abort Relax-and-Recover
#? 5
Start system layout restoration.
Creating partitions for disk /dev/sdb (msdos)

Disk /dev/sdb: 6527 cylinders, 255 heads, 63 sectors/track
Old situation:
Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sdb1   *      0+     91-     92-    731449   83  Linux
/dev/sdb2         91+   3235-   3145-  25258396+  83  Linux
/dev/sdb3       3235+   6527-   3292-  26436900   83  Linux
/dev/sdb4          0       -       0          0    0  Empty
New situation:
Units: sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *      2048   1026047    1024000   7  HPFS/NTFS/exFAT
/dev/sdb2       1026048  36386815   35360768   7  HPFS/NTFS/exFAT
/dev/sdb3      36386816  73400319   37013504  83  Linux
/dev/sdb4             0         -          0   0  Empty
Successfully wrote the new partition table

Re-reading the partition table ...

Creating filesystem of type xfs with mount point / on /dev/sdb3.
Mounting filesystem /
Disk layout created.
Restoring from '/tmp/rear.70zIHqCYsIbtlr6/outputfs/centosd/backup-base_os.tar.gz'...
Restoring usr/lib/modules/3.10.0-514.2.2.el7.x86_64/kernel/drivers/net/wireless/realtek/rtlwifi/rtl8723be/rtl8723be.koRestoring var/log/rear/rear-centosd.log OK
Restored 2110 MiB in 103 seconds [avg. 20977 KiB/sec]
Restoring finished.
Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
Patching '/etc/default/grub' instead of 'etc/sysconfig/grub'
Patching '/proc/1909/mounts' instead of 'etc/mtab'
Skip installing GRUB Legacy boot loader because GRUB 2 is installed (grub-probe or grub2-probe exist).
Installing GRUB2 boot loader
Finished recovering your system. You can explore it under '/mnt/local'.
Saving /var/log/rear/rear-centosd.log as /var/log/rear/rear-centosd-recover-base_os.log

Now we have Linux part restored, GRUB installed and partitions created, hence we can continue with Windows 10 boot partition recovery.

RESCUE centosd:~ # rear -C windows_boot restoreonly
Restore windows_boot.nc.img to device: [/dev/sda1] /dev/sdb1
Ntfsclone image version: 10.1
Cluster size           : 4096 bytes
Image volume size      : 524283904 bytes (525 MB)
Image device size      : 524288000 bytes
Space in use           : 338 MB (64.4%)
Offset to image data   : 56 (0x38) bytes
Restoring NTFS from image ...
Syncing ...

Similarly to Linux restore, we were prompted for restore destination, which is /dev/sdb1 in our case.
As the last step we will recover Windows 10 data partition

RESCUE centosd:~ # rear -C windows_data restoreonly
Restore windows_data.nc.img to device: [/dev/sda2] /dev/sdb2
Ntfsclone image version: 10.1
Cluster size           : 4096 bytes
Image volume size      : 18104709120 bytes (18105 MB)
Image device size      : 18104713216 bytes
Space in use           : 9867 MB (54.5%)
Offset to image data   : 56 (0x38) bytes
Restoring NTFS from image ...
Syncing ...

Again after restoreonly command is launched, ReaR prompts for restore destination.
Now both operating systems are restored and we can reboot.