Let ReaR deal with LVM VGs without LVs

[david-hill]

Relax-and-Recover (ReaR) Issue Template

rear should skip mounted loop devices

[dhill@knox rear]$ git diff
diff --git a/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh b/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
index e01dbf46..dd970944 100644
--- a/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
+++ b/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
@@ -82,6 +82,9 @@ local lvs_exit_code
         # Skip lines that are not describing physical devices
         # i.e. lines where pdev does not start with a leading / character:
         test "${pdev#/}" = "$pdev" && continue
+        if [[ "${pdev}" =~ '/dev/loop' ]] then
+            continue
+        fi
 
         # Output lvmdev header only once to DISKLAYOUT_FILE:
         if is_false $header_printed ; then

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

• ReaR version ("/usr/sbin/rear -V"):
  All

• OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
  RHEL 8.4

• ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

• Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):

• System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):

• Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):

• Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):

• Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):

• Description of the issue (ideally so that others can reproduce it):

• Workaround, if any:

• Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

2022-08-30 12:46:16.127472961 Verifying that the 'disk' entries in /var/lib/rear/layout/disklayout.conf are correct
1200210141184
2022-08-30 12:46:16.130304394 Verifying that the 'part' entries for /dev/sda in /var/lib/rear/layout/disklayout.conf are correct
1048576
1048576
1200208027136
2097152
2022-08-30 12:46:16.132631819 Verifying that the 'part' entries for /dev/sda in /var/lib/rear/layout/disklayout.conf specify consecutive partitions
2022-08-30 12:46:16.136434519 Verifying that the 'lvm...' entries in /var/lib/rear/layout/disklayout.conf are correct
2022-08-30 12:46:16.142924055 LVM no 'lvmvol /dev/cinder-volumes' for 'lvmdev /dev/cinder-volumes'
2022-08-30 12:46:16.149657393 ERROR: 

[gdha]

@david-hill Could you copy/paste the output of the df and mount command please? Loop devices are normally skipped (on my ubuntu ws I have more then 10 loop devices which are skipped during a rear savelayout).

[david-hill]

@david-hill Could you copy/paste the output of the df and mount command please? Loop devices are normally skipped (on my ubuntu ws I have more then 10 loop devices which are skipped during a rear savelayout).

Is it a LVM mounted on a loop ? It has to be LVM otherwise you won't have any issues ...

[gdha]

@david-hill Still need to see some output to understand it better...

[pcahyna]

@gdha do you know how do the loop devices get skipped? I don't see any code to skip them under layout/save. There is code in 200_partition_layout.sh that skips generating disk lines for devices that are not in the known list (currently the list is: hd sd cciss vd xvd dasd nvme mmcblk ) but there seems to be nothing to prevent fs lines being generated for them in 230_filesystem_layout.sh ). Surely I am missing something, as usually it has not been a problem.

[pcahyna]

@david-hill could you please provide the /var/lib/rear/layout/disklayout.conf file generated when you got the error?

[pcahyna]

@gdha I don't see that:

dd if=/dev/zero of=/var/tmp/loopfs bs=1024 count=102400
mkfs.xfs /var/tmp/loopfs
mkdir /home/loopfs
mount -o loop /var/tmp/loopfs /home/loopfs
rear savelayout

results in this in layout:

fs /dev/loop0 /home/loopfs xfs uuid=fbb2b182-5f12-49b0-9ff4-9bbf99e3cc35 label=  options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota

so, loop device is not skipped (unless it changed in ReaR 2.7; this was with ReaR 2.6). Not sure why loop devices on Ubuntu are skipped, aren't they mounted under something in AUTOEXCLUDE_PATH ?

[pcahyna]

@david-hill your issue is only tangentially related to loop devices. Using a PV/VG on a disk:

parted --script -a optimal /dev/vdc -- mklabel gpt mkpart extra ext2 1M -1M set 1 lvm on
pvcreate --yes /dev/vdc1
vgcreate sigvg  /dev/vdc1
rear savelayout

results in the same error:

Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
LVM no 'lvmvol /dev/sigvg' for 'lvmdev /dev/sigvg'
ERROR: 
====================
BUG in /usr/share/rear/layout/save/default/950_verify_disklayout_file.sh line 254:
'Entries in /var/lib/rear/layout/disklayout.conf are broken ('rear recover' would fail)'

without any loop device involved. The underlying issue is that ReaR does not like volume groups without logical volumes. Unused volume groups may be unusual, but I think there is nothing really wrong with them, and PR #2603 fixed a bug ( #2596 ) where an unused PV (without VG) aborted mkrescue. So, by analogy, I think we should also allow a VG without any LV (either exclude it, or add it to the layout as empty).

[pcahyna]

What to do with loop devices is a separate question, but I think should be addressed consistently for all storage objects on them (you see above that filesystems on loop devices are not being excluded either).

[david-hill]

I would suggest we just skip /dev/loop ... is there any possibility we have any useful data on there ?

[gdha]

@pcahyna On my ubuntu ws the loop devices are of type squashfs, so this is the reason of exclusion. The one I created manually is included as you can see:

 cat /var/lib/rear/layout/disklayout.conf 
# Disk layout dated 20220917103612 (YYYYmmddHHMMSS)
# NAME                           KNAME       PKNAME    TRAN   TYPE FSTYPE      LABEL   SIZE MOUNTPOINT                    UUID
# /dev/loop0                     /dev/loop0                   loop squashfs           55.6M /snap/core18/2566             
# /dev/loop1                     /dev/loop1                   loop squashfs          142.2M /snap/chromium/2076           
# /dev/loop2                     /dev/loop2                   loop squashfs              9M /snap/canonical-livepatch/146 
# /dev/loop3                     /dev/loop3                   loop squashfs           63.2M /snap/core20/1623             
# /dev/loop4                     /dev/loop4                   loop squashfs              4K /snap/bare/5                  
# /dev/loop5                     /dev/loop5                   loop squashfs           91.7M /snap/gtk-common-themes/1535  
# /dev/loop6                     /dev/loop6                   loop squashfs          162.9M /snap/gnome-3-28-1804/145     
# /dev/loop7                     /dev/loop7                   loop squashfs          400.8M /snap/gnome-3-38-2004/112     
# /dev/loop8                     /dev/loop8                   loop squashfs          114.9M /snap/core/13741              
# /dev/loop9                     /dev/loop9                   loop squashfs             62M /snap/core20/1611             
# /dev/loop10                    /dev/loop10                  loop squashfs          164.8M /snap/gnome-3-28-1804/161     
# /dev/loop11                    /dev/loop11                  loop squashfs            114M /snap/core/13425              
# /dev/loop12                    /dev/loop12                  loop squashfs           55.6M /snap/core18/2560             
# /dev/loop13                    /dev/loop13                  loop squashfs          142.2M /snap/chromium/2082           
# /dev/loop14                    /dev/loop14                  loop squashfs          346.3M /snap/gnome-3-38-2004/115     
# /dev/loop15                    /dev/loop15                  loop squashfs              9M /snap/canonical-livepatch/138 
# /dev/loop16                    /dev/loop16                  loop squashfs           81.3M /snap/gtk-common-themes/1534  
# /dev/loop17                    /dev/loop17                  loop ext4                100M /home/loopfs                  d89d7f90-1bc9-45a5-a795-de9897577f50
...
fs /dev/loop17 /home/loopfs ext4 uuid=d89d7f90-1bc9-45a5-a795-de9897577f50 label= blocksize=1024 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=4081 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered

So, yes it would be better to exclude loop devices if they are not really required.

[pcahyna]

Why shouldn't be any useful data on /dev/loop ? Is it because the backing files of loop devices can be already saved when the filesystem where they reside is backed up? This makes sense, but beware: if the loop devices are mounted or in any other way used (like being PVs for active VGs, esp. when using thin pools) when the backing files are being backed up, the backup will very likely be inconsistent. This is a similar reasoning like why we usually don't back up data by using just dd on the underlying block device.
Properly saving loop devices and all storage objects on top of them (instead of skipping) would be a way around the problem, but would require some special code (to call losetup during layout recreation to set up the loop devices properly).
Another option would be to exclude them and to warn (or error out) when backing files of loop devices are being backed up and active read-write storage objects (filesystems, LVM) reside on top of them. This would guide the user to think how to temporarily deactivate this stuff during backup to prevent inconsistent backups (similarly to how one can shut down database server before backup and start them after to prevent inconsistent database files, cf the description of POST_BACKUP_SCRIPT in default.conf).

[jsmeix]

Let's keep this issue to
"Let ReaR deal with LVM VGs without LVs"
in the same way as we did it
for #2596
via #2603

For the separated question what to do with loop devices, cf.
#2853 (comment)
I created a separated issue
#2865

[jsmeix]

@pcahyna
regarding your above
#2853 (comment)

In general we do not know in ReaR what files are in the backup.
In particular not for alomst all non-internal backup methods
because ReaR does not implement making a backup with
almost all non-internal (external) backup methods,
see "BACKUP SOFTWARE INTEGRATION" in
https://github.com/rear/rear/blob/master/doc/rear.8.adoc

So in general we cannot compare what files are in the backup
with what storage devices or filesystems are in disklayout.conf.

Furthermore even with internal backup methods we do not know
during "rear mkrescue" what files are actually in the backup
because no backup is actually made during "rear mkrescue".

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message