Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More verbose messages when components are excluded #2495

Merged
merged 8 commits into from Sep 18, 2020
Merged

More verbose messages when components are excluded #2495

merged 8 commits into from Sep 18, 2020

Conversation

jsmeix
Copy link
Member

@jsmeix jsmeix commented Sep 18, 2020

  • Type: Enhancement

  • Impact: Normal

  • How was this pull request tested?

Tested on my home-office laptop while working on
#2493

  • Brief description of the changes in this pull request:

More verbose messages when components are excluded
so that is is easier for the user to see what the actual results are
when he specified to exclude components in his etc/rear/local.conf
and what components are automatically excluded by ReaR.

There is a (perhaps oversophisticated?) distinction what messages should appear

  • only in the log file in debug '-d' mode via 'Debug'
  • in the log file and on the user's terminal in debug '-d' mode via 'DebugPrint'
  • in the log file and on the user's terminal in verbose '-v' mode via 'LogPrint'

so that the info that is shown on the user's terminal (hopefully) looks consistent.

@jsmeix
Update layout-functions.sh
3be2325 
More verbose debug logging when components are excluded.
@jsmeix jsmeix added the enhancement Adaptions and new features label Sep 18, 2020
@jsmeix jsmeix added this to the ReaR v2.7 milestone Sep 18, 2020
@jsmeix jsmeix requested a review from a team September 18, 2020 10:17
@jsmeix jsmeix self-assigned this Sep 18, 2020
@jsmeix
Update 330_remove_exclusions.sh
9cb27cd 
More verbose messages when components are excluded
and less misleading function names that tell what they actually do
plus generic code cleanup in layout/save/default/330_remove_exclusions.sh
@jsmeix
Update 950_verify_disklayout_file.sh
739755f 
Added LogPrint "Created disk layout ..." to layout/save/default/950_verify_disklayout_file.sh
to match the "Creating disk layout" in layout/save/GNU/Linux/100_create_layout_file.sh
so that it is more clear in the user messages when the disk layout was created.
@jsmeix
Copy link
Member Author

jsmeix commented Sep 18, 2020

On my laptop I have

# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME                                                      KNAME     PKNAME    TRAN   TYPE  FSTYPE        SIZE MOUNTPOINT
/dev/sda                                                  /dev/sda            sata   disk              465.8G 
|-/dev/sda1                                               /dev/sda1 /dev/sda         part                  8M 
|-/dev/sda2                                               /dev/sda2 /dev/sda         part  crypto_LUKS     4G 
| `-/dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 /dev/dm-0 /dev/sda2        crypt swap            4G [SWAP]
|-/dev/sda3                                               /dev/sda3 /dev/sda         part  crypto_LUKS   200G 
| `-/dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part3 /dev/dm-1 /dev/sda3        crypt ext4          200G /
|-/dev/sda4                                               /dev/sda4 /dev/sda         part  ext4          100G /nfs
|-/dev/sda5                                               /dev/sda5 /dev/sda         part  ext4          150G /var/lib/libvirt
|-/dev/sda6                                               /dev/sda6 /dev/sda         part  ext2            8G /other
|-/dev/sda7                                               /dev/sda7 /dev/sda         part  crypto_LUKS     1G 
| `-/dev/mapper/luks1test                                 /dev/dm-3 /dev/sda7        crypt ext2         1022M /luks1test
`-/dev/sda8                                               /dev/sda8 /dev/sda         part  crypto_LUKS     1G 
  `-/dev/mapper/luks2test                                 /dev/dm-2 /dev/sda8        crypt ext2         1020M /luks2test
/dev/sr0                                                  /dev/sr0            sata   rom                1024M

where luks2test is the only LUKS2 volume

My etc/rear/local.conf contains (excerpts)

OUTPUT=ISO
OUTPUT_URL=file:///other
BACKUP=NETFS
BACKUP_URL=file:///other/backup
EXCLUDE_COMPONENTS+=( /dev/mapper/luks1test /dev/mapper/luks2test )

With the changes in this pull request
I get the following output on the terminal:

Full output:

# usr/sbin/rear mkrescue
Incomplete values for LUKS device 'luks2test' at '/dev/sda8' (only LUKS version 1 is supported) see /root/rear.github.master/var/lib/rear/layout/disklayout.conf

Excerpt:

# usr/sbin/rear -v mkrescue
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Incomplete values for LUKS device 'luks2test' at '/dev/sda8' (only LUKS version 1 is supported) see /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Excluding component /dev/mapper/luks1test.
Excluding component /dev/mapper/luks2test.
Excluding component fs:/other
Disabling component 'fs ... /luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /luks2test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /other' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'crypt /dev/mapper/luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2'
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)

Excerpt:

# usr/sbin/rear -d mkrescue
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Incomplete values for LUKS device 'luks2test' at '/dev/sda8' (only LUKS version 1 is supported) see /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Excluding component /dev/mapper/luks1test.
Marking component '/dev/mapper/luks1test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Marking dependant fs:/luks1test as done because it is a child of component /dev/mapper/luks1test
Marking component 'fs:/luks1test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Excluding component /dev/mapper/luks2test.
Marking dependant fs:/luks2test as done because it is a child of component /dev/mapper/luks2test
Marking component 'fs:/luks2test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Excluding component fs:/other
Marking component 'fs:/other' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /luks2test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /other' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'crypt /dev/mapper/luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2'
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)

The rear -D terminal output is same as rear -d.

This is the resulting disklayout.conf:

# Disk layout dated 20200918125906 (YYYYmmddHHMMSS)
# NAME                                                      KNAME     PKNAME    TRAN   TYPE  FSTYPE        SIZE MOUNTPOINT
# /dev/sda                                                  /dev/sda            sata   disk              465.8G 
# |-/dev/sda1                                               /dev/sda1 /dev/sda         part                  8M 
# |-/dev/sda2                                               /dev/sda2 /dev/sda         part  crypto_LUKS     4G 
# | `-/dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 /dev/dm-0 /dev/sda2        crypt swap            4G [SWAP]
# |-/dev/sda3                                               /dev/sda3 /dev/sda         part  crypto_LUKS   200G 
# | `-/dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part3 /dev/dm-1 /dev/sda3        crypt ext4          200G /
# |-/dev/sda4                                               /dev/sda4 /dev/sda         part  ext4          100G /nfs
# |-/dev/sda5                                               /dev/sda5 /dev/sda         part  ext4          150G /var/lib/libvirt
# |-/dev/sda6                                               /dev/sda6 /dev/sda         part  ext2            8G /other
# |-/dev/sda7                                               /dev/sda7 /dev/sda         part  crypto_LUKS     1G 
# | `-/dev/mapper/luks1test                                 /dev/dm-3 /dev/sda7        crypt ext2         1022M /luks1test
# `-/dev/sda8                                               /dev/sda8 /dev/sda         part  crypto_LUKS     1G 
#   `-/dev/mapper/luks2test                                 /dev/dm-2 /dev/sda8        crypt ext2         1020M /luks2test
# /dev/sr0                                                  /dev/sr0            sata   rom                1024M 
# Disk /dev/sda
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sda 500107862016 gpt
# Partitions on /dev/sda
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sda 8388608 1048576 rear-noname bios_grub /dev/sda1
part /dev/sda 4294967296 9437184 rear-noname swap /dev/sda2
part /dev/sda 214748364800 4304404480 rear-noname legacy_boot /dev/sda3
part /dev/sda 107374182400 219052769280 rear-noname none /dev/sda4
part /dev/sda 161061273600 326426951680 rear-noname none /dev/sda5
part /dev/sda 8589934592 487488225280 other none /dev/sda6
part /dev/sda 1073741824 496078159872 playground none /dev/sda7
part /dev/sda 1073741824 497151901696 playground2 none /dev/sda8
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part3 / ext4 uuid=f05af948-6075-40a3-9191-354b0a0a9afc label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16383 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
#fs /dev/mapper/luks1test /luks1test ext2 uuid=84e951c1-170d-489d-b1cc-191f95608d97 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
#fs /dev/mapper/luks2test /luks2test ext2 uuid=c1d5b6e6-7760-4b6a-bfe8-70beda9003d7 label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
fs /dev/sda4 /nfs ext4 uuid=4c4a923d-1562-4254-a1fa-4e761278c02f label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
fs /dev/sda5 /var/lib/libvirt ext4 uuid=4a42395e-4f9d-4056-9948-6d5d9d92d990 label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
#fs /dev/sda6 /other ext2 uuid=259dac9c-f2fd-4181-a351-83603398e465 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 uuid=6d8f8998-dd20-412a-bcc2-618eed858662 label=
crypt /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part3 /dev/sda3 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=a6dba0d8-5be8-4970-b1e7-a272ae0cafdd 
#crypt /dev/mapper/luks1test /dev/sda7 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=1b4198c9-d9b0-4c57-b9a3-3433e391e706 
crypt /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 /dev/sda2 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=54fc77c5-8ec2-457f-b558-9deda3b843b2 
# Incomplete values for LUKS device 'luks2test' at '/dev/sda8' (only LUKS version 1 is supported):
#crypt /dev/mapper/luks2test /dev/sda8 cipher=- key_size= hash= uuid=3e874a28-7415-4f8c-9757-b3f28a96c4d2

This is the resulting disktodo.conf

todo /dev/sda disk
todo /dev/sda1 part
todo /dev/sda2 part
todo /dev/sda3 part
todo /dev/sda4 part
todo /dev/sda5 part
todo /dev/sda6 part
todo /dev/sda7 part
todo /dev/sda8 part
todo fs:/ fs
done fs:/luks1test fs
done fs:/luks2test fs
todo fs:/nfs fs
todo fs:/var/lib/libvirt fs
done fs:/other fs
todo swap:/dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 swap
todo /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part3 crypt
done /dev/mapper/luks1test crypt
todo /dev/mapper/cr_ata-TOSHIBA_MQ01ABF050_Y2PLP02CT-part2 crypt

@jsmeix
Copy link
Member Author

jsmeix commented Sep 18, 2020

@rear/contributors
unless there are objections right now
I would like to merge it a bit later today.

@jsmeix
Update 310_include_exclude.sh
398892d 
More descriptive messages that now also tells the config array
that is the reason why the particular component gets excluded.
@jsmeix
Update 320_autoexclude.sh
abc2d06 
Use DebugPrint for messages that tell what is automatically excluded
so that the user could see that info on his terminal in debug '-d' mode.
@jsmeix
Update 335_remove_excluded_multipath_vgs.sh
65806b1 
Use DebugPrint for messages that tell what is automatically disabled
so that the user could see that info on his terminal in debug '-d' mode.
@jsmeix
Update 400_automatic_exclude_recreate.sh
b5f4047 
Use DebugPrint for messages that tell what is automatically added to EXCLUDE_RECREATE
so that the user could make sense of "Excluding component ... in EXCLUDE_RECREATE"
messages that appear on his terminal when using debug '-d' mode.
@jsmeix
Update layout-functions.sh
71d6397 
Less duplicate wording in messages about marking dependant components as done.
@jsmeix
Copy link
Member Author

jsmeix commented Sep 18, 2020 •
edited

With my recent commits here things look now good to me
at least in my case on my terminal in debug mode - excerpts:

# usr/sbin/rear -d mkrescue
...
Adding backup directory mountpoint 'fs:/other' to EXCLUDE_RECREATE
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Incomplete values for LUKS device 'luks2test' at '/dev/sda8' (only LUKS version 1 is supported) see /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Excluding component /dev/mapper/luks1test in EXCLUDE_COMPONENTS
Marking component '/dev/mapper/luks1test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependant component fs:/luks1test is a child of component /dev/mapper/luks1test
Marking component 'fs:/luks1test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Excluding component /dev/mapper/luks2test in EXCLUDE_COMPONENTS
Dependant component fs:/luks2test is a child of component /dev/mapper/luks2test
Marking component 'fs:/luks2test' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Excluding component fs:/other in EXCLUDE_RECREATE
Marking component 'fs:/other' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /luks2test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /other' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'crypt /dev/mapper/luks1test' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2'
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)

In debug mode it is now visible on the terminal what the reason is why there is that
Excluding component fs:/other in EXCLUDE_RECREATE message.

It is a different issue whether or not automatically excluding the fs:/other is right or wrong.
I think I would like to have that filesystem recreated but I would not want to have the
contents of that filesystem included in the backup.
Because things are now clearly shown on the termial I am informed and I could
reenable the disabled '#fs /dev/sda6 /other ext2 ...` entry in my disklayout.conf
before I run "rear recover" if I like to get that filesystem recreated by ReaR.

@jsmeix jsmeix merged commit 612b995 into master Sep 18, 2020
@jsmeix jsmeix deleted the component-exclude-more-verbose branch September 18, 2020 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@jsmeix jsmeix

Labels
enhancement Adaptions and new features fixed / solved / done
Projects
None yet
Milestone
ReaR v2.7
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@jsmeix