Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic 'barrel' support to ReaR #2863

Closed
jsmeix opened this issue Sep 16, 2022 · 2 comments
Closed

Add basic 'barrel' support to ReaR #2863

jsmeix opened this issue Sep 16, 2022 · 2 comments
Assignees
Labels
fixed / solved / done severe improvement Heavy improvements required for long term success of ReaR.
Milestone

Comments

@jsmeix
Copy link
Member

jsmeix commented Sep 16, 2022

Barrel is a new command-line program for storage management
in particular for a SUSE specific storage layout, see

https://github.com/aschnell/barrel

Barrel can save a whole storage layout (as a devicegraph) via

barrel save devicegraph --name barrel-devicegraph.xml

into an XML file.

Barrel can (re)-create that whole storage layout
from scratch on bare metal via

barrel load devicegraph --name barrel-devicegraph.xml --mapping barrel-mapping.json

with device mapping as needed
(e.g. when the disks on the new machine are different
compared to where "barrel save devicegraph" was run).

The crucial advantage when ReaR uses barrel is that
then ReaR uses SUSE's own tool to store and
recreate the SUSE specific storage layout
(in particular the SUSE specific btrfs structure)
instead of how it is currently done
where ReaR uses its own implementation
to store and recreate the storage layout
(because before barrel there was no SUSE tool
to store and recreate a SUSE specific storage layout),
cf. the section
"The limitation is what the special ReaR recovery system can do"
in
https://en.opensuse.org/SDB:Disaster_Recovery
that reads (excerpt):

The ReaR recovery system with the ReaR installer
is totally different compared to the installation system
on an openSUSE or SUSE Linux Enterprise install medium
with the YaST installer and AutoYaST.
This means when ReaR is used to recover your system,
a totally different installer recreates your system.
Therefore when the initial installation of
the basic operating system from an openSUSE
or SUSE Linux Enterprise install medium had worked,
the special ReaR recovery system may not work
in your particular case or it may work
but recreate your system with some
(possibly subtle but severe) differences.

Barrel is available for SLES15-SP4
so basic 'barrel' support in ReaR will be implemented
for SLES15-SP4 and later.

FYI:
Its name is 'barrel' because it even supports
"shooting yourself in the foot" when used without care ;-)

@jsmeix jsmeix added the severe improvement Heavy improvements required for long term success of ReaR. label Sep 16, 2022
@jsmeix jsmeix added this to the ReaR v2.8 milestone Sep 16, 2022
@jsmeix jsmeix self-assigned this Sep 16, 2022
@jsmeix
Copy link
Member Author

jsmeix commented Oct 5, 2022

Additional basic 'barrel' support increases
the ReaR recovery system ISO image size by 2.45 MiB
(from 77.92 MiB to 80.37 MiB)
and the ReaR recovery system size (i.e. what is
in /var/tmp/rear.XXXX/rootfs/ ) by 6.72 MiB
(from 164.97 MiB to 171.69 MiB)
on my test system: SLES15 SP4 KVM/QEMU VM
with the default SUSE storage structure, cf.
#2876 (comment)
with a rather minimal ReaR recovery system via

FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )

but the ReaR recovery system increase because of what
is needed by 'barrel' should not depend on firmware files
or kernel modules.

jsmeix added a commit that referenced this issue Oct 13, 2022
Implement initial basic 'barrel' support as a first step
mainly in layout/recreate/default/200_run_layout_code.sh
and in the new layout/save/default/550_barrel_devicegraph.sh
Barrel is a command-line program for storage management
in particular for a SUSE specific storage layout
which is available since SLES15-SP4, see
https://github.com/aschnell/barrel
Barrel can save and recreate a whole storage layout, see
#2863
The current basic 'barrel' support is implemented as optional
and additional possibility when the 'barrel' program is there.
ReaR's native method via disklayout.conf and diskrestore.sh
is still used and working unchanged as before in parallel
so that when 'barrel' is used during "rear recover" but it fails,
the user can switch back to recreating with diskrestore.sh
which is the reason for some new choices during "rear recover",
in particular "Again wipe the disks in DISKS_TO_BE_WIPED"
to wipe what 'barrel' had setup before diskrestore.sh is run.
Additionally during "rear recover" it shows now 'lsblk' output
on the user's terminal to make it easier for the user to see
what disk layout got recreated to decide if that is right or wrong.
Furthermore there is now the new choice to
"Confirm what is currently on the disks and continue 'rear recover'"
so that the user can manually fix things (in particular smaller things)
by only using the ReaR shell and then confirm what he created
on his disks and continue with restoring the backup, see
#2876
@jsmeix
Copy link
Member Author

jsmeix commented Oct 13, 2022

With #2876 merged
this particular issue is done so far.

What is currently known to be not yet done:

Current 'barrel' device mapping must be done manually via

BARREL_MAPPING_FILE="/etc/barrel-mapping.json"
COPY_AS_IS+=( $BARREL_MAPPING_FILE )

in etc/rear/local.conf with an appropriate manually created
/etc/barrel-mapping.json file with content for example like

{
    "mapping" : {
        "/dev/sda" : "/dev/disk/by-id/ata-QEMU_HARDDISK_QM00001"
    }
}

which works for me for my KVM/QEMU test VMs because
all my VMs have /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001
as a symbolic link that points to ../../sda

Currently undocumented config variables about 'barrel' support.
I cannot know now what the right way will be so I don't want
to have them "officially documented" in default.conf until I got
a better gut feeling how things should be in practice.

The currently function lsblk_output()
is currently intentionally a local function
in layout/recreate/default/200_run_layout_code.sh
regardless of the other places where lsblk output is shown.
This will be cleaned up later via a separated pull request, cf.
#2876 (comment)

There is no problem handling when DISKS_TO_BE_WIPED
is set to an empty string in the 'else' clause in
layout/recreate/default/120_confirm_wipedisk_disks.sh
i.e. when the default DISKS_TO_BE_WIPED="" in default.conf
is used and the create_disk_label function is not called
for any disk in diskrestore.sh which is is normally
an error case where the empty DISKS_TO_BE_WIPED
is a symptom but not the cause of the problem, cf.
#2876 (comment)

Currently it does not work to run diskrestore.sh
then "Again wipe the disks in DISKS_TO_BE_WIPED"
and then re-run diskrestore.sh
because the second run of diskrestore.sh does nothing
because all is marked as already done by the first run
or more precisely: The second run of diskrestore.sh
does not start from the beginning but skips what was
already done by the first run but when the disks got wiped
there is nothing left what was done by the first run.
The reason why things get marked as done and skipped can be seen
how it is meant to behave in the section "The Ad-Hoc Way" in
https://github.com/rear/rear/blob/master/doc/user-guide/06-layout-configuration.adoc
so one can modify diskrestore.sh step by step and re-run it
and it only does the not yet done things, cf.
#2876 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fixed / solved / done severe improvement Heavy improvements required for long term success of ReaR.
Projects
None yet
Development

No branches or pull requests

1 participant