nfs_server as new restore method

[codefritzel]

Relax-and-Recover (ReaR) Pull Request Template

Please fill in the following items before submitting a new pull request:

Pull Request Details:

• Type: New Feature

• Impact: Low

• Reference to related issue (URL):

• How was this pull request tested?
  Manually on Ubu22.04 and OL8.7

• Brief description of the changes in this pull request:
  This pull request adds NFS_SERVER as a new restore method. In the rescue stage rear starts an NFS server and exports all mount points. With NFS version 4 the rootfs to be restored can be mounted with the following command:
  mount -t nfs -o nfsvers=4 <ip>:/ /mnt
  Another server with working backup software can mount this share and restore all data to it.
  Rear waits until the file 'rear_finished.txt' is created and all connections to the nfs server are disconnected. After that Rear continues with the restore process

[jsmeix]

@codefritzel
thank you for your contribution to enhance ReaR!

[jsmeix]

The changes in usr/share/rear/conf/default.conf indicate
that this pull request is currently based on an older code state
because the current changes in usr/share/rear/conf/default.conf
revert things that were meanwhile changed in our current ReaR master code.
In the end this means the changes in this pull request should be
rebased on the state of our current ReaR master code.

[jsmeix]

All new added files are shown in
https://github.com/rear/rear/pull/2973/files
with a "No newline at end of file" mark
which means the last line does not end with a newline character.
As far as I know a newline character at the end of every line
(including the last line) is required at least by some programs
that read input line by line where only the newline character
marks the end of a line so that a line without terminating
newline character will not be read by such programs.

[schlomo]

Thanks for the contribution! Please first of all rebase to current master.

Besides the detailed feedback I'd like to suggest one major change: Let's call this NFS4SERVER and limit the code to NFSv4 only, instead of trying to cover NFS 3 and 4 at the same time.

This change will still cover your use case but significantly reduce the testing efforts and make this code here smaller and simpler.

[schlomo]

You'll have to check also if the modules are actually available

[schlomo]

Main thought is that if rear mkrescue works, then there is a high chance that the configured features will also work. At the moment this is more on the level of "I hope that it will work"

[codefritzel]

What is the best method to check if the modules are available?

[jsmeix]

Regarding a check if kernel modules are available:
I assume what is meant is "available as files"
(i.e. no builtin kernel "modules").
You may have a look at the code and the comments of the

function modinfo_filename ...

in usr/share/rear/build/GNU/Linux/400_copy_modules.sh
which is in our current GitHub master code at
https://github.com/rear/rear/blob/master/usr/share/rear/build/GNU/Linux/400_copy_modules.sh#L26

In that script at its end we add all what there is
in MODULES_LOAD to "$ROOTFS_DIR/etc/modules" which is used in
skel/default/etc/scripts/system-setup.d/40-start-udev-or-load-modules.sh
that gets installed in the ReaR recovery system as
/etc/scripts/system-setup.d/40-start-udev-or-load-modules.sh
which basically calls modprobe -v $module for all
what there is in MODULES_LOAD so

MODULES_LOAD+=( QQQ )

results that modprobe -v QQQ will be called during
ReaR recovery system startupt which does not cause harm
but only an error message

# modprobe -v QQQ
modprobe: FATAL: Module QQQ not found in directory /lib/modules/...

which seems to be OK from my current point of view.

[gdha]

@codefritzel Could you please add some clear clarification why you need a NFS_SERVER workflow? Is NFS client not enough?

[schlomo]

I can add some more context for this, I see three main scenarios for restore via NFS server:

1. In some cases there is a problem running the backup client in our rescue system, and then it might be "cheaper" to use an indirect restore scheme instead of spending more hours on enabling our rescue system to support that piece of commercial software
2. From a security perspective, it is always better to connect from a high-security ("bastion") system to the periphery. For backups this means that one could implement an NFS-based backup solution with pulling the date onto a central server. For restore we then need to start an NFS server in ReaR to push the data back. This could be part of a self-written solution or even use a commercial backup tool that is installed only on the central backup server for licensing or other reasons.
3. Implement asymmetric solutions, using one tool for backup and another tool for restore, in this case via NFS

In all those cases adding NFS server capabilities to the ReaR rescue system is IMHO a useful addition. I'd also consider modifying it to be an optional add-on feature instead of a stand-alone backup method. I also envision introducing a general split of BACKUP and RESTORE, if somebody needs this.

[gdha]

@schlomo Thanks for the heads up as it makes it clear to me why it might be useful.
@codefritzel That being said please add some more comments from your side of the need of this new workflow, not just that it could be useful. And, above all, without proper documentation it will be a workflow only used by a happy few (amongst yourself I'm afraid).

[schlomo]

Suggested change

	# same options works for mountd
	# same options works for rpc.nfsd and rpc.mountd

[schlomo]

Just a thought: This while loop can potentially run forever. If you'd implement the df monitoring suggested below then you could also error out if the amount of restored data doesn't change within a configurable certain time, e.g. 5 mins or so.

[schlomo]

Thanks a lot @codefritzel for the rework, I added you a bunch of minor fixes/changes and then I think we are good to go.

[pcahyna]

Hi, looks useful, but I have a question: will the various extended attributes/ACLs get restored properly? SELinux contexts are probably handled in usr/share/rear/restore/default/500_selinux_autorelabel.sh, but there are other extended attributes that matter. See:

$ getfattr -m- -nsecurity.capability - /usr/sbin/* /usr/bin/* 2>/dev/null
# file: usr/sbin/arping
security.capability=0sAAAAAgAgAAAAAAAAAAAAAAAAAAA=

# file: usr/sbin/clockdiff
security.capability=0sAAAAAgAgAAAAAAAAAAAAAAAAAAA=

# file: usr/bin/newgidmap
security.capability=0sAQAAAkAAAAAAAAAAAAAAAAAAAAA=

# file: usr/bin/newuidmap
security.capability=0sAQAAAoAAAAAAAAAAAAAAAAAAAAA=

AFAIK if one uses NFSv4.2, then one is able to use such attributes over the NFS mount, but I have not tried it. -V 4.2 needs to be passed to rpc.nfsd and mount option v4.2 (or nfsvers=4.2?) should be specified.

[jsmeix]

I am not a NFS expert so I may confuse something but

NFS4SERVER_TRUSTED_CLIENTS=( myserver ...)

looks contradicting i.e. a trusted client called 'myserver'?

[schlomo]

With NFS, a "client" connects to a "server" who authorises the "client" during connection. Therefore the term "allowed clients" or "permitted clients" or even just "clients" would probably be most correct.

[jsmeix]

My point was the possibly misleading word 'myserver' so

# e.g. NFS4SERVER_TRUSTED_CLIENTS=( myclient 172.16.76.0/24 )

seems to be better wording for this example.

[schlomo]

Very nice, I think primarily the verify check for clients and maybe renaming the variable needs to be done.

Everything else, like a df based progress indicator, can also come later.

[pcahyna]

You could omit this check from this file and add a symlink pointing to usr/share/rear/verify/NFS4SERVER/default/400_verify_nfs_server.sh to this directory instead to reduce code duplication a bit.

[schlomo]

I think better to not mess with the MODULES_LOAD variable during verify because there no code cares about it and I don't like code that is meaningless but appears to be important. It might confuse other people who come to work on ReaR.

[pcahyna]

@schlomo oh, I meant only the last line, not the stuff above.

[pcahyna]

Is it intended to turn on debugging unconditionally?

[schlomo]

Yes, in case of error we'll see a bit more useful stuff, otherwise it doesn't matter. As we capture stdout the user isn't impacted.

[pcahyna]

ditto about --debug

[pcahyna]

Another question about NFS. Is NFSv4 required because of its ability to automatically export and access filesystems mounted under the exported filesystem, i.e. to cross mountpoints? Seems pretty crucial for use in ReaR.

[codefritzel]

Another question about NFS. Is NFSv4 required because of its ability to automatically export and access filesystems mounted under the exported filesystem, i.e. to cross mountpoints? Seems pretty crucial for use in ReaR.

@pcahyna that is one reason and it is much easier to support only v4 than v2, v3 and v4 together.

[schlomo]

Thank you everybody for reviewing this. I'm going forward and merging it now, we'll continue to improve this code as it undergoes more testing in production.