If selinux labels are not restored, the autorelabel is not enough in RHEL8.4 and recovered system does not boot

[bwelterl]

Starting in RHEL8.4, the selinux policy does not allow systemd to access unlabeled files anymore, thus if the restored filesystem has not the correct labels (for example /etc/localtime), systemd will not be able to boot and relabel will fail and system will not boot.

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

• ReaR version ("/usr/sbin/rear -V"):
  rear 2.6

• OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
  RHEL 8.4

• ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
  with a backup tool that does not restore selinux labels (TSM, rsync...)

• Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
  VM (and HW)

• System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
  x86

• Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
  UEFI

• Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
  Local

• Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT"):

• Description of the issue (ideally so that others can reproduce it):

When restoring a system with selinux enabled, but where the backup does not restore the label, the system is not bootable because autorelabel will happen to late, systemd already fails with:

Failed to create timezone change event source: Permission denied

because in RHEL 8.4, systemd is not able to read unlabeled files anymore.

This case can happen with different backup tools:
rsync with remote filesystem not able to manage extended attributes
TSM as customer, because labels for link are no saved, thus /etc/localtime will be unlabelled an systemd will fail.

• Workaround, if any:

First reboot in permissive to allow systemd to execute the autorelabel
Fix: execute setfiles before the reboot:

setfiles -r $TARGET_FS_ROOT $TARGET_FS_ROOT/etc/selinux/${policy}/contexts/files/file_contexts $TARGET_FS_ROOT

[pcahyna]

Note that there are other system extended attributes that the system may depend on, so I am afraid that systems restored with "rsync with remote filesystem not able to manage extended attributes" will be subtly broken anyway. Given this, it might be cleaner to desupport entirely backup tools that are not able to backup & restore all the file metadata, and drop workarounds like relabeling. Does TSM have problems with all extended attributes, or is selinux somehow specific?

[bwelterl]

Hello,

I agree that autorelabel is a workaround, that does not work always anymore.
And yes, removing the support of broken backup tools is the easiest way to "fix" this (but with some complains).

I don't have details on TSM, only "SELinux attributes of symbolic links are not backed up and thus cannot be restored.":
https://www.ibm.com/support/pages/ibm-spectrum-protect-v81-unix-and-linux-backup-archive-client-known-problems-and-limitations
It's maybe something similar to rsync with non root user.

I submitted this pullrequest, to be discussed :-) at least for the inclusion of setfiles binary:
d5505e6

Thanks !

[pcahyna]

Thanks for the link. Indeed, TSM docs show that the problem is specific to SELinux and symbolic links, so the proposed workaround is probably enough.
For RSYNC, see #2717 (comment)

[gerhard-tinned]

Hi,

Personally I see this less on an issue in ReaR but more an Issue with RedHats autorelabeling. The problem emerged with RHEL 8.4 but worked before. So RedHat had done something to break autorelebel.

It is very frustrating to see how RedHat is trying to hide the Problem by making other projects add workarounds. :-( At the end of the day, autorelabeling is considered a fix to wrong selinux fcontesxt labels. But it is broken as it is not working when some specific files (my experience, there is more than just this one file)are not labeled correctly.

Oh, YES, the TSM dokumentation states the limitation for the symbolic links.

If I can help with this issue in any way I would be happy to help. As it seems this issue is relöated to my issue reported to RedHat. (not that they would have told me, I had to find that out myself.. :-(

[bwelterl]

The goal of rear is to restore the system in the previous state.

Once it's rebooted to the previous kernel, it no longer has control on the system. If we rely on something dependants of the first reboot and it fails, we can't give a workaround.
Thus it's better to have everything correctly set before the first new reboot. That's why the PR with setfiles helps to configure everything before the reboot.
And of course, the easiest way is to not support backup tool without extended attributes feature.

[pcahyna]

Hi,

Personally I see this less on an issue in ReaR but more an Issue with RedHats autorelabeling. The problem emerged with RHEL 8.4 but worked before. So RedHat had done something to break autorelebel.

It is very frustrating to see how RedHat is trying to hide the Problem by making other projects add workarounds. :-( At the end of the day, autorelabeling is considered a fix to wrong selinux fcontesxt labels. But it is broken as it is not working when some specific files (my experience, there is more than just this one file)are not labeled correctly.

There is a fix for this in Fedora: fedora-selinux/selinux-policy#968

If / when it is included in RHEL, I propose to abandon the proposed workaround, ReaR code is complicated enough already, no need to complicate it further with workarounds for breakage in other tools.

[pcahyna]

If I can help with this issue in any way I would be happy to help. As it seems this issue is relöated to my issue reported to RedHat.

@gerhard-tinned Thank you, the report was very useful already! If you are interested and you are using CentOS Stream, the fixed build of selinux-policy that fixes systemd access to unlabeled symlinks is already in CentOS Stream: selinux-policy-3.14.3-86.el8.noarch.rpm. Of course, no guarantees...

[gerhard-tinned]

If I can help with this issue in any way I would be happy to help. As it seems this issue is relöated to my issue reported to RedHat.

@gerhard-tinned Thank you, the report was very useful already! If you are interested and you are using CentOS Stream, the fixed build of selinux-policy that fixes systemd access to unlabeled symlinks is already in CentOS Stream: selinux-policy-3.14.3-86.el8.noarch.rpm. Of course, no guarantees...

Sadly we are on RHEL not CentOS or CentOS stream or however it is called now. So I guess it still needs a while ... Anyway that is not something to discuss in the rear case. Thanks anyway, ReaR is awesome!!

[github-actions]

Stale issue message

[pcahyna]

bwelterl convinced me that we still may need some fix for selinux relabelling, because although the particular probem that triggered it has been fixed in systemd, there may be other issues that will not be as easy to fix - because for correct relabeling, selinux needs to be in permissive mode, which we don't do (and is hard to do automatically).

[github-actions]

Stale issue message