Override default systemd emergency service behavior with coreos-su-login-force.conf

[noelmiller]

In the short term I'd definitely advocate for Fedora and derivatives to carry that change by default and tell anyone making kisoks etc. that they need to disable it. We just chose to carry it in FCOS derivatives by default currently.

We may want to consider shipping this by default for our images. Thoughts? @bsherman @castrojo @KyleGospo

I support adding the coreos-sulogin-force.conf from https://github.com/coreos/fedora-coreos-config/blob/17b7f15f49c00ba1e522f15a72f6df70db01fdcc/overlay.d/05core/usr/lib/systemd/system/emergency.service.d/coreos-sulogin-force.conf#L1 to our desktop builds, from main on up.

This avoids current hassle for users which doesn't provide any real security benefit.

Originally posted by @bsherman in #469 (comment)

[bsherman]

@travier you asked for someone to reach out in your comment.

I started poking around https://github.com/coreos/fedora-coreos-config/blob/stable/overlay.d/05core/usr/lib/systemd/system/emergency.service.d/coreos-sulogin-force.conf to see how this was being done on CoreOS. But I'm unsure how to get started adding this into the atomic desktops.

We can connect however you prefer, here, discord, email. My email is on my Fedora people profile (@bsherman1 not the same as my github handle).

Thank you for the offer to assist!

[travier]

So I've dived into this more and I've found coreos/fedora-coreos-tracker#805 (comment), which makes it a harder sell for switching it on for all Fedora desktops.

It would probably be a good idea to start with a discussion thread on https://discussion.fedoraproject.org/ or on fedora-devel to get more ideas on how this could be safely done.

[travier]

In general, the process to get such a change in Fedora is to make a Change Request to have it be visible to the community and force the discussion to happen (and a decision to be taken).

Writing / drafting a change page following the instructions in https://docs.fedoraproject.org/en-US/program_management/changes_policy/ is good first step.

[bsherman]

So I've dived into this more and I've found coreos/fedora-coreos-tracker#805 (comment), which makes it a harder sell for switching it on for all Fedora desktops.

I read through the linked issue and coreos/fedora-coreos-config#311 plus again reading through the discussion which spawned this current issue The context was: how to reset a root password on atomic desktops when locked out/can't boot/etc. I dare assume, the REASON one wishes to do this is almost always a need to boot single user, which requires the root user to have their password set.

cgwalters provided a known good solution for how to accomplish the root password reset, but also suggested use of emergency.target as an alternative. #469 (comment)

I hope this is a fair assessment of the history/situation:

1. Fedora has had a non-usable single mode or systemd emergency/rescue boot due to a root password not being assigned
2. In general, this is addressed by root password reset workarounds, which though common in the linux desktop world, look a bit different with selinux in the mix
3. with selinux in play, resetting the root password with init=/bin/bash but NOT loading selinux policies, will break labels (I've done this myself), but on traditional Fedora, the user can request an .autorelabel so that's the workaround there
4. however, on atomic desktops and the immutable root filesystem, relabeling is not possible, thus we have the situation which likely resulted in...
5. overlay/05core: Set SYSTEMD_SULOGIN_FORCE=1 for emergency.service coreos/fedora-coreos-config#311 provided a way boot emergency/rescue
6. coreos/fedora-coreos-tracker#805 (comment) addresses hardening the post boot-loader process, though it is in conflict (for the moment) with the working implementation from overlay/05core: Set SYSTEMD_SULOGIN_FORCE=1 for emergency.service coreos/fedora-coreos-config#311
7. Documentation or Functionality for Resetting Root Password #469 (comment) shares the idea of a protocol which could assist in the hardening implementation, but also the reality of not yet having that ability, and the need for less secure su-login-force, but also warnings and workarounds for hardening on a case by case basis.

So, my view:

• hardened by default is absolutely desirable, but on all Fedora systems, the current default is "unhardened"
• on all Fedora (except FCOS derivatives), there is a way to gain root access with console access, by: rebooting, modifying boot options, resetting root password. but, it happens to be more likely that the uninformed user will break their atomic system's labels when attempting this
• on FCOS, we have su-login-force, which by enabling emergency/rescue systemd targets, provides the same level of root access with lower risk a user damaging their system, by not requiring the user to forcibly set a root password. (one less step to root access; a single reboot instead of two)
• It sounds like solving "hardened by default" is a bigger scope problem, but for the moment, we're trying to give users the ability to recover systems from the console without creating further problems, and the "su-login-force" solution does that without truly increasing security risk.

I needed to get that documented for my own understanding, as I had a few gaps in what/why was going on.

Please let me know if I'm missing something @travier and @cgwalters.

If this is reasonably correct assessment, I'll probably proceed by implementing this in our ublue-os images, but also start a Fedora discussion thread.

[travier]

The main problem is that the work around (auto su-login for single on the kernel command line) also triggers if there is a filesystem check error: coreos/fedora-coreos-tracker#805 (comment)

For systems that don't have a GRUB password set, this is not an issue as someone can edit the GRUB command line and get a shell anyway. But for those that have, this will become something else to harden / change.

I agree that we can argue that not a lot of systems have a GRUB password set right now but it would be nice to find a configuration option that only triggers when single is added to the kernel command line.

[bsherman]

The main problem is that the work around (auto su-login for single on the command line) also triggers if there is a filesystem check error

Most systems don't have a GRUB password set so this is not an issue in most cases.

Thank you for restating this... In my attempt to see the big picture, I'd missed the clear detail... the auto su-login DOES bypass GRUB passwords if a fsck fails.

But yes, I agree...

For those that have, this will become something else to harden / change.

This clarifies the scope of recommendation... those deploying "secure" systems/kiosks/etc, after this suggested change, would need to do both:

1. set a grub password
2. disable auto su-login

[travier]

it would be nice to find a configuration option that only triggers when single is added to the kernel command line.

We can probably ask the systemd folks how to do that.

[cgwalters]

It can be done in a systemd generator

[bsherman]

It can be done in a systemd generator

I was inspired this, @cgwalters and created #488

I've spent some time testing this on a modified FCOS installation, plus my custom Silverblue image. It seems to be working as expected.

While this doesn't go so far as to create a protocol, I believe it generally meets the goal suggested here: #469 (comment)

Also, @travier , though not a full solution to overall hardening, it seems to provide the intent of the SYSTEMD_SULOGIN_FORCE=1 without granting root access when a grub password prevents modifying the cmdline. So, it's a partial solution to coreos/fedora-coreos-tracker#805 (comment) and doesn't require warnings about "if you password protect grub, also disable this feature".

Based on the discussions above, I expect this approach would be more acceptable across all Fedora varieties.

Happy to get any feedback from you both as well as uBlue contributors on this.

[bsherman]

Closing this as we've merged #488 and I'll work on upstream contributions separately.

[travier]

Thanks a lot @bsherman for driving those fixes!