coreos-printk-quiet.service: Lower kernel log level to 4

[cgwalters]

Closes: coreos/fedora-coreos-tracker#1244

A lot going on for this simple service. See the tracker issue above
for more info, but briefly:

Anaconda has historically injected quiet into the kernel command
line in many cases, and this suppresses both kernel and systemd
output. On computers in general, but particularly many bare metal
servers, the serial console can be slow. This can cause reliability
issues.

However for servers, we usually do want to see informational
output when they boot. For example, today the kernel "mitigations"
information for hardware vulnerabilities is output.

This change is a compromise; we boot up at the kernel's default
verbosity level (which for Fedora and derivatives is the upstream 7),
but switch to 4 very early on in the real root. At that point, we've
gotten most of the boot time output, and our initramfs is not
extremely performance sensitive right now.

Also, we explicitly only lower the output log level, and only if
there isn't explicitly debug on the kernel command line.

---

[cgwalters]

I do think this change though is most likely something that should actually happen Fedora wide. Upstreaming this module into dracut might be the easiest.

[lucab]

Out of curiosity, why did you decide against a sysctl.d entry in the real rootfs?

[cgwalters]

I do think this change though is most likely something that should actually happen Fedora wide. Upstreaming this module into dracut might be the easiest.

Alternatively...changing https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-fedora.config#_983 to 4 is likely the better way to do this.

[jlebon]

Out of curiosity, why did you decide against a sysctl.d entry in the real rootfs?

Makes sense to me. It's also easier to disable via Ignition that way. And we'll want to do that for the kola devshell code since I think the state machine there relies on those to know e.g. when to SSH. (But also personally for --devshell-console, I'd just prefer to see everything.)

[cgwalters]

OK 🆕 updated

• Now runs very early in the real root, not initramfs
• No-op if we detect kernel cmdline arguments quiet or debug
• Change logic to always only lower debug output; so if someone e.g. explicitly configures the level to 2, we don't make things more verbose

[cgwalters]

Also now updated to explicitly run Before=systemd-sysctl.service so that configuring printk via Ignition dropping a file in /etc/sysctl.d should still work.

[jlebon]

LGTM!

[bgilbert]

Should we have proper FCOS docs for overriding this?

[cgwalters]

Obviously not opposed to documentation but...in the current design we honor all the standard Linux APIs for this in kernel arguments and injecting a sysctl.d file. The change of default is notable but it's close to a bugfix.

I think I'd say let's tackle documenting the printk level when we push to do this Fedora wide?

[cgwalters]

Addressed all comments!