grub: increase kernel ring buffer size to 2MB#5627
Conversation
rucoder
force-pushed
the
rucoder/increase-kernel-log-buf
branch
from
b66812e to
3f76de0
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5627 +/- ##
==========================================
+ Coverage 19.52% 29.49% +9.96%
==========================================
Files 19 18 -1
Lines 3021 2417 -604
==========================================
+ Hits 590 713 +123
+ Misses 2310 1552 -758
- Partials 121 152 +31 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| set_global dom0 /boot/kernel | ||
| set_global dom0_rootfs "root=$rootfs_root" | ||
| set_global dom0_cmdline "$linuxkit_cmdline $panic_timeout rfkill.default_state=0 split_lock_detect=off" | ||
| set_global dom0_cmdline "$linuxkit_cmdline $panic_timeout rfkill.default_state=0 split_lock_detect=off log_buf_len=4M" |
There was a problem hiding this comment.
I always prefer to be more conservative thinking on devices that uses Satellite connectivity. I understand 128KB is just simple not enough for bigger devices, but 4MB, as you said, looks overkill. @rucoder in your tests, what is the minimum need to not lost kernel logs? 1MB or 2MB would be enough?
There was a problem hiding this comment.
@rene this is hard to tell. I saw servers with 100s of PCI , but yeah, 4Mb may be an overkill. 1M ? (~9x) ? and btw, it won't affect any traffic consumption, this is just RAM: whether we loose logs or not
Add log_buf_len=2M to the kernel command line in grub rootfs.cfg to increase the kernel ring buffer from the default 128KB to 2MB. This prevents early dmesg messages from being lost when the kernel ring buffer overflows before newlogd starts draining /dev/kmsg. This can happen on any system with a large number of PCI devices, where hardware probing and driver initialization generate enough messages to overflow the 128KB buffer even under normal load. It is also critical for RT workloads where kernel-level diagnostics (latency spikes, IRQ storms, OOM events) are the most valuable debug information. The 2MB buffer provides headroom for ~15,000-25,000 kernel messages. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
rucoder
force-pushed
the
rucoder/increase-kernel-log-buf
branch
from
3f76de0 to
8c3bdb7
Compare
rucoder
changed the title
2 participants
Description
NOTE: 2M may look like an overkill but on really big machine it is better to be save than sorry.
Add
log_buf_len=2Mto the kernel command line in grubrootfs.cfgto increase the kernel ring buffer from the default 128KB (CONFIG_LOG_BUF_SHIFT=17) to 2MB. This prevents early dmesg messages from being lost when the kernel ring buffer overflows beforenewlogdstarts draining/dev/kmsg.This can happen on any system with a large number of PCI devices, where hardware probing and driver initialization generate enough messages to overflow the 128KB buffer even under normal load. It is also critical for RT workloads where kernel-level diagnostics (latency spikes, IRQ storms, OOM events) are the most valuable debug information.
The 4MB buffer provides headroom for ~15,000–25,000 kernel messages. The memory cost is negligible on modern systems.
The
log_buf_lenparameter can also be overridden per-device via:CONFIG/grub.cfg:set_global dom0_extra_args "$dom0_extra_args log_buf_len=8M"eve-kernel-extra-cmdlineHow to test and validate this PR
log_buf_len: 20977152 bytesnewlogdoutput and not lost due to ring buffer overflowChangelog notes
Increased kernel ring buffer size from 128KB to 2MB to prevent loss of early boot kernel messages, particularly on systems with many PCI devices or under heavy system load.
PR Backports
Checklist
I've provided a proper description
I've added the proper documentation
I've tested my PR on amd64 device
I've tested my PR on arm64 device
I've written the test verification instructions
I've set the proper labels to this PR
I've checked the boxes above, or I've provided a good reason why I didn't
check them.