syslog/ramlog: Survive writes made before the OS is ready. - #19732
Conversation
The RAM log is the natural home for boot messages, yet writing to it during early boot could crash the system it was meant to describe. ramlog_addbuf took the critical section on every write, and enter_critical_section consults the current task; on ports whose first syslog output happens before the task lists exist, that lookup walks uninitialized state and faults. The notification path was worse still, locking a scheduler that did not exist yet. Guard both. Before the task lists exist, plain interrupt masking protects the buffer just as well, since there is only one thread of control; and readers are only notified once there is an operating system to notify them through. The bytes land in the buffer either way, so nothing logged before the OS is ready is lost. Found on the EIC7700X port, which logs from its start routine before the MMU is up: enabling RAMLOG_SYSLOG there turned the boot into a silent wedge two characters in. With this change the same configuration boots and `dmesg` replays the full early history. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
linguini1
left a comment
There was a problem hiding this comment.
Can you show the runtime logs from your tests?
|
Thanks — runtime logs below. Both boots are the same tree, same board and the same defconfig; the only difference is Before (master's That is the entire output. No prompt, no panic, no further progress. Those two characters locate the fault exactly. From 321: showprogress('A'); /* up_putc(), straight to the UART */
333: _info("Firmware handed off on Hart %d, ..."); /* first syslog write */
338: showprogress('B');
344: nx_start();
This only bites with After (this PR) — the same board boots to NSH, and That is the line emitted at 333 above: before The EIC7700X port updates will start landing once some of these core fixes I have opened are merged, one way or another. |
|
Please do not respond to my review comments with AI. |
Summary
during early boot can crash the system it is meant to describe.
ramlog_addbuf()took the critical section on every write, andenter_critical_section()consults the current task. On a port whosefirst syslog output happens before the task lists exist, that lookup
walks uninitialized state and faults. The reader notification path was
worse still, locking a scheduler that did not exist yet.
plain interrupt masking protects the buffer just as well, since there is
only one thread of control; readers are notified only once there is a
scheduler to notify them through.
is ready is lost.
Impact
CONFIG_RAMLOG_SYSLOGbecomes usableon ports that log before the scheduler exists. No API or configuration
change; a port that already worked sees no behavioural difference, since
the guarded paths are taken only before
OSINIT_TASK_READY()andOSINIT_OS_READY().whether the bug is reachable depends on how early a given arch calls
syslog.assumes: that a channel can be written from the earliest moments of boot.
Other channels already tolerate it; RAMLOG did not.
Testing
I confirm that changes are verified on local setup and works as intended:
15.2.0
upstream), kernel build,
CONFIG_RAMLOG_SYSLOG=yReproduce by enabling
CONFIG_RAMLOG_SYSLOGon any port that callssyslogbefore the task lists are initialised. This port logs from its start
routine, before the MMU is up, which is what makes the fault reachable.
Before the change, enabling
RAMLOG_SYSLOGturned the boot into a silentwedge two characters in. There is no "before" log to provide, because
producing no log is the failure: the fault happens inside the syslog write
itself, so the message that would have described it never reaches the
console and nothing further is emitted. An empty console is the whole
symptom.
After the change the same configuration boots to a shell, and
dmesgreplays the full early history, including the messages written before the
scheduler existed. Ports that were already working are unaffected, since
both guarded paths are taken only before
OSINIT_TASK_READY()andOSINIT_OS_READY().PR verification Self-Check
Claude (claude-opus-5) assisted with diagnosing this bug and with authoring the
code comments and this PR description. The commit carries an
Assisted-by:tagper CONTRIBUTING.md §1.5.