Reset Diagnostics

Reset diagnostics

An unplanned reboot (watchdog reset) usually leaves no panic text: the kernel log is forwarded by a user-space process and the machine restarts within seconds. This page is the short version; the full write-up with all measurements is docs/RESET_DIAGNOSTICS.md. It belongs to the open RC3 blockers #94 (T31L reset after 52 h) and #108 (T23 reset).

What survives a watchdog reset

Store	T31L / T31X	T23
RTC alarm register (RTCSAR)	survives	no RTC block
CPM scratch register	reset to default	same
CP0 ErrorEPC (the reset PC)	survives	survives
DRAM (ramoops / pstore)	not usable on the tested boards	not usable
SD card (FAT)	survives, user space only	same

What we use

• T31: RTC record. The panic/oops notifier stores a short record in RTCSAR; after the reset the boot script reports it (/proc/diag_lastpanic, resetcause in the syslog). Proven on the T31L and the T31X.
• T23: reset-PC sled. The notifier spins at an address that encodes the reason, an uptime class and the task name; the watchdog reset leaves it in ErrorEPC and the boot script decodes it. Proven on the T23 test camera (reboot, oops and panic are told apart). A non-sled PC means a stock restart or a silent hang.
• SD ring log: the last seconds before a reset, written from user space.
• Recovery: a boot-looping camera is recovered with a FAT32 SD card that holds the full image as autoupdate-full.bin (proven on the T31L).

Lessons

• The kernel symbol table (CONFIG_KALLSYMS) changes the module ABI: modules must be rebuilt with the kernel, otherwise the first modprobe misbehaves and the board boot-loops.
• After reset tests clear the ISP boot guard before loading the modules again.
• A full OTA wipes the overlay (settings and local init scripts).