[26.04_linux-nvidia-bos] NVIDIA: SAUCE: watchdog: sbsa_gwdt: stop the watchdog across the whole system-sleep transition - #569
Open
dcemin-nv wants to merge 1 commit into
Conversation
…e system-sleep transition The driver stops a running watchdog in its own device suspend callback and restarts it in its resume callback. That leaves the watchdog armed, with nobody refreshing it, for the entire early part of suspend entry: userspace freeze, kernel thread freeze, and every device suspend callback that runs before this device's own. The same window exists at the tail end of resume. When the watchdog is running from boot (early_enable=1, previously force_enable=1 downstream; 10 s default timeout) and any device stalls its suspend callback past the timeout, the watchdog resets the system in the middle of suspend entry. On N1x (Yukon) this fired on about 7% of suspend attempts in a randomized stress run (9 resets in 124 suspends; the serial console shows the board dropping into the boot ROM mid-entry with the watchdog reset status set in NONRST_REG2). Two elimination runs confirm the mechanism: the identical stress matrix with the parameter off produced zero resets in 118 suspends, and with this patch applied and the watchdog force-enabled, zero resets in 198 suspends across four runs (the baseline rate predicts about 14). Stop the watchdog from a PM notifier at the *_PREPARE events, before tasks are frozen and device callbacks run, and restart it at the PM_POST_* events, after everything has resumed. The existing dev_pm_ops are kept: their stop/start are idempotent register writes and they preserve the current behaviour if the notifier registration ever fails. This is also upstream-relevant as a companion to the early_enable parameter: the armed-during-entry window exists for any system running the SBSA watchdog from boot. Signed-off-by: David Cemin <dcemin@nvidia.com>
Collaborator
BaseOS Kernel ReviewWarning
|
Contributor
PR Validation ReportPatchscan ✅ No Missing FixesAll cherry-picked commits checked — no missing upstream fixes found. PR Lint ✅ All checks passedDetailsChecking 1 commits... Cherry-pick digest: ┌──────────────┬──────────────────────────────────────────────────────────────────┬────────────┬─────────┬───────────────────────────┐ │ Local │ Referenced upstream / Patch subject │ Patch-ID │ Subject │ SoB chain │ ├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤ │ 1237c0608382 │ [SAUCE] watchdog: sbsa_gwdt: stop the watchdog across the whole │ N/A │ N/A │ dcemin │ └──────────────┴──────────────────────────────────────────────────────────────────┴────────────┴─────────┴───────────────────────────┘ Lint: all checks passed. |
Collaborator
|
@dcemin-nv I created the Launchpad bug for this change: Could you please add this link to the PR description? |
Author
|
Tracking info: internal NVbug 6611666 (watchdog initiated reset during suspend entry on the N1x stress runs). Upstream plan: the patch is prepared against current mainline and is being posted to linux-watchdog (Wim Van Sebroeck, Guenter Roeck cc linux-watchdog@vger.kernel.org); I will add the lore link here once it is archived. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BugLink: https://bugs.launchpad.net/ubuntu/+source/linux-nvidia/+bug/2166302
Fix for a watchdog initiated system reset during suspend entry, found and root caused on the N1x laptop program.
The driver stops a running watchdog only in its device suspend callback, which leaves it armed with nobody refreshing it through userspace freeze, kernel thread freeze, and every device suspend callback that runs earlier (same window at the tail of resume). With the watchdog running from boot and a 10 second timeout, any slow device suspend resets the system mid entry. On N1x this fired on about 7 percent of suspend attempts in randomized stress runs (9 resets in 124 suspends); two elimination runs confirm the mechanism (0 resets in 118 with the watchdog off, 0 in 198 with this fix and the watchdog force enabled, where the baseline rate predicts about 14).
The fix moves the stop to a PM notifier at the PREPARE events and the restart to PM_POST, keeping the existing dev_pm_ops as an idempotent fallback.
Validation: board proven on Yukon (the 198 suspend elimination matrix plus a 200 cycle confirmation run on the early_enable rename with zero watchdog resets); shipping in the N1x FastOS kernel.
Notes for review: