Attempting to suspend-resume on RB4 produces crashdump

[mwasilew]

When trying to run suspend resume test the board switches to ramdump mode.

Steps to reproduce:

1. flash meta-qcom build: https://qli-prod-artifacts.qualcomm.com/qcom-prd-gh-artifacts/qualcomm-linux/meta-qcom/31060902153-1//qcom-distro/iq-8275-evk/qcom-multimedia-image-iq-8275-evk.rootfs.qcomflash.tar.gz. The build comes from nightly: https://github.com/qualcomm-linux/meta-qcom/actions/runs/31060902153
2. boot to shell and login as root
3. run systemctl suspend
4. "press" power button. In the test alpaca was used with KPD PWR GPIO. This corresponds to power button and work well on RB8 board.

Manifest of the build reports the kernel as 7.1+7.2+rc3+git0+8d5dbc1b17

# uname -a
Linux iq-8275-evk 7.2.0-rc3-01271-g8d5dbc1b17ad #1 SMP PREEMPT Fri Jul 31 12:37:44 UTC 2026 aarch64 GNU/Linux

[mwasilew]

I pulled the ramdumps and let claude try to make some sense of it. Here is what it produced. I can confirm that the proposed check

echo 1 > /sys/bus/pci/devices/0000:01:00.0/remove

prevents the crash. It doesn't resolve the issue with resume not working though. Output from Claude below.

Monaco EVK (IQ-8275-EVK) suspend hang — ramdump analysis

Dump: ramdump/ (Qualcomm full DDR dump, qcom_scm.download_mode=1)
Kernel: 7.2.0-rc3-01271-g8d5dbc1b17ad, built 2026-07-31, aarch64-qcom-linux-gcc 16.1.0
Build source: https://github.com/qualcomm-linux/meta-qcom/actions/runs/31060902153
Machine: Qualcomm Technologies, Inc. Monaco EVK — hostname iq-8275-evk, KASLR enabled
Reported symptom: board in suspend, power button pressed, never resumed

---

TL;DR

The board never finished suspending. It deadlocked inside dpm_suspend_noirq()
long before it reached s2idle, with the console already suspended — which is
indistinguishable from "asleep and won't wake".

A CPU is permanently stalled in an MMIO read inside dw_pcie_read() because the
generic PCI PM code tried to put the ath11k/WCN6855 endpoint (0000:01:00.0)
into D3hot. The WCN6855 does not complete the configuration request aimed at its
PM Control/Status register, and on a DesignWare/pcie-qcom root complex an
un-completed config request stalls the controller's AXI slave instead of reading
back all-ones.

Fix: keep the PCI core out of the D-state transition for that endpoint —
see 0001-wifi-ath11k-don-t-let-the-PCI-core-D3hot-the-device-o.patch.

---

How the dump was analysed

No CPU register contexts are present (no CPUCTX*.BIN in load.cmm) — this is a
DDR-only dump. Everything below was reconstructed from memory:

1. OCIMEM.BIN → Qualcomm TZ diag log (NoC error records).
2. Located VMCOREINFO in DDR (SYMBOL(kallsyms_*), NUMBER(kimage_voffset),
   KERNELOFFSET) → resolved the KASLR slide.
   • _text = 0xffffc690799e0000, kimage_voffset = 0xffffc681ca200000
   • Kernel image physically at 0xeaf7e0000 (bank DDRCS1_0.BIN)
3. Decoded kallsyms from the running image. Note: this kernel uses
   self-relative offsets — addr[i] = &kallsyms_offsets[i] + offsets[i] —
   not a single relative_base. Verified against every VMCOREINFO symbol.
4. Built a swapper_pg_dir page-table walker for vmalloc/vmap-stack addresses.
5. Decoded the printk ring buffer (prb, 128 KiB text ring) → full dmesg.
6. Scanned all six DDR banks for kernel-text pointers to PM functions to locate
   the relevant task stacks, then unwound them via the frame-pointer chain.
7. Recovered the ath11k module's ELF symtab/strtab out of module memory to
   name module functions, and disassembled with capstone to confirm control flow.

---

Evidence

1. Last kernel log records

[   41.427406] T1324 PM: suspend entry (s2idle)
[   41.436927] T12   Filesystems sync: 0.004 seconds
[   41.456424] T1324 Freezing user space processes
[   41.463185] T1324 Freezing user space processes completed (elapsed 0.002 seconds)
[   41.470427] T1324 OOM killer disabled.
[   41.473759] T1324 Freezing remaining freezable tasks
[   41.480211] T1324 Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[   41.487892] T1324 printk: Suspending console(s) (use no_console_suspend to debug)
[   41.553787] T1324 dwc3-qcom a400000.usb: port-1 HS-PHY not in L2
[   41.651088] T1324 dwc3-qcom a600000.usb: port-1 HS-PHY not in L2

No PM: suspend exit, no Restarting tasks. The ring buffer never wrapped, so
this really is the end of the log. Suspend stopped at t≈41.65 s.

2. The suspend thread is blocked

PID 1324 (the task that wrote /sys/power/state), stack at phys 0xaaec64000:

el0t_64_sync → el0_svc → __arm64_sys_write → ksys_write → vfs_write
  → kernfs_fop_write_iter → sysfs_kf_write → kobj_attr_store
  → state_store+0x60 → pm_suspend+0x23c
  → suspend_devices_and_enter+0x308
  → dpm_suspend_noirq+0x24
  → dpm_noirq_suspend_devices+0x138
  → device_suspend_noirq+0x34
  → dpm_wait_for_subordinate+0x44
  → dpm_wait_fn → wait_for_completion+0x7c → schedule_timeout → __schedule

It is waiting for a subordinate device whose async noirq suspend never finishes.

3. The async worker is wedged in MMIO

kworker/u34:15, vmap stack 0xffff80008e808000–0xffff80008e80c000
(phys 0xaa48a8000). Full frame-pointer unwind:

#00 dw_pcie_prog_outbound_atu+0x19c   <-- returns into dw_pcie_readl_atu -> dw_pcie_read
#01 dw_pcie_other_conf_map_bus+0xa8
#02 pci_generic_config_read+0x2c
#03 dw_pcie_rd_other_conf+0x38
#04 pci_bus_read_config_word+0x74
#05 pci_read_config_word+0x2c
#06 pci_set_low_power_state.isra.0+0xec
#07 __pci_set_power_state+0xe0
#08 pci_prepare_to_sleep+0x90
#09 pci_pm_suspend_noirq+0x218
#10 dpm_run_callback+0x2c
#11 device_suspend_noirq+0x74
#12 async_suspend_noirq+0x20
#13 async_run_entry_fn+0x34
#14 process_one_work+0x160
#15 worker_thread+0x188
#16 kthread+0x12c
#17 ret_from_fork+0x10

Innermost live frame is dw_pcie_read() at +0x120: ldr w5, [x6] — a plain
leaf MMIO load with no way out. Disassembly of dw_pcie_prog_outbound_atu+0x19c
confirms it is the instruction right after bl dw_pcie_readl_atu, i.e. the
iATU-enable poll.

4. Which device, and which address

Fact	Value	Where it came from
Device	0000:01:00.0 — [17cb:1103], wcn6855 hw2.1, driver ath11k_pci	struct device name via stack pointer scan
Parent chain	0000:01:00.0 → 0000:00:00.0 → pci0000:00 → 1c00000.pci → soc@0	dev->parent walk (so the controller had not suspended yet)
pci_dev->pm_cap	0x40	pci_dev+0xac
ATU parent_bus_addr (cfg0_base)	0x40100000	struct dw_pcie_ob_atu_cfg on the stack
Target of the hung access	0x40100044 = cfg0_base + pm_cap + PCI_PM_CTRL	derived
pci_dev->state_saved	1 — pci_save_state() had just succeeded	pci_dev+0x890 bit 22
pci_dev->current_state	PCI_D0 — the D3hot transition never landed	pci_dev+0xa8
saved_config_space[0]	0x110317cb — valid, freshly read	pci_dev+0x8a0

5. Hardware corroboration (TZ log, OCIMEM.BIN)

[0366b2213][-1](700391 GEM_NOC 5 1 41046a 104 40100044 40000000 0 0)
[036ac231c][-1](70023c CONFIG_NOC 605 1 171 4104 100044 0 0 0 0 0 0 0 0 0 0)

The interconnect logged a timeout for exactly 0x40100044. That is not a
coincidence — it is the PM Control/Status register of 0000:01:00.0.

6. ath11k did not park the device this cycle

ath11k_pci_pm_suspend() / ath11k_pci_pm_suspend_late() were called (both are
registered; .suspend_noirq is not set — verified by reading
dev->driver->pm out of memory), but both ath11k_core_suspend() and
ath11k_core_suspend_late() took their early return 0 path: the guard on
ar->state (offset 0x638, confirmed by str wzr, [x19, #0x638] in
ath11k_mac_op_stop) read 1. No failed to suspend core /
failed to late suspend core warning appears in dmesg either.

So the PCI core's unilateral D3hot attempt is what killed the machine.

---

Root cause

pci_pm_suspend_noirq() runs the driver callback first, then still performs the
D-state transition itself — verified against this kernel's actual disassembly,
the whole block is gated only on state_saved:

if (!pci_dev->state_saved) {
        pci_save_state(pci_dev);
        if (!pci_dev->skip_bus_pm && pci_power_manageable(pci_dev))
                pci_prepare_to_sleep(pci_dev);   /* D0 -> D3hot, pokes PMCSR */
}

Sequence at the hang:

1. pci_save_state() completes — config reads of 0x00–0x3f and the
   capabilities still work.
2. pci_prepare_to_sleep() → pci_enable_wake() writes PMCSR at 0x40100044.
   The WCN6855 never completes it (its internal clocks are gated); the write is
   buffered at the AXI level so the CPU walks on.
3. pci_set_power_state(D3hot) → pci_set_low_power_state() issues the next
   config read. That requires reprogramming the outbound iATU, and the iATU
   register read must drain behind the outstanding write — which never
   completes. CPU stalls forever.
4. ~1 s later the NoC reports the timeout for 0x40100044. Too late.
5. dpm_noirq_suspend_devices() deadlocks; PID 1324 waits in
   dpm_wait_for_subordinate() indefinitely. Console is already suspended, so
   nothing reaches the user.

On x86 this same code path is harmless — an unanswered config read comes back as
all-ones. On a DesignWare / pcie-qcom root complex it wedges the bus.

---

Fix

0001-wifi-ath11k-don-t-let-the-PCI-core-D3hot-the-device-o.patch, in
drivers/net/wireless/ath/ath11k/pci.c:

	ret = ath11k_core_suspend(ab);
	if (ret)
		ath11k_warn(ab, "failed to suspend core: %d\n", ret);

+	/* Snapshot config space while the device still answers, and leave
+	 * dev->state_saved set, so pci_pm_suspend_noirq() skips both
+	 * pci_save_state() and pci_prepare_to_sleep().
+	 */
+	pci_save_state(to_pci_dev(dev));
+
	return 0;

state_saved is the only gate on that block, so this removes every config
access the PCI core would otherwise issue after the driver has parked the
device. The endpoint simply stays in D0 across s2idle — which is what ath11k
already assumes.

Confirm the diagnosis first (no rebuild needed)

echo 1 > /sys/bus/pci/devices/0000:01:00.0/remove   # or: modprobe -r ath11k_pci
echo mem > /sys/power/state

If suspend/resume works with the endpoint gone, the diagnosis is confirmed. Add
no_console_suspend to the kernel cmdline to watch the device-suspend sequence
live.