[BUG] dsp not suspended after 15s when runtime suspend on TGLH_RVP_NOCODEC_ZEPHYR

[XiaoyunWu6666]

Describe the bug
TGLH_RVP_NOCODEC_ZEPHYR dsp not suspended after 15s when check-kmod-load-unload
inner daily 5356 model=TGLH_RVP_NOCODEC_ZEPHYR testcase=check-runtime-pm-status-15

To Reproduce
TPLG=/lib/firmware/intel/sof-tplg/sof-tgl-h-nocodec.tplg ~/sof-test/test-case/check-runtime-pm-status.sh -l 15

Environment
Kernel Branch: topic/sof-dev
Kernel Commit: 8877846e storage: 8877846e
SOF Branch: main
SOF Commit: f96e25f storage: f96e25f
Topology:sof-tgl-h-nocodec.tplg
Platform: TGLH_RVP_NOCODEC_ZEPHYR

Screenshots or console output
[console]

2021-07-19 22:47:06 UTC [REMOTE_INFO] ===== Iteration 1 of 15 for DMIC =====
2021-07-19 22:47:06 UTC [REMOTE_COMMAND] arecord -D hw:0,10 -r 48000 -c 4 -f S16_LE /dev/null -q
2021-07-19 22:47:08 UTC [REMOTE_INFO] runtime status: active
2021-07-19 22:47:08 UTC [REMOTE_COMMAND] kill process: kill -9 56918
2021-07-19 22:47:08 UTC [REMOTE_INFO] arecord killed
2021-07-19 22:47:08 UTC [REMOTE_INFO] wait dsp power status to become suspended
2021-07-19 22:47:25 UTC [REMOTE_ERROR] dsp is not suspended after 15s, end test
2021-07-19 22:47:25 UTC [REMOTE_ERROR] Starting func_exit_handler(), exit status=1, FUNCNAME stack:
2021-07-19 22:47:25 UTC [REMOTE_ERROR]  die()  @  /home/ubuntu/sof-test/test-case/../case-lib/lib.sh
2021-07-19 22:47:25 UTC [REMOTE_ERROR]  func_check_dsp_status()  @  /home/ubuntu/sof-test/test-case/check-runtime-pm-status.sh:46
2021-07-19 22:47:25 UTC [REMOTE_ERROR]  main()  @  /home/ubuntu/sof-test/test-case/check-runtime-pm-status.sh:112
2021-07-19 22:47:25 UTC [REMOTE_INFO] Test Result: FAIL!

[kv2019i]

I could reproduce this on a TGL-U platform as well. This looks the same as #4400 . The log I get is this:

[ 3247.161405] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x40010000: GLB_PM_MSG: CTX_SAVE
[ 3247.663096] sof-audio-pci-intel-tgl 0000:00:1f.3: error: ipc timed out for 0x40010000 size 76
[ 3247.663111] sof-audio-pci-intel-tgl 0000:00:1f.3: preventing DSP entering D3 state to preserve context
[ 3247.663114] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump start ]------------
[ 3247.663145] sof-audio-pci-intel-tgl 0000:00:1f.3: error: hda irq intsts 0x00000000 intlctl 0xc0000000 rirb 00
[ 3247.663151] sof-audio-pci-intel-tgl 0000:00:1f.3: error: dsp irq ppsts 0x00000000 adspis 0x00000000
[ 3247.663176] sof-audio-pci-intel-tgl 0000:00:1f.3: error: host status 0x00000000 dsp status 0x00000000 mask 0x00000003
[ 3247.663180] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump end ]------------
[ 3247.663184] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[ 3247.663202] sof-audio-pci-intel-tgl 0000:00:1f.3: unknown ROM status value ffffffff
[ 3247.663393] sof-audio-pci-intel-tgl 0000:00:1f.3: invalid header size 0xffffffff. FW oops is bogus
[ 3247.663397] sof-audio-pci-intel-tgl 0000:00:1f.3: error: unexpected fault 0xffffffff trace 0xffffffff
[ 3247.663402] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[ 3247.663407] sof-audio-pci-intel-tgl 0000:00:1f.3: ctx_save ipc error -110, proceeding with suspend

... in most cases system will recover from this (driver will ignore the error as DSP is power-cycled anyway). Needs to be fixed still of course.

[lgirdwood]

@XiaoyunWu6666 this is the same bug as the APL version, it's on @kv2019i task list. @dcpleung fyi.