SOF: Multicore support failed when _widget_free introduced.

[keyonjie]

We call sof_pipeline_core_enable() in sof_widget_setup() to power up the corresponding DSP cores,

linux/sound/soc/sof/sof-audio.c

Line 160 in 9f92f3c

ret = sof_pipeline_core_enable(sdev, swidget);

but not do the corresponding powering off Cores at sof_widget_free(), e.g. at runtime suspending,

linux/sound/soc/sof/sof-audio.c

Line 94 in 9f92f3c

int sof_widget_free(struct snd_sof_dev *sdev, struct snd_sof_widget *swidget)

this will lead to the subsequent run of the pipeline/widget after the runtime suspend/resume cycle on non-0 DSP core fail as the driver will keep think core 1 is already enabled according to the sdev->enabled_cores_mask,
which is not aligned with the FW side, the FW will complain with "ipc_process_on_core(): core 1 is disable" and reject to run it.
https://github.com/thesofproject/sof/blob/0b0733749c55b39c9e8699538b77c65f0c322397/src/ipc/ipc-common.c#L44

[  272.220926] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx succeeded: 0x30100000: GLB_TPLG_MSG: PIPE_NEW
[  272.220929] sof-audio-pci-intel-tgl 0000:00:1f.3: widget PIPELINE.4.SSP1.IN setup complete
[  272.220931] sof-audio-pci-intel-tgl 0000:00:1f.3: core_enable widget: PCM2P swidget core: 1
[  272.220933] sof-audio-pci-intel-tgl 0000:00:1f.3: sdev->enabled_cores_mask: 0x3
[  272.220935] sof-audio-pci-intel-tgl 0000:00:1f.3: sdev->enabled_cores_mask: 0x3
[  272.220941] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x30010000: GLB_TPLG_MSG: COMP_NEW
[  272.221036] sof-audio-pci-intel-tgl 0000:00:1f.3: error: ipc error for 0x30010000 size 20
[  272.221041] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to load widget PCM2P
[  272.221044] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to restore pipeline after resume -13
[  272.221050] sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at snd_soc_pcm_component_pm_runtime_get on 0000:00:1f.3: -5
[  272.221113]  NoCodec-2: soc_pcm_open() failed (-5)
[  272.221118]  smart-nocodec: ASoC: dpcm_be_dai_startup() failed at NoCodec-2 (-5)
[  272.221122]  smart-nocodec: dpcm_fe_dai_startup() failed (-5)
[  272.233943] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc rx: 0x90020000: GLB_TRACE_MSG

This is actually SOF common issue, not TGL specific, and happen in all scenarios where widgets free is used, including runtime suspend/resume, not dedicated for dynamic pipelines only.

Proposed solution:
add refcount to each DSP core, increase/decrease the refcount when a widget/pipeline using the Core is created/freed, and do the core power up/down and core enable/disable IPC only when needed.

[plbossart]

@keyonjie with all due respect, this is a terrible bug description...

Can you please reword it with code pointers so that we don't have to reverse-engineer your line of thought, or why it's TGL-specific, and whether this is for dynamic pipelines or not.

[keyonjie]

@keyonjie with all due respect, this is a terrible bug description...

Can you please reword it with code pointers so that we don't have to reverse-engineer your line of thought, or why it's TGL-specific, and whether this is for dynamic pipelines or not.

Thanks for feedback, only had 3 minutes to heads up this on github last night, now revised.

[ranj063]

@keyonjie I agree we should power off cores properly depending on refcount with multicore. But we must be missing something in our suspend sequence too. We must not be updating the enabled cores mask when poering up secondary cores via IPC. Something to look into

[keyonjie]

@keyonjie I agree we should power off cores properly depending on refcount with multicore. But we must be missing something in our suspend sequence too. We must not be updating the enabled cores mask when poering up secondary cores via IPC. Something to look into

It looks to me that the powering up of the secondary cores works fine (the enabled cores mask set correctly), but the widget free or runtime suspending not clearing this mask for us. Sorry I assigned this to you as I think you know this part best.

[keyonjie]

@plbossart since @ranj063 is in leave, I may take my old PR #888 back to address this issue.

[keyonjie]

Hi @plbossart @ranj063 I managed to figure out solution for this in my PR #2850, please help to review.

[kv2019i]

This is impacting a key feature in sof-dev, so applying P1.

[plbossart]

@keyonjie isn't this issue solved by PR #2880 already? Please close if this is the case, thank you.

[keyonjie]

@keyonjie isn't this issue solved by PR #2880 already? Please close if this is the case, thank you.

Yes, let's close it, and file new multi-core issues if observed by CI after topology change thesofproject/sof#4153 merged.