[BUG][JSL] Failed to load firmware on JSL #4916

yangxiaohua2009 · 2021-10-26T01:11:30Z

Describe the bug
The sof DSP failed to boot and the dmesg shows FW Poll Status: reg[0x80000]=0x80000012 timedout on JSL.
The problem looks like #2888 but the error code is different.

To Reproduce
Select to use sof on JSL.

Reproduction Rate
10/10

Expected behavior
DSP firmware boots normally and the start-up process goes on.

Impact
firmware boot failure.

Environment

Kernel: 5.14.7
SOF: 1:9:0-fa857
Platform: JSL

Screenshots or console output
[ 6.833291] sof-audio-pci-intel-icl 0000:00:1f.3: FW Poll Status: reg[0x80000]=0x80000012 timedout
[ 6.833296] sof-audio-pci-intel-icl 0000:00:1f.3: error: cl_copy_fw: timeout HDA_DSP_SRAM_REG_ROM_STATUS read
[ 6.833824] sof-audio-pci-intel-icl 0000:00:1f.3: FW Poll Status: reg[0x160]=0x140000 successful
[ 6.833866] sof-audio-pci-intel-icl 0000:00:1f.3: unknown ROM status value 80000012
[ 6.833888] sof-audio-pci-intel-icl 0000:00:1f.3: error: extended rom status: 0x80000012 0x2c 0x0 0x0 0x0 0x0 0x2100186 0x0
[ 6.833890] sof-audio-pci-intel-icl 0000:00:1f.3: error: load fw failed ret: -110
[ 6.833917] sof-audio-pci-intel-icl 0000:00:1f.3: error: failed to reset DSP
[ 6.833919] sof-audio-pci-intel-icl 0000:00:1f.3: error: failed to boot DSP firmware -110
[ 6.884371] sof-audio-pci-intel-icl 0000:00:1f.3: FW Poll Status: reg[0x4]=0x1d003c timedout
[ 6.884375] sof-audio-pci-intel-icl 0000:00:1f.3: error: hda_dsp_core_reset_enter: timeout on HDA_DSP_REG_ADSPCS read
[ 6.884378] sof-audio-pci-intel-icl 0000:00:1f.3: error: dsp core reset failed: core_mask 1
[ 6.884658] sof-audio-pci-intel-icl 0000:00:1f.3: error: sof_probe_work failed err: -110

dmesg2.log

keqiaozhang · 2021-10-26T08:50:34Z

I found the same issue on ADLP chrome laptop. Our device is running with UbuntuOS, so I tested with latest kernel/sof-dev+ sof/master branch. This is not the key error and I have added i915.force_probe=46a8 to grub boot option.

[  808.144710] usbcore: deregistering interface driver snd-usb-audio
[  813.840373] rt5682 i2c-10EC5682:00: Using default DAI clk names: rt5682-dai-wclk, rt5682-dai-bclk
[  814.791377] sof-audio-pci-intel-tgl 0000:00:1f.3: DSP detected with PCI class/subclass/prog-if 0x040100
[  814.791625] sof-audio-pci-intel-tgl 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[  814.798107] sof-audio-pci-intel-tgl 0000:00:1f.3: use msi interrupt mode
[  814.812529] sof-audio-pci-intel-tgl 0000:00:1f.3: hda codecs found, mask 4
[  814.812673] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware info: version 1:8:0-9a35f
[  814.812675] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware: ABI 3:18:1 Kernel ABI 3:18:0
[  814.812682] sof-audio-pci-intel-tgl 0000:00:1f.3: unknown sof_ext_man header type 3 size 0x30
[  814.829361] usbcore: registered new interface driver snd-usb-audio
[  815.716850] sof-audio-pci-intel-tgl 0000:00:1f.3: error: cl_dsp_init: timeout HDA_DSP_SRAM_REG_ROM_STATUS read
[  815.716873] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[  815.716880] sof-audio-pci-intel-tgl 0000:00:1f.3: Boot iteration failed: 3/3
[  815.716886] sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_BOOT_IN_PROGRESS (2)
[  815.716916] sof-audio-pci-intel-tgl 0000:00:1f.3: extended rom status:  0x6000021 0x0 0x0 0x0 0x0 0x0 0x256051f 0x0
[  815.716921] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[  815.716946] sof-audio-pci-intel-tgl 0000:00:1f.3: error: dsp init failed after 3 attempts with err: -110
[  815.717036] sof-audio-pci-intel-tgl 0000:00:1f.3: Failed to start DSP
[  815.717042] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to boot DSP firmware -110
[  815.769469] sof-audio-pci-intel-tgl 0000:00:1f.3: error: hda_dsp_core_reset_enter: timeout on HDA_DSP_REG_ADSPCS read
[  815.769485] sof-audio-pci-intel-tgl 0000:00:1f.3: error: dsp core reset failed: core_mask 1
[  815.770376] sof-audio-pci-intel-tgl 0000:00:1f.3: error: sof_probe_work failed err: -110

Full dmesg is attached:
dmesg.txt

mengdonglin · 2021-10-26T15:33:33Z

@yangxiaohua2009 Can you clone https://github.com/thesofproject/sof-bin/? Please try 2 firmware binaries:

1st one is v1.9.x/sof-v1.9/intel-signed/sof-jsl.ri. It's signed by Intel production key.
2nd one is v1.9.x/sof-v1.9/community/sof-jsl.ri It's signed by SOF community key.

@keqiaozhang I feel the problem you observe on ADL-P Chrome book may be a different one. There is no "W Poll Status: reg[0x80000]=0x80000012" in your test. Can you please try adl-002 debug-signed FW release?

plbossart · 2021-10-26T17:27:00Z

@yangxiaohua2009 are you using a JSL device with ES8336? We've got a similar report here thesofproject/linux#3210 of the same problem (EDIT closed as duplicate of #4916)

@keqiaozhang @mengdonglin I don't think we should add ADL information here, this cannot be the same issue and I don't think it's a Chrome problem either.

keqiaozhang · 2021-10-27T01:28:59Z

@keqiaozhang I feel the problem you observe on ADL-P Chrome book may be a different one. There is no "W Poll Status: reg[0x80000]=0x80000012" in your test. Can you please try adl-002 debug-signed FW release?

yes, I have tried adl-002 debug-signed firmware binary, same one which chrome team is using, but this issue is still there.
I checked with Sathya yesterday, he also checked the coreboot image(except ubuntu payload) provide by Meera, adl-002 debug-signed firmware can be loaded with that coreboot under ChromeOS, so it seems like not a coreboot issue but something related to our driver or firmware.

keqiaozhang · 2021-10-27T01:30:33Z

@keqiaozhang @mengdonglin I don't think we should add ADL information here, this cannot be the same issue and I >don't think it's a Chrome problem either.

Sure, I will file a new bug to track the issue on ADLP and remove the adlp information form this bug.

keqiaozhang · 2021-10-27T07:17:48Z

A new bug #4923 has been filed to track the issue on ADLP.

plbossart · 2021-11-05T20:11:24Z

@lgirdwood @mwasko this problem was reproduced by @hli25, it would appear the firmware was signed either with the wrong key, wrong masks or wrong CSE tools.

mwasko · 2021-11-08T16:53:30Z

@plbossart the JSL 1.9 FW signature looks ok, but I will take a closer look on that

plbossart · 2021-11-10T17:07:43Z

@yangxiaohua2009 @tomty89 @systematicat can you unzip this file
sof-jsl-signed.ri.gz as /lib/firmware/intel/sof/sof-jsl.ri and retry? Internal tests show that this solves the authentication problem. This file would be added to the 1.9.2 release if it's successful. Thanks!

lgirdwood · 2021-11-12T17:49:39Z

@marc-hb fyi - will need a release update.

marc-hb · 2021-11-12T18:14:08Z

it would appear the firmware was signed either with the wrong key,

Indeed it looks like JSL was signed with the ICL key:

for i in ./sof-bin/v1.9.x/sof-v1.9/intel-signed/sof-*.ri; do
  echo "$i"; ./sof/tools/sof_ri_info/sof_ri_info.py -v "$i" | grep -i key
done

sof-bin/v1.9.x/sof-v1.9/intel-signed/sof-glk.ri
        1f f4 58 74 64 d4 ae 90 ... b5 c2 49 4e 2a 5f 47 c2 (APL Intel prod key)
sof-bin/v1.9.x/sof-v1.9/intel-signed/sof-icl.ri
        63 df 54 e3 c1 e5 d9 d2 ... 35 f1 ec 4d 93 73 e6 c4 (ICL Intel prod key)
sof-bin/v1.9.x/sof-v1.9/intel-signed/sof-jsl.ri
        63 df 54 e3 c1 e5 d9 d2 ... 35 f1 ec 4d 93 73 e6 c4 (ICL Intel prod key)
sof-bin/v1.9.x/sof-v1.9/intel-signed/sof-tgl-h.ri
        d3 72 92 99 4e b9 cd 67 ... 5d 62 4a 80 96 31 f8 b5 (TGL Intel prod key)

tomty89 · 2021-11-13T14:17:31Z

@plbossart Yeah the errors are gone with the firmware you linked.

However, I can't actually get proper sound out of my laptop when I have kernel 5.15.2 patched in the way I mentioned in thesofproject/linux#3210 (comment). All I get is a "pop" when the device is opened (when on headphone) and closed. (But well, I suppose I'll won't actually use the internal card anyway unless a driver from the codec vendor is mainlined...)

plbossart · 2021-11-15T18:19:48Z

@tomty89 thanks for testing, much appreciated.

the latest code we have to support Intel platforms with the ES83316 device is here: thesofproject/linux#3107
You would need to pick this branch and install a custom kernel.

If you can give it a try and report your findings in thesofproject/linux#2955 that would be great.

tomty89 · 2021-11-15T22:40:49Z

@plbossart I already did (please see the linked comment above to see what exactly I've applied), and the result is above -- I get an analog alsa device, but no sound actually coming out (other than the pops mentioned).

plbossart · 2021-11-15T23:08:17Z

@plbossart I already did (please see the linked comment above to see what exactly I've applied), and the result is above -- I get an analog alsa device, but no sound actually coming out (other than the pops mentioned).

@hli25 did you get audible sound on the JSL platform you tested, if yes can you share the alsamixer settings used?

hli25 · 2021-11-17T01:03:43Z

@plbossart I already did (please see the linked comment above to see what exactly I've applied), and the result is above -- I get an analog alsa device, but no sound actually coming out (other than the pops mentioned).

@hli25 did you get audible sound on the JSL platform you tested, if yes can you share the alsamixer settings used?

Please try to set "DAC" to 100 via alsamixser @tomty89

tomty89 · 2021-11-17T11:23:14Z

Nope, it doesn't help (it's 100 by default anyway). And toggling/adjusting other mixer controls doesn't seem to help either. Btw I don't know if it's expected but there's no master volume control. There are only the Headphone and Speaker switches and the Headphone Mixer volume control (and a bunch of other whatsoever).

lgirdwood · 2021-11-17T14:08:53Z

@tomty89 can you run amixer contents during playback and paste the output here, it could be we are missing one mixer control setting.

marc-hb · 2021-11-19T01:31:32Z

Please go to thesofproject/linux#2955 or file a different bug for problems happening after loading the firmware.

marc-hb · 2021-11-19T18:03:03Z

v1.9.2 has been merged in sof-bin.git

/sof/tools/sof_ri_info/sof_ri_info.py -v sof-bin/v1.9.x/sof-v1.9.2/intel-signed/sof-jsl.ri | grep -A1 Modulus
./sof-bin/v1.9.x/sof-v1.9.2/intel-signed/sof-jsl.ri
      Modulus size (dwords) 64
        6f e4 d5 c9 52 f4 01 c1 ... f8 7c b0 09 09 c2 a3 cc (Other)

All older intel-signed/sof-jsl.ri files had been signed with the (ICL Intel prod key)

Extracted from https://github.com/thesofproject/sof-bin/blob/main/v1.9.x/sof-v1.9.2/intel-signed/sof-jsl.ri Note previous sof-jsl.ri releases were signed with the ICL key: thesofproject#4916 Signed-off-by: Marc Herbert <marc.herbert@intel.com>

Extracted from https://github.com/thesofproject/sof-bin/blob/main/v1.9.x/sof-v1.9.2/intel-signed/sof-jsl.ri Note previous sof-jsl.ri releases were signed with the ICL key: #4916 Signed-off-by: Marc Herbert <marc.herbert@intel.com>

plbossart · 2021-12-06T15:05:34Z

Chuwi Hi10 Go, jsl Celeron N4500, KDE Neon unstable with signed Debian 5.16rc3 Kernel

try thesofproject/linux#3107 and file a separate issue if this doesn't work for you.

yangxiaohua2009 added the bug Something isn't working as expected label Oct 26, 2021

mengdonglin added the Signing Applies to code signing of firmware label Oct 26, 2021

keqiaozhang added ADL Applies to Alder Lake platform chrome Chromebooks or ChromeOS JSL Applies to Jasper Lake platform P1 Blocker bugs or important features labels Oct 26, 2021

keqiaozhang removed the ADL Applies to Alder Lake platform label Oct 27, 2021

plbossart mentioned this issue Nov 5, 2021

[Alldocube GT Book] cl_copy_fw timeout thesofproject/linux#3210

Closed

This comment has been minimized.

Sign in to view

marc-hb mentioned this issue Nov 19, 2021

Stable v1.9.2 thesofproject/sof-bin#65

Merged

marc-hb mentioned this issue Nov 19, 2021

sof_ri_info: add JSL Intel prod key to list of known keys #5015

Merged

This comment has been minimized.

Sign in to view

marc-hb closed this as completed Dec 6, 2021

plbossart mentioned this issue Dec 10, 2021

[HACK][DO NOT MERGE] es8336 support thesofproject/linux#3107

Closed

Henrythek mentioned this issue Dec 14, 2023

[BUG][JSL]Failed to load FW on JSL #8625

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG][JSL] Failed to load firmware on JSL #4916

[BUG][JSL] Failed to load firmware on JSL #4916

yangxiaohua2009 commented Oct 26, 2021

keqiaozhang commented Oct 26, 2021 •

edited

Loading

mengdonglin commented Oct 26, 2021

plbossart commented Oct 26, 2021 •

edited by marc-hb

Loading

keqiaozhang commented Oct 27, 2021

keqiaozhang commented Oct 27, 2021 •

edited

Loading

keqiaozhang commented Oct 27, 2021

plbossart commented Nov 5, 2021

mwasko commented Nov 8, 2021

plbossart commented Nov 10, 2021

lgirdwood commented Nov 12, 2021

marc-hb commented Nov 12, 2021

tomty89 commented Nov 13, 2021

plbossart commented Nov 15, 2021

tomty89 commented Nov 15, 2021

plbossart commented Nov 15, 2021

hli25 commented Nov 17, 2021

tomty89 commented Nov 17, 2021 •

edited

Loading

lgirdwood commented Nov 17, 2021 •

edited

Loading

This comment has been minimized.

marc-hb commented Nov 19, 2021 •

edited

Loading

marc-hb commented Nov 19, 2021

This comment has been minimized.

plbossart commented Dec 6, 2021

[BUG][JSL] Failed to load firmware on JSL #4916

[BUG][JSL] Failed to load firmware on JSL #4916

Comments

yangxiaohua2009 commented Oct 26, 2021

keqiaozhang commented Oct 26, 2021 • edited Loading

mengdonglin commented Oct 26, 2021

plbossart commented Oct 26, 2021 • edited by marc-hb Loading

keqiaozhang commented Oct 27, 2021

keqiaozhang commented Oct 27, 2021 • edited Loading

keqiaozhang commented Oct 27, 2021

plbossart commented Nov 5, 2021

mwasko commented Nov 8, 2021

plbossart commented Nov 10, 2021

lgirdwood commented Nov 12, 2021

marc-hb commented Nov 12, 2021

tomty89 commented Nov 13, 2021

plbossart commented Nov 15, 2021

tomty89 commented Nov 15, 2021

plbossart commented Nov 15, 2021

hli25 commented Nov 17, 2021

tomty89 commented Nov 17, 2021 • edited Loading

lgirdwood commented Nov 17, 2021 • edited Loading

This comment has been minimized.

marc-hb commented Nov 19, 2021 • edited Loading

marc-hb commented Nov 19, 2021

This comment has been minimized.

plbossart commented Dec 6, 2021

keqiaozhang commented Oct 26, 2021 •

edited

Loading

plbossart commented Oct 26, 2021 •

edited by marc-hb

Loading

keqiaozhang commented Oct 27, 2021 •

edited

Loading

tomty89 commented Nov 17, 2021 •

edited

Loading

lgirdwood commented Nov 17, 2021 •

edited

Loading

marc-hb commented Nov 19, 2021 •

edited

Loading