Skip to content

Test change reflect#64

Open
sgaud-quic wants to merge 1 commit into
qcom-next-stagingfrom
sgaud-quic-patch-33
Open

Test change reflect#64
sgaud-quic wants to merge 1 commit into
qcom-next-stagingfrom
sgaud-quic-patch-33

Conversation

@sgaud-quic
Copy link
Copy Markdown
Collaborator

No description provided.

Signed-off-by: Salendarsingh Gaud <sgaud@qti.qualcomm.com>
@qcomlnxci
Copy link
Copy Markdown
Collaborator

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Tests-monaco-evk

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-qcs615-ride

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-x1e80100-crd

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Test Matrix

Test Case monaco-evk qcs615-ride x1e80100-crd
BT_FW_KMD_Service ❌ Fail ❌ Fail ❌ Fail
BT_ON_OFF ✅ Pass ✅ Pass ✅ Pass
BT_SCAN ✅ Pass ✅ Pass ✅ Pass
CPUFreq_Validation ✅ Pass ✅ Pass ✅ Pass
CPU_affinity ✅ Pass ✅ Pass ✅ Pass
DSP_AudioPD ✅ Pass ✅ Pass ✅ Pass
Ethernet skip skip skip
Freq_Scaling ✅ Pass ✅ Pass ✅ Pass
GIC ✅ Pass ✅ Pass ✅ Pass
IPA ✅ Pass ✅ Pass ✅ Pass
Interrupts ✅ Pass ✅ Pass ✅ Pass
OpenCV skip skip skip
PCIe ✅ Pass ✅ Pass ✅ Pass
Probe_Failure_Check ❌ Fail ❌ Fail ❌ Fail
RMNET ✅ Pass ✅ Pass ✅ Pass
UFS_Validation ✅ Pass ✅ Pass ✅ Pass
USBHost ✅ Pass ✅ Pass ✅ Pass
WiFi_Firmware_Driver ✅ Pass ✅ Pass ✅ Pass
WiFi_OnOff skip skip skip
cdsp_remoteproc ✅ Pass ✅ Pass ✅ Pass
hotplug ✅ Pass ✅ Pass ✅ Pass
irq ✅ Pass ✅ Pass ✅ Pass
kaslr ✅ Pass ✅ Pass ✅ Pass
pinctrl ✅ Pass ✅ Pass ✅ Pass
qcom_hwrng ✅ Pass ✅ Pass ✅ Pass
remoteproc ✅ Pass ✅ Pass ✅ Pass
rngtest ✅ Pass ✅ Pass ✅ Pass
shmbridge ✅ Pass ✅ Pass ✅ Pass
smmu ✅ Pass ✅ Pass ✅ Pass
watchdog ✅ Pass ✅ Pass ✅ Pass
wpss_remoteproc ✅ Pass ✅ Pass ✅ Pass

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Tests-monaco-evk

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-qcs615-ride

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-x1e80100-crd

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Test Matrix

Test Case monaco-evk qcs615-ride x1e80100-crd
BT_FW_KMD_Service ❌ Fail ❌ Fail ❌ Fail
BT_ON_OFF ✅ Pass ✅ Pass ✅ Pass
BT_SCAN ✅ Pass ✅ Pass ✅ Pass
CPUFreq_Validation ✅ Pass ✅ Pass ✅ Pass
CPU_affinity ✅ Pass ✅ Pass ✅ Pass
DSP_AudioPD ✅ Pass ✅ Pass ✅ Pass
Ethernet skip skip skip
Freq_Scaling ✅ Pass ✅ Pass ✅ Pass
GIC ✅ Pass ✅ Pass ✅ Pass
IPA ✅ Pass ✅ Pass ✅ Pass
Interrupts ✅ Pass ✅ Pass ✅ Pass
OpenCV skip skip skip
PCIe ✅ Pass ✅ Pass ✅ Pass
Probe_Failure_Check ❌ Fail ❌ Fail ❌ Fail
RMNET ✅ Pass ✅ Pass ✅ Pass
UFS_Validation ✅ Pass ✅ Pass ✅ Pass
USBHost ✅ Pass ✅ Pass ✅ Pass
WiFi_Firmware_Driver ✅ Pass ✅ Pass ✅ Pass
WiFi_OnOff skip skip skip
cdsp_remoteproc ✅ Pass ✅ Pass ✅ Pass
hotplug ✅ Pass ✅ Pass ✅ Pass
irq ✅ Pass ✅ Pass ✅ Pass
kaslr ✅ Pass ✅ Pass ✅ Pass
pinctrl ✅ Pass ✅ Pass ✅ Pass
qcom_hwrng ✅ Pass ✅ Pass ✅ Pass
remoteproc ✅ Pass ✅ Pass ✅ Pass
rngtest ✅ Pass ✅ Pass ✅ Pass
shmbridge ✅ Pass ✅ Pass ✅ Pass
smmu ✅ Pass ✅ Pass ✅ Pass
watchdog ✅ Pass ✅ Pass ✅ Pass
wpss_remoteproc ✅ Pass ✅ Pass ✅ Pass

@qcomlnxci
Copy link
Copy Markdown
Collaborator

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Tests-monaco-evk

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-qcs615-ride

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-qcs6490-rb3gen2

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

Tests-x1e80100-crd

  • Total: 31 (✅ 26, ❌ 2, ⛔ 0, ⚠️ 3)
    • Failures:
      • Probe_Failure_Check
      • BT_FW_KMD_Service

@qcomlnxci
Copy link
Copy Markdown
Collaborator

Test Matrix

Test Case monaco-evk qcs615-ride qcs6490-rb3gen2 x1e80100-crd
BT_FW_KMD_Service ❌ Fail ❌ Fail ❌ Fail ❌ Fail
BT_ON_OFF ✅ Pass ✅ Pass ✅ Pass ✅ Pass
BT_SCAN ✅ Pass ✅ Pass ✅ Pass ✅ Pass
CPUFreq_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass
CPU_affinity ✅ Pass ✅ Pass ✅ Pass ✅ Pass
DSP_AudioPD ✅ Pass ✅ Pass ✅ Pass ✅ Pass
Ethernet skip skip skip skip
Freq_Scaling ✅ Pass ✅ Pass ✅ Pass ✅ Pass
GIC ✅ Pass ✅ Pass ✅ Pass ✅ Pass
IPA ✅ Pass ✅ Pass ✅ Pass ✅ Pass
Interrupts ✅ Pass ✅ Pass ✅ Pass ✅ Pass
OpenCV skip skip skip skip
PCIe ✅ Pass ✅ Pass ✅ Pass ✅ Pass
Probe_Failure_Check ❌ Fail ❌ Fail ❌ Fail ❌ Fail
RMNET ✅ Pass ✅ Pass ✅ Pass ✅ Pass
UFS_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass
USBHost ✅ Pass ✅ Pass ✅ Pass ✅ Pass
WiFi_Firmware_Driver ✅ Pass ✅ Pass ✅ Pass ✅ Pass
WiFi_OnOff skip skip skip skip
cdsp_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass
hotplug ✅ Pass ✅ Pass ✅ Pass ✅ Pass
irq ✅ Pass ✅ Pass ✅ Pass ✅ Pass
kaslr ✅ Pass ✅ Pass ✅ Pass ✅ Pass
pinctrl ✅ Pass ✅ Pass ✅ Pass ✅ Pass
qcom_hwrng ✅ Pass ✅ Pass ✅ Pass ✅ Pass
remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass
rngtest ✅ Pass ✅ Pass ✅ Pass ✅ Pass
shmbridge ✅ Pass ✅ Pass ✅ Pass ✅ Pass
smmu ✅ Pass ✅ Pass ✅ Pass ✅ Pass
watchdog ✅ Pass ✅ Pass ✅ Pass ✅ Pass
wpss_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass

@qcomlnxci
Copy link
Copy Markdown
Collaborator

@quic-khuzuri
Copy link
Copy Markdown

PR #64 — validate-patch

PR: #64

Verdict Issues Detailed Report
3 pr_only_validate_patch_detailed.md
Verdict: ❌ — click to expand

🔍 Patch Validation

PR: Update Makefile (#64) — #64
Upstream commit: N/A — no lore.kernel.org link found
Verdict: ❌ FAIL

Commit Message

Check Status Note
Subject matches upstream Generic subject "Update Makefile" with no vendor prefix (UPSTREAM:, FROMLIST:, BACKPORT:, QCLINUX:, etc.)
Body preserves rationale Commit body is empty — no description of why PATCHLEVEL/SUBLEVEL are being bumped
Fixes tag present/correct No Fixes: tag present
Authorship preserved ⚠️ Author is Salendarsingh Gaud <sgaud@qti.qualcomm.com>; cannot verify against upstream because no lore link exists
Backport note (if applicable) No Link: tag, no [ upstream commit <sha> ] note, no upstream reference of any kind

Diff

File Status Notes
Makefile Bumps PATCHLEVEL 0→1 and SUBLEVEL 0→1 while EXTRAVERSION = -rc6 remains; cannot verify against any upstream source without a lore link

Issues

  • No lore.kernel.org link — The commit message contains no Link: tag and no lore.kernel.org URL. The validate-patch skill requires a lore link to perform upstream comparison; without it the patch cannot be validated.
  • No vendor prefix — The subject line "Update Makefile" carries none of the required prefixes (UPSTREAM:, FROMLIST:, BACKPORT:, FROMGIT:, QCLINUX:, PENDING:, WORKAROUND:). Every commit in this tree must declare its upstream relationship via one of these prefixes.
  • Empty commit body — There is no rationale explaining why the kernel version numbers are being changed. A Makefile version bump is a significant change that must reference the upstream release commit or explain the intent.
  • Inconsistent version statePATCHLEVEL=1, SUBLEVEL=1, and EXTRAVERSION=-rc6 together form an unusual combination (7.1.1-rc6). Upstream -rc releases always have SUBLEVEL=0; a non-zero SUBLEVEL alongside an -rc suffix is not a valid upstream kernel version string and suggests the diff may be incorrect or applied to the wrong base.

Verdict

Reject: the commit lacks a lore.kernel.org link, a vendor prefix, and any commit body rationale, making upstream faithfulness impossible to verify; additionally the resulting version string 7.1.1-rc6 is inconsistent with standard kernel versioning conventions and must be corrected before merge.

@quic-khuzuri
Copy link
Copy Markdown

PR #64 — checker-log-analyzer

PR: #64
Checker run: https://github.com/qualcomm-linux-stg/kernel-config-test/actions/runs/25633953857

Checker Result Summary
Checker Result Summary
checkpatch WARNING: Missing commit description in commit 53cd93473038
dt-binding-check ⏭️ No changes in Documentation/devicetree/bindings — skipped
dtb-check ⏭️ No changes in Devicetree — skipped
sparse-check ⏭️ No C/H files changed — skipped
check-uapi-headers ⏭️ No relevant files changed — skipped
check-patch-compliance Commit subject "Update Makefile" does not start with a required prefix
tag-check Subject "Update Makefile" missing required prefix (FROMLIST:, UPSTREAM:, etc.) — mandatory for qcom-next-staging
qcom-next-check ⏭️ No FROMLIST:/UPSTREAM: commits — not applicable

Detailed report: pr_only_checker_log_detailed.md

Checker analysis — click to expand

🤖 CI Checker Analysis (checker-log-analyzer)

PR: #64 — "Update Makefile" (53cd93473038)
Target branch: qcom-next-staging
Source: https://github.com/qualcomm-linux-stg/kernel-config-test/actions/runs/25633953857

Checker Result Summary
checkpatch WARNING: Missing commit description in commit 53cd93473038
dt-binding-check ⏭️ No changes in Documentation/devicetree/bindings — skipped
dtb-check ⏭️ No changes in Devicetree — skipped
sparse-check ⏭️ No C/H files changed — skipped
check-uapi-headers ⏭️ No relevant files changed — skipped
check-patch-compliance Commit subject "Update Makefile" does not start with a required prefix
tag-check Subject "Update Makefile" missing required prefix (FROMLIST:, UPSTREAM:, etc.) — mandatory for qcom-next-staging
qcom-next-check ⏭️ No FROMLIST:/UPSTREAM: commits — not applicable

❌ checkpatch

Root cause: The commit body is empty — it has only a Signed-off-by: line and no descriptive text, triggering WARNING: Missing commit description.

Failure details:

WARNING: Missing commit description - Add an appropriate one

53cd93473038dbc63b514a9876b53bbbed022499 total: 0 errors, 1 warnings, 0 checks, 9 lines checked

Commit 53cd93473038 ("Update Makefile") has style problems, please review.

Fix: Amend the commit to add a meaningful description body explaining why the version numbers are being bumped (e.g. what kernel version this targets, what release this corresponds to):

git rebase -i e98c034d3d39   # mark 53cd93473038 as 'edit'
git commit --amend            # add body text below the subject line
git rebase --continue

Reproduce locally:

./scripts/checkpatch.pl --strict --summary-file --ignore FILE_PATH_CHANGES --git e98c034d3d39fcbf7632f48b1795c3caecf5121d..53cd93473038dbc63b514a9876b53bbbed022499

❌ check-patch-compliance

Root cause: The commit subject "Update Makefile" does not start with any of the required prefixes (FROMLIST:, FROMGIT:, UPSTREAM:, BACKPORT:, QCLINUX:, PENDING:, WORKAROUND:).

Failure details:

Checking commit: Update Makefile
Commit summary does not start with a required prefix
Process completed with exit code 1.

Fix: Determine the correct prefix for this change and amend the commit subject:

Scenario Correct prefix
Version bump is a vendor-only change with no upstream equivalent QCLINUX:
Bumping to match an upstream kernel release UPSTREAM: + add Link: <lore/tag URL>
Tracking a mainline-staging version PENDING:

Most likely this is a vendor-only Makefile version bump → use QCLINUX::

git rebase -i e98c034d3d39   # mark commit as 'edit'
git commit --amend -m "QCLINUX: Update Makefile

Bump PATCHLEVEL and SUBLEVEL to 1.1 to reflect the current
vendor kernel version."
git rebase --continue

Reproduce locally:

bash kernel-checkers/check-patch-compliance.sh \
  --kernel-src <path/to/kernel> \
  --base e98c034d3d39fcbf7632f48b1795c3caecf5121d \
  --head 53cd93473038dbc63b514a9876b53bbbed022499

❌ tag-check

Root cause: The PR targets qcom-next-staging (not qcom-next), so every commit subject must start with a recognised prefix. The single commit 53cd93473038 ("Update Makefile") has no prefix at all.

Fix: Same amend as above — prepend the appropriate prefix (QCLINUX:, UPSTREAM:, etc.) to the subject line. This will simultaneously fix both check-patch-compliance and tag-check in a single amend.


Verdict

2 blockers to fix before merge:

  1. Add a commit description body (fixes checkpatch WARNING: Missing commit description).
  2. Add a required subject prefix (fixes both check-patch-compliance and tag-check).

Both issues are in the single commit 53cd93473038 ("Update Makefile") and can be resolved with one git commit --amend. All other checkers (dt-binding-check, dtb-check, sparse-check, check-uapi-headers) correctly skipped because the PR only modifies Makefile.

@quic-khuzuri
Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #64

Job 95057 | SoC unknown_soc_job95057

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95057

No failed cases detected from the LAVA results section.

Job 95058 | SoC unknown_soc_job95058

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95058

No failed cases detected from the LAVA results section.

Job 95059 | SoC unknown_soc_job95059

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95059

No failed cases detected from the LAVA results section.

Job 95060 | SoC unknown_soc_job95060

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95060

No failed cases detected from the LAVA results section.

@rahujosh
Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #64

Job 94607 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94607

Failed test cases in LAVA job 94607 (SoC: x1e80100).

  Case 1: login-action
  1. Failed case: login-action
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94607_1_detailed.md
  Case 2: auto-login-action
  1. Failed case: auto-login-action
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94607_2_detailed.md
  Case 3: fastboot-boot
  1. Failed case: fastboot-boot
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94607_3_detailed.md
  Case 4: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94607_4_detailed.md
Job 94606 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94606

Failed test cases in LAVA job 94606 (SoC: qcs615-ride).

  Case 1: deploy-flasher
  1. Failed case: deploy-flasher
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94606_1_detailed.md
  Case 2: deploy-flasher-retry
  1. Failed case: deploy-flasher-retry
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94606_2_detailed.md
  Case 3: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94606_3_detailed.md
Job 94608 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94608

Failed test cases in LAVA job 94608 (SoC: qcs8300-ride).

  Case 1: cdsp_remoteproc
  1. Failed case: cdsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_1_detailed.md
  Case 2: adsp_remoteproc
  1. Failed case: adsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_2_detailed.md
  Case 3: gpdsp_remoteproc
  1. Failed case: gpdsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_3_detailed.md
  Case 4: remoteproc
  1. Failed case: remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_4_detailed.md
  Case 5: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_5_detailed.md
  Case 6: BT_FW_KMD_Service
  1. Failed case: BT_FW_KMD_Service
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_6_detailed.md
  Case 7: 0_qcom-next-ci-premerge-tests
  1. Failed case: 0_qcom-next-ci-premerge-tests
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_7_detailed.md
  Case 8: lava-test-shell
  1. Failed case: lava-test-shell
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94608_8_detailed.md
Job 95058 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95058

Failed test cases in LAVA job 95058 (SoC: qcs8300-ride).

  Case 1: cdsp_remoteproc
  1. Failed case: cdsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_1_detailed.md
  Case 2: adsp_remoteproc
  1. Failed case: adsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_2_detailed.md
  Case 3: gpdsp_remoteproc
  1. Failed case: gpdsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_3_detailed.md
  Case 4: remoteproc
  1. Failed case: remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_4_detailed.md
  Case 5: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_5_detailed.md
  Case 6: BT_FW_KMD_Service
  1. Failed case: BT_FW_KMD_Service
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_6_detailed.md
  Case 7: 0_qcom-next-ci-premerge-tests
  1. Failed case: 0_qcom-next-ci-premerge-tests
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_7_detailed.md
  Case 8: lava-test-shell
  1. Failed case: lava-test-shell
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95058_8_detailed.md
Job 95059 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95059

Failed test cases in LAVA job 95059 (SoC: qcs615-ride).

  Case 1: deploy-flasher
  1. Failed case: deploy-flasher
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95059_1_detailed.md
  Case 2: deploy-flasher-retry
  1. Failed case: deploy-flasher-retry
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95059_2_detailed.md
  Case 3: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95059_3_detailed.md
Job 95057 | SoC qcs6490-rb3gen2

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95057

Failed test cases in LAVA job 95057 (SoC: qcs6490-rb3gen2).

  Case 1: ** Build Load Failure — Loop device unavailable for rootfs.img during flash
  1. Failed case: ** Build Load Failure — Loop device unavailable for rootfs.img during flash
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh failed at the rootfs mount stage because the worker container could not set up a loop device for rootfs.img; exact error: mount: tmprootfs: failed to setup loop device for rootfs.img. (exit code 1 after 4 s), causing Unable to flash the device on the qcs6490-rb3gen2 board.
  3. Possible fix: Re-trigger the CI job; if it recurs, verify that /dev/loop* devices (or --device=/dev/loop-control) are passed through to the LAVA worker container and that the worker host has available loop device slots (losetup -f). If the worker runs flash-universal.sh inside a nested Docker container, ensure --privileged or explicit --device /dev/loop0 flags are present in the container launch arguments.
  4. Detail analysis attachment: failed_case_job95057_1_detailed.md
  Case 2: deploy-flasher-retry
  1. Failed case: deploy-flasher-retry
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95057_2_detailed.md
  Case 3: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95057_3_detailed.md
Job 95060 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95060

Failed test cases in LAVA job 95060 (SoC: x1e80100).

  Case 1: cdsp_remoteproc
  1. Failed case: cdsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_1_detailed.md
  Case 2: adsp_remoteproc
  1. Failed case: adsp_remoteproc
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_2_detailed.md
  Case 3: 0_qcom-next-ci-premerge-tests
  1. Failed case: 0_qcom-next-ci-premerge-tests
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_3_detailed.md
  Case 4: lava-test-shell
  1. Failed case: lava-test-shell
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_4_detailed.md
  Case 5: lava-test-retry
  1. Failed case: lava-test-retry
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_5_detailed.md
  Case 6: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job95060_6_detailed.md
Job 94604 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94604

Failed test cases in LAVA job 94604 (SoC: qcs8300-ride).

  Case 1: wait-device-boardid
  1. Failed case: wait-device-boardid
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94604_1_detailed.md
  Case 2: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** The qcs8300-ride board (board_id=1315e8d7) never enumerated as a USB fastboot device on the LAVA worker after power-on across all 3 attempts; wait-device-boardid timed out at 200 s / 200 s / 105 s each time with error "wait-device-boardid timed out after 200 seconds", classified by LAVA as error_type: Infrastructure.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, inspect the USB passthrough path from the qcs8300-ride board to the worker container (worker0) — verify the USB cable/hub connecting the board's fastboot USB port is intact, confirm the udev rule that maps board_id=1315e8d7 is present and active inside the container, and check whether the board is actually entering fastboot mode (serial console shows UEFI loading Fastboot.efi correctly, so the issue is likely a USB enumeration/passthrough gap between the board and the container).
  4. Detail analysis attachment: failed_case_job94604_2_detailed.md
  Case 3: ** Build Load Failure — Fastboot USB device not enumerated (board never entered fastboot mode)
  1. Failed case: ** Build Load Failure — Fastboot USB device not enumerated (board never entered fastboot mode)
  2. Root cause: ** Result: Build Load Failure. After PDU reboot, the qcs8300-ride board (1315e8d7) boots through SBL/Hypervisor/UEFI and loads Fastboot.efi, but the USB fastboot device with board ID 1315e8d7 never appears on the LAVA worker's udev — wait-device-boardid timed out after 105 seconds — across all 3 retry attempts; a UEFI DT overlay failure (Bad main_symbols in ufdt_overlay_do_fixups / HypDtFixupEventNotifyFunc: Dtb apply overlay skipped) precedes the USB enumeration failure and likely leaves the Fastboot EFI USB gadget in an incomplete state.
  3. Possible fix: Check and reseat/replace the USB cable between the qcs8300-ride board 1315e8d7 and the LAVA worker (confirm fastboot devices lists the board manually); if the USB path is intact, reflash the board's DTBO partition to resolve the UFDT overlay failure, then re-trigger the job.
  4. Detail analysis attachment: failed_case_job94604_3_detailed.md
Job 94605 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94605

Failed test cases in LAVA job 94605 (SoC: qcs615-ride).

  Case 1: deploy-flasher
  1. Failed case: deploy-flasher
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94605_1_detailed.md
  Case 2: deploy-flasher-retry
  1. Failed case: deploy-flasher-retry
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94605_2_detailed.md
  Case 3: job
  1. Failed case: job
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Collect additional focused logs and rerun the failed test case.
  4. Detail analysis attachment: failed_case_job94605_3_detailed.md

@rahujosh
Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #64

Job 94607 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94607

Failed test cases in LAVA job 94607 (SoC: x1e80100).

  Case 1: ** login-action
  1. Failed case: ** login-action
  2. Root cause: ** After the first kernel boot, mem_sleep_default=s2idle in the kernel cmdline caused the x1e80100-crd board to enter s2idle and reboot instead of resuming; the subsequent cold reboot hit a UEFI firmware ASSERT (ASSERT EnterpriseMgtDxe.c +111: 0) because EnterpriseMgtDxe.efi found its registers already locked from the prior boot, halting the firmware before the kernel could load and leaving LAVA with no login prompt.
  3. Possible fix: Remove mem_sleep_default=s2idle from the LAVA job's kernel command line for this board (or add no_console_suspend to capture the resume failure); additionally, report the EnterpriseMgtDxe ASSERT-on-warm-reboot to the firmware team as a pre-existing UEFI bug on x1e80100-crd that must be fixed to allow reliable reboot/resume cycles.
  4. Detail analysis attachment: failed_case_job94607_1_detailed.md
  Case 2: ** auto-login-action
  1. Failed case: ** auto-login-action
  2. Root cause: ** The x1e80100 CRD board entered s2idle suspend at kernel uptime T+33s (triggered by mem_sleep_default=s2idle in the kernel cmdline), suspending the serial console before LAVA's login-action could complete its shell interaction; on resume the board performed a full cold reboot and the second boot stalled at a UEFI ASSERT EnterpriseMgtDxe.c +111: 0, leaving the serial port silent for all three login-action retry attempts.
  3. Possible fix: Remove mem_sleep_default=s2idle from the LAVA job's kernel cmdline (or replace with mem_sleep_default=freeze) to prevent the board from auto-suspending during the LAVA test window; additionally investigate the UEFI ASSERT EnterpriseMgtDxe.c +111 regression that prevents clean recovery after a suspend-triggered reboot.
  4. Detail analysis attachment: failed_case_job94607_2_detailed.md
  Case 3: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** The X1E80100 CRD board enters s2idle system suspend ~33 seconds after userspace login (triggered by mem_sleep_default=s2idle in the kernel cmdline), suspending the serial console (printk: Suspending console(s)); LAVA's login-action never receives the root@ shell prompt because the console goes silent before LAVA can interact with it, causing all 3 login-action attempts to time out (200 s each).
  3. Possible fix: Remove mem_sleep_default=s2idle from the LAVA job's kernel cmdline (or replace with mem_sleep_default=freeze / mem_sleep_default=shallow) so the board does not auto-suspend during the test window; additionally add no_console_suspend to prevent console blackout if s2idle is intentionally kept.
  4. Detail analysis attachment: failed_case_job94607_3_detailed.md
  Case 4: ** job
  1. Failed case: ** job
  2. Root cause: ** The x1e80100-CRD board entered s2idle suspend (kernel T+33.48s, mem_sleep_default=s2idle in cmdline) before the shell prompt root@[^:]+:~# was ever printed to the serial console, causing the LAVA login-action to time out after 200s and exhaust all 3 fastboot-boot retries within the 10-minute block timeout.
  3. Possible fix: Remove or override mem_sleep_default=s2idle from the kernel cmdline in the LAVA job definition (add mem_sleep_default=freeze or mem_sleep_default=s2idle inhibit_sleep_time=0 or simply remove the parameter) so the board does not auto-suspend before LAVA establishes a shell session; alternatively increase the auto-login-action timeout beyond 10 minutes to survive the suspend/resume cycle.
  4. Detail analysis attachment: failed_case_job94607_4_detailed.md
Job 94606 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94606

Failed test cases in LAVA job 94606 (SoC: qcs615-ride).

  Case 1: ** deploy-flasher — Flash Infrastructure Failure — missing `rootfs.img` in qcomflash artifact
  1. Failed case: ** deploy-flasher — Flash Infrastructure Failure — missing rootfs.img in qcomflash artifact
  2. Root cause: ** flash-universal.sh failed with exit code 32 because rootfs.img (referenced by ROOTFS_IMAGE=rootfs.img in flash.settings) is absent from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact; the tarball contains only firmware partition blobs (xbl.elf, tz.mbn, uefi.elf, vmlinux, dtb.bin, etc.) but no root filesystem image, causing the mount step inside flash-universal.sh to fail with mount: special device rootfs.img does not exist.
  3. Possible fix: Verify that the CI build pipeline for qcs615-ride correctly packages rootfs.img into the .qcomflash.tar.gz artifact; if the image is intentionally absent (firmware-only flash), update flash.settings to remove or conditionally skip the ROOTFS_IMAGE mount step in flash-universal.sh for this device type, then re-trigger the LAVA job.
  4. Detail analysis attachment: failed_case_job94606_1_detailed.md
  Case 2: ** Build Load Failure — Fastboot flash failure
  1. Failed case: ** Build Load Failure — Fastboot flash failure
  2. Root cause: ** Result: Build Load Failure; flash-universal.sh failed at the rootfs mount stage because postprocess.sh (running inside ghcr.io/mwasilew/docker-mkbootimage:master) never extracted rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz — the script exited in ~0 s writing only metadata keys to flash.settings, leaving rootfs.img absent; exact error: mount: special device rootfs.img does not exist.
  3. Possible fix: Inspect and fix postprocess.sh inside the docker-mkbootimage:master image to ensure it extracts rootfs.img from the .qcomflash.tar.gz archive for the qcs615-ride device type; re-trigger the CI job after the corrected image is deployed, and verify rootfs.img is present in /lava-downloads before flash-universal.sh is called.
  4. Detail analysis attachment: failed_case_job94606_2_detailed.md
  Case 3: ** Build Load Failure — Flash script mount failure (missing rootfs.img in artifact)
  1. Failed case: ** Build Load Failure — Flash script mount failure (missing rootfs.img in artifact)
  2. Root cause: ** Result: Build Load Failure — flash stage; flash-universal.sh failed with exit code 32 because rootfs.img is absent from the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact, causing the mount call to fail: "special device rootfs.img does not exist".
  3. Possible fix: Re-trigger the CI job to obtain a freshly built artifact; if the failure recurs, investigate the qcs615-ride rootfs image build pipeline (CI job 25633942419-1) to ensure rootfs.img is correctly packaged into the qcomflash tarball before upload to S3.
  4. Detail analysis attachment: failed_case_job94606_3_detailed.md
Job 94608 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94608

Failed test cases in LAVA job 94608 (SoC: qcs8300-ride).

  Case 1: ** `cdsp_remoteproc` — Remoteproc Firmware Load Failure (`Direct firmware load for qcom/qcs8300/cdsp0.mbn failed with error -2`)
  1. Failed case: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (Direct firmware load for qcom/qcs8300/cdsp0.mbn failed with error -2)
  2. Root cause: ** The rootfs deployed to the qcs8300-ride board in LAVA job 94608 is missing the DSP firmware files under /lib/firmware/qcom/qcs8300/request_firmware() returns -ENOENT (-2) for cdsp0.mbn, leaving remoteproc2 permanently in offline state; the remoteproc driver itself probed correctly (cdsp is available).
  3. Possible fix: Ensure the rootfs artifact used in this LAVA job includes the qcom/qcs8300/*.mbn firmware package (verify the CI rootfs build recipe installs it), then re-trigger the job; add a pre-test guard that checks /lib/firmware/qcom/qcs8300/cdsp0.mbn exists before running remoteproc tests.
  4. Detail analysis attachment: failed_case_job94608_1_detailed.md
  Case 2: ** adsp_remoteproc — Remoteproc Firmware Load Failure (Possibility 1: Missing Firmware)
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure (Possibility 1: Missing Firmware)
  2. Root cause: ** qcom/qcs8300/adsp.mbn (and sibling gpdsp0.mbn, cdsp0.mbn) are absent from the test rootfs firmware search path; request_firmware() returns -ENOENT for all three DSP subsystems, leaving remoteproc0 permanently offline. This is a pre-existing test-environment issue, not introduced by PR Test change reflect #64 (which only bumps Makefile version numbers).
  3. Possible fix: Update the LAVA job definition to use qcs8300-ride.dtb instead of monaco-evk.dtb, and ensure qcom/qcs8300/adsp.mbn, gpdsp0.mbn, and cdsp0.mbn are present under /lib/firmware/ in the test initramfs or via the remotefs firmware partition for this board.
  4. Detail analysis attachment: failed_case_job94608_2_detailed.md
  Case 3: ** `gpdsp_remoteproc` — Remoteproc Firmware Load Failure (missing firmware file)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (missing firmware file)
  2. Root cause: ** The test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain qcom/qcs8300/gpdsp0.mbn under /lib/firmware/, causing request_firmware() to return -ENOENT (-2) at boot; remoteproc1 (gpdsp0) never advances past offline state on qcs8300-ride.
  3. Possible fix: Add the qcs8300 DSP firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) to the initramfs-kerneltest-full-image-qcom-armv8a rootfs by installing the qcom-firmware-qcs8300 package (or equivalent) in the Yocto/OE recipe that builds the test image.
  4. Detail analysis attachment: failed_case_job94608_3_detailed.md
  Case 4: ** remoteproc — Firmware Load Failure (all three DSP subsystems offline)
  1. Failed case: ** remoteproc — Firmware Load Failure (all three DSP subsystems offline)
  2. Root cause: ** All three qcs8300 DSP firmware files (qcom/qcs8300/adsp.mbn, qcom/qcs8300/gpdsp0.mbn, qcom/qcs8300/cdsp0.mbn) are absent from the LAVA firmware overlay; request_firmware() returns -ENOENT (-2) for every subsystem, leaving all three remoteprocs in offline state. This is a pre-existing infrastructure issue — PR Test change reflect #64 only bumps Makefile version numbers and has no impact on firmware loading.
  3. Possible fix: Add the three missing qcs8300 DSP firmware files to the LAVA job's firmware overlay/NFS rootfs at /lib/firmware/qcom/qcs8300/{adsp,gpdsp0,cdsp0}.mbn and verify the Qualcomm remotefs service mounts the firmware partition before the remoteproc driver triggers request_firmware().
  4. Detail analysis attachment: failed_case_job94608_4_detailed.md
  Case 5: ** `Probe_Failure_Check`
  1. Failed case: ** Probe_Failure_Check
  2. Root cause: ** Pre-existing, PR-unrelated probe failures on qcs8300-ride: DSP/VPU firmware files (adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn) are absent from the LAVA test rootfs (error -2/ENOENT); cpufreq-dt conflicts with the already-registered qcom-cpufreq-hw driver (error -17/EEXIST); a second qcom-ice instance at 87c8000 is not present in hardware (error -2); and amc6821/tpm_tis_spi are not physically populated on this board (error -110/ETIMEDOUT). The PR patch only bumps Makefile version numbers and cannot cause any of these.
  3. Possible fix: Add qcs8300 firmware files (adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn) to the LAVA test rootfs image; add the known-benign suppressions for cpufreq-dt -17, qcom-ice 87c8000 -2, amc6821 -110, and tpm_tis_spi -110 to the Probe_Failure_Check test's allowlist for qcs8300-ride, or fix the DT to not describe hardware not present on this board variant.
  4. Detail analysis attachment: failed_case_job94608_5_detailed.md
  Case 6: ** `BT_FW_KMD_Service` — Bluetooth KMD/Firmware Service Failure (WCN6855 UART controller, qcs8300-ride)
  1. Failed case: ** BT_FW_KMD_Service — Bluetooth KMD/Firmware Service Failure (WCN6855 UART controller, qcs8300-ride)
  2. Root cause: ** The WCN6855 Bluetooth controller on UART (serial0-0) does not respond to the QCA version-query HCI vendor command (0xfc00), timing out with -ETIMEDOUT (-110) on all three power-on retries because the three required power supplies (vddio, vddbtcxmx, vddrfa1p7) are absent from the regulator framework (dummy regulators used), leaving hci0 permanently DOWN with a null BD address and no firmware downloaded.
  3. Possible fix: Add the missing WCN6855 regulator supply phandles (vddio-supply, vddbtcxmx-supply, vddrfa1p7-supply) to the Monaco EVK DTS bluetooth node pointing to the correct PMIC rails, and include WCN6855 BT firmware files (msbtfw*/msnv*) in the LAVA test rootfs image; PR Test change reflect #64 (Makefile version bump only) does not need to be blocked on this pre-existing platform issue.
  4. Detail analysis attachment: failed_case_job94608_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests` — Test Shell Timeout — BT_SCAN hung waiting for Bluetooth controller that never became functional due to WCN6855 firmware/USB enumeration failure on qcs8300-ride
  1. Failed case: ** 0_qcom-next-ci-premerge-tests — Test Shell Timeout — BT_SCAN hung waiting for Bluetooth controller that never became functional due to WCN6855 firmware/USB enumeration failure on qcs8300-ride
  2. Root cause: ** The BT_SCAN test case hung indefinitely inside an interactive bluetoothctl loop waiting for hci0 to become visible, because the WCN6855 Bluetooth controller on qcs8300-ride (Monaco EVK) failed to initialize — the USB2 hub port (usb usb2-port1) continuously reported config error / Cannot enable from kernel timestamp ~18s through ~1227s, preventing the BT HCI from enumerating, and the BT_SCAN script had no timeout guard on its bluetoothctl interactive session, consuming the remaining ~4.5 minutes of the 20-minute lava-test-shell budget until LAVA killed it with lava-test-shell timed out after 1200 seconds.
  3. Possible fix: This is a pre-existing hardware/firmware issue on the qcs8300-ride board (WCN6855 USB2 enumeration failure unrelated to the PR patch which only bumps PATCHLEVEL/SUBLEVEL in Makefile); mark BT_SCAN as a known-flaky/skip for this board configuration, or add a timeout guard in the BT_SCAN test script so it exits within a bounded time when hci0 is not available, preventing it from consuming the entire test-shell budget.
  4. Detail analysis attachment: failed_case_job94608_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The lava-test-shell 1200-second timeout was exhausted because the BT_SCAN test suite hung indefinitely inside an interactive bluetoothctl session waiting for a Bluetooth scan result on a board where the Bluetooth stack is non-functional (BD address all-zeros, firmware load failure reported by BT_FW_KMD_Service); the concurrent usb usb2-port1: config error / Cannot enable messages are a background USB enumeration failure on the board's internal hub and are noise, not the cause of the hang.
  3. Possible fix: Re-trigger the CI job to confirm reproducibility; if the hang recurs, add a per-test timeout guard in BT_SCAN/run.sh around the bluetoothctl interactive invocation (e.g. timeout 60 bluetoothctl ...) so a non-functional BT adapter does not block the entire test suite, and investigate the root cause of the Bluetooth firmware load failure on qcs8300-ride (BD address all-zeros indicates the BT firmware/NVM is not loading correctly).
  4. Detail analysis attachment: failed_case_job94608_8_detailed.md
Job 95058 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95058

Failed test cases in LAVA job 95058 (SoC: qcs8300-ride).

  Case 1: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (Missing `qcom/qcs8300/cdsp0.mbn`)
  1. Failed case: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (Missing qcom/qcs8300/cdsp0.mbn)
  2. Root cause: ** The test rootfs (initramfs-kerneltest-full-image-qcom-armv8a) does not contain the QCS8300 DSP firmware blob qcom/qcs8300/cdsp0.mbn; request_firmware() returns -ENOENT (-2), so CDSP (remoteproc2) never boots and stays offline.
  3. Possible fix: Add the QCS8300 DSP firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) to the meta-qcom test rootfs under /lib/firmware/qcom/qcs8300/, or overlay a firmware tarball in the LAVA job before running remoteproc tests.
  4. Detail analysis attachment: failed_case_job95058_1_detailed.md
  Case 2: ** adsp_remoteproc — Remoteproc Firmware Load Failure (missing `qcom/qcs8300/adsp.mbn`)
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure (missing qcom/qcs8300/adsp.mbn)
  2. Root cause: ** The initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz deployed by this LAVA job does not contain the QCS8300 DSP firmware file qcom/qcs8300/adsp.mbn, causing request_firmware() to return -ENOENT (-2) and leaving remoteproc0 (adsp) permanently in offline state.
  3. Possible fix: Add the qcom/qcs8300/ DSP firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) to the meta-qcom initramfs recipe for qcom-armv8a, or deploy them as a LAVA firmware overlay tarball to /lib/firmware/qcom/qcs8300/ before the test shell runs on qcs8300-ride.
  4. Detail analysis attachment: failed_case_job95058_2_detailed.md
  Case 3: ** `gpdsp_remoteproc` — Remoteproc Firmware Load Failure (`qcom/qcs8300/gpdsp0.mbn` not found)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (qcom/qcs8300/gpdsp0.mbn not found)
  2. Root cause: ** The test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain qcom/qcs8300/gpdsp0.mbn; request_firmware() returns -ENOENT (-2) on both load attempts, leaving remoteproc1 permanently offline before the test checks its state.
  3. Possible fix: Add qcom/qcs8300/gpdsp0.mbn (and adsp.mbn, cdsp0.mbn) to the meta-qcom kerneltest-full-image firmware packaging so the blobs are present under /lib/firmware/qcom/qcs8300/ in the LAVA test rootfs.
  4. Detail analysis attachment: failed_case_job95058_3_detailed.md
  Case 4: ** remoteproc
  1. Failed case: ** remoteproc
  2. Root cause: ** All three QCS8300 remoteproc subsystems (ADSP, GPDSP0, CDSP) fail request_firmware with error -2 (ENOENT) at boot because the firmware files qcom/qcs8300/adsp.mbn, qcom/qcs8300/gpdsp0.mbn, and qcom/qcs8300/cdsp0.mbn are absent from the LAVA test initramfs, leaving all three subsystems in offline state; the test expects 3 running but finds 0.
  3. Possible fix: Add the QCS8300 subsystem firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) under /lib/firmware/qcom/qcs8300/ in the initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz used by this LAVA job, or configure the LAVA job to mount/fetch the firmware partition at runtime before the remoteproc test executes.
  4. Detail analysis attachment: failed_case_job95058_4_detailed.md
  Case 5: ** Probe_Failure_Check — Driver/Firmware Probe Failures (Pre-existing, Not PR-Introduced)
  1. Failed case: ** Probe_Failure_Check — Driver/Firmware Probe Failures (Pre-existing, Not PR-Introduced)
  2. Root cause: ** Multiple drivers on qcs8300-ride fail probe at every boot due to missing firmware files (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn, qcom/vpu/vpu30_p4_s6.mbn — all -ENOENT) and hardware-absent peripherals (amc6821, tpm_tis_spi-ETIMEDOUT); identical failures are present in the baseline job 94608 (pre-PR), confirming this is a pre-existing lab/firmware environment issue unrelated to PR Test change reflect #64.
  3. Possible fix: No kernel code change needed — provision the missing QCS8300 subsystem firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn) into the LAVA test rootfs/initramfs under /lib/firmware/qcom/qcs8300/ and /lib/firmware/qcom/vpu/; for -ETIMEDOUT peripherals (amc6821, tpm_tis_spi), update the test allowlist to suppress known-absent hardware on this board variant, or add them to the CI suppression list in lava-known-benign-failures.md.
  4. Detail analysis attachment: failed_case_job95058_5_detailed.md
  Case 6: ** `BT_FW_KMD_Service` — Driver Initialization Failure (WCN6855 BT UART probe hard-fail)
  1. Failed case: ** BT_FW_KMD_Service — Driver Initialization Failure (WCN6855 BT UART probe hard-fail)
  2. Root cause: ** The btqca/hci_uart_qca driver on qcs8300-ride fails to initialize the WCN6855 Bluetooth chip because the chip does not respond to the HCI vendor command 0xfc00 (QCA version query) over UART — all 3 power-ON retries time out with -ETIMEDOUT (-110) — caused by missing power supply regulators (vddio, vddbtcxmx, vddrfa1p7 all fall back to dummy regulators), leaving the chip unpowered/unresponsive; firmware is never downloaded and hci0 stays DOWN with a null BD address.
  3. Possible fix: Add the correct PMIC regulator supply phandles (vddio-supply, vddbtcxmx-supply, vddrfa1p7-supply) and the enable-gpios (BT_EN) property to the qcom,wcn6855-bt DT node in arch/arm64/boot/dts/qcom/qcs8300-ride.dts, matching the qcs8300 hardware schematic; this is a pre-existing board bring-up gap unrelated to PR64 (which only bumps the Makefile version string).
  4. Detail analysis attachment: failed_case_job95058_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests`
  1. Failed case: ** 0_qcom-next-ci-premerge-tests
  2. Root cause: ** The BT_ON_OFF and BT_SCAN Bluetooth test cases in the qcom-next-ci-premerge suite hung indefinitely waiting for the QCA Bluetooth controller (hci0) to become visible after bluetoothctl public-addr, because the QCA BT firmware load over UART (hci_uart_qca serial0-0) failed at boot with repeated command 0xfc00 tx timeout / Reading QCA version information failed (-110) errors (all 3 retries exhausted), leaving hci0 in DOWN state with BD Address 00:00:00:00:00:00; the test script's 15-second re-check loop never resolved, consuming the entire 1200-second lava-test-shell timeout.
  3. Possible fix: Investigate and fix the QCA BT firmware load failure on qcs8300-ride (Monaco EVK) — the hci_uart_qca driver is failing to communicate with the BT SoC over UART at boot (likely missing/mismatched firmware blob, missing regulator supply, or a UART/power sequencing regression); as an immediate mitigation, add a SKIP guard in the BT test scripts when hci0 is DOWN after firmware load retries are exhausted, and add a bounded timeout to the public-addr wait loop to prevent the 20-minute hang.
  4. Detail analysis attachment: failed_case_job95058_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The BT_SCAN test script hung indefinitely inside an interactive bluetoothctl retry loop because the WCN6855 Bluetooth controller (hci0) on the qcs8300-ride board never became operational — the QCA firmware setup failed at boot (hci0: command 0xfc00 tx timeout × 3, Reading QCA version information failed (-110)) — exhausting the full 20-minute lava-test-shell timeout.
  3. Possible fix: Add a hard timeout guard (e.g. 60–90 s) to the BT_SCAN test script's bluetoothctl interactive loop so it exits with SKIP/FAIL instead of hanging; separately, investigate the WCN6855 firmware load failure on qcs8300-ride (missing/mismatched firmware blob or UART regulator supply issue) and the concurrent usb usb2-port1: config error USB enumeration failure on the xHCI bus.
  4. Detail analysis attachment: failed_case_job95058_8_detailed.md
Job 95059 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95059

Failed test cases in LAVA job 95059 (SoC: qcs615-ride).

  Case 1: ** deploy-flasher — Missing `rootfs.img` in flash archive
  1. Failed case: ** deploy-flasher — Missing rootfs.img in flash archive
  2. Root cause: ** flash-universal.sh failed with exit code 32 because rootfs.img is absent from the downloaded flash archive qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz; the archive contains only firmware blobs (.mbn, .elf, .bin, .xml, vmlinux) but no rootfs image, so the mount rootfs.img step inside flash-universal.sh fails with special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job to obtain a freshly built artifact; if the issue recurs, investigate the qcs615-ride image build pipeline to ensure rootfs.img (the ext4/squashfs rootfs partition image) is included in the .qcomflash.tar.gz packaging step — check the IMAGE_INSTALL / do_image_qcomflash Yocto task or equivalent build script that assembles the flash archive for this SoC.
  4. Detail analysis attachment: failed_case_job95059_1_detailed.md
  Case 2: ** deploy-flasher-retry — Flash Image Mount Failure (`rootfs.img` not produced by postprocess)
  1. Failed case: ** deploy-flasher-retry — Flash Image Mount Failure (rootfs.img not produced by postprocess)
  2. Root cause: ** Result: Build Load Failure — flash stage; flash-universal.sh aborted with exit code 32 because rootfs.img was never created: the docker-mkbootimage postprocess script ran for ~0.3 s, wrote only flash.settings entries, and did not extract or build rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz, leaving mount: special device rootfs.img does not exist when flash-universal.sh attempted to mount it.
  3. Possible fix: Inspect and fix the postprocess.sh script inside the ghcr.io/mwasilew/docker-mkbootimage:master image to ensure it extracts the .qcomflash.tar.gz and produces rootfs.img before flash-universal.sh is invoked; re-trigger the CI job after the corrected image is deployed, and verify the postprocess step duration increases to reflect actual extraction work (expected: several seconds to tens of seconds for a 381 MB tarball).
  4. Detail analysis attachment: failed_case_job95059_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — Missing rootfs.img in flash artifact
  1. Failed case: ** Infrastructure Flash Failure — Missing rootfs.img in flash artifact
  2. Root cause: ** The qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact (build 25650106467-1) does not contain a rootfs.img file; flash-universal.sh attempted to mount it as a loop device to inject the LAVA overlay and failed with mount: special device rootfs.img does not exist, causing deploy-flasher to abort with Unable to flash the device.
  3. Possible fix: Re-trigger the CI job to pick up a fresh build artifact; if the failure recurs, inspect the qcs615-ride image build pipeline for build 25650106467-1 to confirm rootfs.img is being packaged into the .qcomflash.tar.gz bundle — the postprocess.sh script sets ROOTFS_IMAGE=rootfs.img but the file is absent, indicating a packaging regression in the artifact build, not in the LAVA infrastructure or the PR under test.
  4. Detail analysis attachment: failed_case_job95059_3_detailed.md
Job 95057 | SoC qcs6490-rb3gen2

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95057

Failed test cases in LAVA job 95057 (SoC: qcs6490-rb3gen2).

  Case 1: ** deploy-flasher — Flash Stage Failure — loop device unavailable for rootfs.img mount
  1. Failed case: ** deploy-flasher — Flash Stage Failure — loop device unavailable for rootfs.img mount
  2. Root cause: ** Result: Build Load Failure (flash stage); flash-universal.sh failed to mount rootfs.img via a loop device inside the LAVA worker container — mount: tmprootfs: failed to setup loop device for rootfs.img. — because /dev/loop* devices are not accessible in the worker's Docker environment, causing the script to exit 1 and LAVA to raise InfrastructureError: Unable to flash the device.
  3. Possible fix: Re-trigger the CI job to rule out a transient loop-device exhaustion; if the failure recurs, ensure the LAVA worker container is launched with loop device pass-through (e.g. --device=/dev/loop-control --device=/dev/loop0 … or --privileged) so that flash-universal.sh can mount rootfs.img during the overlay-injection step on qcs6490-rb3gen2.
  4. Detail analysis attachment: failed_case_job95057_1_detailed.md
  Case 2: ** Build Load Failure — Corrupt/Incomplete Image
  1. Failed case: ** Build Load Failure — Corrupt/Incomplete Image
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh stage; flash.settings specifies ROOTFS_IMAGE=rootfs.img but rootfs.img is absent from the extracted qcom-multimedia-image-rb3gen2-core-kit.rootfs.qcomflash.tar.gz archive, causing mount: tmprootfs: failed to setup loop device for rootfs.img.
  3. Possible fix: Re-trigger the CI job to regenerate the flash artifact; if the issue recurs, inspect the qcs6490-rb3gen2 image build pipeline (build ID 25650106467-1) to confirm rootfs.img is being produced and packaged into the .qcomflash.tar.gz before LAVA picks it up.
  4. Detail analysis attachment: failed_case_job95057_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — Loop Device Setup Error
  1. Failed case: ** Infrastructure Flash Failure — Loop Device Setup Error
  2. Root cause: ** flash-universal.sh failed during the deploy-flasher stage on the qcs6490-rb3gen2 board because the LAVA worker container could not set up a loop device to mount rootfs.img, producing the exact error: mount: tmprootfs: failed to setup loop device for rootfs.img.
  3. Possible fix: Ensure the LAVA worker container has access to loop devices — either by passing --device=/dev/loop-control --device=/dev/loop0 (and additional loop nodes) in the container launch config, or by running the flash container with sufficient privileges (CAP_SYS_ADMIN); then re-trigger the CI job to confirm the flash completes.
  4. Detail analysis attachment: failed_case_job95057_3_detailed.md
Job 95060 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95060

Failed test cases in LAVA job 95060 (SoC: x1e80100).

  Case 1: ** cdsp_remoteproc
  1. Failed case: ** cdsp_remoteproc
  2. Root cause: ** The CDSP firmware blob qcom/x1e80100/cdsp.mbn is absent from the test rootfs/initramfs — request_firmware returns ENOENT (-2) at kernel boot time (~26 s), leaving remoteproc1 permanently in offline state before the test script even runs.
  3. Possible fix: Add the x1e80100 DSP firmware package (containing cdsp.mbn, adsp.mbn, and companion blobs) to the LAVA job's initramfs overlay or deploy it as a separate firmware ramdisk artifact alongside modules.tar; alternatively, ensure the base initramfs-kerneltest-full-image-qcom-armv8a image includes /lib/firmware/qcom/x1e80100/ blobs for this SoC.
  4. Detail analysis attachment: failed_case_job95060_1_detailed.md
  Case 2: ** adsp_remoteproc
  1. Failed case: ** adsp_remoteproc
  2. Root cause: ** Missing DSP firmware blobs in the test rootfs — qcom/x1e80100/adsp.mbn is absent from /lib/firmware/ in the initramfs, causing request_firmware() to return -ENOENT at boot (~26s), leaving remoteproc0 permanently state=offline before the test script runs.
  3. Possible fix: Add qcom/x1e80100/adsp.mbn (and cdsp.mbn) firmware blobs to the initramfs-kerneltest-full-image-qcom-armv8a rootfs image used by this LAVA job; verify the files land under /lib/firmware/qcom/x1e80100/ in the deployed image.
  4. Detail analysis attachment: failed_case_job95060_2_detailed.md
  Case 3: ** Kernel Test Hang — s2idle Suspend Non-Return (CPUFreq_Validation)
  1. Failed case: ** Kernel Test Hang — s2idle Suspend Non-Return (CPUFreq_Validation)
  2. Root cause: ** The CPUFreq_Validation test on x1e80100-crd triggered an s2idle suspend cycle (kernel cmdline mem_sleep_default=s2idle) at kernel uptime ~33s; the kernel entered suspend (PM: suspend entry (s2idle)printk: Suspending console(s)) and never resumed, causing the LAVA test-shell to exhaust its 20-minute timeout with no further output. The adsp_remoteproc FAIL is a pre-existing, unrelated issue (missing qcom/x1e80100/adsp.mbn firmware in the initramfs).
  3. Possible fix: Investigate and fix the s2idle resume path on x1e80100 for the CPUFreq_Validation test; as an immediate mitigation, add mem_sleep_default=freeze (or remove mem_sleep_default=s2idle) to the kernel cmdline in the LAVA job definition to prevent the CPUFreq test from entering a non-returning s2idle cycle, and re-trigger the CI job to confirm the hang is resolved.
  4. Detail analysis attachment: failed_case_job95060_3_detailed.md
  Case 4: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The CPUFreq_Validation test script triggered an s2idle system suspend on the x1e80100-CRD board (kernel cmdline sets mem_sleep_default=s2idle); the board entered suspend at kernel timestamp T+33s (PM: suspend entry (s2idle)printk: Suspending console(s)) and never resumed, silencing the serial console and causing the LAVA test shell to time out after 1200 seconds with no further output.
  3. Possible fix: Add a wakeup alarm (e.g., rtcwake -m no -s 60 before the CPUFreq test) or disable s2idle for the CI job by removing mem_sleep_default=s2idle from the kernel cmdline in the LAVA job definition, so the CPUFreq frequency-sweep test cannot inadvertently trigger a system suspend.
  4. Detail analysis attachment: failed_case_job95060_4_detailed.md
  Case 5: ** lava-test-retry
  1. Failed case: ** lava-test-retry
  2. Root cause: ** The CPUFreq_Validation test script triggered an s2idle system suspend on the x1e80100 CRD board at kernel uptime ~33 s; the board entered suspend (console silenced at [33.544] printk: Suspending console(s)) and never resumed, leaving the LAVA test-shell waiting until it timed out after 1200 seconds with lava-test-shell timed out after 1200 seconds.
  3. Possible fix: Add no_console_suspend to the kernel cmdline in the LAVA job definition to keep the serial console alive through suspend, and investigate the s2idle wakeup path on x1e80100 CRD (missing or misconfigured wakeup source / EC wakeup IRQ) to prevent the board from hanging in suspend during CPUFreq frequency-sweep tests.
  4. Detail analysis attachment: failed_case_job95060_5_detailed.md
  Case 6: ** job — Test Shell Timeout (s2idle suspend hang during CPUFreq_Validation)
  1. Failed case: ** job — Test Shell Timeout (s2idle suspend hang during CPUFreq_Validation)
  2. Root cause: ** The x1e80100-crd DUT entered s2idle suspend at kernel uptime ~33.5 s (triggered by the CPUFreq_Validation test script while iterating CPU frequencies on policy0), and never resumed — the console was suspended (printk: Suspending console(s)) and no wakeup event occurred for the full 1200-second LAVA test-shell timeout window, causing lava-test-shell timed out after 1200 seconds.
  3. Possible fix: Add no_console_suspend to the kernel cmdline in the LAVA job definition to keep the serial console alive through suspend/resume cycles, and investigate why the x1e80100-crd board does not auto-resume from s2idle (missing or broken wakeup source / RTC wakeup configuration for this board in CI); as a short-term mitigation, add mem_sleep_default=freeze (or remove mem_sleep_default=s2idle) to the boot cmdline to prevent the system from entering s2idle during test execution.
  4. Detail analysis attachment: failed_case_job95060_6_detailed.md
Job 94604 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94604

Failed test cases in LAVA job 94604 (SoC: qcs8300-ride).

  Case 1: ** wait-device-boardid
  1. Failed case: ** wait-device-boardid
  2. Root cause: ** After each alpaca_control.py power-on cycle on the qcs8300-ride board, LAVA waited up to 200 s for the board to enumerate a USB fastboot device with board ID 1315e8d7, but the board never appeared on the USB bus in fastboot mode — all three attempts timed out, indicating the board is not entering fastboot mode after power-on (likely booting directly to the OS or hanging before USB enumeration).
  3. Possible fix: Check the qcs8300-ride board's boot mode configuration in the LAVA device dictionary — verify that the fastboot-boot action correctly sets the board to fastboot mode before power-on (e.g., via a set-boot-mode fastboot step using debugboard.py or equivalent); if the board is booting to the OS instead of fastboot, add or fix the pre-boot-mode-set step. Re-trigger the job after confirming the board enumerates in fastboot mode manually.
  4. Detail analysis attachment: failed_case_job94604_1_detailed.md
  Case 2: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** Infrastructure failure — after PDU power-on, the qcs8300-ride board boots through UEFI and loads Fastboot.efi but never enumerates as a USB fastboot device with board ID 1315e8d7 on the LAVA worker, causing wait-device-boardid timed out after 200 seconds on all 3 retry attempts.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, check the USB cable/hub path between the qcs8300-ride board and the LAVA worker container (the board's USB controller reports Cable Attached and Dev Bus Speed: Super in UEFI, so the issue is likely a transient USB enumeration failure or a worker-side udev/USB passthrough problem — verify udevadm monitor inside worker0 during a manual power cycle to confirm the 1315e8d7 device event is received).
  4. Detail analysis attachment: failed_case_job94604_2_detailed.md
  Case 3: ** Build Load Failure — Board fastboot USB enumeration timeout (Infrastructure)
  1. Failed case: ** Build Load Failure — Board fastboot USB enumeration timeout (Infrastructure)
  2. Root cause: ** Result: Build Load Failure. The qcs8300-ride board (board_id=1315e8d7) never enumerated as a fastboot USB device on the LAVA worker after PDU power-cycle across all 3 attempts — UEFI completed successfully and launched Fastboot.efi with USB3 PHY initialized and cable-attached detected, but wait-device-boardid timed out after 105 seconds (final attempt) with error_type: Infrastructure.
  3. Possible fix: Report this to LAVA admins as an infrastructure issue for the qcs8300-ride board (1315e8d7); check the USB cable/hub connection between the board and the LAVA worker host, verify the board's USB device-mode port is physically connected to the worker, and re-trigger the job once the board's USB connectivity is confirmed healthy.
  4. Detail analysis attachment: failed_case_job94604_3_detailed.md
Job 94605 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94605

Failed test cases in LAVA job 94605 (SoC: qcs615-ride).

  Case 1: ** Build Load Failure — Flash image preparation failure (`rootfs.img` not produced by `postprocess.sh`)
  1. Failed case: ** Build Load Failure — Flash image preparation failure (rootfs.img not produced by postprocess.sh)
  2. Root cause: ** Result: Build Load Failure — postprocess.sh (run inside ghcr.io/mwasilew/docker-mkbootimage:master) completed in ~0.55 s without extracting the .qcomflash.tar.gz or building rootfs.img; when flash-universal.sh subsequently attempted mount rootfs.img, it failed with "special device rootfs.img does not exist" (exit code 32).
  3. Possible fix: Re-trigger the CI job; if the failure recurs, inspect postprocess.sh inside the docker-mkbootimage:master image to confirm it contains the extraction/image-build logic for qcs615-ride artifacts, and rebuild/update the ghcr.io/mwasilew/docker-mkbootimage:master image if the script is missing or broken for this device type.
  4. Detail analysis attachment: failed_case_job94605_1_detailed.md
  Case 2: ** deploy-flasher-retry — Flash Infrastructure Failure — missing rootfs.img in qcomflash bundle
  1. Failed case: ** deploy-flasher-retry — Flash Infrastructure Failure — missing rootfs.img in qcomflash bundle
  2. Root cause: ** flash-universal.sh extracted the qcomflash tarball (qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz) and attempted to mount rootfs.img as declared in flash.settings (ROOTFS_IMAGE=rootfs.img), but the file does not exist inside the archive; the mount syscall returned exit code 32 (special device rootfs.img does not exist), aborting the flash with Unable to flash the device.
  3. Possible fix: Verify that the qcs615-ride qcomflash build artifact actually packages a file named rootfs.img; if the rootfs image is built under a different name (e.g. rootfs.ext4), update flash.settings (or the postprocess script that generates it) to set ROOTFS_IMAGE to the correct filename, then re-trigger the CI job.
  4. Detail analysis attachment: failed_case_job94605_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — missing `rootfs.img` in `.qcomflash` artifact
  1. Failed case: ** Infrastructure Flash Failure — missing rootfs.img in .qcomflash artifact
  2. Root cause: ** flash-universal.sh failed during deploy-flasher because rootfs.img (referenced by ROOTFS_IMAGE=rootfs.img in flash.settings) does not exist inside the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact; the mount attempt aborted with mount: special device rootfs.img does not exist (exit code 32), triggering InfrastructureError: Unable to flash the device.
  3. Possible fix: Re-trigger the CI job to obtain a freshly packaged artifact; if the failure recurs, inspect the CI build pipeline for qcs615-ride to confirm that rootfs.img is being generated and bundled into the .qcomflash tarball before the LAVA job is dispatched — the postprocess.sh script unconditionally writes ROOTFS_IMAGE=rootfs.img but the artifact packaging step must ensure that file is present.
  4. Detail analysis attachment: failed_case_job94605_3_detailed.md

@rahujosh
Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #64

Job 94607 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94607

Failed test cases in LAVA job 94607 (SoC: x1e80100).

  Case 1: ** login-action
  1. Failed case: ** login-action
  2. Root cause: ** After the kernel entered s2idle suspend (~33s into first boot), the board performed a warm reset and re-entered UEFI; on that second UEFI pass, EnterpriseMgtDxe.efi hit ASSERT EnterpriseMgtDxe.c +111: 0 (preceded by ERROR: C90000002:V03000007 and EnterpriseMgt : Registers are already locked..!!), halting the UEFI firmware and preventing the kernel from booting again — so LAVA's login-action never received the shell prompt across all 3 retry attempts.
  3. Possible fix: This is a pre-existing firmware/platform issue on x1e80100-crd (not introduced by the PR); short-term: add no_console_suspend and mem_sleep_default=freeze (or =s3) to the kernel cmdline in the LAVA job definition to prevent s2idle entry during the test, or add a systemd inhibitor in the initramfs to block suspend; proper fix: report the EnterpriseMgtDxe ASSERT to the firmware team for the x1e80100 CRD board (the UEFI assert fires when registers are already locked on a warm-reset re-entry, which is a firmware bug in EnterpriseMgtDxe.c:111).
  4. Detail analysis attachment: failed_case_job94607_1_detailed.md
  Case 2: ** auto-login-action
  1. Failed case: ** auto-login-action
  2. Root cause: ** The x1e80100-CRD board entered s2idle system suspend at kernel time ~33.5s (immediately after reaching the login prompt), caused by mem_sleep_default=s2idle in the kernel command line; the serial console was suspended (printk: Suspending console(s)), the board rebooted, and all three login-action attempts (200s, 200s, 176s) timed out waiting for root@[^:]+:~# — which never appeared within the timeout window.
  3. Possible fix: Remove mem_sleep_default=s2idle from the LAVA job's kernel command line (or replace with mem_sleep_default=freeze / no_console_suspend as a diagnostic aid), so the board does not auto-suspend before LAVA can establish a shell session.
  4. Detail analysis attachment: failed_case_job94607_2_detailed.md
  Case 3: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** The x1e80100-crd board's UEFI firmware halts with ASSERT EnterpriseMgtDxe.c +111: 0 (triggered by EnterpriseMgt : Registers are already locked..!! / ERROR: C90000002:V03000007) during the DXE phase on every boot attempt, preventing the kernel from ever starting; LAVA's auto-login-action then times out waiting for Linux version on the serial console.
  3. Possible fix: Re-trigger the CI job; if the ASSERT recurs consistently, escalate to the board/firmware owner to investigate the EnterpriseMgtDxe register-lock state on this specific x1e80100-crd unit — the UEFI firmware may need to be reflashed or the board power-cycled with a full cold reset to clear the locked register state.
  4. Detail analysis attachment: failed_case_job94607_3_detailed.md
  Case 4: ** job
  1. Failed case: ** job
  2. Root cause: ** The x1e80100-CRD board's UEFI DXE phase hit a fatal ASSERT EnterpriseMgtDxe.c +111: 0 on the second boot cycle (triggered by EnterpriseMgt : Registers are already locked..!! + ERROR: C90000002:V03000007), halting the board in firmware before Linux could load; the auto-login-action then timed out across all 3 retries (3×200 s) exhausting the 10-minute fastboot-boot block timeout.
  3. Possible fix: Re-trigger the CI job; if the ASSERT recurs, perform a full power-cycle (cold reset) of the x1e80100-CRD board to clear the locked UEFI register state before the next LAVA run, and investigate whether a prior job left the board in a partially-booted UEFI state that persists across warm resets.
  4. Detail analysis attachment: failed_case_job94607_4_detailed.md
Job 94606 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94606

Failed test cases in LAVA job 94606 (SoC: qcs615-ride).

  Case 1: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure — flash image post-processing stage; postprocess.sh (inside ghcr.io/mwasilew/docker-mkbootimage:master) completed in 2 seconds without extracting the .qcomflash.tar.gz or creating rootfs.img, causing flash-universal.sh to abort with mount: special device rootfs.img does not exist (exit 32).
  3. Possible fix: Re-trigger the CI job; if it recurs, inspect postprocess.sh inside the docker-mkbootimage:master image to confirm it handles the qcs615-ride .qcomflash.tar.gz format and produces rootfs.img — if the script is missing the extraction step for this artifact type, update the postprocess.sh or the docker-mkbootimage image to correctly unpack the tarball and generate rootfs.img before flash-universal.sh is invoked.
  4. Detail analysis attachment: failed_case_job94606_1_detailed.md
  Case 2: ** Build Load Failure — Missing rootfs.img in flash artifact
  1. Failed case: ** Build Load Failure — Missing rootfs.img in flash artifact
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh attempted to mount rootfs.img from the extracted qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact, but the file was not present in the tarball, causing mount to fail with special device rootfs.img does not exist (exit 32) and aborting the EDL flash sequence for the qcs615-ride board.
  3. Possible fix: Re-trigger the CI job to check if this is a transient build artifact packaging issue; if it recurs, inspect the CI build pipeline for PR#64 (build ID 25633942419-1) to confirm that rootfs.img is being correctly generated and included in the .qcomflash.tar.gz packaging step — the flash.settings declares ROOTFS_IMAGE=rootfs.img but the artifact tarball shipped without it.
  4. Detail analysis attachment: failed_case_job94606_2_detailed.md
  Case 3: ** Build Load Failure — Flash script mount error (`rootfs.img` missing from flash archive)
  1. Failed case: ** Build Load Failure — Flash script mount error (rootfs.img missing from flash archive)
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh stage failed because rootfs.img does not exist in the deploy working directory; the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact for qcs615-ride contains only raw partition images (firehose/EDL .mbn/.elf/.bin files) and no rootfs.img, yet flash.settings declares ROOTFS_IMAGE=rootfs.img causing flash-universal.sh to attempt a loop-mount that fails with special device rootfs.img does not exist.
  3. Possible fix: Inspect the postprocess.sh script bundled in the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact to determine whether rootfs.img should be extracted/generated during postprocessing (and fix the script), or update flash.settings / flash-universal.sh for qcs615-ride to not attempt mounting a rootfs image when the flash method is pure EDL/firehose partition flashing; re-trigger the CI job after the fix to confirm deploy-flasher passes.
  4. Detail analysis attachment: failed_case_job94606_3_detailed.md
Job 94608 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94608

Failed test cases in LAVA job 94608 (SoC: qcs8300-ride).

  Case 1: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (CDSP offline)
  1. Failed case: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (CDSP offline)
  2. Root cause: ** The CDSP subsystem (remoteproc2) stays offline because request_firmware() returns -ENOENT (-2) for qcom/qcs8300/cdsp0.mbn — the firmware file is absent from the LAVA test initramfs/rootfs, so the remoteproc driver never boots the DSP.
  3. Possible fix: Populate the LAVA test rootfs with QCS8300 DSP firmware blobs (qcom/qcs8300/cdsp0.mbn, adsp.mbn, gpdsp0.mbn) or ensure the vendor firmware partition is mounted before the remoteproc test runs; additionally fix the LAVA job definition to use qcs8300-ride.dtb instead of monaco-evk.dtb for this board.
  4. Detail analysis attachment: failed_case_job94608_1_detailed.md
  Case 2: ** adsp_remoteproc — Remoteproc Firmware Load Failure (`Direct firmware load for qcom/qcs8300/adsp.mbn failed with error -2`)
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure (Direct firmware load for qcom/qcs8300/adsp.mbn failed with error -2)
  2. Root cause: ** The DSP firmware file qcom/qcs8300/adsp.mbn is absent from the kernel firmware search path on the qcs8300-ride LAVA board; request_firmware() returns -ENOENT on both attempts, leaving remoteproc0 permanently in offline state — this is a lab firmware provisioning gap, not a kernel regression.
  3. Possible fix: Provision the qcs8300-ride board's firmware partition with the correct qcom/qcs8300/adsp.mbn (and gpdsp0.mbn, cdsp0.mbn) blobs, or bundle them into the LAVA deploy step/initramfs overlay so they are visible to request_firmware() before the remoteproc driver attempts to boot the subsystem.
  4. Detail analysis attachment: failed_case_job94608_2_detailed.md
  Case 3: ** `gpdsp_remoteproc` — Remoteproc Firmware Load Failure (missing `qcom/qcs8300/gpdsp0.mbn`)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (missing qcom/qcs8300/gpdsp0.mbn)
  2. Root cause: ** request_firmware("qcom/qcs8300/gpdsp0.mbn") returns -ENOENT (-2) on both direct and ueventd-fallback attempts because the DSP firmware binary is absent from the LAVA test initramfs/rootfs for qcs8300-ride; the Qualcomm remotefs service starts but cannot source the firmware, leaving remoteproc1 permanently offline. The same failure hits adsp and cdsp simultaneously, confirming the entire qcom/qcs8300/ firmware directory is missing from the test image — this is a pre-existing infrastructure issue, not introduced by PR Test change reflect #64 (which only bumps the Makefile version string).
  3. Possible fix: Ensure the qcom/qcs8300/*.mbn DSP firmware files (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) are bundled into the LAVA test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) under /lib/firmware/qcom/qcs8300/, or configure the LAVA job to mount/provide a firmware partition containing these binaries before the remoteproc boot attempt.
  4. Detail analysis attachment: failed_case_job94608_3_detailed.md
  Case 4: ** remoteproc — Firmware Load Failure (missing `.mbn` files)
  1. Failed case: ** remoteproc — Firmware Load Failure (missing .mbn files)
  2. Root cause: ** All three qcs8300 DSP subsystems (adsp, gpdsp0, cdsp0) fail to boot because their firmware files (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn) are absent from the firmware search path on the LAVA rootfs — request_firmware() returns -ENOENT (-2) for every attempt, leaving all rprocs state=offline.
  3. Possible fix: Ensure the LAVA job's rootfs/firmware overlay includes the qcs8300 .mbn firmware files under /lib/firmware/qcom/qcs8300/, or verify the Qualcomm remotefs service correctly mounts the vendor firmware partition before the qcom_q6v5 module triggers remoteproc boot.
  4. Detail analysis attachment: failed_case_job94608_4_detailed.md
  Case 5: ** Probe_Failure_Check
  1. Failed case: ** Probe_Failure_Check
  2. Root cause: ** Multiple pre-existing driver probe and firmware-load failures on qcs8300-ride: cpufreq-dt fails with -EEXIST because qcom-cpufreq-hw is already registered; qcom-ice 87c8000.crypto fails with -ENOENT due to a missing clock dependency; amc6821 and tpm_tis_spi fail with -ETIMEDOUT because those ICs are absent/unpowered on this board; and adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn firmware files are not present in the open-source CI initramfs — none of these are caused by PR Test change reflect #64 (which only changes PATCHLEVEL/SUBLEVEL in the Makefile).
  3. Possible fix: Add the known-benign probe failures for qcs8300-ride to the Probe_Failure_Check test's allowlist (cpufreq-dt EEXIST, ICE/amc6821/tpm ETIMEDOUT/ENOENT, and remoteproc firmware ENOENT); separately, fix the 87c8000.crypto DT clock binding and disable DT nodes for unpopulated hardware (amc6821, tpm_tis_spi) on qcs8300-ride.
  4. Detail analysis attachment: failed_case_job94608_5_detailed.md
  Case 6: ** BT_FW_KMD_Service — WCN6855 BT UART driver initialization failure (driver probe hard-fail, `cots-driver-module-issues.md` Possibility 2)
  1. Failed case: ** BT_FW_KMD_Service — WCN6855 BT UART driver initialization failure (driver probe hard-fail, cots-driver-module-issues.md Possibility 2)
  2. Root cause: ** The WCN6855 BT controller on qcs8300-ride never responds to the QCA version-query HCI vendor command (0xfc00) — all 4 attempts time out with -ETIMEDOUT (-110) — because the three required power-supply regulators (vddio, vddbtcxmx, vddrfa1p7) are absent from the board DT and fall back to dummy regulators, leaving the chip unpowered/unresponsive; additionally, the WCN6855 firmware files (msbtfw*/msnv*) are missing from the CI ramdisk, making firmware download impossible even if UART communication were restored.
  3. Possible fix: Add vddio-supply, vddbtcxmx-supply, and vddrfa1p7-supply regulator entries to the qcs8300-ride DT for the serial@988000/bluetooth node, and add the WCN6855 firmware files (msbtfw11.mbn, msnv11.bin) under /lib/firmware/qca/ in the CI ramdisk image.
  4. Detail analysis attachment: failed_case_job94608_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests`
  1. Failed case: ** 0_qcom-next-ci-premerge-tests
  2. Root cause: ** The lava-test-shell 20-minute timeout was exhausted because the BT_SCAN test hung indefinitely: bluetoothctl interactive mode never returned a result after the Bluetooth firmware failed to initialize on the qcs8300-ride board (hci0 command timeout, BD address all zeros, Reading QCA version information failed (-110)), causing no LAVA_SIGNAL_TESTCASE to be emitted before the 1200-second deadline.
  3. Possible fix: Re-trigger the CI job to confirm reproducibility; if the BT hang recurs, add a timeout guard (e.g. timeout 60s bluetoothctl) inside the BT_SCAN test script, or mark BT_SCAN as SKIP on qcs8300-ride until the QCA Bluetooth firmware initialization issue is resolved on this board.
  4. Detail analysis attachment: failed_case_job94608_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The BT_SCAN test suite hung indefinitely on the qcs8300-ride (Monaco EVK) board: bluetoothctl entered an interactive loop waiting for the BT controller to become visible after public-addr 00:00:00:00:00:00 was applied, but the controller never re-appeared (logged as btensurepublicaddr(hci0) did not report success), causing the test to spin for the remaining ~4.5 minutes until the 1200-second lava-test-shell timeout was hit.
  3. Possible fix: Add a hard timeout guard (e.g. timeout 120s) around the bluetoothctl interactive session in the BT_SCAN test script so a non-responsive controller causes a FAIL result rather than an infinite hang; also investigate why hci0 loses visibility after public-addr on this SoC (likely a firmware/HCI reset issue on qcs8300 that leaves the controller in a DOWN state).
  4. Detail analysis attachment: failed_case_job94608_8_detailed.md
Job 95058 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95058

Failed test cases in LAVA job 95058 (SoC: qcs8300-ride).

  Case 1: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (ENOENT)
  1. Failed case: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (ENOENT)
  2. Root cause: ** The test rootfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain qcom/qcs8300/cdsp0.mbn; request_firmware() returns -2 (ENOENT) at boot, leaving remoteproc2 (cdsp) permanently in offline state before the test runs. All three qcs8300 DSPs (adsp, gpdsp0, cdsp) fail identically, confirming a systemic missing-firmware-package issue in the LAVA rootfs — not a kernel regression from this PR.
  3. Possible fix: Add qcom/qcs8300/cdsp0.mbn (and adsp.mbn, gpdsp0.mbn) to the initramfs-kerneltest-full-image-qcom-armv8a rootfs by including the qcs8300 firmware package in the Yocto/OE recipe, then re-trigger the LAVA job to confirm remoteproc2 reaches running state.
  4. Detail analysis attachment: failed_case_job95058_1_detailed.md
  Case 2: ** Remoteproc Firmware Load Failure — `qcom/qcs8300/adsp.mbn` not found (`-ENOENT`)
  1. Failed case: ** Remoteproc Firmware Load Failure — qcom/qcs8300/adsp.mbn not found (-ENOENT)
  2. Root cause: ** The test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain the QCS8300 DSP firmware files under /lib/firmware/qcom/qcs8300/; request_firmware("qcom/qcs8300/adsp.mbn") returns -ENOENT (-2) on both attempts, leaving remoteproc0 permanently in offline state on the qcs8300-ride board.
  3. Possible fix: Add qcom/qcs8300/adsp.mbn (and gpdsp0.mbn, cdsp0.mbn plus their .bXX split segments) to the test initramfs or add a LAVA job firmware-overlay step that installs these files under /lib/firmware/qcom/qcs8300/ before the remoteproc test suite runs.
  4. Detail analysis attachment: failed_case_job95058_2_detailed.md
  Case 3: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (`-ENOENT`)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (-ENOENT)
  2. Root cause: ** The initramfs used by LAVA job 95058 does not contain qcom/qcs8300/gpdsp0.mbn; request_firmware() returns -ENOENT (-2) on both attempts, leaving remoteproc1 (gpdsp0) permanently in offline state on qcs8300-ride.
  3. Possible fix: Add the qcs8300 DSP firmware blobs (gpdsp0.mbn, and co-failing adsp.mbn, cdsp0.mbn) under /lib/firmware/qcom/qcs8300/ in the initramfs-kerneltest-full-image-qcom-armv8a rootfs image by including the appropriate firmware package in the Yocto/OE recipe that builds this initramfs.
  4. Detail analysis attachment: failed_case_job95058_3_detailed.md
  Case 4: ** remoteproc — Firmware Load Failure (all 3 DSP subsystems offline)
  1. Failed case: ** remoteproc — Firmware Load Failure (all 3 DSP subsystems offline)
  2. Root cause: ** The initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz test rootfs does not contain the QCS8300 DSP firmware files (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn); request_firmware() returns -ENOENT (error -2) for all three, leaving remoteproc0/1/2 permanently offline.
  3. Possible fix: Add qcom/qcs8300/adsp.mbn, qcom/qcs8300/gpdsp0.mbn, and qcom/qcs8300/cdsp0.mbn to the meta-qcom initramfs image under /lib/firmware/qcom/qcs8300/, or configure the LAVA job to mount a firmware overlay before the remoteproc test runs.
  4. Detail analysis attachment: failed_case_job95058_4_detailed.md
  Case 5: ** `Probe_Failure_Check` — Driver/Firmware Probe Failures (pre-existing platform issues)
  1. Failed case: ** Probe_Failure_Check — Driver/Firmware Probe Failures (pre-existing platform issues)
  2. Root cause: ** Multiple pre-existing probe failures on qcs8300-ride triggered the test: (1) missing firmware blobs for remoteproc subsystems (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn) and video codec (qcom/vpu/vpu30_p4_s6.mbn) — Direct firmware load … failed with error -2; (2) qcom-ice 87c8000.crypto: probe with driver qcom-ice failed with error -2 — eMMC ICE DT node present but hardware/resource absent on this UFS-primary board; (3) hardware-absent I2C/SPI devices (amc6821 1-0018, tpm_tis_spi spi0.0) failing with error -110 (ETIMEDOUT). None of these are introduced by PR Test change reflect #64.
  3. Possible fix: Provision the missing qcs8300 firmware blobs in the LAVA rootfs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn); disable the 87c8000.crypto eMMC ICE DT node (status = "disabled") in the qcs8300-ride DTS since eMMC is not present; disable DT nodes for unpopulated hardware (amc6821, tpm_tis_spi); and add a per-board allowlist to the Probe_Failure_Check CI script for known-absent hardware on qcs8300-ride.
  4. Detail analysis attachment: failed_case_job95058_5_detailed.md
  Case 6: ** BT_FW_KMD_Service — WCN6855 BT firmware setup failure (driver probe with missing regulators / UART command timeout)
  1. Failed case: ** BT_FW_KMD_Service — WCN6855 BT firmware setup failure (driver probe with missing regulators / UART command timeout)
  2. Root cause: ** The btqca/hci_uart_qca driver on qcs8300-ride falls back to dummy regulators for vddio, vddbtcxmx, and vddrfa1p7 (DT supply phandles missing or PMIC not configured in test image), leaving the WCN6855 chip unpowered/unreset; all 3 retries of HCI vendor command 0xfc00 time out with -110 (ETIMEDOUT), so firmware is never downloaded and hci0 stays DOWN with BD address 00:00:00:00:00:00.
  3. Possible fix: Add the missing vddio-supply, vddbtcxmx-supply, and vddrfa1p7-supply regulator phandles to the bluetooth child node under serial@988000 in arch/arm64/boot/dts/qcom/qcs8300-ride.dts (or its included dtsi), pointing to the correct PMIC regulator nodes, so the WCN6855 chip is properly powered before the UART version query.
  4. Detail analysis attachment: failed_case_job95058_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests` — BT_SCAN test hang → lava-test-shell timeout
  1. Failed case: ** 0_qcom-next-ci-premerge-tests — BT_SCAN test hang → lava-test-shell timeout
  2. Root cause: ** The WCN6855 Bluetooth chip on the qcs8300-ride (Monaco EVK) board fails to initialize over UART: hci_uart_qca serial0-0 sends HCI vendor command 0xfc00 but receives no response (command 0xfc00 tx timeout, -ETIMEDOUT/-110), exhausting all 3 retries, leaving hci0 permanently DOWN with BD address 00:00:00:00:00:00; the subsequent BT_SCAN test hangs indefinitely in the btensurepublicaddr bootstrap loop waiting for a controller that never comes up, consuming the full 1200 s LAVA test-shell timeout.
  3. Possible fix: This is a pre-existing board-level hardware/firmware issue (WCN6855 not responding on UART) unrelated to the PR (which only bumps Makefile version numbers); re-trigger the job to confirm reproducibility, and investigate the qcs8300-ride board's WCN6855 power/UART wiring or missing BT firmware (msbtfw*/msnv* not found under standard firmware paths) — ensure the correct BT firmware blobs are present in the initramfs or rootfs for this board.
  4. Detail analysis attachment: failed_case_job95058_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The BT_SCAN test script hung indefinitely on the qcs8300-ride board because the WCN6855 Bluetooth controller never initialized — hci_uart_qca failed all 4 retries of command 0xfc00 with -ETIMEDOUT (-110) due to missing power-rail regulators (vddio/vddbtcxmx/vddrfa1p7 all fell back to dummy), leaving hci0 with BD address 00:00:00:00:00:00 and no firmware loaded; the test's bluetoothctl public-addr retry loop never exited, exhausting the 1200-second lava-test-shell timeout.
  3. Possible fix: Skip or gate BT_SCAN (and BT_FW_KMD_Service) on a pre-check that confirms hci0 has a valid non-zero BD address before entering the retry loop; separately, fix the qcs8300-ride DT/regulator configuration so that vddio, vddbtcxmx, and vddrfa1p7 are properly supplied to hci_uart_qca serial0-0 (or confirm the WCN6855 is not populated on this board variant and disable the BT node in the DT).
  4. Detail analysis attachment: failed_case_job95058_8_detailed.md
Job 95059 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95059

Failed test cases in LAVA job 95059 (SoC: qcs615-ride).

  Case 1: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure — flash image extraction stage; flash-universal.sh failed with exit code 32 because rootfs.img declared in flash.settings (ROOTFS_IMAGE=rootfs.img) is absent from the extracted .qcomflash.tar.gz artifact (25650106467-1) for qcs615-ride — the tarball contains only firehose firmware partitions (GPT, MBNs, ELFs) but no rootfs image file, causing the mount step to fail: mount: special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, investigate the qcs615-ride build pipeline to ensure rootfs.img is produced and included in the .qcomflash.tar.gz artifact, or correct flash.settings to point to the actual rootfs image filename present in the tarball (e.g. verify whether the platform uses a different image name or a separate rootfs artifact URL).
  4. Detail analysis attachment: failed_case_job95059_1_detailed.md
  Case 2: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh mount stage failed because rootfs.img is absent from the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz archive; flash.settings declares ROOTFS_IMAGE=rootfs.img but the qcomflash tarball for qcs615-ride contains only firmware partition images (xbl, tz, uefi, rawprogram XMLs, etc.) and no rootfs image file, causing mount to exit 32 with special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job to rule out a transient artifact packaging error; if it recurs, inspect the qcs615-ride image build pipeline to confirm rootfs.img is being generated and bundled into the qcomflash tarball, or update postprocess.sh/flash.settings to set ROOTFS_IMAGE to the correct image filename present in the archive (e.g. the .ext4 or .img partition image actually produced by the qcs615-ride Yocto build).
  4. Detail analysis attachment: failed_case_job95059_2_detailed.md
  Case 3: ** Infrastructure Failure — Flash Artifact Missing `rootfs.img` (`job`)
  1. Failed case: ** Infrastructure Failure — Flash Artifact Missing rootfs.img (job)
  2. Root cause: ** The qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact downloaded from the CI build (run 25650106467-1) does not contain rootfs.img; flash-universal.sh attempted to mount it at /var/lib/lava/dispatcher/tmp/95059/deploy-flasher-x7lv0doa/tmprootfs and failed with mount: special device rootfs.img does not exist, causing flash-universal.sh to exit 32 and LAVA to raise InfrastructureError: Unable to flash the device.
  3. Possible fix: Inspect the CI build artifact 25650106467-1 to confirm rootfs.img is absent from the .qcomflash.tar.gz; if the build is broken (rootfs image not generated), fix the build pipeline to ensure rootfs.img is included in the flash tarball, then re-trigger the LAVA job against a corrected artifact.
  4. Detail analysis attachment: failed_case_job95059_3_detailed.md
Job 95057 | SoC qcs6490-rb3gen2

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95057

Failed test cases in LAVA job 95057 (SoC: qcs6490-rb3gen2).

  Case 1: ** deploy-flasher
  1. Failed case: ** deploy-flasher
  2. Root cause: ** flash-universal.sh failed at the rootfs loop-mount stage with mount: tmprootfs: failed to setup loop device for rootfs.img. (exit 1 after 4 s), preventing EDL-mode flashing of the qcs6490-rb3gen2 board; the failure is an infrastructure issue on the LAVA worker — loop device exhaustion or /dev/loop* nodes not accessible in the flash execution environment.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, investigate loop device availability on the LAVA worker — ensure /dev/loop-control and sufficient /dev/loop* nodes are passed through to the flash container/environment, and check for stale loop mounts (losetup -a) that may be exhausting the pool.
  4. Detail analysis attachment: failed_case_job95057_1_detailed.md
  Case 2: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure — flash-stage mount failure; flash-universal.sh attempted mount -o loop rootfs.img tmprootfs but rootfs.img is absent from the qcom-multimedia-image-rb3gen2-core-kit.rootfs.qcomflash.tar.gz archive, causing the exact error "mount: tmprootfs: failed to setup loop device for rootfs.img" and propagating as "Unable to flash the device".
  3. Possible fix: Re-trigger the CI job; if the failure recurs, inspect the qcom-multimedia-image-rb3gen2-core-kit.rootfs.qcomflash.tar.gz artifact produced by build run 25650106467-1 to confirm rootfs.img is missing, then fix the upstream image build pipeline (Yocto/OE recipe for qcom-multimedia-image-rb3gen2-core-kit) to ensure rootfs.img is packaged into the .qcomflash archive before the LAVA job is triggered.
  4. Detail analysis attachment: failed_case_job95057_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — Loop Device Unavailable
  1. Failed case: ** Infrastructure Flash Failure — Loop Device Unavailable
  2. Root cause: ** Result: Build Load Failure — the flash-universal.sh script failed during the deploy-flasher stage on the qcs6490-rb3gen2 LAVA worker because the flasher environment could not mount rootfs.img as a loop device: mount: tmprootfs: failed to setup loop device for rootfs.img (exit code 1), causing Unable to flash the device.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, ensure the LAVA worker container passes through loop devices (--device=/dev/loop-control --device=/dev/loop0 etc.) or that the loop kernel module is loaded and sufficient loop device slots are available on the host, then redeploy the worker container.
  4. Detail analysis attachment: failed_case_job95057_3_detailed.md
Job 95060 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95060

Failed test cases in LAVA job 95060 (SoC: x1e80100).

  Case 1: ** cdsp_remoteproc
  1. Failed case: ** cdsp_remoteproc
  2. Root cause: ** The CDSP firmware blob qcom/x1e80100/cdsp.mbn is absent from the test initramfs; request_firmware() returns ENOENT (error -2) at remoteproc probe time, leaving remoteproc1 in state=offline on the x1e80100 CRD board.
  3. Possible fix: Add the x1e80100 DSP firmware files (cdsp.mbn, adsp.mbn, and companion blobs) to the linux-firmware package or firmware overlay layer used to build the initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz test rootfs, then rebuild and re-publish the image.
  4. Detail analysis attachment: failed_case_job95060_1_detailed.md
  Case 2: ** adsp_remoteproc — Remoteproc Firmware Load Failure
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure
  2. Root cause: ** The x1e80100 ADSP firmware file qcom/x1e80100/adsp.mbn is absent from the test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz), causing request_firmware() to return -ENOENT (-2) and leaving remoteproc0 permanently in offline state on the x1e80100 CRD board.
  3. Possible fix: Add the x1e80100 DSP firmware blobs (qcom/x1e80100/adsp.mbn, cdsp.mbn, and any split .bXX segments) to the test initramfs under /lib/firmware/qcom/x1e80100/, or add a LAVA job pre-test step that fetches and installs these firmware files before the remoteproc test suite runs.
  4. Detail analysis attachment: failed_case_job95060_2_detailed.md
  Case 3: ** `0_qcom-next-ci-premerge-tests` — Kernel Suspend Hang (s2idle) causing LAVA test-shell timeout; preceded by Remoteproc Firmware Load Failure (adsp/cdsp missing from initramfs)
  1. Failed case: ** 0_qcom-next-ci-premerge-tests — Kernel Suspend Hang (s2idle) causing LAVA test-shell timeout; preceded by Remoteproc Firmware Load Failure (adsp/cdsp missing from initramfs)
  2. Root cause: ** The CPUFreq_Validation test triggered an s2idle suspend cycle on the x1e80100-CRD board at kernel T+33s; the board never resumed (console suspended at printk: Suspending console(s), no wakeup logged), causing the LAVA lava-test-shell to time out after 1200 seconds with "lava-test-shell timed out after 1200 seconds". A secondary pre-existing failure is that qcom/x1e80100/adsp.mbn (and cdsp.mbn) are absent from the test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz), causing request_firmware failed: -2 and adsp_remoteproc / cdsp_remoteproc to report FAIL before the hang occurs.
  3. Possible fix: For the suspend hang: add no_console_suspend to the kernel cmdline in the LAVA job definition to keep the serial console alive through s2idle, and/or configure the CPUFreq_Validation test to skip or guard the s2idle cycle on x1e80100-CRD (which lacks a reliable wakeup source in this LAVA setup); for the firmware failures: add qcom/x1e80100/adsp.mbn and cdsp.mbn blobs to the test initramfs image used by this job.
  4. Detail analysis attachment: failed_case_job95060_3_detailed.md
  Case 4: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The CPUFreq_Validation test on x1e80100-CRD triggered an s2idle system suspend (kernel cmdline mem_sleep_default=s2idle) at kernel uptime ~33 s; the board entered suspend (printk: Suspending console(s)) and never resumed, causing the LAVA test-shell to receive no further output and time out after exactly 1200 seconds.
  3. Possible fix: Add no_console_suspend to the kernel cmdline in the LAVA job definition to keep the serial console alive through suspend/resume cycles, and ensure the CPUFreq_Validation test script uses rtcwake (or equivalent) to schedule a wakeup alarm before triggering s2idle; if the test is not intended to exercise suspend, set mem_sleep_default=freeze or remove the s2idle default from the cmdline for this job.
  4. Detail analysis attachment: failed_case_job95060_4_detailed.md
  Case 5: ** lava-test-retry
  1. Failed case: ** lava-test-retry
  2. Root cause: ** The CPUFreq_Validation test triggered an s2idle suspend cycle on the x1e80100 CRD board (via mem_sleep_default=s2idle in the kernel cmdline); the board entered PM: suspend entry (s2idle) at kernel T+33s, console was suspended (printk: Suspending console(s)), and the system never resumed — hanging silently for the full 1200 s lava-test-shell timeout.
  3. Possible fix: Investigate and fix the s2idle resume path on x1e80100 (check platform suspend hooks, PSCI/firmware, and any pending QDSS clock warnings that may block resume); as an immediate mitigation, add mem_sleep_default=shallow or no_console_suspend to the kernel cmdline in the LAVA job definition to prevent silent hangs during CPUFreq testing.
  4. Detail analysis attachment: failed_case_job95060_5_detailed.md
  Case 6: ** job
  1. Failed case: ** job
  2. Root cause: ** The X1E80100 CRD board entered PM: suspend entry (s2idle) at kernel uptime ~33 s (triggered by the CPUFreq_Validation test script while mem_sleep_default=s2idle was set on the kernel cmdline) and never resumed, causing the lava-test-shell to hang until LAVA's 1200-second timeout expired.
  3. Possible fix: Remove mem_sleep_default=s2idle from the kernel cmdline in the LAVA job definition (or add no_console_suspend temporarily to debug), and investigate why the CPUFreq_Validation test triggers a system suspend on x1e80100 — likely the test script writes to /sys/power/state or a cpufreq governor triggers idle entry that escalates to s2idle; the wakeup source (RTC alarm) is either not armed or not functional on this platform.
  4. Detail analysis attachment: failed_case_job95060_6_detailed.md
Job 94604 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94604

Failed test cases in LAVA job 94604 (SoC: qcs8300-ride).

  Case 1: ** wait-device-boardid
  1. Failed case: ** wait-device-boardid
  2. Root cause: ** After a clean PDU power-cycle (alpaca_control.py off/on, exit 0 each time), the qcs8300-ride board (qcs8300-ride-kci-0301) never enumerated as a fastboot USB device with board-id 1315e8d7 on the LAVA worker — all 3 attempts timed out (200 s, 200 s, 105 s) with no USB udev event seen, indicating the board is not entering fastboot mode on boot (stuck in normal/UFS boot path or USB cable/hub issue).
  3. Possible fix: Check the physical USB cable between the qcs8300-ride board and the LAVA worker container; verify the board's UEFI/ABL fastboot boot path is correctly configured (boot-mode strapping or UEFI BootOrder) so it enters fastboot on power-on, then re-trigger the CI job. If the board consistently fails to enumerate, inspect the worker container's USB passthrough for device 1315e8d7 and confirm udevadm monitor sees the fastboot device when the board is manually power-cycled.
  4. Detail analysis attachment: failed_case_job94604_1_detailed.md
  Case 2: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** The qcs8300-ride board (device qcs8300-ride-kci-0301) boots successfully through PBL/SBL1/XBL/UEFI and loads Fastboot.efi with USB initialized in device mode (SuperSpeed, cable attached), but never enumerates as a udev device with board_id 1315e8d7 on the LAVA worker — wait-device-boardid timed out after 200 seconds on all 3 attempts, classified by LAVA as error_type: Infrastructure.
  3. Possible fix: Re-trigger the CI job; if the failure recurs consistently, investigate the USB cable/hub path between the qcs8300-ride board and the LAVA worker container — the board reaches Fastboot.efi and reports USB SuperSpeed enumeration but the host udev never sees the device, pointing to a USB passthrough or cable connectivity issue in the lab infrastructure.
  4. Detail analysis attachment: failed_case_job94604_2_detailed.md
  Case 3: ** Infrastructure Failure — `wait-device-boardid` timed out (fastboot USB device not enumerated on LAVA worker)
  1. Failed case: ** Infrastructure Failure — wait-device-boardid timed out (fastboot USB device not enumerated on LAVA worker)
  2. Root cause: ** The qcs8300-ride board boots correctly through SBL → UEFI → Fastboot.efi and the USB PHY initialises at SuperSpeed, but the USB device never enumerates on the LAVA worker host — udev event for board ID 1315e8d7 never fires across all 3 retry attempts — error_type: Infrastructure.
  3. Possible fix: Check the USB cable/hub connection between the qcs8300-ride board and the LAVA worker host for board 1315e8d7; verify the USB device appears on the host with lsusb after a manual power cycle, and if the port is flaky, reseat or replace the USB cable/hub and re-trigger the job.
  4. Detail analysis attachment: failed_case_job94604_3_detailed.md
Job 94605 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94605

Failed test cases in LAVA job 94605 (SoC: qcs615-ride).

  Case 1: ** deploy-flasher — Flash Script Failure — rootfs.img missing at mount time
  1. Failed case: ** deploy-flasher — Flash Script Failure — rootfs.img missing at mount time
  2. Root cause: ** flash-universal.sh failed with exit code 32 because rootfs.img was not present in the working directory when the script attempted to loop-mount it at tmprootfs/; postprocess.sh completed in ~2 s and only wrote settings metadata — it did not extract or produce rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz, leaving ROOTFS_IMAGE=rootfs.img in flash.settings pointing to a non-existent file.
  3. Possible fix: Inspect postprocess.sh inside ghcr.io/mwasilew/docker-mkbootimage:master to confirm whether it is supposed to extract rootfs.img from the tar archive for qcs615-ride; if the extraction step is missing or gated on a condition that was not met, fix postprocess.sh (or the image packaging pipeline) to ensure rootfs.img is produced before flash-universal.sh runs; re-trigger the CI job after the fix to verify the mount succeeds.
  4. Detail analysis attachment: failed_case_job94605_1_detailed.md
  Case 2: ** Build Load Failure — Fastboot flash failure (rootfs.img not produced by postprocess.sh)
  1. Failed case: ** Build Load Failure — Fastboot flash failure (rootfs.img not produced by postprocess.sh)
  2. Root cause: ** Result: Build Load Failure — the download-postprocess-docker step (step 1.3) ran postprocess.sh inside ghcr.io/mwasilew/docker-mkbootimage:master for only ~2 seconds and set ROOTFS_IMAGE=rootfs.img in flash.settings, but never actually extracted or created rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz; when flash-universal.sh subsequently tried to mount it, it failed with mount: special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, inspect postprocess.sh inside the ghcr.io/mwasilew/docker-mkbootimage:master image to confirm it correctly extracts rootfs.img from the .qcomflash.tar.gz tarball for the qcs615-ride platform, and verify the Docker container has sufficient /dev/kvm access and disk space to complete the image extraction step.
  4. Detail analysis attachment: failed_case_job94605_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — `rootfs.img` missing after postprocess
  1. Failed case: ** Infrastructure Flash Failure — rootfs.img missing after postprocess
  2. Root cause: ** flash-universal.sh failed with mount: special device rootfs.img does not exist because the postprocess.sh step (run via docker-mkbootimage) completed in ~2 s without extracting or building rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz — the .qcomflash archive contains raw Qualcomm partition binaries (GPT, ELF, XML manifests) but no pre-built mountable rootfs.img, so flash-universal.sh cannot proceed with the UFS flash sequence for qcs615-ride.
  3. Possible fix: Investigate and fix the postprocess.sh script bundled inside the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact (or the CI build pipeline that produces it) to ensure it correctly generates rootfs.img before flash-universal.sh is invoked; alternatively, update the LAVA job's deploy-flasher configuration to point to an artifact that already contains a pre-built rootfs.img, or update flash-universal.sh to extract and assemble rootfs.img directly from the .qcomflash partition files.
  4. Detail analysis attachment: failed_case_job94605_3_detailed.md

@rahujosh
Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #64

Job 94607 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94607

Failed test cases in LAVA job 94607 (SoC: x1e80100).

  Case 1: ** login-action
  1. Failed case: ** login-action
  2. Root cause: ** The x1e80100-crd board entered s2idle ~33 s after the first successful login, then rebooted; on the second boot the UEFI DXE driver EnterpriseMgtDxe hit ASSERT EnterpriseMgtDxe.c +111: 0 (triggered by "Registers are already locked" state left from the previous boot), halting firmware before the kernel could load, so LAVA never received a shell prompt and login-action timed out after 200 s across all 3 retry attempts.
  3. Possible fix: Add mem_sleep_default=no (or remove mem_sleep_default=s2idle) to the kernel command line in the LAVA job definition for this device to prevent the board from auto-suspending during the login window; additionally, report the EnterpriseMgtDxe ASSERT-on-warm-reboot to the firmware team as a pre-existing platform firmware bug on x1e80100-crd that must be fixed to make warm-reboot/resume reliable.
  4. Detail analysis attachment: failed_case_job94607_1_detailed.md
  Case 2: ** auto-login-action
  1. Failed case: ** auto-login-action
  2. Root cause: ** On the x1e80100 CRD, the kernel entered s2idle suspend at T+33s (triggered by mem_sleep_default=s2idle in the kernel cmdline) and never resumed — the board rebooted via PSHOLD Warm Reset instead of resuming, causing the second fastboot-boot attempt to stall in UEFI with ASSERT EnterpriseMgtDxe.c +111: 0 and never produce a console login prompt within the 200s timeout.
  3. Possible fix: Remove mem_sleep_default=s2idle from the LAVA job's kernel cmdline (or add mem_sleep_default=shallow / no_console_suspend) to prevent the board from auto-suspending during the LAVA test session; additionally investigate the UEFI ASSERT EnterpriseMgtDxe.c +111 triggered on warm-reset reboot which blocks the second boot attempt from completing.
  4. Detail analysis attachment: failed_case_job94607_2_detailed.md
  Case 3: ** Kernel Crash — WARNING storm + unexpected s2idle suspend causing board reboot before login
  1. Failed case: ** Kernel Crash — WARNING storm + unexpected s2idle suspend causing board reboot before login
  2. Root cause: ** The x1e80100-CRD board boots the kernel successfully and reaches the login prompt, but immediately enters s2idle suspend (mem_sleep_default=s2idle in cmdline) ~33 seconds after boot; the SBSA watchdog (10 s timeout, action=0) fires during suspend, causing a hard reboot — this repeats across all 3 login-action retry windows, exhausting the 9m41s auto-login-action timeout. The WARNING splats from workqueue.c:4289 (__flush_work / ffs_fs_kill_sb) and clk.c (clk_core_disable/clk_core_unprepare) are pre-existing issues unrelated to the PR patch (which only bumps PATCHLEVEL/SUBLEVEL in Makefile).
  3. Possible fix: Add mem_sleep_default=shallow (or mem_sleep_default=freeze) to the LAVA job's kernel cmdline for the x1e80100-CRD device to prevent automatic s2idle entry during the login window; alternatively add no_console_suspend to keep the console alive through suspend so LAVA can detect the resume. The WARNING splats should be investigated separately as pre-existing kernel bugs.
  4. Detail analysis attachment: failed_case_job94607_3_detailed.md
  Case 4: ** Boot Load Failure — UEFI ASSERT halts board before Linux login
  1. Failed case: ** Boot Load Failure — UEFI ASSERT halts board before Linux login
  2. Root cause: ** The x1e80100-CRD board stalls in the UEFI DXE phase with ASSERT EnterpriseMgtDxe.c +111: 0 (triggered by EnterpriseMgt: Registers are already locked + error C90000002:V03000007); the board produces no further console output, so auto-login-action exhausts all 3 login-action attempts (each 200 s / 176 s) and the fastboot-boot block (10 min) expires with "No time left for remaining 2 retries."
  3. Possible fix: Re-trigger the CI job; if the UEFI ASSERT recurs consistently, escalate to the board/firmware team to investigate the EnterpriseMgtDxe register-lock state on this specific CRD unit — the UEFI firmware may need a cold power-cycle or firmware update to clear the stale lock condition.
  4. Detail analysis attachment: failed_case_job94607_4_detailed.md
Job 94606 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94606

Failed test cases in LAVA job 94606 (SoC: qcs615-ride).

  Case 1: ** Build Load Failure — Missing rootfs image in flash bundle
  1. Failed case: ** Build Load Failure — Missing rootfs image in flash bundle
  2. Root cause: ** Result: Build Load Failure — flash artifact preparation stage; flash-universal.sh failed with exit code 32 because rootfs.img (declared as ROOTFS_IMAGE in flash.settings) is absent from the extracted qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz archive — the file was never included in the build artifact for this qcs615-ride job.
  3. Possible fix: Re-trigger the CI job to obtain a freshly built flash bundle; if the failure recurs, inspect the qcs615-ride image build pipeline to confirm rootfs.img is being generated and packaged into the .qcomflash.tar.gz archive before the LAVA job is dispatched.
  4. Detail analysis attachment: failed_case_job94606_1_detailed.md
  Case 2: ** Build Load Failure — Flash image preparation failure (`rootfs.img` not extracted)
  1. Failed case: ** Build Load Failure — Flash image preparation failure (rootfs.img not extracted)
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh failed at the mount stage because postprocess.sh (inside docker-mkbootimage:master) wrote ROOTFS_IMAGE=rootfs.img into flash.settings but never extracted or created rootfs.img from the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz; the exact error is mount: special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job to rule out a transient issue; if it recurs, inspect postprocess.sh inside ghcr.io/mwasilew/docker-mkbootimage:master to confirm it correctly extracts rootfs.img from the .tar.gz for the qcs615-ride platform, and update the script or the LAVA job's download-postprocess-docker step to ensure rootfs.img is present on disk before flash-universal.sh is invoked.
  4. Detail analysis attachment: failed_case_job94606_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — Missing rootfs image in flash artifact
  1. Failed case: ** Infrastructure Flash Failure — Missing rootfs image in flash artifact
  2. Root cause: ** The qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact produced by CI build 25633942419-1 does not contain rootfs.img; flash-universal.sh aborts with mount: special device rootfs.img does not exist (exit 32) when it attempts to mount the rootfs for overlay injection before flashing the qcs615-ride UFS storage.
  3. Possible fix: Re-trigger the CI job to regenerate the flash artifact; if rootfs.img is consistently absent, inspect the image-packaging step in the CI pipeline (build 25633942419-1) to confirm the rootfs image build target completed and was correctly bundled into the .qcomflash.tar.gz before LAVA downloads it.
  4. Detail analysis attachment: failed_case_job94606_3_detailed.md
Job 94608 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94608

Failed test cases in LAVA job 94608 (SoC: qcs8300-ride).

  Case 1: ** Remoteproc Firmware Load Failure — `Direct firmware load for qcom/qcs8300/cdsp0.mbn failed with error -2`
  1. Failed case: ** Remoteproc Firmware Load Failure — Direct firmware load for qcom/qcs8300/cdsp0.mbn failed with error -2
  2. Root cause: ** The firmware file qcom/qcs8300/cdsp0.mbn is absent from the target rootfs firmware search path (/lib/firmware/qcom/qcs8300/); request_firmware() returns -ENOENT (-2) on both load attempts, leaving remoteproc2 (cdsp) permanently in offline state — this is a missing firmware package in the LAVA rootfs/overlay, not a kernel regression introduced by PR Test change reflect #64.
  3. Possible fix: Populate /lib/firmware/qcom/qcs8300/ with cdsp0.mbn (and adsp.mbn, gpdsp0.mbn) in the LAVA rootfs image or add a pre-test deploy step in the LAVA job definition to place the qcs8300 DSP firmware files on the target before the remoteproc tests run.
  4. Detail analysis attachment: failed_case_job94608_1_detailed.md
  Case 2: ** adsp_remoteproc — Remoteproc Firmware Load Failure (ENOENT)
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure (ENOENT)
  2. Root cause: ** The LAVA test initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain the QCS8300 DSP firmware blob qcom/qcs8300/adsp.mbn; request_firmware() returns -ENOENT (-2) on every attempt, leaving remoteproc0 permanently in offline state.
  3. Possible fix: Ensure the QCS8300 DSP firmware files (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn) are bundled into the test initramfs image used by this LAVA job, or configure the LAVA job to mount a firmware overlay partition that provides these blobs before the remoteproc auto-boot triggers.
  4. Detail analysis attachment: failed_case_job94608_2_detailed.md
  Case 3: ** `gpdsp_remoteproc` — Remoteproc Firmware Load Failure (ENOENT: `qcom/qcs8300/gpdsp0.mbn` missing from CI ramdisk)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (ENOENT: qcom/qcs8300/gpdsp0.mbn missing from CI ramdisk)
  2. Root cause: ** The qcom/qcs8300/gpdsp0.mbn firmware file is absent from the CI test ramdisk (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz), causing request_firmware() to return ENOENT (-2) on the qcs8300-ride board; gpdsp0 (remoteproc1) never advances past powering up and stays offline. This is a pre-existing infrastructure gap — the PR patch only bumps Makefile version numbers and introduces no remoteproc or firmware changes.
  3. Possible fix: Add qcom/qcs8300/gpdsp0.mbn (and adsp.mbn, cdsp0.mbn) to the CI test ramdisk for qcs8300 targets, or add a LAVA job setup step that installs the firmware package into /lib/firmware/qcom/qcs8300/ before running remoteproc tests; do not block the PR on this failure.
  4. Detail analysis attachment: failed_case_job94608_3_detailed.md
  Case 4: ** remoteproc — Firmware Load Failure (all DSP subsystems offline)
  1. Failed case: ** remoteproc — Firmware Load Failure (all DSP subsystems offline)
  2. Root cause: ** All three remoteproc instances on qcs8300-ride (adsp, gpdsp0, cdsp) fail request_firmware with error -2 (ENOENT) because the firmware files qcom/qcs8300/adsp.mbn, qcom/qcs8300/gpdsp0.mbn, and qcom/qcs8300/cdsp0.mbn are absent from the LAVA test rootfs/firmware search path; the remoteproc driver itself probes correctly but cannot boot any subsystem, leaving all three in offline state.
  3. Possible fix: Ensure the LAVA deploy step for qcs8300-ride includes the qcom/qcs8300/ firmware files (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) in the rootfs or firmware partition image — verify the remotefs-mounted path is populated and re-trigger the job.
  4. Detail analysis attachment: failed_case_job94608_4_detailed.md
  Case 5: ** `Probe_Failure_Check` — Driver probe failures and missing firmware on qcs8300-ride
  1. Failed case: ** Probe_Failure_Check — Driver probe failures and missing firmware on qcs8300-ride
  2. Root cause: ** Multiple pre-existing driver probe failures on qcs8300-ride triggered the CI gate: cpufreq-dt fails with -EEXIST (-17) because the Qualcomm HW cpufreq driver registers first; qcom-ice 87c8000.crypto fails with -ENOENT (-2) due to missing clock dependency; amc6821 and tpm_tis_spi fail with -ETIMEDOUT (-110) because the hardware is not physically populated; and remoteproc firmware files (adsp.mbn, cdsp0.mbn, gpdsp0.mbn, vpu30_p4_s6.mbn) are absent from the LAVA test initramfs. None of these failures are introduced by PR64.
  3. Possible fix: Add a per-board allowlist to the Probe_Failure_Check test script for qcs8300-ride that suppresses these known-benign failures; separately, fix the cpufreq-dt conflict by ensuring only one cpufreq driver is active on qcs8300, and add qcs8300 firmware blobs to the LAVA initramfs.
  4. Detail analysis attachment: failed_case_job94608_5_detailed.md
  Case 6: ** `BT_FW_KMD_Service` — Bluetooth KMD/Firmware Service Failure (WCN6855 UART init timeout)
  1. Failed case: ** BT_FW_KMD_Service — Bluetooth KMD/Firmware Service Failure (WCN6855 UART init timeout)
  2. Root cause: ** The WCN6855 BT controller on qcs8300-ride never responds to the hci_uart_qca driver's initial HCI 0xfc00 (QCA get-version) command because all three BT power regulators (vddio, vddbtcxmx, vddrfa1p7) fall back to dummy regulators — the PMIC supply nodes are absent or unmatched in the DT — leaving the BT core unpowered; all 4 power-on retries time out with ETIMEDOUT (-110), hci0 stays DOWN with a zero BD address, and firmware is never downloaded.
  3. Possible fix: Add the missing vddio-supply, vddbtcxmx-supply, and vddrfa1p7-supply regulator references (and verify enable-gpios polarity) in the bluetooth child node of serial@988000 in the qcs8300-ride DTS; ensure the corresponding PMIC regulator nodes are defined and enabled so the WCN6855 BT core is actually powered before the UART handshake is attempted.
  4. Detail analysis attachment: failed_case_job94608_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests`
  1. Failed case: ** 0_qcom-next-ci-premerge-tests
  2. Root cause: ** The BT_SCAN test script hung for the remainder of the 20-minute lava-test-shell timeout because the Bluetooth controller (hci0) on the qcs8300-ride (Monaco EVK) board had an all-zeros BD address and was never visible to bluetoothctl in non-interactive mode; the script repeatedly retried bluetoothctl public-addr 00:00:00:00:00:00 in a loop (each iteration ~75 s) and never exited, while the kernel concurrently flooded the console with usb usb2-port1: config error / Cannot enable. Maybe the USB cable is bad? messages from a persistently failing USB2 port enumeration on the xHCI controller at 0xa600000.
  3. Possible fix: Add a hard timeout guard (e.g. timeout 60s) around the btensurepublicaddr / bluetoothctl retry loop in BT_SCAN/run.sh so the test exits with SKIP/FAIL instead of hanging; separately, investigate the usb usb2-port1 hardware enumeration failure on the Monaco EVK board (check USB cable/hub connection on the xHCI port at 0xa600000) as it may be masking the BT UART attach.
  4. Detail analysis attachment: failed_case_job94608_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The BT_SCAN test hung indefinitely inside an interactive bluetoothctl loop because the WCN6855 Bluetooth controller on qcs8300-ride never initialised — hci_uart_qca timed out on every 0xfc00 QCA version query (-ETIMEDOUT) from boot due to missing firmware regulators (vddio/vddbtcxmx/vddrfa1p7 all using dummy), leaving no BT controller visible; the test never emitted its LAVA_SIGNAL_TESTCASE result and the 1200-second lava-test-shell timeout expired.
  3. Possible fix: Add a hard timeout guard (e.g. timeout 60s) around the interactive bluetoothctl call in BT_SCAN/run.sh so a missing controller causes a SKIP/FAIL result instead of an infinite hang; separately, investigate the WCN6855 regulator/firmware supply configuration in the qcs8300-ride DT or firmware package to fix the underlying BT initialisation failure.
  4. Detail analysis attachment: failed_case_job94608_8_detailed.md
Job 95058 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95058

Failed test cases in LAVA job 95058 (SoC: qcs8300-ride).

  Case 1: ** `cdsp_remoteproc` — Remoteproc Firmware Load Failure (missing `qcom/qcs8300/cdsp0.mbn`)
  1. Failed case: ** cdsp_remoteproc — Remoteproc Firmware Load Failure (missing qcom/qcs8300/cdsp0.mbn)
  2. Root cause: ** The test rootfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain the qcom/qcs8300/cdsp0.mbn firmware file; request_firmware() returns -ENOENT (-2) causing rproc_boot() to abort and leaving remoteproc2 (cdsp) permanently in offline state.
  3. Possible fix: Add a LAVA deploy/overlay step that copies qcom/qcs8300/*.mbn firmware blobs into /lib/firmware/qcom/qcs8300/ on the target before the test suite runs, or include these files in the qcs8300 initramfs build recipe.
  4. Detail analysis attachment: failed_case_job95058_1_detailed.md
  Case 2: ** `adsp_remoteproc` — Remoteproc Firmware Load Failure (`qcom/qcs8300/adsp.mbn` not found)
  1. Failed case: ** adsp_remoteproc — Remoteproc Firmware Load Failure (qcom/qcs8300/adsp.mbn not found)
  2. Root cause: ** The initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz test image does not contain the qcom/qcs8300/adsp.mbn firmware file, causing request_firmware() to return ENOENT (-2) at boot; remoteproc0 (adsp) stays offline and the test fails immediately on the boot-state check.
  3. Possible fix: Add the qcom/qcs8300/adsp.mbn (and gpdsp0.mbn, cdsp0.mbn) firmware files to the qcs8300-ride test initramfs or add a LAVA pre-test deploy step that mounts/copies the firmware package to /lib/firmware/qcom/qcs8300/ before the remoteproc test suite runs.
  4. Detail analysis attachment: failed_case_job95058_2_detailed.md
  Case 3: ** `gpdsp_remoteproc` — Remoteproc Firmware Load Failure (`qcom/qcs8300/gpdsp0.mbn` missing)
  1. Failed case: ** gpdsp_remoteproc — Remoteproc Firmware Load Failure (qcom/qcs8300/gpdsp0.mbn missing)
  2. Root cause: ** The initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) does not contain the qcom/qcs8300/gpdsp0.mbn firmware file; request_firmware() returns -ENOENT (-2) and remoteproc1 (gpdsp0) stays offline — a pre-existing firmware packaging gap unrelated to PR Test change reflect #64.
  3. Possible fix: Add qcom/qcs8300/ DSP firmware binaries (gpdsp0.mbn and its split segments) to the initramfs-kerneltest-full-image-qcom-armv8a rootfs image and republish it to the artifact store used by the LAVA job.
  4. Detail analysis attachment: failed_case_job95058_3_detailed.md
  Case 4: ** remoteproc — Firmware Load Failure (all 3 DSP subsystems offline)
  1. Failed case: ** remoteproc — Firmware Load Failure (all 3 DSP subsystems offline)
  2. Root cause: ** All three QCS8300 DSP firmware files (qcom/qcs8300/adsp.mbn, gpdsp0.mbn, cdsp0.mbn) are absent from the test initramfs; request_firmware() returns -ENOENT (-2) for every subsystem at boot, leaving all remoteprocs in offline state — 0 of 3 expected subsystems running.
  3. Possible fix: Add the QCS8300 DSP firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn) to the CI test initramfs under /lib/firmware/qcom/qcs8300/ in the initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz image used for qcs8300-ride LAVA jobs.
  4. Detail analysis attachment: failed_case_job95058_4_detailed.md
  Case 5: ** `Probe_Failure_Check` — Driver/Firmware Probe Failures (pre-existing infra gap)
  1. Failed case: ** Probe_Failure_Check — Driver/Firmware Probe Failures (pre-existing infra gap)
  2. Root cause: ** Missing Qualcomm firmware blobs (adsp.mbn, gpdsp0.mbn, cdsp0.mbn, vpu30_p4_s6.mbn) in the CI test rootfs cause remoteproc and VPU drivers to fail with -ENOENT; additionally, hardware-absent devices (amc6821, tpm_tis_spi, PCIe ICE 87c8000.crypto) fail with -ETIMEDOUT/-ENOENT because they are not populated on the qcs8300-ride board in this LAVA lab — none of these are introduced by PR#64.
  3. Possible fix: Add the missing qcom/qcs8300/*.mbn and qcom/vpu/vpu30_p4_s6.mbn firmware blobs to the meta-qcom initramfs for qcs8300-ride, and add hardware-absent devices (amc6821, tpm_tis_spi, 87c8000.crypto) to the Probe_Failure_Check test's per-board suppression/allowlist.
  4. Detail analysis attachment: failed_case_job95058_5_detailed.md
  Case 6: ** `BT_FW_KMD_Service` — Driver/Firmware Dependency Failure (WCN6855 BT firmware absent, chip unresponsive)
  1. Failed case: ** BT_FW_KMD_Service — Driver/Firmware Dependency Failure (WCN6855 BT firmware absent, chip unresponsive)
  2. Root cause: ** The WCN6855 BT firmware binaries (msbtfw*/msnv*) are absent from the CI ramdisk; btqca's qca_uart_setup() exhausted all 3 power-on retries with command 0xfc00 tx timeout / Reading QCA version information failed (-110) on every attempt — the chip never responded on UART and hci0 remained DOWN with all-zero BD address. Three BT power-supply regulators also fell back to dummy regulators, indicating the WCN6855 may also be unpowered. This is a pre-existing board/infra issue — PR#64 only bumps PATCHLEVEL/SUBLEVEL in Makefile and has no causal relationship.
  3. Possible fix: Add WCN6855 firmware binaries (msbtfw11.mbn, msnv11.bin or qcs8300-specific equivalents) to the CI ramdisk under /lib/firmware/qca/, and add proper vddio-supply/vddbtcxmx-supply/vddrfa1p7-supply phandles in the qcs8300-ride DT bluetooth node (/soc@0/geniqup@9c0000/serial@988000/bluetooth) to ensure the WCN6855 chip is powered before the driver attempts UART communication.
  4. Detail analysis attachment: failed_case_job95058_6_detailed.md
  Case 7: ** `0_qcom-next-ci-premerge-tests`
  1. Failed case: ** 0_qcom-next-ci-premerge-tests
  2. Root cause: ** The WCN6855 Bluetooth controller on qcs8300-ride (Monaco EVK) fails to initialize at boot — hci_uart_qca serial0-0 exhausts all 4 retries of command 0xfc00 (QCA version query) with -ETIMEDOUT (-110), leaving hci0 permanently DOWN with BD address 00:00:00:00:00:00; the subsequent BT_SCAN test script enters an infinite btensurepublicaddr retry loop that never resolves, consuming the entire 20-minute lava-test-shell timeout.
  3. Possible fix: This is a pre-existing hardware/firmware issue on this board (WCN6855 BT UART not responding) unrelated to the PR; re-trigger the CI job on a healthy board instance, or add a hard timeout/skip guard in BT_SCAN/run.sh so a non-functional BT adapter causes SKIP rather than an infinite hang that blocks the entire test suite.
  4. Detail analysis attachment: failed_case_job95058_7_detailed.md
  Case 8: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The BT_SCAN test suite entered an infinite btensurepublicaddr retry loop (cycling every ~75 s) because the WCN6855 Bluetooth controller on the qcs8300-ride board never initialized — firmware blobs (msbtfw*/msnv*) are absent from the rootfs and the hci_uart_qca driver reported command 0xfc00 tx timeout / Reading QCA version information failed (-110) on every attempt — consuming the entire remaining 20-minute lava-test-shell budget and triggering the lava-test-shell timed out after 1200 seconds exception.
  3. Possible fix: Add a maximum-retry guard or hard timeout (e.g. 120 s) to the BT_SCAN test script's btensurepublicaddr loop so it exits with SKIP/FAIL instead of spinning indefinitely; separately, provision the WCN6855 firmware blobs (msbtfw11.mbn / msnv11.bin) into the test rootfs image for qcs8300-ride so the BT controller can initialize correctly.
  4. Detail analysis attachment: failed_case_job95058_8_detailed.md
Job 95059 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95059

Failed test cases in LAVA job 95059 (SoC: qcs615-ride).

  Case 1: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure; flash-universal.sh failed at the rootfs mount stage (exit 32) because rootfs.img — declared in flash.settings as ROOTFS_IMAGE=rootfs.img — is absent from the extracted qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz archive; the exact error is mount: special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job to rule out a transient packaging glitch; if the error recurs, investigate the qcs615-ride rootfs image build pipeline to ensure rootfs.img is included in the .qcomflash.tar.gz artifact — the flash.settings postprocess step unconditionally sets ROOTFS_IMAGE=rootfs.img but the artifact packager is not producing that file.
  4. Detail analysis attachment: failed_case_job95059_1_detailed.md
  Case 2: ** deploy-flasher-retry — Flash Stage Failure — `rootfs.img` not produced by postprocess
  1. Failed case: ** deploy-flasher-retry — Flash Stage Failure — rootfs.img not produced by postprocess
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh exited with code 32 because rootfs.img was absent from the deploy working directory; postprocess.sh (run inside docker-mkbootimage:master) returned exit 0 but silently failed to extract or generate rootfs.img from the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact, leaving the file referenced by ROOTFS_IMAGE=rootfs.img in flash.settings non-existent at mount time.
  3. Possible fix: Re-trigger the CI job to rule out a transient Docker/postprocess issue; if it recurs, inspect postprocess.sh inside ghcr.io/mwasilew/docker-mkbootimage:master to confirm it correctly extracts/generates rootfs.img from the qcs615-ride .qcomflash tarball, and add a non-zero exit on missing output file so the failure is caught early rather than silently propagating to flash-universal.sh.
  4. Detail analysis attachment: failed_case_job95059_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — missing `rootfs.img` in flash tarball
  1. Failed case: ** Infrastructure Flash Failure — missing rootfs.img in flash tarball
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh exited 32 because rootfs.img was absent from the extracted qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz tarball; the exact error is mount: special device rootfs.img does not exist.
  3. Possible fix: Re-trigger the CI job to obtain a freshly built artifact; if the failure recurs, inspect the qcs615-ride image build pipeline to confirm that the rootfs.img packaging step is not being skipped or that the correct artifact variant (multimedia image with rootfs) is referenced in the LAVA job definition.
  4. Detail analysis attachment: failed_case_job95059_3_detailed.md
Job 95057 | SoC qcs6490-rb3gen2

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95057

Failed test cases in LAVA job 95057 (SoC: qcs6490-rb3gen2).

  Case 1: ** deploy-flasher — Flash Infrastructure Failure — loop device unavailable in worker container
  1. Failed case: ** deploy-flasher — Flash Infrastructure Failure — loop device unavailable in worker container
  2. Root cause: ** flash-universal.sh failed with exit code 1 because the LAVA worker container could not set up a loop device to mount rootfs.img from the qcs6490-rb3gen2 flash tarball: mount: tmprootfs: failed to setup loop device for rootfs.img. — the container lacks /dev/loop* pass-through or the loop module is not accessible, making EDL-mode flashing impossible.
  3. Possible fix: On the LAVA worker host, ensure /dev/loop* devices are passed through to the worker container (add --device=/dev/loop-control --device=/dev/loop0 … or use --privileged for the flash step); alternatively, configure flash-universal.sh to use a bind-mount or pre-extracted rootfs path instead of a loop mount. Re-trigger the job after the container is fixed to confirm.
  4. Detail analysis attachment: failed_case_job95057_1_detailed.md
  Case 2: ** Build Load Failure — loop device unavailable for rootfs.img mount during flash
  1. Failed case: ** Build Load Failure — loop device unavailable for rootfs.img mount during flash
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh failed at the rootfs image preparation stage with mount: tmprootfs: failed to setup loop device for rootfs.img. (exit 1), because the LAVA worker container has no available loop device nodes (/dev/loop*) or lacks the privilege to create them, preventing rootfs.img from being loop-mounted before EDL flashing of the qcs6490-rb3gen2 board.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, ensure the LAVA worker container has loop device access by passing --device=/dev/loop-control --device=/dev/loop0 (or --privileged) in the worker container configuration, or pre-create loop device nodes inside the container with mknod.
  4. Detail analysis attachment: failed_case_job95057_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — loop device unavailable for rootfs.img mount
  1. Failed case: ** Infrastructure Flash Failure — loop device unavailable for rootfs.img mount
  2. Root cause: ** Result: Build Load Failure — the deploy-flasher stage failed when flash-universal.sh attempted to mount rootfs.img via a loop device inside the LAVA worker/flash environment and the operation failed with mount: tmprootfs: failed to setup loop device for rootfs.img., causing flash-universal.sh to exit 1 and LAVA to raise InfrastructureError: Unable to flash the device.
  3. Possible fix: Re-trigger the CI job; if the failure recurs, investigate loop device availability inside the worker container (check losetup -a and /dev/loop* count, ensure the container has --privileged or --device /dev/loop* access, and verify the host kernel has CONFIG_BLK_DEV_LOOP with sufficient max_loop devices configured).
  4. Detail analysis attachment: failed_case_job95057_3_detailed.md
Job 95060 | SoC x1e80100

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/95060

Failed test cases in LAVA job 95060 (SoC: x1e80100).

  Case 1: ** cdsp_remoteproc — Firmware Load Failure (ENOENT)
  1. Failed case: ** cdsp_remoteproc — Firmware Load Failure (ENOENT)
  2. Root cause: ** The initramfs/rootfs used for this LAVA job does not contain qcom/x1e80100/cdsp.mbn (nor adsp.mbn) under any firmware search path; request_firmware() returns -ENOENT (-2) for both DSP subsystems on the X1E80100 CRD, leaving remoteproc1 permanently in offline state before the test script runs.
  3. Possible fix: Ensure the test rootfs/initramfs (initramfs-kerneltest-full-image-qcom-armv8a.cpio.gz) includes the X1E80100 firmware blobs at /lib/firmware/qcom/x1e80100/cdsp.mbn (and adsp.mbn); re-trigger the CI job after the firmware-populated image is rebuilt and published.
  4. Detail analysis attachment: failed_case_job95060_1_detailed.md
  Case 2: ** adsp_remoteproc
  1. Failed case: ** adsp_remoteproc
  2. Root cause: ** The ADSP firmware blob qcom/x1e80100/adsp.mbn is absent from the test initramfs; the kernel's request_firmware() returns -ENOENT (error -2) at boot (~T+26s), leaving remoteproc0 permanently in offline state before the test script runs.
  3. Possible fix: Add the x1e80100 firmware blobs (adsp.mbn, cdsp.mbn, and other qcom/x1e80100/ blobs) to the initramfs-kerneltest-full-image-qcom-armv8a rootfs image used by this LAVA job; rebuild and re-publish the initramfs artifact, then re-trigger the CI job.
  4. Detail analysis attachment: failed_case_job95060_2_detailed.md
  Case 3: ** `0_qcom-next-ci-premerge-tests` — Test Shell Timeout (s2idle suspend hang during CPUFreq_Validation)
  1. Failed case: ** 0_qcom-next-ci-premerge-tests — Test Shell Timeout (s2idle suspend hang during CPUFreq_Validation)
  2. Root cause: ** The CPUFreq_Validation test on x1e80100-CRD triggered an s2idle suspend entry (kernel cmdline mem_sleep_default=s2idle) at ~33s into the test run; the board never resumed, silencing the serial console and causing lava-test-shell to time out after 1200 seconds. Additionally, adsp_remoteproc and cdsp_remoteproc sub-tests failed independently because qcom/x1e80100/adsp.mbn and cdsp.mbn firmware blobs are absent from the initramfs (request_firmware failed: -2).
  3. Possible fix: Add no_console_suspend (or better, mem_sleep_default=freeze or mem_sleep_default=s2idle with a wakeup alarm) to the kernel cmdline in the LAVA job definition to prevent the CPUFreq test from hanging the console; separately, include adsp.mbn/cdsp.mbn firmware blobs in the test initramfs for x1e80100 to resolve the remoteproc FAIL sub-tests.
  4. Detail analysis attachment: failed_case_job95060_3_detailed.md
  Case 4: ** lava-test-shell
  1. Failed case: ** lava-test-shell
  2. Root cause: ** The CPUFreq_Validation test on x1e80100 CRD triggered an s2idle system suspend (kernel cmdline mem_sleep_default=s2idle) at kernel time ~33 s without programming an RTC wakeup alarm, causing the board to enter s2idle and never resume — silencing the serial console for the full 1200-second lava-test-shell timeout.
  3. Possible fix: Update the CPUFreq_Validation test script to program an RTC wakeup alarm (e.g., rtcwake -d rtc0 -s 30 -m no) before triggering any cpufreq scaling that induces suspend on x1e80100, or add no_console_suspend to the kernel cmdline in the LAVA job definition to keep the serial console alive through suspend so LAVA can detect a hang sooner.
  4. Detail analysis attachment: failed_case_job95060_4_detailed.md
  Case 5: ** lava-test-retry
  1. Failed case: ** lava-test-retry
  2. Root cause: ** The x1e80100 CRD board entered s2idle suspend at kernel T+33s during the CPUFreq_Validation test (triggered by mem_sleep_default=s2idle in the kernel cmdline) and never resumed, causing the LAVA test shell to exhaust its 1200-second timeout with no shell prompt returned.
  3. Possible fix: Add mem_sleep_default=shallow (or remove mem_sleep_default=s2idle) to the kernel cmdline in the LAVA job definition for x1e80100 CRD to prevent unintended s2idle entry during CPUFreq testing, or fix the s2idle wakeup path on x1e80100 if this is a known platform regression.
  4. Detail analysis attachment: failed_case_job95060_5_detailed.md
  Case 6: ** job
  1. Failed case: ** job
  2. Root cause: ** The x1e80100-CRD DUT entered s2idle system suspend at kernel uptime ~33.5 s (triggered by the CPUFreq_Validation test script setting CPU frequency on policy0), suspended the serial console (printk: Suspending console(s)), and never resumed — leaving the LAVA test-shell waiting with no output until it timed out after 1200 seconds (lava-test-shell timed out after 1200 seconds).
  3. Possible fix: Add a wakeup alarm (e.g. rtcwake -m no -s 30) before any CPUFreq scaling step in the CPUFreq_Validation/run.sh test script, or suppress spurious s2idle entry during CI by passing mem_sleep_default=freeze or removing mem_sleep_default=s2idle from the kernel cmdline in the LAVA job definition for this device type.
  4. Detail analysis attachment: failed_case_job95060_6_detailed.md
Job 94604 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94604

Failed test cases in LAVA job 94604 (SoC: qcs8300-ride).

  Case 1: ** wait-device-boardid
  1. Failed case: ** wait-device-boardid
  2. Root cause: ** After PDU power-cycle, the qcs8300-ride board boots into its default UFS/normal boot path instead of fastboot mode — the LAVA fastboot-boot action performs only a plain pdu-reboot (power off → power on via alpaca_control.py) without asserting fastboot boot mode first, so the board never enumerates the USB fastboot interface with board-ID 1315e8d7, causing wait-device-boardid to time out after 200 s on all 3 attempts.
  3. Possible fix: Update the LAVA device dictionary / job definition to assert fastboot boot mode before the power-on step — add a debugboard.py --board tac --serial <SERIAL> --set-boot-mode fastboot (or equivalent alpaca_control.py fastboot-mode command) call in the reset-device sequence so the board enumerates in fastboot on the next power-on; then re-trigger the job to verify.
  4. Detail analysis attachment: failed_case_job94604_1_detailed.md
  Case 2: ** fastboot-boot
  1. Failed case: ** fastboot-boot
  2. Root cause: ** The qcs8300-ride board (qcs8300-ride-kci-0301) powered on and reached the UEFI fastboot stage (Fastboot.efi loaded) on every attempt, but never enumerated on the USB bus with the expected board-id 1315e8d7, causing wait-device-boardid to time out after 200 s on all 3 attempts.
  3. Possible fix: Investigate the USB connection between the qcs8300-ride board and the LAVA worker container — check that the USB cable is seated, the board's USB device port is correctly passed through to worker0, and that the udev rule matching board-id 1315e8d7 is present and active; re-trigger the job once connectivity is confirmed.
  4. Detail analysis attachment: failed_case_job94604_2_detailed.md
  Case 3: ** Build Load Failure — `wait-device-boardid` timeout (board never enumerated as USB fastboot device)
  1. Failed case: ** Build Load Failure — wait-device-boardid timeout (board never enumerated as USB fastboot device)
  2. Root cause: ** Result: Build Load Failure. The qcs8300-ride board (board_id=1315e8d7) booted into UEFI Fastboot on all 3 attempts but never enumerated as a USB fastboot device on the LAVA worker host; wait-device-boardid timed out (200s × 2, 105s × 1) with error_type: Infrastructure. A persistent UEFI DT overlay failure (Bad main_symbols in ufdt_overlay_do_fixupsFailed to apply devie tree) on every boot cycle is the likely cause of the USB gadget not being correctly configured, blocking fastboot enumeration. The PR patch (Makefile version bump only) is unrelated.
  3. Possible fix: Report board qcs8300-ride-kci-0301 (board_id=1315e8d7) to LAVA admins; re-trigger the job on a different qcs8300-ride instance, and investigate the UEFI DT overlay failure (Bad main_symbols in ufdt_overlay_do_fixups) on this board — likely requiring a UEFI firmware update or DT overlay fix for this lab unit.
  4. Detail analysis attachment: failed_case_job94604_3_detailed.md
Job 94605 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/94605

Failed test cases in LAVA job 94605 (SoC: qcs615-ride).

  Case 1: ** Infrastructure Flash Failure — rootfs.img missing from flash artifact
  1. Failed case: ** Infrastructure Flash Failure — rootfs.img missing from flash artifact
  2. Root cause: ** Result: Infrastructure Flash Failure — flash-universal.sh exited with code 32 because postprocess.sh unconditionally sets ROOTFS_IMAGE=rootfs.img in flash.settings, but the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact for this qcs615-ride build does not contain a rootfs.img file, causing the mount call inside flash-universal.sh to fail with "special device rootfs.img does not exist."
  3. Possible fix: Inspect the postprocess.sh script inside ghcr.io/mwasilew/docker-mkbootimage:master and fix it to only write ROOTFS_IMAGE=rootfs.img when rootfs.img is actually present in the flash archive; alternatively, verify that the CI artifact build pipeline for qcs615-ride is correctly producing and packaging rootfs.img into the .qcomflash.tar.gz before re-triggering the job.
  4. Detail analysis attachment: failed_case_job94605_1_detailed.md
  Case 2: ** Build Load Failure — Corrupt/incompatible image
  1. Failed case: ** Build Load Failure — Corrupt/incompatible image
  2. Root cause: ** Result: Build Load Failure — flash-universal.sh failed at the rootfs mount stage because the qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact does not contain a rootfs.img file, yet flash.settings unconditionally declares ROOTFS_IMAGE=rootfs.img, causing mount: special device rootfs.img does not exist (exit 32).
  3. Possible fix: Verify whether the qcs615-ride .qcomflash artifact build pipeline is correctly producing and packaging rootfs.img into the tarball; if rootfs.img is intentionally absent for this SoC/image type, update the docker postprocess step in the LAVA job definition to omit or correct the ROOTFS_IMAGE setting so flash-universal.sh does not attempt to mount a non-existent file.
  4. Detail analysis attachment: failed_case_job94605_2_detailed.md
  Case 3: ** Infrastructure Flash Failure — `rootfs.img` missing from qcomflash artifact tarball
  1. Failed case: ** Infrastructure Flash Failure — rootfs.img missing from qcomflash artifact tarball
  2. Root cause: ** flash-universal.sh failed during the deploy-flasher stage because rootfs.img does not exist inside the downloaded qcom-multimedia-image-qcs615-ride.rootfs.qcomflash.tar.gz artifact — the tarball contains only EDL/firehose firmware blobs (XBL, TZ, UEFI, GPT bins, rawprogram XMLs) but no rootfs.img filesystem image, causing mount: special device rootfs.img does not exist (exit 32).
  3. Possible fix: Verify that the CI build pipeline for qcs615-ride correctly produces and packages rootfs.img into the qcomflash tarball before publishing the artifact; re-trigger the job once the artifact is confirmed to contain rootfs.img, or update flash.settings/postprocess.sh to point ROOTFS_IMAGE to the correct image filename present in the tarball.
  4. Detail analysis attachment: failed_case_job94605_3_detailed.md

@rahujosh
Copy link
Copy Markdown

PR #64 — checker-log-analyzer

PR: #64
Checker run: https://github.com/qualcomm-linux-stg/kernel-config-test/actions/runs/25633951916

Checker Result Summary
Checker Result Summary
checkpatch WARNING: Missing commit description on commit 53cd93473038
dt-binding-check ⏭️ No changes in Documentation/devicetree/bindings
dtb-check ⏭️ No changes in Devicetree
sparse-check ⏭️ No C/H files changed
check-uapi-headers ⏭️ No C/H files changed
check-patch-compliance Commit summary does not start with a required prefix
tag-check N/A Target branch is qcom-next-staging — prefix check not required
qcom-next-check N/A No FROMLIST:/UPSTREAM:/BACKPORT:/FROMGIT: commits in this PR

Detailed report: Full report

Checker analysis — click to expand

🤖 CI Checker Analysis (checker-log-analyzer)

PR: qualcomm-linux-stg/kernel #64 — "Update Makefile"
Source: https://github.com/qualcomm-linux-stg/kernel-config-test/actions/runs/25633951916

Checker Result Summary
checkpatch WARNING: Missing commit description on commit 53cd93473038
dt-binding-check ⏭️ No changes in Documentation/devicetree/bindings
dtb-check ⏭️ No changes in Devicetree
sparse-check ⏭️ No C/H files changed
check-uapi-headers ⏭️ No C/H files changed
check-patch-compliance Commit summary does not start with a required prefix
tag-check N/A Target branch is qcom-next-staging — prefix check not required
qcom-next-check N/A No FROMLIST:/UPSTREAM:/BACKPORT:/FROMGIT: commits in this PR

❌ checkpatch

Root cause: Commit 53cd93473038 ("Update Makefile") has no body in the commit message — only a subject line and Signed-off-by:, with nothing in between to describe why the change is being made.

Failure details:

WARNING: Missing commit description - Add an appropriate one
53cd93473038dbc63b514a9876b53bbbed022499 total: 0 errors, 1 warnings, 0 checks, 9 lines checked
Commit 53cd93473038 ("Update Makefile") has style problems, please review.

Fix: Add a meaningful description paragraph between the subject line and the Signed-off-by: tag explaining the purpose of the version bump:

From: Salendarsingh Gaud <sgaud@qti.qualcomm.com>
Subject: [PATCH] QCLINUX: Update Makefile

Bump PATCHLEVEL and SUBLEVEL from 0 to 1 to reflect the updated
kernel version for this release.

Signed-off-by: Salendarsingh Gaud <sgaud@qti.qualcomm.com>

Reproduce locally:

./scripts/checkpatch.pl --strict --summary-file --ignore FILE_PATH_CHANGES \
  --git e98c034d3d39fcbf7632f48b1795c3caecf5121d..53cd93473038dbc63b514a9876b53bbbed022499

❌ check-patch-compliance

Root cause: The commit subject "Update Makefile" carries no upstream-linkable prefix (FROMLIST:, FROMGIT:, UPSTREAM:, BACKPORT:), so the compliance checker rejects it immediately.

Failure details:

Checking commit: Update Makefile
Commit summary does not start with a required prefix
##[error]Process completed with exit code 1.

Fix: This is a vendor-only Makefile version bump with no upstream equivalent. Prepend QCLINUX: to the subject:

git rebase -i e98c034d3d39   # mark 53cd93473038 as 'edit'
git commit --amend -m "QCLINUX: Update Makefile

Bump PATCHLEVEL and SUBLEVEL from 0 to 1 to reflect the updated
kernel version for this release.

Signed-off-by: Salendarsingh Gaud <sgaud@qti.qualcomm.com>"
git rebase --continue

Note: QCLINUX: is the correct prefix for a vendor-only change with no upstream equivalent. However, check-patch-compliance only accepts FROMLIST:, FROMGIT:, UPSTREAM:, and BACKPORT: — it will continue to flag QCLINUX: commits. This is a known checker limitation for vendor-only commits. The checkpatch WARNING: Missing commit description failure is independently fixable and must be resolved regardless.


Verdict

2 blockers to fix before merge:

  1. Add a commit body description to resolve the checkpatch Missing commit description warning.
  2. Add a QCLINUX: prefix to the commit subject — this resolves the tag-check requirement (though check-patch-compliance will still flag it as a known limitation for vendor-only commits; confirm with the repo maintainer whether QCLINUX: commits are accepted on qcom-next-staging).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants