[Dragon Q6A] Venus H.264 encoder hangs in uninterruptible-sleep — firmware never requested (kernel 6.18.2-3-qcom)

[jason-dgio]

On a Radxa Dragon Q6A (QCS6490) running the stock Radxa image with kernel 6.18.2-3-qcom, the Venus video encoder enumerates correctly but any actual encode attempt hangs in kernel uninterruptible-sleep. The driver appears to never call request_firmware for Venus — dmesg shows the binding message but no firmware-load activity, success or failure.

Environment

• Board: Radxa Dragon Q6A (QCS6490, eMMC Module V1.2)
• Image: shipped Radxa Ubuntu 24.04 (fresh boot today)
• Kernel: Linux dragon-q6a 6.18.2-3-qcom #1 SMP PREEMPT ...
• Venus modules: venus_core, venus_enc, venus_dec all loaded
• Firmware dirs: /lib/firmware/qcom/venus-{1.8,4.2,5.2,5.4,6.0}/ contain only .zst-compressed venus.mbn.zst (Jan 2026 dated)

What works

• v4l2-ctl --device /dev/video1 --info reports Card type: Qualcomm Venus video encoder
• --list-formats-ext lists H.264 + HEVC on CAPTURE queue
• Opening the device returns successfully

What fails

Any of the following hangs in D-state indefinitely (cannot be killed with SIGKILL or `timeout`):

```bash
ffmpeg -f lavfi -i testsrc2=size=640x480:rate=15 -t 1 \
-vf format=nv12 -c:v h264_v4l2m2m -b:v 1M -y /tmp/v.h264

gst-launch-1.0 -e videotestsrc num-buffers=30 \
! "video/x-raw,format=NV12,width=640,height=480,framerate=30/1" \
! v4l2h264enc device=/dev/video1 \
! filesink location=/tmp/v.h264
```

Smoking gun: no firmware load attempt

`sudo dmesg -c >/dev/null` followed by an encode attempt — dmesg afterward shows nothing Venus-related. By contrast the WiFi chip's `aic_load_firmware` is verbose about every file it pulls. The Venus driver appears to not be reaching the firmware-request stage at all.

What I tried

1. Decompressed all `/lib/firmware/qcom/venus-*/venus.mbn.zst` → `venus.mbn` (since `CONFIG_FW_LOADER_COMPRESS_ZSTD` is not set per `zcat /proc/config.gz | grep FW_LOADER`).
2. Symlinked `venus.mdt` → `venus.mbn` in each dir.
3. Reloaded the venus modules (failed silently because /dev/video1 was still open by udev).
4. Full reboot.

No change after any of the above. The driver still binds, /dev/video1 still enumerates, encode attempts still hang, dmesg still empty for Venus.

Expected

Either firmware loads successfully (dmesg shows the load), or it fails visibly with a `request_firmware ... failed` line so we know which path to debug.

Hypothesis

The Venus probe path in this kernel build may not be wired to call `request_firmware` at all — the lazy-load on stream-start path appears to be skipping the firmware acquisition step entirely.

Happy to provide the kernel build config, full dmesg, or run any diagnostic. The board is on the bench.

[jason-dgio]

Additional diagnosis from our Q6A this morning — sharing in case it helps narrow this down.

Reproduction is consistent

$ ffmpeg -loglevel warning \
  -f lavfi -i 'testsrc=duration=1:size=1280x720:rate=30' \
  -pix_fmt nv12 -c:v h264_v4l2m2m -b:v 2M /tmp/out.h264 -y

→ ffmpeg enters kernel D-state on the V4L2 streaming path; timeout -s KILL cannot reap it, has to wait ~minutes for the kernel to release.

Kernel state during the hang — completely silent

Cleared the dmesg ring before triggering, ran dmesg -w > /tmp/cap.log in parallel. Zero kernel messages produced during the failure. No errors, no traces, no completion timeouts. The driver hangs without emitting anything.

This rules out a simple request_firmware failure (which would log fail to load video firmware) and suggests the hang is in an hfi_session_* wait_for_completion that never fires.

Single venus message at boot

The entire kernel log contains exactly one Venus line:

[    0.849237] qcom-venus aa00000.video-codec: non legacy binding

So probe completed past drivers/media/platform/qcom/venus/pm_helpers.c:975 (in core_get_v4) without error. v4l2 enumeration works, encoder capabilities list correctly (H264 + HEVC, 128×128–8192×8192).

DT binding observation

The Q6A's device tree binds the Venus IP using compatible = "qcom,sc7280-venus":

$ cat /proc/device-tree/soc@0/video-codec@aa00000/compatible
qcom,sc7280-venus

There is no qcom,qcs6490-venus or qcom,sm7325-venus compatible in upstream (v6.18 drivers/media/platform/qcom/venus/core.c). Radxa is reusing the sc7280 binding because QCS6490 shares the Iris2 IP family — but the matching resources struct (sc7280_res) assumes sc7280-specific clocks, power domains, and .fwname = "qcom/vpu-2.0/venus.mbn", which may not be a clean match for QCS6490's actual peripheral wiring.

Worth checking whether the host-side init handshake (HFI v6xx session-init) is silently incompatible because of clock/regulator differences between the two SoCs that don't surface as a probe-time error.

Environment

• Board: Radxa Dragon Q6A (QCS6490, eMMC Module V1.2)
• Kernel: Linux dragon-q6a 6.18.2-3-qcom #1 SMP PREEMPT aarch64
• Image: shipped Radxa Ubuntu 24.04 (fresh boot)
• Venus modules: venus_core, venus_enc, venus_dec all loaded (autoload at probe)
• Firmware on disk: /lib/firmware/qcom/vpu-2.0/venus.mbn.zst (zstd-compressed; uncompressed not present)
• /lib/firmware/qcom/venus-6.0/venus.mbn does exist uncompressed (1.7 MB, Jan 2026)

Happy to run any further diagnostic (strace on the encode, debugfs dumps, instrumented module rebuild) if it would help triage.

[jason-dgio]

Follow-up after this morning's debugging. Two additions worth sharing — eliminating one hypothesis, escalating severity.

Firmware-naming hypothesis: eliminated

Initial suspicion was a mismatch between the kernel-requested qcom/vpu-2.0/venus.mbn and what Canonical's linux-firmware-dragonwing actually ships at that path. After upgrading to the latest dragonwing package (1.0.0+20260324), which includes Canonical's fix for LP: #2122639 ("add vpu firmware symlink to upstream names in updates"):

$ ls -la /lib/firmware/qcom/vpu-2.0/
venus.mbn.zst -> ../vpu/vpu20_p1.mbn.zst
vpu20_1v.mbn

The kernel-requested path now correctly resolves to a Qualcomm-shipped blob. rmmod + modprobe to force a fresh probe. Encode attempt still hangs identically. So the bug isn't the firmware filename or contents.

Higher severity: full-system instability

In the course of three encode test attempts today:

• ffmpeg process enters TASK_UNINTERRUPTIBLE (D-state) within seconds of VIDIOC_STREAMON
• timeout -s KILL cannot reap it; the kernel holds the task for tens of seconds to minutes
• The board lost network reachability three times (visible only locally via attached keyboard/mouse); one spontaneous full reboot occurred after the operator's manual power-cycle, without any user action

So the failure mode isn't just a process-level D-state — it eventually wedges enough of the kernel that the system goes offline or panics. For users running a service like ours on this hardware, that's a "any unprivileged user can crash the box" footprint.

What's eliminated so far

• ❌ Firmware-filename mismatch (Canonical's fix applied, no change)
• ❌ wait_for_completion_timeout in hfi.c (timeout is msecs_to_jiffies(1000); we'd see -ETIMEDOUT long before the observed wedge)
• ❌ Frame-size dependency (tested 320×180 and 1280×720 NV12 — identical hang)
• ❌ Device-node dependency (both /dev/video0 enc and /dev/video1 dec exhibit the issue when actually streamed)

What still stands as the leading hypothesis

QCS6490 (Snapdragon 778G industrial variant) has no upstream qcom,qcs6490-venus compatible. Radxa's DT binds with qcom,sc7280-venus, matching the sc7280_res resources struct (clocks, power-domain wiring, firmware path) — which is similar but not identical SoC. The HFI session-init handshake likely fails because of subtle clock/regulator differences that probe-time setup doesn't catch.

A proper fix probably requires a new resources struct for QCS6490 in drivers/media/platform/qcom/venus/core.c.

Happy to run further diagnostics if anything specific would help; sysrq is currently masked in this kernel (/proc/sys/kernel/sysrq = 176) so echo w > /proc/sysrq-trigger is filtered, and dmesg gates behind sudo.