[Feature] Support for Qualcomm NPU Hardware Acceleration (QNN Execution Provider)

[Dual-0]

I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.

• Yes

The feature

Hi Immich Team,

with the additions of RKNN and ARM NN support, I would like to propose adding support for Qualcomm NPUs via the ONNX Runtime QNN Execution Provider.

This would benefit users running Immich on high-end Qualcomm-based SBCs like the Radxa Dragon Q6A (QCS6490) or potentially Windows-on-ARM servers.

Technical Details:

• Target Backend: QNNExecutionProvider
• Requirement: Qualcomm AI Runtime (QAIRT) SDK.
• Implementation Path: Similar to the current RKNN implementation with a specific Docker tag (e.g., -qnn) containing the onnxruntime-qnn build and the necessary shared libraries like libQnnHtp.so.
• Reference: As linked above Radxa and Qualcomm have recently documented the process for building ONNX Runtime with QNN support for Linux Aarch64 targets.

Benefits:
The Qualcomm QCS6490 NPU is significantly more powerful than standard ARM NN (Mali) implementations for ML tasks.

Is this something the team or community would be interested in exploring? I am happy to help test on a Radxa Dragon Q6A.

Platform

• Server
• Web
• Mobile

[Dual-0]

Disclosure: this exploration was done with AI assistance. The build, testing and evaluation were performed on real hardware — all results below are from actual NPU execution, not simulated.

I have a working QNN HTP pipeline running on a Radxa Dragon Q6A (QCS6490, Hexagon v68) with Immich v3.0.3. The code changes to enable QNN EP are minimal:

• constants.py: add "QNNExecutionProvider" to SUPPORTED_PROVIDERS
• ort.py: add a case "QNNExecutionProvider" passing backend_path via provider_options
• Dockerfile: add builder-qnn / prod-qnn stages (mirrors the -rknn pattern)
• pyproject.toml: add qnn extra

The NPU engages and executes both ArcFace and CLIP ViT-SO400M models (confirmed via DDR bandwidth monitoring). However, I hit a hardware-level constraint that blocks production use on this specific SoC:

The QCS6490's Hexagon v68 DSP (with the latest available firmware, c4-00004) only supports INT8 and not FP16 or FP32. Running an unquantized model returns The SocModel doesn't support FP16. This means every layer must be INT8-quantized with no precision fallback. For Immich's models:

• ArcFace (buffalo_l): INT8 quantization collapses all face embeddings (LFW verification accuracy = 51%, essentially random). Root cause is the pre-activation BatchNormalization architecture (BN → Conv → PRelu), which doesn't fold cleanly.
• CLIP ViT-SO400M: INT8 degrades embedding diversity significantly (cos_sim between semantically different images compresses from 0.70–0.91 FP32 to 0.99+ INT8).

For comparison, RK3588's NPU supports FP16, so RKNN quantization can keep sensitive layers in higher precision. That's why the existing -rknn models work and -qnn doesn't — at least on this generation of Qualcomm hardware.

What may would help:

• FP16 support on the DSP — newer Qualcomm SoCs or firmware updates may enable this. The pipeline itself works; only the precision constraint is the blocker.
• QAT (Quantization-Aware Training) — if the Immich team publishes QAT-trained INT8 variants of the models, they would survive INT8-only quantization much better.
• Official onnxruntime-qnn plugin — Qualcomm now ships a proper plugin EP (onnxruntime/onnxruntime-qnn), but it requires QAIRT 2.48+ which needs newer DSP firmware than what's available for QCS6490.

Happy to help test if anyone wants to iterate on this — the hardware is ready, the pipeline works, we just need better quantization or FP16 support.