MnnLlmChat v0.8.3.3 — QNN NPU & Hexagon 支持

🎉 MnnLlmChat v0.8.3.3 (pisces fork)

🔧 修复内容

Hexagon Backend 回退 CPU 问题修复

• 上一个 release (v0.8.3.1) 的 APK 中，Hexagon backend 实际未启用，请求 NPU 后静默回退到 CPU
• 根因：build.sh 缺少 -DMNN_HEXAGON=ON 编译选项
• 修复后 Hexagon backend 正常工作，logcat 可见 [MNN::Hexagon] vectorSize=64, vtcmSize=8388608, maxThreads=8

QNN 与 Hexagon 符号冲突修复

• 修复 dsprpc wrapper 符号重复定义问题，QNN 和 Hexagon 两种 backend 可同时编译

StridedSlice 范围修复

• 修复 QNN 模型中负索引未正确处理导致的崩溃

🚀 新增功能

QNN (Qualcomm Neural Network) HTP NPU 后端支持

• 支持 QNN 离线模型（HTP context binary）在 NPU 上运行 LLM 推理
• Qwen3-0.6B 真机验证通过：decode 32 t/s（远超 Hexagon 的 1.5 t/s）
• 集成 QNN SDK 2.39.0 运行时库（libQnnSystem.so 等）
• App 设置页新增 NPU 选项，支持选择 QNN 后端

移除 QNN 模型冗余 weight 依赖

• 新增 use_external_weight 配置项，QNN plugin 模型不再需要 llm.mnn.weight（省 303MB）
• 转换脚本自动生成 "use_external_weight": false

📦 预转换 QNN 模型

预转换好的 Qwen3-0.6B QNN NPU 模型可在 HuggingFace 下载：
👉 https://huggingface.co/pisces312-hf/qwen3-0.6b-mnn-qnn-npu

📊 SM8850 性能对比（Qwen3-0.6B）

指标	CPU fp16 (4T)	Hexagon	QNN HTP (NPU)
Prefill	~306 t/s	~49.7 t/s	12.8~20.7 t/s
Decode	~70 t/s	~1.5 t/s	~32 t/s

💡 小模型 (≤1B) CPU 更快；4B+ 模型 NPU 优势显著（低功耗 + 不占 CPU + 大模型带宽瓶颈下更快）

📦 包含内容

• MnnLlmChat-v0.8.3.3-pisces-standard-signed.apk — 标准版签名 APK (38MB)
• 内含 libMNN.so（QNN + Hexagon 启用）+ QNN 运行时库
• 适用设备：arm64-v8a, Android 8.0+ (API 26)

---

🎉 MnnLlmChat v0.8.3.3 (pisces fork)

🔧 Bug Fixes

Hexagon Backend Fallback to CPU — Fixed

• Previous release (v0.8.3.1) had Hexagon backend silently falling back to CPU
• Root cause: missing -DMNN_HEXAGON=ON in build.sh
• Now Hexagon initializes correctly with HVX vector unit and VTCM

QNN/Hexagon Symbol Conflict

• Fixed duplicate dsprpc wrapper symbols when both QNN and Hexagon backends are compiled together

StridedSlice Range Fix

• Fixed crash from negative indices not being handled correctly in QNN models

🚀 New Features

QNN (Qualcomm Neural Network) HTP NPU Backend

• Run LLM inference on Qualcomm NPU via QNN offline models (HTP context binary)
• Qwen3-0.6B verified on real device: decode 32 t/s (vs Hexagon 1.5 t/s)
• Bundled QNN SDK 2.39.0 runtime libraries
• New NPU option in app settings for QNN backend selection

Remove QNN Model Weight Dependency

• New use_external_weight config option, QNN plugin models no longer need llm.mnn.weight (saves 303MB)
• Conversion scripts auto-generate "use_external_weight": false

📦 Pre-converted QNN Model

Pre-converted Qwen3-0.6B QNN NPU model available on HuggingFace:
👉 https://huggingface.co/pisces312-hf/qwen3-0.6b-mnn-qnn-npu

📊 SM8850 Performance (Qwen3-0.6B)

Metric	CPU fp16 (4T)	Hexagon	QNN HTP (NPU)
Prefill	~306 t/s	~49.7 t/s	12.8~20.7 t/s
Decode	~70 t/s	~1.5 t/s	~32 t/s

💡 Small models (≤1B): CPU is faster. For 4B+ models: NPU wins on power efficiency + freed CPU + bandwidth-bound scenarios.

📦 Package Contents

• MnnLlmChat-v0.8.3.3-pisces-standard-signed.apk — Standard signed APK (38MB)
• Includes libMNN.so (QNN + Hexagon enabled) + QNN runtime libs
• Requirements: arm64-v8a, Android 8.0+ (API 26)

v0.8.3-pisces.2

v0.8.3-pisces.2

Fork release based on upstream MnnLlmChat 0.8.3 (MNN engine synced to upstream master @ e1b8a9cb, 13 commits).

Models for Hexagon can be downloaded from https://huggingface.co/pisces312-hf/qwen3-4b-mnn-hexagon-4bit

Highlights

• Hexagon NPU (HTP) backend support: run LLMs on the Hexagon DSP — Qwen3-4B verified on device
• htp-ops skel rebuilt with upstream PWL FP16 activation optimization (3ed5e9cb) + local Q4 matmul ops (DSP v81)
• Upstream merge: Apple GPU LLM perf overhaul, 9 memory-safety fixes in operators, LLM bugfixes (LoRA-split + transformerFuseC4 garbage, LlmContext cross-thread UAF, Qwen3.5 text-only export, InternVL q/k norm export), OpenCL int32 Select/Reduction fix, prefix KV-cache file support, llm_bench -pg real prefill+decode benchmark

Screenshots

Hexagon backend option in model config:

Qwen3-4B running on Hexagon NPU:

Assets

• MnnLlmChat-v0.8.3.2-pisces-standard-signed.apk — standard flavor, signed release (arm64-v8a, minSdk 26)
• versionName 0.8.3.2 / versionCode 26080101

v0.8.3-pisces.1

v0.8.3-pisces.1

Fork release based on upstream MnnLlmChat 0.8.3 (MNN 3.6.0).

Highlights

• Built with latest MNN 3.6.0 native libraries (libMNN.so), prebuilt and bundled for arm64-v8a
• Model storage path customization: models can now be stored on external storage (e.g. /storage/emulated/0/mnn-models/), making them shareable across apps and accessible via file managers
• Flat mode download: FUSE-compatible layout for external storage — no symlinks, files stored directly in snapshots/ directories
• In-app log viewer: Settings → View Logs to diagnose download failures without logcat
• Download failure logging: detailed error reporting throughout the download chain

Download

File	Flavor	Arch	Build
app-standard-debug.apk	standard	arm64-v8a	debug

Note: Debug build with applicationIdSuffix .debug — can coexist with the release version.

Install

1. Allow installation from unknown sources
2. Install the APK
3. Grant "All files access" permission when prompted (required for external storage path)
4. Settings → Model storage path → choose external storage directory
5. Download models from the Model Market — they will be stored in the chosen path