Skip to content

MnnLlmChat v0.8.3.3 — QNN NPU & Hexagon 支持

Latest
Latest

Choose a tag to compare

View all tags
@pisces312 pisces312 released this 09 Aug 04:18
· 2 commits to master since this release
v0.8.3.3
c065ebd

🎉 MnnLlmChat v0.8.3.3 (pisces fork)

🔧 修复内容

Hexagon Backend 回退 CPU 问题修复

  • 上一个 release (v0.8.3.1) 的 APK 中，Hexagon backend 实际未启用，请求 NPU 后静默回退到 CPU
  • 根因：build.sh 缺少 -DMNN_HEXAGON=ON 编译选项
  • 修复后 Hexagon backend 正常工作，logcat 可见 [MNN::Hexagon] vectorSize=64, vtcmSize=8388608, maxThreads=8

QNN 与 Hexagon 符号冲突修复

  • 修复 dsprpc wrapper 符号重复定义问题，QNN 和 Hexagon 两种 backend 可同时编译

StridedSlice 范围修复

  • 修复 QNN 模型中负索引未正确处理导致的崩溃

🚀 新增功能

QNN (Qualcomm Neural Network) HTP NPU 后端支持

  • 支持 QNN 离线模型（HTP context binary）在 NPU 上运行 LLM 推理
  • Qwen3-0.6B 真机验证通过：decode 32 t/s（远超 Hexagon 的 1.5 t/s）
  • 集成 QNN SDK 2.39.0 运行时库（libQnnSystem.so 等）
  • App 设置页新增 NPU 选项，支持选择 QNN 后端

移除 QNN 模型冗余 weight 依赖

  • 新增 use_external_weight 配置项，QNN plugin 模型不再需要 llm.mnn.weight（省 303MB）
  • 转换脚本自动生成 "use_external_weight": false

📦 预转换 QNN 模型

预转换好的 Qwen3-0.6B QNN NPU 模型可在 HuggingFace 下载：
👉 https://huggingface.co/pisces312-hf/qwen3-0.6b-mnn-qnn-npu

📊 SM8850 性能对比（Qwen3-0.6B）

指标 CPU fp16 (4T) Hexagon QNN HTP (NPU)
Prefill ~306 t/s ~49.7 t/s 12.8~20.7 t/s
Decode ~70 t/s ~1.5 t/s ~32 t/s

💡 小模型 (≤1B) CPU 更快；4B+ 模型 NPU 优势显著（低功耗 + 不占 CPU + 大模型带宽瓶颈下更快）

📦 包含内容

  • MnnLlmChat-v0.8.3.3-pisces-standard-signed.apk — 标准版签名 APK (38MB)
  • 内含 libMNN.so（QNN + Hexagon 启用）+ QNN 运行时库
  • 适用设备：arm64-v8a, Android 8.0+ (API 26)

🎉 MnnLlmChat v0.8.3.3 (pisces fork)

🔧 Bug Fixes

Hexagon Backend Fallback to CPU — Fixed

  • Previous release (v0.8.3.1) had Hexagon backend silently falling back to CPU
  • Root cause: missing -DMNN_HEXAGON=ON in build.sh
  • Now Hexagon initializes correctly with HVX vector unit and VTCM

QNN/Hexagon Symbol Conflict

  • Fixed duplicate dsprpc wrapper symbols when both QNN and Hexagon backends are compiled together

StridedSlice Range Fix

  • Fixed crash from negative indices not being handled correctly in QNN models

🚀 New Features

QNN (Qualcomm Neural Network) HTP NPU Backend

  • Run LLM inference on Qualcomm NPU via QNN offline models (HTP context binary)
  • Qwen3-0.6B verified on real device: decode 32 t/s (vs Hexagon 1.5 t/s)
  • Bundled QNN SDK 2.39.0 runtime libraries
  • New NPU option in app settings for QNN backend selection

Remove QNN Model Weight Dependency

  • New use_external_weight config option, QNN plugin models no longer need llm.mnn.weight (saves 303MB)
  • Conversion scripts auto-generate "use_external_weight": false

📦 Pre-converted QNN Model

Pre-converted Qwen3-0.6B QNN NPU model available on HuggingFace:
👉 https://huggingface.co/pisces312-hf/qwen3-0.6b-mnn-qnn-npu

📊 SM8850 Performance (Qwen3-0.6B)

Metric CPU fp16 (4T) Hexagon QNN HTP (NPU)
Prefill ~306 t/s ~49.7 t/s 12.8~20.7 t/s
Decode ~70 t/s ~1.5 t/s ~32 t/s

💡 Small models (≤1B): CPU is faster. For 4B+ models: NPU wins on power efficiency + freed CPU + bandwidth-bound scenarios.

📦 Package Contents

  • MnnLlmChat-v0.8.3.3-pisces-standard-signed.apk — Standard signed APK (38MB)
  • Includes libMNN.so (QNN + Hexagon enabled) + QNN runtime libs
  • Requirements: arm64-v8a, Android 8.0+ (API 26)
Assets 3
Loading